IPN check returns INVALID because PayPal is sending invalid encoding in POST for IPN listener

In the isolated area of ​​PayPal, all of the following:

I have a PayPal subscription / recurring payment setup that uses an IPN listener as the final step to create or update a user's subscription in my website database. However, when I received POST data from PayPal and submitted it for verification, the only response I get is "INVALID".

I know my code is sending the POST data to PayPal correctly because I get a "VERIFIED" response when I mimic the correct POST to my listener by hard-coding the POST data line with the "IPN Message" value as shown in the PayPal IPN history page.

So my code that sends POST data back to PayPal for final verification is working and my IPN listener receives POST variables for the transaction. There is only a difference between the POST data string my IPN listener creates to send back to PayPal and the POST data string PayPal expects to receive.

PayPal states that the IPN listener should send back the received POST data with the same values, order and encoding that PayPal sent.

I think the issue is related to encoding issue because in PayPal POST for my listener, "charset" has value "windows-1252" which should be "UTF-8". However, "form_charset" has the correct value "UTF-8".

I set the encoding to UTF-8 on both input fields in my merchant's PayPal profile and had a hidden input in the initial PayPal form / button named "charset" and the value "UTF-8". Please tell me if there is another way to set the encoding that I missed.

The portion of the POST data line that PayPal claims to send includes "& notify_version = 3.8" followed by other variables. When I manually insert this line to test my code, "& not" is converted to "¬" when printed to the page, which I understand means the character encoding being used is "windows-1252" because "¬" is in the Windows-1252 character set, but "¬" is not in the UTF-8 character set, so if UTF-8 is used, "& not" will not be converted like this.

How can I get PayPal for the actual POST data for my IPN listener with UTF-8 character encoding? Please be aware that this is a recurring payment, so some aspects may vary.

I don't think there is a problem at my end as I tried different methods, different ways to encode / decode posted data, external libraries, cUrl / sockets, SSL on different versions, different headers, restarting the server, re-creating a purchase, etc. So Paypal is sending me the wrong POST data, or I am forming the post line incorrectly. The latter is possible, but I tried every method I could find to form the message string and none worked.

I've been working on this issue for several days to no avail, so any help or pointers would be much appreciated.

PS: a few notes that may or may not be relevant:

• This subscription system was set up a few months ago and has only inexplicably stopped working at some point in the past few weeks. No relevant code was changed at this time AFAIK.

• The very first day I started debugging this issue, the sandbox merchant account had absolutely wrong details, it was listed as a personal account when it should have been an enterprise account, which meant none of the IPN settings was affordable. The next day, the same account inexplicably worked and manifested itself as the business account that should have been the first thing. I am not aware of any action on my part that caused this change. Maybe PayPal is sending the wrong character encoding a similar problem? Some kind of caching system at cron job? I'm just thinking at this point, I really have no idea.

• Regarding my last point of entry to the sandbox account, the seller was very unreliable. Sometimes I try to log in after a session timed out and get an error code that would only make sense in the context of registering a new account. Other times the login just doesn't work, the result is that part of the time when I try to login after a session timed out, I have to clear my login cookies for it to work.