Bad MAC after porting OpenSSL 1.0.2 to ECOS
We have OpenSSL running on our embedded system that runs ECOS OS . We are now updating our version of OpenSSL to 1.0.2. We have successfully ported and compiled the OpenSSL library. But when we try to connect our device using SSL (over https), the handshake fails with a bad write warning. We turned on the OpenSSL debugging feature, but were unable to determine why it was not working.
Has anyone posted the latest OpenSSL code in ECOS? Do I need to take any special flag compilations with the latest OpenSSL code for ECOS?
For reference, here is the relevant part ssl3_get_record
:
mac = rr->data + rr->length;
i=s->method->ssl3_enc->mac(s,md,0 /* not send */);
if (i < 0 || CRYPTO_memcmp(md, mac, (size_t)mac_size) != 0)
{
al=SSL_AD_BAD_RECORD_MAC;
SSLerr(SSL_F_SSL3_GET_RECORD,SSL_R_DECRYPTION_FAILED_OR_BAD_RECORD_MAC);
goto f_err;
}
source to share
After debugging, we found that the random library ( RAND ) fails for ECOS. There were many places in OpenSSL where it checked the return type of random_bytes. Due to this failure, the decryption of the key with the pre-master was not performed. And the incoming packets were not decrypted properly. Hence, BAD Mac write error has been seen.
We also checked our old ported code (0.9.6), the RAND library was also failing, but there we do not return a check for random_bytes and pseudo_rand_bytes. As a fix, we made RAND return success every time, and we can see the SSL session is established fine with OpenSSL 1.0.2.
source to share