IIS Caching Deleted SSL Certificate

I am using IIS 10 on Windows 2016

Whenever I renew my SSL certificate and apply it to the binding, IIS doesn't use that, but continues to use the old certificate.

What I have tried so far -

• Reload the website
• Restart application pools
• Restart IIS
• Removing the old certificate from the IIS / Windows store

Restarting Windows solves it every time.