Cloudfront TTL not working
I ran into a problem and tried to follow the answers here on the forum, but without success.
To create thumbnails I created the following diagram: S3 account for original images Ubuntu server using NGINX and Thumbor CloudFront
User uploads source images to S3, which will be delivered via Ubuntu Server with Cloudfront before requesting:
http: //cloudfront.account/thumbor-server/http: //s3.aws ...
The big thing is that we often lose objects in Cloudfront, I want them to stay 360 days in the cache. I am getting the following response via the Cloudfront URL:
Cache-Control:max-age=31536000
Connection:keep-alive
Content-Length:4362
Content-Type:image/jpeg
Date:Sun, 26 Oct 2014 09:18:31 GMT
ETag:"cc095261a9340535996fad26a9a882e9fdfc6b47"
Expires:Mon, 26 Oct 2015 09:18:31 GMT
Server:nginx/1.4.6 (Ubuntu)
Via:1.1 5e0a3a528dab62c5edfcdd8b8e4af060.cloudfront.net (CloudFront)
X-Amz-Cf-Id:B43x2w80SzQqvH-pDmLAmCZl2CY1AjBtHLjN4kG0_XmEIPk4AdiIOw==
X-Cache:Miss from cloudfront
After a fresh update, I get:
Age:50
Cache-Control:max-age=31536000
Connection:keep-alive
Date:Sun, 26 Oct 2014 09:19:21 GMT
ETag:"cc095261a9340535996fad26a9a882e9fdfc6b47"
Expires:Mon, 26 Oct 2015 09:18:31 GMT
Server:nginx/1.4.6 (Ubuntu)
Via:1.1 5e0a3a528dab62c5edfcdd8b8e4af060.cloudfront.net (CloudFront)
X-Amz-Cf-Id:slWyJ95Cw2F5LQr7hQFhgonG6oEsu4jdIo1KBkTjM5fitj-4kCtL3w==
X-Cache:Hit from cloudfront
My Nginx responses:
Cache-Control:max-age=31536000
Content-Length:4362
Content-Type:image/jpeg
Date:Sun, 26 Oct 2014 09:18:11 GMT
Etag:"cc095261a9340535996fad26a9a882e9fdfc6b47"
Expires:Mon, 26 Oct 2015 09:18:11 GMT
Server:nginx/1.4.6 (Ubuntu)
Why isn't Cloudfront saving my objects as instructed? Max-Age installed? Thank you very much in advance.
source to share
The second request shows that the object is indeed cached. I assume you can see this, but the question is unclear.
Cache-Control: max-age
only specifies the maximum age of your objects in the Cloudfront cache at any particular edge location. There is no minimum amount of time that your objects will persist ... after all, Cloudfront is a cache that is, by definition, volatile.
If an item at an edge location is not frequently requested, CloudFront can pull the item — delete the item before it expires — create room for more popular items.
- http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Expiration.html
Also, there is no concept of Cloudfront as a whole having a copy of your object. Each edge location cache seems to operate independently of the others, so it's not uncommon to see multiple requests for popular objects coming from different areas of the Cloudfront edge.
If you are trying to notify the load on your back-end server, it might make sense to put some kind of cache that you control in front of it, like varnish, squid, another nginx, or a custom solution that is how I do it on my systems.
Alternatively, you could save each result to S3 after processing and then configure the existing server to test S3 first before trying to resize the object again.
Then why is there a documented "minimum" TTL?
On the same page above, you will also find the following:
For web distributions, if you add Cache-Control or Expires headers to your objects, you can also specify the minimum amount of time that CloudFront keeps the object in cache before sending another request to the beginning.
I can understand why this is, and the hint given in the comment below ...
The minimum amount of time (in seconds) that an object is in the CloudFront cache before CloudFront redirects another request to its original state to determine if an updated version is available.
... seemingly contradicts my answer. However, there is no contradiction.
The minimum ttl, in simple terms, sets a lower bound on internal interpretation Cache-Control: max-age
, overriding - internally Cloudfront - any lower value sent by the origin server. Server talks about cache for 1 day, maximum, but configured TTL minimum is 2 days? Cloudfront forgets what it saw in the max-age header and might not check the origin again on subsequent requests for the next 2 days, rather than checking again after 1 day.
The nature of the cache dictates the correct interpretation of all seeming ambiguity:
Your configuration limits how long Cloudfront CAN serve cached copies of an object, and the point after which it MUST NOT continue to return an object from its cache. They don't define how long Cloudfront SHOULD maintain a cached copy because Cloudfront MAY evict an object at any time.
If you set the header correctly Cache-Control:
, Cloudfront will consider the larger max-age
or your minimum TTL as the longest amount of time that you want them to serve a cached copy without going back to the origin server.
As your site traffic increases, this should become less problematic as your objects will become more "popular", but there is fundamentally no way to indicate that Cloudfront supports a copy of an object.
source to share