Clever Geek Handbook

Check if s3-url exists

I have the following url that exists:

https://s3-us-west-1.amazonaws.com/premiere-avails/458ca3ce-c51e-4f69-8950-7af3e44f0a3d__chapter025.jpg

But this is not:

https://s3-us-west-1.amazonaws.com/premiere-avails/459ca3ce-c51e-4f69-8950-7af3e44f0a3d__chapter025.jpg

Is there a way to check the URL to make sure it is valid without downloading the file (it might be a 1GB file)? Note that I don't want to use boto

to find out if a key exists, I would like to use a query HTTP

.

+3


source to share


2 answers


Try the following:



import httplib
from urlparse import urlparse

def url_exists(url):
    _, host, path, _, _, _ = urlparse(url)
    conn = httplib.HTTPConnection(host)
    conn.request('HEAD', path)
    return conn.getresponse().status < 400
+6


source


You can use curl. The parameter --head

sends a HEAD request, not a GET, so that it doesn't return a body, even if it existed.



curl --head https://s3-us-west-1.amazonaws.com/premiere-avails/458ca3ce-c51e-4f69-8950-7af3e44f0a3d__chapter025.jpg

+1


source






More articles:


All Articles