Clever Geek Handbook

Delphi- downloading files from the Internet using sockets

I've tried Synapse, Indy, and ICS and I'm not happy with them. I want to download multiple files in multiple parts at the same time, resume support, gzip encoded files, cookies, write to websites using POST and so on. So I think I'll just write with sockets. But I can see that Delphi has a lot of sockets: TTcpClient, TRawSocket, TCGIRequest, TClientSocket, and so on. They are all poorly documented - it is very difficult to find examples of use. I tried using TTcpClient but sometimes the program freezes and then hits a timeout and I have no idea why. Looks like a problem, waiting for answers. This is of course not a server issue, because I am testing on localhost. What is the best socket for working with HTTP? Something easy to use?

I want to use in both Delphi 7 and XE2. I don't want to use something like WinAPI, so I don't have to deal with PChars and other non-Delphi stuff.

I also think of something like:

1) Exactly what I want - loads multiple parts with progressbars from many files to the same file

OR

2) Something like telnet, so I just write the HTTP commands as strings and I get the byte arrays in reverse order, which I can either convert to strings or save to streams.

+3


source to share


1 answer


I can't comment on other libraries, but Indy supports everything you ask for.

The component TIdHTTP

can upload files using byte ranges if the server supports it. You can use the method TIdHTTP.Head()

to check if the server is returning a title Accept-Ranges: bytes

for a given url. If so, then you can set the properties TIdHTTP.Request.ContentRangeStart

and TIdHTTP.Request.ContentRangeEnd

to the bytes you want if needed when loading data from that url. To load multiple parts of a resource at the same time, simply execute multiple threads, each with its own component TIdHTTP

, in parallel. You will either have to download multiple chunks to separate the temporary files and then merge them into the final file when finished, or create one pre-defined file and then open multiple objects TFileStream

to it with the desired starting offsets forTIdHTTP

for recording.

TIdHTTP

supports cookies. You can assign a component TIdCookieManager

to a property TIdHTTP.CookieManager

or leave it unassigned and TIdHTTP

will create an object TIdCookieManager

within itself. In any case, also set the property TIdHTTP.AllowCookies

to True.

Login to the site should be done in one of two ways, depending on whether the server is using HTTP authentication or WebForm based authentication:



  • For HTTP authentication, add the desired units IdAuthentication...

    to your clause uses

    like IdAuthentacationDigest

    and IdAuthenticationNTLM

    , or IdAllAuthentications

    unt to register individual authentication classes with TIdHTTP

    and then set the TIdHTTP.Request.UserName

    and properties TIdHTTP.Request.Password

    as needed. If the WebServer requests authentication at the time of the request, it TIdHTTP

    will select the appropriate class and use it to log in with the specified credentials. If the server rejects the credentials, an event TIdHTTP.OnAuthorization

    will be fired to give you the opportunity to change the credentials and try again.

  • For WebForm authentication, there is a class TIdMultipartFormDataStream

    that can be passed to a method TIdHTTP.Post()

    to host a formatted POST request multipart/form-data

    .

TIdHTTP

Indy 10 supports gzip files and photo compressed files ( TIdHTTP

Iny 9 and earlier does not support compression). You can assign the TIdZLibCompressorBase

nested component to, for example TIdCompressorZLib

, a property TIdHTTP.Compressor

, and it TIdHTTP

will handle all the details for you, including sending the appropriate request header Accept-Encoding

and validating server Transfer-Encoding

.

TIdHTTP

has OnWorkBegin

, OnWork

and OnWorkEnd

events that can be used for a progress bar, etc. The event OnWorkBegin

has a parameter AWorkCountMax

that defines the total number of bytes transferred, if known, otherwise it will be 0 (HTTP responses that use a header Content-Length

are set accordingly AWorkCountMax

, but HTTP responses that use a header Transfer-Encoding: chunked

will not). The event OnWork

has a parameter AWorkCount

that indicates how many bytes have been transferred so far, regardless of the value AWorkCounMax

.

+7


source






More articles:


All Articles