Reading and saving part of a file stream via IdHTTP
I want to download a file from an HTTP server via a file stream and only read (and save to a file) the first few lines, say 100. After reading the first 100 lines, the file stream should end: so what I do NOT want to download or read the entire file.
Below you can find what I have. The website is just an example. Can anyone guide me in the right direction?
const
myURL = https://graphical.weather.gov/xml/DWMLgen/schema/latest_DWML.txt
var
fs: TMemoryStream;
http: TIdHTTP;
begin
fs := TMemoryStream.Create;
http := TIdHTTP.Create(nil);
try
fs.Position := 0;
http.Get(myURL, fs);
fs.SaveToFile('test.xml');
finally
fs.Free;
http.free
end;
end;
source to share
If the HTTP server supports byte ranges for the desired URL (via the request header Range
), you can only request the specific bytes you want, and that's all the server will send. You can use a property TIdHTTP.Request.Range
for this when calling TIdHTTP.Get()
. To see if the server supports byte ranges, first use TIdHTTP.Head()
to get the url headers and then check if the header is there Accept-Ranges: bytes
(see property TIdHTTP.Response.AcceptRanges
).
If the server doesn't support byte ranges, you'll have to keep using the code you have now, just make some changes to it:
-
Instead of calling,
fs.SaveToFile()
create a separate objectTFileStream
and passTMemoryStream
it to aCopyFrom()
method so you can specify exactly how many bytes to store. -
Use an event,
TIdHTTP.OnWork
or useTIdEventStream
or output a customTStream
overrideWrite()
to keep track of how many bytes are being loaded, so you can abort (by throwing an exception likeEAbort
afterSysUtils.Abort()
) after you get the number of bytes you want.
Of course, either approach is byte oriented not line oriented . If you need to be line oriented, and especially if the strings are variable length, then you will have to use the second approach above, using TIdEventStream
either a custom one TStream
, so that you can implement the string parsing logic and keep only the completed lines to your file, then abort as soon as you got the number of lines you want.
source to share