File_get_contents () with modified HTTP headers returning garbage html output
The following code is used to extract html using the SIMPLETHTMLDOM parser for php.
include('simple_html_dom.php');
$context = stream_context_create(array(
'http'=>array(
'method'=>"GET",
'header'=>"Accept: text/html,application/xhtml+xml,application/xml\r\n" .
"Accept-Charset: ISO-8859-1,utf-8\r\n" .
"Accept-Encoding: gzip,deflate,sdch\r\n" .
"Accept-Language: en-US,en;q=0.8\r\n",
'user_agent'=>"User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11\r\n"
)
));
$html = file_get_contents('http://www.nseindia.com/content/equities/cmbhav.htm', false, $context);
echo $html;
foreach($html->find('a') as $e)
echo $e->href . '<br>';
The output is Garbage html:
‹µVßsâ6~Ï_¡s§¹ð2¹^:ìNˆÝ†)!ið=äéFØV"K>I˜pMÿ÷®lCšéÝKy@+í¾ý´òè]t}‘ÜÝÄä2¹š’›Oãéä‚x]J'qò¥Q5š=ŸÒxæ…G£Ü À2,·ÂÙ<&]YsËÁ 8eŒ%ãœU©*·#Úئ"Ù$×°<* ôRc<¢AóänÏ/ã8!v[B`áÑRÔâ&Õ¼´D0¹Z³Þ=«X³è£SuohªŠBÉåZ¦¦wo¼pD"o*¦ 7"88ÉTº.@Ú¢ókà[ålþÓ`[ÐæÀà,8yö_¸Pãí$ëŸHVñ³J÷XYÎXAð~Ö¤¬„÷ƒXŠÜ³óôTg/ "Wª×'ñÓSmÖgm‡€åJ’œÉL@¬µÒ'ò—»Ö’X½†áßdÃe¦6=%ÁéIðÒzH0«¸ÂìÍIgxô@>W¾|°´xäÃBЖ5ãëèŽ,V©JÞËúççU0½â2ð‰Uå^n„ø*·¨j¦žÙ<𱾯ˆt˜zùº.GxöjjìV AÅ _pÁívó,9$¥2H[%la"X[":zÐ/‡¤Ù´‘¿v5xô‡uªnÄ,}G6Ì3IÎÇÓ˜¤ „)YÊå ³](FÁ-—,Ëšåö4}ßÿÆ¿wÝî! 3V‘1²rn™¶†\‚†nõÿu£æœ?ºÔí0p,³aQ{¡¤Åz"s ì_ÛÚ¬$8ó}R1ÁW2Àsï{O§7çQ4™ýŽ§t³ùÍùE3_ßFñ-.7ÕúøËi‹h°?øná£+%î…¬@ðdðáÅ^$̘Àx3—&±¶ä˜‰ÃŸ¡óB‡ÇraÊaó&m§ÉU%¶Ž6//¼Ä™s Ç+;$;+l 5Ú6†Âg.VÙº¾PÎ9Üõ¸ïô/,—U»^1ý–Lp®æ·²ËÁ{3 ¤;:Z•±ms€6RâVÈ>ÈaÓñp˜à ¶BTs,¹uu‹Úzôý±M¢FñªÀKĤÅÞ]]ÐáAgÇíMèöiö*[ûN×Ç]±ÚJì@ιÁfÈS&hüç§I2‰çôÔïŸÒ«ó[šþÏ8ºù_|$ªÞW^zï¶Ö@Â&ŒÔF Å2²ä—Hƒ¬CdD?ÿ‡\²ÅòÛ¹ ÑA.Ïø¤xÏ@¿Iè£7)à`pi¢¬Ét‰ÖÐæÄæ°gk0jB<=óšé4çºês>¿Ð.]¤Âhš ›\jvìÒ¦æ"S8>Ѻï9‰V÷±W åu»ixH-[à—º¸ÇÀ͇Å?§m2"
Where am I doing Wrong !! I have checked the header information from the fiddler. This is the actual header information when the link is opened by the browser.
[Client]
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11
[Transport]
Host: www.nseindia.com
Proxy-Connection: keep-alive
+3
source to share
1 answer
In the HTTP request, you explicitly state that you can handle the compressed data, so the server returns:
Accept-Encoding: gzip,deflate,sdch\r\n
You should now decode the compressed data:
$html = gzuncompress($html);
As piotrekkr
mentioned in the comment, you can also remove the title Accept-Encoding
and the web server should return plain text instead.
0
source to share