How to extract text between <h1> </h1> in Python?
I am delaying text extraction between <h1>
and </h1>
.
Please help me.
My code:
import bs4
import re
import urllib2
url2='http://www.flipkart.com/mobiles/pr?sid=tyy,4io&otracker=ch_vn_mobile_filter_Top%20Brands_All#jumpTo=0|20'
htmlf = urllib2.urlopen(url2)
soup = bs4.BeautifulSoup(htmlf)
#res=soup.findAll('div',attrs={'class':'product-unit'})
for res in soup.findAll('a',attrs={'class':'fk-display-block'}):
suburl='http://www.flipkart.com/'+res.get('href')
subhtml = urllib2.urlopen(suburl)
subhtml = subhtml.read()
subhtml = re.sub(r'\s\s+','',subhtml)
subsoup=bs4.BeautifulSoup(subhtml)
res2=subsoup.find('h1',attrs={'itemprop':'name'})
if res2:
print res2
Output:
<h1 itemprop="name">Moto G</h1>
<h1 itemprop="name">Moto E</h1>
<h1 itemprop="name">Moto E</h1>
But I want this:
Moto G
Moto E
Moto E
+3
source to share
1 answer
In any HTML tag, when executed, get_text()
yields the text associated with the tag. Thus, you just need to use get_text()
in res2. i.e.,
if res2:
print res2.get_text()
PS . As a side note, I think this line subhtml = re.sub(r'\s\s+','',subhtml)
in your code is an expensive operation. If all you are doing is getting rid of the extra whitespace, you can do it with
if res2:
print res2.get_text().strip()
+5
source to share