Soup.find_all works but soup.select doesn't work
I am playing with parsing an html page using css selectors
import requests
import webbrowser
from bs4 import BeautifulSoup
page = requests.get('http://www.marketwatch.com', headers={'User-Agent': 'Mozilla/5.0'})
soup = BeautifulSoup(page.content, 'html.parser')
I am having trouble selecting a list tag with a class when using the select method. However, I have no problem using find_all method
soup.find_all('ul', class_= "latestNews j-scrollElement")
This returns the desired result, but for some reason I cannot do the same css selectors. I want to know what I am doing wrong.
Here's my attempt:
soup.select("ul .latestNews j-scrollElement")
which returns an empty list.
I cannot figure out what I am doing wrong with the select method.
Thank.
+3
source to share
1 answer
From the documentation :
If you want to search for tags that match two or more CSS classes, you should use a CSS selector:
css_soup.select("p.strikeout.body")
In your case, you would call it like this:
In [1588]: soup.select("ul.latestNews.j-scrollElement")
Out[1588]:
[<ul class="latestNews j-scrollElement" data-track-code="MW_Header_Latest News|MW_Header_Latest News_Facebook|MW_Header_Latest News_Twitter" data-track-query=".latestNews__headline a|a.icon--facebook|a.icon--twitter">
.
.
.
+4
source to share