How to match both multiline and single line
I feel like I'm trying to ponder some Regex (using Python 2.7) and got into confusion. It's connected with (. *). I know that period matches everything except newline, unless you use the re.DOTALL tag. But when I use the tag it includes too many. Here is some code with several options and results I've tried:
import re
from urllib2 import urlopen
webpage = urlopen('http://trev.id.au/testfiles/rgxtxt.php').read()
# find the instances of pattern in the file
findPatHTMLComment = re.findall('<!--(.*)-->',webpage)
foundItems = len(findPatHTMLComment) # how many instances where found?
# Print results
print "Found " + str(foundItems) + " matches. They are: "
listIterator = []
listIterator[:]=range(0,foundItems)
for i in listIterator:
print "HTML_Comment["+ str(i) +"]: |" + findPatHTMLComment[i] + "| END HTML Comment"
This results in 3 matches as it doesn't find multiline comments.
Using:
findPatHTMLComment = re.findall('<!--(.*)-->',webpage,re.DOTALL)
Finds one match using the first one at the end of the document.
findPatHTMLComment = re.findall('<!--(.*)-->',webpage,re.MULTILINE)
Finds the same as the first, only 3 out of 5 comments that are in the file.
QUESTION: What should I use in this case as a regular expression? Could you explain this for me and others?
Appreciate any recommendations you can provide. Thanks and had a good day.
EDIT: Include example data that was in the link in the code above (will be deleting data from the server soon):
<html>
<!--[if lt IE 9 ]>
<script type="text/javascript">
jQuery(function ($) {
function TopSearchIE9(input,inputBtn){
var $topSearch=$(input);
var $topSearchBtn=$(inputBtn);
$topSearch.keydown(function(e) {
if (e.keyCode == 13) {
$topSearchBtn.trigger("click");
return false;
}
});
}
TopSearchIE9(".J-txt-focus1",".J-txt-focus1-btn");
TopSearchIE9(".J-txt-focus2",".J-txt-focus2-btn");
});
</script>
<![endif]-->
<!--[if lt IE 10 ]>
<style>
.new-header-search .hdSch-txt{ width: 225px;}
.new-header-search .hdSch-del{width: 0px; padding: 5px 0px;}
.new-header-search .hdSch-del.del{background:none; padding: }
</style>
<![endif]-->
<body>
<!-- This is a text file with a number of items to allow testing of some regex methods. It has no actual meaning -->
<div head1>Item heading for first item</div>
<!--By the way, this is a comment in a block of HTML text.-->
<div itembody>We can jump over the moon if we are fast enough, but we really shouldn't try it cause we may get a blood nose. When we do try and succeed it feels quite good.</div>
<div head1>Item heading for second item</div>
<div itembody>If this is showing, its the second body within the itembody div tags for this file</div>
<div head1>Item heading for the third item</div>
<div itembody>
Going to add another div tag
<div highlight>
and closing div tag
</div>
in this body to see how it handles that.
</div>
<!-- The above itembody data should
have it own div and closing div tags -->
<div head1>Item heading for the fourth item</div>
<div itembody>
<p><a href="mailto:fred@flinstone.com">email fred</a> or phone him on +63 493 3382 3329 when you are ready to try more regex stuff.</p>
<p>You can also check with Barney by <a href="mailto:barney@rubble.com">emailing him</a> or phone him of +44 394 394 3992 if that is easier</p>
</div>
<!-- Thats all folks... -->
</body>
source to share
But when I use the tag it includes too many.
*
is greedy , meaning it will match as much as it can and still allows the remaining regex to match. You must follow the *
c operator ?
for a non-greedy match, which means zero or more - preferably as little as possible.
re.findall('<!--(.*?)-->', webpage, re.DOTALL)
↑
The flag re.MULTILINE
is called multiline because bindings ^
and $
work on multiple lines when implemented, which is redundant in this case using a multiline modifier.
In another post I would look at BeautifulSoup for this task.
from bs4 import BeautifulSoup, Comment
soup = BeautifulSoup(html)
comments = soup.find_all(text=lambda text:isinstance(text, Comment))
source to share