WebBrowser control throws a seemingly random NullReferenceException
I have been working on a WebBrowser based web browser for several days. After a few prototypes working with Threads and DocumentCompleted events, I decided to give it a try and see if I could make a simple and straightforward Webscraper.
The goal is to create a Webscraper that does not include actual Thread objects. I want it to work consistently (e.g. navigate to a url, take actions, navigate to a different url, etc.).
This is what I got so far:
public static class Webscraper
{
private static WebBrowser _wb;
public static string URL;
//WebBrowser objects have to run in Single Thread Appartment for some reason.
[STAThread]
public static void Init_Browser()
{
_wb = new WebBrowser();
}
public static void Navigate_And_Wait(string url)
{
//Navigate to a specific url.
_wb.Navigate(url);
//Wait till the url is loaded.
while (_wb.IsBusy) ;
//Loop until current url == target url. (In case a website loads urls in steps)
while (!_wb.Url.ToString().Contains(url))
{
//Wait till next url is loaded
while (_wb.IsBusy) ;
}
//Place URL
URL = _wb.Url.ToString();
}
}
I am a beginner programmer, but I think this is pretty simple code. This is why I hate the fact that for some reason the program is throwing a NullReferenceException in this code snippet:
_wb.Url.ToString().Contains(url)
I just called the _wb.Navigate () method, so the NullReference cannot be in the _wb object itself. So the only thing I can imagine is that the _wb.Url object is NULL. But while the _wb.IsBusy () loop should prevent this from happening.
So what is going on and how can I fix it?
source to share
Waiting busy ( while (_wb.IsBusy) ;
) on the UI thread is not recommended. If you are using the new async / await features of .Net 4.5 you can get a similar effect (like navigate to a url, take an action, navigate to a different url, etc.) you want
public static class SOExtensions
{
public static Task NavigateAsync(this WebBrowser wb, string url)
{
TaskCompletionSource<object> tcs = new TaskCompletionSource<object>();
WebBrowserDocumentCompletedEventHandler completedEvent = null;
completedEvent = (sender, e) =>
{
wb.DocumentCompleted -= completedEvent;
tcs.SetResult(null);
};
wb.DocumentCompleted += completedEvent;
wb.ScriptErrorsSuppressed = true;
wb.Navigate(url);
return tcs.Task;
}
}
async void ProcessButtonClick()
{
await webBrowser1.NavigateAsync("http://www.stackoverflow.com");
MessageBox.Show(webBrowser1.DocumentTitle);
await webBrowser1.NavigateAsync("http://www.google.com");
MessageBox.Show(webBrowser1.DocumentTitle);
}
source to share