HttpClient vs HtmlUnit
I know what the HtmlUnit
browser is mimicking and HttpClient
not.
Q HtmlUnit
, when the page is loaded and there is JavaScript inside, will the script execute? If the script sets a cookie, will the cookie be set in the browser HtmlUnit
and available from Java code?
Is there anything that can be done with HttpClient
, but not using HtmlUnit
? As HtmlUnit
possible to start from a POST request and change any part of the HTTP-request, including the method, URI, HTTP version, headers and body?
What are the advantages HttpClient
over HtmlUnit
?
source to share
HttpClient
is a lower level library for sending HTTP requests and receiving responses.
HtmlUnit
is at a higher level and uses internally HttpClient
to make HTTP requests, but also handles JavaScript (via Rhino
and internal DOM implementation), XPath (via Xalan
), CSS (via CSSParser
), mangled HTML (via NekoHtml
), WebSockets (via Jetty
), etc. ...
You can change outgoing requests and response HtmlUnit
as follows:
new WebConnectionWrapper(webClient) {
public WebResponse getResponse(WebRequest request) throws IOException {
WebResponse response = super.getResponse(request);
if (request.getUrl().toExternalForm().contains("my_url")) {
String content = response.getContentAsString("UTF-8");
//change content
WebResponseData data = new WebResponseData(content.getBytes("UTF-8"),
response.getStatusCode(), response.getStatusMessage(), response.getResponseHeaders());
response = new WebResponse(data, request, response.getLoadTime());
}
return response;
}
};
as outlined here .
You can change the used HttpClient
in HtmlUnit
by overriding HttpWebConnection.createHttpClient()
.
You can make a request POST
:
WebRequest webRequest = new WebRequest(url, HttpMethod.POST);
HtmlPage page = webClient.getPage(webRequest);
source to share