Can Linq AsParallel () dispose of SoapHttpClientProtocol objects prematurely?
In an ASP.Net MVC 4 web application that I am working on. I have one page that basically generates a report by fetching data from a SOAP service.
My code basically looks like this
List<CustomThings> serverInfos = ServerInfos;
serverInfos.AsParallel().ForAll(srvInfo =>
{
SoapHttpClientProtocol soapProxy = CreateProxy(srvInfo);
//call make soap calls through the soap client
//store results in the proper places
}
The reason I am doing AsParallel is because doing multiple HTTP requests in sequential order takes forever. I must point out that this code works, albeit sporadically .
Is it possible that things are getting rid of in an unpredictable way and PLINQ is not a good solution for what I am trying to do here?
Is it possible that another threading problem could cause an error that would cause the soap client to "opt out"?
Additional Information
This particular mouse proxy talks to ArcGIS Server. You can usually check the server logs and see when certain requests are being initiated and if the requests have failed. Nothing is displayed in these logs.
Here is an example of an internal exception stack trace from AsParallel code.
Exception: System.AggregateException: One or more errors occurred. ---> System.Net.WebException: The underlying connection was closed: The connection that was expected to be saved was closed by the server. ---> System.IO.IOException: Unable to read data from transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote hostat System.Net.Sockets.Socket.Receive (Byte [] buffer, Int32 offset, Int32 size, SocketFlags socketFlags) to System.Net.Sockets.NetworkStream.Read (Byte [] buffer, Int32 offset, Int32 size) --- End of internal exception stack trace --- at System.Net.Sockets.NetworkStream.Read (Byte [] buffer, Int32 offset, Int32) at System.Net.PooledStream.Read (Byte [] buffer, Int32 offset, Int32 size) at System.Net.Connection.SyncRead (HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead) --- End of internal exception stack trace --- when System.Web.Services.Protocols.WebClientProtocol.GetWebResponse (WebRequest request) in System.Web.Services .Protocols.HttpWebClientProtocol.GetWebResponse (WebRequest request) at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke (String methodName, Object []) at ESRI.ArcGIS.SOAP.FeatureServerProxy.Query (Int32 LayerOrTableID, String DefinitionExpression, QueryFilter QueryFilter, ServiceDataOptions ServiceDataOptions, String GdbVersion, Double MaximumAllowableOffset) in System.Linq.Parallel.SelectQueryOperator
2.SelectQueryOperatorResults.GetElement(Int32 index) at System.Linq.Parallel.QueryResults
1.get_Item (Int32 index) in System.Linq.Parallel.PartitionedDataSource1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey) at System.Linq.Parallel.PipelineSpoolingTask
2.SpoolingWork () at System.Linq.Parallel.SpoolingTaskBase.Work () in System.Linq.Parallel.QueryTask.BaseWork (no object used) in System. Linq.Parallel.QueryTask. <.cctor> b__0 (Object o) at System.Threading.Tasks.Task.InnerInvoke () at System.Threading.Tasks.Task.Execute ()
source to share
PLINQ doesn't even know your connection object exists. He cannot close it.
Read the message carefully:
An existing connection was forcibly closed by the remote host.
The server closed the connection unexpectedly. It's not your client's fault.
Interpreting an exception for sure is an essential debugging skill. That information was right there in the exception message.
You may be putting too much load on it. Set a steady degree of parallelism. The default heuristic is for CPU operation, not I / O.
.WithDegreeOfParallelism(10)
The connection that should have been saved has been closed by the server.
This may mean that the server does not support HTTP.
source to share
I don't think you are doing anything terribly wrong with AsParallel
your Soap HTTP requests, and I don't think this is a threading issue.
However, parallel requests are clearly pushing your client / server into a number of connection restrictions, which is why you see connections being closed.
I would bet that your client, server, or both are not configured to handle the number of concurrent connections you are sending. This is why it works when you serially run queries.
I think you don't have access to the server config, so one thing you can do is control the number of parallel requests you issue to the server, at the same time setting a parameter ParallelEnumerable.WithDegreeOfParallelism
like in the following snippet
.AsParallel()
.WithDegreeOfParallelism(15)
This way, you manage parallelism and don't run the risk of overwhelming the server with a lot of requests for a small number of connections.
On the client side, you need to make sure you set the maximum level. the number of concurrent client connections to the corresponding number to ensure that your requests can use separate connections to the server and prevent the reuse of connections that could cause Keep-Alive issues. The server may close a connection if the number of requests using the connection has exceeded the maximum number of connections that can support the maximum number of connections, or if it exceeds the timeout setting.
You can set the client connection level limit using the parameter ServicePointManager.DefaultConnectionLimit
. For example. you can set it to 50:
System.Net.ServicePointManager.DefaultConnectionLimit = 50;
Here's an example of setting max. connecting to 50 using a config file:
<configuration>
<system.net>
<connectionManagement>
<add address="*" maxconnection="50" />
</connectionManagement>
</system.net>
I used "50" as an example, you should determine / calculate / measure which is the best parameter for your installation.
Also make sure you delete your HTTP connections correctly after every request to prevent connection timeouts.
source to share