Using Parallel.ForEach with / or async / await
I am trying to check my image urls to see if they are valid. I have so many of them that it takes hours to complete this task. So I decided to do it asynchronously. I would like to know if there are any big differences or advantages in my code as shown below.
My main functions:
Async Function testUrl_async(ByVal myImageurl As String) As Task(Of Boolean)
myHttpResponse = Await myHttpClient.GetAsync(myImageurl)
If myHttpResponse.IsSuccessStatusCode Then
mySuccess = True
Else
mySuccess = False
End If
Return mySuccess
End Function
Function testUrl(ByVal myImageurl As String) As Boolean
myHttpResponse = myHttpClient.GetAsync(myImageurl)
If myHttpResponse.IsSuccessStatusCode Then
mySuccess = True
Else
mySuccess = False
End If
Return mySuccess
End Function
1) using async.
For Each myImage In myImages
Dim result=await testUrl_async(myImageUrl).Result
'some code
Next
2) using parallel foreach
Parallel.ForEach(myImages,
Sub(myImage)
testUrl(pictureComponent.websiteShop.hqpatronen, myImageUrl)
'some code
End Sub)
3) using parallel foreach and asnyc / await
Parallel.ForEach(myImages,
Sub(myImage)
await testUrl_async(pictureComponent.websiteShop.hqpatronen, myImageUrl)
'some code
End Sub)
The third option might be a better solution, but it won't let me call Await
/ Async
in ForEach
.
If I use the second, the function testurl
has an asynchronous HTTP call, but not with Await
, so it throws an error:
[TaskCanceledException: Task canceled.]
on the line that calls myHttpClient.GetAsync
. I am assuming that it is throwing this exception because it ForEach
ended and cancellation was requested, but the httpclient has not yet exited. How can I handle this if this might be the best solution?
Alternatively any other solution that makes my job faster.
source to share
You certainly don't want to use Parallel.ForEach
. Parallel
is designed to distribute CPU-bound algorithms across multiple cores, which will give you no benefit (in your scenario, your algorithm is not CPU-bound).
What you really want is concurrency, not parallelism. Asynchronous concurrency can be accomplished with Task.WhenAll
:
Dim tasks = myImages.Select(Function(x) testUrl_async(x))
Dim results = Await Task.WhenAll(tasks)
source to share