c#


ServicePointManager, Parallel Execution, HttpClient Very odd and unstable behavior


I have see an odd behavior with c#.
Let me explain my aim which I 'm struggling with. I want to send so many http request as fast as I can. You can imagine I'm writing it for spidering some website. I want to my program to be as fast as possible and 100% accurate. no responses of http request should have an incorrect response or unseen.
I have chosen to work with HttpClient because it was the newest technology and most important, it doesn't throw exceptions in case of http protocol error codes (unlike HttpWebRequest which does) which affects performance heavily.
For maximum performance I have decided to uses Parallel.For to run requests as fast as it can. I simplified my code a little and just requested a simple unique page (for example my wifi modem administration page because of it had low latency and ping nearly about 1ms)
I counted the the successful requestes which made with loop counter, failed requests because of exception with exceptioncount ,successful http responses with successful and unsuccessful http responses with unsuccessful variables.
All things was Ok until I played with ServicePointManager.DefaultConnectionLimit value. I found that maximum concurrent connection which can be made is limited to this value, so for example if this value be 8 ,only 8 concurrent connections can be made and this affects the HttpClient.GetAsync() in the Parallel.For
I set ServicePointManager.DefaultConnectionLimit to diffrent values and saw diffrent so amazing odd results:
-If you set it to a little value (for example less than 16 or maybe less) you see everything is nearly fine. all 1000 httprequests are made and all are successfull.
-If you set it to a larger value (for example bigger than 32) you see some exception in some threads created by Parallel.For (usually IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.) It's ok, because maybe my Wifi modem may treat to reset some tcp connections.
--but if you set it to a big value (like 1000 or more) you see only a percent of request are successfull and no exception will raise! Note: no exception, but requests were lost without any notification. you should see the result your self!
Why and why and why????
using System;
using System.Net;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
namespace ConsoleApp1
{
class Program
{
static void Main(string[] args)
{
ServicePointManager.DefaultConnectionLimit = 1000;
dox();
Console.ReadKey();
}
static void dox()
{
string LoopOrder = "";
string ss = "";
int successful = 0, unsuccesful = 0;
int exceptionCount = 0;
int successfulLoops = 0;
HttpClient hc = new HttpClient();
Parallel.For(0, 1000, async (i) =>
{
try
{
var res = await hc.GetAsync("http://192.168.0.1");
Interlocked.Increment(ref successfulLoops);
if (res.IsSuccessStatusCode)
{
LoopOrder = LoopOrder + i + "-";
Interlocked.Increment(ref successful);
}
else
{
Interlocked.Increment(ref unsuccesful);
}
}
catch (Exception e)
{
Interlocked.Increment(ref exceptionCount);
Console.WriteLine(e.Message);
}
});
Thread.Sleep(10000); // I had used this because Parallel.For don't respect and wait for the execution of async methods.
//This is just waiting enough to parallel tasks to be finished
Console.WriteLine("successful http:" + successful);
Console.WriteLine("unsuccessful http:" + unsuccesful);
Console.WriteLine("loops raised exception:"+exceptionCount);
Console.WriteLine("successfull loops:" + successfulLoops);
Console.WriteLine(LoopOrder);
}
}
}

Related Links

Code Conversion from Classic ASP to ASP.NET
Easiest way to read single int from front of string
How does custom syntax highlighting in Scintilla work (and why doesn't mine)?
Defining Routes in .NET 3.5 with WCF
How to cast object to generic type?
How to achieve hierarchical data binding?
Getting A Window's Region
How to determine that a UserControl Collection Property has changed at Design Time?
Need Help for VB6 Code to C# Conversion
Checking for IEnumerable<T> with reflection
Custom Dispose in designed form or user control
Retrieve the stream file name from a XmlTextReader
C# WinForms - App Installation Error - Reference in the manifest does not match the identity of the downloaded assembly
ASP.NET MVC Form Validation. How do you do it on a non-model object?
SocketAsyncEventArgs Encryption — SSL vs. Custom Pre-Transmission Encryption
How to wait for Textbox to finish rendering?

Categories

HOME
asterisk
asp.net
max
postgresql
ide
cluster-analysis
popover
latex
x509certificate
xss
jshell
docker-compose
annyang
coding-style
google-app-maker
osclass
point-of-sale
minimum
responsive
game-engine
airbnb
polybase
vala
sha
setup-project
delphi-2010
android-scrollview
restler
incapsula
ponylang
abide
stencyl
var
julia-jump
susy
google-cloud-powershell
rexx
ack
spring-ide
opos
denodo
jscoverage
aot
hellosign
selection-sort
overhead
node.js-client
timex
acceleo
esprima
breadcrumbs
hdr
require-handlebars
patching
jmap
clique
sqldataadapter
commercetools
azure-mobile-app
borland-c++
sparkcore
p4python
waterline
ultraedit
pymol
report-builder2.0
zoneminder
pytz
nsmutableurlrequest
disabled-control
rails-migrations
reactjs-native
nativequery
facebook-likebox
xcode6.1-gm-seed
mencoder
google-checkout
symbian3
pudb
optimistic-concurrency
advanceddatagrid
helicontech
method-call
miniport
for-xml
scala-designer
bootstrap-protocol
high-load

Resources

Encrypt Message



code
soft
python
ios
c
html
jquery
cloud
mobile