The underlying connection was closed: The connection was closed unexpectedly in HTTPWebRequest

[EDN Admin]

I have an application written in c#.NET 2.0 Windows app. I am trying to implement web crawler using HttpWebRequest & HttpWebResponse. In this application i am using both http get & post request for a third party web sites. this web site provides user account for its members.
i am trying to login this sites using positing data then entering into my profile page & then trying to make another GET request & then make another POST request with post data. but i am not able to make successfull request .
it through an error

[background=transparent]
"The server committed a protocol violation. Section=ResponseHeader Detail=Header name is invalid"[/background]​

then i set <httpWebRequest useUnsafeHeaderParsing="true" /> in app.config files. after that it throw an error The underlying connection was closed:

[background=transparent]
The connection was closed unexpectedly : System.Net.WebException.[/background]​

anybody can tell me what the reason for showing this error
my code below. HttpWebRequest request = null;
HttpWebResponse response = null;
Stream dataStream = null;
StreamReader reader = null;

CookieContainer objContainer = new CookieContainer();
request = (HttpWebRequest)HttpWebRequest.Create(strBaseURL + "login.xhtml");
request.CookieContainer = objContainer; request.KeepAlive = true; response = (HttpWebResponse)request.GetResponse(); dataStream = response.GetResponseStream(); reader = new StreamReader(dataStream); strServerResponse = reader.ReadToEnd(); reader.Close(); dataStream.Close(); response.Close(); //======================================================================= StringBuilder strLinkBuilder = new StringBuilder(); strLinkBuilder.Append("j_username=username"); strLinkBuilder.Append("&j_password=password"); request = (HttpWebRequest)HttpWebRequest.Create(strBaseURL + "j_security_check"); // Set the Method property of the request to POST. request.Method = "POST"; // Create POST data and convert it to a byte array. string postData = strLinkBuilder.ToString(); byte[] byteArray = Encoding.UTF8.GetBytes(postData); // Set the ContentType property of the WebRequest. request.ContentType = "application/x-www-form-urlencoded"; request.KeepAlive = true; // Set the ContentLength property of the WebRequest. request.ContentLength = byteArray.Length; for (int i = 0; i < response.Cookies.Count; i++) { response.Cookies.Path = String.Empty; } request.CookieContainer = objContainer; request.CookieContainer.Add(response.Cookies); // Get the request stream. dataStream = request.GetRequestStream(); // Write the data to the request stream. dataStream.Write(byteArray, 0, byteArray.Length); // Close the Stream object. dataStream.Close(); // Get the response. response = (HttpWebResponse)request.GetResponse(); // Get the stream containing content returned by the server. dataStream = response.GetResponseStream(); // Open the stream using a StreamReader for easy access. reader = new StreamReader(dataStream); // Read the content. strServerResponse = reader.ReadToEnd(); // Clean up the streams. reader.Close(); dataStream.Close(); response.Close(); //------------------------------------------------------- request = (HttpWebRequest)HttpWebRequest.Create(strBaseURL + "ded/nsdlconsofile.xhtml"); //request.CookieContainer = objContainer; request.KeepAlive = true; // request.Method = "POST"; for (int i = 0; i < response.Cookies.Count; i++) { response.Cookies.Path = String.Empty; } request.CookieContainer = objContainer; request.CookieContainer.Add(response.Cookies); response = (HttpWebResponse)request.GetResponse(); dataStream = response.GetResponseStream(); reader = new StreamReader(dataStream); strServerResponse = reader.ReadToEnd(); reader.Close(); dataStream.Close(); response.Close(); //------------------------------------------------------- strLinkBuilder = new StringBuilder(); strLinkBuilder.Append("finYr=2012"); strLinkBuilder.Append("&qrtr=3"); strLinkBuilder.Append("&frmType=24Q"); strLinkBuilder.Append("&download_conso=Go"); strLinkBuilder.Append("&requestnsdlconsoForm_SUBMIT=1"); Dictionary<string, string> objNameval = TraceHiddenField(strServerResponse, ""); foreach (KeyValuePair<string, string> pair in objNameval) { strLinkBuilder.Append("&" + pair.Key + "=" + pair.Value); } request = (HttpWebRequest)HttpWebRequest.Create(strBaseURL + "ded/nsdlconsofile.xhtml"); // Set the Method property of the request to POST. request.ContentType = "application/x-www-form-urlencoded"; request.Method = "POST"; request.KeepAlive = false; // request.ServicePoint.Expect100Continue = false; request.UserAgent = "Mozilla/4.0 (compatible"; request.ProtocolVersion = HttpVersion.Version10; // Create POST data and convert it to a byte array. postData = strLinkBuilder.ToString(); byte[] byteData = Encoding.UTF8.GetBytes(postData); // Set the ContentType property of the WebRequest. // Set the ContentLength property of the WebRequest. request.ContentLength = byteData.Length; for (int i = 0; i < response.Cookies.Count; i++) { response.Cookies.Path = String.Empty; } request.CookieContainer = objContainer; request.CookieContainer.Add( response.Cookies); // Get the request stream. dataStream = request.GetRequestStream(); // Write the data to the request stream. dataStream.Write(byteData, 0, byteData.Length); // Close the Stream object. dataStream.Close(); // Get the response. response = (HttpWebResponse)request.GetResponse(); // Get the stream containing content returned by the server. dataStream = response.GetResponseStream(); // Open the stream using a StreamReader for easy access. reader = new StreamReader(dataStream); strServerResponse = reader.ReadToEnd(); reader.Close(); dataStream.Close(); response.Close();

gettting error at request.GetResponse();

View the full article