Z
zydjohn
Guest
Hello:
I want to use HTTP Client to post some data to a web site.
First, I did this by hand: launch Chrome browser to visit the web site, and click on the web form, then fill in the form, then click “submit” button, as it always works.
Second, I setup a HTTP proxy using Fiddler Version 5.0, so all the HTTPS traffic went through the proxy.
I can see the HTTPS traffic when posting the web form, and I record all the information; then I tried to create a http client to post exactly the same data.
Here is my code:
public static async Task<string> Place_Order1(string encoded_body1)
{
try
{
string response1 = "";
HttpClientHandler handler = new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip
};
HttpClient client = new HttpClient(handler);
ServicePointManager.Expect100Continue = false;
try
{
string place_order_url1 = "https://dot.com/betslip/";
StringContent http_content = new StringContent(encoded_body1);
client.DefaultRequestHeaders.Add("Accept", "*/*");
client.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate, br");
client.DefaultRequestHeaders.Add("Connection", "keep-alive");
client.DefaultRequestHeaders.AcceptLanguage.TryParseAdd("en-gb;q=0.8");
client.DefaultRequestHeaders.Add("authority", "dot.com");
client.DefaultRequestHeaders.Add("origin", "https://dot.com");
client.DefaultRequestHeaders.Add("Referer", "https://dot.com");
client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3835.0 Safari/537.36");
client.DefaultRequestHeaders.Add("Content_Type", "application/x-www-form-urlencoded");
client.DefaultRequestHeaders.TryAddWithoutValidation("Cookie", DotCom_Cookies);
HttpResponseMessage reply1 = await client.PostAsync(place_order_url1, http_content);
response1 = await reply1.Content.ReadAsStringAsync();
return (response1);
}
return (response1);
}
catch (HttpRequestException ex)
{
Console.WriteLine("[Place_Order1] Exception: {0}", ex.Message);
return (null);
}
}
The DotCom_Cookies looks like the following data:
csrftoken=gWmnlHZSxxwJvLSXPUFygqVAszlNNf1DikwgIxIIF9TY5vAHwQHYtJHeONotUp; _gid=GA1.2.148678906.1565117481; _fbp=fb.1.1565117480950.1449659034; _gat_UA-41965734-1=1; sessionid=uqn8oyxqtyyws2ciizfc4x8rp5sxw42x; _ga=GA1.2.2133559866.1565117481; __cfduid=d4f5da4642b9bea9fcda2fc38bb33cca01565117479
The post data payload is something like the following:
csrfmiddlewaretoken=XYZ&side=0&price=1.10&market=123&runner=456&type=1&price_formatted=1.10&amount=1.00
I can see from Fiddler proxy, I made all the data in http client exactly the same as it appear in the proxy, but when I run my program, after post the data, I got the following reply:
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta name="robots" content="NONE,NOARCHIVE">
<title>403 Forbidden</title>
<style type="text/css">
html * { padding:0; margin:0; }
body * { padding:10px 20px; }
body * * { padding:0; }
body { font:small sans-serif; background:#eee; color:#000; }
body>div { border-bottom:1px solid #ddd; }
h1 { font-weight:normal; margin-bottom:.4em; }
h1 span { font-size:60%; color:#666; font-weight:normal; }
#info { background:#f6f6f6; }
#info ul { margin: 0.5em 4em; }
#info p, #summary p { padding- }
#summary { background: #ffc; }
#explanation { background:#eee; border-bottom: 0px none; }
</style>
</head>
<body>
<div id="summary">
<h1>Forbidden <span>(403)</span></h1>
<p>CSRF verification failed. Request aborted.</p>
</div>
<div id="explanation">
<p><small>More information is available with DEBUG=True.</small></p>
</div>
</body>
</html>
I check again and again to make sure the CSRF token values are the same in Fiddler proxy and in http client. It seems the web site detected I am not a human to post the form data, but why bother?
I have included the __cfduid cookie in the http client headers.
In this example, it is like this:
csrftoken=gWmnlHZSxxwJvLSXPUFygqVAszlNNf1DikwgIxIIF9TY5vAHwQHYtJHeONotUp
But why I got this kind of error.
Minor issue: I can’t add those cookies in pairs, if I did so, then I can’t use the http client to get the post form, but if I put all the cookies in one string and add them together as my code did, it worked, at least, I can use the httpclient to get the form to post the data.
By the way, when I used web browser to fill in the form and post the data, it seems web browser used XHR request, but I think http client should be able to do the same as XHR request, right?
If not, then how I can post the same data to the web server using Dot net (C#)?
Finally, I am use Visual Studio 2019 (Version 16.2.0) on Windows 10 (Version 1903)
Best Regards,
Continue reading...
I want to use HTTP Client to post some data to a web site.
First, I did this by hand: launch Chrome browser to visit the web site, and click on the web form, then fill in the form, then click “submit” button, as it always works.
Second, I setup a HTTP proxy using Fiddler Version 5.0, so all the HTTPS traffic went through the proxy.
I can see the HTTPS traffic when posting the web form, and I record all the information; then I tried to create a http client to post exactly the same data.
Here is my code:
public static async Task<string> Place_Order1(string encoded_body1)
{
try
{
string response1 = "";
HttpClientHandler handler = new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip
};
HttpClient client = new HttpClient(handler);
ServicePointManager.Expect100Continue = false;
try
{
string place_order_url1 = "https://dot.com/betslip/";
StringContent http_content = new StringContent(encoded_body1);
client.DefaultRequestHeaders.Add("Accept", "*/*");
client.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate, br");
client.DefaultRequestHeaders.Add("Connection", "keep-alive");
client.DefaultRequestHeaders.AcceptLanguage.TryParseAdd("en-gb;q=0.8");
client.DefaultRequestHeaders.Add("authority", "dot.com");
client.DefaultRequestHeaders.Add("origin", "https://dot.com");
client.DefaultRequestHeaders.Add("Referer", "https://dot.com");
client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3835.0 Safari/537.36");
client.DefaultRequestHeaders.Add("Content_Type", "application/x-www-form-urlencoded");
client.DefaultRequestHeaders.TryAddWithoutValidation("Cookie", DotCom_Cookies);
HttpResponseMessage reply1 = await client.PostAsync(place_order_url1, http_content);
response1 = await reply1.Content.ReadAsStringAsync();
return (response1);
}
return (response1);
}
catch (HttpRequestException ex)
{
Console.WriteLine("[Place_Order1] Exception: {0}", ex.Message);
return (null);
}
}
The DotCom_Cookies looks like the following data:
csrftoken=gWmnlHZSxxwJvLSXPUFygqVAszlNNf1DikwgIxIIF9TY5vAHwQHYtJHeONotUp; _gid=GA1.2.148678906.1565117481; _fbp=fb.1.1565117480950.1449659034; _gat_UA-41965734-1=1; sessionid=uqn8oyxqtyyws2ciizfc4x8rp5sxw42x; _ga=GA1.2.2133559866.1565117481; __cfduid=d4f5da4642b9bea9fcda2fc38bb33cca01565117479
The post data payload is something like the following:
csrfmiddlewaretoken=XYZ&side=0&price=1.10&market=123&runner=456&type=1&price_formatted=1.10&amount=1.00
I can see from Fiddler proxy, I made all the data in http client exactly the same as it appear in the proxy, but when I run my program, after post the data, I got the following reply:
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta name="robots" content="NONE,NOARCHIVE">
<title>403 Forbidden</title>
<style type="text/css">
html * { padding:0; margin:0; }
body * { padding:10px 20px; }
body * * { padding:0; }
body { font:small sans-serif; background:#eee; color:#000; }
body>div { border-bottom:1px solid #ddd; }
h1 { font-weight:normal; margin-bottom:.4em; }
h1 span { font-size:60%; color:#666; font-weight:normal; }
#info { background:#f6f6f6; }
#info ul { margin: 0.5em 4em; }
#info p, #summary p { padding- }
#summary { background: #ffc; }
#explanation { background:#eee; border-bottom: 0px none; }
</style>
</head>
<body>
<div id="summary">
<h1>Forbidden <span>(403)</span></h1>
<p>CSRF verification failed. Request aborted.</p>
</div>
<div id="explanation">
<p><small>More information is available with DEBUG=True.</small></p>
</div>
</body>
</html>
I check again and again to make sure the CSRF token values are the same in Fiddler proxy and in http client. It seems the web site detected I am not a human to post the form data, but why bother?
I have included the __cfduid cookie in the http client headers.
In this example, it is like this:
csrftoken=gWmnlHZSxxwJvLSXPUFygqVAszlNNf1DikwgIxIIF9TY5vAHwQHYtJHeONotUp
But why I got this kind of error.
Minor issue: I can’t add those cookies in pairs, if I did so, then I can’t use the http client to get the post form, but if I put all the cookies in one string and add them together as my code did, it worked, at least, I can use the httpclient to get the form to post the data.
By the way, when I used web browser to fill in the form and post the data, it seems web browser used XHR request, but I think http client should be able to do the same as XHR request, right?
If not, then how I can post the same data to the web server using Dot net (C#)?
Finally, I am use Visual Studio 2019 (Version 16.2.0) on Windows 10 (Version 1903)
Best Regards,
Continue reading...