C# httpwebrequest批量提交数据到网站。

非新文,由自己以前在baidu网站转来的。(现在百度的博客不喜欢,转来这存档。很久以前的工作笔记了)

因为工作需要要将大致300多条数据提交到网站,而网站在提交 错数据后无法正常进入相应的数据页面进行修改,

不得己自己动手开发一个。

关键字:httpwebrequest, httpwebrespone, excel ,datagradview,httpwatch.cookie

首先下载一个httpwatch安装,在IE中截获提交信息

登录流:

POST /index.php/Public/checkLogin/ HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/x-ms-application, application/x-ms-xbap, application/vnd.ms-xpsdocument, application/xaml+xml, */*
Referer: http://syd.fjedu.gov.cn/index.php/Public/login
Accept-Language: zh-cn
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)
Host: syd.fjedu.gov.cn
Content-Length: 101
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: PHPSESSID=22g4terki0cnj498q17duhim46

account=账号名的URL编码(具体要编码后才知道)&password=XXXXXXXX&verify=8251

红色因保密需要而修改。

提交流:

POST /index.php/Sydmd/update/navTabId/index HTTP/1.1
x-requested-with: XMLHttpRequest
Accept-Language: zh-cn
Referer: http://syd.fjedu.gov.cn/index.php/Index/index#index
Accept: application/json, text/javascript, */*; q=0.01
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)
Host: syd.fjedu.gov.cn
Content-Length: 270
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: PHPSESSID=22g4terki0cnj498q17duhim46; _currentUrl_=czoyMjoiL2luZGV4LnBocC9TeWRtZC9pbmRleCI7

id=数据ID&name=姓名sex=性别&sfzid=身份证&minzhu=民族cityid=58&xianid=573&jhr_name=%E6%9E%97%E6%9C%88%E5%89%8D&jhr_tel=13799492115&address=%E5%BE%B7%E5%8C%96%E5%8E%BF%E6%9D%A8%E6%A2%85&knlx=2&ndkje=6000.00&mdremark=

红色部分保密,也为输入的参数。id为己提交数据时产生的ID,要在网站复制下来才可以得到。

二、登录

1cookie取得。为了取得cookie先get主页面

此代码在form1中执行,即程序运行即自动获取。

HttpWebRequest req;
            WebResponse wrp;
            Cookies = new CookieContainer();
           // Cookies.SetCookies("http://syd.fjedu.gov.cn/index.php/Public/login", "PHPSESSID=22g4terki0cnj498q17duhim46");
            InitializeComponent();
            req = WebRequest.Create(Index_URL) as HttpWebRequest;
            //req.Method = "GET";
            req.ProtocolVersion = HttpVersion.Version11;
           // req.Headers.Add("1.1"); 
            req.ContentType = "application/x-www-form-urlencoded";
            req.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)";
            //req.CookieContainer=Cookies ;
 
            //在这里对接收到的页面内容进行处理
            try
            {
                HttpWebResponse hwrs = (HttpWebResponse)req.GetResponse();
                StreamReader reader = new StreamReader(hwrs.GetResponseStream());
                String message = reader.ReadToEnd();
                RTB.Text = message;
                strCookie = hwrs.GetResponseHeader("Set-Cookie");//这里的strcookie是字符,不是C#中的Cookie类型,所以后面的使用以真正的cookie类型不同
                req.Abort();

}
            catch (Exception ex)
            {
            }
            finally
            {
                if (req != null)
                    req.Abort();
            }

2.获取验证码图片并显示。

private void pictureBox1_Click(object sender, EventArgs e)
        {
            HttpWebResponse wresponse;
            Stream s;
            HttpWebRequest req;
            req = (HttpWebRequest)HttpWebRequest.Create(Image_URL);
            req.Headers.Add("Accept-Language", "zh-cn");
            req.Headers.Add("Cookie", strCookie);//加入cookie才有效,没有cookie则取得的无效。。
            wresponse = (HttpWebResponse)req.GetResponse();
            try
            {

s = wresponse.GetResponseStream();
                System.Drawing.Image simg = System.Drawing.Image.FromStream(s);
                Bitmap newimg = new Bitmap(simg);
                pictureBox1.Image = simg;
                //Cookies = req.CookieContainer;
                //wresponse.Close;
            }
            catch (Exception ex)
            {
            }
            finally
            {
                if (req != null)
                    req.Abort();
            }

//pictureBox1.Load("http://syd.fjedu.gov.cn/index.php/Public/verify/");这个代码无cookie无效,所以不用。虽可显示。
        }

3.登录代码:

private void button1_Click(object sender, EventArgs e)
        {
            //登录
            //HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("");
            HttpWebRequest req1;
            string postData =String.Format("account={0}&password={1}&verify={2}",textBox1.Text , textBox2.Text, textBox3.Text);

req1 = WebRequest.Create(LOGIN_URL) as HttpWebRequest;
            try
            {

//StreamWriter requestWriter1 = new StreamWriter(req1.GetRequestStream());
                req1.Method = "POST";

req1.ContentType = "application/x-www-form-urlencoded";
                req1.ProtocolVersion = HttpVersion.Version11;
                req1.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322)";
                //Cookie: PHPSESSID=22g4terki0cnj498q17duhim46
                req1.Headers.Add("Accept-Language", "zh-cn");
                req1.Headers.Add("Accept-Encoding", "gzip, deflate");
                //req.Connection = Keep - Alive;
                req1.KeepAlive = true;
                //req.RequestUri = "/index.php/Public/checkLogin/";
                req1.Headers.Add("Cookie", strCookie);//加入cookie才有效,没有cookie则取得的无效。。

Encoding encoding = Encoding.GetEncoding("utf-8");
                
                //UrlEncode(postData, encoding);
                byte[] bytesToPost = encoding.GetBytes(postData);
                req1.ContentLength = bytesToPost.Length;

System.IO.Stream requestStream = req1.GetRequestStream();
                requestStream.Write(bytesToPost, 0, bytesToPost.Length);
                requestStream.Close();

HttpWebResponse hwrs1 = (HttpWebResponse)req1.GetResponse();
                StreamReader reader1 = new StreamReader(hwrs1.GetResponseStream());
                String message1 = reader1.ReadToEnd();
                RTB.Text = message1;
                //requestWriter.Close();成功....

}
            catch (WebException ex)
            {
                throw ex;
            }
                finally
                {
                    if (req1 != null)
                        req1.Abort();
                }

}

4.读取excel并填 充datagridview.

须引用:using System.Data.OleDb;

string strConn = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=D:/data.xls;" +
    "Extended Properties=‘Excel 8.0;HDR=Yes;IMEX=1‘";
            DataSet ds = new DataSet();
            OleDbDataAdapter oada = new OleDbDataAdapter("select * from [index$]", strConn);
            
            oada.Fill(ds);
            try
            {
                dataGridView1.DataSource = ds.Tables[0];//读取第一个表格,以excel表的顺序为准。具体可以自己试。
            }
            catch (Exception ex)
            {
                throw ex;
            }
      5.批量提交:

string Sid, sName,ssex,ssfzid,sminzhu,scityid,sxianid,sjhr_name,sjhr_tel,saddress,sknlx ;//提交顺序处理表格,并定义。
            //HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("");
            HttpWebRequest req2;
            //sName,ssex,ssfzid,sminzhu,scityid,sxianid,sjhr_name,sjhr_tel,saddress,sknlx

string postData = "";
            int i = 0,j=0;
            for (j = 0; j < 302; j++)
            {
                //sName = dataGridView1[i, j].Value.ToString(); i++;

req2 = WebRequest.Create(Post_URL) as HttpWebRequest;//httpwebrequest赋值后不能修改,所以放在循环中生成
                try
                {

//StreamWriter requestWriter1 = new StreamWriter(req1.GetRequestStream());
                    req2.Method = "POST";

req2.ContentType = "application/x-www-form-urlencoded";
                    req2.ProtocolVersion = HttpVersion.Version11;
                    req2.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322)";
                    //Cookie: PHPSESSID=22g4terki0cnj498q17duhim46
                    req2.Headers.Add("Accept-Language", "zh-cn");
                    req2.Headers.Add("Accept-Encoding", "gzip, deflate");
                    //req.Connection = Keep - Alive;
                    req2.KeepAlive = true;
                    //req.RequestUri = "/index.php/Public/checkLogin/";
                    req2.Headers.Add("Cookie", strCookie);

//id Sname Xb sfzid jhr_name jhr_tel address knlx minzhu cityid xianid
                    i = 0;
                    Sid = dataGridView1[i, j].Value.ToString(); i++;
                    sName = dataGridView1[i, j].Value.ToString(); i++;
                    ssex = dataGridView1[i, j].Value.ToString(); i++;
                    ssfzid = dataGridView1[i, j].Value.ToString(); i++;
                    sjhr_name = dataGridView1[i, j].Value.ToString(); i++;
                    sjhr_tel = dataGridView1[i, j].Value.ToString(); i++;
                    saddress = dataGridView1[i, j].Value.ToString(); i++;
                    sknlx = dataGridView1[i, j].Value.ToString(); i++;
                    sminzhu = dataGridView1[i, j].Value.ToString(); i++;
                    scityid = dataGridView1[i, j].Value.ToString(); i++;
                    sxianid = dataGridView1[i, j].Value.ToString();
                    postData = String.Format("id={0}&name={1}&sex={2}&sfzid={3}&minzhu={4}&cityid={5}&xianid={6}&jhr_name={7}&jhr_tel={8}&address={9}&knlx={10}&ndkje=6000.00&mdremark=", Sid, sName, ssex, ssfzid, sminzhu, scityid, sxianid, sjhr_name, sjhr_tel, saddress, sknlx);//中间的&号不要掉了,会出错,却记
                    Encoding encoding = Encoding.GetEncoding("utf-8");
                    byte[] bytesToPost = encoding.GetBytes(postData);

req2.ContentLength = bytesToPost.Length;

System.IO.Stream requestStream = req2.GetRequestStream();
                    requestStream.Write(bytesToPost, 0, bytesToPost.Length);

HttpWebResponse hwrs2 = (HttpWebResponse)req2.GetResponse();
                    StreamReader reader2 = new StreamReader(hwrs2.GetResponseStream());
                    String message2 = reader2.ReadToEnd();
                    RTB.Text = message2;//下面最好有结果判断。。。
                    requestStream.Close();
                   // progressBar1.Increment(1);
                    progressBar1.Value = j;
                    
                    
                    //requestWriter.Close();鎴愬姛....

}
                catch (WebException ex)
                {
                    throw ex;
                }
                finally
                {
                    if (req2 != null)
                        req2.Abort();//关闭httpwebrequest以便再生成。。。。
                }
            }//j循环结束

}
三小结。

此程序历时一天半才完成,期间遇到问题有:

1.httpwebrequest的使用问题,不可重复使用真让人郁闷。。

2.excel导入到datagridview

3..httpwebrequest使用cookie也是很大的问题。。。

时间: 2024-10-11 16:05:22

C# httpwebrequest批量提交数据到网站。的相关文章

Mobile Services批量提交数据

Mobile Services批量提交数据,参考了文章:Inserting multiple items at once in Azure Mobile Services.里面其实已经介绍得比较清楚了,但由于是英文,而且有些地方交待得不清楚,也没有Android的示例,故下文以Android版本的开发为例作个补充. 首先在Mobile Services项目里新建AllToDoItems以及ToDoItem表,点击AllToDoItems,再点击script标签,将里面的内容替换如下: funct

C# HttpWebRequest post提交数据,提交对象

1.客户端方法 //属于客户端 //要向URL Post的方法 public void PostResponse() { HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("http://demo2.cm-force.com/appapi/apiaccount/aa");//这是你要提交的url Encoding encoding = Encoding.UTF8; //参数少的情况下,直接拼接参数字符串 //strin

使用 HttpWebRequest 向网站提交数据(转)

使用 HttpWebRequest 向网站提交数据 http://www.blogjava.net/killme2008/archive/2011/05/20/105023.html 转自:http://www.cnblogs.com/webman/archive/2006/11/17/564106.html HttpWebRequest 是 .net 基类库中的一个类,在命名空间 System.Net 下面,用来使用户通过 HTTP 协议和服务器交互. HttpWebRequest 对 HTT

C#中使用 HttpWebRequest 向网站提交数据

HttpWebRequest 是 .NET 基类库中的一个类,在命名空间 System.Net 里,用来使用户通过 HTTP 协议和服务器交互. HttpWebRequest 对 HTTP 协议进行了完整的封装,对 HTTP 协议中的 Header, Content, Cookie 都做了属性和方法的支持,很容易就能编写出一个模拟浏览器自动登录的程序. 下面是HttpWebRequest的一些属性,这些属性对于轻量级的自动化测试程序是非常重要的. AllowAutoRedirect:获取或设置一

C# HttpWebRequest提交数据方式浅析

原文:http://developer.51cto.com/art/200909/149995.htmC# HttpWebRequest提交数据方式其实就是GET和POST两种,那么具体的实现以及操作注意事项是什么呢?那么本文就向你详细介绍C# HttpWebRequest提交数据方式的这两种利器. AD:2014WOT全球软件技术峰会北京站 课程视频发布 C# HttpWebRequest提交数据方式学习之前我们先来看看什么是HttpWebRequest,它是 .net 基类库中的一个类,在命

【转】C# HttpWebRequest提交数据方式

[转]C# HttpWebRequest提交数据方式 HttpWebRequest和HttpWebResponse类是用于发送和接收HTTP数据的最好选择.它们支持一系列有用的属性.这两个类位 于System.Net命名空间,默认情况下这个类对于控制台程序来说是可访问的.请注意,HttpWebRequest对象不是利用new关键字通过构 造函数来创建的,而是利用工厂机制(factory mechanism)通过Create()方法来创建的.另外,你可能预计需要显式地调用一个"Send"

批量插入数据(基于Mybatis的实现-Oracle)

前言:做一个数据同步项目,要求:同步数据不丢失的情况下,提高插入性能. 项目DB框架:Mybatis.DataBase:Oracle. ---------------------------------------------------------------------------- 批量插入数据方式: 一.Mybatis 全局设置批处理: 二.Mybatis 局部设置批处理: 三.Mybatis foreach批量插入: ①SELECT UNION ALL: ②BEGIN INSERT I

向mysql中批量插入数据的性能分析

MYSQL批量插入数据库实现语句性能分析 假定我们的表结构如下 代码如下   CREATE TABLE example (example_id INT NOT NULL,name VARCHAR( 50 ) NOT NULL,value VARCHAR( 50 ) NOT NULL,other_value VARCHAR( 50 ) NOT NULL) 通常情况下单条插入的sql语句我们会这么写: 代码如下   INSERT INTO example(example_id, name, valu

用python操作mysql数据库(之批量插入数据)

#!/usr/bin/env python # -*- coding: utf-8 -*- import MySQLdb #建立连接 conn = MySQLdb.connect(host='127.0.0.1',user='root',passwd='1qaz#EDC',db='test_db') cur = conn.cursor() #对数据进行操作 li = [('tanzhenx','shaoguan'),('huangmengdie','shaoguan')] #定义一个列表,列表中