EDN Admin
Well-known member
Im trying to create a quick little program in VB 2008 that can parse extract the google search results based on the search criteria of the user input.
Basically what I have is a textbox1.text that enters the keywords into google, google will then put the url results back into my richtextbox1.text.
Here is the code that I have:
<div style="color:Black;background-color:White; <pre>
<span style="color:Blue; Private <span style="color:Blue; Sub WebBrowser1_DocumentCompleted(<span style="color:Blue; ByVal sender <span style="color:Blue; As System.Object, <span style="color:Blue; ByVal e <span style="color:Blue; As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) <span style="color:Blue; Handles WebBrowser1.DocumentCompleted
<span style="color:Blue; Me.Text = WebBrowser1.Url.AbsoluteUri
<span style="color:Blue; If WebBrowser1.Document.Body.InnerHtml.Contains(<span style="color:#A31515; "http://www.google.com/search?q=") <span style="color:Blue; Then
<span style="color:Blue; Dim htmlele <span style="color:Blue; As HtmlElementCollection
htmlele = WebBrowser1.Document.GetElementsByTagName(<span style="color:#A31515; "h3")
<span style="color:Blue; For <span style="color:Blue; Each htm <span style="color:Blue; As HtmlElement <span style="color:Blue; In htmlele
<span style="color:Blue; Dim chld <span style="color:Blue; As HtmlElementCollection = htm.GetElementsByTagName(<span style="color:#A31515; "a")
<span style="color:Blue; For <span style="color:Blue; Each ch <span style="color:Blue; As HtmlElement <span style="color:Blue; In chld
RichTextBox1.AppendText(ch.GetAttribute(<span style="color:#A31515; "href") & vbCrLf)
<span style="color:Blue; Next
<span style="color:Blue; Next
<span style="color:Blue; End <span style="color:Blue; If
<span style="color:Blue; End <span style="color:Blue; Sub
[/code]
<br/>
The issue with this code is that it will scrape the google search engine results, but it also brings back links like this:
<pre>http://forum.ableton.com/viewtopic.php?f=10&t=166233
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CCMQmAcwAA
http://www.gearslutz.com/board/rap-hip-hop-engineering-production/637344-making-song-hip-hop-forum-collaboration-part-2-production.html
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CDMQmAcwAQ
http://www.soundonsound.com/forum/showflat.php?Board=PCMus&Number=938564
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CD0QmAcwAg
http://www.homerecordingconnection.com/forum.php?action=view_thread&id=20071&frm=8
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CEUQmAcwAw
https://www.propellerheads.se/forum/showthread.php?t=138153
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CFEQmAcwBA
http://www.kvraudio.com/forum/viewtopic.php?p=4651455
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CF4QmAcwBQ
http://www.inthemix.com.au/forum/forumdisplay.php?forumid=34
http://serato.com/forum/discussion/378801
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CHIQmAcwBw
http://www.boyinaband.com/forum/viewtopic.php?f=12&t=5543
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CHoQmAcwCA
http://www.japan-guide.com/forum/quereadisplay.html?0+16859
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CIMBEJgHMAk[/code]
<br/>
I do not want the links/urls that contain:
<pre>http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CCMQmAcwAA[/code]
I rather only have the links that look like this to enter into Richtextbox1.text:
<pre>http://forum.ableton.com/viewtopic.php?f=10&t=166233
http://www.gearslutz.com/board/rap-hip-hop-engineering-production/637344-making-song-hip-hop-forum-collaboration-part-2-production.html
http://www.soundonsound.com/forum/showflat.php
http://serato.com/forum/discussion/378801.php
http://www.boyinaband.com/forum/viewtopic.php?f=12&t=5543
http://www.japan-guide.com/forum/quereadisplay.html?0+16859
[/code]
<br/>
All the links/URLs will be extracted from the google SERPS and sent to Richtextbox1.text after button1 is clicked.
Any help would be appreciated.
<hr class="sig Inforequester123 - social.msdn.microsoft.com forums
View the full article
Basically what I have is a textbox1.text that enters the keywords into google, google will then put the url results back into my richtextbox1.text.
Here is the code that I have:
<div style="color:Black;background-color:White; <pre>
<span style="color:Blue; Private <span style="color:Blue; Sub WebBrowser1_DocumentCompleted(<span style="color:Blue; ByVal sender <span style="color:Blue; As System.Object, <span style="color:Blue; ByVal e <span style="color:Blue; As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) <span style="color:Blue; Handles WebBrowser1.DocumentCompleted
<span style="color:Blue; Me.Text = WebBrowser1.Url.AbsoluteUri
<span style="color:Blue; If WebBrowser1.Document.Body.InnerHtml.Contains(<span style="color:#A31515; "http://www.google.com/search?q=") <span style="color:Blue; Then
<span style="color:Blue; Dim htmlele <span style="color:Blue; As HtmlElementCollection
htmlele = WebBrowser1.Document.GetElementsByTagName(<span style="color:#A31515; "h3")
<span style="color:Blue; For <span style="color:Blue; Each htm <span style="color:Blue; As HtmlElement <span style="color:Blue; In htmlele
<span style="color:Blue; Dim chld <span style="color:Blue; As HtmlElementCollection = htm.GetElementsByTagName(<span style="color:#A31515; "a")
<span style="color:Blue; For <span style="color:Blue; Each ch <span style="color:Blue; As HtmlElement <span style="color:Blue; In chld
RichTextBox1.AppendText(ch.GetAttribute(<span style="color:#A31515; "href") & vbCrLf)
<span style="color:Blue; Next
<span style="color:Blue; Next
<span style="color:Blue; End <span style="color:Blue; If
<span style="color:Blue; End <span style="color:Blue; Sub
[/code]
<br/>
The issue with this code is that it will scrape the google search engine results, but it also brings back links like this:
<pre>http://forum.ableton.com/viewtopic.php?f=10&t=166233
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CCMQmAcwAA
http://www.gearslutz.com/board/rap-hip-hop-engineering-production/637344-making-song-hip-hop-forum-collaboration-part-2-production.html
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CDMQmAcwAQ
http://www.soundonsound.com/forum/showflat.php?Board=PCMus&Number=938564
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CD0QmAcwAg
http://www.homerecordingconnection.com/forum.php?action=view_thread&id=20071&frm=8
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CEUQmAcwAw
https://www.propellerheads.se/forum/showthread.php?t=138153
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CFEQmAcwBA
http://www.kvraudio.com/forum/viewtopic.php?p=4651455
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CF4QmAcwBQ
http://www.inthemix.com.au/forum/forumdisplay.php?forumid=34
http://serato.com/forum/discussion/378801
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CHIQmAcwBw
http://www.boyinaband.com/forum/viewtopic.php?f=12&t=5543
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CHoQmAcwCA
http://www.japan-guide.com/forum/quereadisplay.html?0+16859
http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CIMBEJgHMAk[/code]
<br/>
I do not want the links/urls that contain:
<pre>http://www.google.com/search?q=hip+hop+drums+%22Post+a+reply%22+OR+%22post+a+comment%22++OR+-comments+closed+inurl:forum&hl=en&prmd=imvnsfd&tbm=dsc&tbo=u&sa=X&ei=LrSLTtrXLMigtweQnYSKCA&ved=0CCMQmAcwAA[/code]
I rather only have the links that look like this to enter into Richtextbox1.text:
<pre>http://forum.ableton.com/viewtopic.php?f=10&t=166233
http://www.gearslutz.com/board/rap-hip-hop-engineering-production/637344-making-song-hip-hop-forum-collaboration-part-2-production.html
http://www.soundonsound.com/forum/showflat.php
http://serato.com/forum/discussion/378801.php
http://www.boyinaband.com/forum/viewtopic.php?f=12&t=5543
http://www.japan-guide.com/forum/quereadisplay.html?0+16859
[/code]
<br/>
All the links/URLs will be extracted from the google SERPS and sent to Richtextbox1.text after button1 is clicked.
Any help would be appreciated.
<hr class="sig Inforequester123 - social.msdn.microsoft.com forums
View the full article