I want to make a search spider?

agolangf · · 679 次点击    
这是一个分享于 的资源,其中的信息可能已经有所发展或是发生改变。
<p>So i want to make program that will allow the user to search for a term on Google and then the program will give all the websites that match the term.</p> <p>Ie. </p> <p>go run Google.go GoLang</p> <p>will give you all the links that show when you Google GoLang.</p> <p>I am guessing i could use some API, but there aren&#39;t any examples that i could find that do this......</p> <hr/>**评论:**<br/><br/>xrstf: <pre><p>Google has no interest in giving away free search results without the option of displaying their ads right next to them and tracking who you are and what you click.</p> <p>This explains why there is no Google API (don&#39;t know if Google offers commercial access via Google Apps for Business) and trying to automate access can get you throttled or your IP banned for a short time.</p> <p>These are the reasons why I would not even attempt to code a google search client. And why there is probably little sample code on how to accomplish that task.</p></pre>master_of_deception: <pre><p><a href="https://www.google.com/work/search/products/gss.html#pricing_content" rel="nofollow">https://www.google.com/work/search/products/gss.html#pricing_content</a></p></pre>tinke: <pre><p>You could try using the AJAX API, although I think it&#39;s being deprecated but am not sure. And while it still works, I wouldn&#39;t rely on it for anything serious.</p> <p>The URL you need to use is: <code> https://ajax.googleapis.com/ajax/services/search/web?v=1.0&amp;rsz=&lt;results&gt;&amp;start=&lt;offset&gt;&amp;q=&lt;query&gt; </code></p> <p>Where <code>&lt;results&gt;</code> would be the amount of results you want returned (limited to max 8, if I remember correctly) and <code>&lt;offset&gt;</code> would be the offset of the search. And <code>&lt;query&gt;</code> would be your search query, eg. golang.</p> <p>So assume you want 16 results, you would need to set the <code>&lt;results&gt;</code> to 8 and then make two requests, one where <code>offset</code> is set to 0, and another one where it&#39;s set to 8.</p> <p>So an example query of the first 16 results of &#34;golang&#34; would like like:</p> <ol> <li><code>https://ajax.googleapis.com/ajax/services/search/web?v=1.0&amp;rsz=8&amp;start=0&amp;q=golang</code></li> <li><code>https://ajax.googleapis.com/ajax/services/search/web?v=1.0&amp;rsz=8&amp;start=8&amp;q=golang</code></li> </ol></pre>nhooyr: <pre><p>Try <a href="https://github.com/headzoo/surf" rel="nofollow">https://github.com/headzoo/surf</a></p></pre>

入群交流(和以上内容无关):加入Go大咖交流群,或添加微信:liuxiaoyan-s 备注:入群;或加QQ群:692541889

679 次点击  
加入收藏 微博
暂无回复
添加一条新回复 (您需要 登录 后才能回复 没有账号 ?)
  • 请尽量让自己的回复能够对别人有帮助
  • 支持 Markdown 格式, **粗体**、~~删除线~~、`单行代码`
  • 支持 @ 本站用户;支持表情(输入 : 提示),见 Emoji cheat sheet
  • 图片支持拖拽、截图粘贴等方式上传