get first 1000 results from a google search

I'm trying to get the first 1000 search result links in google with C#. So far, I've modified Shiv Kumar's Finding links on a Web page to look for links in: string webpageUrl = "http://www.google.com/se开发者_如何学Goarch?hl=en&num=100&q=" + "concept"; however since the google page doesn't show all the 1000 results i need to find the way to get the rest of them. could that be accomplished without google api?

gilibi

---

I'd recommend you use the API.

Using "screen scraping" from HTML is problematic and requires frequent maintenance work - especially on a page like Google which will almost certainly change several times a year and which often uses redirects to track link usage.

---

Alternatively, if you really want to use the HTML route then take a look at the query parameters - e.g. "&start=10" - this should allow you to iterate over the pages.

But there's no guarantee that the query parameters will remain constant forever.