WebCrawling User-authenticated websites

Is 开发者_开发技巧it possible to crawl user-authenticated websites using c#?

---

Yes, it is possible assuming you have the credentials. Depending on the authentication mechanism used by the site different methods exist. WebClient could be used to send HTTP requests to the remote sites.

---

To crawl user-authenticated websites, your crawler should be able to handle cookies and POST-requests.

The question is: do you want your crawler to sign up, validate itself by email and then sign in to crawl all contents?