Automated spider test

I'm looking to add a very simple layer of automated integration testing to our current Continuous Integration setup. (CI currently only checks for build breaks).

Is there a product that will:

• From a base URL, s开发者_如何学JAVApider a site & report back any 404/500 error codes?
• Allow me to add a step to logon, to be able to spider the authorized pages?

Bonuses / would-be-nice:

• Report JS errors
• Report 404s linked from CSS

I've had a quick look at SilkTest & Selenium, and they don't seem to feature quite such a site-agnostic approach. (The logon step is obviously something they can do...)

We're simply wanting to cull out the simplest/dumbest of regression errors, and we have an absolute minimum of time to implement such an automated check - hence the spidering. Ideally the solution can be run on the command line, and output its results in something I can parse into TeamCity (continuous integration package).

Much appreciated.

---

Here is a list of utilities to look at.

---

SilkTest should be able to handle your use case, you'll need to write a script that navigates through your page, depending on the complexity of your page, a simple recursive descent might be sufficient. If it gets more complex, you might need some sort of already visited URLs to avoid infinite loops.

As for the results, if you use either Silk4J or Silk4Net which both use xUnit runners to drive the tests, I assume you should be able to get the results into TeamCity.