Anchor tags in urls & search engines, HOWTO?

How do I implement anchor tag urls so that search engines crawl my pages? Here's an example from twitter:

In search results it's: http://twitter.com/username

When I click on it, it re开发者_JAVA技巧directs me to http://twitter.com/#!/username

How does twitter know when to redirect? Relying on a User-Agent doesn't seem such a good idea.

---

Twitter isn't optimizing their site for SEO. They have a special deal with Google so I wouldn't use them as an example. Google has support for hash URLs, which you can read about here https://developers.google.com/webmasters/ajax-crawling/docs/specification.

The main idea is that a URL like http://www.example.org/#!/my-url the crawlers convert to http://www.example.org/?_escaped_fragment_=/my-url. When Google encounters that URLs it makes a get requests to the alternative URL and will use that content to index it.