parsing robots.txt file using c++

is there is any library to parse robots.txt, and if开发者_开发百科 it does not exist, how can i write it in c++ with boost regex?

---

Check out the examples in the Boost Regex library. If you edit your question to give a better idea of what exactly you are looking for in your robots.txt file, someone can help you with the Regex syntax.

For example, if you are trying to find the names of all User-agents in the file, you could use an expression like this.

boost::regex expression("^User-agent:\s*(.*)");