Converting untagged corpora to tagged (NLTK)

开发者_StackOverflow中文版

I have a plaintext corpora, that I want to tag and save, so I can use it further. What's the best way to do this?

I already have my tagger made, but I can't figure out a way to change the corpora that isn't messy

---

Take a look at other tagged corpora, like brown, for output examples. This will give you an idea of what a tagged corpus should look like. Next, load your corpus (with the PlaintextCorpusReader) and iterate over the sentences, tagging each sentence. Then write each tagged sentence to a file by making a string from the tagged sentence, as in ' '.join([tuple2str(t) for t in tagged_sent]) (after you do from nltk.tag.util import tuple2str). And it's ok if your code is "messy" as long it does the job correctly. You're not looking for an elegant algorithm here, you're running a very specific script to create a custom corpus.

---

Are you doing simple unigram tagging, or are you actually parsing the text? I believe NLTK parses/tags such that the output of every token is (token, PoS). Is an array of tuples unacceptable for storing your corpora? Why do you find this messy?