开发者

How to smooth unigrams

开发者 https://www.devze.com 2023-04-12 02:55 出处:网络
I have a unigram language model andi want to smooth the counts. Is add one smoothing the only way or can i use some other smoothing also. I dont think we can use knesser nay as that is for Ngrams with

I have a unigram language model and i want to smooth the counts. Is add one smoothing the only way or can i use some other smoothing also. I dont think we can use knesser nay as that is for Ngrams with N>=2. Any other smoothing method you kno开发者_运维技巧w?

How about witten bell?


For unigram smoothing, Good-Turing would be optimal ...and it's easy (to apply)!

http://en.wikipedia.org/wiki/Good%E2%80%93Turing_frequency_estimation

For higher orders, modified interpolated Kneser-Ney is a good choice.

0

精彩评论

暂无评论...
验证码 换一张
取 消