开发者

Scikits-learn: Use custom vocabulary together with Pipeline

开发者 https://www.devze.com 2023-03-17 17:40 出处:网络
In my scikits-learn Pipeline, I would like to pass a custom vocabulary to CountVectorizer(): text_classifier = Pipeline([

In my scikits-learn Pipeline, I would like to pass a custom vocabulary to CountVectorizer():

text_classifier = Pipeline([
    ('count', CountVectorizer(vocabulary=myvocab)),
    ('tfidf', TfidfTransformer()),
    ('clf', LinearSVC(C=1000))
])

However, as far as I 开发者_如何学Gounderstand when I call

text_classifier.fit(X_train, y_train)

Pipeline uses the fit_transform() method of CountVectorizer(), which ignores myvocab. How could I modify my Pipeline to use myvocab? Thanks!


This was a bug in scikit-learn that I fixed five minutes ago. Thanks for spotting it. I suggest you either upgrade to the newest version from Github, or separate the vectorizer from the pipeline as a workaround:

count = CountVectorizer(vocabulary=myvocab)
X_vectorized = count.transform(X_train)

text_classifier = Pipeline([
    ('tfidf', TfidfTransformer()),
    ('clf', LinearSVC(C=1000))
])

text_classifier.fit(X_vectorized, y_train)

UPDATE: since this answer was posted, this fix has been incorporated in several scikit-learn releases.

0

精彩评论

暂无评论...
验证码 换一张
取 消