输入值
$ cat test.csv ID,Task,label,Text1,Collect Information,no response,cozily married practical athletics Mr. Brown flat2,New Credit,no response,active married expensive soccer Mr. Chang flat3,Collect Information,response,healthy single expensive badminton Mrs. Green flat4,Collect Information,response,cozily married practical soccer Mr. Brown hierachical5,Collect Information,response,cozily single practical badminton Mr. Brown flat
TL; DR
>>> from nltk import word_tokenize, pos_tag, pos_tag_sents>>> import pandas as pd>>> df = pd.read_csv('test.csv', sep=',')>>> df['Text']0 cozily married practical athletics Mr. Brown flat1 active married expensive soccer Mr. Chang flat2 healthy single expensive badminton Mrs. Green ...3 cozily married practical soccer Mr. Brown hier...4 cozily single practical badminton Mr. Brown flatName: Text, dtype: object>>> texts = df['Text'].tolist()>>> tagged_texts = pos_tag_sents(map(word_tokenize, texts))>>> tagged_texts[[('cozily', 'RB'), ('married', 'JJ'), ('practical', 'JJ'), ('athletics', 'NNS'), ('Mr.', 'NNP'), ('Brown', 'NNP'), ('flat', 'JJ')], [('active', 'JJ'), ('married', 'VBD'), ('expensive', 'JJ'), ('soccer', 'NN'), ('Mr.', 'NNP'), ('Chang', 'NNP'), ('flat', 'JJ')], [('healthy', 'JJ'), ('single', 'JJ'), ('expensive', 'JJ'), ('badminton', 'NN'), ('Mrs.', 'NNP'), ('Green', 'NNP'), ('flat', 'JJ')], [('cozily', 'RB'), ('married', 'JJ'), ('practical', 'JJ'), ('soccer', 'NN'), ('Mr.', 'NNP'), ('Brown', 'NNP'), ('hierachical', 'JJ')], [('cozily', 'RB'), ('single', 'JJ'), ('practical', 'JJ'), ('badminton', 'NN'), ('Mr.', 'NNP'), ('Brown', 'NNP'), ('flat', 'JJ')]]>>> df['POS'] = tagged_texts>>> df ID Task label 1 Collect Information no response 1 2New Credit no response 2 3 Collect Information response 3 4 Collect Information response 4 5 Collect Information response Text cozily married practical athletics Mr. Brown flat 1 active married expensive soccer Mr. Chang flat 2 healthy single expensive badminton Mrs. Green ... 3 cozily married practical soccer Mr. Brown hier... 4 cozily single practical badminton Mr. Brown flat POS 0 [(cozily, RB), (married, JJ), (practical, JJ),... 1 [(active, JJ), (married, VBD), (expensive, JJ)... 2 [(healthy, JJ), (single, JJ), (expensive, JJ),... 3 [(cozily, RB), (married, JJ), (practical, JJ),... 4 [(cozily, RB), (single, JJ), (practical, JJ), ...在长：
首先，您可以将
Text列提取到字符串列表中：
texts = df['Text'].tolist()
然后可以应用该
word_tokenize功能：
map(word_tokenize, texts)
注意，@Boud的建议几乎是相同的，使用
df.apply：
df['Text'].apply(word_tokenize)
然后将标记化的文本转储到字符串列表中：
df['Text'].apply(word_tokenize).tolist()
然后，您可以使用
pos_tag_sents：
pos_tag_sents( df['Text'].apply(word_tokenize).tolist() )
然后将列添加回Dataframe中：
df['POS'] = pos_tag_sents( df['Text'].apply(word_tokenize).tolist() )
