在 NER 的评价里有很多选择，以下介绍两个常用的。这里只介绍用法，基础的计算公式之类的大家应该都会了。
一、sklearn.metrics官方文档：API Reference — scikit-learn 1.1.1 documentation
这里以 NER 评估最常用的 f1-score 为例。
from sklearn.metrics import f1_score score = f1_score(y_true, y_pred, labels=None, pos_label=1, average='binary', sample_weight=None, zero_division='warn')参数简介
这里常用的参数是 y_true,y_pred,average。
举例y_true：1d数组，或标签指示符数组/稀疏矩阵 基本事实（正确）目标值。
y_pred：1d数组，或标签指示符数组/稀疏矩阵 分类器返回的估计目标。
average：{‘micro’, ‘macro’, ‘samples’, ‘weighted’, ‘binary’} or None, default=’binary’。对于多分类/多标签的目标这个参数是必需的。如果是 None，则返回每个分类的分数。否则，这将确定对数据执行的平均类型：
'binary':
Only report results for the class specified by pos_label. This is applicable only if targets (y_{true,pred}) are binary.
'micro':
Calculate metrics globally by counting the total true positives, false negatives and false positives.
'macro':
Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
'weighted':
Calculate metrics for each label, and find their average weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance; it can result in an F-score that is not between precision and recall.
'samples':
Calculate metrics for each instance, and find their average (only meaningful for multilabel classification where this differs from accuracy_score).
from sklearn.metrics import f1_score
y_true = [0, 0, 2, 1, 1, 1]
y_pred = [0, 0, 1, 1, 0, 1]
score = f1_score(y_true, y_pred, average='micro')
print(f'f1_score:{score:.3f}')
>>>f1_score:0.667
二、seqeval.metrics
这个模块我个人更喜欢一点。
官方文档：seqeval · PyPI；chakki-works/seqeval: A Python framework for sequence labeling evaluation(named-entity recognition, pos tagging, etc…)
模块说明| metrics | description |
|---|---|
| accuracy_score(y_true, y_pred) | Compute the accuracy. |
| precision_score(y_true, y_pred) | Compute the precision. |
| recall_score(y_true, y_pred) | Compute the recall. |
| f1_score(y_true, y_pred) | Compute the F1 score, also known as balanced F-score or F-measure. |
| classification_report(y_true, y_pred, digits=2) | Build a text report showing the main classification metrics. digits is number of digits for formatting output floating point values. Default value is 2. |
参数和 sklearn.metrics 的看着差不多，其实有一点小区别，sklearn.metrics 的 y_true 和 y_pred 为 id，也就是 int 类型，而 seqeval.metrics 的参数为 str 类型。使用的时候需要用自建的 tag2id 的字典转化一下即可。
举例from seqeval.metrics import classification_report
y_true = [['O', 'O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
y_pred = [['O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
score = classification_report(y_true, y_pred)
print(score)
>>> precision recall f1-score support
MISC 0.00 0.00 0.00 1
PER 1.00 1.00 1.00 1
micro avg 0.50 0.50 0.50 2
macro avg 0.50 0.50 0.50 2
weighted avg 0.50 0.50 0.50 2
