Evaluation of 9 BEL models
- Paper:
- PDF: [[|论文.pdf]]
- Venue / year: 2023
- Topic:
1. To Understand List
- 两阶段的第二阶段, 如果用 Reranker， 是不是一定需要 context?
- SapBERT 能用于中文词表吗，中文有类似的"SapBERT"吗
2. Alias Matching EL
- MetaMap
- SciSpacy
- BioSyn
- SapBERT SapBERT 和 BioSyn 的训练思路和方法类似，所以主要关注 SapBERT就行
模型对比
5.1 Error Analysis
For errors in the dataset, we analyze the following:
Stage of EL failure: For incorrectly linked mentions, did the failure occur in CG or NED phase? For failures that occur in candidate generation phase, what proportion of generated candidates have the correct semantic type/semantic group?
Failure subgroups: When a model fails, can we identify slices with high/low chances of failure? Inspired by Orr et al. (2021) and Chen et al. (2021), we investigate possible failure modes including:
- Entity type. Are entities of particular types frequently linked incorrectly? Are generated candidates in the correct semantic type/group?
- Popularity. How often are incorrectly linked entities present in the training data?
- Available metadata. Do incorrectly linked surface forms match aliases in the KB? Are KB entities with few aliases and/or no definition more likely to be incorrectly linked?
Common Misunderstandings: There are some cases where all models in our comparison find the incorrect entity link in our data. We manually examined cases where all BioEL models provided an incorrect entity link and describe common mistakes made by current BioEL models.