SapBERT - Outline

Outline

1. Self-Alignment Pretraining for Biomedical Entity Representations
2. To learn:
3. 1. To understand list
4. 2. Abstract
5. 3. Related works:
6. 4. Introduction:
7. To understand of introduction:
8. 5. Methods:
9. What's the meaning of figure2, what does the author want to imply?
10. To understand of methods:
11. Question of formula(2):
12. 6. Experiments:
13. To understand of experiments:
14. I skip the "The ADAPTER Variant" part
15. Actual compute steps of online hard samples mining
16. 第一步：计算全成对相似度矩阵 $S$
17. 第二步：构建标签掩码 (Label Masks)
18. 第三步：利用 $S$ 进行在线硬采样
19. 第四步：计算 MS Loss (公式 2 的具体实现)