Precision, Recall, and F1 Analysis for LMW Candidates from SpVar Model - Paper on SpVar
I. Introduction
In the previous study (AMIA paper), multiple MES and ES models are used to retrieved SpVar from the distilled MEDLINE n-gram set and then use as filter to retrieve LMWs. This model is OK with some issues:
An improved model, M2CES, is developed to address these issues.
II. Development
Case | Description | TP | FP | T. Retrieved | T. Relevant | FN | TN | Precision | Recall | F1 | Accuracy |
---|---|---|---|---|---|---|---|---|---|---|---|
31 | Parenthetic Acronym - Gold Standard | 14,805 | 1,870 | 16,675 | 14,805 | 0 | 0 | 0.8879 | 1.0000 | 0.9406 | 0.8879 |
Filters or a single matcher | |||||||||||
33 | SpVar Matcher - MES + ES, WC: 150 | 7,509 | 482 | 7,991 | 14,805 | 7,296 | 1,388 | 0.9397 | 0.5072 | 0.6588 | 0.5336 |
33A | SpVar Matcher - M2CES, WC: 150 | 3,695 | 226 | 3,921 | 14,805 | 11,110 | 1,644 | 0.9424 | 0.2496 | 0.3946 | 0.3202 |
33B | SpVar Matcher - M2CES, WC: 100 | 4,485 | 283 | 4,768 | 14,805 | 10,320 | 1,587 | 0.9406 | 0.3029 | 0.4583 | 0.3641 |
33C | SpVar Matcher - M2CES, WC: 50 | 5,748 | 421 | 6,169 | 14,805 | 9,057 | 1,449 | 0.9318 | 0.3882 | 0.5481 | 0.4316 |
33D | SpVar Matcher - M2CES, WC: 30 | 6,682 | 513 | 7,195 | 14,805 | 8,123 | 1,357 | 0.9287 | 0.4513 | 0.6075 | 0.4821 |
Combination: SpVar + CUI + Distrilled | |||||||||||
36 | SpVar - MES + ES, WC: 150 | 5,510 | 206 | 5,716 | 14,805 | 9,295 | 1,664 | 0.9640 | 0.3722 | 0.5370 | 0.4302 |
36A | SpVar - M2CES, WC: 150 | 2,793 | 106 | 2,899 | 14,805 | 12,012 | 1,764 | 0.9634 | 0.1887 | 0.3155 | 0.2733 |
36B | SpVar - M2CES, WC: 100 | 3,319 | 118 | 3,437 | 14,805 | 11,486 | 1,752 | 0.9657 | 0.2242 | 0.3639 | 0.3041 |
36C | SpVar - M2CES, WC: 50 | 4,102 | 162 | 4,264 | 14,805 | 10,703 | 1,708 | 0.9620 | 0.2771 | 0.4302 | 0.3484 |
36D | SpVar - M2CES, WC: 30 | 4,699 | 189 | 4,888 | 14,805 | 10,106 | 1,681 | 0.9613 | 0.3174 | 0.4772 | 0.3826 |
Combination: SpVar + CUI + Distrilled | |||||||||||
37 | SpVar - MES + ES, WC: 150 | 727 | 11 | 738 | 14,805 | 14,078 | 1,859 | 0.9851 | 0.0491 | 0.0935 | 0.1551 |
37A | SpVar - M2CES, WC: 150 | 354 | 5 | 359 | 14,805 | 14,451 | 1,865 | 0.9861 | 0.0239 | 0.0467 | 0.1331 |
37B | SpVar - M2CES, WC: 100 | 427 | 5 | 432 | 14,805 | 14,378 | 1,865 | 0.9884 | 0.0288 | 0.0560 | 0.1375 |
37C | SpVar - M2CES, WC: 50 | 568 | 8 | 576 | 14,805 | 14,237 | 1,862 | 0.9861 | 0.0384 | 0.0739 | 0.1457 |
37D | SpVar - M2CES, WC: 30 | 654 | 10 | 664 | 14,805 | 14,151 | 1,860 | 0.9849 | 0.0442 | 0.0846 | 0.1508 |