Releases applied this approach to retrieve the optimized SD-rule set are copared as follows since 2014:
Release | New SD-Rules | Baseline | Results | Notes
|
---|
2014 | First Release (based on 2013 SD-Rule)
|
- Total candidates SD-pairs: 43,375
- Total valid candidates SD-pairs (SD-Facts: relevant): 37,136
|
- N/A (All SD-Rules are first timer)
|
|
---|
2015 | Added 15 new SD-Rules to the previous release
|
- Total candidates SD-pairs: 53,905
- Total valid candidates SD-pairs (SD-Facts: relevant): 46,950
|
- 2 are duplicated (child rule of existing rules).
- 11 (84.62%, 11/13) of them are evaluated as good rules in the optimized set
- 2 (15.38%, 2/13) are bad rules
|
|
---|
2016 | Added 12 new SD-Rules to the previous release
|
- Total candidates SD-pairs: 58,422
- Total valid candidates SD-pairs: 50,814
| |
---|
Year | Stats | Optimized Diagram
|
---|
2014
|
- Baseline Set (include parent-child rules): 107
- Total Unique Rules: 96
- Total Good Rules: 73
- Total Valid SD-pairs (SD-Facts: Relevant): 42,552
- Opti. System Precision: 95.30%
- Opti. System Recall: 95.01%
- Opti. System Performance: 1.9031
- Cutoff Rule:
ar$|adj|e$|noun
- Optimized Set: 2014 Optimized Set
|
|
---|
2015
|
- Baseline Set (include parent-child rules):120
- Total Unique Rules: 101
- Total Good Rules: 76
- Total Valid SD-pairs (SD-Facts: Relevant): 46,950
- Opti. System Precision: 95.22%
- Opti. System Recall: 95.70%
- Opti. System Performance: 1.9093
- Cutoff Rule:
ar$|adj|e$|noun
- Optimized Set: 2015 Optimized Set
|
|
---|
2016
|
- Baseline Set (include parent-child rules):132
- Total Unique Rules: 111
- Total Good Rules: 82
- Total Valid SD-pairs (SD-Facts: Relevant): 50,814
- Opti. System Precision: 95.00%
- Opti. System Recall: 95.26%
- Opti. System Performance: 1.9026
- Cutoff Rule:
$|noun|ist$|noun
- Optimized Set: 2016 Optimized Set
|
|
---|
2017
|
- Baseline Set (include parent-child rules):142
- Total Unique Rules: 119
- Total Good Rules: 86
- Total Valid SD-pairs (SD-Facts: Relevant): 51,788
- Opti. System Precision: 95.09%
- Opti. System Recall: 94.92%
- Opti. System Performance: 1.9001
- Cutoff Rule:
$|noun|ist$|noun
- Optimized Set: 2017 Optimized Set
|
|
---|
2020
|
- Baseline Set (include parent-child rules):153
- Total Unique Rules: 130
- Total Good Rules: 93
- Total Valid SD-pairs (SD-Facts: Relevant): 53,440
- Opti. System Precision: 95.00%
- Opti. System Recall: 94.48%
- Opti. System Performance: 1.8948
- Cutoff Rule:
ar$|adj|e$|noun
- Optimized Set: 2020 Optimized Set
|
|
---|
The conclusion is the optimized set of SD-Rules is very steady (consistent) as we expected.