Performance Tests on Training Set

I. Test Setup

• Data: Training Set
• The corrected data of ESpell and Jazzy from Dr. Kilicoglu are used directly for this test result.
• The Ensemble program from Dr. Kilicoglu was enhanced from Ensemble paper. Thus, the result is slightly better.

II. Test Results

• Non-word Only:

  Non-word, Detection
  Method	TP	FP	FN	T. Ret	T. Rel	Precision	Recall	F1
  ESpell	395	785	379	1180	774	0.3347	0.5103	0.4043
  Jazzy	324	69	450	393	774	0.8244	0.4186	0.5553
  Ensemble	655	170	119	825	774	0.7939	0.8463	0.8193
  CSpell	667	55	107	722	774	0.9238	0.8618	0.8917

  Non-word, Correction
  Method	TP	FP	FN	T. Ret	T. Rel	Precision	Recall	F1
  ESpell	237	943	537	1180	774	0.2008	0.3062	0.2426
  Jazzy	187	206	587	393	774	0.4758	0.2416	0.3205
  Ensemble	552	273	222	825	774	0.6691	0.7132	0.6904
  CSpell	607	115	167	722	774	0.8407	0.7842	0.8115

• Real-word Included:

  Real-word Included, Detection
  Method	TP	FP	FN	T. Ret	T. Rel	Precision	Recall	F1
  ESpell	410	770	554	1180	964	0.3475	0.4253	0.3825
  Jazzy	334	59	630	393	964	0.8499	0.3465	0.4923
  Ensemble	580	138	384	718	964	0.8078	0.6017	0.6897
  CSpell	692	53	272	745	964	0.9289	0.7178	0.8098

  Real-word Included, Correction
  Method	TP	FP	FN	T. Ret	T. Rel	Precision	Recall	F1
  ESpell	245	935	719	1180	964	0.2076	0.2541	0.2285
  Jazzy	191	202	773	393	964	0.4860	0.1981	0.2815
  Ensemble	517	201	447	718	964	0.7201	0.5363	0.6147
  CSpell	627	118	337	745	964	0.8416	0.6504	0.7338

• Speed:
  • Elapse: 56.91 sec

III. Discussion

• The Ensemble outperformed ESpell and Jazzy (ASpell) by a large margin (over 30%) because Ensemble was developed to correct errors in consumer health questions.
• The improvement from Ensemble to CSpell for non-word detection and correction is 7.24% and 12.11%.
• The improvement from Ensemble to CSpell for real-word detection and correction is 12.01% and 11.91%.