Dictionary Analysis

I. Introduction

Four dictionaries (Jazzy, Ensemble, Medline, and Lexicon) are compared. Here are the summary:

II. Analysis and Tests

Analysis and performance tests are conducted from various dictionaries to obtain a better dictionary generation. Please see the following URL for details:

• Factor Analysis
• Performance Tests

From the above results, we observe:

• The verified dictionary from Lexicon is the best for spelling error checking
• The Ensemble dictionary is the best for spelling error suggestion/correction
• Multiwords are useful in split cases
• The abbreviations and acronyms are useful in split cases (to avoid invalid split)
• Extra dictionary from Lexicon (numbers and units) are useful

• The combination of Ensemble and Lexicon seems reach the best performance
• The Lexicon dictionary lacks of terms from drugs, problem, ..

III. Overlap and Contain Check