UMLS-Core: Synonym Files

• Descriptions:
  • A Java class, GenDefaultSynonymsFile, is implemented in STMT package to read in synonym files (with different format) and stored into normTerm-synonym table
  • There are 8 files (from different sources) with different formats and needs to be standardized first:

    File	Descriptions	Notes
    acronym_edited.txt	acronyms and abbreviations	acronym|upper case|punctuation
    british.txt	British English	spelling varaints|lower case|punctuation
    ecri.txt	ECRI (Emergency Care Research Institute) Medical Device	mixed case|multiple sysnonym pairs
    greco-latin.txt	Greco-Latin	lower case|bi-direction
    lvg.txt	Lexical Tools	mixed case|genitive|punctuation|bi-direction
    ramiller-prune_kwf.txt	Ramiller (after human review)	mixed case|genitive|punctuation|multiple sysnonym pairs
    To_add_UMLS_syn_consolidated.txt	To be added to UMLS	mixed case|punctuation|multiple sysnonym pairs
    UMLS_syn_consolidated.txt	UMLS	lowercase|punctuation

  • All files have different formats (needs to be standardized)
  • Synonym variations:

    Variations	Standardization	Lvg flow	Notes and Examples
    genitive	remove genitive	-f:g	down's|mongoloid bristowe's|split brain cowper's|bulbourethral
    parenthetic plural from	remove (s), (es), (ies)	-f:rs	This is needed to standardize the term (not the synonym)
    punctuation	replace with space	-f:o	trim multiple spaces % is removed
    Upper, lower, mixed cases	lower case	-f:l
    spelling variants	use citation form instead	-f:Ct
    inflectional variants	use citation form instead	-f:Ct
    stopWord	do nothing
    suffixes	manually remove them		Knife|-tome
    recursive synonyms	not implemented		abut|adjoin abut|to touch =>adjoin|to touch anaemia|anemia chlorosis|chlorotic anemia =>chlorosis|chlorotic anaemia
    bi-direction	all synonym pairs are bi-directional		TB|TUBERCULOSIS =>TUBERCULOSIS|TB
    multiple synonym pairs	convert to all possible double pairs		AF|ATRIAL FIBRILLATION|A FIB =>AF|ATRIAL FIBRILLATION =>AF|A FIB =>ATRIAL FIBRILLATION|A FIB =>ATRIAL FIBRILLATION|AF =>A FIB|AF =>A FIB|ATRIAL FIBRILLATION
    redundancy	remove if the normalized synonyms are the same. This remove redundant synonyms which is taken care in Norm, such as spelling variants, inflectional variants, etc.		fecal|fecal

• Examples - Test Cases:

  Skip

• Algorithm:
  • manually reviewed all 8 files and remove suffix synonyms
  • Read in all 8 files, one by one
  • decompose multiple synonym pairs:
    • Input:
      • AF|ATRIAL FIBRILLATION|A FIB
    • Outputs:
      • AF|ATRIAL FIBRILLATION
      • AF|A FIB
      • ATRIAL FIBRILLATION|A FIB
  • Ignore comments (line starts with #)
  • normalize key:
    • use SynonymNorm: -f:g:rs:Ct:o:l (also trim spaces)
  • Generate symmetric synonym pair
    • Input:
      • AB|ANTIBODY
    • Outputs:
      • ab|ANTIBODY
      • antibody|AB
  • remove duplications:
    • Remove duplicated synonyms from different files
    • Remove if the normalized synonyms in synonym pair are the same, which means no new CUI will be found because they have same key (normTerm) in CuiMapping.