GENIA Sentence Splitter

GENIA Sentence Splitter (GeniaSS) [1] is a sentence splitter optimized for biomedical texts. GeniaSS reads a text and splits it into sentences by inserting line breaks.

Features

• The classification model is based on supervised leaning method using maximum entropy modeling (using simple MaxEnt library[2]).
• Trained on the GENIA corpus [3]. The classifier achieved an F-score of 99.7 on 200 unseen GENIA abstracts.

Classification model

First, GeniaSS detects candidate positions for splitting using selected delimiters: periods, commas, single/double quotation marks, right parentheses, etc. Then, it classifies whether each candidate really splits the sentence or not.

Classifier Features

• Delimiters of the candidate point
• Previous/Next words
• Previous/Next words deleted brackets, commas, etc. (word2)
• Capitalization of first/other character(s) of the next word
• Existence of capitals in the previous word
• Existence of numbers in prev/next words
• Existence of brackets/quotations in prev/next words
• Existence of comma in prev/next words
• Combination feature of previous word & delimiter
• Combination feature of previous word2, delimiter & next word2
  

Download

• version 1.00 geniass-1.00.tar.gz (Jan, 2007)

How to use

1) make
2) ./geniass arg1 arg2

arg1 is a target file to split. arg2 is an output file name. If you want to get stand-off format file, please run

3) ruby sentence2standOff.rb arg1 arg2 arg3

arg1 and arg2 are same as 2). arg3 is an output stand-off file name.

Note: you need to run GeniaSS in the directory which includes EventExtracter.rb, Classifying2Splitting.rb, model1-1.0.

Reference

[1] Sætre, Rune, Kazuhiro Yoshida, Akane Yakushiji, Yusuke Miyao, Yuichiro Matsubayashi and Tomoko Ohta., AKANE System: Protein-Protein Interaction Pairs in BioCreAtIvE2 Challenge, PPI-IPS subtask. In Proceedings of the Second BioCreative Challenge Evaluation Workshop. pp. 209--212, April 2007. CNIO. 

[2] Yoshimasa Tsuruoka., A simple C++ library for maximum entropy classification, http://www-tsujii.is.s.u-tokyo.ac.jp/~tsuruoka/maxent/ , 2005.

[3] Kim J.D., Ohta T., Tateishi Y., and Tsujii J., GENIA corpus - a semantically annotated corpus for bio-textmining, Bioinformatics, 19(suppl. 1):180–i182, 2003.

Created and maintained by Yuichiroh Matsubayashi