BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of CNTS-LTG, University of Antwerp
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 35) 46 ( 35) 20.11 76.09 31.82
Binding 347 ( 42) 145 ( 42) 12.10 28.97 17.07
Gene_expression 722 ( 364) 539 ( 364) 50.42 67.53 57.73
Transcription 137 ( 32) 82 ( 32) 23.36 39.02 29.22
Protein_catabolism 14 ( 9) 21 ( 9) 64.29 42.86 51.43
Phosphorylation 135 ( 37) 56 ( 37) 27.41 66.07 38.74
==[EVT-TOTAL]== 1529 ( 519) 889 ( 519) 33.94 58.38 42.93
------------------------------------------------------------------------------------
Regulation 291 ( 23) 110 ( 23) 7.90 20.91 11.47
Positive_regulation 983 ( 104) 363 ( 104) 10.58 28.65 15.45
Negative_regulation 379 ( 40) 140 ( 40) 10.55 28.57 15.41
==[REG-TOTAL]== 1653 ( 167) 613 ( 167) 10.10 27.24 14.74
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 686) 1502 ( 686) 21.56 45.67 29.29
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 36) 46 ( 36) 20.69 78.26 32.73
Binding 347 ( 45) 145 ( 45) 12.97 31.03 18.29
Gene_expression 722 ( 371) 538 ( 371) 51.39 68.96 58.89
Transcription 137 ( 34) 82 ( 34) 24.82 41.46 31.05
Protein_catabolism 14 ( 9) 21 ( 9) 64.29 42.86 51.43
Phosphorylation 135 ( 38) 56 ( 38) 28.15 67.86 39.79
==[EVT-TOTAL]== 1529 ( 533) 888 ( 533) 34.86 60.02 44.10
------------------------------------------------------------------------------------
Regulation 291 ( 26) 110 ( 26) 8.93 23.64 12.97
Positive_regulation 983 ( 114) 363 ( 114) 11.60 31.40 16.94
Negative_regulation 379 ( 41) 140 ( 41) 10.82 29.29 15.80
==[REG-TOTAL]== 1653 ( 181) 613 ( 181) 10.95 29.53 15.98
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 714) 1501 ( 714) 22.44 47.57 30.49
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 36) 46 ( 36) 20.69 78.26 32.73
Binding 347 ( 45) 145 ( 45) 12.97 31.03 18.29
Gene_expression 722 ( 371) 538 ( 371) 51.39 68.96 58.89
Transcription 137 ( 34) 82 ( 34) 24.82 41.46 31.05
Protein_catabolism 14 ( 9) 21 ( 9) 64.29 42.86 51.43
Phosphorylation 135 ( 38) 56 ( 38) 28.15 67.86 39.79
==[EVT-TOTAL]== 1529 ( 533) 888 ( 533) 34.86 60.02 44.10
------------------------------------------------------------------------------------
Regulation 291 ( 26) 110 ( 26) 8.93 23.64 12.97
Positive_regulation 983 ( 115) 363 ( 115) 11.70 31.68 17.09
Negative_regulation 379 ( 42) 140 ( 42) 11.08 30.00 16.18
==[REG-TOTAL]== 1653 ( 183) 613 ( 183) 11.07 29.85 16.15
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 716) 1501 ( 716) 22.50 47.70 30.58
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 35) 46 ( 35) 20.11 76.09 31.82
Binding 398 ( 83) 163 ( 83) 20.85 50.92 29.59
Gene_expression 722 ( 364) 539 ( 364) 50.42 67.53 57.73
Transcription 137 ( 32) 82 ( 32) 23.36 39.02 29.22
Protein_catabolism 14 ( 9) 21 ( 9) 64.29 42.86 51.43
Phosphorylation 135 ( 37) 56 ( 37) 27.41 66.07 38.74
==[EVT-TOTAL]== 1580 ( 560) 907 ( 560) 35.44 61.74 45.03
------------------------------------------------------------------------------------
Regulation 338 ( 32) 122 ( 32) 9.47 26.23 13.91
Positive_regulation 1186 ( 144) 404 ( 144) 12.14 35.64 18.11
Negative_regulation 416 ( 50) 152 ( 50) 12.02 32.89 17.61
==[REG-TOTAL]== 1940 ( 226) 678 ( 226) 11.65 33.33 17.27
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 786) 1585 ( 786) 22.33 49.59 30.79
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 36) 46 ( 36) 20.69 78.26 32.73
Binding 398 ( 88) 163 ( 88) 22.11 53.99 31.37
Gene_expression 722 ( 371) 538 ( 371) 51.39 68.96 58.89
Transcription 137 ( 34) 82 ( 34) 24.82 41.46 31.05
Protein_catabolism 14 ( 9) 21 ( 9) 64.29 42.86 51.43
Phosphorylation 135 ( 38) 56 ( 38) 28.15 67.86 39.79
==[EVT-TOTAL]== 1580 ( 576) 906 ( 576) 36.46 63.58 46.34
------------------------------------------------------------------------------------
Regulation 338 ( 35) 122 ( 35) 10.36 28.69 15.22
Positive_regulation 1186 ( 159) 404 ( 159) 13.41 39.36 20.00
Negative_regulation 416 ( 51) 152 ( 51) 12.26 33.55 17.96
==[REG-TOTAL]== 1940 ( 245) 678 ( 245) 12.63 36.14 18.72
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 821) 1584 ( 821) 23.32 51.83 32.17
------------------------------------------------------------------------------------
Task 2
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 30) 46 ( 30) 17.24 65.22 27.27
Binding 349 ( 38) 145 ( 38) 10.89 26.21 15.38
Gene_expression 722 ( 364) 539 ( 364) 50.42 67.53 57.73
Transcription 137 ( 32) 82 ( 32) 23.36 39.02 29.22
Protein_catabolism 14 ( 9) 21 ( 9) 64.29 42.86 51.43
Phosphorylation 139 ( 21) 56 ( 21) 15.11 37.50 21.54
==[EVT-TOTAL]== 1535 ( 494) 889 ( 494) 32.18 55.57 40.76
------------------------------------------------------------------------------------
Regulation 292 ( 22) 110 ( 22) 7.53 20.00 10.95
Positive_regulation 987 ( 96) 363 ( 96) 9.73 26.45 14.22
Negative_regulation 379 ( 40) 140 ( 40) 10.55 28.57 15.41
==[REG-TOTAL]== 1658 ( 158) 613 ( 158) 9.53 25.77 13.91
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 652) 1502 ( 652) 20.42 43.41 27.77
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 31) 46 ( 31) 17.82 67.39 28.18
Binding 349 ( 41) 145 ( 41) 11.75 28.28 16.60
Gene_expression 722 ( 371) 538 ( 371) 51.39 68.96 58.89
Transcription 137 ( 34) 82 ( 34) 24.82 41.46 31.05
Protein_catabolism 14 ( 9) 21 ( 9) 64.29 42.86 51.43
Phosphorylation 139 ( 22) 56 ( 22) 15.83 39.29 22.56
==[EVT-TOTAL]== 1535 ( 508) 888 ( 508) 33.09 57.21 41.93
------------------------------------------------------------------------------------
Regulation 292 ( 25) 110 ( 25) 8.56 22.73 12.44
Positive_regulation 987 ( 106) 363 ( 106) 10.74 29.20 15.70
Negative_regulation 379 ( 41) 140 ( 41) 10.82 29.29 15.80
==[REG-TOTAL]== 1658 ( 172) 613 ( 172) 10.37 28.06 15.15
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 680) 1501 ( 680) 21.30 45.30 28.97
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 31) 46 ( 31) 17.82 67.39 28.18
Binding 349 ( 41) 145 ( 41) 11.75 28.28 16.60
Gene_expression 722 ( 371) 538 ( 371) 51.39 68.96 58.89
Transcription 137 ( 34) 82 ( 34) 24.82 41.46 31.05
Protein_catabolism 14 ( 9) 21 ( 9) 64.29 42.86 51.43
Phosphorylation 139 ( 22) 56 ( 22) 15.83 39.29 22.56
==[EVT-TOTAL]== 1535 ( 508) 888 ( 508) 33.09 57.21 41.93
------------------------------------------------------------------------------------
Regulation 292 ( 25) 110 ( 25) 8.56 22.73 12.44
Positive_regulation 987 ( 112) 363 ( 112) 11.35 30.85 16.59
Negative_regulation 379 ( 42) 140 ( 42) 11.08 30.00 16.18
==[REG-TOTAL]== 1658 ( 179) 613 ( 179) 10.80 29.20 15.76
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 687) 1501 ( 687) 21.52 45.77 29.27
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 239 ( 36) 47 ( 36) 15.06 76.60 25.17
Binding 419 ( 84) 167 ( 84) 20.05 50.30 28.67
Gene_expression 722 ( 364) 539 ( 364) 50.42 67.53 57.73
Transcription 137 ( 32) 82 ( 32) 23.36 39.02 29.22
Protein_catabolism 14 ( 9) 21 ( 9) 64.29 42.86 51.43
Phosphorylation 191 ( 37) 56 ( 37) 19.37 66.07 29.96
==[EVT-TOTAL]== 1722 ( 562) 912 ( 562) 32.64 61.62 42.67
------------------------------------------------------------------------------------
Regulation 349 ( 32) 124 ( 32) 9.17 25.81 13.53
Positive_regulation 1220 ( 150) 418 ( 150) 12.30 35.89 18.32
Negative_regulation 425 ( 52) 156 ( 52) 12.24 33.33 17.90
==[REG-TOTAL]== 1994 ( 234) 698 ( 234) 11.74 33.52 17.38
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3716 ( 796) 1610 ( 796) 21.42 49.44 29.89
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 239 ( 37) 47 ( 37) 15.48 78.72 25.87
Binding 419 ( 89) 167 ( 89) 21.24 53.29 30.38
Gene_expression 722 ( 371) 538 ( 371) 51.39 68.96 58.89
Transcription 137 ( 34) 82 ( 34) 24.82 41.46 31.05
Protein_catabolism 14 ( 9) 21 ( 9) 64.29 42.86 51.43
Phosphorylation 191 ( 38) 56 ( 38) 19.90 67.86 30.77
==[EVT-TOTAL]== 1722 ( 578) 911 ( 578) 33.57 63.45 43.90
------------------------------------------------------------------------------------
Regulation 349 ( 36) 124 ( 36) 10.32 29.03 15.22
Positive_regulation 1220 ( 167) 418 ( 167) 13.69 39.95 20.39
Negative_regulation 425 ( 53) 156 ( 53) 12.47 33.97 18.24
==[REG-TOTAL]== 1994 ( 256) 698 ( 256) 12.84 36.68 19.02
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3716 ( 834) 1609 ( 834) 22.44 51.83 31.32
------------------------------------------------------------------------------------
Task 3
N/A
N/A
N/A
N/A
N/A