BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of Team 24 (anonymized)
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 47) 80 ( 47) 27.01 58.75 37.01
Binding 347 ( 67) 223 ( 67) 19.31 30.04 23.51
Gene_expression 722 ( 332) 486 ( 332) 45.98 68.31 54.97
Transcription 137 ( 36) 89 ( 36) 26.28 40.45 31.86
Protein_catabolism 14 ( 3) 4 ( 3) 21.43 75.00 33.33
Phosphorylation 135 ( 59) 95 ( 59) 43.70 62.11 51.30
==[EVT-TOTAL]== 1529 ( 544) 977 ( 544) 35.58 55.68 43.42
------------------------------------------------------------------------------------
Regulation 291 ( 21) 92 ( 21) 7.22 22.83 10.97
Positive_regulation 983 ( 103) 576 ( 103) 10.48 17.88 13.21
Negative_regulation 379 ( 19) 133 ( 19) 5.01 14.29 7.42
==[REG-TOTAL]== 1653 ( 143) 801 ( 143) 8.65 17.85 11.65
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 687) 1778 ( 687) 21.59 38.64 27.70
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 48) 80 ( 48) 27.59 60.00 37.80
Binding 347 ( 79) 223 ( 79) 22.77 35.43 27.72
Gene_expression 722 ( 339) 486 ( 339) 46.95 69.75 56.13
Transcription 137 ( 38) 89 ( 38) 27.74 42.70 33.63
Protein_catabolism 14 ( 3) 4 ( 3) 21.43 75.00 33.33
Phosphorylation 135 ( 60) 95 ( 60) 44.44 63.16 52.17
==[EVT-TOTAL]== 1529 ( 567) 977 ( 567) 37.08 58.03 45.25
------------------------------------------------------------------------------------
Regulation 291 ( 22) 92 ( 22) 7.56 23.91 11.49
Positive_regulation 983 ( 112) 576 ( 111) 11.39 19.27 14.32
Negative_regulation 379 ( 20) 133 ( 20) 5.28 15.04 7.81
==[REG-TOTAL]== 1653 ( 154) 801 ( 153) 9.32 19.10 12.52
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 721) 1778 ( 720) 22.66 40.49 29.06
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 48) 80 ( 48) 27.59 60.00 37.80
Binding 347 ( 79) 223 ( 79) 22.77 35.43 27.72
Gene_expression 722 ( 339) 486 ( 339) 46.95 69.75 56.13
Transcription 137 ( 38) 89 ( 38) 27.74 42.70 33.63
Protein_catabolism 14 ( 3) 4 ( 3) 21.43 75.00 33.33
Phosphorylation 135 ( 60) 95 ( 60) 44.44 63.16 52.17
==[EVT-TOTAL]== 1529 ( 567) 977 ( 567) 37.08 58.03 45.25
------------------------------------------------------------------------------------
Regulation 291 ( 22) 92 ( 22) 7.56 23.91 11.49
Positive_regulation 983 ( 113) 576 ( 112) 11.50 19.44 14.45
Negative_regulation 379 ( 20) 133 ( 20) 5.28 15.04 7.81
==[REG-TOTAL]== 1653 ( 155) 801 ( 154) 9.38 19.23 12.61
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 722) 1778 ( 721) 22.69 40.55 29.10
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 47) 80 ( 47) 27.01 58.75 37.01
Binding 398 ( 103) 194 ( 103) 25.88 53.09 34.80
Gene_expression 722 ( 332) 486 ( 332) 45.98 68.31 54.97
Transcription 137 ( 36) 89 ( 36) 26.28 40.45 31.86
Protein_catabolism 14 ( 3) 4 ( 3) 21.43 75.00 33.33
Phosphorylation 135 ( 59) 95 ( 59) 43.70 62.11 51.30
==[EVT-TOTAL]== 1580 ( 580) 948 ( 580) 36.71 61.18 45.89
------------------------------------------------------------------------------------
Regulation 338 ( 23) 92 ( 23) 6.80 25.00 10.70
Positive_regulation 1186 ( 149) 576 ( 149) 12.56 25.87 16.91
Negative_regulation 416 ( 20) 133 ( 20) 4.81 15.04 7.29
==[REG-TOTAL]== 1940 ( 192) 801 ( 192) 9.90 23.97 14.01
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 772) 1749 ( 772) 21.93 44.14 29.30
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 48) 80 ( 48) 27.59 60.00 37.80
Binding 398 ( 116) 194 ( 116) 29.15 59.79 39.19
Gene_expression 722 ( 339) 486 ( 339) 46.95 69.75 56.13
Transcription 137 ( 38) 89 ( 38) 27.74 42.70 33.63
Protein_catabolism 14 ( 3) 4 ( 3) 21.43 75.00 33.33
Phosphorylation 135 ( 60) 95 ( 60) 44.44 63.16 52.17
==[EVT-TOTAL]== 1580 ( 604) 948 ( 604) 38.23 63.71 47.78
------------------------------------------------------------------------------------
Regulation 338 ( 24) 92 ( 24) 7.10 26.09 11.16
Positive_regulation 1186 ( 162) 576 ( 161) 13.66 27.95 18.35
Negative_regulation 416 ( 21) 133 ( 21) 5.05 15.79 7.65
==[REG-TOTAL]== 1940 ( 207) 801 ( 206) 10.67 25.72 15.08
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 811) 1749 ( 810) 23.04 46.31 30.77
------------------------------------------------------------------------------------
Task 2
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 38) 95 ( 38) 21.84 40.00 28.25
Binding 349 ( 64) 228 ( 64) 18.34 28.07 22.18
Gene_expression 722 ( 332) 486 ( 332) 45.98 68.31 54.97
Transcription 137 ( 36) 89 ( 36) 26.28 40.45 31.86
Protein_catabolism 14 ( 3) 4 ( 3) 21.43 75.00 33.33
Phosphorylation 139 ( 54) 119 ( 54) 38.85 45.38 41.86
==[EVT-TOTAL]== 1535 ( 527) 1021 ( 527) 34.33 51.62 41.24
------------------------------------------------------------------------------------
Regulation 292 ( 21) 94 ( 21) 7.19 22.34 10.88
Positive_regulation 987 ( 103) 588 ( 103) 10.44 17.52 13.08
Negative_regulation 379 ( 19) 136 ( 19) 5.01 13.97 7.38
==[REG-TOTAL]== 1658 ( 143) 818 ( 143) 8.62 17.48 11.55
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 670) 1839 ( 670) 20.98 36.43 26.63
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 39) 95 ( 39) 22.41 41.05 29.00
Binding 349 ( 76) 228 ( 76) 21.78 33.33 26.34
Gene_expression 722 ( 339) 486 ( 339) 46.95 69.75 56.13
Transcription 137 ( 38) 89 ( 38) 27.74 42.70 33.63
Protein_catabolism 14 ( 3) 4 ( 3) 21.43 75.00 33.33
Phosphorylation 139 ( 55) 119 ( 55) 39.57 46.22 42.64
==[EVT-TOTAL]== 1535 ( 550) 1021 ( 550) 35.83 53.87 43.04
------------------------------------------------------------------------------------
Regulation 292 ( 22) 94 ( 22) 7.53 23.40 11.40
Positive_regulation 987 ( 112) 588 ( 111) 11.35 18.88 14.17
Negative_regulation 379 ( 20) 136 ( 20) 5.28 14.71 7.77
==[REG-TOTAL]== 1658 ( 154) 818 ( 153) 9.29 18.70 12.41
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 704) 1839 ( 703) 22.05 38.23 27.97
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 39) 95 ( 39) 22.41 41.05 29.00
Binding 349 ( 76) 228 ( 76) 21.78 33.33 26.34
Gene_expression 722 ( 339) 486 ( 339) 46.95 69.75 56.13
Transcription 137 ( 38) 89 ( 38) 27.74 42.70 33.63
Protein_catabolism 14 ( 3) 4 ( 3) 21.43 75.00 33.33
Phosphorylation 139 ( 55) 119 ( 55) 39.57 46.22 42.64
==[EVT-TOTAL]== 1535 ( 550) 1021 ( 550) 35.83 53.87 43.04
------------------------------------------------------------------------------------
Regulation 292 ( 22) 94 ( 22) 7.53 23.40 11.40
Positive_regulation 987 ( 113) 588 ( 112) 11.45 19.05 14.30
Negative_regulation 379 ( 20) 136 ( 20) 5.28 14.71 7.77
==[REG-TOTAL]== 1658 ( 155) 818 ( 154) 9.35 18.83 12.49
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3193 ( 705) 1839 ( 704) 22.08 38.28 28.01
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 239 ( 61) 101 ( 61) 25.52 60.40 35.88
Binding 419 ( 104) 199 ( 104) 24.82 52.26 33.66
Gene_expression 722 ( 332) 486 ( 332) 45.98 68.31 54.97
Transcription 137 ( 36) 89 ( 36) 26.28 40.45 31.86
Protein_catabolism 14 ( 3) 4 ( 3) 21.43 75.00 33.33
Phosphorylation 191 ( 90) 128 ( 90) 47.12 70.31 56.43
==[EVT-TOTAL]== 1722 ( 626) 1007 ( 626) 36.35 62.16 45.88
------------------------------------------------------------------------------------
Regulation 349 ( 25) 94 ( 25) 7.16 26.60 11.29
Positive_regulation 1220 ( 156) 589 ( 156) 12.79 26.49 17.25
Negative_regulation 425 ( 22) 136 ( 22) 5.18 16.18 7.84
==[REG-TOTAL]== 1994 ( 203) 819 ( 203) 10.18 24.79 14.43
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3716 ( 829) 1826 ( 829) 22.31 45.40 29.92
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 239 ( 62) 101 ( 62) 25.94 61.39 36.47
Binding 419 ( 117) 199 ( 117) 27.92 58.79 37.86
Gene_expression 722 ( 339) 486 ( 339) 46.95 69.75 56.13
Transcription 137 ( 38) 89 ( 38) 27.74 42.70 33.63
Protein_catabolism 14 ( 3) 4 ( 3) 21.43 75.00 33.33
Phosphorylation 191 ( 91) 128 ( 91) 47.64 71.09 57.05
==[EVT-TOTAL]== 1722 ( 650) 1007 ( 650) 37.75 64.55 47.64
------------------------------------------------------------------------------------
Regulation 349 ( 26) 94 ( 26) 7.45 27.66 11.74
Positive_regulation 1220 ( 171) 589 ( 170) 14.02 28.86 18.87
Negative_regulation 425 ( 23) 136 ( 23) 5.41 16.91 8.20
==[REG-TOTAL]== 1994 ( 220) 819 ( 219) 11.03 26.74 15.62
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3716 ( 870) 1826 ( 869) 23.41 47.59 31.38
------------------------------------------------------------------------------------
Task 3
N/A
N/A
N/A
N/A
N/A