BioNLP'09 Shared Task on
Event Extraction
in conjunction with
BioNLP, a NAACL-HLT 2009
workshop, June 4-5 2009, Boulder, Colorado
Final Evaluation Results
of VIB - Ghent University
Task 1
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 75) 98 ( 75) 43.10 76.53 55.15
Binding 347 ( 129) 342 ( 129) 37.18 37.72 37.45
Gene_expression 722 ( 421) 527 ( 421) 58.31 79.89 67.41
Transcription 137 ( 51) 90 ( 51) 37.23 56.67 44.93
Protein_catabolism 14 ( 9) 15 ( 9) 64.29 60.00 62.07
Phosphorylation 135 ( 74) 85 ( 74) 54.81 87.06 67.27
==[EVT-TOTAL]== 1529 ( 759) 1157 ( 759) 49.64 65.60 56.52
------------------------------------------------------------------------------------
Regulation 291 ( 28) 136 ( 28) 9.62 20.59 13.11
Positive_regulation 983 ( 161) 525 ( 161) 16.38 30.67 21.35
Negative_regulation 379 ( 84) 247 ( 84) 22.16 34.01 26.84
==[REG-TOTAL]== 1653 ( 273) 908 ( 273) 16.52 30.07 21.32
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1032) 2065 ( 1032) 32.43 49.98 39.34
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 76) 97 ( 76) 43.68 78.35 56.09
Binding 347 ( 132) 342 ( 132) 38.04 38.60 38.32
Gene_expression 722 ( 429) 526 ( 429) 59.42 81.56 68.75
Transcription 137 ( 54) 89 ( 54) 39.42 60.67 47.79
Protein_catabolism 14 ( 9) 15 ( 9) 64.29 60.00 62.07
Phosphorylation 135 ( 76) 85 ( 76) 56.30 89.41 69.09
==[EVT-TOTAL]== 1529 ( 776) 1154 ( 776) 50.75 67.24 57.85
------------------------------------------------------------------------------------
Regulation 291 ( 31) 136 ( 31) 10.65 22.79 14.52
Positive_regulation 983 ( 169) 525 ( 169) 17.19 32.19 22.41
Negative_regulation 379 ( 86) 247 ( 86) 22.69 34.82 27.48
==[REG-TOTAL]== 1653 ( 286) 908 ( 286) 17.30 31.50 22.34
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1062) 2062 ( 1062) 33.38 51.50 40.50
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 76) 97 ( 76) 43.68 78.35 56.09
Binding 347 ( 132) 342 ( 132) 38.04 38.60 38.32
Gene_expression 722 ( 429) 526 ( 429) 59.42 81.56 68.75
Transcription 137 ( 54) 89 ( 54) 39.42 60.67 47.79
Protein_catabolism 14 ( 9) 15 ( 9) 64.29 60.00 62.07
Phosphorylation 135 ( 76) 85 ( 76) 56.30 89.41 69.09
==[EVT-TOTAL]== 1529 ( 776) 1154 ( 776) 50.75 67.24 57.85
------------------------------------------------------------------------------------
Regulation 291 ( 31) 136 ( 31) 10.65 22.79 14.52
Positive_regulation 983 ( 169) 525 ( 169) 17.19 32.19 22.41
Negative_regulation 379 ( 87) 247 ( 87) 22.96 35.22 27.80
==[REG-TOTAL]== 1653 ( 287) 908 ( 287) 17.36 31.61 22.41
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3182 ( 1063) 2062 ( 1063) 33.41 51.55 40.54
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 75) 98 ( 75) 43.10 76.53 55.15
Binding 398 ( 176) 328 ( 176) 44.22 53.66 48.48
Gene_expression 722 ( 421) 527 ( 421) 58.31 79.89 67.41
Transcription 137 ( 51) 90 ( 51) 37.23 56.67 44.93
Protein_catabolism 14 ( 9) 15 ( 9) 64.29 60.00 62.07
Phosphorylation 135 ( 74) 85 ( 74) 54.81 87.06 67.27
==[EVT-TOTAL]== 1580 ( 806) 1143 ( 806) 51.01 70.52 59.20
------------------------------------------------------------------------------------
Regulation 338 ( 45) 144 ( 45) 13.31 31.25 18.67
Positive_regulation 1186 ( 236) 553 ( 236) 19.90 42.68 27.14
Negative_regulation 416 ( 94) 255 ( 94) 22.60 36.86 28.02
==[REG-TOTAL]== 1940 ( 375) 952 ( 375) 19.33 39.39 25.93
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1181) 2095 ( 1181) 33.55 56.37 42.07
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 76) 97 ( 76) 43.68 78.35 56.09
Binding 398 ( 179) 327 ( 179) 44.97 54.74 49.38
Gene_expression 722 ( 429) 526 ( 429) 59.42 81.56 68.75
Transcription 137 ( 54) 89 ( 54) 39.42 60.67 47.79
Protein_catabolism 14 ( 9) 15 ( 9) 64.29 60.00 62.07
Phosphorylation 135 ( 76) 85 ( 76) 56.30 89.41 69.09
==[EVT-TOTAL]== 1580 ( 823) 1139 ( 823) 52.09 72.26 60.54
------------------------------------------------------------------------------------
Regulation 338 ( 49) 144 ( 49) 14.50 34.03 20.33
Positive_regulation 1186 ( 251) 551 ( 251) 21.16 45.55 28.90
Negative_regulation 416 ( 101) 255 ( 101) 24.28 39.61 30.10
==[REG-TOTAL]== 1940 ( 401) 950 ( 401) 20.67 42.21 27.75
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3520 ( 1224) 2089 ( 1224) 34.77 58.59 43.64
------------------------------------------------------------------------------------
Task 2
N/A
N/A
N/A
N/A
N/A
Task 3
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 75) 98 ( 75) 43.10 76.53 55.15
Binding 347 ( 129) 342 ( 129) 37.18 37.72 37.45
Gene_expression 722 ( 421) 527 ( 421) 58.31 79.89 67.41
Transcription 137 ( 51) 90 ( 51) 37.23 56.67 44.93
Protein_catabolism 14 ( 9) 15 ( 9) 64.29 60.00 62.07
Phosphorylation 135 ( 74) 85 ( 74) 54.81 87.06 67.27
==[EVT-TOTAL]== 1529 ( 759) 1157 ( 759) 49.64 65.60 56.52
------------------------------------------------------------------------------------
Regulation 291 ( 28) 136 ( 28) 9.62 20.59 13.11
Positive_regulation 983 ( 161) 525 ( 161) 16.38 30.67 21.35
Negative_regulation 379 ( 84) 247 ( 84) 22.16 34.01 26.84
==[REG-TOTAL]== 1653 ( 273) 908 ( 273) 16.52 30.07 21.32
------------------------------------------------------------------------------------
Negation 227 ( 17) 52 ( 17) 7.49 32.69 12.19
Speculation 208 ( 16) 115 ( 16) 7.69 13.91 9.91
==[MOD-TOTAL]== 435 ( 33) 167 ( 33) 7.59 19.76 10.96
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3617 ( 1065) 2232 ( 1065) 29.44 47.72 36.42
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 76) 97 ( 76) 43.68 78.35 56.09
Binding 347 ( 132) 342 ( 132) 38.04 38.60 38.32
Gene_expression 722 ( 429) 526 ( 429) 59.42 81.56 68.75
Transcription 137 ( 54) 89 ( 54) 39.42 60.67 47.79
Protein_catabolism 14 ( 9) 15 ( 9) 64.29 60.00 62.07
Phosphorylation 135 ( 76) 85 ( 76) 56.30 89.41 69.09
==[EVT-TOTAL]== 1529 ( 776) 1154 ( 776) 50.75 67.24 57.85
------------------------------------------------------------------------------------
Regulation 291 ( 31) 136 ( 31) 10.65 22.79 14.52
Positive_regulation 983 ( 169) 525 ( 169) 17.19 32.19 22.41
Negative_regulation 379 ( 86) 247 ( 86) 22.69 34.82 27.48
==[REG-TOTAL]== 1653 ( 286) 908 ( 286) 17.30 31.50 22.34
------------------------------------------------------------------------------------
Negation 227 ( 18) 52 ( 18) 7.93 34.62 12.90
Speculation 208 ( 16) 115 ( 16) 7.69 13.91 9.91
==[MOD-TOTAL]== 435 ( 34) 167 ( 34) 7.82 20.36 11.30
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3617 ( 1096) 2229 ( 1096) 30.30 49.17 37.50
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 76) 97 ( 76) 43.68 78.35 56.09
Binding 347 ( 132) 342 ( 132) 38.04 38.60 38.32
Gene_expression 722 ( 429) 526 ( 429) 59.42 81.56 68.75
Transcription 137 ( 54) 89 ( 54) 39.42 60.67 47.79
Protein_catabolism 14 ( 9) 15 ( 9) 64.29 60.00 62.07
Phosphorylation 135 ( 76) 85 ( 76) 56.30 89.41 69.09
==[EVT-TOTAL]== 1529 ( 776) 1154 ( 776) 50.75 67.24 57.85
------------------------------------------------------------------------------------
Regulation 291 ( 31) 136 ( 31) 10.65 22.79 14.52
Positive_regulation 983 ( 169) 525 ( 169) 17.19 32.19 22.41
Negative_regulation 379 ( 87) 247 ( 87) 22.96 35.22 27.80
==[REG-TOTAL]== 1653 ( 287) 908 ( 287) 17.36 31.61 22.41
------------------------------------------------------------------------------------
Negation 227 ( 24) 51 ( 23) 10.57 45.10 17.13
Speculation 208 ( 18) 114 ( 18) 8.65 15.79 11.18
==[MOD-TOTAL]== 435 ( 42) 165 ( 41) 9.66 24.85 13.91
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3617 ( 1105) 2227 ( 1104) 30.55 49.57 37.80
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 75) 98 ( 75) 43.10 76.53 55.15
Binding 398 ( 176) 328 ( 176) 44.22 53.66 48.48
Gene_expression 722 ( 421) 527 ( 421) 58.31 79.89 67.41
Transcription 137 ( 51) 90 ( 51) 37.23 56.67 44.93
Protein_catabolism 14 ( 9) 15 ( 9) 64.29 60.00 62.07
Phosphorylation 135 ( 74) 85 ( 74) 54.81 87.06 67.27
==[EVT-TOTAL]== 1580 ( 806) 1143 ( 806) 51.01 70.52 59.20
------------------------------------------------------------------------------------
Regulation 338 ( 45) 144 ( 45) 13.31 31.25 18.67
Positive_regulation 1186 ( 236) 553 ( 236) 19.90 42.68 27.14
Negative_regulation 416 ( 94) 255 ( 94) 22.60 36.86 28.02
==[REG-TOTAL]== 1940 ( 375) 952 ( 375) 19.33 39.39 25.93
------------------------------------------------------------------------------------
Negation 220 ( 23) 51 ( 23) 10.45 45.10 16.97
Speculation 201 ( 18) 112 ( 18) 8.96 16.07 11.50
==[MOD-TOTAL]== 421 ( 41) 163 ( 41) 9.74 25.15 14.04
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3941 ( 1222) 2258 ( 1222) 31.01 54.12 39.43
------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
Event Class gold (match) answer (match) recall prec. fscore
------------------------------------------------------------------------------------
Localization 174 ( 76) 97 ( 76) 43.68 78.35 56.09
Binding 398 ( 179) 327 ( 179) 44.97 54.74 49.38
Gene_expression 722 ( 429) 526 ( 429) 59.42 81.56 68.75
Transcription 137 ( 54) 89 ( 54) 39.42 60.67 47.79
Protein_catabolism 14 ( 9) 15 ( 9) 64.29 60.00 62.07
Phosphorylation 135 ( 76) 85 ( 76) 56.30 89.41 69.09
==[EVT-TOTAL]== 1580 ( 823) 1139 ( 823) 52.09 72.26 60.54
------------------------------------------------------------------------------------
Regulation 338 ( 49) 144 ( 49) 14.50 34.03 20.33
Positive_regulation 1186 ( 251) 551 ( 251) 21.16 45.55 28.90
Negative_regulation 416 ( 101) 255 ( 101) 24.28 39.61 30.10
==[REG-TOTAL]== 1940 ( 401) 950 ( 401) 20.67 42.21 27.75
------------------------------------------------------------------------------------
Negation 220 ( 24) 50 ( 24) 10.91 48.00 17.78
Speculation 201 ( 18) 112 ( 18) 8.96 16.07 11.50
==[MOD-TOTAL]== 421 ( 42) 162 ( 42) 9.98 25.93 14.41
------------------------------------------------------------------------------------
==[ALL-TOTAL]== 3941 ( 1266) 2251 ( 1266) 32.12 56.24 40.89
------------------------------------------------------------------------------------