Relation Annotation in CHF
Annotation scheme
In the narrative EHR reports of PhenoCHF, three types of relations involving entity mentions have been annotated. Compared to entity mention annotation, which identifies mentions of concepts occurring in text, relation annotation encodes some of the more complex pieces of information expressed in text. Specifically, a sentence will often describe how different concepts are are linked toogether in particular ways, or will provide a particular interpretation of an entity.
Table 1 provides details of the three types of relations that have been annotated in PhenoCHF (narrative EHR reports only). Please see the Entity Mention annotation page for descriptions of the different types of entities involved in the relationships. All annotated relationships occur within the scope of a single sentence. Two of these relationships involve pairs of entity mentions which, according to the information provided in the sentence, are associated with each other in specific ways. The semantic label assigned to the relation determines the nature of the association between the two entity mentions. The third relationship (Negate) is annotated when the context of the sentence alters the default interpretation of an entity (i.e., it becomes negated).
Relation Type | Description | First entity type(s) | Second entity type(s) | Example |
---|---|---|---|---|
Causality | The concept referred to by the first entity mention is responsible for the concept refered to by the second entity mention | Chief complaint Cause Risk Factor Non- traditional Risk factor | Non- traditional Risk Factor Cause | |
Finding | The mentioned organ is associated with the manifestation or abnormal variation that is observed during the diagnosis process. | Organ | Sign & Symptom | |
Negate | A word or phrase denoting negation (a polarity cue) is annotated and linked to the mention of the condition that it negates | Polarity cue | Finding Cause Non- traditional risk factor |
The two relationships between pairs of entity mentions are are based on relationships in the UMLS semantic network:
- Causality - based on the causes relation in the UMLS semantic network that holds between two diseases or a disease and a pathologic function
- Finding - based on the manifestation relation in the UMLS semantic network that holds between a sign or symptom and a body part or organ
Relation annotation statistics
All relationships of the types shown in Table 1 were annotated in all narrative EHR records. The total counts of each type of relationship annotated are shown in Table 2
Relation Type | No of annotated relations in narrative EHR reports |
---|---|
Causality | 125 |
Finding | 364 |
Negate | 692 |
Agreement
The relationship annotations were undertaken by two medical doctors. The quality and consistency of the annotations were verified through the calculation of inter-annotator agreement (IAA). We calculated IAA in terms of F-Score, and found that high levels of agreement were acheived (i.e., a macro-averaged F-Score of 0.91).
Featured News
- Invited talk at the 15th Marbach Castle Drug-Drug Interaction Workshop
- Call for papers: CL4Health @ NAACL 2025
- BioNLP 2025 and Shared Tasks accepted for co-location at ACL 2025
- Prof. Junichi Tsujii honoured as Person of Cultural Merit in Japan
- Participation in panel at Cyber Greece 2024 Conference, Athens
- Shared Task on Financial Misinformation Detection at FinNLP-FNP-LLMFinLegal
- New Named Entity Corpus for Occupational Substance Exposure Assessment
- FinNLP-FNP-LLMFinLegal @ COLING-2025 - Call for papers
Other News & Events
- Keynote talk at Manchester Law and Technology Conference
- Keynote talk at ACM Summer School on Data Science, Athens
- Invited talk at the 8th Annual Women in Data Science Event at the American University of Beirut
- Invited talk at the 2nd Symposium on NLP for Social Good (NSG), University of Liverpool
- Invited talk at Annual Meeting of the Danish Society of Occupational and Environmental Medicine