NaCTeM

Automatic Summarisation for Systematic Reviews using Text Mining (ASSERT)

Overview

The ASSERT project centres around providing for broader institutional involvement in text mining through a community call, whilst at the same time developing an exemplar service for the social sciences domain.

The overall objectives of the proposed community call are:

  1. to liase with domain experts from other institutions who will use text mining as an integral part of their projects supported under the community call
  2. to configure appropriate text mining workflows for the specified purposes of the supported projects
  3. to demonstrate the applicability of the text mining technology in social sciences;
  4. to disseminate the benefits of text mining to the wider social sciences community via NaCTeM services.

The overall objectives of the proposed exemplar service are:

  1. to develop cost-effective and rapid methods for locating relevant studies for input to a systematic review using a combination of text mining techniques;
  2. to apply a suite of text mining tools which will support novel methods of information management in the domain of social science systematic reviews (document clustering, information extraction and text summarisation);
  3. to demonstrate the applicability of the text mining technology in social sciences, in cooperation with the EPPI currently heavily involved in producing systematic reviews;
  4. to disseminate the benefits of text mining to the wider social sciences community via NaCTeM services.

What is a Systematic Review

Before undertaking any new policy, practice, research, or before making any other decisions it can be useful to find out what is already known about an issue in a fair, unbiased manner, in order to be of any scientific value to the community. This knowledge may include the findings of individual research studies that might, alone, be limited in their applicability and vulnerable to bias. In order to minimise this bias, therefore, a large number of people and organisations, such as the Cochrane Collaboration, the Centre for Reviews and Dissemination's guidelines have developed methods for locating research evidence and synthesising it in order to inform decision-making. They have developed ways of conducting literature reviews of research in a systematic way, which provide users with a 'short-cut' to relevant evidence.

Systematic reviews usually proceed along the following stages:

  1. First, extensive searches are carried out in order to locate as much relevant research as posible according to a query. These searches often include electronic databases, scanning reference lists and searching for unpublished literature. This stage includes the definition of a set of inclusion and exclusion criteria on which the researchers base their searching.
  2. Then the mass of the data retrieved by this process is screened until only the most relevant and reliable literature remains to form the focus of the review.
  3. Finally, the literature is synthesised and summary reports are written to inform policy and practice. The summaries of research which are produced in this systematic way are then used to help users of the research to make evidence based decisions.

Currently, this task is performed manually by the researchers, who often encounter many problems. The proliferation of information in textual form means that the quantity of potentially relelvant literature retrieved in the early stages of review can become unmanageable. Reviewers have been accustomed to sacrificing specificity in their searches in order to ensure they have not missed any relevant studies. They conduct searches which yield large numbers of 'hits'. They download the titles and abstracts, usually into bibliographic software, and look through them manually. This process is often called 'screening' and is the most time-consuming part of the review. Reviewers are finding that they sometimes need to scan through tens of thousands of titles and abstracts which are retrieved from large databases such as ERIC and Medline to decide whether or not they meet the inclusion criteria for a review.