Building confidence in process and outcomes of systematic literature reviews in healthcare
Systematic literature reviews are the highest form of research in clinical healthcare (see Figure 1) and aim to review or analyse the existing body of research in a given topic with the same level of rigour that should be used in producing that research evidence in the first place. These reviews have increasingly replaced traditional narrative reviews and expert commentaries as a tool for summarizing research evidence. The outcome of such reviews is often a change to procedure, practice, treatment or health policy at country or global level depending on the scope of the review. It is easy to see the desire for a more systematic method of analysing and synthesising the robust scientific evidence to change healthcare at this level. The challenge posed by Imperial College was to design a simple and elegant to use digital platform that is more accessible to researchers and reinforces the rigour of the systematic review process and evidence synthesis in healthcare.
The failings of some of the less rigorous review methods are well documented in the clinical research world. These reviews are sometimes called 'literature reviews', 'narrative reviews', 'critical reviews' or 'commentaries'. One of the main characteristics that sets them apart is that they are not led via a peer-reviewed process and thus making it very difficult to replicate the findings. One of the main objectives and a differentiator for this solution was to document the process of the review with the utmost rigour so that anyone within the platform can view it, scrutinise it and, if desired, to replicate it. This level of transparency and rigour across the entire process is what allows the clinical community to find the confidence within the process and ultimately in the outcome of the systematic literature review.
This confidence is the most important outcome of the review process and it is also one of the key drivers of decision making in the process. As reviewers gather the data they intend to review based on their review question, they begin to assess the confidence they have with the study design, the principal researchers that have orchestrated the studies and the type of study data they are dealing with. For example, a Random Controlled Trial data (quantitative) that relates to the review question directly is considered of the least biased and therefore inspires much confidence. On the other hand, a qualitative questionnaire that loosely relates to the study would not build much confidence in the result and it's relationship to the review objective. In short, the rigour and discipline in evaluating each step of the process is critical to the confidence of the reviewers in the outcomes of their selections an ultimately their analysis and recommendations.
The project was designed to consist of two distinct phases and two outcomes. Phase I was dedicated to an in-depth understanding of the systematic literature review process and the reviewer's needs across that process. Imperial College experts in systematic literature reviews provided in-depth personal accounts and insight into the inner workings of systematic review teams, roles, methods, tools formal and informal activities as well as outcomes across the entire review process (see Figure 3).
These iterative in-depth interviews and process visualization sessions with the world's top experts in evidence synthesis in healthcare were supplemented by suggested readings which you can find in the reference section at the bottom of this page. The focus of Phase 2 was on the development of the platform concept, the design of the features and functions of the concept, and the design of high fidelity dashboard wireframes to communicate the potential capability of the platform to the various internal stakeholders as well as to the development team.
The platform would enable reviewers to manage the entire systematic review process including administrative tasks related to team management, designing the question, filtering down their review sample of studies based on the criteria they deemed as important, and finally processing the studies (less most complex statistical analysis) and reporting of the results.
The experience model revolved around various modes relevant to the process and was rooted in concrete user needs across the systematic review process. Below you will find a few examples of dashboards that illustrate the overall design direction of the solution. They represent only a subset of the solution and are included here only to help illustrate the output delivered.
- Evidence Synthesis in Healthcare: A practical handbook, Athanasiou, Darza (Eds), Springer, 2011
- What is a systematic review?: What is a...? [series], [online] www.whatisseries.co.uk, accessed December, 2015
- Avoidable waste in the production and reporting of research evidence, Chambers and Glasziou, Oxford University Press, Lancet 2009
- Scopus: Quick reference guide, [online], www.elsevier.com/scopus, Elsevier 2016
- Harmonization, [online], www.clearstorydata.com, Clear Story Data, 2014
Banner source: http://www.mesotheliomahelpnow.com/wp-content/uploads/MESO2015_40.jpg