Evaluating research papers is straightforward when you know how to critique. Karen Baker explains more.
Midwives magazine: Issue 2 :: 2014
Part of the midwife’s job is to find and evaluate evidence about the effectiveness of their practice and healthcare interventions (NMC, 2008).
To do that, a midwife needs to be able to work out whether or not a piece of research is robust. This is where the ability to critique is invaluable.
Critiquing enables you to assess the quality of a piece of literature and identify its strengths and weaknesses.
When evaluating a research paper, critiquing helps you to make a judgement on whether the results of the study have been influenced, perhaps by the characteristics of the study design or how it was carried out (Aveyard, 2010).
This is essential as faults in the design or conduct of a study can result in bias and influence the findings. Therefore, assessing the quality of a study, by critiquing it, helps measure whether a study is robust enough to affect decisions regarding practice (Steen and Roberts, 2011). Critiquing is also an essential part of the systematic review process (Centre for Reviews and Dissemination, 2009).
Prior to conducting a critique of the literature, you will have decided on an area of investigation and carried out a detailed search of it, identifying relevant sources of information.
Finding the right literature
When thinking about an area of investigation, bear in mind that some categories of literature are better suited to providing answers than others. Different categories answer different research questions. Furthermore, different categories of literature are arranged in a hierarchy of evidence to assess their quality in answering the research question (see Figure 1).
For example, if you are investigating whether an intervention is effective, then quantitative research studies consisting of randomised controlled trials (RCTs) should be at the top of your hierarchy of evidence.
However, if you are investigating people’s experiences or views regarding an issue, then qualitative research studies consisting of phenomenological studies are more appropriate.
Once you have amassed your evidence, group together the different categories of literature (see Figure 2). For example, all the qualitative research papers should be together, the quantitative pieces grouped with each other and, likewise, all the practice papers together.
You then need to read and reread your literature until you are familiar with it and can summarise it. Consider what it is about, identify its aims and think about what the author did and what the results show.
Once you are familiar with your literature, you can then conduct a detailed critical appraisal. The best way to do this is to use a specific critical appraisal tool to ensure a consistent systematic approach. It also allows you to assess whether the literature is of high enough quality to answer your research question or contribute to your area of investigation (Aveyard, 2010).
Tools for critique
When critiquing a piece of literature, you need to familiarise yourself with it so you can assess its relevance, strengths and limitations to your research question or area under investigation.
Having a framework by which to examine a research article can be helpful. When reviewing a research paper, adopting a critical appraisal framework provides general and more specific questions to ask.
This helps the reviewer to identify whether the research has been well designed and conducted, whether it has any limitations and whether it has any benefit to the wider population (Steen and Roberts, 2011).
In quantitative research, the terms of validity and reliability are used (Bryman, 2008). And, in qualitative research, the terms credibility, transferability, dependability and confirmability are used to assess trustworthiness (Aveyard, 2010).
Different frameworks are used, depending on whether the research is qualitative or quantitative. The Critical Appraisal Skills Programme (CASP) (2010) has developed critical appraisal tools, which have been validated, to ensure that studies are assessed and critically appraised in a standardised way.
When critiquing RCTs, in conjunction with CASP, refer to the CONSORT (Consolidated Standards of Reporting Trials) statement (2010) (Greenhalgh, 2010; Schulz et al, 2010). Rochon et al’s guide to critical appraisal (2005) can also be used in conjunction with CASP when critiquing cohort studies. The AGREE collaboration has also developed a website (agreetrust.org) that offers advice and tools for appraising the quality of clinical guidelines.
Also, Walsh and Down (2006) have developed a framework to assess qualitative research after finding the ones that already exist are needlessly detailed and lengthy. There are also critical appraisal tools for all types of research, such as Polit and Beck (2009).
For non-research papers, there are generic critical appraisal tools, such as Woolliams et al (2009) and Cottrell (2005). The ideal, though, is to find an appraisal tool that is specific to the type of literature you are critiquing (Aveyard, 2010). However, regardless of your choice, a critical appraisal tool will not help you if you do not understand the study’s research design (Greenhalgh, 2010).
Once you have critiqued your pieces of literature, bring it all together to address your research question or area under investigation.
Using your completed critical tool(s), write a short paragraph summarising the aim of each piece of literature, the method and the main findings, the strengths and the weaknesses of each paper and the conclusions.
Decide whether the conclusions of the piece of literature are convincing from the results of the main study or discussion.
A chart can help you make this decision. Plotting or laying out the findings on a chart can help summarise your findings and work out whether the literature you have identified is robust enough to affect decisions regarding practice (Aveyard, 2010).
Taking a methodical approach in which the relevant available tools are employed means that a critique of literature is straightforward and even enjoyable – when you know how.
Midwife, Calderdale Birth Centre, West Yorks
For modules on critiquing, visit RCM i-learn.