Introduction
Critical appraisal is a process which is used to help you identify the strengths and weakness of a research paper. Understanding how appropriate the study design is for the question you are seeking to answer, how well the study was carried out, and how good the reporting in the paper is helps you to assess whether the paper is likely to provide reliable evidence.
The purpose of this page is to help you assess the level of evidence from the many different types of research study that are being published.
This page will help you:
- understand what we mean by levels of evidence
- understand how the study design relates to the level of evidence.
Research studies in veterinary science can be designed in a variety of ways, depending on the type of question they are trying to answer. These different study designs are often arranged into a hierarchy known as the ‘levels of evidence’ with practitioners encouraged to find the highest level of evidence possible to answer their clinical questions.
Many versions of the hierarchy of evidence concentrate on clinical studies, placing Randomised Controlled Trials (RCTs), and Systematic Reviews and metanalyses of such studies, at the top of the evidence hierarchy.
However, we are increasingly seeing the publication of social science papers, looking at the human factors involved in the delivery of veterinary care, and studies which use new technology to provide evidence based on large data sets or report on the development of computer models of animal disease, or machine learning based diagnostics.
What are the levels of evidence?
The levels of evidence are often portrayed as a pyramid of evidence, which helps to convey that some types of study provide a higher level of evidence than others, and also that higher levels of evidence are often built on and supported by lower levels of evidence (Arlt & Heuweiser 2016; Cockcroft & Holmes 2008).
There are many different types of study design, so it is helpful to start by considering in broad terms what makes some types of study more likely to provide a higher level, that is more reliable and accurate, evidence than others.
Synthesised evidence
At the top of the pyramid, and considered to provide the highest level of evidence, we have synthesised evidence, which involves the critical evaluation and summary of previously published studies.
Intervention studies
Below this we have intervention studies, such as clinical trials, where the researcher controls the intervention (diagnostic test or treatment) that the animal is exposed to. The reliability of these types of studies can be increased by providing a suitable comparator or control to the intervention being assessed, randomisation of the participants between the trial and control arm of the study and blinding of participants and researchers to which participants received the intervention.
Observational studies
Observational studies are those where the researcher observes and compares the outcomes between two or more groups without having any control over which animals receive the intervention or exposure. While these studies may provide some evidence of differences between the groups, there may be many confounding factors which could influence the outcome other than the intervention under investigation.
Descriptive studies
Descriptive studies are those where the author records the details of individuals or groups of animals, but due to lack of a comparator group cannot analyse variables and causal relationships.
Background information
Background information does not count as strong evidence, but it may provide the information that enables you to formulate an answerable question or evaluate higher level published evidence.
How study design relates to level of evidence
| Type of study | Description | Examples |
|---|---|---|
| Synthesised evidence | Critical evaluation and summary of previously published studies. | Meta-analysis, Systematic review, Knowledge Summary, Critically Appraised Topics, Evidence-based guidelines |
| Intervention studies / Clinical trials | Studies where the researcher controls the intervention (diagnostic test or treatment) that the animal is exposed to. | Randomised controlled trials (RCTs), Non-randomised controlled trials, Crossover trials |
| Observational studies | Studies where the researcher observes and compares the outcomes between 2 or more groups without having any control over which animals receive the intervention or exposure. | Case-control studies, Cohort studies, Cross sectional studies, Diagnostic test, validation studies |
| Descriptive studies | Studies where the author records details of individuals or groups of animals. | Case series, Case reports |
| Background information | Information which provides us with general information. | Narrative reviews, Textbooks/ lectures, Non-clinical studies |
Remember: Study design is not the only factor
- The levels of evidence are designed to help you to concentrate your efforts on sources that are most likely to provide reliable evidence. However, is important to remember that study design is not the only factor to consider when critically appraising a paper.
- It should be noted that each type of study has its own strengths and limitations. For example, a case-control study is an appropriate way to study the aetiology of a disease and a qualitative study would be appropriate address questions regarding the perceived quality of life of a patient after an intervention.
- Randomised controlled trials are often considered to provide the highest level of evidence, because their methodological design inherently reduces bias. However, a poorly designed Randomised Controlled Trial (RCT) may provide less reliable evidence than a good cohort study. The information below on the different types of study should help you to identify different types of study and assess the level of evidence it is likely to provide.
- Alongside the study design it is also relevant to look at whether the data was collected prospectively, for the purpose of the study, or whether the study uses retrospective data that had been collected for another purpose, for example as part of the animal’s clinical records. For some types of study design data can only be collected prospectively, e.g. a Randomised Controlled Trial, this has the benefit that the data is likely to be more complete. However, retrospective data, while it may be incomplete, can often allow analysis from a larger population of participants to be collected and analysed more quickly.
References and further reading
- Arlt, S.P. and Heuwieser, W., 2016. The staircase of evidence–a new metaphor displaying the core principles of evidence-based veterinary medicine. Veterinary Evidence, 1(1). https://doi.org/10.18849/ve.v1i1.18
- Cardwell, J.M. (2008) An overview of study design. Journal of Small Animal Practice, 49 (5), pp. 217-218. https://doi.org/10.1111/j.1748-5827.2008.00594.x
- Cockcroft, P. and Holmes, M., 2008. Handbook of evidence-based veterinary medicine. John Wiley & Sons.
- Dean, R. (2013) How to read a paper and appraise the evidence. In Practice, 35 (5), pp. 282-285. DOI: https://doi.org/10.1136/inp.f1760
- Greenhalgh, T. (2014) How to read a paper: The basics of evidence-based medicine. 5th ed. Wiley Blackwell
- Hennessey, E., DiFazio, M., Hennessey, R. and Cassel, N., 2022. Artificial intelligence in veterinary diagnostic imaging: A literature review. Veterinary Radiology & Ultrasound, 63, pp.851-870. https://doi.org/10.1111/vru.13163
- Hernán, M.A., Wang, W. and Leaf, D.E., 2022. Target trial emulation: a framework for causal inference from observational data. Jama, 328 (24), pp.2446-2447. https://doi.org/10.1001/jama.2022.21383
- Pereira, A.I., Franco-Gonçalo, P., Leite, P., Ribeiro, A., Alves-Pimenta, M.S., Colaço, B., Loureiro, C., Gonçalves, L., Filipe, V. and Ginja, M., 2023. Artificial Intelligence in Veterinary Imaging: An Overview. Veterinary Sciences, 10 (5), p.320. https://doi.org/10.3390/vetsci10050320