EBVM toolkit 4: Identifying the study design 

Introduction

The information needed to identify the type of study design is normally found in the methodology (i.e. in the methods section of the paper).  It is good practice not to rely solely on the abstract when identifying study design as it rarely gives sufficient information for you to be sure that the description is accurate.  For example, the abstract may say the study was ‘a randomised controlled trial’ but you would need to read the methodology to see how the randomisation was achieved in order to confirm if this was indeed the case.  

What types of clinical research are there?

The table in EBVM Toolkit 3 shows a broad categorisation of the different type of clinical research, arranged according to the level of evidence that they are likely to provide for clinical decision making.

View this table in EBVM Toolkit 3 – How study design relates to level of evidence.

Background information

Background information is generally considered low quality evidence for decision making. However, it can be useful for giving an overview of a subject and providing the knowledge you need to find and assess higher quality evidence.

In terms of the published veterinary literature the most common forms of background information are opinion pieces and narrative reviews. Textbooks and CPD notes may also fall in this category.

Examples of background information include:

• Opinion pieces: These are not based on a literature search. Instead, the authors give their opinions on a subject without any explicit appraisal of existing literature though they may provide a couple of references to back up their claims.
• Narrative reviews: These provide an overview of a subject.  They may include references, but they rarely include details of specific search protocols or explicit criteria by which papers are included or excluded. Instead, they rely on the author to draw conclusions based on the papers they find most relevant or interesting.
  
  While background information is generally considered to provide low quality evidence, its usefulness may be improved if it provides references to a range of appropriate sources, contains explicit discussion of the evidence on which it is based, and acknowledges areas of uncertainty and knowledge gaps.

Descriptive studies

Descriptive, or non-comparative studies, are designed to record what is seen. They provide a description of what has been seen in an individual or group of animals, but do not attempt any comparison to a control group. These studies have value if the aim of the paper is to highlight an unusual finding, such as a new disease, or report a rare occurrence.

Descriptive studies will not be able to demonstrate causation, so when using this type of study care should be taken to avoid over-interpreting the findings by making conclusions regarding causal links.

Examples of descriptive studies include:

• Case reports: These are reports on a single patient. They describe the presentation and/or course of a disease.
• Case series: These are collections of case reports and can provide descriptive quantitative data.

Observational studies

Observational studies are those where the researcher examines the outcomes of an intervention within two groups without having any influence on which animals get the intervention.

For example, a researcher could consider the rate of complication following different types of surgery by looking back at all the surgical cases and analysing those that resulted in complications. Where two groups are compared it is possible to make inferences about the association of risk factors and outcomes. However, because there are many possible confounding factors which have not been controlled it is not possible to prove causation.

Examples of observational studies include:

• Case-control studies: These are where animals which have a disease condition are identified and any causal or risk factors are compared to a control group (without the disease). Information regarding the exposure is historical These studies start with groups that already have the outcome (e.g. diabetes) and it looks back to examine what might have been the exposure factors (e.g. obesity).

Large data sets can be used to undertake nested case-control studies, where cases of a disease are identified and, for each, a specified number of matched controls is selected from among those in the data set that have not developed the disease, by the time of disease occurrence in the case.

• Cohort studies: These identify a group of animals and follow them over time to see how their exposures affect their outcomes compared to another group that were not exposed to that factor, either the general population or another cohort of animals.
  

• Cross-sectional studies: These are studies that describe the characteristics of sample groups of animals. Data is collected at one point in time and two groups are identified – usually animals with a specified disease and those without. The relationships within the groups to given parameters are then considered, and typically expressed as an odds ratio. As the data is taken at one point in time direction of case and effect cannot be established.

• Controlled before-and-after/interrupted time series: These are studies that measure the characteristics of a group of animals before and after an event or intervention. The two sets of data are then compared to judge the effect of the event or intervention.

Interventional studies

Interventional studies are those where there is an intervention (e.g. treatment, drug therapy, surgical method, exposure to a chemical etc) with a researcher responsible for designing the intervention and deciding which animals are exposed to it. By controlling the conditions in which the intervention takes place the researcher can control for potential confounding variables and increase the likelihood that differences between the groups are the result of the intervention.

• Randomised controlled trials (RCTs): These can either be experimental laboratory studies or clinical trials. RCTs have two important features:  there are at least two groups – a treatment group and a control group and animals are randomly assigned into the two groups.  Randomisation of participants and blinding of the investigator to which animals are in which group reduce bias, and lead to randomised controlled trials being considered the ‘gold standard’ when assessing the efficacy of a treatment.
• Non-randomised controlled studies: Not every intervention can, or should, be randomised. Non-randomised controlled trials can detect associations between an intervention and an outcome, but they cannot rule out the possibility that the association was caused by a third factor linked to both intervention and outcome.
• Cross-over trials: These involve the administration of two or more interventions one after the other in a specified or random order to the same group of patients. In these trials each participant acts as its own control, minimising the risk of confounding.
  

Synthesised evidence

These are studies which systematically review and critically evaluate already published literature. They may also be referred to as secondary sources of evidence, in contrast to the primary sources of evidence form published research studies.

When appraising secondary sources, it is still important to consider how reliable the evidence is, by looking for a clear description of how the evidence has been gathered and evaluated.  With secondary sources of evidence, it is particularly important to consider whether any important or relevant evidence has been left out, such as newly published primary research.

Sources of synthesised evidence include:

• Systematic reviews: These are comprehensive surveys of a topic in which all the primary studies of the evidence have been systematically identified, those providing the most relevant evidence have been selected and critically appraised, and then graded and summarised according to explicit and reproducible methodologies. A systematic review should be performed in a highly structured way, with the question and methods clearly defined in advance to minimise any bias in the selection of papers and grading of evidence.

• Meta analyses: These are secondary studies the in which the quantitative data from several similar research studies are combined and analysed as if they were a single study.  Analyses of this type are normally accompanied by some sort of graphical representation, for example a forest plot.
• Knowledge summaries/critically appraised topics: A knowledge summary, or critically appraised topic, is a concise appraisal of the best available evidence on a defined clinical question. They should follow an explicit methodology to identify and critically appraise the evidence.

• Evidence-based guidelines: Guidelines can provide an accessible way for practitioners to access recommendations for the care of patients with specific conditions. However, if they are to provide a high level of evidence and a reliable basis for clinical decision making it is important that they are based on a systematic and critical evaluation of the available evidence.

Social science studies

Social science studies seek to understand humans and their interactions. They may use quantitative methods, like those that have been described in the section on clinical research. However, they may also use qualitative methods which seek to understand the experiences of the human subjects involved.

Studies using social science methods may be relevant to the delivery of veterinary care or the experience of caring for animals with certain conditions. They may involve veterinary professionals or owners as participants.

Even if you are unfamiliar with the particular methodology being used it should be possible to place them within the broad categories of the levels of evidence.

You can view this ‘Levels of evidence’ in EBVM Toolkit 3: introduction to ‘Levels of evidence’ and study design.

If you are still unsure of the types of study you are looking at the  questions in the link below will help you identify the  study design in the paper you are reading and the relevant critical appraisal checklist.

You should work your way through the questions until you are satisfied with the answer and that you have identified a study design. If you reach the end and are still unsure please contact us at library@rcvsknowledge.org and we will try and help you.

Further assistance

If you get to the end of the questions and are still unsure about the type of study design please email library@rcvsknowledge.org and we will try to help you identify the study design and find a checklist that will allow you to appraise the paper.

References

• Arlt, S.P. and Heuwieser, W. (2016) The staircase of evidence–a new metaphor displaying the core principles of evidence-based veterinary medicine. Veterinary Evidence, 1(1). https://doi.org/10.18849/ve.v1i1.18
• Cardwell, J.M. (2008) An overview of study design. Journal of Small Animal Practice, 49 (5), pp. 217-218. https://doi.org/10.1111/j.1748-5827.2008.00594.x
• Cockcroft, P. and Holmes, M. (2008) Handbook of evidence-based veterinary medicine. John Wiley & Sons.
• Dean, R. (2013) How to read a paper and appraise the evidence. In Practice, 35 (5), pp. 282-285. https://doi.org/10.1136/inp.f1760
• Greenhalgh, T. (2014) How to read a paper: The basics of evidence-based medicine. 5th ed. Wiley Blackwell
• Hennessey, E. et al. (2022) Artificial intelligence in veterinary diagnostic imaging: A literature review. Veterinary Radiology & Ultrasound, 63, pp. 851-870. https://doi.org/10.1111/vru.13163
• Hernán, M.A., Wang, W. and Leaf, D.E. (2022) Target trial emulation: a framework for causal inference from observational data. Journal of the American Medical Association, 328 (24), pp. 2446-2447. https://doi.org/10.1001/jama.2022.21383
• Pereira, A.I. et al. (2023) Artificial Intelligence in Veterinary Imaging: An Overview. Veterinary Sciences, 10 (5), p. 320. https://doi.org/10.3390/vetsci10050320