EBVM toolkit 4: Types of study design

Introduction

The information needed to identify the type of study design is normally found in the methodology (i.e. in the methods section of the paper).  It is good practice not to rely solely on the abstract when identifying study design as it rarely gives sufficient information for you to be sure that the description is accurate.  For example, the abstract may say the study was ‘a randomised controlled trial’ but you would need to read the methodology to see how the randomisation was achieved in order to confirm if this was indeed the case.  

The following questions will help you identify the different types of study design in the paper you are reading and the relevant critical appraisal checklist. You can work through the questions as they appear on this page, or download our algorithm.

You should work your way through the questions until you are satisfied with the answer and that you have identified a study design. If you reach the end and are still unsure please contact us at library@rcvsknowledge.org and we will try and help you.

Levels of evidence and study design

Although the levels of evidence are usually presented as a pyramid showing synthesised evidence at the top, it is also important to realise that higher levels of evidence are built on and supported by lower levels of evidence, so this information will be presented from the bottom up.

What types of clinical research are there?

The table in EBVM Toolkit 3 shows a broad categorisation of the different type of clinical research, arranged according to the level of evidence that they are likely to provide for clinical decision making.

The study types at the top of the table correspond to increasing quality and reliability of the evidence. The higher the level of evidence, the more confident you can be in the accuracy of the results with less the chance of error or bias.

View this table in EBVM Toolkit 3 – How study design relates to level of evidence.

Background information

Background information is generally considered low quality evidence for decision making. However, it can be useful for giving an overview of a subject and providing the knowledge you need to find and assess higher quality evidence.

In terms of the published veterinary literature the most common forms of background information are opinion pieces and narrative reviews. Textbooks and CPD notes may also fall in this category.

Examples of background information include:

• Opinion pieces: These are not based on a literature search. Instead, the authors give their opinions on a subject without any explicit appraisal of existing literature though they may provide a couple of references to back up their claims.
• Narrative reviews: These provide an overview of a subject.  They may include references, but they rarely include details of specific search protocols or explicit criteria by which papers are included or excluded. Instead, they rely on the author to draw conclusions based on the papers they find most relevant or interesting.
  
  While background information is generally considered to provide low quality evidence, its usefulness may be improved if it provides references to a range of appropriate sources, contains explicit discussion of the evidence on which it is based, and acknowledges areas of uncertainty and knowledge gaps.

Descriptive studies

Descriptive, or non-comparative studies, are designed to record what is seen. They provide a description of what has been seen in an individual or group of animals, but do not attempt any comparison to a control group. These studies have value if the aim of the paper is to highlight an unusual finding, such as a new disease, or report a rare occurrence.

Descriptive studies will not be able to demonstrate causation, so when using this type of study care should be taken to avoid over-interpreting the findings by making conclusions regarding causal links.

Examples of descriptive studies include:

• Case reports: These are reports on a single patient. They describe the presentation and/or course of a disease.
• Case series: These are collections of case reports and can provide descriptive quantitative data.

Observational studies

Observational studies are those where the researcher examines the outcomes of an intervention within two groups without having any influence on which animals get the intervention.

For example, a researcher could consider the rate of complication following different types of surgery by looking back at all the surgical cases and analysing those that resulted in complications. Where two groups are compared it is possible to make inferences about the association of risk factors and outcomes. However, because there are many possible confounding factors which have not been controlled it is not possible to prove causation.

Examples of observational studies include:

• Case-control studies: These are where animals which have a disease condition are identified and any causal or risk factors are compared to a control group (without the disease). Information regarding the exposure is historical These studies start with groups that already have the outcome (e.g. diabetes) and it looks back to examine what might have been the exposure factors (e.g. obesity).

Large data sets can be used to undertake nested case-control studies, where cases of a disease are identified and, for each, a specified number of matched controls is selected from among those in the data set that have not developed the disease, by the time of disease occurrence in the case.

• Cohort studies: These identify a group of animals and follow them over time to see how their exposures affect their outcomes compared to another group that were not exposed to that factor, either the general population or another cohort of animals.
  

• Cross-sectional studies: These are studies that describe the characteristics of sample groups of animals. Data is collected at one point in time and two groups are identified – usually animals with a specified disease and those without. The relationships within the groups to given parameters are then considered, and typically expressed as an odds ratio. As the data is taken at one point in time direction of case and effect cannot be established.

• Controlled before-and-after/interrupted time series: These are studies that measure the characteristics of a group of animals before and after an event or intervention. The two sets of data are then compared to judge the effect of the event or intervention.

Interventional studies

Interventional studies are those where there is an intervention (e.g. treatment, drug therapy, surgical method, exposure to a chemical etc) with a researcher responsible for designing the intervention and deciding which animals are exposed to it. By controlling the conditions in which the intervention takes place the researcher can control for potential confounding variables and increase the likelihood that differences between the groups are the result of the intervention.

• Randomised controlled trials (RCTs): These can either be experimental laboratory studies or clinical trials. RCTs have two important features:  there are at least two groups – a treatment group and a control group and animals are randomly assigned into the two groups.  Randomisation of participants and blinding of the investigator to which animals are in which group reduce bias, and lead to randomised controlled trials being considered the ‘gold standard’ when assessing the efficacy of a treatment.
• Non-randomised controlled studies: Not every intervention can, or should, be randomised. Non-randomised controlled trials can detect associations between an intervention and an outcome, but they cannot rule out the possibility that the association was caused by a third factor linked to both intervention and outcome.
• Cross-over trials: These involve the administration of two or more interventions one after the other in a specified or random order to the same group of patients. In these trials each participant acts as its own control, minimising the risk of confounding.
  

Synthesised evidence

These are studies which systematically review and critically evaluate already published literature. They may also be referred to as secondary sources of evidence, in contrast to the primary sources of evidence form published research studies.

When appraising secondary sources, it is still important to consider how reliable the evidence is, by looking for a clear description of how the evidence has been gathered and evaluated.  With secondary sources of evidence, it is particularly important to consider whether any important or relevant evidence has been left out, such as newly published primary research.

Sources of synthesised evidence include:

• Systematic reviews: These are comprehensive surveys of a topic in which all the primary studies of the evidence have been systematically identified, those providing the most relevant evidence have been selected and critically appraised, and then graded and summarised according to explicit and reproducible methodologies. A systematic review should be performed in a highly structured way, with the question and methods clearly defined in advance to minimise any bias in the selection of papers and grading of evidence.

• Meta analyses: These are secondary studies the in which the quantitative data from several similar research studies are combined and analysed as if they were a single study.  Analyses of this type are normally accompanied by some sort of graphical representation, for example a forest plot.
• Knowledge summaries/critically appraised topics: A knowledge summary, or critically appraised topic, is a concise appraisal of the best available evidence on a defined clinical question. They should follow an explicit methodology to identify and critically appraise the evidence.

• Evidence-based guidelines: Guidelines can provide an accessible way for practitioners to access recommendations for the care of patients with specific conditions. However, if they are to provide a high level of evidence and a reliable basis for clinical decision making it is important that they are based on a systematic and critical evaluation of the available evidence.

Social science studies

Social science studies seek to understand humans and their interactions. They may use quantitative methods, like those that have been described in the section on clinical research. However, they may also use qualitative methods which seek to understand the experiences of the human subjects involved.

Studies using social science methods may be relevant to the delivery of veterinary care or the experience of caring for animals with certain conditions. They may involve veterinary professionals or owners as participants.

Even if you are unfamiliar with the particular methodology being used it should be possible to place them within the broad categories of the levels of evidence.

You can view this ‘Levels of evidence’ in EBVM Toolkit 3: introduction to ‘Levels of evidence’ and study design.

Cost Benefit analyses

Cost benefit analyses in veterinary healthcare seek to compare the economic costs and effectiveness of an intervention with one or more alternatives. They may use actual data or computer modelling, but in either case, it is important to critically evaluate the way that costs and outcomes have been assessed.

Computer-based studies

Technological advances in computing, machine learning, and artificial intelligence are opening new approaches to research and the development of clinical decision aids. As these are new approaches to evidence generation we are still learning their strengths and limitations. However, the basic principles of critical evaluation and assigning levels of evidence (descriptive, observational, intervention studies and synthesis of evidence) can still be applied.

Large data sets

Large data sets can provide access to much larger samples than can be achieved with conventional research methods. However, the reliability of research based on these data sets will always be dependent on the quality of data available for analysis.

• Clinical data sets. Computerised clinical records and large datasets can provide access to very large samples for analysis. It should be remembered that as the data has been recorded for clinical rather than research purposes it may not include all the information that the researcher would want. Most of the research carried out on this sort of data will be descriptive or observational. However, new techniques such as “Trial Emulation” can be used to mimic clinical trials. In all cases it is important to carefully evaluate the selection criteria for cases and controls and consider the limitations of using retrospective data (Hernán et al. 2022).

• Genome Wide Association Studies (GWAS) are a type of observational study that screen for genetic variants within a population to look for associations with a particular trait or disease.

Generative AI

Thisis a type of artificial intelligence which can create new content, including text, images and audio. Although it is “trained” on a large data set it uses generative algorithms to generate new content which is based on, but not identical to, the data it was trained on.

Computer modelling

There are some areas of research where it is very difficult to control variables and interventions, for example in one off events or complex systems such as disease outbreaks. In these cases, computer models can be used to examine the effect of different variables on outcomes. Until recently the variables were selected by those carrying out the research, however tools based on artificial intelligence are now starting to be used in veterinary research. In all cases the accuracy of computer models will depend on the assumptions on which they are based.

Machine learning

Machine Learning is a type of artificial intelligence using algorithms and statistical models to enable computers to “learn” and make predictions without being explicitly programmed. The algorithms are “trained” on large data sets to identify patterns and associations and then use this information to make predictions about new data sets. Machine learning can be subdivided based on whether the data has already been labelled (supervised learning) or whether the machine is left to identify patterns in the data (unsupervised learning). Deep learning is a further type of machine learning which use neural networks, with multiple layers to analyse complex relationships and patterns in data (Hennesey et al. 2022; Pereira et al. 2023).

This type of research is now being carried out to develop tools which aid diagnosis, particular in diagnostic imaging.  It is important that these tools should be validated not only on separate data sets but in a range of clinical settings.

Further assistance

If you get to the end of the questions and are still unsure about the type of study design please email library@rcvsknowledge.org and we will try to help you identify the study design and find a checklist that will allow you to appraise the paper.