The Use and Interpretation of Quasi-Experimental Studies in Medical Informatics PMC

Table Of Content

• Research Methods in Psychology – 2nd Canadian Edition
• QUASI-EXPERIMENTAL DESIGNS FOR PROSPECTIVE EVALUTION OF INTERVENTIONS
• Differences Between Quasi-Experiments And True Experiments
• STRATEGIES TO STRENGTHEN EXTERNAL VALIDITY
• Part 2: “quasi-experimental” designs used by health care evaluation researchers
• Workplace financial education and change in financial knowledge: A quasi-experimental approach (Horwitz et al.,

Quasi-experimental studies can use both preintervention and postintervention measurements as well as nonrandomly selected control groups. Nonetheless, there are design strategies for non-experimental studies that can be undertaken to improve the internal validity while not eliminating considerations of external validity. In comparison to simple pre-post designs in which the average outcome level is compared between the pre- and post-intervention periods, the key advantage of ITS designs is that they evaluate for intervention effect while accounting for pre-intervention trends. Such trends are common due to factors such as changes in the quality of care, data collection and recording, and population characteristics over time.

Research Methods in Psychology – 2nd Canadian Edition

Clustering can also arise implicitly, from naturally occurring hierarchies in the data set being analyzed, that reflect clusters that are intrinsically involved in the delivery of the intervention or comparator. Particular features are often chosen to address the logistical challenges of evaluating particular research questions and settings. Therefore, it is not possible to illustrate them with examples drawn from a single review as in part 1; instead, studies exemplifying each design are cited across a wide range of research questions and settings. This can present great difficulties when trying to classify studies, for example, to describe eligibility for inclusion in a review. Relying on the study design labels used by primary researchers themselves to describe their studies can lead to serious misclassifications. In the case of Bolsa Alimentação in Brazil, a computer error led eligible participants whose names contained nonstandard alphabetical characters to be excluded from the program.

QUASI-EXPERIMENTAL DESIGNS FOR PROSPECTIVE EVALUTION OF INTERVENTIONS

A quasi-experimental study (also known as a non-randomized pre-post intervention) is a research design in which the independent variable is manipulated, but participants are not randomly assigned to conditions. In conclusion, quasi-experimental designs offer valuable opportunities for researchers to investigate cause-and-effect relationships in diverse fields, ranging from healthcare to social sciences. Despite their limitations, such as the risk of confounding bias and lower internal validity compared to true experiments, quasi-experimental methods provide a practical and ethical approach to studying complex phenomena. In a pretest-posttest design, the dependent variable is measured once before the treatment is implemented and once after it is implemented.

Differences Between Quasi-Experiments And True Experiments

Other reviews including quasi-experimental studies have evaluated health insurance schemes [14], [15] and maternal and child health programs [16]. In this paper, we use studies from the reviews on the effects of CCT programs to illustrate the wide range of quasi-experimental methods used to quantify causal effects of the programs (Table 1). If the average posttest score is better than the average pretest score, then it makes sense to conclude that the treatment might be responsible for the improvement.

Quasi-experimental research designs find applications in various fields, ranging from education to public health and beyond. One significant advantage of quasi-experiments is their feasibility in real-world settings where randomization is not always possible or ethical. Natural experiments take advantage of naturally occurring events or circumstances that mimic the random assignment found in true experiments. Participants are exposed to different conditions in situations identified by researchers without any manipulation from them. Table 2 provides examples of studies using SWD that have used one or more of the design approaches described above to improve the internal validity of the study. In the study by Killam et al 2010 (31), a non-randomized SWD was used to evaluate a complex clinic-based intervention for integrating anti-retro viral (ART) treatment into routine antenatal care in Zambia for post-partum women.

Part 2: “quasi-experimental” designs used by health care evaluation researchers

Beyond Randomised Controlled Trials – expanding the horizon for experimental research techniques in the social ... - LSE Home

Beyond Randomised Controlled Trials – expanding the horizon for experimental research techniques in the social ....

Posted: Tue, 21 Jan 2020 08:00:00 GMT [source]

As we have outlined in this paper, a relative hierarchy and nomenclature of quasi-experimental study designs exist, with some designs being more likely than others to permit causal interpretations of observed associations. Strengths and limitations of a particular study design should be discussed when presenting data collected in the setting of a quasi-experimental study. Future medical informatics investigators should choose the strongest design that is feasible given the particular circumstances. There is a relative hierarchy within these categories of study designs, with category D studies being sounder than categories C, B, or A in terms of establishing causality.

The health care evaluation community has historically been much more difficult to win around to the potential value of nonrandomized studies to evaluate interventions. We think that the checklist helps to explain why, that is, because designs used in health care evaluation do not often control for unobservables when the study features are examined carefully. However, to the extent that studies may be possible with features that promote the credibility of causal inference, health care evaluation researchers may be missing an opportunity to provide high-quality evidence. For example, a manufacturing company might measure its workers’ productivity each week for a year. In an interrupted time series-design, a time series like this one is “interrupted” by a treatment. In one classic example, the treatment was the reduction of the work shifts in a factory from 10 hours to 8 hours (Cook & Campbell, 1979)[5].

Ethical Reasons

Quasi-experimental study designs, often described as nonrandomized, pre-post intervention studies, are common in the medical informatics literature. Yet little has been written about the benefits and limitations of the quasi-experimental approach as applied to informatics studies. This paper outlines a relative hierarchy and nomenclature of quasi-experimental study designs that is applicable to medical informatics intervention studies. They hope that future medical informatics studies will implement higher level quasi-experimental study designs that yield more convincing evidence for causal links between medical informatics interventions and outcomes. Randomized controlled trials (RCTs) in which individuals are assigned to intervention or control (standard-of-care or placebo) arms are considered the gold standard for assessing causality and as such are a first choice for most intervention research. Random allocation minimizes selection bias and maximizes the likelihood that measured and unmeasured confounding variables are distributed equally, enabling any difference in outcomes between intervention and control arms to be attributed to the intervention under study.

Effectiveness of implementing of an infection control link nurse program to improve compliance with standard ... - BMC Medical Education

Effectiveness of implementing of an infection control link nurse program to improve compliance with standard ....

Posted: Wed, 19 Apr 2023 07:00:00 GMT [source]

Doctors, on the other hand, are unlikely to agree with this genuine experiment since they believe it is unethical to treat one group while leaving another untreated. A hypothetical quasi-experimental study was conducted in an economically developing country in a mid-sized city. For instance, it's impractical to trawl through large sample sizes of participants without using a particular attribute to guide your data collection.

Researchers prefer to apply quasi-experimental design when there are ethical or practical concerns. Researchers can use this method to determine whether a particular intervention has had an impact on the target population over time. Questions 2–4 are also new, replacing the first question (“Was there a relevant comparison?”) in the original checklist [1], [2]. These questions are designed to tease apart the nature of the research question and the basis for inferring causality. Question 1 in the checklist distinguishes individual allocation, cluster allocation (explicit clustering), and clustering due to the organizational hierarchy involved in the delivery of the interventions being compared (implicit clustering). Users should respond factually, that is, with respect to the presence of clustering, without making a judgment about the likely importance of clustering (degree of dependence between observations within clusters).

Patients undergoing the new procedure are compared with similar patients who received conventional treatment in the past. Although not randomly assigned, comparing outcomes between the two groups provides valuable insights into the procedure’s effectiveness. Clustering occurs when implementation of an intervention is explicitly at the level of a cluster/organizational unit (as in a cluster-randomized controlled trial, in which each cluster is explicitly allocated to control or intervention).

If the new curriculum was so tough that many of the slowest readers dropped out of school, the school with the new curriculum would experience an increase in the average reading scores. The reason they experienced an increase in reading scores, however, is because the worst readers left the school, not because the new curriculum improved students' reading skills. Recall that when participants in a between-subjects experiment are randomly assigned to conditions, the resulting groups are likely to be quite similar. When participants are not randomly assigned to conditions, however, the resulting groups are likely to be dissimilar in some ways. A nonequivalent groups design, then, is a between-subjects design in which participants have not been randomly assigned to conditions. A true experiment would, for example, randomly assign children to a scholarship, in order to control for all other variables.

The prefix quasi means “resembling.” Thus quasi-experimental research is research that resembles experimental research but is not true experimental research. Although the independent variable is manipulated, participants are not randomly assigned to conditions or orders of conditions (Cook & Campbell, 1979)[1]. Because the independent variable is manipulated before the dependent variable is measured, quasi-experimental research eliminates the directionality problem.