The Career and Technical Education (CTE) Research Network seeks to expand the evidence base on the impact of CTE programs on student outcomes and is working to encourage more rigorous research—studies designed to show a causal impact—to meet that goal. This blog post aims to demystify causal impact research and explain its value to CTE practitioners.

There are three broadly defined types of research design:

• Descriptive research, which describes the current state of a program or school.
• Correlational research, which determines whether there is a relationship between two or more variables, such as whether a student’s gender is associated with a score in a career and technical student organization competition.
• Causal research, which determines whether one variable caused, or did not cause, a change in another variable, such as whether enrollment in a career academy influences the likelihood of graduating from high school.

Designed to show cause and effect, causal research identifies whether and how CTE programming changes students’ education and workforce outcomes. In doing so, it informs educators, policymakers, and the public about the effectiveness of CTE programs. It also brings researchers and practitioners together, especially during the research’s implementation phase.

Randomized controlled trials (RCTs) are considered among researchers as key to establishing cause-effect relationships, reducing inherent student selection biases, and producing the greatest confidence in a study’s findings. In an RCT, students who are similar to one another are randomly assigned to either an experimental group that participates in a program, or a control group that does not. Each group is tested before and after to measure their respective outcomes.

What does an RCT look like on the ground? You may have a CTE program with limited seats and more applicants than you can accommodate. Instead of picking the students that you think will benefit the most from the program, you randomly choose from among the interested learners. Then you test both groups before and after to measure their progress. If the experimental group shows an increase in technical knowledge and skills after the program, and the control group does not, then you have demonstrated the positive impacts of the program. More than any other research method, RCTs are able to show cause and effect by eliminating the likelihood that learner outcomes can be attributed to variables other than the CTE program itself, such as student interest or socioeconomic status.

Quasi-experimental designs have become very common in education research. These studies establish cause and effect by using naturally formed experimental and control groups. For instance, a current CTE Network quasi-experimental study is comparing the outcomes for students who scored just above a cut-off to get into a CTE high school to the outcomes for students who scored just below. The individuals in these two groups were not randomly assigned, but they are assumed to be similar since their scores are very close and all of them chose to apply to the CTE high school. Researchers conduct careful analyses to identify any differences between the two groups and use statistical methods to control for those differences.

Educators are often worried about causal research denying services to learners. Yet in education, causal studies are most often used in situations where some students will inevitably be unable to participate because there is more interest in the program than can be accommodated. In these situations, randomization helps guard against implicit biases that can creep in during the application or student selection process.

Practitioners play an important role in developing causal studies by providing researchers with background and expertise grounded in their experience working with learners. And, practitioners can benefit from causal research, particularly during the implementation phase. For instance, research showing student outcomes from CTE dual enrollment courses can help CTE educators determine how much to invest in implementing or expanding a dual enrollment program, identify relevant student populations that can benefit from the dual enrollment program, and reproduce impacts identified in the study findings.

Causal research provides the data to show if a program is working and helps CTE practitioners determine where to invest resources. Researchers need the boots-on-the-ground insights from CTE administrators and instructors to help understand where there are gaps in the evidence base and inform new and innovative CTE programming.

This article is based on the webinar, Demystifying “Causal” Impact” Research and Understanding Why It Is Important to CTE, featuring the following experts:

• Michelle Conrad, Ph.D., Associate Professor & Program Coordinator, CTE Graduate Programs, University of Central Missouri (moderator)
• Shaun M. Dougherty, Ed.D., Co-Principal Investigator, CTERN, Associate Professor of Public Policy & Education, Department of Leadership, Policy, and Organizations, Vanderbilt University
• D. Crystal Byndloss, Ph.D., Senior Associate, K12 Education Policy Area and Director, Outreach, Diversity & Inclusion, MDRC
• Julie Edmunds, Ph.D., Program Director, SERVE Center at University of North Carolina at Greensboro