In Conversation With Katie Graham: A State Leader’s Thoughts on the Need for Outcome Data in CTE

In Conversation With Katie Graham: A State Leader’s Thoughts on the Need for Outcome Data in CTE

Career and technical education (CTE) provides a powerful model for equipping students with relevant knowledge and skills for a 21st century global workforce. CTE programs challenge students to take ownership of their education and offer unique opportunities for hands-on learning and for contextualizing academic subjects to build relevance. Research on the specific outcomes of CTE programs remains limited, however, posing a challenge for educators’ efforts to promote CTE and its benefits.

Alisha Hyslop, director of public policy at the Association for Career and Technical Education (ACTE), sat down with Katie Graham, state director for CTE in Nebraska, to talk about the state of CTE research, what it offers and lacks, and Nebraska’s efforts to leverage the research available.

Hyslop: Can you tell me about your role in Nebraska?

Graham: I am the state director for career and technical education with the Nebraska Department of Education, which is really exciting! Before that I was the data, research, and evaluation specialist for CTE for about a year. Then, I was the assistant director, and later I was promoted to director. I've been at the department about 4 years, and I oversee everything from the Perkins Grant to professional development, and a staff of 21. I work heavily within the agency, across different divisions, and on tasks outside of CTE as well.

Hyslop: What are some key ways you use CTE research in that role?

Graham: It can be hard because there's not a lot of research out there, and the research that is out there is often anecdotal in nature. One way we use this information is to determine the best ways to communicate the message about CTE. For instance, we can use anecdotal data to pinpoint emerging areas of interest or success stories that later help direct our statewide efforts and communication about them. There are also data—again, anecdotal and maybe not empirical research studies—that show some trends in CTE. In Nebraska, we use a lot of the trend data to highlight best practices or positive outcomes from around the country. We use any existing research to talk about promising practices.

Career academies are a great example. It's a phenomenal model with a strong national research base, and we know a lot of good things are happening within career academies in Nebraska. But there isn't a lot of research to validate that here and to say, “Because these students participated in this specific experience, they had these outcomes.” So we use the research we do have to, again, give options and highlight best practices.

The other way we use data is in an evaluative function: finding ways to compare what we're doing to others or to these other studies, and analyzing our own practices.

Hyslop: Are there specific questions about CTE programs in your state that you cannot answer because you lack evidence?

Graham: The big question everybody wants to ask is, “What leg up do students have if they participate in CTE?” And businesses also want to know the implications for workforce development. How does CTE contribute to an employer’s bottom line and strengthen the pipeline of skilled workers?

Causal research would be great but, as with most things in education, that's hard. So even more correlational research would be really helpful. One thing that I'm very personally interested in is demonstrating how the nature of CTE programming presents an ideal setting for the contextualized core academic courses. We keep saying it; it's in the [Perkins] law. We know we need to do it, but there's not a lot of research showing how to do it and what the outcomes could be. I think that the affordances CTE provides are so over and above what a traditional classroom setting offers. And we know this, but we need the outcome data.

Participation in CTSOs [Career and Technical Student Organizations] is another area in which I'd love to be able to show some hard data. These student organizations are not clubs; there's not a lot out there to replicate the true cocurricular nature in which classroom knowledge connects with real-world learning. Students compete at a national level to demonstrate their skills. That's such a unique thing to CTE, but even the individual CTSOs don't have enough research on the impact they can have on student outcomes and community outcomes and national outcomes.

It’s about more than the opportunities students have. It's, collectively, what all those opportunities contribute.

Hyslop: What would be the benefit to CTE if we had more causal evidence, more outcome data?

Graham: First and foremost would be awareness. We could answer the question, for students and their parents, “What is this?” Being able to show the benefit—that these experiences lead to these potential outcomes—would be hugely beneficial. I truly believe a lot of the anecdotal information that shows students benefit from the experiences they have in CTE could be validated if we had more research and outcome data.

Most teachers don't know. I mean, the CTE teachers know. But, even in Nebraska where I like to think CTE is pretty strong and we're pretty loud and proud, it’s unlikely that non-CTE teachers could tell you what career and technical education is or how their work aligns with it. Outcome data could counter that.

Hyslop: You've already done a lot in Nebraska with data to share the CTE story. Can you tell us a little about the work you are doing?

Graham: I think our work is just beginning because we're limited by the data we have available. All we can really do right now is review the accountability data that are required federally. Strategically, we have worked in Nebraska to pull in other data points, not just [those] related to graduation rates or only academic attainment on standardized tests. We're looking at things like honors courses and AP [Advanced Placement] courses and, you know, trying to bust the myth that CTE students are those “other kids.”

We try to be very intentional about telling the story—that bigger picture of when students start and the experiences they have. Even in some of our data pieces, we've tried to display the information in a compelling way, using the data to support the story rather than just throwing the data out there.

As for research, we currently have a couple of different things going on. Within a Regional Educational Laboratory program, Nebraska is working with other states to identify what CTE data are commonly available and to determine which predictor and outcome variables may be worth exploring further. And we're doing something similar in-house with one of our psychometricians at the department. But, again, it's kind of half data mining, maybe half research agenda. The biggest barrier, as I talk with colleagues, is still access.

Some of the data points that would help us really show impact aren't comparable across states, and we know they won't be. Within Nebraska, we have data-sharing agreements between the 4-year and the 2- year [institutions of higher education] and all of secondary, even some limited workforce, but there's no agreement on how we're going to use the data. So we collect it; it's all there. But we can't really dig in. And I know the states that have been able to analyze data have really been able to show some incredible things.

If we were able to access data at that big level, we'd be able to focus on a smaller level, to show what's needed in every area or region, or what some best practices are for specific subsets of students or for rural versus urban. You know, there are so many different ways to approach it.

Learn more about ACTE, one of several partners leading the CTE Research Network.

Learn more about the Nebraska Career Education program at the Nebraska Department of Education.