Posted by: Carol | April 26, 2011

CTE & Data: Why You Can’t Have One Without the Other

It seems like everyone in education is under fire these days.  CTE’s been in the hot seat before, so this is nothing new to us – and while some view Arne Duncan’s  recent address to the National Association of State Directors of Career Technical Education Consortium and the U.S. Office of Vocational and Adult Education as a sort of warning shot over the bow, it seems that overall, CTE has been receiving some favorable attention. However, in consideration of the Secretary of Education’s comments, Tom touched on a topic that’s long been a pebble in our collective shoe: the lack of systemic, long-term data for CTE programs and activities, particularly in the state of California.

While the activity and content in this blog does not fall under the purview of any evaluation that we’re working on, it may occasionally touch on the topic of evaluation and what that means to the CTE field. This is going to be one of those posts. So please excuse me for a moment, while I put on my “evaluator” cap, and make a case to the CTE community – particularly that which exists in California.

As many of you know, for the past 3 years, WestEd has worked on the statewide evaluation of the CTE Pathways Initiatives. We’ve spent many, many hours traveling across the state, visiting your sites, observing your events, talking to you, your partners, and your students, and poring over your reports. We’re convinced. What you do is really important, not to mention really fun and engaging too! The greater challenge for us, however, is that we’ve been unable to collect any really solid outcomes data.

You may be asking, “what about all those reports we complete and send in? Isn’t that data?”

Yes, that’s data. And it’s okay data. It’s process data – the data that tracks things like how many students were enrolled in programs, how many students attended an event, how many hours you spent working on any given task, how many industry partners were involved in any particular project, what sort of systems were created to facilitate communication between groups, etc. Unfortunately, none of these data actually answers the question: “And then what?”

The Questions

What does that mean, you ask? It means that when all is said and done, Arne wants to know what happened after. After the investment of funds, after the implementation of the programs, after students participate in and complete a program, what happens? Do they get a job?

To be sure, not all CTE programs are about sending students into the workplace upon completion. In fact, more and more of these programs – particularly the ones targeted at younger students – are just as much about giving students exposure to careers and the workplace, than providing them with concrete, employable skills (though the two often go hand in hand).  But regardless, we need to know what happens to these students – your students – after the program ends. Did they go on to 2 or 4 year colleges? Did they complete and acquire industry-approved and accepted certifications? Did they enroll in more courses towards earning their certificates or degrees? Did they get a job doing what they possibility spent years learning about and training for? And if not, what did they do instead?

And answering these questions – that’s our great challenge. Ours as program evaluators, but yours as well, because without the answers, funding for these valuable services could very well dry up and disappear. We have a lot of anecdotal, qualitative data about the value of these programs. Sit down with any given group of students, staff, or partners, and you’ll hear all about the positive impacts of CTE projects. However, as with anything else, in order to be considered good data, qualitative data needs to be supported with quantitative data – the hard numbers.

The Challenges

The largest obstacle to being able to collect the data we need is the lack of matching ID numbers between California’s K-12 school systems, community colleges, and universities. For reasons that defy our efforts to explain, students seem to spend 13 years in the K-12 school system with one ID number. Then, when they move on to a community college or a university, that number is discarded, and they’re assigned something completely new and different. What does that mean? Well, in a nutshell, that we can’t easily track students from one system to the others. Currently, there’s no simple way for us to know whether a student has moved on from K-12 to higher ed  – to get the information we need, we’d have to delve deeply  into other, trickier numbers and records which we’d like to stay clear of (SSNs, birthdays, addresses, etc.) or expend an inordinate amount of time and money trying to physically track down each student and asking them, “Did you decide to continue your education? If so, how?” And even trickier than tracking the leap from K-12 to higher ed, is tracking students into employment – how do we find out where, when, and how they found jobs, if they don’t tell us themselves?

By the way – we’ve heard rumors of community colleges that use K-12 student IDs. If you belong to one of these institutions, EMAIL ME, please! Or at least leave a comment and tell us who you are! We would love to speak with you.

We’ve wracked our brains trying to come up with work-arounds to these issues, and so far, we’ve only come up with one that seems relatively viable: using pre-existing social networks – specifically, LinkedIn, which is particularly appropriate due to its career-focus – to track students as they move through various educational systems and into the workplace. If CTE professionals were willing to help us coordinate it, students could be asked via their courses to register on LinkedIn (if they didn’t already have an account), and update their profiles once or twice a year at minimum. The Pros: Maintaining a LinkedIn account would be good for students as they’re a great way for individuals to maintain a professional presence online. We would be able to gather pertinent data easily and with minimal cost and intrusion. The Cons: There are privacy issues concerning students who are minors – we are totally aware of the pitfalls of asking students under 18 to post personal information online, and we’d like to avoid that if possible. Also, it’d require students to actually remember to keep their accounts updated consistently and accurately.

What do you think? Would you participate in something like the LinkedIn idea to help us collect the data we all need? Or better yet – do you have an idea we haven’t considered? Tell us, in the comments.


Responses

  1. I think it’s a great idea! I am working with our campus IO to create a LinkedIn group and I believe some of the faculty members are using Google groups as well.

  2. I thought all of our contributions to CalPass and those of our secondary partners were supposed to take care of the problem of being able to consistently track students’ movements from secondary into postsecondary. No? And if no why were we all required to go to great lengths to make sure we all joined and participated in the CalPass system.

  3. I participate in Linkedin and definitely this is worth trying. Also wonder if there is a possibility of utilizing an online “monkey” type sort – quick survey to collect info from high school students, especially in articulated classes? This might give us some indication of their intent and encourage them to participate in Linkedin.
    High schools in our area do not have this follow up information available for students, at least most of them, so it is a nearly impossible task.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: