Evaluating CTE

Not that kind of CTE

Breaking Bogart (14)

 Evaluating Career Technical Education

Hello all. Enjoy reading and please dive in if you have anything to add!

Career technical education is a unique beast in the world of andragogy. Automotive teaching even more so. With this uniqueness comes unique evaluation challenges, as well as some aspects which allow vivid comparison to Workplace Training and Leadership. Specifically, automotive CTE andragogy is especially focused on how knowledge and skill learned at the college transfers to a student's ability to be hired and their performance on the job afterwards. I will expound on this idea in this discussion.

Probably chews a pack of Red Hot every day.
Miroslav Vajdić

...

Firstly, I must discuss my confidence levels with different evaluation techniques. I will refer to the 4 main techniques of evaluation referred to within "Evaluation Basics" written by Donald V McCain. The four levels of evaluation are laid out as follows: Level 1: Audience Reaction to Training, Level 2: Learning and Application During Training, Level 3: Transfer to the Job and Environment, and Level 4: Impact of the Training and Return on Investment. McCain (2016). 
Level one and level two are the two techniques which I am most familiar with. That is to say that evaluating audience reaction and the application of training are the evaluation techniques which I am most confident in. I believe this is due to the nature of my teaching and training which heavily involves observing students in a laboratory environment which they can succeed and fail in hands on application of concepts. I may be a unique resource to my peers as I can provide both anecdotal and data points of evidence as to how evaluating training applications and learning is challenging in being accurate and fair, and how our department has seen improvements in those areas.

I am familiar with, but not confident in the third level of evaluation. I would like to improve by better understanding and applying the evaluation of this transfer of training to the job. This goal of skill transfer is one of the main, stated goals of our department. We strive to produce effective, skilled, and knowledgeable technicians. My teaching and how it molds my students to do so should reflect that. However, I am not so sure that I am where I would like to be on that.

Honestly, none of the evaluation techniques or issues in the reading were new to me. I both work in education, attend trainings and school, and have studied evaluation and HRD extensively in OLP 4401. Level 4 (Training Impact and Return on Investment) is discussed at my institution and was covered extensively in Human resource development: A concise introduction. Carbery, et al (2015).

High speed data transfer in a Cat-5 cable.
Darren Hester

...

Our department of automotive repair and transportation technologies stands fairly ready and competent for several forms of evaluation. I will discuss level 3: Transfer, thusly. Our department utilizes a technology called Handshake. This technology is both a hiring tool for industry partners, and a census tool for the college in order to see how many of our students are being hired, and by whom.  Data is therefore provided by this system. However, there are follow through issues in the proper utilization of the data for evaluation purposes. What I will say here is that due to various issues, the college has historically had an impactfully narrow data set. This data set then equals uncertainty in conclusions of training effectiveness. The systems, its utilization, and follow through in using it are improving. Through these scenarios, the following questions are answered concerning the "Organization's Evaluation Readiness." I will be looking at level 3:

Are specific individuals held accountable for program evaluations? [YES] (But I am unsure of the nature of it)
Is there appropriate data collection? [NO] (But it is being attempted).
Does instructional strategy support level 3 evaluation? [YES]
Do instruments for level 3 evaluation provide for on-the-job application assessment? [YES] (instructors visit industry partners to have on site surveys and questionnaires concerning the employee/former students' company standings and performance)
Evaluations are used for program improvement? [YES] (our program proudly exclaims how it adapts to industry needs. We have an entire industry advisory board who speaks to department chairs as well as instructors in large meetings to inform of industry wants/needs)

Automotive Service Excellence Logo

...

Lastly, as I reflect on my writings, I think of two stand out ideas which repeatedly hold strong truths in my life. The first is a phrase or concept taught to me by my father. He would tell me that when you are presenting information to people you should do the following:

1. Tell your audience what you are going to tell them.
2. Tell them what you have to say.
3. Tell them what you just told them.

I believe he was specifically referring to public speaking at the time (which this works fantastically within, I must add). However, I have found that this concept applies well to other forms of information delivery. So far McCain's text leverages this principal in an excellent manner.

I also keep noticing that blended learning appears to have diverse and effective applications and benefits for learners, teachers, and clients.

Thank you for reading this far.

Peace out!

-Vaughn.