Section 3 Evaluating and managing instructional programs and projects

Chapter 10 discusses evaluation in instructional design and provides you with two evaluation models, the CIPP and Kirkpatrick models for evaluation. Search for at least two other models used for evaluation and summarize these models. Describe how you would use them to evaluate your instruction. Reflect on what other questions that instructional design evaluation should address besides whether the instructional design leads to comparable amounts of learning and learner satisfaction as traditional methods. What else would be useful to know? 

In my search for other models for evaluation, I discovered the ADDIE model.  The ADDIE model consists of 5 steps – analysis, design, developmental, implementation, and evaluation.  In the original model, each phase must be completed before moving on to the next phase.  This idea has been modified over time.

In the analysis phase, who the learners are and the knowledge and skills that they have are identified.  The learning environment is also assessed as to what is available and what is needed.  A timeline is established based on these findings.  The next phase is the design phase, which establishes the learning objectives, assessment instruments, activities to be used, what media will be used, and lesson plans for the project are developed.  In the developmental phase, the learning objectives are actually developed into lessons, storyboards and graphics are created, materials and procedures are tested and revised if necessary, and if e-learning is involved, the development and integration of technology occurs.  The implementation phase involves the development of procedures for training the facilitators as well as the learners involved.  It also involves evaluating the design and ensuring that all the materials that are needed are in place and functioning properly.  The evaluation phase involves both formative assessments that will occur throughout every level of the program but also a summative assessment at the end of the program.

The other model that I discovered was Kaufman’s 5 Levels of Evaluation, also known as Kirkpatric Plus).  This model was developed as a modification to Kirkpatric’s model of evaluation.  In the first stage, resources and processes are analyzed to find out what is available and its quality.  Also the quality and effectiveness of processes being used is analyzed not only for their effectiveness for the organizational factors but also for how satisfied are the learners with the processes being used.  In the next stage, acquisition, the focus is on the small group and individual benefits of the project.  Also, whether learning objectives are being met and desired outcomes of the learning interventions are effective is being assessed.  In the application stage, whether the learners are using their new skills in new ways is assessed.  Organization payoffs stage involves looking at the overall results and deciding if the learners met the desired learning goals/objectives.  In the fifth stage, societal contributions, how the new material learned relates to the client and/or the needs of society are assessed.

I feel that instructional design evaluation should also address how relevant the instructional design is to needs of the learner.  If the learner sees little to no relevance to how it will help them or why they should know the information being presented then they will not be as successful because it will be of little importance to them.  Therefore the more relevant the design to the learner, I feel the more effective it will be for the learner. 

I would like to know how learners continue to use the information they learned in other ways.  This information would be a great asset to revise the original design so that it would better benefit the learner in their future. In many of the trainings that I have been to in my 14 years as an educator, I have only been asked to evaluate the training immediately following the training.  In many cases, I was introduced to some very creative ideas that would have been very useful in my classroom but unfortunately enough time was not spent on the topic for me to be comfortable enough to take it back to my classroom to use with my students.  If the trainer had done a follow up evaluation several weeks or a month later, they would have known that what they had presented was interesting to the learners but that the learners were having difficulty implementing it into their classrooms.  I think follow up evaluations would be a great asset to any training for both the learner and the trainer.

Chapters 12 & 13 focus on project management and how to manage projects when resources are scarce. You have been assigned to develop a series of professional development sessions focusing on technology use in the classroom for teachers during a time of economic decline. How will you use Situational Leadership to facilitate this project and manage scarce resources? 

I would first survey the educators about what kind of technology they currently use, their comfort level with that technology, and what technology they want to learn more about.  I would ask about some specific technology resources such as Edmodo, power point presentations, prezi presentations, and website design.  I would also survey the campus administrators to find out exactly what kinds of technology resources are available on each campus. 

From this survey, I would meet with educators who felt comfortable with specific technology resources that are considered of high need and train them to become presenters to those who are lacking in skills with those resources identified.  Then provide after school or Saturday trainings for those teachers who need to learn new resources to attend at campuses that have sufficient resources to accommodate the resources being taught.  If necessary, more than one date could be available or training could be ongoing for several weeks so that problems or questions that arise could be addressed and answered more quickly.  In order to see if this is effective, I would ask for frequent feedback from both the presenters and the learners.  As an incentive to participate in this training, I would provide both the presenters and the learners to count the time that they are giving up after school hours to count towards any professional development comp time requirements that they would have to have.