Tips for Driving the Appropriate Use of Learning Technologies through Practical ISD

Like many powerful tools, learning technology can help or hurt – it depends on how it is used. Getting others to use these technologies appropriately can be a challenge. Those of you around when eLearning was young were able to observed mismatches in learning technology to solution. In that case, eLearning often mismatched the needs and job context of the learner especially when the requirement included some form of behavior or skill acquisition that needed to be applied on the job. This problem is equally if not more prevalent today with new social media and mobile learning technologies available. Many are building wikis, writing blogs, and syndicating podcasts while grouping them under the Learning 2.0 umbrella, but does their content really support learning or a form of communication by providing access to information? If we are going to use or evangelize any learning technology, we need to get back to the basics – instructional systems design. If other learning professionals lack this basic competency, we need to help them.

Ideally, learning professionals have some training and background in instructional systems design (ISD), but we do not always see that in the real world. Even instructional design degree and certification programs tend to focus on the academic versus practice and lack preparedness for applicability when the graduate gets a job. Anecdotally, how many folks do you know with degrees in an area of expertise but lack the real-world experience required to apply it effectively? The old theory versus reality problem… At the same time, there is a basis for the application of ISD, and that’s the trick, applying ISD at practical and appropriate levels.

As learning technologists, practical ISD at a simplistic level tells us we need to help stakeholders and other learning professionals focus on alignment with business goals, business tools, and the work environment. Then we need to identify the needs of the learner by role and what must be accomplished on the job in the ideal world. We need to determine if a training or learning solution is required (it’s not when the problem is systemic or motivational), create a profile of the learner to include what they do in the real world, and map all that to the type of content that will be required (auditory, visual, kinesthetic, blended). Factor in budget, obstacles, and time to proficiency requirements and we can prescribe a set of learning technologies that meet those needs. As learning technologists, we need to push this approach.

If a learning technologist is not available, job-aides like tool matrices or decision work-flows can help stakeholders and development teams make decisions where learning technology is not a core competency, but later on, experts in the required technology need to be involved - especially in the analysis, design, and development phases. Learning technologists or experts will need to hold hands when their stakeholders are too far from their comfort zones.

If starting fresh or embarking on bleeding edge technologies, do some research to benchmark what’s been done successfully by others. When you have identified which technology will meet your needs, incubate and pilot. Start with small groups and the low hanging fruit during the initial test phases and then focus on the larger wins with larger groups as you proceed. Over communicate your wins but document your mistakes and don't forget the lessons learned - they drive efficiency and cost savings later on.

Lastly, ensure you communicate the purpose and value of the learning technologies up and down the food chain as this will drive adoption with your stakeholders and learners, along with prescriptive usage the learning professionals at your organization will need to drive.

Evaluation

“Knowledge is power…”
– Sir Francis Bacon – Founder of the scientific method

Why should anyone know how effective their training efforts were? Why should anyone take the time to evaluate learning initiatives? The answers are simple… So stakeholders can:

• Baseline current performance
• Assess what employees, partners, and customers are achieving
• Set attainable goals based on data
• Track their investment in learning
• Continuously improve what they do by asking the right questions

The most effective way to improve development and delivery of courseware is through Evaluation. The goals are to assess the value of the course to the stakeholder and to continuously improve that value based on valid metrics. Valid metrics include stakeholder feedback received during the development cycle while success measures should be based on the required skill-sets, performance criteria, and delivery modalities identified during the Analysis phase. As the required skill-sets and performance criteria should be translated into the learning objectives that drive Design and how the course should be Developed, evaluative elements built into the design and developed as part of the course should measure if those learning objectives and success measures are met, or not. It is important to note that levels of evaluation that are measured (or not) are often determined by stakeholder budget, resource availability, and statements of work. When not measured, this often attributed to fear of reprisal based on misuse and misinterpretation of measured outcomes, versus the drive to improve process and the overall learning solution. In the long-term, such misunderstanding detriments all by depriving the process of a feedback loop that adds value for the learner and saves margin or increases profit for the stakeholders. The four levels of training evaluation have evolved into six and are listed in order of robustness below:

Levels 0 through 5 (Kirkpatrick and Phillips):

Level 0 - Participation
Level 1 - Reaction
Level 2 - Learning
Level 3 - Application
Level 4 - Results
Level 5 - ROI

A description of each level will follow shortly...

Implementation (Delivery)

Courseware is typically deployed and delivered in the classroom, onsite, and or through computer and Web-based media during Implementation. Goals during this phase should include maximizing usability, satisfaction, transfer, behavior-change, and impact and should be therefore measured. Implementation requirements are identified during analysis based on learner profiles and are designed and developed based on technical and media specifications. Requirements for delivery through an instructor, content server, or Learning Management System (LMS) should be identified in advance during analysis so that content is designed and developed in alignment with delivery requirements, standards, and goals.

Typical delivery media include:

• Instructor-Led Training (ILT) > in a classroom, lab, or onsite
  
• Virtual Classroom > synchronous eLearning delivered via live Web connection, Web instructor-led learning
• Virtual Lab > live and synchronous virtual machine or a self-paced sandbox
• Self-Paced Training > asynchronous eLearning delivered as hosted or through CD or file on the local machine)
• Mobile Learning > mLearning for MP3/MP4 player, PPA, and smartphones
• On-the-Job Training (OJT)
• Job Aides > cards, posters, ePaper
• Collaborative and Social Learning > includes collaboration suites, coaching, mentoring discussion threads, learning Wikis, expert-produced blogs, and relevant RSS feeds
• Learning Management System (LMS) and supports
• Evaluation Systems > reaction/satisfaction, knowledge/skill transfer, application on the job, on the job performance improvement, return on the training investments, and predictability of future success
• Blended media delivery combining any or all of the above as indicated by the learner and media specifications analyses and design documents
  

Risks if not conducted properly:

• Extended deployment resulting in missed delivery dates and costs
• Project failure at final implementation resulting in lost and dissatisfied stakeholders and learners
  

ISD and the ADDIE Model

I have done a lot of talking about learning technologies, but I think now is a really good time to get down to basics, specifically, how any learning solution should be built, as in "instructional design." ADDIE is useful model of Instructional Systems Design (ISD) and is based on a 5-phased approach to courseware development... ADDIE is an acronym that stands for:

A - Analysis
D - Design
D - Development
I - Implementation
E - Evaluation

Execution of each phase is dependent on how a development team and stakeholders agree on the project's approach typically specified within a statement of work or project charter. As a system of checks and balances, each phase and milestone requires stakeholder approval to assure that what is delivered at project completion is what the stakeholder has approved from the beginning.

When no or little preexisting courseware exists, the most critical phase of a project is Analysis. Each phase follows sequentially with exception to Evaluation, which if conducted properly, occurs continuously throughout the process. Details regarding each phase will follow in subsequent blog entries. However, for a high level explanation of how this should work, see the graphic below.

It is fair to say that many in the business of course development skip phases within this model and some even ignore the model completely. This is a mistake, but please do not misinterpret that to mean that this process cannot be performed efficiently or within short time-frames. Those of us who work in internal training departments or on the customer business side need only to perform an analysis on a periodic basis for each department or customer-base we serve.

Also, variations in this model exist along with compressed methodologies and adaptations based on stakeholder needs. How this can be accomplished will be addressed in subsequent entries. Stay tuned for details on each phase of the ADDIE model.