Development

Courseware is built during the Development phase, where the solution is created based on the specifications of the approved design architecture, and within the required timeframes identified at planning. The training product should be built in delivery media based on the learners’ profile, required testing instrumentation, hands-on images (real, virtual, or simulated), augmentative instructional aides, and pre-deployment technologies for blended media where appropriate. The training product and content should be audited, quality assured, and piloted (Alpha and Beta) for content validity (by SMEs), technical compliance (image, SCORM, AICC, etc.), learner utility, and publication-standards purposes. Beta pilots must simulate the delivery environment to minimize issues. Subsequent revision is typically required to meet stakeholder needs identified during analysis and therefore should be accounted for in project schedules. Curriculum owner involvement in testing, sign-off, and approval is required for each development milestone.

Based on statements of work and or a project charter, development formats include:

- Instructor-Led Training (ILT - classroom-based)
- Virtual Classroom (synchronous eLearning delivered via live Web connection)
- Self-Paced Training (asynchronous eLearning delivered as hosted or through CD or file on the local machine) (SPT, WBT, CBT, WILR, SPS)
- Mobile Learning (mLearning delivered to MP3/MP4 players, PDA, mobile phone, and smartphones like BlackBerry and iPhone)
- On-the-Job Training (OJT)
- Job Aides
- Collaborative and Social Learning 2.O tools (includes collaboration suites, coaching, mentoring discussion threads, learning Wikis, expert-produced blogs, and relevant RSS feeds)
- Learning Management System (LMS) supports
- Evaluation Systems (reaction/satisfaction, knowledge/skill transfer, application on the job, on the job performance improvement, return on the training investments)
- Blended media delivery combining any or all of the above as indicated by the learner and media specifications analyses and design documents

Risks if not conducted properly:

- A product that is not properly tested before final implementation to the customer and or their learners
- A product that is not fine-tuned to the needs of target learning groups
- Discovery of problems or technical difficulty with the product at implementation resulting in high issue counts as opposed to identification and resolution before delivery
- Project failure or extended development cycles resulting in missed milestones, deadlines, and cost-targets- A course that does not meet design specifications falls short of stakeholder expectations, and results in learner dissatisfaction

More on Design - PI

It's important to address the science behind instructional design. For reasons unknown, programmed instruction (PI) in eLearning seems to be all but abandoned in much of the learning content I review. In my opinion, this is a flaw in instructional design and worse, is only supported by popular eLearning development tools, because they omit this capability.

Gagne actually pioneered this mode of instruction in the mid 1960s. For those unfamiliar, programmed instruction models assess a learner's needs through some form of testing and loop back, branch-forward, or multi-path based on performance to pass/fail criteria. More robust PI models incorporate both pretesting an posttesting. The biggest advantage to this design model is that delivery of content is customized to a learner's needs because only filtered content is delivered (based on what the learner did not pass). In essence, a form of needs assessment is built into course delivery and the user experience. How cool is that?

Development tools like Macromedia Authorware or CA KnowledgeTRACK were great at facilitating this mode of design. Unfortunately these tools are no longer available, (in all honesty - they were cumbersome to use.) Meanwhile, social and Learning 2.0 suites like Q2 Learning's eCampus currently bake PI functionality into a learning path (Check it out at: http://www.q2learning.com/). None-the-less, this is a methodology we should revive. Take another look at the process illustrated above and decide for yourself.

In the Middle of the Curve: CA Case Study - Informal Learning

In the Middle of the Curve: CA Case Study - Informal Learning

Design

We specify how to build a course during the Design stage. When a house is to be built by a construction crew, an architect creates a blueprint that specifies the layout and measurement of materials for the contractors and construction crews. In similar fashion, instructional designers frame-out the architecture and content flow of the courseware solution in a set of design documents and or storyboards. A robust design leverages the data gathered during the analysis stage, which results in a record that outlines the learning objectives and instructional supports through appropriate delivery media. A valid design must be derived from accurate analysis data in order to develop learning objectives that can predict success through performance on the job. Learning objects based on the task-KSA analysis and support components should be designed as modular so they can be easily grouped, sequenced, updated, and reused. If it is practical, design document templates should be as media-agnostic, so they can be efficiently converted to any blended media with or without an LCMS (Learning Content Management System). At the curriculum level and based on learner analysis data, blended learning paths can be designed for any combination of instructor-led (classroom based), Web instructor-led (virtual class), or self-paced media (Web-based or computer-based). Based on media and technical specifications, instructional supports such as graphics, animations, video, audio, simulations, classroom-based images, virtual labs, pretest, and posttest objects, or any required interactive elements should be indicated. Blended designs and learning paths can also include job-aides, on-the-job training (OJT), coaching, mentoring, and collaboration through social media. Design documents should be submitted to both subject matter experts (SMEs) and stakeholders for validation and approval prior to development.

Design specifications include:
- Development of learning objectives that map to on the job requirements
- Section titles and topologies that indicate breakdown and sequence by module, lesson, and topic, which is also known as MLT
- Specification of instructional supports
- Targeting of critical learning objects for test, interactive, and certification development
- Specification of learning group differences and roles where applicable (multi-user design)
- Use of interactive exercises and practicums that map to job tasks
- Media types
- Timing estimates

Practical design structure at the module or lesson levels:
- Tell'em
> Learning objectives
> Explain what the learner will learn and why
- Show'em
> Demonstrate the task or concept
- Walk'em through it
> Guide and coach through the task
> Provide feedback as prompting
- Have'em try it
> Provide the learner with an opportunity to experience task with minimum feedback
- Have'em apply it
> Provide a real-world problem scenario where the learner can use the new skill
> Test the learner in a situation he or she will likely encounter on the job
- Tell'em again
> Reiterate the learning objective and explain what was accomplished

The structure above can be modified by using the parts that are appropriate. What is important is the progression from what is explained or demonstrated - to the transfer of learning.

Risks if not conducted or conducted improperly:
- Misdirected development efforts due to a lack-of or inaccurate architectural specifications
- (Think of trying to get from point A to point B in unfamiliar territory without a map)
- Extended development timelines and cost due to design error and misdirection
- A course that does not meet the required needs of the participants resulting in project failure and a dissatisfied stakeholder

Analysis and Planning: Technical and Media Specifications Analysis

A technical and media specifications analysis should identify the technical, graphical, and media-based requirements and constraints to assure eLearning delivery is compatible with the hardware and software requirements of the stakeholder’s learning base. Graphic look, feel, and specifications should be identified. Requirements should be mapped into courseware development. Summaries for technical and media specifications should be submitted. Data collected at this stage should:

• Identify infrastructure capabilities for courseware delivery
• Identify requirements for optimal delivery media
• Identify stakeholder standards for text, graphics, media, and branding

Risks if not conducted or conducted improperly:

• Graphics and media that do not meet the standards and satisfaction of the stakeholder
• Incorrect delivery formats
• Product failure due to incompatibility of delivery medium and/or components to the hardware and software requirements of the stakeholder and/or the stakeholder’s customers
• Missed deadlines and extended project plans that do not meet stakeholder expectations

In the end, all analysis data (organizational, task-KSA, learner, and technical/media specifications) should be used to design a learning solution that:

• Is based on the needs of the organization/department
• Addresses the gap between the required task-KSAs and what’s actually done by the learner on the job
• Is delivered using the appropriate mediums and technologies that map to learner profiles

Enough with Analysis. We move to the Design phase next.

National Learning Week

We will get back to ISD, the ADDIE model, and analysis in my next post, but this is a good time for a break because it’s ASTD National Learning Week. The local ASTD chapter I serve sponsored and prepared for this by asking members to provide some learning tid-bits that fit within an acronym we defined for L-E-A-R-N:

L - Learning

E - Evaluation or eLearning

A - Assessment or Activities

R - Reinforcement or ROI

N - Networking

They are posting and distributing responses out to members with one letter of the acronym for each day of the week. Here is my submission:

L:
At some level, we are all in the business of Learning, but what do we mean when we refer to learning, behavior, or training? The following definitions can be useful from an operational standpoint... Let’s start with our subject - the learning-participant or learner. The assumption is that learners behave in a certain way until they learn to behave in another, through some sort of experience. Let’s define behavior as something that a participant does that can be observed and measured. If so, then we can agree that any measurable change in behavior is some sort of learning. That being said, the training experience needs to be the purposeful cause of that measurable change in behavior so that what is learned is consistent and applicable.

E:
Some Evaluation questions to ask your self after learning events or better yet - to prepare for during course development: Do they like it? Do they know more now than they did before training? Can they do more now than they could before training? Are work-performance outcomes better now than they were before training? Are departmental or business metrics better now than they were before training? How am I ruling out confounding variables (reasons “other than training” for your evaluation data)?…

A:
What to analyze when Assessing the need for training: Define the sponsoring organization’s/department’s mission, goals, vision, culture, obstacles to performance and budget for training. Identify and what learners need to do in “an ideal world” for their job-roles in terms of tasks, knowledge, skills, and abilities. Validate what learners are actually doing in “the real world” for their job-roles in terms of tasks, knowledge, skills, and abilities. Identify the capabilities and limitations of your learners in terms of education-level, experience, and available technologies. Use assessment results to design and develop content that focuses on the gap between the ideal and real world requirements while accounting for organizational needs and learner capabilities.

R:
When designing and developing content, apply Reinforcement principles to learning events as ABCs:

A = Antecedents – or what happens before any change in behavior, in our case a learning event that is aligned with the learner’s needs and is effectively delivered before the she meets the learning objective.
B = Behavior – is what the learner does; ideally, meeting the learning objective.
C = Consequences – these are are anything that happens after behavior that can either reinforce (increase the likelihood) or punish (decrease the likelihood) that what was learned will be applied on the job. Build and deliver activities that reinforce: Tell’em > Show’em > Let’em Try It (guided) > Let’em Apply It (unguided) > and Test’em. Also, use reminders, job aides, feedback, and supervisor support to reinforce behavior back on the job.

N:
Use social Networking and Web 2.0 tools to learn: Blog to broadcast subject matter expertise; micro-blog to keep your team abreast of newly learned knowledge and skills; write wikis to share and update reference-able knowledge-bases between team members; video short info-bytes or demonstrations from vid-cam at your desktop for your corporate YouTube. Don't have a budget? Get permission to use free tools like Blogger, WordPress, PBwiki, WetPaint, Twitter, Google Docs and Reader.

Happy National Learning Week!

Analysis and Planning: Learner Analysis

The Learner Analysis identifies what is actually done on the job and required in the real world. A learner analysis (also known as a Person Analysis) should identify the range of learner needs based on the differences between job demands and role characteristics to assure coverage of the gap between actual learner skill sets, the expectations and standards established during the organizational analysis, and the criteria set from Task-KSA Analysis. Based on subsequent technical and media specification analysis, large differences in learner groups such can be leveraged into an appropriate medium through design and delivery supports or facilitated through blended delivery media. Learner analysis role-summaries should be submitted. Data collected at this stage should:

• Identify logistics of the base learning pool
• Detail role characteristics for learner profiles
• Gauge current performance measures and characteristics for comparison against what is required in training
• Develop an approach to providing targeted recurrent training based on identified performance gaps as required by stakeholder
• Ensure training program complements activities relating to recruitment and performance appraisal as required by stakeholder

Risks if not conducted or conducted improperly:

• Not understanding the learner - your customer
  
• A course that does not focus on the gap between tasks/KSAs assessed as required in “a perfect world” and task/KSAs the learner was doing before the learning event in “the real world”
• Delivery modalities that do not meet the differing needs of course participants
• Training that does not result in improved on the job performance criteria
• Dissatisfied course participants and a dissatisfied stakeholder

We will review technical and media requirements next.

Analysis and Planning: Task-KSA Analysis

The Task-KSA Analysis identifies what should be done on-the-job in the ideal world. A task-knowledge-skill-ability (Task-KSA) analysis targets the critical tasks, knowledge, skill-sets, and criteria required for performance-based behavior and outcomes resulting from courseware delivery. Task-KSAs are commonly referred to as competencies, and should be obtained from:

• Existing courseware provided by the stakeholder when possible
• Supporting documentation as provided by the stakeholder when possible
• Information as solicited through Subject Matter Experts (SMEs) provided by the stakeholder

This data should support the training content for design and courseware development. Criteria from identified tasks and required KSAs should be mapped to the learning objectives of the course design and testing criteria for evaluation as facilitated by the learning management solution. Task-KSA documents should be submitted, which will be used to support design documentation. Data collected at this stage includes:

• Identification of critical tasks, knowledge, skills, and abilities
• Identification of critical success factors for training outcomes
• Review of existing curriculum and suggest necessary modifications
• Specification of information sources/references
• Development of parameters for on-the-job training
• Development of parameters for training targeted to specific technologies/equipment
• Development of performance metrics and standards to govern the training process, as well as learner performance

Risks if not conducted or conducted improperly:

• Coverage of material already mastered by course participant/learners
• Missing coverage of material required on the job for course participant/learners based on material not captured during the Task/KSA Analysis
• Loss of internal and predictive validity as there is no baseline for success comparison
• A course that does not focus on the gap between tasks/KSAs assessed as required in “a perfect world” and task/KSAs occurring in “the real world”
• Delivery modalities that do not meet the differing needs of course participants
• Training that does not result in improved on the job performance criteria
• Inhibited transfer of training due ignorance of group perceived function and/or climate issues not planned for during training design (e.g. The difference between how upper management perceives what is required for sales database training and what sales people out in the field perceive as required)
• Dissatisfied course participants and a dissatisfied stakeholder

We talk about Learner Analysis next.

Analysis and Planning: Organizational Analysis

Organizational Analysis:

This sub-phase identifies organizational readiness for the learning solution. The organizational analysis validates and aligns the stakeholder’s mission and objectives with strategic delivery of a courseware solution, as mutually confirmed by stakeholder decision makers and the delivery team. The organizational analysis identifies critical information for the developed courseware solution, which should be mapped to criteria for behavior-based learning outcomes, performance outcomes, learner characteristics, and return on investment measures. Data should be mapped to the design phase as appropriate. An organizational analysis summary should be submitted.

Data collected at this stage includes:

• Critical success factors for synchronous and/or asynchronous training
• Confirmation of mission objectives
• Identification of training goals, expectations, and success measures
• Identification for points-of-contact
• Identification of systemic support factors
• Identification of constraining variables
• Confirmation that the problem to be addressed is based on a skill deficit vs. a performance or systemic deficit
• Paralleling project charter and planning

Risks if not conducted or conducted improperly:

• The solution does not meet expectations of decision-makers
• Misalignment of the course expectations with organizational or departmental goals, values, mission, and vision
• Support factors, external constraints, and climate related factors that do not facilitate transfer of training and thus result on project failure
• Losses of internal and predictive validity as benchmarks are not tailored to organizational needs for predicting success. (Internal validity looks at if training made any difference at all. Predictive validity is how training should predict success on the job based on criterion measures.)
• Miscommunication between the various constituencies of the stakeholder and the development team
  
• Development of a training solution for an outcome-oriented problem that requires a systemic or motivational solution
• Missed deadlines - extended project plans, and unforeseen obstacles to product success that do not meet stakeholder expectations due to a lack of proactive planning
• A dissatisfied stakeholder due to misdirected planning

We will discuss Task-KSA Analysis next.

Analysis and Planning

The first phase of ADDIE is the Analysis and Planning. This is the phase where we we gather information and identify requirements for a course or curriculum (a learning path to a set of courses based on roles). The Analysis phase is also known as Needs Analysis or Needs Assessment. The goal is to establish requirements for success by targeting outcomes and setting up the design for a training curriculum and subsequent courseware solution. This phase entails an examination of mission-critical needs; identification of required skill-sets mapped to respective performance criteria; an assessment of the learner’s actual competencies (Task-KSAs), performance measures, demands, role characteristics; and lastly, the training delivery media requirements.

The Analysis and Planning phase should include some level of the following sub-phases:

• Organizational Analysis
• Task-KSA Analysis
• Learner Analysis
• Technical and Media Specifications Analysis

We will break each sub-phase of analysis down in subsequent entries. Although Analysis is the most important phase, it is at times overlooked or not given the attention it deserves. This is most evidenced when training solutions and learning events appear ineffective or seem to "miss the mark." In my next entry, I will commence our breakdown of the Analysis Phase by examining Organizational Analysis.