Analysis and Planning: Technical and Media Specifications Analysis

A technical and media specifications analysis should identify the technical, graphical, and media-based requirements and constraints to assure eLearning delivery is compatible with the hardware and software requirements of the stakeholder’s learning base. Graphic look, feel, and specifications should be identified. Requirements should be mapped into courseware development. Summaries for technical and media specifications should be submitted. Data collected at this stage should:

• Identify infrastructure capabilities for courseware delivery
• Identify requirements for optimal delivery media
• Identify stakeholder standards for text, graphics, media, and branding

Risks if not conducted or conducted improperly:

• Graphics and media that do not meet the standards and satisfaction of the stakeholder
• Incorrect delivery formats
• Product failure due to incompatibility of delivery medium and/or components to the hardware and software requirements of the stakeholder and/or the stakeholder’s customers
• Missed deadlines and extended project plans that do not meet stakeholder expectations

In the end, all analysis data (organizational, task-KSA, learner, and technical/media specifications) should be used to design a learning solution that:

• Is based on the needs of the organization/department
• Addresses the gap between the required task-KSAs and what’s actually done by the learner on the job
• Is delivered using the appropriate mediums and technologies that map to learner profiles

Enough with Analysis. We move to the Design phase next.

National Learning Week

We will get back to ISD, the ADDIE model, and analysis in my next post, but this is a good time for a break because it’s ASTD National Learning Week. The local ASTD chapter I serve sponsored and prepared for this by asking members to provide some learning tid-bits that fit within an acronym we defined for L-E-A-R-N:

L - Learning

E - Evaluation or eLearning

A - Assessment or Activities

R - Reinforcement or ROI

N - Networking

They are posting and distributing responses out to members with one letter of the acronym for each day of the week. Here is my submission:

L:
At some level, we are all in the business of Learning, but what do we mean when we refer to learning, behavior, or training? The following definitions can be useful from an operational standpoint... Let’s start with our subject - the learning-participant or learner. The assumption is that learners behave in a certain way until they learn to behave in another, through some sort of experience. Let’s define behavior as something that a participant does that can be observed and measured. If so, then we can agree that any measurable change in behavior is some sort of learning. That being said, the training experience needs to be the purposeful cause of that measurable change in behavior so that what is learned is consistent and applicable.

E:
Some Evaluation questions to ask your self after learning events or better yet - to prepare for during course development: Do they like it? Do they know more now than they did before training? Can they do more now than they could before training? Are work-performance outcomes better now than they were before training? Are departmental or business metrics better now than they were before training? How am I ruling out confounding variables (reasons “other than training” for your evaluation data)?…

A:
What to analyze when Assessing the need for training: Define the sponsoring organization’s/department’s mission, goals, vision, culture, obstacles to performance and budget for training. Identify and what learners need to do in “an ideal world” for their job-roles in terms of tasks, knowledge, skills, and abilities. Validate what learners are actually doing in “the real world” for their job-roles in terms of tasks, knowledge, skills, and abilities. Identify the capabilities and limitations of your learners in terms of education-level, experience, and available technologies. Use assessment results to design and develop content that focuses on the gap between the ideal and real world requirements while accounting for organizational needs and learner capabilities.

R:
When designing and developing content, apply Reinforcement principles to learning events as ABCs:

A = Antecedents – or what happens before any change in behavior, in our case a learning event that is aligned with the learner’s needs and is effectively delivered before the she meets the learning objective.
B = Behavior – is what the learner does; ideally, meeting the learning objective.
C = Consequences – these are are anything that happens after behavior that can either reinforce (increase the likelihood) or punish (decrease the likelihood) that what was learned will be applied on the job. Build and deliver activities that reinforce: Tell’em > Show’em > Let’em Try It (guided) > Let’em Apply It (unguided) > and Test’em. Also, use reminders, job aides, feedback, and supervisor support to reinforce behavior back on the job.

N:
Use social Networking and Web 2.0 tools to learn: Blog to broadcast subject matter expertise; micro-blog to keep your team abreast of newly learned knowledge and skills; write wikis to share and update reference-able knowledge-bases between team members; video short info-bytes or demonstrations from vid-cam at your desktop for your corporate YouTube. Don't have a budget? Get permission to use free tools like Blogger, WordPress, PBwiki, WetPaint, Twitter, Google Docs and Reader.

Happy National Learning Week!

Analysis and Planning: Learner Analysis

The Learner Analysis identifies what is actually done on the job and required in the real world. A learner analysis (also known as a Person Analysis) should identify the range of learner needs based on the differences between job demands and role characteristics to assure coverage of the gap between actual learner skill sets, the expectations and standards established during the organizational analysis, and the criteria set from Task-KSA Analysis. Based on subsequent technical and media specification analysis, large differences in learner groups such can be leveraged into an appropriate medium through design and delivery supports or facilitated through blended delivery media. Learner analysis role-summaries should be submitted. Data collected at this stage should:

• Identify logistics of the base learning pool
• Detail role characteristics for learner profiles
• Gauge current performance measures and characteristics for comparison against what is required in training
• Develop an approach to providing targeted recurrent training based on identified performance gaps as required by stakeholder
• Ensure training program complements activities relating to recruitment and performance appraisal as required by stakeholder

Risks if not conducted or conducted improperly:

• Not understanding the learner - your customer
  
• A course that does not focus on the gap between tasks/KSAs assessed as required in “a perfect world” and task/KSAs the learner was doing before the learning event in “the real world”
• Delivery modalities that do not meet the differing needs of course participants
• Training that does not result in improved on the job performance criteria
• Dissatisfied course participants and a dissatisfied stakeholder

We will review technical and media requirements next.

Analysis and Planning: Task-KSA Analysis

The Task-KSA Analysis identifies what should be done on-the-job in the ideal world. A task-knowledge-skill-ability (Task-KSA) analysis targets the critical tasks, knowledge, skill-sets, and criteria required for performance-based behavior and outcomes resulting from courseware delivery. Task-KSAs are commonly referred to as competencies, and should be obtained from:

• Existing courseware provided by the stakeholder when possible
• Supporting documentation as provided by the stakeholder when possible
• Information as solicited through Subject Matter Experts (SMEs) provided by the stakeholder

This data should support the training content for design and courseware development. Criteria from identified tasks and required KSAs should be mapped to the learning objectives of the course design and testing criteria for evaluation as facilitated by the learning management solution. Task-KSA documents should be submitted, which will be used to support design documentation. Data collected at this stage includes:

• Identification of critical tasks, knowledge, skills, and abilities
• Identification of critical success factors for training outcomes
• Review of existing curriculum and suggest necessary modifications
• Specification of information sources/references
• Development of parameters for on-the-job training
• Development of parameters for training targeted to specific technologies/equipment
• Development of performance metrics and standards to govern the training process, as well as learner performance

Risks if not conducted or conducted improperly:

• Coverage of material already mastered by course participant/learners
• Missing coverage of material required on the job for course participant/learners based on material not captured during the Task/KSA Analysis
• Loss of internal and predictive validity as there is no baseline for success comparison
• A course that does not focus on the gap between tasks/KSAs assessed as required in “a perfect world” and task/KSAs occurring in “the real world”
• Delivery modalities that do not meet the differing needs of course participants
• Training that does not result in improved on the job performance criteria
• Inhibited transfer of training due ignorance of group perceived function and/or climate issues not planned for during training design (e.g. The difference between how upper management perceives what is required for sales database training and what sales people out in the field perceive as required)
• Dissatisfied course participants and a dissatisfied stakeholder

We talk about Learner Analysis next.

Analysis and Planning: Organizational Analysis

Organizational Analysis:

This sub-phase identifies organizational readiness for the learning solution. The organizational analysis validates and aligns the stakeholder’s mission and objectives with strategic delivery of a courseware solution, as mutually confirmed by stakeholder decision makers and the delivery team. The organizational analysis identifies critical information for the developed courseware solution, which should be mapped to criteria for behavior-based learning outcomes, performance outcomes, learner characteristics, and return on investment measures. Data should be mapped to the design phase as appropriate. An organizational analysis summary should be submitted.

Data collected at this stage includes:

• Critical success factors for synchronous and/or asynchronous training
• Confirmation of mission objectives
• Identification of training goals, expectations, and success measures
• Identification for points-of-contact
• Identification of systemic support factors
• Identification of constraining variables
• Confirmation that the problem to be addressed is based on a skill deficit vs. a performance or systemic deficit
• Paralleling project charter and planning

Risks if not conducted or conducted improperly:

• The solution does not meet expectations of decision-makers
• Misalignment of the course expectations with organizational or departmental goals, values, mission, and vision
• Support factors, external constraints, and climate related factors that do not facilitate transfer of training and thus result on project failure
• Losses of internal and predictive validity as benchmarks are not tailored to organizational needs for predicting success. (Internal validity looks at if training made any difference at all. Predictive validity is how training should predict success on the job based on criterion measures.)
• Miscommunication between the various constituencies of the stakeholder and the development team
  
• Development of a training solution for an outcome-oriented problem that requires a systemic or motivational solution
• Missed deadlines - extended project plans, and unforeseen obstacles to product success that do not meet stakeholder expectations due to a lack of proactive planning
• A dissatisfied stakeholder due to misdirected planning

We will discuss Task-KSA Analysis next.

Analysis and Planning

The first phase of ADDIE is the Analysis and Planning. This is the phase where we we gather information and identify requirements for a course or curriculum (a learning path to a set of courses based on roles). The Analysis phase is also known as Needs Analysis or Needs Assessment. The goal is to establish requirements for success by targeting outcomes and setting up the design for a training curriculum and subsequent courseware solution. This phase entails an examination of mission-critical needs; identification of required skill-sets mapped to respective performance criteria; an assessment of the learner’s actual competencies (Task-KSAs), performance measures, demands, role characteristics; and lastly, the training delivery media requirements.

The Analysis and Planning phase should include some level of the following sub-phases:

• Organizational Analysis
• Task-KSA Analysis
• Learner Analysis
• Technical and Media Specifications Analysis

We will break each sub-phase of analysis down in subsequent entries. Although Analysis is the most important phase, it is at times overlooked or not given the attention it deserves. This is most evidenced when training solutions and learning events appear ineffective or seem to "miss the mark." In my next entry, I will commence our breakdown of the Analysis Phase by examining Organizational Analysis.

ISD and the ADDIE Model

I have done a lot of talking about learning technologies, but I think now is a really good time to get down to basics, specifically, how any learning solution should be built, as in "instructional design." ADDIE is useful model of Instructional Systems Design (ISD) and is based on a 5-phased approach to courseware development... ADDIE is an acronym that stands for:

A - Analysis
D - Design
D - Development
I - Implementation
E - Evaluation

Execution of each phase is dependent on how a development team and stakeholders agree on the project's approach typically specified within a statement of work or project charter. As a system of checks and balances, each phase and milestone requires stakeholder approval to assure that what is delivered at project completion is what the stakeholder has approved from the beginning.

When no or little preexisting courseware exists, the most critical phase of a project is Analysis. Each phase follows sequentially with exception to Evaluation, which if conducted properly, occurs continuously throughout the process. Details regarding each phase will follow in subsequent blog entries. However, for a high level explanation of how this should work, see the graphic below.

It is fair to say that many in the business of course development skip phases within this model and some even ignore the model completely. This is a mistake, but please do not misinterpret that to mean that this process cannot be performed efficiently or within short time-frames. Those of us who work in internal training departments or on the customer business side need only to perform an analysis on a periodic basis for each department or customer-base we serve.

Also, variations in this model exist along with compressed methodologies and adaptations based on stakeholder needs. How this can be accomplished will be addressed in subsequent entries. Stay tuned for details on each phase of the ADDIE model.

Learning/Web 2.0 Collaboration for Mentoring and to Wrap Learning Events

For those of you who are interested, I was recently interviewed and quoted for an article in Training Magazine regarding use of Web 2.0/learning collaboration for training and mentoring. We used wikis and discussion threading to wrap learning events as previously described in this blog. The effort has been so successful that we have extended the original Java programming curricula to instructing and mentoring a global workforce of over 600 programmers on the use of C++, C#, .NET, Assembler, and Oracle SQL. To read directly from the article, click below:

http://www.trainingmag.com/msg/content_display/publications/e3i34ff3e76aa1f48788ed288b9cf20826a

How to Use Discussions within Learning Contexts

Oddly enough, of all the Web and Learning 2.0 tools available, the ability to support threaded discussion groups is the most useful. Wikipedia currently offers an excellent definition:

A threaded discussion is an electronic discussion - in which the software aids the user by visually grouping messages. Messages are usually grouped visually in a hierarchy by topic. A set of messages grouped in this way is called a topic thread or simply "thread". A discussion forum - is said to have "threaded topics" if it groups messages on the same topic together for easy reading in this manner. Moreover, threaded discussions typically allow users to reply to particular posting within a topic's thread. As a result, there can be a hierarchy of discussions within the thread topic. Various types of software may allow this hierarchy to be displayed in what's called Threaded Mode. (The alternative being Linear Mode, which typically shows all of the posts in date order, regardless of who may have specifically replied to whom.)

View more of the Wikipedia description here.

From a learning context, how they can be blended-with or attached-to other learning objects are what really make them powerful. Discussions can be used as pre-work, post-work, or breakouts to transition and bind self-paced training to the instructor-led, while supporting conversation around various blogs and wikis. From a peer to peer level, setting up discussions that query and prompt a community response is a great way to socially harvest information while sharing with those who may also benefit from the interaction. Most discussion groups require a facilitator or virtual instructor who takes responsibility for keeping the discussion's focus on-topic while promoting further response and conversation. Depending on the situation, seeding discussions with initial questions, responses, and comments while preparing content in-advance can fertilize activity within the discussion. Some examples of usage follow:

Focused Discussions:
These are a no brainer and are therefore the most popular. What is key is ensuring the topics of the discussion are directly related to the learning objectives. Focused discussions can prep your learners for upcoming events, serve as a breakouts between sessions, or keep the conversations going and relevant to the job long after any formal learning activity. If the discussion tool supports threaded mode, participants will be able to branch into sub-topics where appropriate.

Note-taking:
Great synchronous activity - while in-class, take notes, but not in your notepad or to a doc on your laptop, but within a note-taking discussion group. Later, participants of the discussion can share and compare notes. The instructor can correct, provide, and get feedback if appropriate. Everyone benefits.

Homework:
An instructor gives an assignment - the learner is asked to post it within a homework discussion room. Great for group assignments, but if privacy is required due to concerns regarding sensitivity or copying - have them submit homework in password protected zip packages or set-up 1:1 discussion rooms between the instructor and each learner.

Brainstorming Sessions:
Typically a breakout or between session activity, a brainstorming session may be assigned as an activity where learners are asked to apply content they just learned to resolve an open-ended scenario or existing problem at their workplaces. If the tool has the ability to freeze idea threads as they are posted - even better as this functionality supports the brainstorming philosphy of "no idea is a bad idea."

To-do Lists:
To-do lists are a great follow-up to a learning event. The learner just took a course, now have them publicly post what they will do with their new knowledge, skills, and abilities. While others read, they may be inspired to do the same, the similar, or completely innovative based on the sparks that are lit.

Goal Planning:
Goal planning discussions are an excellent post-work activity where learners must turn learning objectives from a course into SMART goals they will achieve on the job. (SMART = specific, measurable, attainable but difficult, relevant, and time-framed.) Goals set by others can motivate participants to set similar goals or build on their ideas. Coaches and or peers can review and provide feedback.

Action Planning:
In the last activity participants identified their goals - an action planning discussion is where they can collaborate and plan on how to achieve those goals with their managers and supervisors. Great way to transform what was learned into practice.

Scorecards:
Great post-work activity where teams are required to post business metrics relevant to how the content that was learned is improving efficiencies, productivity, sales metrics, or any other measurable outcome related to the learning experience and job context. Scorecards, scoreboards, and dashboards work on the premise that what gets monitored changes and when combined with learning activity, provide a very powerful feedback and reinforcers. You might have them brainstorm on what to measure and how prior to this... Bottom line - these really work and managers who send their employees to training love them.

Lounge:
Classroom based training typically offers a lounge, lobby, cafe, or even water cooler area where participants can take a break. As learning experiences become more electronic, we may need to provide virtual learners with a virtual lounge area so they can take that break and come back fresh into the learning experience. If you are concerned about the type of discussion that may take place, provide guidelines and moderate as required.

Wrapping:
Wrapping using a discussion thread is the truly packaged and blended learning experience. The entire learning activity can be coordinated through discussion if appropriate - from scheduling, to preparation, and evaluation. Keep the discussions relevant by embedding links, attaching appropriate materials, providing feedback, and facilitating discussion on relevant topics that speak to the learners needs. Instructors can assign pre-work, breakout sessions, home work, and post-work around self-paced or live learning events.

Of course, there are other ways to use discussions as part of the blended learning experience. Membership in an ongoing discussion can be leveraged as a reinforcer to learning. They can be used as an incentive to participation and or completion where access to subject matter experts and peers with similar problems or experience is perceived as a benefit. In sum, use discussion threads to expand the digital tribe through social contact and by tying formal learning to the informal.

How to Use Wikis within Learning Contexts

What is a Wiki? Well, Wikipedia, the granddaddy of all wikis, defines a wiki as “a collection of web pages designed to enable anyone who accesses it… the ability to contribute or modify content…” Wiktionary, an off-shoot of Wikipedia, defines a wiki as “a collaborative website which can be directly edited by anyone with access to it.”

A wiki is best known as a Web 2.0 tool. Although at times criticized because the information in them is not always valid, the premise behind why wikis work has much to do with how the collective wisdom, opinion, and judgment of the masses can outweigh the same characteristics of any one individual. Groups with a common purpose and even totally unrelated contributors can centralize information, participate in wiki development, and update it in a collective, self-paced, and or synchronous manner. Wikis operate on the concept of wabi-sabi, where contributions that are incomplete or unfinished are encouraged as it is expected that others in the community will add-to and perpetually complete the document. If one person creates a wiki item, another can add more content, edit what exists, or augment it with supporting links and references. Because most wikis by nature are versioned, anyone can easily review its change log for editing history and revert back to a previous version as applicable. Reviewers might even openly debate a wiki’s validity in an attached discussion thread. Including open discussions as an attached feature of the wiki facilitates a method of judges through peer reviews and ultimately results in highly valid and relevant content. At the end of the day, it is the community of authors who ensures the accuracy and importance of the wiki as it is consumed and critiqued by its readers.

From a learning perspective (Learning 2.0 tool), a wiki allows its contributors and readers to review, add, and update useful information so they can collaborate and share at a peer level, while learning about topics of interest through projects and group assignments as they are relate to other learning events, resources, and job demands. Some examples follow:

HOWTO-type Wikis:
Wikis are great places to display and collaborate on task-based HOWTO-type documentation such as manuals or guidelines on the use of a tool, method, or process. Sure you could post the actual manual or training guide, but wiki items can address the most important and frequently used tasks as well as the contexts in which those tasks will be applied. Additionally, they can be updated in real-time as the subject matter changes. Depending upon the situation, this type of wiki may only be editable by a group of subject matter experts (SMEs as opposed to all). At the same time, if the tool method or process is new, all who are diving in can be publishing and updating HOWTOs to be shared by all. None-the-less, HOWTO-type wikis are a great way to teach and train others either on a stand-alone basis or as a wrapped follow-up to other learning objects or events.

Best Practice Wikis:
Let’s say you have a new tool or method and need to get beyond the task-based HOWTO-type material because the technology is multi-faceted, complex, or just plain real important. A best practice wiki is a great repository for collecting and documenting information for this set of needs. Again depending upon the situation, this type of wiki may only be editable by a group of SMEs as opposed to all, but because such material tends to be more suited for a globally dispersed audience, keeping it open to all increases the likelihood that unknown experts will contribute. Like with the HOWTO-type wiki, if the technology itself is so spankin’ new you have no experts, you will want to give everyone a place to document best practices so as to capture the collective discovery-process and subsequent experiences. Sure, in this type of scenario, not everything captured may yet be a best practice, but what a convenient way to document those lessons in real time so the community can glean out what is truly best or not while applying them. This way, others can learn about and later avoid the mistakes made by others while openly taking advantage of lessons learned. Depending upon the learning need, best practice wikis can be referred-to asynchronously or on a pre-work, mid-curriculum, or post-work basis.

Wikis as Knowledge Bases:
At a consumer level, knowledge bases allow users to collect, organize and retrieve information. They are either part of an automated system, like with helpdesk applications, or are readable, reference-able, and searchable by users. By design, most wikis bake-in this capability. Thusly, using a collection of wikis combined with full text and or federated search engines can emulate a knowledge base. From a pulled learning perspective, they provide an almost database-like experience for learners who need just-in-time information on the job or while participating in a tandem learning activity. From a training delivery perspective, a wiki knowledge bases allows users and SMEs to collaborate on real time updates. This particular usage will become more prolific as baby boomers retire out of the workforce and millennials (Generation Y) onboard into organizations. Based learning needs and opportunities, this type of wiki works well as an asynchronous learning object or as a follow-up reference to a preceding learning activity.

FAQ Wikis:
If you know what questions or issues your participants will ask, or have advanced notice regarding the issues and problems they will experience, FAQ (Frequently Asked Question) wikis can ease the burden for both the instructor and the learner. These are especially useful when rolling out new programs and assigning homework or group-projects. As such, FAQ wikis work well when presented just prior-to and during program roll-out, and just after assigning homework or group-related projects.

Process Wikis:
If your objective is to teach about a new or updated process, a process wiki allows instructors and process owners to collaborate with new learners of the process. Such a wiki combined with discussion threading could support Q&A regarding policies, procedures, and even graphic of the workflow (a picture is worth a thousand words). A process wiki works well as an asynchronous learning object or when paired as a follow-up reference to a preceding lesson on the process.

Learning Path Wikis:
So the instructional design team has created their blended learning masterpiece and your instructors and facilitators need a place to post the curriculum’s learning path and learning objectives… Hey, your learners need a place where they know to find these things. Because the components of blended curricula now go beyond the conventional first module or lesson in a classroom based student guide or self-paced eLearning course, the learning path wiki can be an excellent focal point for converging the various mediums the curriculum is delivered in. It is especially powerful when the wiki function is part of a larger collaboration suite supporting the blended delivery of the curriculum. Naturally, the learning path wiki is best when made available just before delivery or just as participants are beginning their learning experience.

Assignment Wikis:
Let’s say you are mid-session for a virtual class and you were given a group-assignment or your team was given ‘homework’ instructions through a discussion thread. An assignment wiki is a convenient location for collaborating on the assignment. The versioning features allow the group to track progress and to revert to previous iterations while discussing threading allows the group to discuss, debate, and collaborate while drafting the assignment. While they are doing this, the instructor or facilitator can monitor progress and see who is contributing by reviewing the saved versions. Useful, no? Obviously, an assignment wiki is best delivered as a breakout during mid-session, between sessions, or as part of an overall collaborative experience.

So, what’s a wiki? Set up and positioned correctly, it’s a pretty cool learning medium.