Ask Harold

I would like some advice on paper versus electronic job aids. I'm concerned that paper gets outdated and people still hold onto the old information, even when updates are provided. The electronic format is good, but it's not as convenient so may not be used as much. Please share your thoughts on this subject? Answer

I am in a class on Entrepreneurs and we are required to put a business plan together. I will be putting together a plan for my newly formed performance consulting company and part of the plan requires us to put together our pricing. Do you know where this information might be available? Answer

What is conceptual skill and why is it important to top management? Answer

Is there any research that shows that learning transfer happens more effectively and efficiently when inside trainers are used versus outside consultants? We are implementing a new IT system and our COO thinks it is more effective to use the new systems trainer who will learn our processes and teach the new workflows, versus a T3 approach using trainers from each department who have cultural knowledge and understand the work and the application of the new system in their environment. Answer

When/under what circumstances should I use a RASCI model and how do I use it? Answer

I'm working on a performance improvement project and just finished the cause analysis. The gaps I found have fall mostly under trust issues and open communication. Should I concentrate on the examples and factors of why that trust is not there or should I state that lack of trust is the reason they are having these gaps? Also, when looking at the Behavioral Engineering Model, where would lack of trust fall under? Should it go under information or capacity? Answer

Do any of the consultants you work with who are internal (company) consultants report difficulties in getting closure with their internal clients? We currently have some 19 projects in process but are having difficulty with some when it comes to closure. It seems that other priorities delay meetings, implementation activities, etc. If we were external and continuously invoicing clients for hours worked, it would probably be a different story. Any thoughts or suggestions? Answer

At the ISPI Conference in Chicago this past fall, you stated that learning styles are pretty much discredited now as they only account for 2% of learning effectiveness. You said there was no firm evidence to support them. Can you give more information on any academic literature to back up what you're saying? I agree with you but have colleagues who strongly disagree. Answer

Learning packets are one way in which our employees receive education on new topics. We struggle with finding a way to consistently and realistically determine how long it should them to finished the packet. Sometimes they are doing them at home and then they are paid for the time. So pay becomes a part of the scene and can cause problems when people feel the time estimated to complete the packet is in error. We also offer classes that require study time out side of the classroom. They are also paid for this time. We struggle with determining the correct time for this work as well. Any insights or experiences you have had with determining independent study time? Some staff feel they are not given enough time while others seem to need less. What is the most equitable way to determine these time frames? Answer

What is the difference between efficiency and effectiveness? Answer

I read your article "Keys to Performance Consulting Success." However, it still seems like there's a huge canyon to bridge between doing training-only and performance improvement. You have to acquire the skill set (hire new people) to get the clients for performance improvement. How do you justify acquiring the skill set if you don't have the clients? Answer

What are some standard questions that should always be asked when speaking with a SME to ensure you find out everything you need to know? Answer

What is the difference between ability and skill? Answer

In one of your articles you mentioned "not more than 10 percent of the expenditures actually result in transfer to the job." Does this mean:

Transfer rate of 10% was achieved because proper analysis was conducted (including transfer strategies)?
We should have invested the 90% to improving work environment factors as training was obviously not the ideal solution? Answer

Can HPT be applied in an educational organization? Answer

There is increasing interest from the training, consulting, and HR communities in HPT but there is also worry whether this interest is going to translate into increased practice and opportunities. Please comment. Answer

What is the difference between skill and competency? Answer

When conducting a needs analysis, what are the most important questions to ask?Answer

Do you have any additional information on the topic of transforming performance capability?Answer

I am most interested in supporting my managers in having a discussion with their employees pre and post training. Do you already have a template of what this discussion might look like?Answer

Why is it so hard to apply a few proven best practices in our work environment?Answer

How do accelerated learning and brain-based learning relate to the emotions of fear?Answer

What is the difference between a training needs analysis and a performance analysis?Answer

We frequently hear the term "learning styles" tossed around like "I know there are tests for people to identify their learning style" and so on. People use this to argue for one delivery media over another like "My learning style prevents me from engaging in e-learning!" I would like to know if there are any scientific grounds for the existence and definition of learning styles. If so, what are they and what are the main messages coming out of this literature?Answer

Several years ago I encountered a training philosophy that I liked, but I don't know how to find resources that would teach it. The philosophy is that you let people know up front what specific knowledge and skills they will be taught and expected to know. An outline is made which essentially becomes a written test, but not in the traditional academic format. The test items are written in a verbal, proactive way, such as "Describe ...", or "Relate these items.." or "Demonstrate ..." for those skills that can be observed. So the "test outline" is seen at the beginning, the trainee knows exactly what is to be learned, and that the objective is to complete it with 100%, not a lesser passing grade. The concept is that, if you are training someone in an important task, are they competent at 80% or wouldn't it be better if they were 100% on things you need them to know. Can you identify this philosophy and lead me to resources?Answer

What is the difference between organizational effectiveness and human performance technology (HPT)? Answer

What is the difference between Organizational Development and Human Performance Improvement? Answer

Would you have any advice for new consultants to the field? What skills, aptitudes and attitudes do you think are important to succeed in today's competitive marketplace? Answer

To what extent do we, as instructional designers and human performance technologists, need to immerse ourselves in the contexts of our client's business? In other words, must we acquire or possess in-depth subject knowledge of their field, or is it sufficient to be merely familiar with their content? Answer

What are the best most popular needs assessment (competency-based or performance-based) models? Answer

De plus en plus, on essaie d'homogénéiser les clientèles en formation, donc de regrouper les mêmes niveaux d'habiletés, capacités d'apprentissage au lieu de diversifier la clientèle (apprentissage rapide, régulier ou lent dans des groupes séparés); qu'en pensez-vous ? Réponse

What do you think the future holds for training professionals with respect to Reusable Learning Objects? Answer

How does one objectively measure a candidate's personality in an interview? How important is the personality aspect in a candidate vis a vis his technical skills in today's competitive scenario? Answer

What is meant by a "blended solution"? Must it include a high tech component? Answer

What is the difference between front-end analysis, needs assessment, and needs analysis? What can I say to my management to persuade them to invest in front-end analysis for my projects? Answer

Why don't men listen? Answer

Can you develop interactive, online learning teaching learners to use a software product when the software is being developed concurrently? If yes, how is this done to ensure cost containment and efficient use of resources? Answer

What is the difference between a performance, an accomplishment, a competency and a skill? Answer

Is e-learning all it's cracked up to be? Answer

What is performance consulting? Answer

What is the difference between Human Performance Technology (HPT) and Human Performance Improvement (HPI)? Answer

How do you calculate Return on Investment (ROI) in training or human performance improvement interventions? Answer

Is ISD outdated? Answer

Paper or electronic, both are media and each has its place in the array of job aids we use. Part has to do with the use people will make of the job aid. If you are baking a cake or making a stew and have stuff all over your kitchen counter, a paper cookbook or index card is probably more convenient than an electronic device. Similarly a job aid pasted to a telephone with emergency numbers is more practical than some electronic instrument. On the other hand, job aids for trouble shooting equipment are more helpful in an electronic mode, particularly if they allow access to databases and expert advice to deal with highly variable circumstances.

So how should you decide which medium to use? Here are some rules of thumb:

For simple procedures that do not change much over time, paper job aids work well. They also tend to be more cost effective and can be carried around, pasted up or mounted in a prominent place very easily.
For complex procedures or requirements to access a variety of procedures, some of which are complex, electronic job aids are generally better.
Consider the level of technical sophistication of the end-users and their preferences. Some people cannot adapt to PDAs. They prefer paper daybooks and planners. For less technologically experienced users, paper job aids are generally easier to use.
Consider the environment in which the job aid will be used. In some, electronic devices will not work well (e.g. in an underground tunnel; at a buzz saw). In other environments, electronic job aids are by far superior (e.g. inside the cockpit of an airplane; at a terminal). Analyze the implementation context to determine which medium is more compatible with environmental characteristics.
Consider the volatility of the content. The more frequently the content of the job aid is likely to change, the more appropriate electronic job aids are if there is a means to update these automatically. If not, they are no better than paper ones.
Finally, consider cost. If electronics are at hand and are cheap and easy to use, then that's the way to go. If some of the end users will not have access to the electronics and you can produce a paper-based job aid quickly and cheaply, then paper is your best bet.

The bottom line is that practical considerations reign when it comes to selecting the appropriate job aid medium. Implementation variables usually dominate here. The most important aspect of a job aid is its design for accuracy, clarity, ease of use and accessibility. Focus more on these than which medium is better. The right choice will naturally emerge.

I am in a class on Entrepreneurs and we are required to put a business plan together. I will be putting together a plan for my newly formed performance consulting company and part of itn requires us to put together our pricing. Do you know where this information might be available?

There are no published figures on what individual private performance consultants charge. A lot depends on their noteiety. I know several who charge $6,500 per day and more. Others are at the $300 per day level. Three possible sources are:

ASTD State of the Industry, 2003. In that year, ASTD published salaries for internal performance consultants. The range for 2002 was $65,218 to $124,200 with a mean of $80,671. The sample size was 276.
The second source is the Instructional Systems Association
(ISA). However, they only tend to share this type of information with their members.
ISPI in Washington. They may have some data on fees.

What is conceptual skill and why is it important to top management?

The term "conceptual skills" is a convenient one to describe the general analytic ability of people to formulate and deal with ideas. Conceptual skills are used most frequently as part of an overall description of what managers require to do their work effectively. Often, you will run across the following:

To be an effective manager you require three major sets of skills. The first is "technical skills," which involves process or specific abilities to perform tasks related to the job. This is also known as "hard skills." The second set is "human skills" and centers on the ability to interact effectively with people. The third is the "conceptual skills" set that deals with the formulation and manipulation of ideas. This includes the ability to work with abstract notions and relationships and to do creative problem solving.

While this brief response probably does not do the term full justice, there are those who have specialized in this area. Skills of an Effective Administrator by Robert L. Katz is considered to be a Harvard Business Review classic.

There is no research on this issue. It's not a researchable question and here's why. There is a whole host of variables that affect the impact of a training program. Many of these have nothing to do with the training itself as demonstrated in my book, Training Ain't Performance. There is also the nature of what is to be learned and the experience of the learners with respect to this content. Finally, the characteristics and capabilities of both inside and outside trainers affect the learning and performance results.

More important than who does the training are three essential activities on which you should focus:

The front-end analysis to identify all of the key variables that will affect learning, motivation to apply and actual competent application on the job.
The design of the instruction itself. If it is well designed with lots of practice, contains job aids that can be used in the work context and includes cases and/or simulations that mirror actual work situations, the probability of success rises regardless of where the trainer comes from.
A really good train-the trainer preparation that focuses on transforming and not transmitting, that gets the trainers ready to work the solid instructional design and that emphasizes how to provide feedback and encouragement. In this instance there is an advantage to using well-selected internal people as trainers. You develop internal skills and then can convert the trainers into on-the job coaches and consultants post-training.
An excellent support system. Get the supervisors up to speed to encourage and assist performers. Have them insist on appropriate application. Create a temporary technical support hotline for those in need post-training.

It's not where the trainers come from that is the issue. It's the system you build that results in desired performance valued by all stakeholders that counts.

When/under what circumstances should I use a RASCI model and how do I use it?

The RASCI chart is a planning tool. This means that you should use it at the very beginning of a project once you have determined its size and scope and who the project "players" will be. In our learning and performance improvement project planning work, we usually begin with a timeline that lays out all of the tasks to be accomplished and the estimated number of person days each will require for completion. Once the timeline is established, we then turn to our RASCI chart to list the same tasks as in the timeline, list the project participants, determine the roles each person will play for each task and - our innovation we believe - the estimate of time required for the person to execute his or her assigned role for that task.

There is an interplay between the estimates in the timeline and the time assignments in the RASCI chart. They ultimately have to match. You adjust back and forth until they are perfectly coordinated.

The meaning of each of the letters is as follows:

R = Responsible (the person who must ensure that the task is completed)
A = Approval (the person/s who must sign off on the task)
S = Support (the person/s who can help/facilitate in the execution of the task)
C = Consult (persons you can turn to for expert information and advice: technical, administrative, legal, marketing, instructional)
I = Inform (persons or groups who must be kept up to date on what is happening in this task).

Every step requires an R. Many steps require an A, S or C. I is only noted if someone requires information on the project tasks but plays no active role (e.g., a manager; unions). Within each step, estimate the number of hours (or days) it will take each participant to perform her/his part. Enter the estimate in the lower right box for each task/participant. Here is an example of a RASCI chart filled in.

Project RASCI Chart (time specified in person hours)

When you have completed the chart and it matches the timeline as well as the expectations of the organization, you can use this to inform all players about the total amount of time each will have to budget over the life of this project. You can also generate individual calendars for each of them with the dates on which they will be required to work on the project and the specific task. We generally present the RASCI chart during the Project Launch meeting - a must for successful projects.

We (my colleagues and I at HSA) have found that when they are done well and approved by the client/organization, the timeline and RASCI chart make project management so much easier. We usually assign someone we euphemistically call the "project nag" to manage the timeline and RASCI chart. This is often an intern or junior team member. The purpose is to provide this person with a developmental experience. She or he keeps people on track and reminds them of due dates. No one may argue with this person. This adds a layer of protection to the project manager and usually results in the work getting done on time.

I'm working on a performance improvement project and just finished the cause analysis. The gaps I found fall mostly under trust issues and open communication. Should I concentrate on the examples and factors of why that trust is not there or should I state that lack of trust is the reason they are having these gaps? Also, when looking at the Behavioral Engineering Model, where would lack of trust fall under? Should it go under information or capacity?

It's great that you uncovered these issues. The first cut is to state that key factors inhibiting performance are lack of trust and stifled communication. The impact they have lowers the motivation of performers to engage in whatever it is that is being sought. I would continue to dig more deeply and state specifically what the lack of trust and poor communication indicators are and the impact of them. Gather data to support your claims so that no one can argue that this is "just your opinion." Obtain solid evidence.

With respect to the adapted Behavior Engineering Model, I would place these in the "motivation" box. Why? Because motivation is affected by value, confidence and mood. All of these factors can raise or lower motivation to perform. Mood, in a work setting, is greatly influenced by the atmosphere of the workplace. Intimidation, perceived threats, lack of trust, a sense of unfairness and a fear to communicate openly can create a negative "mood" in performers. They back off or lie low. Negative factors have a stronger impact on motivation than do positive ones. In other words, negative factors lower motivation and performance more rapidly than positive factors raise them. We get "bummed out" very quickly and withdraw from the game. Appropriate interventions will fall under the category of "motivational systems." These include elimination of the issues underlying the lack of trust or in dealing with them squarely as well as dealing with those related to the communication barriers.

Internal or external, it doesn't seem to make a whole lot of difference. Most of the time, we work with the internal people so that both of us are affected. In addition, internal people get paid regardless of delays whereas the externals can't bill which leads to devastating repercussions for their employees and freelancers.

Having established a band of brothers culture between us, let me turn to some suggestions. These are based on our own practices and do help to some extent:

Create contracts with your clients in the same way that external consultants do. Establish clear expectations with respect to deadlines, penalties, etc. Become a consulting and service provider to your clients using the same tools as your externals.
Find a senior level champion with political clout. Take her/him to lunch. Explain the problem. Build a business case with examples and show what this is costing the company. Work with the senior manager to put reasonable pressure on the clients.
One of the biggest problems we all face is accessing busy subject-matter experts (SME) or key sign-off managers. This can make things drag intolerably. Along with your initial project timelines, create what we call RASCI charts. These identify all project players, note which role each one plays at each step of the project and estimates time requirements for each task individual players are responsible to perform. This informs them of their time commitments over the life of the project. If anyone has difficulties, problem solve with him/her and identify alternate personnel to replace parts of their tasks. Use this along with your project time line to generate individual calendars so that each person knows exactly when he/she has to do whatever is required. This works very well.
Assign a "project nag." Yes, you read it right. At the project launch meeting, gather all the players (including client representatives) to go over the project , identify potential delay spots and create means for overcoming these. Then, get everyone to accept an identified individual (we often use an intern or a junior person) as the project nag. No one may argue with, insult, pull rank on, etc. this person whose responsibility is to inform people of their due dates and how their delays may affect others. They literally nag people, reminding them of commitments. It's great for the team because they have other duties and forget project due dates. This keeps them on task. It helps the project manager who can focus on project matters and not on negotiating new timelines for individuals. It is a wonderful development experience for the person tasked with project nagging. We have found it very successful.
Track costs of delays and regularly inform team members and clients of them. Copy the senior manager on your memos. Money often talks.
If you know of some people who in all probability will cause delays, create artificial deadlines for them. Hold them to these deadlines, painfully retreating from them to the real deadlines with much hand-wringing.
Finally, find an active sport where you can take out your frustrations. I run long distances. It helps soothe my troubled spirits when those **@*&%^$$^&! project delays occur.

I mentioned two different things. One had to do with sensory modalities. Some people are more visual than auditory...or kinesthetic. If you explore the literature on what are called ATI studies (aptitude by treatment interaction) you will come across significant differences with respect to favoring one sense over another, but there is not enough "power" in this difference to be overly concerned compared to many other variables that affect learning. That's why I recommend stimulus variation. This is pretty potent if done well and meaningfully.

This is different from "learning styles." Here is a previous question and answer from the Ask Harold Archives:

There is a lot written on learning styles and a lot more folklore circulating about it. Here are two useful articles that deal with your question. The first, McLoughlin, C. (1999). The implications of the research literature on learning styles for the design of instructional material, Australian journal of educational technology, 15 (3), 222-241, provides a good overview of the main currents and definitions of terms that are similar, yet different. These include: learning preferences, cognitive styles, personality types and aptitudes. It also examines two main learning style theoretical approaches, one that divides learners into wholist-analytic versus verbaliser imager and the other that suggests four categories: activists, reflectors, theorists and pragmatists. The second article, Stahl, S. (1999). Different strokes for different folks? A critique of learning styles. American educator, 23 (3), 27-31, questions the validity of the learning style construct itself. In the article, Stahl examines learning style inventories and questions their reliability.

My take on all of this is that there are individual differences that affect the way we learn. However, there are also many rules with respect to learning that apply to all of us as human learners. While it may be useful to factor in variations in learning approaches, it is probably more useful to apply those universal principles that research on learning consistently suggests result in higher probabilities of learning. What are they? Six simple ones:

When learners know why they are supposed to learn something (a rationale with a credible benefit to them), the probability of learning and retention increases.
When learners know what they are supposed to learn (a clear, meaningful objective), the probability of learning and retention also increases.
If what is to be learned is clearly structured and organized so that the learner easily sees the organization and logic, again, learning and retention probabilities increase.
If learners have an opportunity to actively respond and engage in the learning in meaningful ways, once again, learning and retention probabilities increase.
If learners receive corrective and confirming feedback with respect to responses they emit or activities in which they engage, their learning increases along with retention.
Finally, if the learner feels rewarded for the learning, has a sense of accomplishment or is given an external recompense for the learning that he or she values, learning and retention have an increased probability of occurring.

With respect to being more visual or auditory, while there may be significant differences among learners, more important, however, is stimulus variation.

Concerning the use of media or self-pacing, issues about learning tend to focus more on the design of the mediated or self-paced material than on the medium itself. Richard Clark has written extensively on media not being the message. If the mediated and/or self-paced material follows the universal rules and is well supported, it will work. Some learners who are not used to learning on their own may require additional support and control. Distance universities, such as the British Open University or Athabasca University in Canada, have learned how to do this well.

So, to conclude, there is a lot of folklore and some science concerning individual differences in learning. Best to apply universally sound methods to enhance learning. Vary activities to maintain interest and attention. Provide support and control mechanisms to help learners "stick with it." This way, you address all learning styles.

There are several methods for doing this, one of which is somewhat radical. A simple method, once the package is complete, is to ask a few (4-6) learners to go through it while someone who acts as a proctor is present. Time the learners and establish an average time. Of course, your learners for the trial should be representative of the range of learners in the entire population.

A second approach is to establish a time for each section of the packet (also based tryout testing) and to state up front that this learning package should take "X" number of minutes to complete. Create a matrix that contains the various sections and suggested time allotments. Explain that you are providing this in the interest of the learners so that in advance, they can budget sufficient time. Explain that this is based on tryouts and that it is 20% longer than the average time taken. Please make sure that this is true.

A third, more radical approach, but nevertheless perfectly acceptable, is to state that the job requires continuing education and that there are periodic tests which employees must complete. The results are placed on their records and considered an essential part of their performance evaluation. The requirement is for employees to pass the test. If they desire, they can request a self-study packet to prepare. It is free and contains all the content and sample tests to help prepare for the official test. This places the responsibility to learn on the learners themselves.

This last approach is more radical in that the requirement is not the training, but the passing of a test. This requires a system that makes continuing education essential and valued for the job and truly rewards success. You can even have it linked with professional CEUs from various organizations so that the learning and testing is officially and externally recognized.

What is the difference between efficiency and effectiveness?

Efficiency is productivity with minimum waste. It is the achievement of maximal results with the least amount of resources and energy expended. An example would be your attainment of a goal in the least costly manner with the least amount of your energy and resources used. Highly skilled persons usually perform tasks that achieve high results with far less effort than novices. Watch an expert swimmer do laps in a pool. She or he seems to glide effortlessly back and forth. Caloric expenditure is minimal compared to the novice who splashes around and is soon exhausted doing far fewer laps.

Effectiveness is achieving the desired result from the expenditure of resources and effort. Think of a medication for an ailment. If it is effective, it will cure the ailment. An aspirin for the common headache is a simple example. Taking a cough medicine for the headache would be ineffective.

In learning and performance, we strive to create interventions that exhibit the two characteristics of effectiveness and efficiency. Suppose a person had to be able to dial the correct number for a given emergency, but the number varied depending on the nature of the emergency. You could drill the numbers into the person's head and make him or her repeat these until a perfect result was achieved. It would be effective, but not necessarily efficient. Another way might be to provide a telephone with clearly labelled emergency numbers on it and spend a few minutes training the person on how to press the appropriate buttons. Hence, for a given emergency, the person would simply press the right button. Much more efficient than all the drill and practice and the possibility of error over time.

Remember, determine what will be effective first. Then determine the most efficient means for getting there. Focus effectiveness on results and efficiency on the means for attaining them.

I would imagine that you already have clients asking for your services. I am guessing that they are requesting "training." Your first step is to reassure them that you are there to help. Your response to their request is, "I can help you solve your problem." Then you begin probing for the true business need: "Imagine that whatever we do for you works perfectly and you're totally delighted. What is different in this scenario from the ways things currently are being done and accomplished?" By documenting the ideal and current you will begin to see (and perhaps so will your client) that there are other factors requiring attention beyond training. Now is the time to offer a broader, "performance" vision. And thus a client is born.

I know that this sounds a bit simplistic. However, the basic approach is sound. I recommend beginning with a current client who is open to this form of questioning and would be supportive to a broader, systemic vision of what is needed beyond checking off a "training box." Work with this client on a high-probability-of-success project. Then use the project success and the client to market to others. Write up the project for internal newsletters. Invite potential clients to a show and tell. Let your successful client be your advocate. If you build it and market it, they will come.

As for the skill sets, unless your current instructional people are too set in their ways, undertake a diagnosis of current competencies and examine those they require. Establish a development program for them. Without sounding too self-serving, please get a copy of our latest book Training Ain't Performance from www. astd.org. It includes a diagnostic tool and suggestions for becoming a performance consultant. The resources section offers a number of resources to help build your team. I hope this helps as a starting point.

What are some standard questions that should always be asked when speaking with a SME to ensure you find out everything you need to know?

This is a tough question in that subject matter content greatly varies. Also the nature of the subject matter expert (SME) is highly variable. Is the person someone who has deep content knowledge? Or is the SME one who has a very large repertoire of practical experience? This makes a big difference.

A content person, someone who can speak at length about a specialty area, may possess a great deal of declarative knowledge. This person can describe, explain, give reasons, and provide a highly articulate verbal presentation of a field. Examples would be lawyers, scholars and professors.

An expert who has been doing the job for a long time and automatically performs complex tasks possesses strong procedural knowledge. She or he can do the job. Examples of this might be a fine chef, a software roubleshooter, a pilot or a New York cab driver. The expertise resides in the ability to "do" something.

Each type of expert requires a different approach. Those who can "talk about" subject must be directed and probed. They will assume you understand the jargon and possess the concepts which are very familiar to them. With them, you have to stop frequently and ask for examples. You must have them illustrate what may appear to you (and novice learners) to be highly abstract. The questioning technique requires constant demands for clarification with illustrative examples. With these types of SMEs, you may be more in danger of learning too much rather than too little. You have to lay out the limit and scope of what you require. Then you must verify with other SMEs to ensure that the explanations you have received are not one-sided.

With "doing" experts, the best approach is to have them show you. As they do this, ask them to talk out loud. Constantly ask them to describe what they are doing and why they are doing it. Probe for how they make decisions. Then, if possible, play back to them what they have demonstrated and explained. Do it, even in simulation, as they watch you perform. Capture their feedback.

In all cases, once you have written up what you have gleaned from your SMEs, go back and confirm your write-ups. Have them give you feedback. Always verify your understandings.

To summarize, there is no perfect set of questions to ask. However, by preparing SMEs for what you wish to get from them, by listening, probing and continuously playing back your comprehension and by returning with a written summary of what you gathered from them, you will increase the probability that you are on solid ground. Ensure that what you have is accurate, current and complete, given your needs.

What is the difference between ability and skill?

A bit of a tricky question here as with so much of terminology. Because most of the social sciences have not nailed down specific and universally accepted definitions for all the constructs they commonly (and often loosely) use, it is difficult to come up with "official" meanings for them. The usually accepted definition for "ability" is an individual's potential to perform, given the opportunity to do so. This is in contrast to "aptitude" which is an individual's potential for performance, once they have been trained up to a specified level of ability. I have drawn this distinction from the Penguin Dictionary of Psychology, second edition (1995). "Skill," on the other hand, is the capacity to carry out complex, well organized, patterns of behavior smoothly and adaptively so as to achieve some end or goal. In its origins, skill generally was associated with psychomotor activities. Now, we use it for the verbal and social domains as well. Once again, I have relied on the same specialized dictionary to help me with this.

To make all of the above concrete, you may possess a broad range of abilities -- things you can perform, from judging when to invest to discriminating between a perfectly pitched note and one that is slightly off. You may have various aptitudes including, for example, musical ones, but if these are not fostered, they may never manifest themselves as part of your repertoire of abilities. The skills you posses do show up by the smooth way you execute them. You are "dexterous" at them -- from bowling a perfect game to singing an operatic aria without error. Addition and subtraction are skills you possess, even though you may not have great math ability and, in fact, your mathematical aptitude may be low.

In one of your articles you mentioned "not more than 10 percent of the expenditures actually result in transfer to the job." Does this mean:

Transfer rate of 10% was achieved because proper analysis was conducted (including transfer strategies)?
We should have invested the 90% to improving work environment factors as training was obviously not the ideal solution?

Wonderful question. There are various percentages published concerning the amount of training that "sticks" and is applied back on the job. Depending on the question posed, the research numbers vary. In general, what we see is that when training courses or programs "teach" a host of content and then later surveys are carried out to determine how much of what was taught is actually being applied back on the job, the percentages are very low. Let me give a couple of examples.

Timm Esque and Joel McCausland (Performance Improvement Quarterly, 1997, pp.116-133) investigated the transfer of a skill set that 400 managers at Intel Corp had been trained on. They asked the managers for examples of application. Approximately 20 percentsaid that they had used this skill set (Breakthrough System) in their work. However, when they went out to substantiate the reported use, they only found fourexamples (one percent). In this case, the form of transfer was percentage of persons using a skill set taught in a training program. Other studies have focused on how much of what was taught was applied on the job. Most have used self or supervisor reports as the means for measuring transfer. Time frames for doing this have varied considerably, from days to months. Brenda Cruz (Performance Improvement Quarterly, 1997, pp.83-97) has questioned this method and demonstrated that it is a poor form of verification compared with actual observation. So, I must admit that there is a great deal of confusion in what we mean by the word, "transfer" as well as how we measure it.

Despite this confusion, what we do learn from the accumulation of writings and studies is that little of what is taught in training programs gets noticeably used back on the job. Often this is due to the following: the people sent to training are not in a position to apply what they are taught (in one company we found that 30 percent of persons enrolled in training classes would never be able to use what was in the class - improper enrollment), insufficient resources back on the job, training not aligned with job reality, supervisors unaware of what is included in the training, lack of on-job support and the list goes on.

To summarize and respond to your question, which I assume really meant to ask "because proper analysis was not conducted," my answer is that so little transfer occurs due to a host of reasons, inadequate analysis among them as well as useless investment in training if, to start with, the problem was not skills and knowledge related. I also fault researchers for not being more specific in their definition of transfer to the workplace and organizations in general for either not measuring application of what the training programs intended for people to do differently back on the job, for relying on weak measures such as self-report and especially, for trying to solve with training, performance problems that are not skill and knowledge based.

You and I have a challenge here. Let's do more to change the situation. We can encourage organizations to conduct better front-end analyses, establish appropriate metrics and then systematically verify "transfer" in a variety of ways - through observation sampling and by examination of the results of the required transfer. We can also encourage researchers to be more precise in their definitions and to provide tools and methods for easier transfer verification. Until then, we will have to live with the estimates of 10 to 30 percent global numbers, meaning "not very good."

Can HPT be applied in an educational organization?

The answer to your question is, "Of course!" While HPT has grown up in the world of work, there is no reason why its fundamental principles and many of its processes cannot be applied to both educational and social settings. What happens, however, is that the educational establishment is often uncomfortable with the tough talk HPT uses. Frequently, educators hide their fuzzy thinking behind flowery and socially acceptable phrases to avoid confronting what it truly takes to achieve valued accomplishments.

Let me share with you two experiences in non-business settings where we used HPT effectively. In the first case, one of my doctoral students did a study using HPT principles to help decrease rehospitalization rates and medical emergencies and improve the quality of life in the chronically ill aged. In a second instance, a few students and I applied HPT to improving the performance of workers and volunteers in a home for the aged by getting them to apply a "Bill of Rights for the Elderly."

In educational settings, Joe Harless has been strongly advocating application of HPT. He wrote a book called the Eden Conspiracy which deals with this issue.

So, to conclude, there is no reason other than political pressure and social inertia that should inhibit applying HPT to non-business and industry sectors.

Despite your concerns, I do see progress. Let me step my way through this slowly and carefully. First, you have to take a long-term perspective on fundamental as opposed to superficial change. Here's the distinction. TQM blew in like a storm (and now six-sigma). It was high profile for a while and then faded out. Something remains, but the terms and the specifics have largely vanished. HPT does not offer dramatic salvation . It is a field of professional practice. It is an evolutionary change from single focus approaches that are more concerned with solution than cause. It is certainly not "sexy." So we mustn't expect the same excitement and enthusiasm engendered by the other "miracle cures" that have come...and gone.

So what are the signs that HPT is healthy and growing? As I mentioned in my interview for PerformanceXpress: increased number of ISPI members; great international activity; increased number of publications; more training groups calling themselves something other than training; number of training groups trying to integrate HPT concepts into their daily routines..stumbling around very often, I admit. As I work with different organizations, I see more sophistication in their analyses and solutions than ever before. Definitely, there is heightened awareness of and interest in HPT (or as many call it, HPI).

Okay, now for the hard facts. If all of this is true, do we see changes in the corporations and lots of HPT cases? What about ROI? For the first of these two, you are not always going to see cases. Someone asks for training. A "performance consultant," so named formally or by the person's activities, does a front-end analysis (checks out the legitimacy of the request) and finds that something else is required. S/he helps the requester to see other possibilities other than training or in addition to training. A different set of interventions emerges. Nothing dramatic, but this is HPT in action. Of this, I can give you many cases done internally. Deciding to select rather than train. Developing job aids to replace or complement training. Cleaning up job expectations. Setting standards. Changing the work flow or process. This happens all the time. These are rarely documented.

About ROI. This is a problem that goes beyond HPT. In the HR field in general, in training and in HPT, we are constantly pressed to undertake the next venture before we have completed the last one. We don't take the time to conduct ROI studies/calculations. In the 2002 ASTD Industry Report, even though demand from management for ROI results in training was the number one trend, the data on what they call Level 4 evaluation shows a downward trend line over the last ten years in the percentage of organizations doing it. Why? Pressure to move on to the next thing. It ain't glamorous. "No one will believe me." "I don't know how to do it." Doing ROI calculations is not that hard. Erica and I have just finished a toolkit on front-end analysis and return on investment that will be published by Pfeiffer in April 2004. The guidelines and templates are pretty easy to use. Jack and Pat Phillps have been writing, teaching and preaching ROI for years. The guilty parties are the internal training or performance support groups. I have had the devil of a time getting my clients to spend some money and energy on ROI.

To conclude, I see lots of signs of progress for HPT. The weaknesses lie in two areas. One, lack of effort to do the ROI work. This is bad. We don't show the value of our interventions and then we complain when no one sees how good they were. The second is that we are lousy marketers. We wannabe HPT professionals. We don't take the time to define what that means, tool ourselves up and then let our clients know about our services through brochures, demonstrations, lunchtime show and tell activities, and write-ups in internal newsletters...with data.

Yes, times are slowly changing. It's up to the internal groups - with help from outside specialists, sometimes - to take the initiative to advance the cause of HPT within their organizations.

What is the difference between skill and competency?

Although often used interchangeably, there is a practical reason to recognize the difference between the two terms. "Skill," if you hunt the word in the Oxford Complete Dictionary (British) or the Random House Unabridged (American), has a host of meanings. With respect to the workplace, the key ones are: capability of accomplishing something with precision or certainty (Oxford); the ability, coming from one's knowledge, practice, aptitude, etc. to do something well (Random House). The important point is that "skill" suggests the ability to do something. If you play the guitar well or speak a foreign language fluently, you posses skills. "Competency" and its virtually synonymous cousin "competence," while close in meaning to skill contain an important nuance: "sufficiency of qualification, ability to deal adequately with a subject [bears a relationship to the word compete] (Oxford). The word competent is defined as "possessing the requisite qualifications." Random House offers "possession of required skill, knowledge, qualifications or capacity... based on meeting of certain minimum requirements" and "competent" is defined as "properly qualified."

Why are these definitions so important for us in the performance improvement field? Because there is a great deal of confusion in terminology and as a result, some fuzzy thinking and acting.

Here is what I propose. "Skill" is something you are able to do. "Competency" (or competence) is an ability to do something that is required by a job or task situation. Ability to juggle balls is a skill. A clown's job in the circus requires competency in ball-juggling. You assess a person for skills. You define a job in terms of required competencies.

Competencies are, therefore, what is required to do or perform with respect to external requirements. They state what is essential to the job.

Please don't confuse competency with performance, which is defined as behaviors that result in valued accomplishments, or characteristics, the traits or attributes people possess.

So much similar terminology and yet it is important to create a precise vocabulary in Human Performance Improvement. This is a requisite of a scientific and professional field.

For more terms and definitions, please visit the Terminology Lexicon.

When conducting a needs analysis, what are the most important questions to ask?

What you ask is not so easy to answer. A lot depends on the circumstances of your needs analysis, in other words, at what stage you are asking the questions and how much information you already possess about the specific context and performance gaps. Sometimes, also, your data comes from observation or business results. Let's start at the beginning.

If you review one of the previous questions on our Website in the Ask Harold section, you'll find that I discuss needs analysis. There are a whole host of terms used that are similar in meaning, but may vary in practice. I assume you are interested in improving workplace performance. Therefore, the first question is one you ask yourself. Am I doing this analysis proactively or reactively? If you are operating as a performance consultant within your company and have a set of clients for whom you are responsible to provide learning and performance support, your questions will focus on how well people are currently generating desired results and on what changes are anticipated that will have an impact on what people do and how they do it. These are questions you would ask of your clients and their management. You should be constantly scanning your client universe for performance gaps and changes. As you spot anomalies, you must ask questions to verify what you notice and determine the implications.

When someone comes to you with a request for training or other intervention, then the pattern of questioning is different. You must ask questions to clarify the request first. What outcomes are they seeking? Then, you probe to identify the business need that triggered the request. Your questioning must go beyond those directed to the client. Based on existing data, documents or business decisions and expert, management, customer and performer input you must establish in operational terms what the ideal state should be. Through further questioning, observation and data gathering, you have to determine what the current state is. Even if a new system is being introduced, you might ask questions about previous new system implementations and what problems occurred for performers back then as well as what parts of the previous system never got implemented. This provides clues for new implementations.

So, to conclude my response, the questions you ask depend on the nature of the needs analysis, what triggered it and how informed you already are about the performance issues. There is no magic set of questions.

Do you have any additional information on the topic of transforming performance capability?

Yes, a whole library full of material. A good start would be to join the International Society for Performance Improvement (ISPI). Their Website is www.ispi.org. Check for local chapters. Start reading their publications. Two good books for your bookshelves are: Handbook of Human Performance Technology (editors H.D. Stolovitch and E.J. Keeps). It is available through HSA's Website at book order form, through ISPI's Website at www.ispi.org or from Amazon.com at www.amazon.com. I also like Moving from Training to Performance: A Guidebook (editors Dana and Jim Robinson). It is available through the American Society for Training and Development's (ASTD) Website at www.astd.org or from Amazon.com.

I am most interested in supporting my managers in having a discussion with their employees pre and post training. Do you already have a template of what this discussion might look like?

I do not possess a template. However, I can offer recommendations. Before training, I suggest you send a note to the managers/supervisors informing/reminding them that "next week, XXXX will be attending a XX hour/day training session. Here is the rationale for the session and for XXXX's participation in attending. Here are the objectives and an outline of the session. Please schedule a 20-minute meeting with XXXX. (This could include one or several persons participating). During the meeting, discuss the following:

The reason why XXXX is/are attending training.
What they should anticipate during the session.
How you expect them to participate during the session (which may include questions they should ask).
What you expect XXXX to bring back and apply upon returning from training.

Set up a time for a post-training meeting shortly after the training has been completed to discuss more specifically application to the job.

Following the training, meet again at the pre-selected time. Debrief the training with questions such as:

What key points or issues were raised during the training that is relevant to our group and your job?
What did you learn that we can integrate and/or that you can apply immediately? Soon?
What do you require to apply what you learned?
What can I do to help you make it happen?"

The "Learning and Performance Support" group must help managers and supervisors to see the value of this, train and support them, and monitor the results. This should help you get started.

Why is it so hard to apply a few proven best practices in our work environment?

The answer is relatively simple. Implementation of a solution is far more complex. First, there is inertia in any system. We tend to perpetuate habits and frequently exhibited past behaviors. Even though new, credible information convinces us of the necessity to change, we persist with old, counterproductive patterns of behavior. A simple example: we have all received plenty of information on proper food/eating habits, nutrition and fitness. We know what we should eat and that we should exercise regularly. So, do we implement these "best practices?" Some of us do; some of us try; many of us nod our heads and then continue with what we know are inappropriate ways.

How to change this? We have to begin with the environment. Clear, valued, meaningful expectations, timely, specific feedback and consistent, persistent support are necessary. Appropriate infrastructure and adequate resources are also essential. Meaningful incentives and consequences must be put in place and administered fairly. Where skills and knowledge are lacking, training and coaching are required. In the work setting, careful selection of employees whose background and characteristics align with performance requirements increase the probability of success (e.g., they already eat healthfully and exercise regularly). And we must ensure that we truly value these new practices we desire, have confidence to implement them and have patience to support approximations toward the goal. It's a tough job, demanding commitment and discipline on the part of senior managers (not just lip service), but it can be done.

How do accelerated learning and brain-based learning relate to the emotions of fear?

The fundamental problem with accelerated learning is that it is not what one would call a well-defined construct (something that is not directly observable or objectively measurable but is assumed to exist because it gives rise to measurable phenomena). A number of people use the term toward a number of ends. It's like "learning styles." What exactly does it encompass?

In accelerated learning, there have been many trends. Work done by the Bulgarian Psychiatrist, Lozanov, was placed in this approach with his "suggestology" and "suggestopaedia." It created a sensation in the learning of second languages but did not have staying power once scientific research investigated learning beyond the basics.

Other contributors to this movement have been the split-brain theorists. Once again, the explanations they offer have proven too simplistic from a neuro-scientific perspective. The brain is far more complex than simply dividing it along hemispheric lines. Triune brain theory, likewise does not handle neuro-scientific research very thoroughly although it borrows from this literature. Much is being made of Gardner's multiple intelligences, but how this affects learning is still unclear. Other contributors have been those involved in creativity and in cooperative learning...and the list continues.

This is what I mean by accelerated learning not being a solid scientific construct. Thus it is difficult to know what precisely we mean by it and how it works.

I recommend reading more about how the human brain (and mind) processes information and how it perceives and retains information for later retrieval. The two most fruitful areas in my opinion are in cognitive psychology and in the neuro-sciences themselves. Not easy reading, but more valid and reliable for truly understanding learning processes.

Ellen Gagne's book (along with the Yekovitches) is a good starting place for cognitive psychology. Robert Ornstein has written a lot in a fun way on the evolution of consciousness. It's easy reading and will get you started.

What is the difference between a training needs analysis and a performance analysis?

Training needs analysis is quite a tricky term. It suggests that you are out hunting for gaps that can be filled by training. I, personally, am uncomfortable with this term. Why? Because it suggests a solution in search of problems. The literature on human performance at work suggests that training, by itself, rarely is sufficient to create lasting workplace change. It can be an essential part of an integrated set of interventions, but alone, has not got the staying power to make a significant, long-term impact except in a very few instances. Read about transfer of training to the workplace and you will soon see the discouraging results.

Performance analysis, a term coined by Gilbert and Rummler, is more investigative and comprehensive. Performance analysis is a methodology that permits you to identify and analyze performance gaps, to identify the array of factors influencing the gaps and to determine the range of interventions required to eliminate the gaps.

I strongly recommend the latter approach. I, myself, like Joe Harless' term front-end analysis, mostly because of the chronological sense it offers. It's what you do up front to determine what the business requirement is, what desired and actual human performance is, what the magnitude, urgency and value of the gap is, what factors affect the gap and what interventions are appropriate, economical, feasible and acceptable organizationally and to the performers. It is a wonderful means for helping to solve human performance problems.

I recommend that you view yourself as a performance consultant more than as a training person and that you conduct performance analyses or front-end analyses. They will allow you to have a greater impact than with a training needs analysis.

When learners know why they are supposed to learn something (a rationale with a credible benefit to them), the probability of learning and retention increases.
When learners know what they are supposed to learn (a clear, meaningful objective), the probability of learning and retention also increases.
If what is to be learned is clearly structured and organized so that the learner easily sees the organization and logic, again, learning and retention probabilities increase.
If learners have an opportunity to actively respond and engage in the learning in meaningful ways, once again, learning and retention probabilities increase.
If learners receive corrective and confirming feedback with respect to responses they emit or activities in which they engage, their learning increases along with retention.
Finally, if the learner feels rewarded for the learning, has a sense of accomplishment or is given an external recompense for the learning that he or she values, learning and retention have an increased probability of occurring.

With respect to being more visual or auditory, while there may be significant differences among learners, more important, however, is stimulus variation.

If I understand this question correctly, you are talking about criterion-referenced testing. You create a task analysis that focuses on what people have to be able to do as a result of the instruction, develop performance objectives based on the task analysis and then create test items that perfectly match the objectives. If as a counter person in a delicatessen, you have to be able to slice a bagel into two perfectly equal halves with smooth surfaces and no injuries, you develop a test item that requires demonstration of this precise task performance along with a checklist or observation instrument to verify performance to standard. This is also true for firefighting, analyzing poetry or performing a swan dive. Two wonderful resources are Robert Mager's book, Measuring Instructional Intent, published by ISPI and William Coscarelli and Sharon Shrock's book on criterion-referenced testing also available at www.ispi.org.

What is the difference between organizational effectiveness and human performance technology (HPT)?

All organizations want to be effective. How they define effectiveness varies, especially over time. Going back to the early part of the 20th century, management pioneers such as Frederick Taylor defined effectiveness as being the result of production maximization, cost minimization and technical excellence. Henri Fayol viewed effectiveness as the outcome of clear authority and discipline within an organization. Elton Mayo humanized the concept of organizational effectiveness. He defined it as productivity resulting from employee satisfaction.

Today, we tend to view organizational effectiveness as the ability of the organization to be effective in accomplishing its purposes, efficient in the acquisition and use of scarce resources, and a source of satisfaction to its owners, employees, customers and society. Organizational effectiveness is also concerned with the organization's ability to be adaptive to new opportunities and obstacles and capable of developing the ability of its members and of itself to meet new challenges. In the long term, organizational effectiveness is about the organization surviving in a world of evolving uncertainties.

To summarize, organizational effectiveness is the ability of an organization to fulfill its mission through a blend of sound management, strong governance and a persistent rededication to achieving results. It includes meeting organizational and stakeholder objectives - immediately and long term - as well as adapting and developing to the constantly changing business environment.

We measure organizational effectiveness in a variety of distinct yet interrelated ways: in the organization's ability to:

Accomplish its stated goals.
Acquire needed resources.
Satisfy all strategic constituencies.
Combine internal efficiencies with affective health.

Criteria for organizational effectiveness include:

For owners and shareholders: return on investment; earnings growth.
For employees: compensation; benefits; job satisfaction.
For creditors: satisfaction with debt payments.
For unions: satisfaction with competitive wages and benefits; satisfaction with working conditions; fairness in bargaining.
For local communities: involvement in and support of local affairs; environmental respect.
For government agencies: compliance with laws; avoidance of penalties.

Professionals in human resources and organizational development are most directly concerned with helping to facilitate organizational effectiveness. They generally assist in the following:

Defining organizational direction.
Creating appropriate organizational structures.
Defining required organizational skills (generally non-technical).
Defining organizational processes.
Improving communication and building trust.

Hence those involved in organizational effectiveness focus on the current and future overall direction, functioning and health of the organization.

Human Performance Technology (HPT), as its name suggests, is a "technology," defined as the application of scientific and organized knowledge to achieve practical outcomes. Its focus, within the organizational context, is the achievement of desired performance from people. HPT tends to deal with concrete, specific performance gaps. While organizational effectiveness is a desired state, HPT is a field of study and practice. HPT practitioners are professionals who work with organizational clients to identify gaps between desired and actual performance states whenever these occur (technical as well as non-technical) and then measure these gaps in terms of magnitude, value and urgency. They identify factors affecting the gap and recommend interventions to eliminate these gaps. In many instances, they design, develop, help implement, and monitor and maintain performance interventions.

HPT provides processes, tools, principles and clearly defined methods for operating at every stage and step of performance improvement. The results of applying HPT are verifiable through concrete measures.

To conclude, organizational effectiveness is about the overall functioning of an organization. HPT is about engineering effective human performance in specific ways. There is an obvious link. To achieve organizational effectiveness requires the efforts of many managers and professionals. As gaps are identified, the HPT professional can play an extremely valuable role in investigating these and creating solutions. HPT is one of the key professional means for achieving organizational effectiveness.

What is the difference between Organizational Development and Human Performance Improvement?

This is a frequently asked question and understandably so. Both fields are aimed at improving the performance of people and organizations. However, there is considerable difference in style and practice between the two. Organizational Development (OD) adopts a very broad-brush approach to overall organizational performance improvement. It generally operates at a macro-level. It is most frequently employed when there is anticipated or actual, significant organizational change. Human Performance Technology (HPT) is more of an engineering discipline. It focuses on identified gaps in human performance and seeks to eliminate (or at least reduce) these. Here are a few definitions of OD:

OD is the field of study and practice that focuses on various aspects of organizational life, aspects that include culture, values, systems and behavior. The goal of OD is to increase organizational effectiveness and organizational health, through planned interventions in the organization's processes or operations. Most often, OD services are requested when an organization (or a unit within an organization) is undergoing a process of change. OD is a planned effort to improve an organizational unit and the people who comprise it through the use of a consultant and sets of structured activities dealing with organizational health and effectiveness.

OD is a distinct consulting method that focuses on the people, culture, processes and structure of the whole organization. A primary goal of organizational development is to optimize the system by ensuring system elements are harmonious and congruent. Performance suffers when structure, strategy, culture and processes are misaligned.

Compare these to the following typical definitions of HPT:

HPT is a set of methods and processes for solving problems or realizing opportunities related to the performance of people. It may be applied to individuals, small groups or large organizations.

HPT is an engineering approach to attaining desired accomplishments from human performers by determining gaps in performance and designing cost-effective and efficient interventions.

HPT is a field of study and professional practice aimed at engineering accomplishments from human performers. HP technologists adopt a systems view of performance gaps, systematically analyze both gaps and systems, and design cost-effective and cost-efficient interventions that are based on the analysis of data, scientific knowledge, and documented precedents, in order to close the gaps in ways all stakeholders value.

Note that both are concerned with improving performance. They are in many ways complementary. However, the words and style of the definitions immediately reflect the fundamental differences between the two approaches. OD deals with the overall health of the organization. It comes into play when the organization is not functioning properly or will have to alter its way of operating. HPT focuses on identifying and closing specific performance gaps. It springs into action when specific results are not being achieved. OD emphasizes organizational effectiveness. It is generally called upon when a major change affecting the entire organization is anticipated or occurring. It relies on consultant intervention, bringing together all the affected players and groups, and facilitating communication and decision-making. HPT is applicable to any human performance gap, whether these result from change or fr