JULY 2016
Volume 15, Number 3

To contact us, click HERE.


For more information on our best-selling, award-winning books, click HERE.

Need help with HPT terminology? Click HERE for The HSA Lexicon.

Click HERE to read our published articles.


Click HERE to read the latest Ask Harold question and Harold's response or ask a question of your own!


July 28-29, 2016
ATD Telling Ain't Training Event, Los Angeles, CA

November 3-4, 2016
ATD Telling Ain't Training Event, Chicago, IL

For details about these events, click HERE

To learn more about engaging Harold Stolovitch to speak at your organization, click HERE


Harold Stolovitch and Erica Keeps are Certified Performance Technologists (CPT). The CPT designation is awarded by the International Society for Performance Improvement (ISPI) to experienced practitioners in the field of organizational performance improvement, whose work meets both the performance-based Standards of Performance Technology and application requirements. For more information, visit www.ispi.org.

Our Professional Family

We individually joined the International Society for Performance Improvement (ISPI, formerly known as NSPI) in 1975 and both served on the Executive Board a few years later at the same time. We met each other through our professional organization. Interestingly enough, several other couples have met at ISPI as well!

Over the years, we have grown through our professional organization and have contributed to our society as board members, committee chairs, presenters, mentors and more. We were both active in our local chapters, served in leadership roles and became life members.

Through ISPI we met incredible colleagues who became friends. We attended numerous ISPI conferences where we had an opportunity to maintain our ISPI friendships and professional relationships while building our skills and sharing our work with others.

The ISPI Conference in Philadelphia this April was no exception. We had a wonderful time seeing old friends and meeting new ones. We were inspired by the leadership. We were thrilled with the up and coming graduate students. They will become leaders and contribute to the society while growing and developing their skills and careers. They are the future of ISPI.

We look forward to seeing "our professional family" again in Montreal for the ISPI Conference April 30-May 2, 2017. We hope to see you all there.

Join the family! If you are not already a member of ISPI, what are you waiting for? Learn more about membership benefits.

Erica and Harold


Harold's Reading and Viewing Guide
By Harold D. Stolovitch

Here are three very different readings that have caught my attention over time. You can access them by simply clicking on the image beside each. Enjoy the reads - very well worth your time.

This is a brief, intriguing book written by New Yorker business columnist James Surowiecki. It explores a deceptively simple idea: Large groups of people are smarter than an elite few, no matter how brilliant/better at solving problems, fostering innovation, coming to wise decisions, even predicting the future.

The book is easy to read and travels across a wide range of fields from popular culture, psychology, ant biology, behavioral economics, artificial intelligence, military history, and politics, to name only a few as it demonstrates how the combination of individual, free choice, when aggregated can make better and more accurate decisions and predictions than subject-matter experts. Surowieki not only provides evidence for his book's assertions, but also offers ways we can use his principles in our own day-to-day work. A fun and thought-provoking volume that both informs and entertains. Available on Amazon.

A report by the Chartered Institute of Personnel & Development, University of North Carolina, found that evaluation was a top priority among learning and development (L&D) professionals. Despite this, however, the authors also found that calculating return on investment (ROI) on learning and development programs is still rarely done; a McKinsey Quarterly report, for example, found that only eight percent of organizations actually evaluate the value of L&D initiatives. And too often, those who do track ROI rarely go beyond asking for feedback from participants immediately after the event. Common explanations given for this lack of evaluative effort are: senior management doesn't really care; don't know how to do it; you can't really calculate ROI or even create a clear connection between L&D initiatives and bottom line results. Frequently, the real reasons revolve around rushing from one project to the next, with no time and resources dedicated to verify if what was done had the desired, or even any, impact. All of this leads to little effort invested for measuring results and ROI of L&D activities. Smiley sheets prevail even though, long ago, researchers established a virtual zero correlation between the data generated by these and meaningful outcomes.

This brief report, easy to read and implement, offers a good start toward changing this mentality and practice. While the reader will require more technical guidance to operationalize and implement effective evaluation practices, including calculating ROI, it should get the L&D professionals' juices started at working to obtain better and more accurate readings on the results of their activities and accomplishments.

What a provocative, yet very well documented book this is! It presents a powerful attack on the latest pseudo-scientific claims about the differences between the sexes - with the scientific evidence to back it up. Sex discrimination is supposedly a distant memory. Yet popular books, magazines and even scientific articles increasingly defend inequalities by citing immutable biological differences between the male and female brain. Why are there so few women in science and engineering, so few men in the laundry room? Well, they say, it's our brains. Drawing on the latest research in developmental psychology, neuroscience, and social psychology, Delusions of Gender rebuts these claims, showing how old myths, dressed up in new scientific finery, help perpetuate the status quo.

Cordelia Fine reveals the mind's remarkable plasticity, shows the substantial influence of culture on identity, and, ultimately, exposes just how much of what we consider "hardwired" is actually malleable. This startling, original and witty book shows the surprising extent to which boys and girls, men and women are made - and not born. Available on Amazon.

Now Available! Training Ain't Performance Webinar: Adding Value as a Performance Improvement Professional

Due to popular demand, Harold now delivers his two-day workshop in a five-session webinar. Ideally, the webinar series is delivered once a week for 2.5 hours per session over a period of five weeks. This 12.5 hour course (with approximately four hours of homework) is now an affordable option for organizations with a decentralized employee group who can benefit from performance consulting skill development, but cannot meet on site for our in-house workshop. Participants will acquire the skills and tools to immediately begin functioning as Performance Improvement Professionals. The sessions are highly interactive and entirely focused on workplace application. An added benefit is that participants get to experience a truly interactive virtual course. They can replicate the process with their own content.

The NEW webinar series is based on our books, Training Ain't Performance and Beyond Training Ain't Performance Fieldbook, as well as our many years of experience conducting performance improvement projects and training/coaching/mentoring performance consultants worldwide and across many industries.

HSA can also provide support and mentoring following the sessions.

For a course description and intended audience, click here. For a detailed agenda, costs and scheduling, contact Erica Keeps at ekeeps@hsa-lps.com or 310-497-8466.

Personal Musings: One and One is NOT Two
By Erica J. Keeps

The back, back story: Years ago when Harold and I first started dating and discussing marriage, we wondered what it would be like to work with our spouse long term? Would we choose to be generalists or each specialize in specific aspects of our work? Would it be best to have an outside office and separate our professional work from family life? Were we better suited to work from home and share the same space day and night. (BTW, I love being in a noisy environment and have several balls in the air at the same time. Harold needs silence and is very linear.) Would my clients accept him? Would his clients accept me? We actually visited with a number of married colleagues in our field to see how they made it work. We learned a lot about what to do…and what not to do!

The back story: We decided we could make it work! We got married, merged our consulting practices and chose to work from home. Fortunately Harold had a university office to run away to when he needed absolute quiet. Later, once he left the university, he used the local library when he needed to focus on writing. Ultimately, we bought a second home…8.5 miles from our primary one! Harold refers to it as his "private writing retreat."

Initially, we offered our clients a two-for-one sale. Two consultants for the price of one. They were thrilled with the financials, and we were happy to be able to travel and work together. Soon our clients felt comfortable interacting with either one of us, which was, of course, our plan.

When we wrote together, Harold would come up with the article or book concept. He would then interview me to create the structure and key points. He would write a detailed outline, which I would then edit. Harold and I would draft assigned chapters and would then edit each other's work until we no longer could identify who wrote what. Once I wrote an entire audio product knowledge script for a high tech company modeled after one he had done. I read it to him and he seriously asked when he had written it!

The story: We learned I was better at managing the business while Harold was best at the content and creativity. I took over marketing, sales, legal, accounting, human resources and project management. This left Harold free to do research and development, write proposals, do instructional design and performance consulting. We were a team. We had synergy. We had complementary skills and some overlapping ones. Decisions we made together were always more thoughtful and sound than those made alone. However, we each had final say in our own areas of responsibility.

We built a business from a two-person professional practice to an international consulting firm with 80 associates. We did bigger projects with larger organizations. Our model was working! Then 9/11 brought us to the realization that if we didn't scale back, we could lose our practice and retirement savings. Client companies were halting travel; consulting projects, workshops and conferences were being cancelled. We wisely chose to follow our accountant's advice and go back to a two-person professional practice. Full circle!

The moral to the story: One and one is NOT two; it's a whole lot more! What we did together, neither of us could have done alone. What we accomplished was more than we ever expected.

Two important concepts we have incorporated into our lives are quality time together and shared independence. The first of these simply means: Make activities you do together, whether they are for work or play, meaningful and enjoyable to both of you. The second focuses on demonstrating respect for each other's individual differences when doing things apart. Share the independent experience with one another, emphasizing the learnings that emerged.

We have ended the prefaces to every book we have co-authored with the following message:

"One of the greatest joys of writing a book in which you share what you have learned with readers is that of sharing the experience with a co-author you admire. We are not just co-authors; we are also professional colleagues and life partners. To each other, then, thanks for continuing the journey together."

There is a long-standing adage that "two heads are better than one." We would like to modify it, asserting that "two heads are substantially better than two." Our over thirty years of collaboration attest to this.

Guest Author Series

From time to time, we encounter interesting people in our work whom we feel have something important to share with others. Our Guest Author Series includes writings and outright articles by these professional colleagues. The latest contribution in our series is by Will Thalheimer, PhD. He is a consultant and research translator, providing organizations with learning audits, research benchmarking, workshops, and strategic guidance. Will shares his wisdom through keynotes, research reports, job aids, workshops, and blog posts. Compiler of the Decisive Dozen, one of the authors of the Serious eLearning Manifesto (eLearningManifesto.org), founder of The Debunker Club (Debunker.Club), and author of the highly-acclaimed book Performance-Focused Smile Sheets (SmileSheets.com), Will blogs at WillAtWorkLearning.com, tweets as @WillWorkLearn, and consults through Work-Learning Research, Inc. (Work-Learning.com). Will regularly publishes extensive research-to-practice reports and gives them away for free. His clients have included a wide variety of organizations, including the Navy Seals, MIT, ADP, Microsoft, Walgreens, DIA, Pfizer, and Liberty Mutual.

The Smile-Sheet Blind Spot
By Will Thalheimer

Smile sheets, happy sheets, response forms, reaction forms. Call them what you will, they may be the most dangerous tool in the instructional design toolbox! Here are some of their problems (taken from the book Performance-Focused Smile Sheets):

  1. They are not correlated with learning results.
  2. They don't tell us whether our learning interventions are good or bad.
  3. They misinform us about what improvements should be made.
  4. They don't enable meaningful feedback loops.
  5. They don't support smile-sheet decision making.
  6. They don't help stakeholders understand smile-sheet results.
  7. They provide misleading information.
  8. They hurt our organizations by not enabling cycles of continuous improvement.
  9. They create a culture of dishonest deliberation.

Of course, there are many reasons we might use smile sheets (taken from the book and modified from the legendary Rob Brinkerhoff):

  1. Red-flagging training programs that are not sufficiently effective.
  2. Gathering ideas for ongoing updates and revision of a learning program.
  3. Judging strengths and weaknesses of a pilot program to enable revision.
  4. Providing instructors with feedback to aid their development.
  5. Helping learners reflect on and reinforce what they learned.
  6. Helping learners determine what (if anything) they plan to do with their learning.
  7. Capturing learner satisfaction data to understand - and make decisions that relate to - the reputation of the training and/or the instructors.
  8. Upholding the spirit of common courtesy by giving learners a chance for feedback.
  9. Enabling learner frustrations to be vented - to limit damage from negative back-channel communications.

In the book, I focus on the first four imperatives - those that provide us with feedback so we can improve what we're doing as instructional designers and trainers.

When I first saw the research showing that smile sheet results are not correlated with learning results - actually they are correlated with an r = 0.09 (virtually no correlation) - I thought, well, maybe we should just stop using smile sheets. But then I got real. Read more...

Readers and Participants Speak Out

"Harold, I greatly appreciate your perspective on our work. I respect and admire the dedication that you and your partner have had to training and development methods - especially because you've dedicated your life to it. Not many people can say that they've dedicated their life to something that I believe is both noble and useful.

Thank you for being an influence for the good of our team - both now and for the future. However the next few weeks play out, I'm confident that you have provided skills for each and every team member for their current job or potentially for another opportunity."

- Darlene Hampson, Senior Learning Advisor, Cenovus Energy Inc.

Do you have feedback on our books, workshops or our newsletter?
We would love to hear from you. Please share your comments
with us at ekeeps@hsa-lps.com.

Ask Harold

Do you have a burning learning and performance question? Visit ASK HAROLD and ask your question for Harold Stolovitch to answer. Here is a recent submission that might intrigue you:

Do you have any information on the effectiveness of short and spaced learning segments compared to longer, more concentrated learning sessions? I have seen a number of articles touting these.

To read the response, visit ASK HAROLD. To ask your own question, click on the crystal ball above, fill out the form and click submit.

Double Dipping: River Cruise and Mediterranean Cruise
By Erica J. Keeps

Isn't it staggering how expensive airfares have become? And, if you travel business class, you may have to choose between a trip or a new car! While not everyone can get away for longer periods of time, fortunately those who can, may elect to "double dip."

Harold and I did exactly that this year. We planned a river cruise to the south of France with colleagues and friends and then piggybacked with a family Mediterranean cruise with family. One round-trip air ticket, one battle with jet lag and two completely unique experiences!

River cruising is an adult activity. It is serene gliding along a river where you see land on both sides. Every day your small ship (with around 120 passengers) tucks into a port, you hop off and are met with a local guide to explore a city, town or village.

Erica and Harold enjoying an anniversary toast in Provence

River cruise companions, Jane and Darryl Sink shopping one of the many open air markets in the south of France

No days at sea and no sea sickness! Lots of walking, exploring and taking in the culture of the area. No need to choose what to see as the excursions are decided for you. You can just enjoy and appreciate the scenery and the highlights of every place you visit. Sounds wonderful, doesn't it? In spite of high water levels causing itinerary changes, train strikes and Euro soccer, we had a great time. Ocean or sea cruising is a different phenomenon. These vessels carry 1,000s of passengers, many of whom are children (even during non-school closing times). There is so much to do on the ship (for both kids and adults) that often passengers opt out of port excursions altogether.

Days at sea are welcomed! You are fed, entertained, exercised, spa-treated and can shop or gamble day or night. It is so much of a whirlwind that one cruise line places the day of the week in the elevator floor to keep passengers aware. You carry a ship activities schedule in your pocket or on your cell phone. You communicate with your family passenger members by intranet services on board and occasionally meet up for a meal or evening "nap." Sounds exciting, doesn't it? We had a fabulous time introducing our family to a taste of Europe!

Our ocean cruise family companions enjoying a day in Rome

Which do we personally prefer? Both! Each one has its pros and cons. We were blessed with two incredible vacations this summer.

Interested in learning more about cruising? Contact Erica Keeps at ekeeps@hsa-lps.com or 310-497-8466.




At HSA LEARNING & PERFORMANCE SOLUTIONS LLC, we've seen a lot over the years.  We know the business of learning.  We know the role human performance plays in business success.  We know how to uncover and address needs, then create appropriate solutions.  We pride ourselves on helping organizations achieve high levels of performance - and success.  HSA is a leader in workplace learning and performance improvement.  Our proven learning and performance solutions have helped maximize employee performance at hundreds of organizations throughout the world.  Our principals, Harold D. Stolovitch and Erica J. Keeps, share a common passion - developing people. Together they have devoted a combined total of over 85 years to make workplace learning and performance both enjoyable and effective. Their dedication to improving workplace learning and performance is reflected in the workshops they run internationally on training delivery, instructional design and performance consulting. Together, they are co-editors of the first two editions of the Handbook of Human Performance Technology and co-authors of the best-selling, award-winning series of books Telling Ain't Training - Updated, Expanded and Enhanced, Training Ain't Performance, Beyond Telling Ain't Training Fieldbook and Beyond Training Ain't Performance Fieldbook published by ATD Press. They are also co-authors of the Wiley/Pfeiffer Learning & Performance Toolkit Series.  Learn more HERE.


© 2016 Harold D. Stolovitch and Erica J. Keeps