Validation: The Need for Feedback

Posted: April 7th, 2014

Noll Note – Article by William T. Noll
March, 2014

When people ask me to explain our analysis process, my high school chemistry class comes to mind. I was fascinated by qualitative analysis, and I remember a particular experiment involving yellow powder. We were given a testing solution and two samples of powder, which looked almost identical, and our assignment was to determine which sample was sulfur and which one was merely colored powder. With the right test, it became very clear which one was sulfur.

The instrument we use in our behavioral profile test is made up of carefully selected questions, which we feel very accurately measure specific themes or elements that impact the behavior of an individual. These questions have been studied, adjusted and refined, some of them for over 70 years. About 60 percent of most of the characteristics we measure are established by age 10. We have had the opportunity to interview some very young individuals, and when they were interviewed again 10 to 15 years later, we found their results to be both consistent and accurate in their observed behaviors.

Continuing with the chemical analogy, our instrument, made up of these questions, accurately measures basic elements such as raw-creativity, assertiveness, competitiveness and many others. Once we have determined the presence of these basic elements, we look at groups of themes or what could be described as compounds. For example, a person’s ability to negotiate is defined by their assertiveness, persuasiveness and competitiveness. When these themes are present, the person is very capable of bringing others around to their way of thinking and will generally find themselves on the positive side of negotiations, hopefully ending in a win/win situation. To complete the chemical analogy, the next step is the formula stage, or the combination of themes that are necessary to do a specific job, and this is when things get tougher. It takes a lot of performance data collection to come up with the right combination of a complex set of ingredients and compounds.

And this is where we need your help.

Each job requires a different formula, each company may need a different formula, and each manager may manage better with a different formula.

It is said that salesmen are salesmen, but there are inside sales, outside sales, remote outside sales, and technical support sales, all of which may require different levels of talent and supervision. The person who requires what we call “sight-supervision” may be very capable in sales but needs to be held accountable and managed closely. Conversely, there is the person who is self-driven with a strong level of work orientation and ego-drive that gets them out of bed and on track every day, to the point where they do not like, need or want to be micro-managed. There are an infinite number of different styles that are effective in sales.

We find the same is true with management. There is the foreman or lead-person who may be pretty good at motivating people and may even be a fairly good teacher. Then there is a level where the person can develop and motivate others to get results. And at the highest level, there is the leader who has the management strengths to make critical strategic and tactical decisions. All these positions require different formulas.

Any time we start a new project, we try to get as many models as possible within each category, and we get the best information by having contrast, so we interview both top, successful models and poor, failure models. We suggest that our clients “force-rank” their models by asking themselves, “If I had to fire all but one person, who is the person I would keep?” That is our top person on the list. To determine the person on the bottom of the list, or the poorest model, we ask them who they would let go if they had to fire someone immediately and keep everyone else. It is a different way of thinking and can be a little uncomfortable for the managers, but it is amazing how quickly they can come up with the top person, and they generally know who the person on the bottom is, as well. It can get a little “fuzzy” in the middle, and sometimes going even two or three deep can spoil our data for separation and validation.

We regularly try to send out requests for feedback on performance by email, and when people respond, it can be very beneficial. We know it takes your valuable time, and we know it may even be a bit of a nuisance, but your feedback is extremely valuable in helping us help you. If someone you hired did not work out, we want to know why.

  • Did they leave on good terms and perform well while they were doing the job?
  • Were they unable to do the job, although we decided together that they could?

We also need to know where you see strengths and weaknesses. Generally, as soon as rejection sets in, it’s easy to see lots of things wrong with the individual, but if a strong positive connection is present, it can be hard to see flaws and easy to see strengths. We find that some individuals are not very perceptive of talent or capability, and they can be a little negative about people, while others are very accurate in how they read people. Nonetheless, even though evaluation methods may vary, the detailed information you provide is critical in improving the accuracy of the instrument, specific to your needs. Whether the person is succeeding or failing, we need to know, and we would like your help to know why.

We really want to continue to provide you with the best service we possibly can. If you are utilizing the instrument as a developmental appraisal, we want you to have an accurate tool that will provide you with coaching opportunities to help the person grow from strength and avoid “de-railers” that may cause failure. If you use the instrument for predictive analysis, we want to continue to refine it, so we can more accurately predict success and help you build your varsity team.