2005 - Panel on Capability Assessment Methods

Speaker A Start with Judy Holbrook from BiOps, followed by Glenn Meltredder, who's with People Fit. And then Jerry Cranes is with the Leventon Institute.

Speaker B So I will turn it over to Judy.

Speaker A So you really don't need to hear from me.

Speaker B Thanks you. As Katie said, what I would like to do can you hear me at the back? What I would like to do is not just to describe CTA and what we do, but talk a little bit about the way in which we use it. So within the Bio community we are bound by a very strong sense of values. And it comes to global, one of the things that does it binds us together. So we're very clear about the ethics that we work with and the values and respecting the dignity of individuals and also valuing the diversity of the communities in which we work. And I think the other important thing is when we're talking about individual capability, is that people need to have they have a choice about how they use their capability. They don't have to use their capability at work, they can use it in any number of other ways. So we need to be aware of that. We need to be open minded and we need to encourage the integration of practice, theory and research as an ongoing continuing process in terms of the application of the capability evaluation, we have a global client base that's growing. And one of the things that we have done is to ensure that we apply that in a consistent way and have developed a set of global processes that set standards for training, accreditation and for quality. So we work both with the individual and the organization always working within the individual context. The only caveat around that might be some of the recruitment work we do. We're recruiting for an organization that we know and involving the line managers who do have overall accountability for the development of people. And the work that we do contributes to talent bank succession planning, development plans, coaching, development, recruitment and selection. One of the ways of looking at this, and we've talked a lot, I think during the last two or three days about the challenge bit, is the work that we're asking people to do. And this session is focusing on capabilities and the importance of bringing those two together in a way that enables people to have a sense of flow about what they're doing so that we're not keeping roles too small for them for too long so that they fail. And that kind of underpins a lot of the work that we're doing. And the other thing is to think about capability in terms of what we call a jigsaw potential. And we can talk about applied capability, but it's converting that often raw capability into effective performance. Because I'm sure everybody knows very capable people who we say can't fight their way out of a brown paper bag because they don't have all the other attributes that are needed. So it's important to look at the total picture. When we're talking about individual capability, we're talking about the way that people pattern an order and their world. It enables people to make judgments and decisions when experience and knowledge alone don't necessarily give them the answer, data doesn't give them the answer. There's a judgment that needs to be to be made. And as we all know, capability continues to grow. That's the good news for all of us, I think with the models that we use, but it's not ruled out yet. After 21 actually our capability does continue to grow and it does grow at different paces and it unfolds along consistent path. And as we talked about in the jigsaw, it's the necessary but not sufficient within bias. We have a set of tools that look at capability career Path Appreciation, which I think is probably one that most of you have heard of, and those of you that have been in other sessions will have heard of the modified Career Path Appreciation which is an online version. And I'll come on to what the difference is shortly. And then we also have Iris, which has been designed on the same platform, but is a very structured interview primarily for people to use in house, with graduates and with people who are in shop floor roles or first line supervisors. And that has structured questionnaire with a.

Speaker C Flooring.

Speaker B Because we just put that online primarily to address our markets in China and in go back to a bit of history, because Casey alluded to this. It was in the 1970s that Elliot asked Jillian to develop a measurement capability tool and that led to the development originally of the symbol card task. Further developments in the 80s led to the development of the phrase cards which form the key part of both graphos appreciation and the modified MCTA today. And the term Appreciation chose very deliberately to reflect the spirit in which the process was used. If we go back to pro conspicuation, it is a one to one guided conversation and it offers people the opportunity to explore the way that they make decisions, the way that they frame their world. And it consists of a number of parts. So there are a set of phrase cards and I'll show you an example.

Speaker C Of one in a moment for those.

Speaker B Who aren't familiar with that. We also look at preferred approach to problem solving. The degree to which people work very conceptually or very pragmatically, the degree to which they use intuition in that problem solving process. We also explore their career path today and looking particularly for significant transitions because clearly people's growth from one level to another doesn't just take place overnight. It is a gradual process and depending on the context in which that transition is occurring, it will depend on how easy that transition is for someone. And then there is the feedback process. So if we look at the core of preopath appreciation itself, we have nine sets of cards. Each of the cards represents a different level of complexity. And what we're doing is asking people to choose a card that best reflects the way they like to work and also to perhaps choose a card that they don't agree with during that process. We also will explore with them whether there are phrases that reflect the way their job requires them to work. So it's starting to tease out the difference between the way they like to work and the way that their job requires to work. But it's not enough just to take the number of the phrase cards that are chosen. We have to get the meaning behind that. So I put these two samples up. I didn't think you wanted to have a look at all nine. And I think katie Shout we did. But there's a card there that represents level two work strategy, work, service work about working within a given framework. But very often when you explore that with people, the framework that they work within is not necessarily a framework that's of two complexity. And it may well be a much broader framework that they have developed, designed, put in place, been accountable for themselves. And the other set, the approach to uncertainty is there's one there who is common sense that's not in the STPA set. And very often people will choose that harbor what's very interesting is to say to them, okay, what does common sense mean for you? And there are some wonderful discussions and descriptions that come out from that in terms of what common sense means for them. And we work through. And the whole time that you're going through that process, you're making a hypothesis about what it is that you're hearing from this particular individual and exploring that and probing for that and asking, for example, wanting to look at the output. And I think that most of you will be familiar with the curve and the notion that people travel on different growth paths and so that they make transitions at different times of their career. The other output for us from a career path appreciation is to give an individual their own development path. And what we highlight on there is that across their current level of capability, we look at the transitions that they've made and if we're using the predictive ability will make in the future. And we're also mapping their career to date to start to look at where some of the gaps might be. And that might be experience gaps. Because if people are working below their capability for a long time, as this person was, they can develop all sorts of behaviors that are not particularly effective within the organization that they work and can cause problems. And when that happens, I don't know what your experience is, but artists that it takes people a long time to rehabilitate themselves once they've developed a reputation within the organ. That's just another example of somebody who had an amazing performance track record until they were put in a position where the job was. A question for you.

Speaker A In talking with Mark Van Cleave and.

Speaker B I had described this to him, one of the questions he raised with me.

Speaker A Was, does your company ever look at.

Speaker B The output of the work irregardless of what the level of work was required for the role, but actually look at what they were able to produce?

Speaker A And in those situations where they then.

Speaker B Either worked closer to their level of capability or further away?

Speaker A So in other words, the job may.

Speaker B Require two minutes type of work but maybe they actually produce two cry work. Two things to that, the appreciation process and the exploration and the information that you gain from that does. But also because we're working closely with the organization, we have, if you like, a validation process, a check on that. And very often when we've identified people that have got more capabilities in their.

Speaker C Role.

Speaker B Part of that debate will happen in terms of what work they have been doing. So the answer is yes to that. And of course the other thing that the outputs when you start looking at a large group of people and this is what will feed into the talent bank and the session management piece is developing a pattern, a picture, a map within the organization. This one we've just demonstrated like this, just to show the difference between job content, which is the red circles and the blue stars, which is the capability. And there's one person in that particular example who was struggling with it. Their role was just that much step too far for them at that point in time. Now, because of their growth, they would in, I think it was the year, 18 months in that case, be able to really handle that. But it was then the support and the mentoring that we put in place to help them along the way. And another way of showing that is to plot capability onto the development curve and then you can start to see what your pool is like and what that's going to look like in 510 15 years. That's purely on the capability. What of course happens in the discussions is to look at all the other elements that are needed for success in that particular organization. So those will be experiences, they'll be fit with culture, values, a whole range of things that will come into whether they're going to be seen to be credited in this particular organization where you have people on what we would call no SIS. We pulled that out very separately to present the board to have a discussion with them because they were all individuals that had a bit of a reputation in the organization and they were the future leaders of that organization. So there was going to be a need to take some fairly serious decisions about whether they were going to be given the chance to perform or whether their reputation and then we've got some reliability data and there are a lot of research papers available so that hopefully gives you a flavor of what we do.

Speaker A Back to a process question.

Speaker B Your one to one guiding conversation. Yeah, who does that?

Speaker A Do you set that up in an organization so it's the mor conversation or.

Speaker B Someone in we do it in a range of ways. Part of our global processes are to put in place some training programs so that we're actually training people to do that. Now we train in house but we also provide that of a level of service in our market. What we find is that for people working up to probably high level three there is more ease about people doing that internally. Always there's input from the line because we're providing one piece of data it needs to come in with a whole lot of other data and we're really clear that people don't make decisions just based on our input. Does that answer? Yes. Thank you. I think we'll hold questions for now.

Speaker A So that you hear all three and.

Speaker B Then there'll be rolling plenty of time.

Speaker A For questions and also to hear the.

Speaker B Three of them debate among each other.

Speaker D Why don't you get one more question while I change?

Speaker B Anybody else have a question? Yes.

Speaker E It relates to the previous one, the question of potential conflict of interest. Are people on the inside of the organization relation?

Speaker C To.

Speaker B Repeat the question I was asking.

Speaker E What the potential for conflict of interest was if the assessment was done by someone within the organization or a colleague or potential colleague of the person.

Speaker B Very often around senior levels and the level of credibility that the person conducting the interview has. And there is a consistent pattern there in terms of the work that we do. We're very clear with the organization we set up a contract with them that obviously they need the organized, they need the information 90% of the time. There are some projects we've done to be purely developmental for the individual but we have to raise up front. If you say this is for the individual you can't then six months down the line say I'd really like that information on that. So what we do is we're clear, we help organizations with briefing programs, we brief participants. The conversation we have with them is confidential and that is for two weeks. One is there can be things that are impacting the way they work in an organization that they don't want to shout about. It just makes life difficult or they can be working on something commercially sensitive. And if you think about the levels we're working at and variety of industry sectors sometimes that is absolutely fundamental. The feedback is a participative process. It's part also of the exploration with the individual. And when we write the report, the individual signs that off. The organization doesn't have that information until the individual is happy with that, I think. And yes, so that's that's, that's all part of the process. We don't experience conflict of interest if we're really clear about.

Speaker C Okay, well.

Speaker D What I'm going to attempt to do in a short period of time is to talk a little bit about this instrument and actually let you get a feeling of it or a flavor for it, and then talk about some of the we've had discussions about when is it appropriate to have an outside practitioner evaluate potential versus manager once removed, ultimately accountable for that judgment. A number of us, and I have the people up there and you'll recognize some of the names I said, what's a list of things that we ought to consider. And this is not a final list, this is not like a set of rules, but it's kind of our feelings about things to consider in the inside versus outside. Not surprised. All right. Some foundation, and I'm going to hit some basics that most of us understand but I'm not sure if everybody understands. The foundation is each person possesses a maximum ability to deal with complexity, which corresponds to a specific level of work and which defines the maximum level of work a person has the potential to perform at the present time and which increases with maturation or age.

Speaker C Now, there are a lot of implications.

Speaker D There because particularly that last one said, it's not something we can change. So what we're concerned with is how do we develop people to work up to their potential, how do we place them in role of working up to their potential? But here we're talking about a tool that is aimed at measuring which CPA also is aimed for. So we're aiming at the same thing. In fact, all three things are talking about identifying current potential. All right. Quick definition. Complexity of information processes is the current terminology we use. Complexity of mental processes is a term you see in a lot of Elliot's literature. Now, what happened is he was dealing with these issues right up until he left and he made a change formally from mental processes to information processes because he said, I can't see into a person's brain, so I don't know what the mental processes are. And his philosophy was to look at the total entity, organism and the behavior in the society. I can look at the information by the way you speak. So that's why he made that change. Now, current applied capability is what could I do at work today if I take my present skills, knowledge and experience, take my values and place them on the job? Current potential, notice the word is current for both of them, says what could I do? What could I do if I hadn't acquired skills and knowledge? I may not have them, but if I had them. What could I do if I valued the work and if I were free from any temperamental hindrance or other things that would keep me from working with potential? So, in effect, current potential capability is equal to complexity of information processing. So what it is we're looking for some pattern in speech, and I'll be explicit about that. That is a direct relationship to the work. Now, what I found is I think it's critically important that we do more and more work on helping managers understand levels of work. But what I've also found is when they understand thinking processes or information processes, they begin to relate that to the work and it enhances their ability to understand human capability. Was a study on the consistency of judgment made various ways. What he did is he looked at two people and it was he and Captain Patron did interviews and then did assessments of those interviews and they did them independent. So we had two outside observers. So two people using a technique that is the basis then they had the manager once removed make a judgment about the subordinate once removed, okay? And that is the accountable place. That's where they had the manager make a judgment about the subject. And they had the manager and the nor together make a judgment about the subject. And they did it with this number of people. So there's a sample size. And it was two organizations that were using organizations. So it was two organizations that had experience. They were requisite organizations. And I think that's a key point. We have data to show that the organization is not requisite. We have to be very careful about those yieldments.

Speaker C Correlations.

Speaker D Those kind of correlations are phenomenal. I mean, they're, they're the kind of things you get engineers. So there are two things that we have to deal with.

Speaker C There's a little bit of complexity of.

Speaker D What we're going to cover right now. One of them is the process or the way by which the information is compared. And then the second one is the complexity of the information itself. Now, what we found is these processes, it's pretty easy to teach people the process and get agreement on a passage of what processes showed. So I would say that's kind of a settled fact. But when you come to the complexity of the information, it's much more complex. There's art until in fact, before Elliot passed away, within the last month, michelle and I were on the phone with him and he was still struggling with the formation of the complexity of the information and trying to get a crystal clear image of what we're looking at. Now, those of us that are doing this work recognize that you can make these judgments and it gets pretty clear. But it takes some practice. And there is somewhat of a tendency that the person making the judgment be of a similar capacity. So in other words, differentiating stratum four and five is particularly difficult. What we find is somebody at stratum four can learn how to do that, but they're not entirely stable. They tend to drift. Somebody at strategy five can do that. Differentiation, you can do it pretty clearly. All right, what are mental processes? I didn't change that. What are information processes? It's grouping and comparing of information. And in fact you use the same expression, right? Talking about grouping and comparing of information.

Speaker C There are four.

Speaker D First one is called clarity. And what my symbols are saying is you can basically akin to boolean type logic. What happens is the person just speaks individual standalone statements. You need to do that because it's the right thing to do. Now they may also speak statements that are something like that. If you do that this is going to happen. But together that one set. If you do that, this is going to happen. Declaration, single declaration, and those declarations will kind of just stand out there. You'll have some examples. The next step is the person on purpose accumulates the information so that it's very clear to the listener that I mean, not just I was late for work because I lost my keys. You lost your keys? Well, I was late for work because I got caught in traffic. Okay, that's independent declarative type statement. But if the person came out and said, you know, I was late for work because I lost my keys and then this plus this serial in order for us to get to this final place, it's going to take a number of steps. It's not just if we do this will happen, but if we do that then this will happen. It's got to be at least three. It's got to be a series of things and then in parallel we're dealing with a series of things and then a connection together. Now, what's interesting is these are ten to Stratum 1234, stratum two level of first level manager. I can accomplish my job with list if you force me to set priorities on those lists that are causal type priorities. That doesn't process here. What do you think I can do with that job? Now, one of the things about parallel, parallel is a little tricky. The first three are easy to pick out. Parallel tricky because kind of the textbook would say a nice parallel speech, you see a fish, you see another series of things down here and that's not quite it. You might think of it this way. Parallel has the ability to take an idea, express that idea, put that idea on the shelf, go over here and take another idea and develop that idea. And now say and remember what I was saying before and begin to people start to say things like from the perspective of that's a typical strategy, by the way, that's a flag. That's a flag that says look here. But it is not the expression itself. You don't score something on the flag, but you look when you see a flag. Now, what happens is, as we mature, we go through these four things and expresses they're recursive. So you get four things at one level of language, and then you go through each of the four process, and then what happens is you cross into another level of language. Now, this is what takes place between the stratum four and five change. This is why it's such a major change, because the nature of the information is changing. What we're finding out is, just before you get to stratum one, there's a similar kind of thing. And we're starting to see that there's a huge similarity between Elliot I guess it's Catherine likely to call it zero, but there's really a big jump from zero to one, and the jump is very similar to the jump from word five. You get more complex information. It's not the information. Yes, he's correct. What the fine point is that if I use a word like total quality management pardon?

Speaker A Shotgun approach.

Speaker D Shotgun approach, okay. Shotgun approach. That can be at it can be at one level or another level, and it depends on my ability to unpack the word, not your ability as a listener. And this is one of the challenges in trying to learn how to do this method, is to not read into it. When I take high straddle people through this, they overrate people. Well, of course they meant that. Well, tell me exactly what they said concerning what they meant. Okay. The answer was, no, they're not. So there are techniques or guidelines that say when I look at the information, I need to look through certain filters to prevent misinterpreting what the information is really there.

Speaker B Do you also find that it's picking up on Alan's point that actually it's the breadth of the information that they need in order to.

Speaker D Okay, I'm going to say no, but I think I know where you're going, because one of the questions or one of the things is what does mode mean? What's the difference? Everybody not know what mode is? Mode is the path that you're on. So a mode three individual is going to end their career at stratum three, current potential capability. Mode six individual end their we don't fully understand what mode is. I think the BIOS people probably understand a lot better than I do from some of the things I've seen written. We think myself, Captain Kason, don't know about bias. Think that higher mode when they process information the young high potential who just if you have two people who are at the same stratum on the chart like this and two different ages, which places them on different modes. You got 23 year olds Dragon Four and a 45 year old strategic. My experience is that the 23 or four year old has an awfully scattered argument, and it's just all over the place with all kinds of things, and the 45 year old had this crisp, polished argument. But when you analyze the structure, there's still a strategic fork.

Speaker E Yeah, I think mode was invented to explain precociousness.

Speaker C This person wasn't at level three, but they were talking.

Speaker D Can we know? Question. But I would expect that with the younger four, you might hear much higher language popping out, much higher stratum language. I think that's a difference.

Speaker C But.

Speaker D You hear it, but it doesn't have the evidence that makes you rate it that way. Anyway. The right hand side is kind of the progression of the four steps declared as queen of the Power Theorem. The left hand side are the numbers, which are the best way to identify each. And then the middle is one of the set of terms that Elliot went through, and he did. This is the one I use because it kind of works for me, but it doesn't entirely hold together. It's useful self explanatory gestures. I'll push you off the boat. Specific he pushed me off the boat. Classes, shipping, boating, boats, things like that. Abstract concepts. Actually, shipping is probably an abstract concept. General principle. There's a gentleman here, John Morgan, had just did a dissertation on high level leadership and Herb couple of and I did analysis of Pope John 23rd and Martin Luther King and the attitude. And what we found is that we could kind of when they were down when they were down at six, seven, and eight, we could identify it, but then something happened, and it was like, I don't live up there. I can't tell with that language. But there was some evidence that, yeah, this is what this higher level language looked like. Going back on that slide one. Are you suggesting then that there is Stratum 16? Elliot suggested that I wouldn't have to. Brain power suggest that, I guess. Why do we stop at 16?

Speaker C I'm kind of looking at this kind.

Speaker D Of wondering why we didn't try to.

Speaker E Get to 100 or something.

Speaker D Jerry one part of the population. I guess, from a practical standpoint, we look at it's difficult enough to find.

Speaker E Five in this world.

Speaker D And I believe Elliot mentioned one time.

Speaker C He met a nine Chinese man.

Speaker D I don't know. Does anybody know? He had predicted some numbers and put him in the life and behavior of living organisms. He had a chart in there about what he thought was the distribution. The reason I said is because I'm not sure that abstract concept does, in fact, include everything above it. Well, then you have to find a.

Speaker C Nine to prove your purpose of general.

Speaker D Principle, and you have to find the eleven. We got a mode eleven. We got that functioning now. What was this function called? Second and Martin Luther King, both on the same path that King was at age 27 and was on the same path that the Pope was on, only the Pope made it through. Higher.

Speaker C If you recall the way Elliot, when he was consulting, would respond to questions, he'd always begin. Well, first, let me help you understand the general principle. He was operating at level ten, and he was framing the body of knowledge or the model, the conceptual model that Herb talked about this morning. And then he went to the abstraction, and then he went to the concrete specific. Ellie was offering attention.

Speaker D All right, now we got tests for you, at least paper out. Get ready to give an answer. Michelle is going to read three excerpts. I would like you to rate those three excerpts. You want to give us the name first?

Speaker A So we're going to rank order from least complex.

Speaker D Oh, I want this group not only rank order, I want them to say declarative, serial, cumulative or parallel.

Speaker A Okay.

Speaker D This group will be able to do.

Speaker A That based on what I'm saying, pattern of what I'm saying. So the question is, if you were to get this job, how would you improve customer service to our external clients?

Speaker D And this is Bob.

Speaker A Here's what Bob has to say. I'd like to see more contact between the sales manager and the clients. The sales reps do what they can, but if the sales managers were to visit occasionally, I think that would send a message to our clients that they are important. And we get a lot of complaints about late deliveries. We need to fix that. It makes the sales reps look bad and it angers the customers. This is Rosa, and she says we should ask them what we can be doing better. It's important that they know we care. We need to train our representatives on how to offer exceptional customer service, make it a part of their visit procedures. Treating customers like family is what I do to make sure they are happy. And my customers always say I give good customer service. Next is Joe.

Speaker B This is our last one.

Speaker A Where do I start? We need to try some new things, be innovative. I think we need to separate the salespeople from the account managers. The people who are good at maintaining relationships are not necessarily good at finding new customers. I like the challenge of finding new customers and seeing them through their first sales, but then I'm ready to move on to find new customers. That's where the challenge is. We have other people who enjoy handholding our customers, and we should let them do that. I think they are two different personality types. Asking somebody to do both adds stress when they are working on a side they're not comfortable with. So if we separate the two tasks into two jobs, people will be able to spend 100% of their time doing what they're good at and enjoy.

Speaker D Here are the transcript. Who would you hire for the frontline sales role? Drafting one frontline. Who would you hire for her?

Speaker C What?

Speaker D How about Bob's boss? Actually, everybody's there okay. Now, all right, the contribution there are a couple of contributions that we feel like we've made. Delia developed the thing. It's in human capability. We're using the process in human capability. What have we contributed to it? We've contributed ways to make this as explicit as possible, and we contributed a process which I will review, which I feel increases the accuracy and reduces the probability for wrong judgment. So I'll go into those things. So there is an example of what I'll call what we're calling tool solution I would like to see. And the context works this way, is red notes. Things that are underlined are kind of the kernel of the thought, and there's nothing sacred about those. It's just how we identify kernel of the thought. And the blue are the connecting words, the things that kind of tell us what we're looking for about how the language is being related. More contact between sales managers and the clients that you've got. Sales management clients. And I'm not getting into the language thing. This is all strapping one in my heart. This is all third order language. I'll let you read that. I'll read it with intimate let me read a couple of start with the result.

Speaker A We found that this one and here's.

Speaker C Why.

Speaker D We need to fix that. It makes sales representatives look bad and it angers the client, makes sales guns feel bad. Now, your initial reaction may sometimes be, no, it can't be this way. Somebody's going to talk higher than this, but it actually works. So our response to that would be three declarative. Two, no evidence. Higher than declarative. This is new strategy. A couple of evidence. All right? And so we're saying that person ought to be able to handle a job with time span about six months, six to nine months.

Speaker A No evidence. Tired of cumulative I'm sorry, no evidence.

Speaker D Joe. We have other people. We have other people who enjoy handful. We should let them do that. That's declarative. I think there are two different personality types. Now we get into something. Asking someone to do both, it adds to it causes something to happen when they are working on the side they're not comfortable with. We got a one thing. So if we separate the two tasks into two jobs, now we have another effect, another relationship, right. People will be able to so here you have an example of a serial with a lot of series that's fairly advanced, but that's good, solid strategy. Now, I remember when I was first learning how to do this, I was saying to Elliot, you know, this particular interview, it has two, but it just kind of has a hint being a little higher than that. And he said, yeah, that's how you feel when you're starting to get to move into the higher level, is that they begin to in fact, they can pull you. Sometimes you read it and you say, well, that's cereal but then you go and try to outline it like this and you find it really isn't quite serious. You're reading it into it and that's an indication that isn't behind. So what we're finding is that you can be pretty accurate. And I expect people that are coding, like if I have two people next to each other coding that they're going to be within a third of strap, I kind of feel like the tool is good for about a third of the strap. There are no example, doesn't require a state treating customers like family. What I do to make sure that they are accurate application, as I said, the thing I think we've contributed is we've experimented with a way of doing this that seems to get good, reliable results. Now you notice my words are seem clear and all that because we don't have anything like the background. Now as far as I've been able to track down, there's kind of documented maybe 1200. Daniel works with both. And what was the name of the company? The company?

Speaker C Enhancer.

Speaker D Enhancer. And when I checked with them about a year ago, they had done a couple hundred things. Okay, one, Patel has done four or 500 of them. I've done four or 500 of them. So kind of the order of magnitude of how much is being used is about there. What I recommend people let me go back further than that. We interview people and we interview them with a protocol of problem solving type questions. One of the concerns is, can you get the person fully engaged? They have to be fully engaged in the problem solving, not in talking down to you or something like that, that wouldn't get the right result, but in the problem solving. And what we found is that there are certain questions that work better, but we find we also want multiple checks. So a question as simple as what do you see as the most serious situation facing your organization? Present time usually gets their highest response. The trouble is you sometimes get company talk and you're not quite sure whether they have originated the conceptual or whether they're talking buzwords from the company. Now Elliot's questions were remote questions. They had to do with drug use or legalization of drugs, abortion, things like that. I have found this is that we have asked questions like talk about something that's particularly important to you and tell your position on that. And that question hasn't worked as well because my interviewers sometimes move over into the emotional side and they say, I want something to feel really strongly about, really strongly about. No, I don't. That kind of gets them or something you're very engaged in and you can apply yourself and problem solving anyway. We do the interview and the interview, four questions, something in that order a number of times to get checked somewhere between. We have kind of a set up way to transcribe these that's about 600 words per page, and we typically get four pages is acceptable. Sometimes we get six or seven pages. That's kind of the length of what we're dealing with. And we review it by two compare results, and whether the results agree or not, we do a consensus review of why each other said what they said, if they were different, we look for what their differences are. What I found is that at stratum four, one of the things you're looking for are connections, where they connect different parts of their argument together. And what I found is it's exceedingly difficult to read the document and pick all those things up. And if you use multiple graders, you're much more likely to be able to pick those up. Now, the first major piece of work that we did was involved up here in Alison Brown and I did it. What we found was that we were trying to differentiate primarily four and five, which is the most difficult thing to do, and found that by having two people would be much more, then we get a comparison. And then what we did in the formation of this is we rank order 20 of them. And then we looked for breakpoints where it seemed to make sense, where there were breakpoints. And then that started to be our library. And that's now a benchmark library that we can thread to. So we evaluate them, and then we link it back to benchmark. We take it back and say, okay, this really does fit in here, and that validates it. There's a number of validation check. Now, there's another thing that we've done. One of the things about assessing people's conversation, and if you try to do this, some people tell exactly where they are as soon as they open their mouth. Some people, it's pretty difficult. So that means that there has to be where there should be some way to show your concern for risk and how well and how accurate the judgment is. So the way that we've approached it is to say, okay, let's do three things in the scoring. Let's set a high. What's the maximum that the transcript contains? How high could we just and we really don't see a hint of anything above that. And then on the other side, what's below? What is there constant, solid use of that? We can say, it's got to be at least this. And then we say within that what's the observer's best judgment as to what that transcript represents. Now, in doing 250 of them, of that I'm sorry, 150 of them in one particular case, we then plotted them with a band, plotted the bands in a central point. And what we found was that 140 of them were very tight bands, and then a couple were a little wider than that, and a couple were real wide. So we said, really? That was a bad we've done all those things to to try, try to increase the precision and the confidence that this is a good evaluation. Now, what I just alluded to was a project that was inside an organization, was sponsored by succession planning. And what happens is, the CEO says to the succession planning chief, can you identify people who could potentially run this company of 140,000 people in our pool? So what they did is they had used standard methodologies for identifying potential, and they selected well, I'm sorry, they screened the 1200 director and executive director level people. Out of the 144,000, they picked 150, and then later 50 more. And we put them through this process. And basically, this is the result of the first 150. What we have is stratum here and mode here, and this blue line represents the kind of the expected location of the role. We didn't go in and do role work, so we don't know what they were, but the titles and what they were targeting. The organization were directors and executive directors, which are kind of four to mode five. It was a major telecommunications company. They had some very large general manager kind of roles. Probably would take a mode nine individual to move into leadership of that organization. So we found three out of 103% out of 150 people, and also pushing up into that another four people. We had eight or nine people out of that pool. Now, how do we test it? The way we tested it is we plotted all of them and we separated them into cohorts, and we said, look, if these people look like they ought to be moving together now, from what you know internally, does it make sense? And they came back and said, yeah, that looks pretty good. Now, this was done 1999, and my most recent check, which was a year ago, was the data is holding up very well. Predicted power of the data, use of the data, they're still using the data. Last one.

Speaker C Question, did I read there that 26% of their management.

Speaker D And there are number who are higher than that, 26% of this sample. This represents a sample not this is 150 out of 1200 that are all at the same level.

Speaker C A quarter of the 150, plus the ones above.

Speaker D You had directors running 1200 people, major operations. In fact, what happened is, I didn't take this job. All right, back to the issue of accountability. I'm not challenging, I'm not taking it as a challenge. Manager wants to move accountable for making judgments of capacity. Before we did this job, we said to the client, I'm not going to do this job unless you demonstrate internally that process. So we did a talent hearing process in one of the divisions, and it was a woman running a division. She was in an executive directed role, which is a stratum four high transport role, clearly operating at six and working for other words like fours and five. So very complex. I can't mention what I'm aware of. Anyway, what happened is we had a training session on this and these are the folks that were at that session and it was last September of 2003. And we said, okay, what should control our behavior and the use of this thing? And these were the things that we thought about. Anybody doing this work should maintain their training and experience. They should ensure that there is at least minimal conditions of trust in the organization before doing an interview. If the interview is an employee, they should ensure that the process is voluntary. They should explain the process to the interviewee. Explain all who would have access to the results. Ensure that it's a quiet, proper environment for the interview. Offer four to six questions of which they can choose.

Speaker C Questions.

Speaker D I don't go by that one, but that was one of the things that we listened to something a little bit different. Joel Curveball give them a question that you pretty sure that they don't know from somebody else that that's what they're going to be asked. Seek to have the interview last at least 20 minutes with the interview's permission. Record the interview and transcribe it. We actually get that on the face recorder. Have two blind rated for each interview. Use at least two sources of rating transcript, resume, rear, History Manager, Mr rating to one other points to zero in. Ensure that the data is from different sources that they converge when appropriate feedback the results, and it's almost always appropriate to feedback the results. Keep a library of past interviews in a systematic means of collecting and storing data. Place each transcript where it belongs relative to other transcripts to determine whether it feels more or less complex. So you're using the rigid structure, but then you're also using gut reading that says, yeah.

Speaker A Jerry, why don't you come on start and then we'll have time for questions kind of rush through. But we got 2 hours and we want to hear all of.

Speaker B Mic.

Speaker C Because the methods that have been described so far are probably very accurate ways to make inferences about someone's potential. And as a matter of fact, the methodology that Glenn just described was the basis used with the US Army Research Institute in being able to validate the judgment of potential. And the measurements of information complexity correlated nearly one to one. So the question then is when you're trying to help an organization implement consistently and across the entire organization the methodology for evaluating current and future potential, what methodology should you use? And I would like to answer the question in the following way. You need to answer the question based on what is going to best support the accountability leadership system. And now I'll explain what I mean by that. I'd like to make some assumptions about potential similar to what's already been described. First of all, potential is a construct. It doesn't exist in reality, it's a construct it refers to the possibility of becoming and the inference, the hypothesis that most of us have it's determined by the brain's capacity, the brain's capacity to make novel connections that are necessary to solve problems. That's what potential is. Mental horsepower. Now, potential cannot be directly measured. None of these three processes measure potential. What they do is they measure properties of potential or they judge properties of potential. Some methods look at the patterns that people recognize when formally tested. Some methods look at the types of connections that people exhibit when thinking of the context about what field I'm playing in. Now, the value of assessing current potential. We've all been talking about it. If the determinant of success in any particular role, if the sin quan no is having the raw potential to handle the complexity that is embedded in that role, then we need that to significantly improve the accuracy with which we fit the person to roll. If that were the only need, any of these three methods I think would be equally useful for me. On the other hand, of equal importance is the need to take managers through the process of judging current potential subordinates once and twice removed. Because it's the hard mental work of thinking through that manager's experiences with those subordinates one and two levels down and comparing the way they sense and have experienced that person handling work against what they're learning about these levels of complexity and what they mean that that is what's necessary to sharpen the manager's thinking about work itself. I don't believe managers get work, they really get it until they go through this process. It focuses their greater attention on all the supported capabilities. In order to judge people's potential, you've got to take a look at how effective they are and why in your gut, it feels this person is not demonstrating the degree of effectiveness that you believe he has the potential. What does a manager mean when he says, I don't think he's working to his potential? For a manager to say that that manager is implicitly saying, I believe this person has more raw capacity, this person could be more effective than he actually is demonstrating. I believe that getting managers to go through the process of doing this then helps them identify the source of the gap between their potential effectiveness and their demonstrated effectiveness. If you want managers to develop their people, managers need to accurately diagnose why they're not as effective as they could be. And then finally, that's a prerequisite for effective coaching and mentoring. When I have clients who are evaluating people outside the company, I now have real confidence in sending them to either of these groups. I'm really impressed with the presentations you do, CIP, better than I do. I'm very impressed with the quality and the integrity of the CPA process. So we then need to think of the value of assessing current potential, the future potential. Sorry. In terms of if we have confidence that we can translate strategy into the organizational analog and we can then say, based on the organization's current requirements, if our strategy is successful, we will need to have this organization in two years, this one in five years, this one in ten years. Then you need to have some way to evaluate your pipelines of potential so that you can determine whether or not you have sufficient people in tow to meet the requirements of that organization in two, five, and ten years. The same time, it helps the managers meet their own accountabilities and being proactive in making adjustments as they identify gaps or overages and to begin to do the appropriate mentoring to prepare the pool of talent for their requirements in the future. Now, for the employee, if the organization is committed to assisting each employee in being developed so that he or she has the potential, the opportunity to realize his or her potential, then the mapping of that person's career potential is a necessary part of the dialogue. And I think you talk beautifully about that. Okay, now, what are the practical issues in judging detention? And here's where this is now 15 years of experience. It's not theory, it's not scientifically based. This is my empirical understanding, is managers are not able to accurately judge people's current potential until they build in a very hard, personal way, a mental model about what these levels of role complexity are really about. If you don't understand how work differs from this level and this level and this level, how in the heck are you going to really say, I believe this person is big enough for work at this level or this level or this level? So when we work with clients, whether or not it's a full organizational project, or whether it is simply to help them map their talent pool, the first piece of work is getting them to understand how their existing organization maps out against levels of complexity. Well, this requires a fair amount of rigor in role sizing and using time span. Initially, our experience is not completely you need to do enough time, save information to have enough benchmark roles at each level so that you can establish with managers an understanding of why those roles are level five, high level five, low level five, why those roles are mid level four. And in that discussion, then after they get it about those 20 or 25 roles, then you can start piling on the hundreds of additional roles in a much more efficient way of sizing those roles relative to the others. I, quite frankly, have not ever found a client that would be patient enough or willing enough even to have time spans of 1000 roles or 10,000 roles. That's not reality. It's in these discussions where managers are actively debating, discussing the underlying basis for complexity of each role that they begin to construct this dynamic model. They begin to feel what that complexity is about, as they describe to each other how this kind of job requires dealing with more variables than that kind of job, or this kind of job has more changing variables, then they start to really understand what that complexity is about. The problem is, these discussions will inevitably trigger hot claims about status and turf protection. Because, see, if I evaluate the roles in my part of the organization low, that must mean I have a lower organization than you do. This requires firm admonition on the part of the consultant and on the part of the manager once removed, to exhort and explain and encourage and cajole managers to be mature and impartial and as objective as they know how, and to bring it to their attention when clearly they're not. That's all there is to it the first time out. When you do, this process gets messy. And I finally concluded it must be messy because you're trying to break new ground in the way managers think. It's very much what Herb said this morning. It's anacletic. You're pulling people into an entirely different conceptual world. My argument is, without developing this model fully, asking managers to assess the potential of their people is far less useful, far less accurate. So what do we do? As I'm sure all of you do, you spend a lot of time training people about levels of complexity. You use one of Elliot's favorite expressions when he and I were talking with the CEO of Ford about twelve years ago, about what organizational structure they needed to go to should they go from their current 13 or their current twelve, up to 13 like GM, or down to eleven like Toyota. Ali said to the CEO of Ford, you can have any number of levels that you want as long as it's eight. But the key here is helping managers then understand the relationship between levels of complexity and different levels of mental capability. And I found it very useful to take a page from what Glenn was just describing. Book level one work.

Speaker D And the more you can do, it.

Speaker C Inductively, and you can get people to explain to you what is the nature of work, what kind of decisions do people need to make on the front line? And they'll say they're short, they're immediate, they're simple, and eventually someone says they're binary. And then, well, if that person gets stuck and has to go to a supervisor, what do we expect her to be able to do that we don't expect the subordinate to be able to do to solve the problem, even if she didn't have any more education. After a while someone says, well, you got to collect information and analyze it and come up with some hypothesis about what the problem is. And what is it that the department head or the unit head or the superintendent needs to be able to do that? The supervisor doesn't need to be able to do well, they got a plan. They got to look backwards in time. They got to project forwards in time.

Speaker B And they begin to describe the types.

Speaker C Of decisions they have to make. After we go through the model, which I won't bore you with, I say, look, this is the tactical role. Level three is the unit in any business unit of where things get going, that's where current cash flow is generated. Current PNL comes from level five, and that's the strategic world. And that's how I make the connection about abstract conception, because the true strategic order, not everybody says, I'm working on my strategy, but real strategic work is dealing with abstraction, cash flow, asset worth. You can't measure asset worth. You have to develop models that will help you infer the worth of the asset. But you can measure cash, and you can essentially infer directly cash flow. Those are different levels of complexities within the normal adult symbolic world. Level four, that's universal joint, that's your operational transducer. That's where the general managers of the functions within a level five business unit are taking the model and driving it into tactical elements that their subordinates need to implement. That's where the parallel processing comes from. That's where the critical path analysis is. Then I start to talk about information. You know, if you think of one sphere of information complexity, you throw thoughts at a problem, you weave them together. You construct serial patterns. You weave multiple serial patterns together. As a matter of fact, what Ellery Jacks did, what Glenn does, is they literally look at the mental process that corresponds to that. But you know what? The nature of the evolution of intelligence and the nature of the maturation of human intelligence is that as our brain matures into its ultimate architecture, it moves up progressively to higher and higher steps of complexity. And when it gets to the top of this, it kicks over to the next sphere of information complexity. This is the area of measurable work in organizations. That's the area that people like Elliot Jackson and Richard Feynman live in. This is the area where people are concerned with dealing with things in days, months, maybe years. Here, people are concerned with decades and generations. Here, we're talking about people focusing on systems of knowledge that are enduring for generations. So then I build up gradually to the Rosetta Stone. Every manager has a big Rosetta Stone in his or her life as we go through this process, because the objective is to get managers to feel in their bones how these different roles are different from each other in terms of the degrees and levels of complexity. Well, now, the way we used to do it up until about five years ago is we just had huge nine foot high wall charts, and we would have all of the data in Excel spreadsheets or access databases, and we'd print out transparent labels and put them on. Different colored post it notes. Elliot and I would stay up till four in the morning doing this level zero work, and then we'd lose half of the data points anyway, and then they'd slip off of the wall charts. It was terribly frustrating. So we developed software originally based on an Access now on SQL Server using Visual Basic, now Visual Studio, in which we could build the tree in the organization, in the software, and then pull up any part of the organization as dynamic icons. And then we start with the benchmark roles and get them established. And then we start to examine the additional roles. It's in the discussions where you start to see the light bulbs turn on, not simultaneously. The real bright, high mode folks, they're already on. And then the next group starts to turn on, and there's always a group that's low. Now, this, when they're done, is about first time out is about 85% accurate in terms of what they eventually arrive at, because they're still not clear what this means. But with the software, then instead of having these wall charts, we then sit down with the group of managers and say, okay, these are the roles. Let's have the discussion, and we can pull up any combination. Compare this unit to that unit, engineers to engineers, engineers to accountants. Now, what are the practical issues now in judging current potential? Here's where you have to build the mental model that Judy described about. In the middle of the Juxtaposo puzzle is potential. But the famous Elliott formula that someone's demonstrated effectiveness is a byproduct of the interplay between raw potential, acquired skilled knowledge, the degree to which the person applies that knowledge, and potential to solving problems, and the maturity of that person's behavior. Well, at the outset, every manager will have some difficulty in making separation. Every manager will have some. Our experience is we saw this first with postit notes, is the moment they get four postits on the wall, all of a sudden they start adjusting. Takes about four I can't do this. Do it. One. I can't do this. Two, I can't do this. Three. I think this is a little lower. They start to get an intuitive feel. Still not accurate, but they're moving towards it. There are two major categories of resistance. There are the flower child who still live on an organization. I object on principle. All people are created equal. Right. As Herd said this morning, Einstein is Einstein special in his way. Larry Bird is special in his way. But there's also a group that are simply obsessionally rigid. They may have potential, just get stuck in a group. Those are the harder ones to change eventually. The peer pressure works on these, and the simple elegance of the model works on these. Yeah.

Speaker D Is there a level below which a manager just cannot grasp the points of Ro?

Speaker C Not our experience. The issue is at what level of complexity do you want them to grasp? We have very effective training programs for first line supervisors. Level two supervisors who leave the room transformed, and we have half day, one day and day and a half programs for people on the shop floor to understand what is relevant for them. That's the key.

Speaker D You don't dump the whole load on the left.

Speaker C No, that's just of no value. Katie made a very interesting observation to me on the phone the first time we ever talked. Actually, level three is harder than level two because level two, you know where you got to get to. It's about the leadership practices. It's understanding accountability, it's boom, boom, boom, boom. Level four, it's clear. Level three, you're constantly straddling the fence. And the high capable level three folks, they move up. The low capable. Level three folks, they get dragged down. So now you have to have a model. And we have a four x four model that we define the role by its size, level of role complexity, by the types of work inherent in the role, the functions, the processes, the practices, the nature of work in the role. Is it an individual contributor role? Is it an analytical role? Is it one that has a high degree of interaction with other people? Is it primarily a managerial role? And what are the role relationships? Our role specifications follow this model. What's the size of the role, the type of work in the role, the nature of work in the role, and what are the working relationships? Because if people can begin to translate the roles accountabilities into those dimensions, we can then say, okay, the first question is, does this person possess the cognitive capacity so that you feel confident he could do the work at this level if he knew enough and cared, could he work at this level? And if he could, does the person possess sufficient skilled knowledge for the types of work that he can do the work of this role? And if she could, does the person value the nature of work enough so that she will apply herself to get the results required by this role? And finally, does the person possess the maturity so that he or she will work well in this role, given the working relationships required by the role? This model works as well with CEOs of Fortune 100 companies as it does with that long in the tooth first line shop manager who has been there for 40 years. I remember this is ten years ago going to shop floor and arie from England with inkle alloys. And one of the supervisors stopped me in the evening as I was making rounds with some of with the superintendent. He says, hey doc, remember that discussion we had last month about that guy who just can't do crap? Well, I finally realized he's got enough in the can and he sure as hell knows what to do. And he's got the fire and ability. The problem is that he's just an asshole, but he was using the four x four model. Well, here's the first thing I learned. Elliot and I spent four years in Argentina, three years at Ford, and we could never get managers to stop thinking about potential as future. So I no longer talk about potential. I talk about maximum capacity. What is the person's maximum capacity today? What is the person's maximum capacity in ten years? What is the person's maximum capacity by the end of his career? Because potential confuses people. Now, for us, the major breakthrough in being able to deal with small groups, a company with 1000 people, 150 managers, where we could do the post ups to dealing with a company with 10,000, 50,000, 10,0000 people, was the development of our software. Because what this does is it allows managers to have these calibration sessions, which are wonderful brawl, compare their judgments about subordinate current future potential against the judgments that have already been made about other folks. Because it's in that trying to they squint at the wall, they're trying to see the raw potential, and then they have discussions. And the more opportunities managers have to review and compare and discuss their assessments of people against other people, the more subtle is their ability to make the distinction about why that person is a low level four guy and that gal is a high level four gal. So this allows managers then to further refine their judgments and improve the accuracy of the judgments. But then we're also able in real time to say, now if this is what you think people's current potential is, then by career end, this is what empirical data suggests their career ultimate potential would be. So the impact of seeing immediately what happens if you change current potential on that person's career potential or vice versa, allows for further correction. In particular, empirically, we began to see that the young high potential people always have a halo effect. Some of it may be the high mode, some of it may be the question that we ask, which is imagine this person fully experienced. Well, if you got a 23 year old and you start to imagine them fully experienced, it's ten years from now. And so they literally project them forward in time anecdotally our experiences first pass. This is before we've reorganized, because we have to do this in order to reorganize and figure out who goes in what role. The first timers, the beginners the assessment they end up with is 85% to 90% consistent with their third assessment a year and a half later. So there's some variability. Well, we start with definitions. This is a screenshot from our software that shows the final judgments of people's current potential. What you don't see on the screen is the person's current role. But as a way of explaining to managers, this is headroom. This says that this person has a whole level of headroom today, they've got one level of potential greater than the size of royalty family. And these are biological maturation curves that aren't theoretical, they're empirical. And this is where interpolating this person ought to be in ten years. And you've already confirmed that you believe it's consistent with this person being there by career end. The difference between his current potential and his career potential, that's runway. We've been finding language that works for people headroom, and runway works.

Speaker D I get that.

Speaker C Now, the problem is that all organizational processes for evaluating potential and effectiveness is that the peers doing the evaluating are reluctant to confront each other's judgments, their biases, the different standards they have, both because in Canada, because you're also polite, and also because casting the first stone, I don't want you to challenge my judgment. So for that reason, the mor must be held accountable by his own manager, who's not in the room for ensuring the process with that mor. Subordinate managers is one that holds those subordinate managers accountable for making honest and accurate judgments and encouraging a debate and a discussion that surfaces the biases and the different standards. It's got to be within an accountability framework to work. So since this involves judgments, the Nor has to set the expectation and model open discussion. Honest discussion, encourage debate and must be willing to challenge people's judgments. Not to say I overrule you, but, gee, I have a different sense of Jane, tell me what you see, and I'll tell you what I see. And to encourage that kind of discussion, this is an equilibrium process, not a vote, not a consensus. My belief it's the mor that's accountable for the final judgment of his subordinates once removed and her subordinates once removed. But it's in the context of the discussion. What's the question? Very simple. Imagine this individual in this size role, as bright as she is, but having had a whole different set of life experiences in a role that she gets speeding tickets on the way into work in the morning. She's so excited about it. And imagine throwing a cocktail of Prozac and lithium and psychotherapy to get rid of the dysfunctional behavior under those hypothetical conditions. How big a role do you believe this person could handle? And so software pops up first where their current roles are, and then they start doing headroom. They go back and forth and comparing people to people after they leave. The first task, we ask them to look in the crystal ball. And where do you think this person could operate by career end? Not is it likely? Not will we break the glass ceiling? But could this person operate at a role of what size? So now we have current and age 65 compared against each other. And then we start saying, does this track does this track do these people look like they're going to be passed from the same mold? Do these people appear to have the same current potential, and it's an iterative discussion. So now you end up with your pipelines of potential. If at the same time you want to now say which of these people are underemployed and which of those people are Peter principal, we can superimpose that just using a color code. All of a sudden, managers eyes light up, they can begin. And the folks in HR that are accountable for staffing and succession plan, all of a sudden things start to make sense. You start zeroing in on areas. Here's someone who's got current potential at mid four, but is in a role that size well matched. Here's someone else well mid five, but he's in a row, low four. This person's got a huge amount of headroom. Very important implication. Or you can organize by structure business unit A, business unit B, business unit C, and all of a sudden you see why business unit A is much more effective than business unit B and business unit C. They're beginning to correlate very strongly the median level of effectiveness of the employees in the business. Correlate very closely with their actual business. Or you can look at the pipelines through the dimension of effectiveness, and you can then what Elliot calls minus C, we call the X factor, the extremes of dysfunctional behavior. This happens to be an offshore oil drilling company where they breed a lot of dysfunctional behavior. Okay, I think I'll stop at this point.

Speaker B That's probably a good idea. All right, why don't the three of.

Speaker A You come back up here and.

Speaker B Give us the time?

Speaker A I was going to ask you to talk about what you thought about each of these items, but I think probably.

Speaker B There'S a lot of questions out there.

Speaker C I'd love to hear biographies really wondering.

Speaker E What the choice of big.

Speaker B Go for it.

Speaker C The question was posed to you.

Speaker B I think what's very interesting is I think when you start getting down to some of the basic one or two differences, I don't think any of us are saying anything particularly different. I think there are some differences in application, but I think if you take some of the core principles, there's not a huge amount of.

Speaker C I would agree, we all agree in the importance of judging and its critical nature for Council Mathing, critical nature for having effective mentoring. The real issue is who should be doing the judgments of potential.

Speaker B I think it's one is in the right role and has the capabilities it because if not, I think my experience there is a danger that people map the ability to.

Speaker C Well, see, we start this when it isn't requisite because that's part of the process of becoming requisite but it's not anymore making the judgment this cannot be done without these equilibrium sessions. And those equilibrium sessions, man, they are not down dragging them out of the ear. If it were just the Mr. I agree with you. I think it would be maybe 70% reliable.

Speaker D Is each method missing the same scent.

Speaker C So that if you do all three.

Speaker D Chances are you get 99%?

Speaker B I think what we haven't been able to do as fully as we would like is to be able to say in 2005 that this has this potential. There are some studies, and we're embarking on this more that in 15 years time. We actually can say that.

Speaker C Have you observed specific characteristics individually or collectively?

Speaker D Where your methodologies?

Speaker C Don't you seem to have some type.

Speaker D Of trend in terms of not being able to be able to see that potential? Is there a common individually or across between?

Speaker C I'll say the most difficult clients, most difficult employees for managers to get their arms around in terms of judging current potential are the ones who sit significant. Managers have a hell of a time saying, I could see that person having that future potential because there's no way in hell that person's ever going to be allowed to get there. And they won't let the words out of their mouth. That's the truth. It's the high potential, high dysfunctional folks that we have to work the hardest to get an accurate assessment of their current potential.

Speaker E Lynn, I know that you used both methods at bestpac, and maybe you could comment on the relative accuracy.

Speaker D Yeah, because it's something related to what we did is we had been doing pre hire interviews where we were doing one other sacred thing. We did CIP and we did Personality Inventory, and we put them into the organization. Now, that information was generally not shared with the managers. It was involved in the recruiting process, but it was never a big point. And one year later, we did a talent hearing, which we did as Jerry talked, very similar to what he talked about. And when we got the results back, we found that there were 22 in the sample, but ten of them were essentially exactly the same. In other words, I judged them in a certain stratum, and the Manager once removed team judged them to be within one third of the stratum. Of that, nine of them were judged to be one stratum lower than I judged them. Now, from a statistical standpoint, that is really weird. So we started digging into the information, and what we found is that every person that was judged one strand lower, their manager was below their own role. Okay, so to state it this way, listen carefully. It's a little hard to say on if my manager is lower than he or she should be. It's highly likely that my manager manager will judge me lower than now, what we did as a result of that is we instituted Ron, we instituted the Manager once removed kind of mandatory contact with that skip step in enough so that they would have an input into that judgment, because obviously the information was being filtered. And think about it this way. If the manager was one step lower. It's highly likely they were at the same stratum. So you got a dysfunctional relationship there. They are reporting feedback from that.

Speaker C I.

Speaker E Think, Jerry, I mean, from my point of view, having observed that your comments about having to establish a model within the manager wants to remove brain to do this is an important concept that we set the concept in place before they're really able to judge the complexity.

Speaker C Just empirically. We haven't found this problem, as I said, about 85% accurate the first time.

Speaker E But you do set the stage for them to be able to do that, to do an accurate evaluation. You said that they needed to understand the complexities within the organization, the work within the organization level, the work level.

Speaker C Of every company we work with gets at least three days of training right up front and then gets a bolus of specific training about this process before it starts. Question.

Speaker E About CIP and getting to the engaged conversation. One is do you have to have the same question for all the interviews you've given set? If the answer that is not necessarily, then I'm wondering whether there's perhaps the opportunity to ask what I would call an engagement question which would help to seek out subject matter that that particular interviewee would find fascinating and then to pose a question in that field that would lead them into an engaged discussion right away so they wouldn't have, as you said, the risk of reciting to.

Speaker D You all the what he's saying is, would it be advisable to do some probing, to find an area of interest and ask questions? I don't see we just haven't done that. One of the things I have to be careful. My background is engineering, and I do it. And when I talk to some of the OD folks, they say, well, gee, weren't you careful about building rapport? And no. Don't you have to do it in purpose? No, I'd rather do it on telephone. I get a better tape. No, I prepare what I call a protocol for the organization. Or what I do is I will ask neutral questions that are the same and then I will ask recruiting situation. I will collect recruiting data during the interview.

Speaker A I slightly disagree that we ask what.

Speaker B Is the biggest issue facing your organization right now?

Speaker A That's the probing question because they pick and then we say, if you are in a position to do something about.

Speaker B That, you could have the money, the.

Speaker A Staff, the authority, whatever you needed to fix that problem.

Speaker B What would you do?

Speaker A So in my experience, that is the probing question. Get them where they feel they're michelle.

Speaker E But that's the sequence, I think, that Glenn said.

Speaker C Put you in a position.

Speaker E Where you had to get past the corporate gobbledygook that people would recite because.

Speaker D Okay, but that is a minor problem. The corporate goblin is a minor problem.

Speaker E I had a question relative to the capability of an individual, relative to the job that they're in, if they're much more capable than their current job would reflect? And how quickly does an organization have to move to move them through regression to get them to the higher job before they get frustrated and leave?

Speaker C Well, I think Judy's comments were very illuminating about this. If someone has been underemployed for a long enough period of time, they start to become demoralized. Sometimes you got to work very hard just to get them back in the game. But if you've got someone who has got a lot of headroom and is already demonstrated to be very effective in the role, my feeling is if you don't have a plan to get them out within a year, you'll lose them.

Speaker D I definitely agree with that year, and I want to go a little bit further. Sometimes I've been called on for the application of a problem child. There's a problem child who's been around for a long time and they're thinking about firing them and somebody says, Wait a minute, I think that person what you find out, he's got 3000 employees for 25 years, and once they find that out, if they're willing to take the risk to give them some accountability of that highlight.

Speaker B I've had some similar examples where we had somebody very high level capability who was working at a level two role when their capabilities and they were about to be fired. And whilst we don't normally get involved in situations, we were doing a development program and as a result of that they were moved into a three role, their behaviors have really changed because then you want to expect you to be grateful for this. But I think the other thing is, it depends how long it takes. I do a lot of work in the northeast of England, and there's an amazing community spirit there with some things that are going on in the environment, in the social environment, and the encouragement from the company to be involved in that community. There is a huge amount of untapped capability. They're all very well paid. There are environmental issues, there are family issues, and there is a payoff.

Speaker C I think one of my favorite stories of all times that Elliot would tell is when he was working with Worldfall, tom Hilton really was the main reason for bringing him in. The organization was ambivalent, I think. But Tom was just a real force and they went through a process with the CEO's permission to first of all, do some role sizing, get a general sense of where the structure looked like, and then do some talent pool assessments. But to illustrate their results to the CEO, they said, we want to show you a different use of this. We want to show you the career progression of one of our employees. So this is 1234-5678. The employee came in here after a couple years, was here after a couple years was here. After another year, was here. Left the company for six weeks, came back here, here, and here. CEO Dave Whitwin was a very bright fellow, and he says, you know what this means? This was all wasted potential. And then he said, Wait a minute. That's me, isn't it? And then he got this big shooting grin on his face, according to Elliot. And he said, Let me tell you what know, I freight fellow. I got out of graduate school, and I had a chip on my shoulder. So I went in and I did a job. And after about two years, my boss couldn't stand me and foisted me on someone else and did my job for two years. Got foisted off, got voiced one more.

Speaker B Time, and I quit.

Speaker C Six weeks later, I got a call from the chairman of the board asking a lunch with me. The chairman said, we feel we have to share in the responsibility for your leaving here, because we don't think anyone has effectively coached you or mentored you in the past ten years, but we really need you back. And so Woodland said, why? He said, well, within a week, your boss's unit folded. Within two weeks, the general manager's unit folded. Within a month, the business unit folded. This guy was carrying the entire business unit. They said, look, if you come back, we'll give you assignments appropriate to catching up with your potential. We'll give you our best coaches and mentors. You've got to go to charm school to get rid of that chip on your shoulder. And within 13 years, he was the CEO.

Speaker A It's 03:00, and anybody who wants to leave.