Developing 360º Leadership-Feedback Instruments
Competencies, Skills, Traits, And Practices
John E. Jones, PhD, & William L. Bearley, EdD Organizational Universe Systems
What should we measure in instruments that provide comprehensive feedback on how people are perceived by self and others? This paper focuses on how to establish the proper mix of content for such instruments. As we have argued elsewhere (Jones & Bearley, 1996a, 1996b), creating such instruments within the organization can offer an increased chance of achieving satisfactory validity and reliability than simply using "off-the-shelf" surveys that are commercially available.
Definitions
There is some confusion on terminology in the practice of providing 360º feedback to leaders and others. Here are our working definitions of the major concepts:
Ability. A person's capability, either developed or not. An individual may have undiscovered and unrealized abilities. For example, a design engineer may be able to sell but may have never tried or learned selling methods.
Assessment. Gathering data on "how much," not to be confused with performance appraisal, or review. This is the second step in 360º feedback (developing the tool is the first), and it provides the basis for feedback. No evaluation is implied in assessment.
Attitude. A tendency to act in predictable ways toward an object or class of objects. A leader may have an "attitude" toward participative management, for example; he or she would then tend strongly to behave according to that predisposition.
Behavior. What a person actually does. These are the acts of the individual, how he/she does things. Behavior is often observable, but thinking (which is a form of behavior) is not unless the person behaves expressively, that is, talks out what he/she is thinking.
Competency. A developed ability. A single competency can subsume multiple skills In order for a leader to be competent in facilitating meetings, for example, he/she must be reasonably skillful in meeting design, harmonizing conflicts, initiating change, observing and manipulating group process, managing participation, etc.
Knowledge. What an individual knows and understands. He/she may have accumulated knowledge through experience, training, or formal education. What the person does with the knowledge is, of course, another dimension.
Management. Traditionally: planning, organizing, controlling, and evaluating the work of others. Modern organizational thinkers emphasize leadership more than management; Bennis (1985) says, "Manage yourself and lead others."
Potential. The predicted future performance of an individual. This can be an overall expectation (our "High Potentials"), or it can refer to a probable undeveloped or underdeveloped ability.
Performance. Work output against expectations. This is the set of outcomes that the personachieves on his/her goals. This is the data base on which "appraisal" and "review" ("how good") are carried out.
Practice. The person's acting out of his/her (or the organization's) policies and strategies. The individual's work behavior may, for example include coaching subordinates, delegating challenging tasks, and involving people in decisions that affect them.
Skill. Any work behavior that can be improved. This is more specific than competency. Skills involve "micro behaviors"; for example, listening (often referred to as a skill) requires the capacity for accuracy in several dimensions, such as observing nonverbal cues, interpreting intended meanings, attending to the emotional overlay of messages, and probably many more.
Style. Patterns of behavior. A person's leadership "style," for example is his/her characteristic ways of behaving in order to get work done through others.
Trait. Personal characteristic. An individual may be achievement-oriented, aggressive, amiable, arrogant, assertive, astute, etc. Many personality dimensions are not observable and, therefore, are not ratable reliably and validly by others.
It is important to tie the content of 360º feedback instruments to the purposes of the intervention. "Measuring everything that moves" and testing "pet theories" can lead to over assessment and a perversion of purpose. Ask yourself, "What are we up to here? What do we hope to accomplish for the benefit of the organization?"
Working from Models
We believe that it is useful to engage in model-building before attempting to develop instruments for 360º feedback. Our generalized leadership model, below, shows the interrelationships among traits, competencies, and leadership practices. They are best thought of in terms of the organizational context in which they are played out, and they are best interpreted in terms of the results that the organization requires.
In this model the leader's personal characteristics, combined with his/her competencies and "read" of the organizational realities and the demands of the immediate situation, guide practices (behavior) and get results. Knowledge of results can assist in improving leadership practices, developing competencies, and modifying how the leader displays his/her traits.
We call this a Leadership-Outcomes Model because the emphasis is on results rather than "style." Since style is often thought of as immutable, we avoid emphasizing it in our assessments. We have seen little evidence that developmental programs that focus on leadership style actually make a difference in achieving the aims of organizations. In addition, emphasizing style can easily lead to categorizing and stereotyping individuals, as sometimes happens with the use of such instruments as the Myers-Briggs.
A Method for Developing 360º Feedback Instruments
The most thorough approach to ensuring that we have reliable, valid instruments for providing feedback to people to compare how they see themselves and how they are viewed by others is spelled out in the steps below. A given intervention should, in our judgment, be an adaptation of a procedure such as this.
- Getting Top-Management Buy-In. Here you outline the major benefits of 360º feedback, the rationale for the intervention, and the intervention design. You enroll executives in supporting the work through modeling participation. Ideally the intervention begins and ends at the top, with their active participation throughout.
- Administration of our Survey of Needed Competencies. A stratified random sample of managers and supervisors is surveyed to determine which competencies leaders in the organization need to have in order to actualize its vision and carry out its mission. This survey is anonymous, and there is no individualized feedback to respondents, just a summary of findings, distributed after the next step.
- Executive Session. Here you survey the executives with the same instrument. Then you provide a statistical summary of the Survey of Needed Competencies. The executive group then selects the critical competencies to be measured in the target instrument.
- Pilot Study. Draft a questionnaire, selecting appropriate items and a rating scale. Using a volunteer manager, go through all of the phases of gathering 360º assessments, develop a confidential report for the manager, and go over it with him/her. Optionally, you can gather the data in interviews, in which raters are encouraged to talk out loud as they carry out the task. Listen for ambiguities and competencies that the process may have overlooked.
- Development of the Final Instrument. Modify the instrument as needed, and prepare the final form. Get it approved by the executive group.
Developing instruments for 360º feedback requires sharp focus on purpose and attention to content as well as form. Content validity is assured when what the questionnaire measures covers the domains of competencies, skills, traits, and practices that the organization needs in order to accomplish its objectives. For more detailed instructions on how to construct multi-rater instruments, see our manual (Jones & Bearley, 1996b).
References
Bennis, W., & Nanus, B. (1985). Leaders: The Strategies for Taking Charge. New York: Harper & Row.
Jones, J.E., & Bearley, W.L. (1996a). 360° Feedback: Strategies, Tactics, and Techniques for Developing Leaders. Amherst, MA: HRD Press.
Jones, J.E., & Bearley, W.L. (1996b). Surveying Employees: A Practical Guidebook. Amherst, MA: HRD Press.