<Previous Lesson

Organization Development

Next Lesson>

Lesson#25

Evaluating and Institutionalizing Organization Development Interventions

Evaluating and Institutionalizing Organization Development Interventions

This lecture focuses on the final stage of the organization development cycle— evaluation and institutionalization. Evaluation is concerned with providing feed-back to practitioners and organization members about the progress and impact of interventions. Such information may suggest the need for further diagnosis and modification of the change program, or it may show that the intervention is successful. Institutionalization involves making a particular change a permanent part of the organization’s normal functioning. It ensures that the results of successful change programs persist over time. Evaluation processes consider both the implementation success of the intended intervention and the longterm results it produces. Two key aspects of effective evaluation are measurement and research design. Time institutionalization or long- term persistence of intervention effects is examined in a framework showing the organization characteristics, intervention dimensions, and processes contributing to institutionalization of OD interventions in organizations.
Evaluating OD Interventions:


Assessing organization development interventions involves judgments about whether an intervention has been implemented as intended and, if so, whether it is having desired results. Managers investing resources in OD efforts increasingly are being held accountable for results—being asked to justify the expenditures in terms of hard, bottom-line outcomes. More and more, managers are asking for rigorous assessment of OD interventions and are using the results to make important resource allocation decisions about OD, such as whether to continue to support the change program, to modify or alter it, or to terminate it and try something else. Traditionally, OD evaluation has been discussed as something that occurs after the intervention. That view can be misleading, however. Decisions about the measurement of relevant variables and the design of the evaluation process should be made early in the OD cycle so that evaluation choices can be integrated with intervention decisions. There are two distinct types of OD evaluation—one intended to guide the implementation of interventions and another to assess their overall impact. The key issues in evaluation are measurement and research design.

Implementation and Evaluation Feedback:


Most discussions and applications of OD evaluation imply that evaluation is something done after intervention. It is typically argued that once the intervention is implemented, it should be evaluated to discover whether it is producing intended effects. For example, it might be expected that a job enrichment program would lead to higher employee satisfaction and performance. After implementing job enrichment, evaluation would involve assessing whether these positive results indeed did occur. This after-implementation view of evaluation is only partially correct. It assumes that interventions have been implemented as intended and that the key purpose of evaluation is to assess their effects. In many, if not most, organization development programs, however, implementing interventions cannot be taken for granted. Most OD interventions require significant changes in people’s behaviors and ways of thinking about organizations, but they typically offer only broad prescriptions for how such changes are to occur. For example, job enrichment calls for adding discretion, variety, and meaningful feedback to people’s jobs. Implementing such changes requires considerable learning and experimentation as employees and managers discover how to translate these general prescriptions into specific behaviors and procedures. This learning process involves much trial and error and needs to be guided by information about whether behaviors and procedures are being changed as intended. Consequently, we should expand our view of evaluation to include both during-implementation assessment of whether interventions are actually being implemented and after-implementation evaluation of whether they are producing expected results. Both kinds of evaluation provide organization members with feedback about interventions. Evaluation aimed at guiding implementation may be called implementation feedback, and assessment intended to discover intervention outcomes may be called evaluation feedback. Figure 36 shows how the two kinds of feedback fit with the diagnostic and intervention stages of OD. The application of OD to a particular organization starts with a thorough diagnosis of the situation, which helps identify particular organizational problems or areas for improvement, as well as likely causes underlying them. Next, from an array of possible interventions, one or some set is chosen as a means of improving the organization. The choice is based on knowledge linking interventions to diagnosis and change management. In most cases, the chosen intervention provides only general guidelines for organizational change, leaving managers and employees with the task of translating those guidelines into specific behaviors and procedures. Implementation feedback informs this process by supplying data about the different features of



the intervention itself and data about the immediate effects of the intervention. These data, colleted repeatedly and at short intervals, provide a series of snapshots about how the intervention is progressing. Organization members can use this information, first, to gain a clearer understanding of the intervention (the kinds of behaviors and procedures required to implement it) and, second, to plan for the next implementation steps. This feedback cycle might proceed for several rounds, with each round providing members with knowledge about the intervention and ideas lot the next stage of implementation.
 Implementation and Evaluation Feedback


Once implementation feedback informs organization members that the intervention is sufficiently in place, evaluation feedback begins, in contrast to implementation feedback, it is concerned with the overall impact of the intervention and with whether resources should continue to be allocated to it or to other possible interventions. Evaluation feedback takes longer to gather and interpret than does implementation feedback. It typically includes a broad array of outcome measures, such as performance job satisfaction, absenteeism, and turnover. Negative results on these measures tell members either that the initial diagnosis was seriously flawed or that tile wrong intervention was chosen. Such feedback might prompt additional diagnosis and a search for a more effective intervention. Positive results, on the other hand, tell members that the intervention produced expected outcomes and might prompt a search for ways to institutionalize the changes, making them a permanent part of the organizations normal functioning. An example of a job enrichment intervention helps to clarity the OD stages and feedback linkages shown in Figure 36. Suppose the initial diagnosis reveals that employee performance and satisfaction are low and that jobs being overly structured and routinized is an underlying cause of this problem. An inspection of alternative interventions to improve productivity and satisfaction suggests that job enrichment might be applicable for this situation. Existing job enrichment theory proposes that increasing employee discretion, task variety, and feedback can lead to improvements in work quality and attitudes and that this job design and outcome linkage is especially strong for employees who have growth needs—needs for challenge, autonomy, and development. Initial diagnosis suggests that most of the employees have high growth needs and that the existing job designs prevent the fulfillment of these needs. Therefore, job enrichment seems particularly suited to this situation.



Managers and employees now start to translate the general prescriptions offered by job enrichment theory into specific behaviors and procedures. At this stage, the intervention is relatively broad and must be tailored to fit the specific situation. To implement the intervention, employees might decide on the following organizational changes: job discretion can be increased through more participatory styles of supervision; task variety can be enhanced by allowing employees to inspect their job outputs; and feedback can be made more meaningful by providing employees with quicker and more specific information about their performances. After three months of trying to implement these changes, the members use implementation feedback to see how the intervention is progressing. Questionnaires and interviews (similar to those used in diagnosis) are administered to measure the different features of job enrichment (discretion, variety, and feedback) and to assess employees’ reactions to the changes. Company records are analyzed to show the short-term effects on productivity of the intervention. The data reveal that productivity and satisfaction have changed very little since the initial diagnosis. Employ perceptions of job discretion and feedback also have shown negligible change, but perceptions of task variety have shown significant improvement. In-depth discussion and analysis of this first round of implementation feedback help supervisors gain a better feel for the kinds of behaviors needed to move toward a participatory leadership style. This greater clarification of one feature of the intervention leads to a decision to involve the supervisors in leadership training to develop the skills and knowledge needed to lead anticipatively. A decision also is made to make job feedback more meaningful by translating such data into simple bar graphs, rather than continuing to provide voluminous statistical reports. After these modifications have been in effect for about three months, members institute a second round of implementation feedback to see how the intervention is progressing. The data now show that productivity and satisfaction have moved moderately higher than in the first round of feedback and that employee perceptions of task variety and feedback are both high. Employee perceptions of discretion, however, remain relatively low. Members conclude that the variety and feedback dimensions of job enrichment are sufficiently implemented but that the discretion component needs improvement. They decide to put more effort into supervisory training and to ask OD practitioners to provide online counseling and coaching to supervisors about their leadership styles. After four more months, a third round of implementation feedback occurs. The data now show that satisfaction and performance are significantly higher than in the first round of feedback and moderately higher than in the second round. The data also show that discretion, variety, and feedback are all high, suggesting that the job enrichment intervention has been successfully implemented. Now evaluation feedback is used to assess the overall effectiveness of the program. The evaluation feedback includes all the data from the satisfaction and performance measures used in the implementation feedback. Because both the immediate and broader effects of the intervention are being evaluated, additional outcomes are examined, such as employee absenteeism, maintenance costs, and reactions of other organizational units not included in job enrichment. The full array of evaluation data might suggest that after one year from the start of implementation, the job enrichment program is having expected effects and thus should be continued and made more permanent.
Measurement:


Providing useful implementation and evaluation feedback involves two activities: selecting the appropriate variables and designing good measures.

Selecting Variables


: Ideally, the variables measured in OD evaluation should derive from the theory or conceptual model underlying the intervention. The model should incorporate the key features of the intervention as well as its expected results. The general diagnostic models described in Chapters 5 and 6 meet these criteria, as do the more specific models introduced in Chapters 12 through 20. For example, the job-level diagnostic model described in Chapter 6 proposes several major features of work: task variety, feedback, and autonomy. The theory argues that high levels of these elements can be expected to result in high levels of work quality and satisfaction. In addition, as we shall see in Chapter 16, the strength of this relationship varies with the degree of employee growth need: the higher the need, the more that job enrichment produces positive results. The job-level diagnostic model suggests a number of measurement variables for implementation and evaluation feedback. Whether the intervention is being implemented could be assessed by determining how many job descriptions have been rewritten to include more responsibility or how many organization members have received cross-training in other job skills. Evaluation of the immediate and long- term impact of job enrichment would include measures of employee performance and satisfaction over time.



Again, these measures would likely be included in the initial diagnosis, when the company’s problems or areas for improvement are discovered. Measuring both intervention and outcome variables is necessary for implementation and evaluation feedback. Unfortunately, there has been a tendency in OD to measure only outcome variables while neglecting intervention variables altogether. It generally is assumed that the intervention has been implemented and attention, therefore, is directed to its impact on such organizational outcomes as performance, absenteeism, and satisfaction. As argued earlier, implementing OD interventions generally take considerable time and learning. It must be empirically determined that the intervention has been implemented; it cannot simply be assumed. Implementation feedback serves this purposes guiding the implementation process and helping to interpret outcome data Outcome measures are ambiguous without knowledge of how well the intervention has been implemented. For example, a negligible change in measures of performance and satisfaction could mean that the wrong intervention has been chosen, that the correct intervention has not been implemented effectively, or that the wrong variables have been measured. Measurement of the intervention variables helps determine the correct interpretation of outcome measures. As suggested above, the choice of intervention variables to measure should derive from the conceptual framework underlying the OD intervention. OD research and theory increasingly have come to identify specific organizational changes needed to implement particular interventions. These variables should guide not only implementation of the intervention but also choices about what change variables to measure for evaluative purposes. Additional sources of knowledge about intervention variables can be found in the numerous references at the end of each of the intervention chapters in this book and in several of the books in the Wiley Series on Organizational Assessment and Change. The choice of what outcome variables to measure also should be dictated by intervention theory, which specifies the kinds of results that can be expected from particular change programs. Again, the material in this book and elsewhere identifies numerous outcome measures, such as job satisfaction, intrinsic motivation, organizational commitment, absenteeism, turnover, and productivity. Historically, OD assessment has focused on attitudinal outcomes, such as job satisfaction, while neglecting hard measures, such as performance. Increasingly, however, managers and researchers are calling for development of behavioral measures of OD outcomes. Managers are interested primarily in applying OD to change work-related behaviors that involve joining, remaining, and producing at work, and are assessing OD more frequently in terms of such bottom-line results. Macy and Mirvis have done extensive research to develop a standardized set of behavioral outcomes for assessing and comparing intervention results.

<Previous Lesson

Organization Development

Next Lesson>

Home

Lesson Plan

Topics

Go to Top

Copyright © 2008-2013 zainbooks All Rights Reserved
Next Lesson
Previous Lesson
Lesson Plan
Topics
Home
Go to Top