 |
|
|
|
|
|
|
|
 |
|
|
|||||
 |
 |
|
|
||||
 |
 |
 |
 |
|
|||
 |
|
||||||
| |||||||||||||||||||||||||||||
|
Statline article from The Quality Magazine,
7(2), April 1998, 69-72
Questioning Your Measurement System Teresa Dickinson, Ian Saunders and Doug Shaw CSIRO Mathematical and Information SciencesIntroduction The previous articles in this series have addressed various aspect of measurement and how it can be used to enhance organisational performance. This article brings together some of the threads and suggests some questions that will help investigate how an overall system of measurement should be assessed. Questioning is a powerful method of discovering opportunities to improve. The questions we suggest are superficially simple, but they may require substantial effort to answer. Even the basic question of ‘Who are our stakeholders?’ leads to substantial discussion about the purpose and direction of an organisation. Our focus in this article is on the system of measurement that organisations use to manage their performance. Solving individual problems will often require additional data collection and specific analysis. As we noted in an earlier article in this series, what is measured affects what is done. The questioning approach we outline here assesses the alignment of the measurement system with the overall goals of an organisation. A properly aligned measurement system drives the behaviour needed for success. Those organisations using the Australian Quality Awards Framework as a basis for assessment may also find useful material in an earlier article ‘Self-Assessment Using the Australian Quality Awards Criteria. Part 5. Data and Information in the Australian Quality Awards Criteria’, The Quality Magazine, 4 (5) (1995), 10-17. The examples in the present article are drawn from the Australian Quality Awards Case Studies published by the Australian Quality Council. The examples may not necessarily reflect current practice in these organisations. ‘What to Measure about Processes’, The Quality Magazine, 6 (6) (1997), 67-68. See also ‘Remember that what you measure affects behaviour’, The Quality Magazine, 3 (1) (1994), 83-84. ‘What to Measure about Organisational Performance’ The Quality Magazine, 7 (1) (1998), 71-72 Creating Alignment The previous article in this series gave a structure for developing an aligned measurement system.
Figure 1. Sequence for defining performance indicators Success measurements record our performance in satisfying the needs of our stakeholders. Key Performance Indicators are internal measurements that predict our future performance at a corporate level. To identify actions that are needed to improve performance, these must be translated to localised performance indicators at the departmental and process level. Stakeholders and Organisational Mission The value of any organisational activity must be assessed in terms of the benefits it brings to the organisation’s stakeholders. A measurement system supports this evaluation by identifying the contribution of activities and processes to stakeholder value. The stakeholder groups may be identified via a mission statement for the organisation. In any case, analysis of stakeholder needs must form the basis of the measurement system. Questions for an organisation Who are your stakeholders? How do they benefit from your operation? How do you measure the benefits they receive? How do you measure progress towards achieving your mission? While a mission should include all of an organisation’s stakeholders, it is essential that customers be attracted and retained. The needs of customers are likely to fluctuate from day to day, in contrast to the other stakeholder groups. The next questions therefore concern how well the mission relates to increasing customer satisfaction and loyalty. Understanding Customer Needs Since customers are the recipients of the organisation’s products and services, monitoring of their satisfaction is an essential part of the measurement system. This will also include determining of the needs of potential customers by direct contact or through market research. Questions for an organisation Who are your customers? What is your market? How do you measure your customers’ needs and their satisfaction? How do you determine the needs of potential customers? How do you measure customer loyalty? Example Enhance Systems, a textile products rental company, monitors:
The next step is to understand how customer needs translate into processes. Key Performance Indicators To link all of the parts of the system together, we need to understand how the process measurements support the overall measurement of performance of the organisation. The final outputs of the processes are aimed at meeting the needs of customers and other stakeholders. The stakeholder success measurements will identify the stakeholders’ view of our organisation, but this is inevitably a lagging indicator. The final outcomes such as defect-free products, timely and accurate service, attractive return on investment and other stakeholder requirements are the result of a range of factors that can be monitored internally. Questions for organisations What internal factors influence the outcomes for stakeholders? How do you measure these factors? What are appropriate targets for these factors that will lead to satisfaction of stakeholder needs? Example Ford Motor Company has a defined methodology that links information on customer satisfaction to the ‘significant characteristics’ of their vehicle. These characteristics can then be monitored and controlled. Understanding Processes Achieving the outputs that customers require, while meeting the needs of other stakeholders for efficiency, working conditions, financial returns, and so on, requires effective measurement of processes. This will require monitoring of such aspects as throughput, inventory, efficiency, process capability, cycle time and quality. The quality measurements provide a direct link from customer needs to the process. It is important that they relate as directly as possible to the customers’ requirements. When direct measurement is not feasible, it is important to ensure that a surrogate measurement is appropriate. For example, the measurement of tensile strength of vehicle axle nuts was used as a surrogate for durability. The supplier of steel, in order to achieve greater capability lifted their tensile strength of this steel, but in doing so increased the brittleness. Suddenly many more axle nuts snapped on vehicles. This increased customer complaints and re-work. Measurements give information about individual items, and also processes. The process information is often more important because, while individual item information tells us about the past and present, it is process information that tells us about the future.
Individual item measurements answer the question ‘Did we do a good job?’, while process measurements answer the vital questions ‘Can we do a good job?’ and ‘Will we do a good job?’ Questions for organisations What processes are you involved in? What are the most important outcomes of this process? How do you measure the outcomes? Are the measurements linked to the needs of the customers of the process? Are they improving? How do you know? How do you know that the outcomes will be achieved in the future? Examples Enhance Systems: Study of variation in rewash figures led to separation of items TVS Partnership: Rework costs monitored weekly NRMA: Monitoring of error rates of keypunch staff identified areas for elimination of checking Measurement of processes inevitably leads to thinking about variation. Variation A universal feature of processes is variation. Variation makes decisions more difficult but must be allowed for to reach proper conclusions. In assessing its use of data, an organisation should evaluate how far the inevitable variation in processes is measured and allowed for in planning and managing processes. Some variation is predictable, such as seasonal or weekly patterns or differences between demands at different branches. This predictable variation does not give new information and should be removed before analysis. Other variation is unpredictable. Excessive variation in manufacturing processes leads to products that are outside specification and must be rejected or reworked. Unpredictable variation in demand and production levels may have to be allowed for through ‘safety margins’ of extra staff, inventory or resources. All of these are costs to the organisation. A central issue for a process is its ‘capability’: its ability to deliver what its customers require. The key aspects of a product or service, perhaps the dimensions of a component, or the waiting time for a service, need to be identified. Then determining capability requires an understanding of variation in these key aspects: What is the range of variation? Only if this range is well within the needs of the customer can we be sure of satisfying those needs. Questions for organisations How much unpredictable variation is present in your processes? Are the processes ‘capable’? What are the causes of the variation? What are you doing to allow for variation in the future? How do you cope with unpredictable variation? How much is it costing you? Examples Few good examples are available. Manufacturing companies may monitor process capability as an overall indicator of the impact of variation. Ford and Ford Plastics monitored their ‘significant characteristic capability index’ defined as the percentage of significant characteristics that were stable and capable. However, the costs of variation are apparently not commonly monitored. A system of measurements How should an organisation deal with the answers to these questions? The answers will highlight areas where there is a lack of measurement or a lack of integration. For example, if there are no measurements relating to a particular stakeholder group, then there is an opportunity to work with that group to develop an appropriate set of success measurements. If the performance indicators for a department are not related to the overall key performance indicators, there is a need to review the indicators and perhaps also the individual roles of managers. It is very tempting to develop measurements separately for each process and each department. This piecemeal approach is simpler than undertaking the consultation and negotiation required for an integrated system. However, the piecemeal approach leads to confusion and conflict. Developing measurements as a system, with stakeholder needs giving a common direction, avoids these problems. Regular review of the measurements will keep them aligned with changing stakeholder needs and priorities. An integrated and reliable measurement system can act as the information centre for achieving organisational success. CSIRO Mathematical and Information Sciences has an ongoing research project focussed on the use of data and information for managing. Please call Ian Saunders for further information about this work on (08) 8303 8788. |
