Statline article from The Quality Magazine, 6(6), December 1997, 67-68.

What to Measure About Processes

Teresa Dickinson, Ian Saunders and Doug Shaw 

Previous articles in this series have looked at why we should measure and why we should use data for decision-making. Having agreed with the need to measure and the need to use measurements (data) in decision-making, the question that causes great difficulty for many managers is "What should we measure?".

In this article we will discuss what should be measured at the operational process level in the organisation, that is, at the level where the production and service processes of the organisation take place. Later articles in the series will address measurement at higher management levels in the organisation.

As discussed in our second article (August, 1997) entitled "Why measure?", measurements are used:

• to monitor and control a process. That is, to have quantitative information about how the process is performing and to use that information to ensure that the process consistently performs as we want it to; and
• to learn and improve. That is, to better understand why the process performs as it does and to use this information to improve its performance

There are two types of measurements that we need to have for a manufacturing or a service process – measurements about how well the process is satisfying the requirements of its customers, and measurements about how well the process is meeting the requirements of the business of which it is a part.

Customers may be internal or external. Whichever they are, we should find out, in measurable terms, what they will regard as a quality outcome from the process. And we should institute measurements on the process to ensure that the requirements are met or exceeded. Sometimes customers can be quite specific about their requirements – setting them out quite clearly in a contract or a specification document, for example. Other times it requires skill and effort on our part to establish customers’ requirements – it’s almost an exercise in psychoanalysis! Internal customers should be treated the same as external customers. Some organisations go as far as written internal customer/supplier contracts.

Examples of process outcome measurements are

• for manufacturing processes -

physical dimensions of the product

percentage of active ingredient in a pharmaceutical product

a measure of reliability such as Mean Time To Failure

• for service processes

elapsed time until service is completed

difference between service person’s arrival time and appointed time

proportion of faults recurring within one month of repair

We must be sure that what we measure as the outcome of the process really represents the customer’s requirements, and that what we choose to measure is not chosen just because it is easy to measure or because we have traditionally measured it. Are we measuring product hardness, when the customer’s real concern is about product durability? If we are, then we need to do some experimentation and data analysis to make sure that hardness and durability are highly correlated. A car manufacturer followed this approach to establish that door closing force was an appropriate measurement to use to check that it was meeting the customers’ requirement that "the door closes easily". (See Eureka, W.G. "Introducing Quality Function Deployment" in Quality Function Deployment, American Supplier Institute, Dearborn, Mich., 1989.)

It is also essential to establish what the customer’s priorities are amongst the various requirements specified. We should not be putting all our effort into measuring product attributes if the customer’s main requirement is on-time delivery.

When measurements are available on the outcomes of the process, we can determine the capability of the process, that is, what the process is capable of achieving. This is valuable information, which can be compared with customers’ requirements to see if those requirements are being, or can be, met. If customer requirements are not being met, process improvement is necessary. Process capability is also valuable information for the marketing function in the organisation. It provides a basis for setting out the organisation’s ‘offering’ to future customers.

The process owner needs to know whether the process is meeting the requirements set for it in the business planning of the organisation. Measurements that might be appropriate in this context are

• operating costs
• amount and value of waste
• yield (some measure of output achieved in relation to amount of raw materials input)
• labour productivity
• staffing levels

The measurements to be used for a particular process will be the subject of negotiation between the process owner and his or her immediate manager, and occur as part of the planning process.

These ‘business requirements’ measurements will be reviewed regularly as part of the process of measuring the organisation’s performance against its plans.

The measurements described above are all outcome measurements. They are all measured after the process has been completed. It is highly desirable, particularly if we want to use measurements to control the process, to have measurements that will tell us during the course of the process how it is performing.  See Figure 1.

With such ‘in-process’ measurements, we can take corrective action before the process is complete and before unsatisfactory products or services result. There are a number of ways of determining which in-process measurements should be used.

• Specialist knowledge about the process (engineering, chemical, etc.) can tell us. For example, if it is known that the pressure level in a particular vessel in a chemical process largely determines the level of impurities in the output, effort needs to be concentrated on measuring and controlling that pressure level.
• Flow-charting of the process, with input from those who work in the process, can reveal appropriate in-process measurements. This is particularly the case for service processes, where total process time is often the main concern of the customer. Flow-charting the process reveals where delays can occur, and this can be verified by data collection and analysis. A large electronics manufacturer applied this approach to its process for inspecting incoming goods – a process with internal customers. Flow-charting of the process, and data collection and analysis, revealed that the major component of total process time was the front-end activity of sorting the incoming items and assigning them to appropriate test procedures. Steps were taken to streamline that part of the process, and to measure, record and monitor the time that part of the process took.
• Statistical analysis of existing data, or data specially collected for the purpose, can reveal which in-process measurements are related to the outcome measurements. Data collection and analysis on a snack food production line showed that dimensional variability in the end product was influenced by the temperature and holding time of a syrup additive, the holding time of the mix in an intermediate hopper, and the temperature and gap setting of a forming roller. For control of the outcome dimensions, regular monitoring of these in-process measurements is required.

The way we choose to measure our processes will send very strong messages to our customers, and to the people who work in the processes, about what we consider to be important. Some approaches to measurement can produce negative effects. See Dickinson, T.A. and Robinson, G.K. (1994) "Remember that what you measure affects behaviour", The Quality Magazine, 3 (1), 83-84, for examples of how inappropriate measurement can encourage staff to act contrary to customer requirements.

Careful selection of process measurements, both outcome and in-process, will provide us with information on how our process is performing in relation to customer requirements, and the ability to control and improve the process to meet business requirements.