Statline article from The Quality Magazine, 7(1), February 1998, 71-72

What to Measure about Organisational Performance

Teresa Dickinson, Ian Saunders and Doug Shaw

Managers need to know about the overall success of their organisation. They need to know how it has performed in the past and also how likely it is to be successful in the future. How should an organisation define its success and measure its achievements?

This, the fifth article in our series on measurement, focuses on what to measure at the organisational or business unit level. It forms a natural companion to the previous article, which concentrated on measurement at the operational process level.

All enterprises, whether in the public or private sectors, whether they are profit or not-for-profit organisations, have stakeholders. Stakeholders are individuals or groups who have an interest in how the enterprise performs because it affects them in some way – that is they have a stake in the organisation. Traditionally, the owners of a business (for example, shareholders in a publicly listed company or the government in public service situations) were seen as the most important stakeholders, and meeting their requirements was the sole reason for the existence of the enterprise.

A more contemporary view is that there is a broader range of stakeholders, all of whose requirements must all be met in order for an enterprise to be considered successful. This view is reflected strongly in criteria for awards in quality and business excellence such as the Baldrige Award and the Australian Quality Awards. Such stakeholder groups will include Customers, Suppliers, Staff and Communities (where communities may be geographical, professional, business communities etc.).]

Under the stakeholder view of the business, stakeholders are the final judges of organisational performance. A successful business will be one that properly balances the requirements of all stakeholders. Senior managers must be able to ascertain the extend to which these requirements are being met. This involves determining what stakeholders require of the organisation, and measuring performance against those requirements. Success measurements are the set of measurements that together indicate the extent to which the stakeholder requirements are met.

Success measurements can only be determined once a business has identified its stakeholders, the requirements of each stakeholder group are known, and it is understood how stakeholders will judge whether their requirements have been met. Stakeholder requirements and success measurements are key inputs to the long term strategic planning of any organisation. Strategic planning is about how the organisation will satisfy the requirements of its stakeholders.

Defining appropriate success measurements is not always a simple matter. It involves a focus on what stakeholders want rather than what the organisation is delivering. Often determining success measurements requires a long-term view.

For example, consider a school as an enterprise. Community requirements may include intangible concepts such as "former students making a useful contribution to society" and "former students enjoying a life-long love of learning". These cannot be measured in the short term and a process of translation from the long-term outcome to short term measurable results is required. Measurable quantities in this example would include the percentage of students continuing with further education, success of former students, and views of recent students on how the school contributed to their personal growth and love of learning.

Success measurements are collected outside the organisation and they should be:

• exhaustive, covering the full complement of stakeholder requirements,
• quantifiable,
• collected from stakeholders themselves,
• reliable, that is the measurement must accurately reflect the quantity it is designed to measure,
• repeatable, so that if the quantity being measured remains unchanged over time data collected over time will reflect this, and
• revised as stakeholder needs evolve.

While success measurements answer the questions "How well have we done in the past?", and "How well are we doing now?" organisations need information about how well they are likely to meet stakeholders’ requirements in the future.

We define key performance indicators (KPIs) (see footnote) as measurements collected internally to an enterprise which help, in conjunction with broader information such as environmental, industry and competitor analysis, predict the future success of the business. KPIs are collected within the business itself, but are related to the external success measurements.

Key performance indicators should also be:

• derived from, and predictive of stakeholder requirements, that is each KPI should directly relate to at least one success measure,
• exhaustive, that is, taken together a suite of KPIs should indicate how well stakeholders requirements have been met,
• reliable and repeatable
• timely, so that organisations can respond to unsatisfactory data before stakeholder satisfaction is adversely affected,
• if possible represent an aggregation of information obtained from process measurements,
• revised as stakeholder requirements change.

Linking the KPIs to the external success measurements may require some research. For example, in the school described above the link of ‘love of learning’ to teaching methodologies could be investigated by a review of education research. This would encourage teachers to use approaches aligned with the stakeholders’ requirements.

Just as success measurements should be reflected in the long-term vision, strategy and goals of an enterprise, KPIs should reflect an organisations’ shorter-term objectives such as are often found in business plans.

While KPIs represent the highest level internal measurements that an organisation uses to assess its performance they are often not directly actionable. In order to control process output as measured by KPIs, enterprises need a system of lower level performance indicators and process measurements, which are actionable.

For example, a communications equipment company had ‘Percentage of customers satisfied with installation performance’ as a success measurement. Customer research indicated that keeping appointments was a major concern for customers, so that one of the KPIs associated with this success measurement was ‘Service appointment kept’. This is measured as the proportion of appointments for which the technician arrives within 5 minutes of the agreed time. If this KPI becomes unsatisfactory the situation can only be remedied by identifying systemic causes of lateness and working to eliminate them. Doing this requires more detailed measurements of timeliness at the region or depot level (localised performance indicators) and process measurements. In this example such process measurements may include data on staff and equipment availability as well as reasons for time overruns on jobs.

It can be seen that in order for an organisation to be able to respond to KIPs, and ultimately affect the successfulness of the business, KPIs must be cascaded down to lower organisational levels and ultimately related to process measurements. Developing such process measurements was discussed in the previous article in this series.

Organisations who understand what requirements their stakeholders have of them, and have measured how well they are doing against those requirements understand how well they are performing. Further, organisations that measure themselves internally in relation to stakeholder requirements are in a strong position to predict how successful they are likely to be in the future. These organisations will have a head start in their ability to respond to performance shortfall, to improve their business and ultimately to control their own destiny.

Measuring organisational performance indicates how well the vision and strategy of the organisation are being translated into outcomes. More than any other marker, looking at how an enterprise chooses to judge its own performance illuminates what matters to the organisation.

The reader should recognise that our series of articles has presented some components of a broader and more comprehensive approach to organisational performance measurement. This broader and more comprehensive approach is the subject of ongoing research and development. For a description of the current state of this research, the reader should see "Using Statistics and Statistical Thinking to Improve Organisational Performance" by S.E. Dransfield, N.I. Fisher and N.J. Vogel, submitted to the International Statistical Review for publication.

Footnote: We are well aware that the term 'key performance indicator' has been given many different meanings.  We choose to define key performance indicators as the small suite of high level key measurements in an organisation or business unit that can be used to predict the success of the organisation in the eyes of its stakeholders.