Start by using your information security objectives to determine your measurement information needs

Monitoring, measurement, analysis, and evaluation

Introduction

Clause 9.1 of all management system standards is entitled “Monitoring, measurement, analysis, and evaluation”. This article explores the purpose of this clause and how it can be fulfilled in the context of ISO/IEC 27001.

What do Annex SL and ISO/IEC 27001 say?

Annex SLAppendix 2 of the ISO/IEC Directives (Part 1 Consolidated ISO Supplement), Annex SL specifies the high level structure and identical core text for all ISO management system standards (MSS)., Clause 9.1 begins with a requirement to determine what needs to be monitored and measured and ends with a requirement to evaluate the XXX performance and effectiveness of the XXX MS. ISO/IEC 27001 reproduces this requirement with the following changes:

• XXX, the discipline place-holder, has been replaced with “information security“
• A recommendation has been added to list item b): “The methods selected should produce comparable and reproducible results to be considered valid.”
• A new list item “d) who shall monitor and measure.“ has been added
• A new list item “f) who shall analyse and evaluate these results.” has been added.

The recommendation is considered problematic by some experts as there circumstances when measurement results are not repeatable (consider a roulette wheel, for example). However, ISO/IEC 27004 provides a detailed explanation.

What is ISO/IEC 27004?

ISO/IEC 27004 – Monitoring, measurement, analysis and evaluation – is a guidance standard in support of ISO/IEC 27001. It takes its name from Clause 9.1 of ISO/IEC 27001 and provides guidance to organisations on how to fulfil the requirements of that clause.

What are its key recommendations?

The key recommendation in ISO/IEC 27004 is to first determine the organisation’s information needs necessary to determine its information security performance and the effectiveness of its ISMS. Knowing what one wants to achieve in making measurements, and the setting of worthwhile goals are key to obtaining the greatest value from an ISMS. Clause 9 corresponds to the CHECK in the Deming PLAN-DO-CHECK-ACT model and allows organisations to take stock of their information security performance and ISMS effectiveness by direct measurements, thereby facilitating informed decisions to be made regarding corrective actions and improvements.

ISO defines monitoring as “determining the status of a system, process or an activity”, whereas measurement is a “process to determine a value”. Thus, monitoring is a series of measurements.  A measure is a variable to which a value is assigned as the result of measurement. Given these definitions, and having determined one’s information needs, the next step is to determine the corresponding measure (or measures), how the measure should be evaluated, calculated, or scored, and the desired result of the measurement or target.

ISO/IEC 27004 also recommends the use of measurement programmes. For example, some organisations are known to have measurement programmes with Key Performance Indicators (KPIs), each with an associated measurement construct, which are derived from their business objectives and are updated each year. Note, however, that having a measurement programme is not an ISO/IEC 27001 requirement.

Where is a good place to start?

A good place to start is with your information security objectives. Edition 3 of ISO/IEC 27001 introduces a direct linkage between Clauses 6.2 and 9.1. Clause 6.2 d) says that your information security objectives shall be monitored, and Clause 9.1 presents the requirements for monitoring.

How should my measurement programmes develop?

ISO/IEC 27004 describes two types of measure — performance and effectiveness — terms which should not be confused with the same terms as used in the goal of evaluating “information security performance and ISMS effectiveness”.
Performance measures are measures based on intentions, whereas effectiveness measures consider how well something meets its objectives, for example:

• performance: “is our training program meeting its planned targets of so many people being trained per week?”
• effectiveness: “do our training sessions meet their training objectives?” and (to test the training objectives themselves) “are our training sessions having the desired effect on reducing the number of internal security incidents?”

Effectiveness measures are better at meeting the goal of evaluating information security performance and ISMS effectiveness. However, in practice they are more difficult to design. In this example, one would need to somehow measure the trainees’ understanding before and after the training, and the influence of training and other measurable quantities, such as the number of incidents caused by user error.

For this reason, ISO/IEC 27004 advocates starting with performance measures, and augmenting these with effectiveness measures as the ISMS matures.

Is there a scientific basis for this?

Yes — it is the science of metrology.

Metrology is the science of measurements. It refers to information need and shows how so-called base measures can be combined to create derived measures from which the information need can be satisfied.

As an example of base measures and derived measures consider the measurement of vehicle speed. The base measures are distance travelled and time, the distance travelled traditionally being measured by wheel rotation and latterly by GPS, and time by a clock. Speed is a derived measure and is calculated, i.e., derived, by dividing the distance travelled by time.

Conclusions

ISO/IEC 27001, Clause 9.1 is an important requirement as its diligent fulfilment facilitates informed decisions to be made about corrective actions and improvements to an ISMS.