Start by using your information security objectives to determine your measurement information needs

Monitoring, measurement, analysis, and evaluation

Introduction

Clause 9.1 of all management system standards is entitled “Monitoring, measurement, analysis, and evaluation”. ISO/IEC 27004, by the same name, provides guidance to organisations on how to fulfil the requirements of this clause.

What are its key recommendations?

The key recommendation in ISO/IEC 27004 is to first determine the organisation’s information needs necessary to determine its information security performance and the effectiveness of its ISMS. Knowing what one wants to achieve in making measurements, and the setting of worthwhile goals are key to obtaining the greatest value from an ISMS. Clause 9 corresponds to the CHECK in the Deming PLAN-DO-CHECK-ACT model and allows organisations to take stock of their information security performance and ISMS effectiveness by direct measurements, thereby facilitating informed decisions to be made regarding corrective actions and improvements.

ISO defines monitoring as “determining the status of a system, process or an activity”, whereas measurement is a “process to determine a value”. Thus, monitoring is a series of measurements.  A measure is a variable to which a value is assigned as the result of measurement. Given these definitions, and having determined one’s information needs, the next step is to determine the corresponding measure (or measures), how the measure should be evaluated, calculated, or scored, and the desired result of the measurement or target.

ISO/IEC 27004 also recommends the use of measurement programmes. For example, some organisations are known to have measurement programmes with Key Performance Indicators (KPIs), each with an associated measurement construct, which are derived from their business objectives and are updated each year. Note, however, that having a measurement programme is not an ISO/IEC 27001 requirement.

Where is a good place to start?

A good place to start is with your information security objectives. Edition 3 of ISO/IEC 27001 introduces a direct linkage between Clauses 6.2 and 9.1. Clause 6.2 d) says that your information security objectives shall be monitored, and Clause 9.1 presents the requirements for monitoring.

A principle of corporate governance is that the exploitation of opportunities to fulfil the business objectives encounters risks, which must be managed. The process is illustrated in the figure to the right of this text.

Want to see an example?

Determine key information needs

As an example, a business objective of an ISMS consultancy is to make a profit. Its ability to produce quality software could be exploited to “increase productivity through self-developed SAAS”. This opportunity exploitation is accompanied by two principle risks:

• The SAAS fails to produce ISMS that conform to ISO/IEC 27001.
• The SAAS is insecure.

If any of these risks is not an an IS risk, it would be discounted from further consideration at this point. However, both are IS risks. Treatment is straightforward:

• Gain experience of using the SAAS in the consultancy’s own ISMS and its client’s ISMS.
• Gain confidence in SAAS security through testing and live operations.

These two treatment plans give rise, respectively to two IS objectives.

• Use the SAAS to establish consultancy’s own ISMS.
• Assure SAAS security.

Thus, we have determined two essential inputs to Clause 9.1.

Determine what else you want to know

The intent of Clause 9.1 is to determine the extent to which planned activities are realised and planned results are achieved. For the fulfilment of the first IS objective there are several relevant measures:

• progress of the ISMS build against its project plan
• fulfilment of planned risk assessments, internal audits and management reviews
• timely achievement of ISMS-relevant actions
• achievement of certification

and for the second:

• client bug reports and incidents
• security anomaly reports and attempted attacks
• results of OWASP testing
• SAAS bugs and enhancements
• ICT performance measurements (for botnet detection).

It should be evident from this list, that the measurements include the organisation’s IS processes and controls.

Create measurement constructs

ISO/IEC 27004 (Monitoring, measurement, analysis and evaluation) advocates the use of measurement constructs to detail the how, what, who and when for each measure.

Create a programme

The measurement constructs specify when the measurements are to take place. Putting these together creates a measurement programme.

Review, learn and update

As the measurements progress, you will learn about your IS performance and ISMS effectiveness. Since business objectives are traditionally set on an annual basis, reformation of those objectives should herald a review and reformation of your measurement programme.

Want to know more or need some help?

David Brewer’s book “An introduction to ISO/IEC 27001:2022/Amd 1:2024”, gives a detailed explanation of the whole of ISO/IEC 27001:2022/Amd 1:2024.There is a detailed section on monitoring, measurement, analysis and evaluation. It is available on Amazon in e-book and paperback formats.

IMS-Smart also provides ISMS consultancy and we can help you to better your measurement programmes and increase the value of your ISMS to your organisation. Use the Ask facility to enquire. We will be pleased to assist.

How should my measurement programmes develop?

ISO/IEC 27004 describes two types of measure — performance and effectiveness — terms which should not be confused with the same terms as used in the goal of evaluating “information security performance and ISMS effectiveness”.

Performance measures are measures based on intentions, whereas effectiveness measures consider how well something meets its objectives, for example:

• performance: “is our training program meeting its planned targets of so many people being trained per week?”
• effectiveness: “do our training sessions meet their training objectives?” and (to test the training objectives themselves) “are our training sessions having the desired effect on reducing the number of internal security incidents?”

Effectiveness measures are better at meeting the goal of evaluating information security performance and ISMS effectiveness. However, in practice they are more difficult to design. In this example, one would need to somehow measure the trainees’ understanding before and after the training, and the influence of training and other measurable quantities, such as the number of incidents caused by user error.

For this reason, ISO/IEC 27004 advocates starting with performance measures, and augmenting these with effectiveness measures as the ISMS matures.

Is there a scientific basis for this?

Yes — it is the science of metrology.

Metrology is the science of measurements. It refers to information need and shows how so-called base measures can be combined to create derived measures from which the information need can be satisfied.

As an example of base measures and derived measures consider the measurement of vehicle speed. The base measures are distance travelled and time, the distance travelled traditionally being measured by wheel rotation and latterly by GPS, and time by a clock. Speed is a derived measure and is calculated, i.e., derived, by dividing the distance travelled by time.

Conclusions

ISO/IEC 27001, Clause 9.1 is an important requirement as its diligent fulfilment facilitates informed decisions to be made about corrective actions and improvements to an ISMS.