Without proper Quality Assurance (QA) controls it would be impossible to measure the success of a Security Awareness Program. It would be impossible to identify areas in need of improvements so that appropriate corrective actions can be carried out to ensure the continued success of the program. The best way to apply Quality Assurance controls for a Security Awareness Program is by way of collecting and analyzing metrics.
Security Awareness metrics has traditionally been challenging, perhaps because there isn’t a lot of academic research or public papers discussing metrics, what to measure, and what the benefits are. The lack of readily available information on metrics could explain the high percentage of organizations that do not collect them as part of their Quality Assurance efforts. In fact, according to recent studies as many as 1/3rd of organizations do not measure the effectiveness of their Security Awareness Program. However, statistics clearly show that applying metrics is essential to any Security Awareness Program. Would employees recognize an incident if it occurred? Would they know what to do? Is your Security Awareness program reducing high-risk behaviors? Is it reinforcing desired behaviors? Is your Security Awareness Program reducing unexpected costs? Metrics is vital to the success of a Security Awareness Program and help to measure its success and to identify areas that need improvement.
Collecting Security Awareness metrics is therefore the first step of The Security Awareness Cycle. The concept is to create a set of baseline metrics that you can compare against as you begin new iterations of the cycle. Let me give you an example. Let’s assume that you have decided to measure the percentage of employees who are aware that there is an Acceptable Use Policy. Ideally, you would want all of your employees to be familiar with this policy and to adhere to it. During your initial baseline metrics you might find that only 50% of your employees are aware of this regulatory policy, so you create and deliver Security Awareness material that informs your employees of the existence of the policy, its importance, and their role and responsibility in adhering to it. During the next iteration of the Security Awareness Cycle you collect metrics again and find that the number has increased to 60%. This number, although not what we might have expected, provides some valuable information. First and foremost it tells us that the Security Awareness program has provided value by successfully increasing the percentage of employees familiar with the policy by 10%. However, it also tells us that we were unsuccessful in making all of our employees aware and that there is still significant room for improvement. Had we not used metrics we would likely have made assumptions that our first pass was sufficient and would have failed to identify the need to strengthen our efforts to reach complete awareness.
Metrics can be collected and measured multiple ways, for example by systems or through employee surveys.
