Information Security Management Metrics: A Definitive Guide to Effective Security Monitoring and Measurement

212 Pages·2009·11.696 MB·English

by  W. Krag Brotby CISM

Information
Security
Management
Metrics
A Definitive Guide to Effective
Security Monitoring and Measurement
© 2009 by Taylor & Francis Group, LLC
AU5285.indb 1 2/20/09 7:46:54 PM
OTHER INFORMATION SECURITY BOOKS FROM AUERBACH
Architecting Secure Software Systems Information Technology Control and Audit,
Asoke K. Talukder and Manish Chaitanya Third Edition
ISBN: 978-1-4200-8784-0 Sandra Senft and Frederick Gallegos
ISBN: 978-1-4200-6550-3
Building an Effective Information Security Policy
Architecture Intelligent Network Video: Understanding Modern
Sandy Bacik Video Surveillance Systems
ISBN: 978-1-4200-5905-2 Fredrik Nilsson
ISBN: 978-1-4200-6156-7
Business Resumption Planning, Second Edition
Leo A. Wrobel IT Auditing and Sarbanes-Oxley Compliance:
ISBN: 978-0-8493-1459-9 Key Strategies for Business Improvement
Dimitris N. Chorafas
CISO Leadership: Essential Principles for Success
ISBN: 978-1-4200-8617-1
Todd Fitzgerald and Micki Krause
ISBN: 978-0-8493-7943-7 Malicious Bots: An Inside Look into the
Cyber-Criminal Underground of the Internet
CISO Soft Skills: Securing Organizations Impaired by
Ken Dunham and Jim Melnick
Employee Politics, Apathy, and Intolerant Perspectives
ISBN: 978-1-4200-6903-7
Ron Collette, Michael Gentile, and Skye Gentile
ISBN: 978-1-4200-8910-3 Oracle Identity Management: Governance, Risk,
and Compliance Architecture, Third Edition
Critical Infrastructure: Understanding Its Component Parts,
Marlin B. Pohlman
Vulnerabilities, Operating Risks, and Interdependencies
ISBN: 978-1-4200-7247-1
Tyson Macaulay
ISBN: 978-1-4200-6835-1 Profiling Hackers: The Science of Criminal
Profiling as Applied to the World
Cyber Fraud: Tactics, Techniques and Procedures
of Hacking
Rick Howard
Raoul Chiesa, Stefania Ducci, and Silvio Ciappi
ISBN: 978-1-4200-9127-4
ISBN: 978-1-4200-8693-5
Enterprise Systems Backup and Recovery: A Corporate
Security in an IPv6 Environment
Insurance Policy
Daniel Minoli and Jake Kouns
Preston de Guise
ISBN: 978-1-4200-9229-5
ISBN: 978-1-4200-7639-4
Security Software Development: Assessing
How to Complete a Risk Assessment in 5 Days or Less
and Managing Security Risks
Thomas R. Peltier
Douglas A. Ashbaugh
ISBN: 978-1-4200-6275-5
ISBN: 978-1-4200-6380-6
How to Develop and Implement a Security Master Plan
Software Deployment, Updating, and Patching
Timothy Giles
Bill Stackpole and Patrick Hanrion
ISBN: 978-1-4200-8625-6
ISBN: 978-0-8493-5800-5
HOWTO Secure and Audit Oracle 10g and 11g
Terrorist Recognition Handbook: A Practitioner’s
Ron Ben-Natan
Manual for Predicting and Identifying Terrorist
ISBN: 978-1-4200-8412-2
Activities, Second Edition
Information Assurance Architecture Malcolm Nance
Keith D. Willett ISBN: 978-1-4200-7183-2
ISBN: 978-0-8493-8067-9
21st Century Security and CPTED: Designing
Information Security Management Handbook, Sixth for Critical Infrastructure Protection and
Edition, Volume 3 Crime Prevention
Harold F. Tipton and Micki Krause, Editors Randall I. Atlas
ISBN: 978-1-4200-9092-5 ISBN: 978-1-4200-6807-8
Information Security Management Metrics: A Definitive Understanding and Applying Cryptography and
Guide to Effective Security Monitoring and Measurement Data Security
W. Krag Brotby Adam J. Elbirt
ISBN: 978-1-4200-5285-5 ISBN: 978-1-4200-6160-4
AUERBACH PUBLICATIONS
www.auerbach-publications.com
(cid:53)(cid:80)(cid:1)(cid:48)(cid:83)(cid:69)(cid:70)(cid:83)(cid:1)(cid:36)(cid:66)(cid:77)(cid:77)(cid:27)(cid:1)(cid:18)(cid:14)(cid:25)(cid:17)(cid:17)(cid:14)(cid:19)(cid:24)(cid:19)(cid:14)(cid:24)(cid:24)(cid:20)(cid:24)(cid:1)(cid:114)(cid:1)(cid:1)(cid:39)(cid:66)(cid:89)(cid:27)(cid:1)(cid:18)(cid:14)(cid:25)(cid:17)(cid:17)(cid:14)(cid:20)(cid:24)(cid:21)(cid:14)(cid:20)(cid:21)(cid:17)(cid:18)(cid:1)
E-mail: [email protected]
© 2009 by Taylor & Francis Group, LLC
AU5285.indb 2 2/20/09 7:46:54 PM
Information
Security
Management
Metrics
A Definitive Guide to Effective
Security Monitoring and Measurement
W. Krag Brotby, CISM

© 2009 by Taylor & Francis Group, LLC
AU5285.indb 3 2/20/09 7:46:54 PM
Auerbach Publications
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2009 by Taylor & Francis Group, LLC
Auerbach is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed in the United States of America on acid-free paper
10 9 8 7 6 5 4 3 2 1
International Standard Book Number-13: 978-1-4200-5285-5 (Hardcover)
This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher can-
not assume responsibility for the validity of all materials or the consequences of their use. The
authors and publishers have attempted to trace the copyright holders of all material reproduced
in this publication and apologize to copyright holders if permission to publish in this form has not
been obtained. If any copyright material has not been acknowledged please write and let us know so
we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copy-
right.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222
Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that pro-
vides licenses and registration for a variety of users. For organizations that have been granted a
photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and
are used only for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Brotby, W. Krag.
Information security management metrics : a definitive guide to effective
security monitoring and measurement / W. Krag Brotby.
p. cm.
Includes bibliographical references and index.
ISBN 978-1-4200-5285-5 (hardcover : alk. paper)
1. Information technology--Security measures. 2. Computer security.
3. Business enterprises--Computer networks--Security measures. 4. Data
protection. I. Title.
HD30.2.B78 2009
658.4’78--dc22 2009000669
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the Auerbach Web site at
http://www.auerbach-publications.com
© 2009 by Taylor & Francis Group, LLC
AU5285.indb 4 2/20/09 7:46:55 PM
Contents
Acknowledgments ..........................................................................................xi
Introduction ................................................................................................xiii
Chapter 1
Security Metrics Overview ..............................................................................1
1.1 Metrics and Objectives ......................................................................4
1.2 Information Security .........................................................................7
1.3 IT Security ........................................................................................8
1.3.1 Why the IT Metric Focus .....................................................8
1.4 Other Assurance Functions ...............................................................8
1.5 Stakeholders ....................................................................................10
Endnotes ....................................................................................................10
Chapter 2
Security Metrics ............................................................................................13
2.1 Security Program Effectiveness ........................................................14
2.2 Types of Metrics ..............................................................................15
2.3 Information Assurance / Security Metrics Classification .................17
2.4 Monitoring vs. Metrics ....................................................................18
Endnotes ....................................................................................................18
Chapter 3
Current State of Security Metrics ..................................................................21
3.1 Quantitative Measures and Metrics .................................................21
3.1.1 Performance Metrics ...........................................................22
3.1.2 Discussion ..........................................................................25
3.2 Financial Metrics .............................................................................25
3.2.1 Return on Investment (ROI) ..............................................26
3.2.1.1 Payback Method .................................................26
3.2.1.2 ROI Calculation .................................................27
v
© 2009 by Taylor & Francis Group, LLC
AU5285.indb 5 2/20/09 7:46:55 PM
vi ◾ Contents
3.2.1.3 NPV ...................................................................29
3.2.1.4 IRR ....................................................................29
3.2.2 Return on Security Investment (ROSI) ..............................30
3.2.2.1 SLE and ALE .....................................................30
3.2.2.2 ROSI ..................................................................31
3.2.2.3 A New ROSI Model ...........................................31
3.2.2.4 A More Complex Security ROI ..........................32
3.2.3 Security Attribute Evaluation Method
(SAEM) ..............................................................................35
3.2.4 Cost-Effectiveness Analysis .................................................35
3.2.4.1 Cost-Benefit Analysis ..........................................36
3.2.5 Fault Tree Analysis .............................................................36
3.2.6 Value at Risk (VAR) ...........................................................37
3.2.7 ALE/SLE ............................................................................37
3.3 Qualitative Security Metrics ............................................................38
3.3.1 Cultural Metrics .................................................................39
3.3.2 Risk Management through Cultural
Theory ................................................................................39
3.3.3 The Competing Values Framework ....................................40
3.3.4 Organizational Structure ...................................................42
3.4 Hybrid Approaches ..........................................................................43
3.4.1 Systemic Security Management ..........................................43
3.4.2 Balanced Scorecard ............................................................44
3.4.3 The SABSA Business Attributes
Approach ...........................................................................46
3.5 Quality Metrics ...............................................................................48
3.5.1 Six Sigma ............................................................................48
3.5.2 ISO 9000 ...........................................................................49
3.5.3 Maturity Level ....................................................................49
3.5.4 Benchmarking ....................................................................50
3.5.5 Standards ............................................................................50
3.5.6 OCTAVE ...........................................................................51
Endnotes ....................................................................................................51
Chapter 4
Metrics Developments ...................................................................................53
4.1 Statistical Modeling .........................................................................54
4.2 Systemic Security Management .......................................................55
4.3 Value at Risk Analysis......................................................................56
4.4 Factor Analysis of Information Risk
(FAIR) .............................................................................................57
4.5 Risk Factor Analysis ........................................................................58
4.6 Probabilistic Risk Assessment (PRA) ...............................................58
Endnotes ....................................................................................................61
© 2009 by Taylor & Francis Group, LLC
AU5285.indb 6 2/20/09 7:46:55 PM
Contents ◾ vii
Chapter 5
Relevance .......................................................................................................63
5.1 Problem Inertia ...............................................................................64
5.2 Correlating Metrics to Consequences .............................................64
Chapter 6
The Metrics Imperative .................................................................................67
6.1 Study of ROSI of Security Measures................................................68
6.2 Resource Allocation .........................................................................69
6.3 Managing without Metrics ..............................................................70
Endnotes ....................................................................................................71
Chapter 7
Attributes of Good Metrics ...........................................................................73
7.1 Metrics Objectives ...........................................................................75
7.2 Measurement Categories ..................................................................75
7.3 Effective Metrics .............................................................................77
7.3.1 What Is Being Measured? ...................................................79
7.3.2 Why Is It Measured? ...........................................................80
7.3.3 Who Are the Recipients? ....................................................81
7.3.4 What Does It Mean? ..........................................................81
7.3.5 What Action Is Required? ..................................................81
Chapter 8
Information Security Governance .................................................................83
8.1 Security Governance Outcomes .....................................................84
8.2 Defining Security Objectives ...........................................................85
8.2.1 Sherwood Applied Business Security Architecture
(SABSA) .............................................................................86
8.2.2 CobiT .................................................................................86
8.2.3 ISO 27001 ..........................................................................89
8.2.4 Capability Maturity Model ...............................................90
8.3 Current State ...................................................................................91
8.4 Information Security Strategy .........................................................91
Endnotes ....................................................................................................92
Chapter 9
Metrics Development—A Different Approach ..............................................93
9.1 The Information Security Manager .................................................94
9.2 Activities Requiring Metrics ............................................................96
9.2.1 Criticality and Sensitivity ...................................................97
9.2.2 Degree of Risk or Potential Impact .....................................97
9.2.3 Risk over Time ...................................................................97
9.2.4 Options and Cost-Effectiveness ..........................................97
© 2009 by Taylor & Francis Group, LLC
AU5285.indb 7 2/20/09 7:46:55 PM
viii ◾ Contents
9.3 Ranking Metrics and Monitoring Requirements .............................98
9.3.1 Monitoring, Measures, or Metrics? .....................................98
Chapter 10
Information Security Governance Metrics ..................................................101
10.1 Strategic Security Governance Decisions .......................................101
10.1.1 Strategic Security Governance Decision Metrics...............102
10.2 Security Governance Management Decisions ................................103
10.2.1 Strategic Direction ............................................................103
10.2.2 Ensuring Objectives Are Achieved ....................................104
10.2.3 Managing Risks Appropriately .........................................104
10.2.4 Using Resources Responsibly ............................................105
10.3 Security Governance Operational Decisions ..................................105
Chapter 11
Information Security Risk Management .....................................................107
11.1 Information Security Risk Management Decisions........................108
11.2 Management Requirements for Information Security Risk ............109
11.2.1 Criticality of Assets ...........................................................109
11.2.2 Sensitivity of Assets ..........................................................110
11.2.3 The Nature and Magnitude of Impacts .............................110
11.2.4 Vulnerabilities...................................................................110
11.2.5 Threats ..............................................................................111
11.2.6 Probability of Compromise ...............................................111
11.2.7 Strategic Initiatives and Plans ...........................................111
11.2.8 Acceptable Levels of Risk and Impact ...............................112
11.3 Information Security Operational Risk Metrics ............................112
Chapter 12
Information Security Program Development Metrics ................................115
12.1 Program Development Management Metrics ................................116
12.2 Program Development Operational Metrics ..................................117
Chapter 13
Information Security Management Metrics ................................................119
13.1 Security Management Decision Support Metrics ...........................120
13.2 Security Management Decisions ....................................................122
13.2.1 Strategic Alignment ..........................................................123
13.2.2 Risk Management ............................................................125
13.2.2.1 Metrics for Risk Management ..........................126
13.2.3 Assurance Process Integration ...........................................132
13.2.4 Value Delivery ..................................................................134
13.2.5 Resource Management .....................................................136
13.2.6 Performance Measurement ...............................................136
© 2009 by Taylor & Francis Group, LLC
AU5285.indb 8 2/20/09 7:46:56 PM
Contents ◾ ix
13.3 Information Security Management Operational Decision
Support Metrics ...................................................................................137
13.3.1 IT and Information Security Management .......................137
13.3.2 Compliance Metrics .........................................................138
Endnotes ..................................................................................................147
Chapter 14
Incident Management and Response ...........................................................149
14.1 Incident Management Decision Support Metrics ..........................150
14.1.1 Is It Actually an Incident? .................................................150
14.1.2 What Kind of Incident Is It? .............................................151
14.1.3 Is It a Security Incident? ...................................................151
14.1.4 What Is the Severity Level? ...............................................151
14.1.5 Are There Multiple Events and/or Impacts? ......................152
14.1.6 Will an Incident Need Triage? ..........................................152
14.1.7 What Is the Most Effective Response? ..............................152
14.1.8 What Immediate Actions Must be Taken? ........................153
14.1.9 Which Incident Response Teams and Other Personnel
Must be Mobilized? ...............................................................153
14.1.10 Who Must be Notified? ..................................................153
14.1.11 Who Is in Charge? ..........................................................153
14.1.12 Is It Becoming a Disaster? ...............................................153
Chapter 15
Conclusions .................................................................................................155
15.1 Predictive Metrics ..........................................................................155
Acronyms ............................................................................... 157
Appendix A
Metrics Classifications ................................................................................165
A.1 IA Program Developmental Metrics ..............................................165
A.1.1 Policy Management Metrics .............................................165
A.1.2 Process Maturity Metrics ..................................................165
A.2 Support Metrics .............................................................................166
A.2.1 Personnel Support Metrics ................................................166
A.2.2 Resource Support Metrics .................................................166
A.3 Operational Metrics ......................................................................166
A.3.1 Operational Readiness Metrics .........................................166
A.3.1.1 Management Readiness Metrics .......................167
A.3.1.2 Technical Readiness Metrics .............................167
A.3.2 Operational Practice Metrics ............................................167
A.3.3 Operational Environment Metrics ....................................167
A.4 Effectiveness Metrics .....................................................................168
© 2009 by Taylor & Francis Group, LLC
AU5285.indb 9 2/20/09 7:46:56 PM
x ◾ Contents
A.4.1 Metrics for Technical Target of Assessment (TTOA) .......168
A.4.1.1 Metrics for Strength Assessment ........................168
A.4.1.2 Metrics for Weakness Assessment ......................169
Acknowledgments ....................................................................................170
Endnotes ..................................................................................................170
References ................................................................................................170
Appendix B
Cultural Worldviews ...................................................................................171
Endnotes ..................................................................................................173
Appendix C
The Competing Values Framework .............................................................175
C.1 Cultural Dimensions .....................................................................175
C.1.1 Horizontal: In/Out ...........................................................175
C.1.2 Vertical: Stability/Flexibility .............................................175
C.2 The Competing Values Map ..........................................................175
C.2.1 Hierarchy..........................................................................176
C.2.2 Market ..............................................................................176
C.2.3 Clan..................................................................................176
C.2.4 Adhocracy ........................................................................177
Appendix D
The Organization Culture Assessment Instrument (OCAI) .......................179
Appendix E
SABSA Business Attribute Metrics ..............................................................181
Endnotes .................................................................................................200
Appendix F
Capability Maturity Model ...........................................................................201
Level 1—Initial ........................................................................................201
Level 2—Repeatable .................................................................................201
Level 3—Defined .....................................................................................202
Level 4—Managed ...................................................................................202
Level 5—Optimizing ...............................................................................202
Appendix G
Probabilistic Risk Assessment .....................................................................205

DR. MIChAel StAMAtelAtOS
G.1 What Is Probabilistic Risk Assessment? .........................................205
G.2 What Are the Benefits of PRA? .....................................................207
Index ...........................................................................................................211
© 2009 by Taylor & Francis Group, LLC
AU5285.indb 10 2/20/09 7:46:56 PM