Information Technology: An Audit Guide for Assessing Acquisition Risk GAO/IMTEC-8.1.4

101 Pages·1995·0.36 MB·English

United States General Accounting Office
GAO Information Management and
Technology Division
December 1992 Information
Technology: An Audit
Guide for Assessing
Acquisition Risks
GAO/IMTEC-8.1.4

Preface
The federal government depends heavily on a variety
of information technology products and services to
serve the public. Each year the government spends
billions of dollars on computer equipment, services,
software, and telecommunications. The success or
failure of information system acquisitions affects
executive agencies’ credibility with the Congress and
the public as well as their abilities to carry out their
missions effectively and efficiently.
The General Accounting Office (GAO) and offices of
inspectors general have consistently identified
problems with information technology acquisitions.
Problems identified in numerous evaluations include
information systems that do not meet users’ needs,
exceed cost estimates, or take significantly longer
than expected to complete.
This guide provides a logical framework for
evaluating information technology acquisitions. It
incorporates a risk assessment methodology intended
to reduce audit planning time and ensure that
significant issues are included. It is based on a model
of the acquisition process developed by GAO in
cooperation with a wide range of federal and private
sector officials. 1 The model outlines the process used
to acquire information technologies and identifies
elements of the process that are essential for planning
and carrying out acquisitions.
This guide is intended for use in planning and
conducting risk assessments of computer hardware
and software, telecommunications, and system
development acquisitions. A risk assessment is the
process of identifying potential risks in a system
under development and then determining the
significance of each risk in terms of its likelihood and
impact on the acquisition’s cost, schedule, and ability
1Information Technology: A Model to Help Managers Decrease
Acquisition Risks (GAO/IMTEC-8.1.6, August 1990).
Page 1 GAO/IMTEC-8.1.4 Assessing Acquisition Risks
Preface
to meet the agency’s needs. Such assessments may
have their greatest impact if carried out early, when
an agency can more easily alter its acquisition plans
and strategy to manage and control the identified
risks.
The audit guide consists of 10 chapters, with
appendixes. Chapter 1 introduces the purpose of the
guide, explains its essential concepts and techniques,
and provides direction in tailoring the guide for use
on specific assignments. Chapters 2 through 10
address specific activities in the acquisition process.
The nine chapters present audit guidance on:
• management and user support,
• project staffing,
• needs/requirements/ specifications,
• alternatives,
• acquisition planning,
• solicitation document,
• source selection,
• contract management, and
• test and acceptance.
Each chapter lists audit objectives, commonly
expected documentation, detailed audit questions,
and references to federal regulations and guidance.
The chapters are intended to assist in the
identification of specific risk areas and to contribute
to an overall assessment of how well an agency is
meeting its acquisition objectives.
This audit guide is also available in a software format,
accompanied by reference materials such as the GAO
model of the acquisition process, relevant federal
acquisition regulations, General Services
Administration (GSA) guidance, and Office of
Management and Budget (OMB) circulars. The
software version utilizes a hypertext software
package to help auditors quickly and flexibly review
Page 2 GAO/IMTEC-8.1.4 Assessing Acquisition Risks
Preface
all the included documents. This software may be
requested from GAO by returning the card included
with this guide.
This guide supplements and does not replace other
GAO policies or procedures. It was prepared under
the direction of Jack L. Brock, Jr., Director,
Government Information and Financial Management,
who can be reached at [phone]. Other major
contributors to the guide are listed in appendix IV.
Ralph V. Carlone
Assistant Comptroller General
Information Management and
Technology Division
Werner Grosshans
Assistant Comptroller General
for Policy
Page 3 GAO/IMTEC-8.1.4 Assessing Acquisition Risks
Contents
Preface 1
Chapter 1 8
Objectives 8
Introduction
Approach 9
Assignment Planning 9
Organization and Use of the Audit Guide 10
Application of Audit Guide to Prototyping 13
Methodology
Presenting Conclusions and 14
Recommendations
Chapter 2 16
Audit Objectives 17
Management and
Documentation Required 17
User Support
Audit Steps: Top Management Support 17
Audit Steps: User Involvement 19
References 21
Chapter 3 22
Audit Objective 22
Project Staffing
Documentation Required 22
Audit Steps: Project Management 22
Audit Steps: Project Staff 23
References 24
Chapter 4 25
Audit Objectives 26
Needs /
Documentation Required 26
Requirements /
Audit Steps: Needs Determination 26
Specifications Audit Steps: Requirements Analysis 27
Audit Steps: Specifications 30
Audit Steps: Test Plans 32
References 32
Page 4 GAO/IMTEC-8.1.4 Assessing Acquisition Risks
Contents
Chapter 5 34
Audit Objectives 34
Alternatives
Documentation Required 34
Audit Steps 35
References 39
Chapter 6 40
Audit Objective 40
Acquisition
Documentation Required 40
Planning
Audit Steps 41
References 43
Chapter 7 44
Audit Objectives 45
Solicitation
Documentation Required 45
Document
Audit Steps 46
References 49
Chapter 8 50
Audit Objective 50
Source Selection
Documentation Required 51
Audit Steps 51
References 54
Chapter 9 55
Audit Objectives 56
Contract
Documentation Required 56
Management
Audit Steps 56
References 59
Chapter 10 60
Audit Objectives 61
Test and
Documentation Required 61
Acceptance
Audit Steps 61
References 63
Appendixes Appendix I: Acquisition Profile 64
Page 5 GAO/IMTEC-8.1.4 Assessing Acquisition Risks
Contents
Appendix II: Management Metrics 68
Appendix III: Reference Materials 80
Appendix IV: Major Contributors to This 86
Audit Guide
Glossary 87
Table Table I.1: Map to the Audit Guide 11
Figures Figure II.1: Problem Reports in Software 73
Project
Figure II.2: Problems By Priority Levels and 74
Number of Months Unresolved
Figure II.3: Software Requirements Changes 75
Figure II.4: Development Progress 76
Figure II.5: Software Size Estimates 78
Page 6 GAO/IMTEC-8.1.4 Assessing Acquisition Risks
Contents
Abbreviations
ANSI American National Standards Institute
APR agency procurement request
CO Contracting Officer
COCOMO Constructive Cost Model
COTR Contracting Officer Technical
Representative
DPA delegation of procurement authority
FAR Federal Acquisition Regulation
FIPS Federal Information Processing
Standards
FIRMR Federal Information Resource
Management Regulation
GAO General Accounting Office
GSA General Services Administration
IEEE Institute for Electrical and Electronic
Engineers
IFB invitation for bid
IMTEC Information Management and
Technology Division
NIST National Institute of Standards and
Technology
OMB Office of Management and Budget
PM program manager
RFP request for proposals
SSA source selection authority
SSAC Source Selection Advisory Committee
SSEB Source Selection Evaluation Board
Page 7 GAO/IMTEC-8.1.4 Assessing Acquisition Risks
Chapter 1
Introduction
This audit guide is based on and incorporates GAO’s
information technology acquisition model. The model
describes three phases in the acquisition process:
presolicitation, solicitation and award, and
postaward. It sets out essential activities in each
phase along with critical factors related to those
activities. The model is intended to give managers an
overview of the acquisition process and to help them
decrease acquisition risks.
The model’s critical factors were drawn from the
judgment and expertise of a wide range of
knowledgeable individuals from government and
private industry. Compliance with these critical
factors can increase the likelihood that an acquisition
will meet an agency’s needs at a reasonable cost and
in a timely manner.
Objectives This guide is intended to help auditors conduct more
focused reviews of information technology
acquisitions by enabling them to quickly identify
significant areas of risk. Using this guide will help
auditors identify critical factors not addressed by
management, make a general assessment of any
procurement risks, and provide rapid feedback to
agency officials so they can take corrective action in a
timely and efficient manner. Use of the guide should
be selectively tailored to the requirements of
particular reviews and adapted to the status of the
acquisition.
Auditors will need to exercise professional judgment
in assessing the significance of audit results or
findings. For example, the guide assists auditors in
determining how an agency identified and defined its
requirements. Professional judgment is necessary to
evaluate this information and determine if the agency
conducted an adequate requirements analysis.
Page 8 GAO/IMTEC-8.1.4 Assessing Acquisition Risks