3.5.1 ANSI/EIA-748-A-1998
The Electronics Industries Alliance (EIA)—formerly the Electronic Industries Asso- ciation (until 1997) and before that the Radio Manufacturers Association (RMA)—is an alliance of several high-tech associations and companies. The EIA headquarters is in Arlington, Virginia. For more information about the EIA, see the organization’s Web site [57].
The EIA standards of most interest to SQA people are developed by the Govern- ment Electronics & Information Technology Association (GEIA). For more infor- mation about GEIA, see its Web site [58]. Recently, GEIA has collaborated with several other associations in publishing a very influential standard on earned value management systems (EVMS).
ANSI/EIA-748-A-1998 [59] (which was reaffirmed in 2002) presents common terminology and guidelines for establishing and applying an EVMS.
The standard was prepared under the guidance of the Program Management Systems Committee (PMSC) of the National Defense Industrial Association (NDIA). For more information about the NDIA PMSC, see its page on the NDIA Web site [60]. Currently, the U.S. Office of Management and Budget requires that U.S. Federal agencies “…must use a performance-based acquisition management or earned value management system, based on the ANSI/EIA Standard 748, to obtain timely information regarding the progress of capital investments” [61].
Clause 1 of the standard, the Introduction, states seven EVMS principles, and it makes a useful distinction between these principles and the EVMS guidelines that follow in clause 2. The distinction is this: Every program management system should make use of an EVMS application that is “compliant” with the principles. However, the EVMS guidelines in clause 2 are only applicable to “large complex and/or high-risk programs….”
Clause 2 contains the guidelines, 32 of them, for establishing and applying an integrated EVMS. The guidelines are collected into five categories:
• Organization;
• Planning, Scheduling, and Budgeting;
• Accounting Considerations;
• Analysis and Management Reports; • Revisions and Data Maintenance.
The guidelines depend upon common terms that are defined in clause 2.6. The guidelines are described at a high level. The intent of the standard is to state them in a way that does not mandate implementation details. Here is a sample of the guidelines:
• “Define the authorized work elements for the program. A work breakdown structure (WBS), tailored for effective internal management control, is com- monly used in this process.” (Organization)
• “Identify physical products, milestones, technical performance goals, or other indicators that will be used to measure progress.” (Planning, Scheduling, and Budgeting)
• “Record all indirect costs, which will be allocated to the contract.” (Account- ing Considerations)
As the sample shows, the guidelines in clause 2 are written in a way that the manual for writing GEIA standards calls a “direct instruction.” They are not written as requirements, which must be followed to conform to the standard. They do not depend on “shall.”
Clause 3 contains supplementary information that clarifies some of the terms and instructions in the guidelines in clause 2. For example, in clarifying what a pro- gram organization is, clause 3.3 discusses control accounts, control account manag- ers, subcontract management, and intercompany work transfers.
Clause 4 explains that the form of EVMS documentation should be whatever is standard for documenting systems and policies and procedures within the company where the EVMS is used.
Clause 5 discusses how a company might go about assuring that its EVMS achieves “conformity” to the guidelines in clause 2 (and the language in this clause suggests that “conformity” and “compliance” are used interchangeably here). This is the only claim of conformance that the standard offers, because the standard is written without requirements. If a company has an EVMS that has achieved earlier acceptance against C/SCSC (U.S. Department of Defense Cost/Schedule Control Systems Criteria) for a government contract, clause 5 suggests that the company might benefit more from citing the C/SCSC acceptance than from documenting con- formity to the guidelines in this standard. However, in other cases, the basic process for assuring conformity to the EVMS guidelines in clause 2 is to document that the company’s program management system “meets the full intentions of the guide- lines” [59]. The clause makes it clear that the company is responsible for the evaluation of its system, and for preparing the documentation.
Demonstrating conformity to the guidelines in clause 2 depends upon an under- standing of their “full intentions.” However, there is no explicit explanation of those intentions within the standard itself. The content of clause 3 does provide some clarification, but it does not explain the intentions to a degree that is adequate to demonstrate conformity.
The best explanation of conformity to ANSI/EIA-748-A-1998 appears in a pub- lication by the NDIA PMSC—the same group that guided the development of the standard. The publication is the Earned Value Management Systems Intent Guide [62]. For each guideline in clause 2 of the standard, the publication provides:
• An explanation of its intent;
• A list of typical attributes that business processes and system documentation would have if they complied with the guideline;
• A list of typical outputs that provide objective evidence that the business pro- cesses and system documentation do comply with the guideline.
To document that business processes and system documentation comply with (are in conformity to) the guidelines in the standard, the NDIA PMSC publication recommends that the processes and documentation be mapped to the intent, the typical attributes, and the typical outputs, and that this mapping be verified by an “independent” party. The publication provides mapping templates for all the guide- lines in the standard, and also it provides an example of how to use them.
3.5.2 RTCA/DO-178B
RTCA/DO-178B [63] provides guidelines for producing software that will be used in airborne systems. The intent is that software developed according to the guide- lines in the standard will not compromise the safety of a system in which it is embed- ded or the system’s compliance with airworthiness requirements.
The standard was developed by RTCA, Inc. (formerly the Radio Technical Com- mission for Aeronautics), which has headquarters in Washington, D.C. RTCA is a not-for-profit corporation whose mission is to develop “consensus-based recommen- dations regarding communications, navigation, surveillance, and air-traffic manage- ment (CNS/ATM) system issues.” Its members are government, industry, and academic organizations. For more information about the RTCA, see its Web site [64]. Sections 1, 2, and 10 of this standard describe the context in which the standard will be used. The airworthiness of aircraft systems and their engines must be certi- fied, and software that is part of an aircraft or an engine is considered during the certification process. Sections 1 and 2 explain key relations between the software and an aircraft or engine system that contains it. Section 2 also defines six levels of failure that software might cause or allow.
Section 3 discusses the software life cycle. Also, it introduces the concept of transition criteria between processes.
Sections 4 through 9 of the standard describe a software planning process, and four software development processes (software requirements, software design, soft- ware coding, and integration). They also define four integral processes (software verification, software configuration management, software quality assurance, and certification liaison) that provide assistance to the software development processes and to each other. For each process that it describes, the standard states objectives, and it provides guidance on how to achieve the objectives. Most of the objectives in these sections are associated with the software verification process, in section 6, which includes software testing.
Section 11 provides a topical outline for each of the major software life-cycle data items that is mentioned in one or more of the nine processes in sections 4 through 9.
Section 12 of the standard contains discussions of considerations that, for vari- ous reasons, do not fit neatly into other sections of the standard. These consider- ations include, for example, use of previously developed software (and the quality assurance considerations associated with that), criteria for qualifying tools, and alternative methods for achieving the objectives.
In Annex A, which is normative, the standard presents an important collection of tables. For each process in sections 4 through 9, other than verification, there is one table in the collection. For the verification process, there are five tables because the verification process verifies each of the four development processes and it per- forms testing. Each table maps objectives of the process to the levels of failure to indicate which objectives should be satisfied for which levels, and to indicate whether or not the objective should be satisfied “with independence.” The table also indicates for each level and each objective how rigorous the process should be for controlling changes to related outputs.
Conformance or compliance is not defined explicitly by RTCA/DO-178B. How- ever, language in the standard indicates that a software practice (e.g., use of robust- ness test cases), or software method (e.g., an alternative method), or a life-cycle process, can be said to comply with the standard if it satisfies the related objectives in the standard.