The word "culture" does not have a precise definition, but its meaning in context is clear. There is no substitute for executive awarness of the importance of quality. Industry leaders care about quality and know how it might be achieved or assessed. This is not less true when applied to software, although many companies and organizations seem to be confused about the activities related to software quality. In the software arena several activities related to software quality can be identified:
● Software Quality Achievement: Represents the process of obtaining methods about how to
engineer quality into the software design, for example, during its development stages.
● Software Quality Control: Represents the process of checking at the end of some development
stages (design stage, implementation stage) or at the end of some development process that has quality built in, that the quality required has been achieved using the specification, design, and implementation methods chosen during the software product life cycle.
● Software Quality Assurance: Represents that aspect of the overall management function that
determines and implements the software quality policy that contains the overall quality intentions and objectives of an organization as formally expressed by senior management.
● Software Quality Evaluation: The process that involves measurements, tests static/dynamic
analysis, and V&V techniques to determine the quality of the software (process/product).
● Software Quality Assessment: The process of comparing the actual tests and measurements of
the characteristics of interest with the specifications of those characteristics.
● Software Quality Certification: The process by which a third independent party gives written
assurance that a product, process, or service conforms to specified characteristics. In order to certificate, it is necessary to evaluate against a standard or some sort of requirement.
Within these activities, verification and validation plays an important role. V&V is one of the primary activities that requires addressing by software developers, independent V&V teams, and certification specialists. In this section we outline V&V as performed by indepenedent evaluation teams, and V&V in the view of certification. We conclude the section with some aspects related to the cost of V&V.
4.2. INDEPENDENT VERIFICATION AND VALIDATION
Independent verification and validation (IV&V) is a process whereby the products of the software
development life-cycle phases are independently reviewed, verified, and validated by an organization that is neither the provider nor the acquirer of the software. The IV&V activities duplicate the V&V activities step-by-step during the life cycle, with the exception that the testing performed is not "informal," but rather formal.
It is important to probe more closely into the relationship between verification, validation and testing, and IV&V. Testing and debugging refer to the analysis, exercise, and repair of software by the developers. IV&V is a process which is independent of normal development and testing activities. It is called IV&V to indicate that it is accomplished by parties who are not part of the immediate development team. The purpose of IV&V is not to directly assist in the development of reliable software, but to provide
independent evidence that this is the case and that the system performs according to its requirements. Traditionally, V&V agents only discover problems, but they do not fix them. Correcting the problem is the task of the development team. However, the IV&V team is more likely to have a deeper understanding of testing methods, test-case construction, and a much better understanding of how to accomplish the
your e-mail
subscribe
Go!
ITLibrary
EarthWeb sites Crossnodes Datamation Developer.com DICE EarthWeb.com EarthWeb Direct ERP Hub Gamelan GoCertify.com HTMLGoodies Intranet Journal IT Knowledge IT Library JavaGoodies JARS JavaScripts.com open source IT RoadCoders Y2K Info
complex task of repairing the detected problems completely, generally, and with fewer side effects. There can easily be confusion regarding the relative roles of software quality assurance (SQA) and IV&V. Development management uses SQA to monitors its own organization and to ensure that established standards and procedures are followed. IV&V does essentially the same thing for the customer. The two types of activities need not overlap. In fact, an effective SQA program makes the IV&V process easier. IV&V cannot and should not replace SQA.
Expert systems are being increasingly applied to nontrivial problem domains, where erroneous advice may lead to loss of life, loss of considerable amounts of money, or loss of properties. There is an increased request for IV&V, especially from customers who want to make sure that the expert system they buy has the quality they need.
4.3. V&V IN SUPPORT OF CERTIFICATION FOR CRITICAL SYSTEMS
Expert systems may be safety-critical where they must perform without failure. An inappropriately specified system may cause death, injury, or huge property losses. For example, many medical systems are safety-critical, meaning that errors in operation or use can lead to death or injury (and legal liability). Even where human safety is less of a consideration, a high degree of reliability is a requirement for an expert system. In the SE tradition, a safety-critical (computer) system is one whose malfunction could lead to "unacceptable consequences." This term has different meanings depending on the domain and tasks for which the systems are built: shipping, aerospace, process control, nuclear power plant, etc.
When safety issues are a primary consideration, a software-based system may be required to undergo certification corresponding to its application and criticality. Critical applications of software require rigorous attention to methods for reducing the probability of failure and ensuring the safety of those potentially affected by the failure of such software. This is not less true when applied to expert systems.
However, existing certification guidelines were not designed with AI or expert system software in mind. Although the major processes applied to conventional software are also applied to AI-based software, the unique characteristics associated with the development and testing of AI-based software are not covered in the standards existing in critical domains. Since AI is a relatively new technology, a lack of
standardization exists. Standards relating to AI software are only beginning to emerge. Adherence to standards can have many benefits for developers, independent V&V teams, and certification specialists. Key to the certification process for critical systems will be the faithful application of a thorough verification and validation process. Consequently, a new challenge in the field of V&V of expert systems is now to develop methods and techniques in support of certification.
Previous Table of Contents Next
footer nav
Use of this site is subject certain Terms & Conditions.
Copyright (c) 1996-1999 EarthWeb, Inc.. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of EarthWeb is prohibited. Please read our privacy policy for details.
Brought to you by EarthWeb
IT Library Logo Click
Here!
Click Here!
Search the site:
EXPERT SEARCH ---Programming Languages Databases Security Web Services Network Services Middleware Components Operating Systems User Interfaces Groupware & Collaboration Content Management Productivity Applications Hardware Fun & Games
EarthWeb Direct EarthWeb Direct Fatbrain Auctions Support Source Answers
Previous Table of Contents Next
4.4. HOW MUCH V&V?
A general comment on V&V is that it is poorly understood and generally disliked. Adequate V&V is also expensive. This is not less true when applied to expert systems. Traditionally, it is one of the first activities to be cut when the project costs get tight. The need for V&V, especially for the benefits of reduced expert system maintenance, is seldom understood by management. Nevertheless, there is considerable agreement that V&V will always show a positive cost-avoidance benefit over the life of a system. Careful V&V will do so in some cases even when assessed just for the development stage.
How much V&V? is then a natural question within industrial applications. Nonfunctional features are relatively easy to define, and there are methods that can be applied straightforwardly to check them. However, conformity to the functional part of the specification is another matter. It is generally agreed that we cannot prove (guarantee) by testing that a program conforms to its specification. Therefore, it is important to define what level of checking will be sufficient for the specified purposes. However, there is no simple answer to the question "How much V&V should we do to ensure the quality of the
system/subsystem/module/procedure?," as several factors should be taken into consideration. Of particular concern from the point of view of V&V is the system complexity. The complexity of the system is defined by several characteristics which make the system harder to develop and analyze. Generally, the higher the complexity, the greater the opportunity for errors and the greater the need for V&V. However, there is another factor that needs to be considered in determining the extent of estimated V&V required: system integrity. System integrity refers to the joint capability of a system to operate for long periods without failures, to fail gracefully with reasonable warnings, to be able to recover rapidly without much difficulty, and to avoid causing expensive damage to property or harm to people or the environment. How much integrity is required of a system will be a function of several factors. Thus, a highly complex system with a low degree of required integrity should probably not need as much V&V as a highly complex system with a very high degree of required system integrity.
If we translate the system complexity and integrity requirements into quality requirements, then the above- mentioned question should be answered in connection with another one, i.e.: "How critical is the quality of the system/subsystem/ module/ procedure?"