Software Testing Quality Standards – Software Testing

Software Testing Quality Standards, ISO standards for Software quality, IEEE Test documentation standards, and Software Testing Certification.

Software Testing Standards

ISO – International Organization for Standardization (1947)

IEEE – Institute of Electrical and Electronics Engineers (1963)

IEC – International Electrotechnical Commission (1906)

BS (BSI) – British Standards (1901)

CMMI– Capability Maturity Model Integration

1. IEEE 610.12-1990

IEEE Standard Glossary of Software Engineering Terminology

2. IEEE 829-1998

IEEE Standard for Software Test Documentation

The Types of Document:

There are eight document types in the IEEE 829 standard, which can be used in three distinct phases of software testing:

Preparation of Tests

Test Plan: Plan how the testing will proceed.

Test Design Specification: Decide what needs to be tested.

Test Case Specification: Create the tests to be run.

Test Procedure: Describe how the tests are run.

Test Item Transmittal Report: Specify the items released for testing.

Running the Tests

Test Log: Record the details of tests in time order.

Test Incident Report: Record details of events that need to be investigated.

Completion of Testing

Test Summary Report: Summarise and evaluate tests.

3. IEEE Standard 1012-1998

IEEE Standard for Software Verification and Validation -Description

4. IEEE Standard 1028-1997

IEEE Standard for Software Reviews -Description

5. IEEE 1008

IEEE Standard for Software Unit Testing

6. IEEE 1044-1993

IEEE Standard Classification for Software Anomalies

7. IEEE 1219-1998

The standard for Software Maintenance

8. ISO/IEC 9126-1:2001

(Software engineering – Software product quality- part 1)

Support for review, verification and validation, and a framework for quantitative quality evaluation, in the support process;

Support for setting organisational quality goals in the management process.

9. ISO/IEC 12207:2008

Systems and software engineering — Software life cycle processes

10. ISO/IEC 14598-1:1999

Information technology — Software product evaluation

11. ISO/IEC 2382-1:1993

Data Processing — Vocabulary — Part 1: Fundamental terms

12. ISO 9000:2000

Quality management systems – Fundamentals and vocabulary

13. BS 7925-2:1998

Software testing, Software component testing

(This standard defines the process for software component testing using specified test case design and measurement techniques. This will enable users of the standard to directly improve the quality of their software testing, and improve the quality of their software products)

14. DO-178B:1992

– Software Considerations in Airborne Systems and Equipment

15) BS7925-1

The British software testing standard governing testing terminology

* Software Testing standards from Non-governmental organizations *

Interview Questions on Quality Standards

1. What are the Standards?

Standards are a set of mandatory requirements established by consensus and maintained by a recognized body to prescribe a disciplined uniform approach or specify a product, that is, mandatory conventions and practices. Standards provide a body of knowledge that provides the basis for a professional discipline.

2. What is the basis for standards?

The basis for standards:
1. Communication – common terminology.
2. Professional qualifications.
3. Certification/compliance schemes.
4. Benchmark of good industry practice.
5. Contracts.
6. Interoperability and consistency.

3. How many categories of standards are there in software testing?

Testing Standards are broadly categorized into two types: External standards and internal standards.

4. What ISO?

ISO (International Organization for Standardization) is the world’s largest developer of voluntary International Standards. International Standards give state of the art specifications for products, services and good practice, helping to make industry more efficient and effective. Developed through global consensus, they help to break down barriers to international trade.

ISO develops International Standards. ISO founded in 1947, and since then published more than 19 000 International Standards covering almost all aspects of technology and business.

5. How ISO is related to Software Testing?

The aim of ISO/IEC 29119 Software Testing is to provide one definitive standard for software testing that defines vocabulary, processes, documentation, techniques and a process assessment model for software testing that can be used within any software development life cycle.

6. What is ISO/IEC 29119?

ISO/IEC 29119 comprises 4 parts:

Part 1: Definitions & Vocabulary
Part 2: Test Process
Part 3: Test Documentation
Part 4: Test Techniques

7. What is ISO Test Documentation Standard?

Test Documentation standard covers test documentation across the entire software testing life cycle.

It Covers:

Organizational Test Process Documentation:
– Organizational Test Policy
– Organizational Test Strategy

Test Management Process Documentation:
– Test Plan (includes test strategy)
– Test Status Report
– Test Completion Report

Dynamic Test Process Documentation:
– Test Design Specification
– Test Case Specification
– Test Procedure Specification
– Test Data Requirements
– Detailed Test Environment Requirements
– Test Environment Readiness Report
– Test Outcome
– Test Result
– Test Execution Log
– Test Incident Report

IEEE have given ISO permission to use the well-known IEEE 829 Test Documentation standard as a basis for this part of the standard.

8. What is ISO Test Techniques Standard?

ISO Test Techniques standard covers a variety of common dynamic software testing techniques:

Specification-Based Testing Techniques:
– Equivalence Partitioning
– Classification Tree Method
– Boundary Value Analysis
– State Transition Testing
– Decision Table Testing
– Cause-Effect Graphing
– Syntax Testing
– Combinatorial Test Techniques, including:
-> All Combinations
-> Pairwise Testing
-> Each Choice Testing
-> Base Choice Testing
– Scenario Testing
– Error Guessing
– Random Testing

Structure-Based Testing Techniques:
– Statement Testing
– Branch Testing
– Decision Testing
– Condition Testing, including:
-> Branch Condition Testing
-> Branch Condition Combination Testing
-> Modified Condition Decision Condition (MCDC) Testing
Etc…

9. What are the Concepts and Vocabulary ISO/IEC 29119 Software Testing?

Concepts and Vocabulary ISO/IEC 29119 Software testing:

The aim of part 1 of the standard is to give an overview of the standard and of general software testing concepts and to provide a vocabulary of software testing terms covering the entire software testing life cycle. A working draft of part 1 is currently under development. It is anticipated that this part will include (but not be limited to) the following topics:
• Introduction to software testing
• The role of verification and validation
• Testing as a heuristic
• Exhaustive testing
• Software testing in an organizational and project context
• The testing process
• Generic testing processes in the system life cycle
• Development project sub-processes and their results
• On-going maintenance and its results
• Support processes for the software development life cycle
• Risk-based testing
• Test sub-processes
• Test objectives
• Test item
• Testing of quality characteristics
• Test basis
• Retesting and regression testing
• Test techniques
• Approaches to testing, including risk-based, requirements-based, analytic, model-based, scripted, unscripted and automated
• Metrics and measures
• Testing in different life cycle models, including agile, evolutionary and sequential

10. What is IEEE?

Institute of Electrical and Electronics Engineers describes itself as “the world’s largest technical professional society.It provides standards for Power, Energy, Telecom, Information Technology and Aviation etc… Industries.

11. What is IEEE 829 Standard?

An IEEE standard for Software Test Documentation, Provides Test documentation templates from Test Planning to Test Closure.

Test Plan: A detail of how the test will proceed, who will do the testing, what will be tested, in how much time the test will take place, and to what quality level the test will be performed.

Test Design Specification: A detail of the test conditions and the expected outcome. This document also includes details of how a successful test will be recognized.

Test Case Specification: A detail of the specific data that is necessary to run tests based on the conditions identified in the previous stage.

Test Procedure Specification: A detail of how the tester will physically run the test, the physical set-up required, and the procedure steps that need to be followed.

Test Item Transmittal Report: A detail of when specific tested items have been passed from one stage of testing to another.

Test Log: A detail of what tests cases were run, who ran the tests, in what order they were run, and whether or not individual tests were passed or failed.

Test Incident Report: A detail of the actual versus expected results of a test, when a test has failed, and anything indicating why the test failed.

Test Summary Report: A detail of all the important information to come out of the testing procedure, including an assessment of how well the testing was performed, an assessment of the quality of the system, any incidents that occurred, and a record of what testing was done and how long it took to be used in future test planning. This final document is used to determine if the software being tested is viable enough to proceed to the next stage of development.

12. What is IEEE Test Plan template?

IEEE 829 Test Plan Template
1) Test Plan Identifier
2) Introduction
3) Test Items
4) Features To Be Tested
5) Features Not To Be Tested
6) Approach
7) Item Pass/Fail Criteria
8) Suspension Criteria And Resumption Requirements
9) Test Deliverables
10) Testing Tasks
11) Environmental Needs
12) Responsibilities
13) Staffing And Training Needs
14) Schedule
15) Risks And Contingencies
16) Approvals

13. IEEE Test Case Specification Template

1) Test Case Specification Identifier
2) Test Items
3) Input Specifications
4) Output Specifications
5) Environmental needs
6) Special Procedural Requirements
7) Inter-case Dependencies

14. What is IEEE Test Incident Report template?

Test Incident Report
1) Purpose
2) Outline
3) Cover Page
4) Section Description
5) Test-Incident-Report Identifier
6) Summary
7) Incident Description
8) Impact

15. What is the IEEE Test Design Specification template?

IEEE Test design specification Template

1) Test Design Specification Identifier
2) Features to be Tested
3) Approach Refinements
4) Test Identification
5) Feature Pass/Fail Criteria

16. What is IEEE Test Procedure Template?

Test Procedure Specification Template

1) Test Procedure Specification Identifier
2) Purpose
3) Special Requirements
4) Steps

17. What is the IEEE Item Transmittal Report template?

Test Item Transmittal Report
1) Purpose
2) Outline
3) Cover Page
4) Section Description
5) Transmittal-Report Identifier
6) Transmitted Items
7) Location
8) Status
9) Approvals

18. What is IEEE Test Log Report Template?

Test Log Template

1) Test Log Identifier
2) Description
3) Activity and Event Entries

19. What is the IEEE Test Summary Report template?

IEEE Test Summary Report
1) IEEE Test Summary Report Identifier
2) Summary
o Test Items
o Environment
o References
3) Variances
4) Comprehensiveness Assessment
5) Summary of Results
6) Evaluation
7) Summary of Activities
8) Approvals

Follow me on social media:

Follow me on social media: