This paragraph is an excerpt from Test automation.[edit]

Test automation is the use of software (separate from the software being tested) for controlling the execution of tests and comparing actual outcome with predicted.[18] Test automation supports testing the system under test (SUT) without manual interaction which can lead to faster test execution and testing more often. Test automation is a key aspect of continuous testing and often for continuous integration and continuous delivery (CI/CD).[19]

Levels

Software testing can be categorized into levels based on how much of the software system is the focus of a test.[20][21][22][23]

Unit testing

This paragraph is an excerpt from Unit testing.[edit]

Unit testing, a.k.a. component or module testing, is a form of software testing by which isolated source code is tested to validate expected behavior.[24]

Integration testing

This paragraph is an excerpt from Integration testing.[edit]

Integration testing is a form of software testing in which multiple software components, modules, or services are tested together to verify they work as expected when combined. The focus is on testing the interactions and data exchange between integrated parts, rather than testing components in isolation.

System testing

This paragraph is an excerpt from System testing.[edit]

System testing, a.k.a. end-to-end (E2E) testing, is testing conducted on a complete software system.

Static, dynamic, and passive testing

There are many approaches to software testing. Reviews, walkthroughs, or inspections are referred to as static testing, whereas executing programmed code with a given set of test cases is referred to as dynamic testing.[25][26]

Static testing is often implicit, like proofreading, plus when programming tools/text editors check source code structure or compilers (precompilers) check syntax and data flow as static program analysis. Dynamic testing takes place when the program itself is run. Dynamic testing may begin before the program is 100% complete in order to test particular sections of code and are applied to discrete functions or modules.[25][26] Typical techniques for these are either using stubs/drivers or execution from a debugger environment.[26]

Static testing involves verification, whereas dynamic testing also involves validation.[26]

Passive testing means verifying the system's behavior without any interaction with the software product. Contrary to active testing, testers do not provide any test data but look at system logs and traces. They mine for patterns and specific behavior in order to make some kind of decisions.[27] This is related to offline runtime verification and log analysis.

Exploratory

This section is an excerpt from Exploratory testing.[edit]

Exploratory testing is an approach to software testing that is concisely described as simultaneous learning, test design and test execution. Cem Kaner, who coined the term in 1984,[28] defines exploratory testing as "a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project."[29]

Preset testing vs adaptive testing

The type of testing strategy to be performed depends on whether the tests to be applied to the IUT should be decided before the testing plan starts to be executed (preset testing[30]) or whether each input to be applied to the IUT can be dynamically dependent on the outputs obtained during the application of the previous tests (adaptive testing[31][32]).

Black/white box

Software testing can often be divided into white-box and black-box. These two approaches are used to describe the point of view that the tester takes when designing test cases. A hybrid approach called grey-box that includes aspects of both boxes may also be applied to software testing methodology.[33][34]

White-box testing

Main article: White-box testing

White box testing diagram

White-box testing (also known as clear box testing, glass box testing, transparent box testing, and structural testing) verifies the internal structures or workings of a program, as opposed to the functionality exposed to the end-user. In white-box testing, an internal perspective of the system (the source code), as well as programming skills are used to design test cases. The tester chooses inputs to exercise paths through the code and determines the appropriate outputs.[33][34] This is analogous to testing nodes in a circuit, e.g., in-circuit testing (ICT).

While white-box testing can be applied at the unit, integration, and system levels of the software testing process, it is usually done at the unit level.[35] It can test paths within a unit, paths between units during integration, and between subsystems during a system–level test. Though this method of test design can uncover many errors or problems, it might not detect unimplemented parts of the specification or missing requirements.

Techniques used in white-box testing include:[34][36]

API testing – testing of the application using public and private APIs (application programming interfaces)
Code coverage – creating tests to satisfy some criteria of code coverage (for example, the test designer can create tests to cause all statements in the program to be executed at least once)
Fault injection methods – intentionally introducing faults to gauge the efficacy of testing strategies
Mutation testing methods
Static testing methods

Code coverage tools can evaluate the completeness of a test suite that was created with any method, including black-box testing. This allows the software team to examine parts of a system that are rarely tested and ensures that the most important function points have been tested.[37] Code coverage as a software metric can be reported as a percentage for:[33][37][38]

Function coverage, which reports on functions executed
Statement coverage, which reports on the number of lines executed to complete the test
Decision coverage, which reports on whether both the True and the False branch of a given test has been executed

100% statement coverage ensures that all code paths or branches (in terms of control flow) are executed at least once. This is helpful in ensuring correct functionality, but not sufficient since the same code may process different inputs correctly or incorrectly.[39]

Black-box testing

Main article: Black-box testing

Black box diagram

Black-box testing (also known as functional testing) describes designing test cases without knowledge of the implementation, without reading the source code. The testers are only aware of what the software is supposed to do, not how it does it.[40] Black-box testing methods include: equivalence partitioning, boundary value analysis, all-pairs testing, state transition tables, decision table testing, fuzz testing, model-based testing, use case testing, exploratory testing, and specification-based testing.[33][34][38]

Specification-based testing aims to test the functionality of software according to the applicable requirements.[41] This level of testing usually requires thorough test cases to be provided to the tester, who then can simply verify that for a given input, the output value (or behavior), either "is" or "is not" the same as the expected value specified in the test case. Test cases are built around specifications and requirements, i.e., what the application is supposed to do. It uses external descriptions of the software, including specifications, requirements, and designs, to derive test cases. These tests can be functional or non-functional, though usually functional. Specification-based testing may be necessary to assure correct functionality, but it is insufficient to guard against complex or high-risk situations.[42]

Black box testing can be used to any level of testing although usually not at the unit level.[35]

Component interface testing

How To Guide