What is Black Box Testing?

Black box testing is done to know the external functionality of what the product should do.
Doesn't look at the program code but looks at the product from an external perspective.
Done without the knowledge of internals of system under test.
Done from the customer's viewpoint.
The test engineer engaged in BBT only knows set of inputs and expected outputs, and is unaware of how these inputs are processed internally by the product.
Convenient to administer because they use the complete finished product and do not require any knowledge of its construction.

Why Black Box Testing?

BBT helps in overall functionality verification of the system under test.

Done based on requirements. Helps in identifying any incomplete, inconsistent requirements or issues involved when the system is tested as a complete entity.
Addresses the stated requirements as well as implied requirements. All requirements are not stated explicitly, but are deemed implicit.
Encompasses the end user perspectives.
Handles valid and invalid inputs.

When To Do Black Box Testing?

BBT requires involvement of the testing team right from the beginning of the software project life cycle.
Testers get involved right from requirements gathering phase.
Test scenarios and test data are prepared during the test construction phase of the test life cycle, when the software is in design phase.
Once the code is ready and delivered for testing, test execution is done.
All the test scenarios developed during the construction phase are executed.
A subset of these test scenarios are selected for regression testing.

How To Do Black Box Testing?

BBT exploits specifications to generate test cases in a methodical way to avoid redundancy and to provide better coverage. The techniques include :

Requirements Based Testing
Positive Testing
Negative Testing
Boundary Value Analysis
Decision Tables
Equivalence Partitioning
State Based or Graph Based Testing
Compatibility Testing
User Documentation Testing
Domain Testing

Requirements Based Testing

Requirements testing deals with validating the requirements given in the Software Requirements Specification(SRS) of the software product.
Explicit requirements are stated and documented clearly in the SRS. Implied or implicit requirements are those that are not documented but assumed to be incorporated in the system.
Precondition is a detailed review of the requirements specifications to ensure they are consistent, correct, complete and testable.
The review ensures that some implied requirements are converted and documented as explicit requirements, making the testing more effective.
All explicit and implied requirements are collected and documented as 'Test Requirements Specification(TRS)'.
The TRS documented is used for Requirements based testing.
Requirements are tracked by a 'Requirements Traceability Matrix(RTM)'. An RTM traces all the requirements from their beginning through design, development and testing.
The matrix evolves through the life cycle of the project. Each requirement is given a unique id along with a brief description, which are taken from the Requirements specification.
The requirements are named with a naming convention, which depends on each organization.
Each requirement is assigned a requirement priority, classified as high, medium or low. Tests for high priority requirements will get precedence over low priority ones, which ensures that the functionality with high risk or high stakes is tested earlier, and the defects fixed at the earliest.
The RTM contains columns like 'test conditions', 'test case IDs' and 'phase of testing'.
A requirement is subjected to multiple phases of testing - unit, component, integration, system testing etc.

The RTM helps in identifying the relationship between the requirements and test cases. The following combinations are possible:

One to one
One to many
Many to one
Many to many
One to none

The Requirements Traceability Matrix or RTM provides a wealth of information on various test metrics:

Requirements addressed priority wise helps addressing the high priority requirements first.
Number of test cases requirement wise
Total number of test cases prepared.

Once the test cases are executed, the test results can be used to collect metrics such as:

Total number of test cases passed.
Total number of test cases failed.
Total number of defects in requirements.
Number of requirements completed.
Number of requirements pending.

These metrics can also be displayed graphically to better visualize what we are dealing with.

Positive Testing

Positive testing tries to prove that a given product does what it is supposed to do.
When a test case verifies the requirements of the product with a set of expected output, it is called positive test case.
The purpose of positive testing is to prove that the product works as per specification and expectation.
A product delivering an error when it is expected to give an error, is also a part of positive testing.
Positive testing is done to verify the known test conditions.

Negative Testing

Negative testing is done to show that the product does not fail when an unexpected input is given.
The purpose of negative testing is to try and break the system with unknowns.
Covers scenarios for which the product is not designed and coded. The input values may not have been represented in the specification of the product. These are termed as unknown conditions.
It is important to know for testers to know the negative situations that may occur at end-user level so that the application can be tested and made foolproof.
A negative test would be a product not delivering an error when it should or delivering an error when it should not.

The difference between Positive and negative testing is in their coverage. For positive testing, if all documented requirements and test conditions are covered, then coverage can be said to be 100 percent. In contrast, there is no end to negative testing, and 100 percent coverage is impractical. Negative testing requires a high degree of creativity among the testers to cover as many unknowns as possible to avoid failure.

Boundary Value Analysis

Most of the defects in software product hover around conditions and boundaries. Boundaries means the 'limits' of values of the various variables.
BVA is useful for catching defects that happen at boundaries. It believes that the density of defects is more towards boundaries.
Boundary Value Analysis is useful to generate test cases when the input(or output) data is made up of clearly indentifiable boundaries or ranges.
Another instance where boundary value testing is extremely useful in uncovering defects is when there are internal limits placed on certain resources, variables, or data structures.
Look for any kind of gradation or discontinuity in data values which affect computation - the discontinuities are the boundary values, which require thorough testing.
Look for internal limits on resources.
Look for documented limits on hardware resources.
BVA applies for white box testing as well. Internal data structures like arrays, stacks etc need to be checked for boundary or limit conditions. The way linked lists behave in the beginning and ending have to be tested thoroughly.
Boundary values and decision tables help identify the test cases that are most likely to uncover defects. A generalization of both these concepts is the concept of equivalence classes.

Decision Tables

A decision table lists the various decision variables, the conditions assumed by each of the decision variables, and actions to take in each combination of conditions.

The steps in forming a decision table are as follows:

Identify the decision variables.
Identify the possible values of each of the decision variables.
Enumerate the combinations of the allowed values of each of the variables.
Identify the cases when values assumed by a variable are immaterial for a given combination of other input variables. Represent such variables by 'dont care' symbol, which is usually the greek character phi.
For each combination of values of decision variables, list out the action or expected result.
From a table, listing in each but the last column a decision variable. In the last column, list the action item for the combination of variables in that row, including dont cares, as appropriate.

Equivalence Partitioning

Equivalence partitioning is a software testing technique that involves identifying a small set of representative input values that produce many different outputs conditions as possible.
Reduces the effort involved in testing and increases coverage.
The set of input values that generate one single expected output is called a partition.
When the behavior of the software is the same for a set of values, then the set is termed as an equivalence class or a partition.
One representative sample from each partition, also called member of equivalence class is picked up for testing.
The benefits of using equivalence classes : choosing a minimal set of input values that are truly representative of the entire spectrum and uncovering a higher number of defects.
Equivalence partioning is useful to minimize the number of test cases when the input data can be divided into distinct sets, where the behaviour or outcome of the product within each member of the set is the same.

The equivalence partitions table has columns:

Partitions definition
Type of input(valid/invalid)
Representative test data for that partition
Expected results

The steps to prepare an equivalence partitions table are:

Choose criteria for doing the equivalence partitioning(range, list of values etc).
Identify the valid equivalence classes based on the above criteria.
Select a sample data from that partition.
Write the expected results based on the requirements given.
Identify special values, if any, and include them in the table.
Check to have expected results for all the cases prepared.
If the expected result is not clear for any particular test case, mark it and escalate for corrective actions.

State Based or Graph Based Testing

State or graph based testing is useful in situations where:

The product under test is a language processor(like a compiler), wherein the syntax of the language automatically lends itself to a state machine.
Workflow modeling, where, depending on the current state and appropriate combinations of input variables, specific workflows are carried out, resulting in new output and state.
Dataflow modeling, where the system is modeled as a set of dataflow, leading from one state to another.

A general outline for using state based testing methods with respect to language processors is :

Identify the grammar for the scenario.
Design test cases corresponding to each valid state-input combination.
Design test cases corresponding to the most common invalid combinations of state-input.

Compatibility Testing

Compatibility Testing(CT) is done to ensure that the product features work consistently with different infrastructure components is called compatibility testing. Requires high degree of effort as there are a large number of parameter combinations.

The parameters that generally affect the comptability of the product are:

Processor and number of processors.
Architecture(32bit, 64bit and so on).
Resource availability(RAM, disk space, network card etc).
Equipment that the product is expected to work with.
Operating system.
Middle-tier infrastructure components such as web server, application server, network server.
Backend components such as database servers.
Services that require special hardware-cum-software solutions like clusters, load balancers etc.
Any software used to generate product binaries.
Various technological components used to generate components(SDK, JDK etc).

A compatibility matrix is created. It has its columns as various parameters the combinations of which have to be tested. Each row represents a unique combination of a specific set of values of the parameters.

Techniques for CT include:

Horizontal Combination
Intelligent Sampling

Compatibility testing of a product involving parts of itself can be classified into two types:

Backward compatibility testing
Forward compatibility testing

User Documentation Testing

User documentation covers all the manuals, user guides, installation guides, setup guides, read me file, software release notes, and online help that are provided along with the software to help the end user to understand the software system.

User documentation testing has two objectives:

To check if what is stated in the document is available in the product.
To check if what is there in the product is explained correctly in the document.

Benefits of user documentation testing include:

User documentation testing aids in highlighting problems overlooked during reviews.
Ensures consistency of documentation and product.
Results in less difficult calls for the support staff.
New programmers and testers who join a project group can use the documentation to learn the external functionality of the product.
Customers need less training and can proceed more quickly to advanced training and product usage if the documentation is of high quality and is consistent with the product.

Domain Testing

Domain testing goes beyond white box and black box testing, where we do not even look at the specifications of a software product but are testing the product, purely based on domain knowledge and expertise in the domain of application.
This testing approach requires critical understanding of the day to day business activities for which the software is designed.
Domain testing is an extension of black box testing.
The test engineers performing this type of testing are selected because they have in-depth knowledge of the business domain.
Domain testing exploits the tester's domain knowledge to test the suitability of the product to what the users do on a typical day.
Domain testing is the ability to design and execute test cases that related to the people who will buy and use that software.
Domain testing involves testing the product, not by going through the logic built into the product. The business flow determines the steps, not the software under test. This is also called 'business vertical testing'.
Domain testing is done after all components are integrated and after the product has been tested by other black box techniques.

Summarizing Black Box Testing Techniques

Scenarios	Most Effective Black Box Testing Technique
Output values dictated by certain conditions depending upon values of input variables.	Decision Tables
Input values in ranges, with each range exhibiting a particular functionality.	Boundary Value Analysis
Input values divided into classes, with each class exhibiting a particular functionality.	Equivalence Partitioning
Checking for expected and unexpected input values.	Positive & Negative Testing
Workflows, process flows, or language processors.	Graph or State Based Testing
To ensure that requirements are tested and met properly.	Requirements Based Testing
To test domain expertise rather than product specification.	Domain Testing
To ensure that the documentation is consistent with the product.	Documentation Testing