Defining your Test Strategy

1. Identify Stakeholders (i.e. the people who have an interest in the outcome of the tests, and the information that the testers provide)

   1. Sponsors - the system should support their business goals

   2. Users - the system should “work” for them

   3. Project Management - needs information on the status of the deliverables

      • replanning

      • managing expectations

   4. Designers and Developers - need to know where the product fails

      • fixing defects

      • adjusting designs

2. Define your test strategy

Test Case Management Solution - a product that helps streamline test execution, reporting, future strategy, management, and overall quality

Test Planning Strategy - defines the process that the test team will be using to achieve the testing goals

• Presents some decisions that can be made ahead of time.

• Defines the process/ method/information that will allow decisions to be made.

• Sets out the principles or process to follow for uncertain situations.

Developing a Test Strategy Framework

1. Stakeholder Objectives

   1. Stakeholders

   2. Goal and Risk Management

   3. Decisions

   4. Confidence

   5. Assessment

   6. Scope

2. Design Approach

   1. Sources of Knowledge

   2. Sources of Uncertainty

   3. Models

   4. Prioritization Approach

3. Delivery Approach

   1. Test Sequencing

   2. Retesting

   3. Environment requirements

   4. Information delivery approach

   5. Incident Management approach

   6. End-game approach

Shift-Left Testing - whenever possible, feedback should be provided to help the team to understand, challenge, or improve goals, requirements, design, or implementation

• developers take more ownership and responsibility for their testing

• testers can get involved in the project earlier

• users, BAs, and developers take full responsibility for the testing

• no test team

• no testers

Mind Maps

Mind Mapping - a method for generating test plans by mapping out all possible related ideas. This is helpful to organize ideas in the process of brainstorming while trying to come up with ideas and solutions.

Parts of the Mind Map:

1. Central Node

2. Branches

   1. Testing Scope

      1. in-scope

      2. out-of-scope

   2. Timescale

   3. Testing Resources

   4. Testing approaches

   5. Risks and Assumptions

3. Items and Activities

One-page Test Plan

1. Resourcing

2. In-scope

3. Out-of-scope

4. New Functionality

   1. Feature Description

   2. Depth of Testing

5. Performance & Load Testing

6. User Acceptance Test

7. Infrastructure Considerations

8. Assumptions

9. Risks & Mitigation Plans

10. Sprint Test Timeline

11. Test-planning

Risk-Based Testing

Types of Software Risks:

1. Project Risk - e.g. external dependencies for the project

2. Process Risk

3. Product Risk

Two Dimensions of Risk Assessment:

1. Probability - 0 to 100%

2. Consequence or Severity - potential cost of damage

Risk Exposure - how serious a risk is

$$Probability * Severity Rank = Risk Exposure$$

Risk-based Test Planning

• formulating a set of tests to assess the likelihood of the risk materializing

• Simulate the failure mode in as many ways as possible

A. Failures - by detecting the number of bugs there are to be fixed, the risk from that mode of failure is reduced

B. Passes - increases the confidence that a mode or failure is unlikely to occur

Fault Detection - reduces the residual risks of failures in production

Designing your Tests

Test Design - the process of selecting the tests that we believe will be of value to the stakeholders

Test Model - a checklist, set of criteria, diagram based on the design documents, or analysis of a narrative text that helps to select tests systematically

• Requirements Documents

• Use Cases

• Flowcharts

• Swim-lane Diagrams

• State models

• Collaboration diagrams

• Sequence charts

  Uses:

  1. Simplify the context of the test

  2. Focus attention on one perspective of the system’s behavior

  3. Generate a set of tests that are unique and diverse within the context of the model

  4. Enable the testing to be estimated, planned, monitored and evaluated for its completeness or coverage

Coverage Measurement

• What has been tested?

• What hasn’t been tested?

• Is the project finished?

• How many tests remain?

Managing & Executing your testing

Keys to a Successful Plan Execution

1. People

2. Environments

3. Knowledge

4. System Under Test

Agile Test Interventions

Test strategy in Agile - a series of test interventions, i.e. a loop for feedback and analysis

1. Identify Critical moments

2. Propose contributions

3. Negotiate

Project-level interventions

1. Story Challenge

2. Story Definition

3. Integration Tests

4. System Tests

5. User Acceptance Tests

Sprint-level interventions

1. Daily standup

2. Story Refinement

3. Developer Testing

4. Integration/System testing

Source