Test Management: From Ad-hoc Testing to a Systematic Quality Process
Good testing doesn't happen by accident. It requires structure, methodology, and the right tools.
Wilson Campero
ISTQB Certified Tester Advanced Level (Full)
I hold the black belt in software testing: ISTQB Full Advanced since 2014, all 3 modules. What I learned in 200+ projects, I share here.
22+
Years IT
200+
Projects
15+
Years Testing
5 Signs Your Test Management Isn't Working
Most projects test. But few manage their testing systematically. These 5 symptoms show the difference.
No test concept, no test strategy
Testing is done on whatever comes up. There's no document defining which test types, test levels, and test techniques are used.
Test cases exist only in testers' heads
No test management tool, no structured test cases. When a tester leaves the project, the test knowledge leaves with them.
No meaningful test reports
Test status is communicated via email or in dailies. There's no dashboard, no trend analysis, no decision basis for management.
Test coverage is unknown
Nobody can say which requirements are tested and which aren't. The traceability matrix doesn't exist or is outdated.
Bugs are found but not analyzed
Defects get fixed, but nobody asks: Why wasn't this bug found earlier? Where's the gap in the test process?
Recognized 3 or more? Your project tests, but the management behind it is missing.
Test management brings structure, traceability, and control to your testing.
4 Maturity Levels in Test Management
Where does your project stand? The 4 TMMi levels show how mature your test management is.
Level 1: Chaotic (Ad-hoc)
Typical: Testing happens, but unplanned. Test cases aren't documented. Test status is unclear. Quality gates don't exist.
Context: Typical for startups and projects without dedicated testers. Testing is seen as a developer responsibility.
Next step: First step: Create a minimal test strategy. Define what gets tested, who tests, and when testing happens.
Level 2: Managed (Defined)
Typical: There's a test strategy and test plan. Test cases are documented. Test status is reported regularly. But: Everything depends on individual people.
Context: Processes exist but aren't institutionalized. When the test manager changes, the system breaks down.
Next step: Standardize processes and capture them in templates. Define entry/exit criteria for each test level.
Level 3: Optimized (Measurable)
Typical: Test processes are standardized and measurable. Metrics (defect density, test coverage, defect leakage rate) are systematically collected. Reviews are established.
Context: The organization understands that testing is a steering instrument. Quality isn't just checked, it's managed.
Next step: Use metrics to continuously improve test processes. Deploy test automation strategically.
Level 4: Optimizing (Continuous)
Typical: Testing is fully integrated into CI/CD. Test processes are optimized data-driven. Quality is a team responsibility, not a department's task.
Context: The organization has established testing as a continuous improvement process. Shift-left and shift-right are lived practices.
Next step: Benchmark against industry standards. Evaluate AI-powered test optimization.
What maturity level is your test management at?
Start free maturity checkThe ISTQB Test Management Process
ISTQB defines 5 core activities in test management. Each activity is essential for systematic testing.
✓ Planning Phase
- Create test strategy and test concept
- Identify test scope and risks
- Define resources and schedule
- Plan and provision test environments
- Define entry and exit criteria
✗ Control Phase
- Monitor and report test progress
- Detect and escalate deviations from test plan
- Adjust test prioritization under time pressure
- Defect management and root cause analysis
- Prepare go/no-go decisions
7 Metrics Every Test Manager Must Know
- 1
Test Coverage: What percentage of requirements are covered by tests?
- 2
Defect Density: How many defects per 1,000 lines of code or per feature?
- 3
Defect Leakage Rate: How many bugs reach production that should have been found in testing?
- 4
Test Execution Rate: How many planned tests were actually executed in the sprint?
- 5
Defect Removal Efficiency: How effective is the test process at finding bugs before release?
- 6
Mean Time to Detect (MTTD): How long until a bug is discovered after introduction?
- 7
Cost of Quality: What does testing cost relative to the cost of quality failures?
Calculate Your Test Management Maturity
A simple indicator for your test management maturity:
Maturity Score = (Documented Processes + Measurable Metrics + Automation Rate) / 3
Example: Test strategy exists (1/1), 3 of 7 metrics collected (0.43), 20% automation (0.2). Score: 0.54 = Level 2.
Test Management Tools Compared
The right tool depends on your process, your budget, and your tool landscape.
| Criterion | Jira + Xray | TestRail | Zephyr Scale | qTest |
|---|---|---|---|---|
| Type | Plugin für Jira (Cloud & Server) | Standalone Testmanagement-Tool | Plugin für Jira (ehem. TM4J) | Enterprise Testmanagement-Plattform |
| Integration | Nahtlos in Jira integriert, Atlassian Marketplace | REST API, Integrationen mit Jira, Jenkins, Selenium | Native Jira-Integration, CI/CD Pipelines | Breite Integrationen, Requirements-Traceability |
| Strength | Marktführer bei agilen Teams, starke Traceability | Intuitive Oberfläche, starkes Reporting | Skalierbar, gute Automatisierungs-Anbindung | Enterprise-Features, Compliance-Reports |
| Cost | Ab 10 USD/User/Monat (Xray Cloud) | Ab 36 USD/User/Monat (Cloud) | Ab 10 USD/User/Monat (Cloud) | Auf Anfrage (Enterprise-Pricing) |
| Learning Curve | Mittel (Jira-Kenntnisse vorausgesetzt) | Niedrig (schneller Einstieg) | Mittel (ähnlich wie Xray) | Hoch (umfangreiche Konfiguration) |
| Recommendation | Wenn Ihr Team bereits mit Jira arbeitet | Wenn Sie ein dediziertes Testmanagement-Tool wollen | Wenn Sie Xray-Alternative in Jira suchen | Wenn Sie Enterprise-Anforderungen und Compliance haben |
Jira + Xray
- Type:
- Plugin für Jira (Cloud & Server)
- Integration:
- Nahtlos in Jira integriert, Atlassian Marketplace
- Strength:
- Marktführer bei agilen Teams, starke Traceability
- Cost:
- Ab 10 USD/User/Monat (Xray Cloud)
- Learning Curve:
- Mittel (Jira-Kenntnisse vorausgesetzt)
Wenn Ihr Team bereits mit Jira arbeitet
TestRail
- Type:
- Standalone Testmanagement-Tool
- Integration:
- REST API, Integrationen mit Jira, Jenkins, Selenium
- Strength:
- Intuitive Oberfläche, starkes Reporting
- Cost:
- Ab 36 USD/User/Monat (Cloud)
- Learning Curve:
- Niedrig (schneller Einstieg)
Wenn Sie ein dediziertes Testmanagement-Tool wollen
Zephyr Scale
- Type:
- Plugin für Jira (ehem. TM4J)
- Integration:
- Native Jira-Integration, CI/CD Pipelines
- Strength:
- Skalierbar, gute Automatisierungs-Anbindung
- Cost:
- Ab 10 USD/User/Monat (Cloud)
- Learning Curve:
- Mittel (ähnlich wie Xray)
Wenn Sie Xray-Alternative in Jira suchen
qTest
- Type:
- Enterprise Testmanagement-Plattform
- Integration:
- Breite Integrationen, Requirements-Traceability
- Strength:
- Enterprise-Features, Compliance-Reports
- Cost:
- Auf Anfrage (Enterprise-Pricing)
- Learning Curve:
- Hoch (umfangreiche Konfiguration)
Wenn Sie Enterprise-Anforderungen und Compliance haben
Frequently Asked Questions About Test Management
What's the difference between test management and quality management? +
Test management focuses on planning, controlling, and evaluating tests within a project. Quality management is broader: it encompasses all quality assurance measures, including processes, standards, and continuous improvement. Test management is a subset of quality management.
Do I need a test management tool or is Jira enough? +
Jira alone is not a test management tool. It can track bugs and tasks, but cannot manage test cases in a structured way, establish traceability to requirements, or deliver test reporting. Plugins like Xray or Zephyr Scale add these capabilities to Jira. For small teams, a simple setup (Jira + Confluence) often suffices. From 5 testers onward, a dedicated tool pays off.
What belongs in a test strategy? +
A test strategy defines: (1) Test objectives and test scope, (2) Test types and test levels (unit, integration, system, acceptance), (3) Test techniques (risk-based, experience-based, model-based), (4) Entry and exit criteria, (5) Test environments and test data, (6) Roles and responsibilities, (7) Metrics and reporting. It's created once per project and updated as needed.
How much test effort is normal? +
Rule of thumb: 25-35% of total project effort for testing (including test management, test design, test execution). In regulated industries (medical, automotive, finance), the share can rise to 40-50%. What matters isn't the absolute number, but whether the effort is distributed based on risk.
What is risk-based testing? +
In risk-based testing, tests are prioritized by the risk a failure would cause. Risk = severity x probability. Critical business processes are tested more intensively than edge functions. This saves time and money by investing test effort where it delivers the most value.
Test management in agile projects: How does it work? +
In agile projects, test management isn't a separate phase but a continuous activity. The test manager (or QA lead) defines the test strategy across sprints, supports the team with the definition of done, drives test automation, and delivers sprint reports. Test planning happens in sprint planning, test control in dailies and retrospectives.
Which ISTQB certification is relevant for test management? +
The ISTQB Certified Tester Advanced Level: Test Manager (CTAL-TM) is the reference certification. It covers test planning, test control, risk management, and metrics. Prerequisite: ISTQB Foundation Level. Additionally recommended: ISTQB Agile Tester for agile contexts and TMMi Professional for process improvement.
How do I start with test management if there is none? +
In 4 steps: (1) As-is analysis: How is testing done currently? What tools, processes, roles exist? (2) Write a test strategy: What gets tested at which level? (3) Introduce a tool: Store test cases in a structured way and link them to requirements. (4) Set up reporting: Weekly test reports with 3-5 key metrics. The entire setup takes 2-4 weeks.
About the Author
Wilson Campero
ISTQB Certified Tester Advanced Level (Full)
I hold the black belt in software testing: ISTQB Full Advanced since 2014, all 3 modules. What I learned in 200+ projects, I share here.
22+
Years IT
200+
Projects
15+
Years Testing
The content in this guide is based on 200+ projects at companies like Deutsche Telekom, Deutsche Bank, and SAP. From test strategy to tool selection to reporting.