I am surprised how sketchy test management tools are out there. I can compare the feeling when one makes an evaluation on test management tools and makes a decision on which one to use to choose between the two hands to bite off.
Definition of good is: fits my needs.
What are my needs against a test management tool?
- Integrated in Jira
Yes. Sorry. Reducing the administration cost of the “yet another tool”, LDAP, authorization are amongst the reasoning.
- Import/Export functionality
Import test cases from csv is the minimum.
Export test cases, test plans, reports to doc, pdf, csv.
Ability to search on keywords inside the TC description, steps. That is super cool, if search works inside the attachments as well.
- Custom workflow setup
Workflow for test cases so test case review process can be defined
- Version control for test cases
When the TC is modified and by whom.
- Custom field setup for test cases
To make distinction on which test case is automated, which one is manual.
- Custom field setup for high level/master test plans
- Test suite creation
Between requirement, test case, bug report
- Active support
To get the latest bugfix packages. There is no bugfree software, right?
I take some things for granted, e.g. test plan creation, test case id, test execution status etc, that is why I don’t mention these in the list above. Listed item are weaknesses in some of the tools. Implemented poorly, or not implemented at all.
I found that reporting is the main pain point of TM tools. Yes, reporting belongs to the list above.
- Report creation by releases
- Report creation by test suites
- Customizable filters
- Cumulative report for traceability
- Reporting along with known issues and their actual status
- Storing reports
- Email notification
In some cases REST APIs are provided so one can spend time on it. In other cases extension plug-ins are available (of course for extra money) which tries to fill in the gap between reality and expectations.
But either way, testers along with product people have to give up some of their needs and have to make a consensus based on what would hurt the least.