The following text and points are directed towards testing tools in particular while some of the points may in-fact be universally true. Tools do not replace the need for human process. Rather, tools should be used to enhance our capabilities.
Testing tools can offer higher test coverage, save time and resources, increase support for multiple platforms, support bug free releases, help identify defects to be fixed, and facilitate faster releases.
Tools should be easy to use, be compatible with the test environment, support multiple frameworks, contain robust object identification, and support a good debugging facility.
Considerations before selecting a new tool include (but are not limited to):
- Assessment of the maturity of the organization
- Opportunities for an improved test process supported by tools
- Understanding of the technologies used by the test object(s)
- Compatibility and integration with build and continuous integration tools
- Evaluation of the tool against clear requirements and objective criteria
- Whether the tool is available for a free trial period
- Evaluation of the vendor or support for noncommercial tools
- Internal requirements for coaching and mentoring in the use of the tool
- Evaluation of training needs
- Consideration of pros and cons of various licensing models
- Estimation of a cost-basis ration based on a concrete business case
- A proof-of-concept evaluation
- Estimation of a cost-basis ration based on a concrete business case
- Consideration of pros and cons of various licensing models
- Evaluation of training needs
- Internal requirements for coaching and mentoring in the use of the tool
- Evaluation of the vendor or support for noncommercial tools
- Whether the tool is available for a free trial period
- Evaluation of the tool against clear requirements and objective criteria
- Compatibility and integration with build and continuous integration tools
Pilot project for introducing a tool into an organization should have the following objectives:
- Learn more detail about the tool
- Evaluate how the tool fits with existing processes
- Decide on standard ways of using the tool
- Assess cost vs benefits
- Understanding the metrics that you with the tool to collect and report, and configuring the tool to ensure these metrics can be captured and reported
- Assess cost vs benefits
- Decide on standard ways of using the tool
- Evaluate how the tool fits with existing processes
Success factors for tools:
- Incremental roll-out to the rest of the organization
- Adapting and improving processes
- Providing training, coaching, and mentoring to new users
- Defining and communicating usage guidelines
- Gather usage information
- Monitoring tool use and benefits
- Providing support
- Gathering lessons learned
- Providing support
- Monitoring tool use and benefits
- Gather usage information
- Defining and communicating usage guidelines
- Providing training, coaching, and mentoring to new users
- Adapting and improving processes
Steps to successfully implement new tools:
- Recognize a need
- Give adequate time to test the tool
- Share with management
- Choose a project manager
- Evangelize the tool
- Customize hands-on learning
- Ready. Set. Test. Don’t roll out during a busy time.
- Ask for candid feedback.
- Make adjustments
- Roll out