Evaluating Modern Testing Practices: A Comprehensive Look

Evaluating Modern Testing Practices: A Comprehensive Look

11 November 2011

In the evolving landscape of software development, particularly for enterprise systems like SAP, understanding the methodology driving your project is crucial. Major players like Deloitte Consulting and IBM offer proprietary frameworks like Thread Manager and Ascendant™ to guide you. Even SAP offers its Roadmap methodology through its Solution Manager platform. These frameworks are also backed by standards from established organizations like IEEE and the U.S. Department of Defense.

Smaller corporations without a defined methodology can also find guidance in classical software development models like the waterfall, spiral, and evolutionary approaches. These models are flexible enough to adapt to different project scopes and levels of requirement stability. If your organization already has a successful track record with other large-scale software projects, that experience can be invaluable in shaping your SAP implementation.

The key takeaway? Ensure that your chosen methodology provides sufficient guidance for testing your ERP system. Some frameworks designed for building software from scratch may not be suitable for off-the-shelf solutions like SAP.

Testing in Alignment with Project Methodologies #

Whether or not a formal methodology exists within your project, meticulous attention must be paid to aligning your testing activities with your overarching approach. Test managers should focus on generating comprehensive test plans that adhere to the project’s broader methodology to meet testing criteria effectively.

The Limitations of Manual Testing #

While manual testing is a go-to option for many projects, it’s not without its drawbacks:

  • Time-consuming: Documenting and executing each test manually can stretch timelines.
  • Complexity: The growing complexity of computing environments demands more thorough test coverage, nudging teams towards automated testing.
  • Globalization: Distributed teams require standardized processes that manual testing struggles to deliver.
  • Documentation challenges: Without automation, keeping documentation in sync with the testing process becomes a monumental task.
  • Error-prone: Manual tests are more susceptible to human errors compared to automated tests.

Record and Playback: Not a Silver Bullet #

Automated record and playback testing tools may appear to be a quick fix but often disappoint in the long run. These scripts are tightly bound to specific features or elements in the application, making them brittle and hard to maintain as the software evolves. The up-front effort to adapt and annotate these raw scripts often outweighs the benefits, defeating the purpose of automation.

Finding the Balance: Questions to Consider #

Before automating everything, ask:

  • Are your current manual tests cost-effective?
  • Could trimming or revising tests make them more manageable?
  • Would additional testers alleviate the workload?
  • Are the test procedures clear and well-understood by the team?

Leveraging Tools for Test Result Comparisons #

Comparison utilities are available not just in dedicated test tools but also within most operating systems. These can be powerful aids in assessing test results and can be a stepping stone to further automation.

Test Documentation: An Overlooked Aspect #

Detailed record-keeping is a must. Test documentation varies from management-level test plans to granular test scripts. Automated tools can offer solutions for documentation challenges, but the critical part is to have a well-defined process in the first place.

When is the Right Time for Automation? #

  • No ongoing organizational crises
  • A dedicated person responsible for tool selection and implementation
  • Dissatisfaction with current testing practices
  • Management support for the investment in tools and process improvement

If these conditions don’t align for you, it doesn’t mean automation is off the table. It just signifies that you might have to work a bit harder to implement it successfully.

While automated tools can significantly aid the testing process, they are not a panacea. A balanced approach that aligns with your organization’s unique needs and the specifics of your SAP implementation is essential for long-term success.