Automated Testing Makes for Less Risky WMS Rollouts

Supply Chain Test Automation


Defects cost time and money, whether it’s a problem in your process or configuration, or more commonly in the form of a software failure or bug.  As the graph above shows, defects only get more expensive in terms of downtime(s), labor intensiveness, and reputation damage the longer they persist.  There is a key “cost avoidance” metric called defect leakage percentage which measures the how many defects were “leaked” from testing to production, and everyone – except your merciless competition that hate-watches your operation’s every move with bloodlust in their eyes – wants this number to be as low as possible.  Consider the ramifications of a software defect that prevents picking found via a nightly test, versus if that same issue was discovered in production after a go live.

Automated testing is, simply put, automating the execution and pass/fail reporting of repetitive tests against a system under test with the end goal of catching potentially costly defects as early as possible.  Manual testing is fine for those rare edge cases, visual acceptance testing, and when conducting exploratory testing, but it can be laborious, pricey, and inconsistent if it is performed regularly to test an entire system.  Automated testing is faster (more tests run in less time), more accurate, and cheaper while allowing for much greater test coverage and reporting capabilities than manual testing.  Whether regression or performance testing, manual testing can’t be solely relied upon to test – and in-turn reduce business risk for – warehouse management systems in a timely manner due to the overall complexity of such a deep, enterprise-wide application.  

Reasons for including automated testing in your WMS projects:

  • Customers want to anticipate solving supply chain execution problems
  • Staying current allows for taking advantage of the latest functionality and fixes
  • WMSes are now significantly more complex than the old days
  • More external integrations are required to meet both internal and external demands which adds to complexity
  • The current warehouse labor shortage means less available testers
  • The market demand for change is tightening up timelines
  • Full test coverage minimizes the chance of WMS outages/downtimes
  • You can take advantage of digital twin simulation testing
  • Stress / Load / Performance testing via automated testing (versus an extra shift doing it manually) really has no better example business case than mimicking a warehouse in full operation to accurately stress virtual and physical infrastructure, and establish reliable metrics
WMS Bugs


What types of tests are the most ripe for automation?

  • Older, repetitive tests that have crystal clear pass/fails for verification and/or validation.  A good example would be a test case that tries various username and password combinations for the WMS Web interface.
  • Data-driven tests and/or any test that requires lengthy, in-depth preparation.  Things like configuring and cleaning up large volumes of test data, or the submissions of lengthy forms with various combinations of input data.  Let your test automation solution take the brunt of the repetitive configuration management workload.
  • Sanity and smoke tests, especially any typical paths that your users take through the system under test.  For example, you should make doubly sure that warehouse personnel can pick the top three best-selling products.
  • Time-consuming tests that involve lengthy and repetitive user interactions.  Not only is it expensive to tie up a human tester with long, monotonous tests like this, but over time the manual tester becomes glassy-eyed and bored from mindlessly clicking – and in-turn the overall quality of testing drops.
  • Performance tests, as the very nature of determining speed, reliability, scalability, and resource usage is what automated tests are perfectly designed to tackle.  The process of finding system benchmarks demands the consistency and precision afforded by automated testing.  After the big network upgrade, does the WMS crash when a large team of warehouse personnel are all performing receiving, putaway, and picking operations at the same time?


Remember, good test cases have…

  • A clear objective with a refined scope
  • Obvious and meaningful pass/fail verifications
  • Clear and concise documentation
  • Traceability to requirements
  • Reusability
  • Independence from other test cases while testing one thing
  • Permutations taken into account by the test case designer


Can your team get away with manual testing just the most critical WMS processes?  The answer to that question:  It’s risky but…possibly?  Plenty of go lives and upgrades have been successful at the end of the day without adequate test coverage, though there was usually some unnecessary nail-biting and at least a few disruptive bumps in the road that could have been otherwise smoothed out in advance.  Plenty of rollouts have also bombed due to major defects that reared their ugly head in production which could have been caught earlier with the increased test coverage afforded by automated testing.  As profits, as well as jobs, are on the line we think automated testing is worth the extra initial “transitional effort” to drastically reduce business risk.   Are you in the market for less risky WMS go lives or upgrades, and/or looking for a test automation solution?  Drop us a line!

This post was written by:

James Prior

James Prior Sales Ops Manager  James has been working in software pre-sales and implementation since 2000, and more recently settled into working with a pre-sales team and occasionally writing blog posts. Drop him a line at: james.prior[at]tryonsolutions[dot]com.

Recommended Content

Similarities between Manufacturing Supply Chain and the Human Body

What? Come on now… How can an efficient manufacturing supply chain be similar to a healthy human body? They got nothing to do with each other? One is a living organism…and one is…well…an industrial process. Actually, the count is endless. A well oiled industrial...

6 Tips to Properly Maintain Your Warehouse Management System

Your operation has successfully implemented a warehouse management system.  Congratulations, as not all WMS implementations are successful – or so we hear, at Tryon Solutions we don’t have firsthand experience with that.  Your next step post-implementation is ensuring...

5 Supply Chain Systems Enhancements for Consumer Goods

Line up! The battle for the top systems enhancements for consumer packaged goods (CPG) companies starts now. Every single supply chain professional has their own list of tools needed and processes required. Every organization has their inventory issues with changes to...

4 Compelling Reasons to Outsource Supply Chain Execution Services

Why would one outsource when it’s technically possible to handle everything in-house?  I’m a fan of a nice haircut, and I’m willing to pay a little more to get it.  Before coming to this conclusion, I tried a well-known super-cheap option out there (rhymes with...