Automated Testing Makes for Less Risky WMS Rollouts

Supply Chain Test Automation


Defects cost time and money, whether it’s a problem in your process or configuration, or more commonly in the form of a software failure or bug.  As the graph above shows, defects only get more expensive in terms of downtime(s), labor intensiveness, and reputation damage the longer they persist.  There is a key “cost avoidance” metric called defect leakage percentage which measures the how many defects were “leaked” from testing to production, and everyone – except your merciless competition that hate-watches your operation’s every move with bloodlust in their eyes – wants this number to be as low as possible.  Consider the ramifications of a software defect that prevents picking found via a nightly test, versus if that same issue was discovered in production after a go live.

Automated testing is, simply put, automating the execution and pass/fail reporting of repetitive tests against a system under test with the end goal of catching potentially costly defects as early as possible.  Manual testing is fine for those rare edge cases, visual acceptance testing, and when conducting exploratory testing, but it can be laborious, pricey, and inconsistent if it is performed regularly to test an entire system.  Automated testing is faster (more tests run in less time), more accurate, and cheaper while allowing for much greater test coverage and reporting capabilities than manual testing.  Whether regression or performance testing, manual testing can’t be solely relied upon to test – and in-turn reduce business risk for – warehouse management systems in a timely manner due to the overall complexity of such a deep, enterprise-wide application.  

Reasons for including automated testing in your WMS projects:

  • Customers want to anticipate solving supply chain execution problems
  • Staying current allows for taking advantage of the latest functionality and fixes
  • WMSes are now significantly more complex than the old days
  • More external integrations are required to meet both internal and external demands which adds to complexity
  • The current warehouse labor shortage means less available testers
  • The market demand for change is tightening up timelines
  • Full test coverage minimizes the chance of WMS outages/downtimes
  • You can take advantage of digital twin simulation testing
  • Stress / Load / Performance testing via automated testing (versus an extra shift doing it manually) really has no better example business case than mimicking a warehouse in full operation to accurately stress virtual and physical infrastructure, and establish reliable metrics
WMS Bugs


What types of tests are the most ripe for automation?

  • Older, repetitive tests that have crystal clear pass/fails for verification and/or validation.  A good example would be a test case that tries various username and password combinations for the WMS Web interface.
  • Data-driven tests and/or any test that requires lengthy, in-depth preparation.  Things like configuring and cleaning up large volumes of test data, or the submissions of lengthy forms with various combinations of input data.  Let your test automation solution take the brunt of the repetitive configuration management workload.
  • Sanity and smoke tests, especially any typical paths that your users take through the system under test.  For example, you should make doubly sure that warehouse personnel can pick the top three best-selling products.
  • Time-consuming tests that involve lengthy and repetitive user interactions.  Not only is it expensive to tie up a human tester with long, monotonous tests like this, but over time the manual tester becomes glassy-eyed and bored from mindlessly clicking – and in-turn the overall quality of testing drops.
  • Performance tests, as the very nature of determining speed, reliability, scalability, and resource usage is what automated tests are perfectly designed to tackle.  The process of finding system benchmarks demands the consistency and precision afforded by automated testing.  After the big network upgrade, does the WMS crash when a large team of warehouse personnel are all performing receiving, putaway, and picking operations at the same time?


Remember, good test cases have…

  • A clear objective with a refined scope
  • Obvious and meaningful pass/fail verifications
  • Clear and concise documentation
  • Traceability to requirements
  • Reusability
  • Independence from other test cases while testing one thing
  • Permutations taken into account by the test case designer


Can your team get away with manual testing just the most critical WMS processes?  The answer to that question:  It’s risky but…possibly?  Plenty of go lives and upgrades have been successful at the end of the day without adequate test coverage, though there was usually some unnecessary nail-biting and at least a few disruptive bumps in the road that could have been otherwise smoothed out in advance.  Plenty of rollouts have also bombed due to major defects that reared their ugly head in production which could have been caught earlier with the increased test coverage afforded by automated testing.  As profits, as well as jobs, are on the line we think automated testing is worth the extra initial “transitional effort” to drastically reduce business risk.   Are you in the market for less risky WMS go lives or upgrades, and/or looking for a test automation solution?  Drop us a line!

This post was written by:

James Prior

James Prior Sales Ops Manager  James has been working in software pre-sales and implementation since 2000, and more recently settled into working with a pre-sales team and occasionally writing blog posts. Drop him a line at: james.prior[at]tryonsolutions[dot]com.

Recommended Content

We Saw Robot Picking in Action…

    An efficient operation is no longer a “nice to have” in this era of labor shortages and ruthless competition.  We recently toured the warehouse of a large electronics company located in the Southern United States that saved 35,000 US dollars a month...

Top Five Warehouse Management System KPIs

  How do we know if our warehouse or distribution center is running as expeditiously as possible?  Accounting department touts great numbers, warehouse personnel are clearly working hard, and everything “looks and feels” right, so isn’t that good enough?  You...

Waving Strategies for Your Warehouse or DC

  Wave picking is a warehouse order picking system where a worker will receive a picking list, and rather than walking back-and-forth, the items are grouped together by a common and specific order characteristics which can be zone, shipping priority, carrier,...

Choosing the Right Tech for Your Picking Strategy

  In Part I of our “Picking Strategy and Tech” series, I mentioned that Distribution Centers are going to have to deal with more diverse order profiles in this new digital age as wholesale orders are changing from Pallet size orders to case size orders.  Digital...

Choosing the Right Picking Strategy for Your Warehouse or DC

  As consumer product companies “go digital” with orders coming from a growing list of omni-channel sources, order profiles are evolving rapidly. Often times these profiles are reduced to just a few items per order, and to meet these changing demands digital...