As with mabl, I was encountered enough problems on first encounter with the product that it swamped my ability to stay focused and keep track of them all.
I stumbled over several problems that are not included in this report, and I perceive many systemic risks to be investigated. In each case, Katalon’s recording functions failed to record my behaviours reliably. I pointed Katalon Record to three different Web applications. My first impression is that the tool is unstable, brittle and prone to systematic errors and omissions.Ī very short encounter with the product reveals startlingly obvious problems, including hangs and data loss. The idea behind sympathetic testing is not to find bugs, but to exercise a product’s features in a relatively non-challenging way. It’s brief, shallow testing to determine whether if the product is fit for deeper testing, or whether it has immediately obvious or dramatic problems.
Sanity testing might also be called “smoke testing”, “quick testing”, or “build verification testing”. It provides a foundation for effective, efficient, deliberative, deep testing later on. Survey testing tends to be spontaneous, open, playful, and relatively shallow. The general mission of survey testing is learning about the design, purposes, testability, and possibilities of the product. My self-assigned charter was to explore and survey Katalon Studio, focusing on claims and identifying features in the product through sympathetic use.Īs before, I will include some meta-notes about the testing in indented text like this.
This time, the application is Katalon Studio. Last time, the application under test was mabl. The work was performed in September 2021, with follow-up work November 3-4, 2021. This is an experience report of attempting to perform a session of sympathetic survey and sanity testing on a “test automation” tool. But hey, it’s worth it, right? Introduction