Meet Automated Exploratory Testing

5 min read
477 views

For years we have treated automated tests as a regression-testing approach. That is what they are good at. The problem is the path they run is the path you wrote — hardcoded, repeatable, blind to anything outside it. A test can pass on a screen where a real user is lost, because the test only ever does the steps it was told. Technically, it works. Practically, it tells you very little about how the product feels to use.

New functionality has a different problem. No automation at all. A QA engineer has to invent the scenarios, write the Playwright code, debug the selectors, and verify the outcomes. Coding agents like Cursor or Copilot help with the typing, but the loop from “what should we test” to “this is signed off” needs a person driving it from end to end.

Both gaps bothered me enough to build something. I work as an engineer in the QA space. I created my own testing frameworks — Codeception (PHP) and CodeceptJS — and co-founded Testomat.io, a Test Management System. I have built many different tools around testing, but none of them actually tested an application by itself. In conversations with people from non-technical backgrounds I kept hearing the same question: why can’t we have a product that automatically my app? I had plenty of excuses and technical explanations to give. But when the AI era began, I thought: why not?

That is how I decided to build Explorbot — a self-driven test automation framework that actually tests an application on its own. At first I was just curious: how far could it go on a real web interface, what obstacles would it hit, what bugs would it find? It became an interesting challenge. What started as a pet project is now being adopted into our internal process.

From my laptop to our CI

One evening, while a session was still running on my laptop, I decided to go to sleep without fully closing the lid. The bot was still working when I shut it. In the morning I opened the laptop, and Explorbot was still going — twelve hours of continuous testing, real interactions, real verifications, real findings, and nobody had touched it. I was astonished.

I started reviewing the tests it produced and sharing the findings — with screenshots — with my team. There were various UI issues and edge-case scenarios we had never touched. It was like looking at our app from an unbiased perspective. Running Explorbot every evening became my daily routine, and I kept sharing what it found with the tech team.

At some point I decided Explorbot was mature enough to have its own space. I set it up on our CI server, and our SDET configured a workflow to launch it on a cron schedule every business day. It now runs on its own, alongside the rest of our testing pipelines. Along with the pass/fail statuses of our unit and end-to-end tests, we get a report on what functionalities Explorbot checked, and how.

How Explorbot works

I built Explorbot as a general web testing agent, so I didn’t hardcode any of our system specifics into its prompts. I could have attached our documentation or source code to it, but I didn’t. I was curious what it could discover on its own.

How Explorbot works
How Explorbot works

So how does Explorbot know what to test?

The same way your users know how to use your app. It reads the UI, figures out the domain, looks at the elements on the page, and picks plausible paths. It starts with the actions that change state — CRUD operations, toggling filters, switching views — and once the main page is exercised, it follows into the sub-pages it discovered along the way. Given enough time on a deep hierarchy, it walks the whole thing.

Explorbot also runs in multiple testing styles, so it covers more than the happy path. It will try SQL and JS injection in your inputs. It will type characters and combinations you did not expect to see in a form. It checks the edges, not just the centre. If you have never deliberately tested those flows, Explorbot will — and you will know what happens.

Pace and cost

In our runs, Explorbot produces 30 to 50 meaningful tests per hour. We have logged 2-hour continuous sessions in 4 parallel threads. It is strict about outcomes — every test goes through verification before it is reported as passed.

The cost is roughly $1 per hour in AI tokens through OpenRouter. At five hours a working day, that is about $110 a month. Explorbot itself is free. The figure above is the bare cost of the AI calls, nothing more.

Where it fits next to your existing tests

Explorbot is not a replacement for your regression suite, and it is not a replacement for your QA team. It fills the blind spots — areas with low coverage, new functionality, the screens nobody has had time to revisit. It produces evidence of what worked and what did not, plus a summary of each run, so your QA team can review the results and decide what to act on. Sometimes that means signing the test off, sometimes it means trying the flow manually, sometimes it means raising a defect.

How Explorbot works
How Explorbot works in Testomat.io

The flows Explorbot completes successfully are saved automatically as Playwright or CodeceptJS test files. Behaviour the bot discovers today can join your regression suite tomorrow.

Free and open source

Explorbot is free and open source. It is not hidden behind our cloud and it does not require a Testomat.io subscription. You install it like any other tool and run it on your own infrastructure, your own CI, or your laptop.

We keep it open because no two websites on the internet are alike. The only way Explorbot gets better in new environments is to be tried in them. We want to see it run on as many real applications as possible, and we want feedback from the teams that run it.

Think of Explorbot as Playwright with AI on top. Wherever Playwright can be installed, Explorbot can be installed too — locally, on any CI, on your own infrastructure.

Try it

– npm: <https://www.npmjs.com/package/explorbot>

– GitHub: <https://github.com/testomatio/explorbot#readme>

– Slack — we are happy to help you get it set up on your application

AI is reshaping testing along with the rest of software. Automated exploratory testing already works. The question is whether you start using it now or wait until everyone else has.

📋 Test management system for Automated tests
Manage automation testing along with manual testing in one workspace.
Follow us