Web UI Test Automation with Hermes: From Manual Testing to Automation
The Same Bottleneck Before Every Release
Before releasing a new version, two members of the team had to dedicate their full attention to the testing process. Cordatus has 259 scenarios across 7 modules: device connection in Device Hub, stream opening and recording management in Camera Hub, container deployment and VRAM calculations in Application Hub. Each scenario involves 5-10 steps: click, wait, check the result, save, move to the next.
As hours passed, focus dropped. It became harder to track which scenario we were testing and which step we were on. A feature marked as “working” in one scenario was actually failing silently, but we missed it due to fatigue. In another scenario, a known bug reappeared, but we’d note “it was there in the last release too” and move on.
The process was consuming us: human error, inconsistent verification, lost steps. And most importantly, as the team members doing the testing, we couldn’t develop new features during this time. The product waited, testing ran in circles.
The question was simple: What if an AI agent ran these tests itself?
We Can’t Build While Testing
Cordatus is a comprehensive platform consisting of 7 modules: Device Hub, Application Hub, Camera Hub, Cordatus Chat, Jobs, Logs, and Networks. Each module has an average of 30-70 scenarios, totaling 259+ test scenarios. Regression testing is mandatory on every release, because a change in one place can break another.
The cost of manual testing isn’t just time. Steps skipped due to fatigue, inconsistent verifications, features marked as “working” but silently failing. In one scenario, the “start recording” button is clicked, recording doesn’t start, but it gets marked as PASS.
And the biggest problem: We can’t develop while testing. While two people test for 3 days, those two people can’t develop new features. The product waits, testing runs. This isn’t sustainable for a rapidly growing product.
Hermes Takes the Field
Hermes Agent is an AI agent. It can use the browser, run terminals, read and write files, and most importantly, access the application’s internal state. This means instead of just looking at “what’s on the screen,” it can verify what’s actually happening in the background.
First step: the cordatus-ui-test skill was written. This skill is like a recipe: what to do for each scenario is written step by step. Preconditions, Steps, Assertions, Cleanup. 5 analysis files per module: frontend structure, backend endpoints, client layer, data flow, and scenarios.
Then the orchestrator pattern kicked in. Hermes works as a coordinator, running subagents sequentially to test in the browser. Each subagent gets a maximum of 3-4 scenarios; more than that, it gets stuck at the tool call limit. Lesson learned: Sequential, not parallel. Because the browser session is shared, if two agents log in at the same time, one’s session drops.
Technically, there are a few areas where Hermes stands out:
- Smart Login: Login in Cordatus is two-step: first email, then password. Hermes optimizes the login process by directly accessing the application’s internal state. The login that normally takes 5-6 steps in Hermes is completed in 3 steps. This is a significant saving given the limited operation budget.
- Automating Multi-Step Flows: Many operations in Cordatus require 5-10 step flows. To open a camera’s stream and start recording: select grid, select cell, select camera, select device, wait for stream to start, go to detail page, and start recording. Hermes automates these flows end to end, verifying the result at each step.
- Auto-Passing Confirmation Dialogs: Operations like deleting recordings trigger confirmation dialogs like “Please write DELETE to confirm.” Hermes automatically detects these dialogs, enters the required value, verifies the button is active, and clicks it. Without human intervention, in seconds.
- Real State Verification: Hermes doesn’t just verify what it sees on screen, it verifies the application’s internal state. When “recording started” appears, it checks whether it actually started in the background. The UI can be misleading, but internal state reflects the truth. When a misleading result is detected, the scenario is marked as FAIL.
- Observability: If you want, you can watch what Hermes does step by step. In visible Chrome mode, every click, every value typed, every dialog opened by Hermes can be viewed in real time. Screen recordings can be taken and reviewed later. You can literally see the AI’s testing process.
The Numbers Speak
The comparison below summarizes the difference between the traditional manual process and the AI-assisted workflow.
The values may vary depending on scenario complexity, infrastructure, and review requirements.
| Metric | Before | After |
|---|---|---|
| Test Scenarios | 259 scenarios | 259 scenarios |
| Required Resources | 2 human resources | 1 AI agent |
| Processing Time | Dependent on team workload and manual execution speed | Approximately 130 minutes using subagent batches |
| Process Consistency | May be affected by fatigue, skipped steps, and manual differences | More consistent execution through automated workflows |
| Bug Detection | Manual and dependent on reviewer consistency | 8 potential bugs identified during the automated analysis |
On Stage: llama.cpp Deployment
Numbers can feel abstract. To make the process more tangible, we recorded a complete deployment scenario from start to finish using Cordatus Application Hub.
In this example, the llama.cpp application is selected from the catalog, configured, deployed, and verified automatically. The goal is not only to confirm that the deployment starts successfully, but also to validate every step of the process, including container creation, model loading, API availability, and port assignment.
The entire workflow completes in approximately 44 seconds.
| Time | Stage | Verification |
|---|---|---|
| 0–5 s | Catalog Search | The Application Hub catalog is opened, “llama” is searched, and the application card is verified for its name, description, icon, and supported platforms. |
| 5–12 s | Application Details | The application detail page is opened. Version information, documentation, and supported platforms are validated before starting the deployment wizard. |
| 12–20 s | Device & Version Selection | The target device and application version are selected. The workflow verifies that navigation remains disabled until all required selections are completed. |
| 20–32 s | Configuration | An environment name, GPU, and AI model are selected. Readiness indicators are validated before the deployment button becomes available. |
| 32–40 s | Deployment | The environment is deployed. Container creation, running status, startup logs, and assigned network ports are automatically verified. |
| 40–44 s | API Validation | The deployed API is called through the assigned port and the returned model list confirms that the service is operational. |
One of the most valuable observations came during repeated execution. In an early run, the Start Environment button remained disabled after the model selection. Instead of stopping the process, the automation retried the scenario using a different model and browser interaction strategy. The deployment completed successfully and the issue was recorded for further investigation.
This demonstrates one of the key advantages of automated testing. The same workflow can be executed repeatedly with identical steps, allowing every run to validate previous fixes while producing consistent and reproducible results. Unlike manual testing, automated execution never becomes fatigued, skips a verification step, or forgets to document the outcome.
You can watch the complete deployment video below. The video has been played back at 2× speed !
“Did It Really Test?” Proof Reports
Let’s be honest. If an AI agent claims to have tested 259 scenarios in approximately 130 minutes, the first question is likely to be:
“How do we know it actually performed the tests? It could have simply marked every scenario as PASS.”
This is where detailed execution reports become important. Hermes creates a report for every tested scenario. It does not simply record PASS or FAIL. It documents each action step by step, including which button was clicked, what information was entered, what appeared on the screen, and which conditions were verified.
Example Scenario: Creating a New Job
The following example comes from the Jobs module. In this scenario, a new job must be created by selecting a model, target device, camera, and alarm rule before starting the inference process.
Once the job starts, the selected model processes the camera stream. If the configured alarm condition is triggered, the system generates a notification. Although the workflow may appear simple, it contains several dependent steps that must each be verified.
| Step | Stage | Recorded Actions and Verifications |
|---|---|---|
| Step 1 | Model Selection | The Start New Job button was clicked. The workflow reached the Models & Pipelines step. The TRAFFICCAMNET model was selected, and the model’s action button was used to continue to the next step. |
| Step 2 | Device Selection | In the Select Target Device step, the ASUS device checkbox was selected. The Select Device button was then clicked. |
| Step 3 | Camera and Alarm Configuration | The workflow reached the Deploy Multiple Models step. The job name was entered as Test-Job-Alarm-001. The Ofis 84 camera was selected. The Add Alarm button was clicked, the Select Alarms window opened, and the previously created Test-Alarm-001 alarm was assigned to the camera. |
| Step 4 | Job Launch | The Start Job button was clicked. The message “Deploying inference job… Step 6 of 8” appeared on the screen. The automation waited approximately 15 seconds, after which the job status changed to Running. |
| Result | Final Verification | PASS — The job was created successfully, the camera was assigned, the alarm was configured, and the job entered the Running state. |
Anyone reviewing this report can see that Hermes did not simply state that the scenario worked. It documented the selected model
(TRAFFICCAMNET), the target device (ASUS), the job name (Test-Job-Alarm-001), the camera (Ofis 84), the assigned alarm (Test-Alarm-001), the deployment message (Step 6 of 8), the waiting period (15 seconds), and the final status (Running).
If someone asks, “Did it really test the scenario?”, the complete execution record is available. The report shows which buttons were pressed, what information was entered, what appeared on the screen, and how the final result was verified.
This is the difference between saying “it works” and proving that it works. Hermes provides the proof.
Humans Don’t Test, Agents Test
Manual testing is rapidly giving way to AI agents. This is not a prediction—it is something we experienced firsthand at Cordatus. Hermes executed 259 test scenarios, identified 8 issues, and completed the entire test campaign in approximately 130 minutes.
Beyond speed, the real advantage is consistency. Every scenario is executed using the same workflow, every verification is documented, and every result is reproducible. There are no skipped steps, forgotten notes, or variations caused by fatigue.
Today, the Cordatus team spends less time repeating manual test procedures and more time improving the platform. Routine validation is handled by Hermes, allowing engineers to focus on designing, developing, and solving new problems.
This blog post was written by Hermes. The test scenarios were executed by Hermes. The issues were identified by Hermes. The CSV files were updated by Hermes. The final report was prepared by Hermes.
Now it’s your turn.
Try Hermes Agent, build your own skills, and let AI handle repetitive work while you focus on creating new solutions.
