How to beta test a robot pool cleaner
To beta test a robot pool cleaner well, run the same cycle each week, record the conditions every time (water, debris, pool surface, battery or cable) and back every fault with a timestamped photo or clip. BetaSuite: Tester's Toolkit (betasuite.app) is an iPhone, iPad and Android app built for this: it burns the date and time into every photo and video and turns your sessions into a report for the manufacturer. The checklist below works with any tool and covers cordless and corded cleaners, and there is a free 12-step protocol to import.
What to test in a robot pool cleaner beta
A robot pool cleaner beta is mostly about shape, surface and water, because every pool is different and the water changes with the weather. Start with the first cycle and the pool floor, then work through the rest, and repeat anything that fails so you can say how often it fails. This covers cordless and corded models.
| Area | What you are checking | Evidence worth capturing |
|---|---|---|
| Set-up and first run | Time to get going, whether it powers on first time, and where the power supply, cable or charger sits in relation to the water. For a cordless model, how it charges and how long that takes. | Photos of the set-up (the cleaner, the power supply or charger, where the cable runs) and a note of the time taken. |
| Floor coverage | Whether it covers the whole floor, including the deep end, corners and the area under steps and around fittings, and how that changes between modes and cycle lengths. | Photos of the floor from the deck after a cycle, from the same spots each time. |
| Wall and waterline climbing | Whether it climbs each wall type in your pool, how cleanly it reaches and follows the waterline (where the model claims to), and whether it slips back, stalls or breaks the surface at the wrong moment. | A clip from the deck of the climb and the turn at the top, and a photo of the waterline afterwards. |
| Debris pickup | Fine debris such as sand and silt, and larger debris such as leaves and insects. Use the repeatable test below. | Before and after photos from the same spot, and a photo of what is in the filter. |
| Obstacles and pool features | Steps, ladders, main drains, lights, jets, benches and tight corners. Whether it snags, rides up, gets stuck or loses its way. On a corded model, whether the cable tangles or pulls it off course. | A clip of each encounter, and a photo of any marks or wear on the pool or the cleaner. |
| Filter and basket | How easily the filter comes out, empties and refits, how quickly it clogs, and whether suction drops as it fills. | A photo of the filter after each cycle and a note of when suction changed. |
| Power: battery or cable | Cordless: run time per cycle, charge time and low-battery behaviour. Corded: cable length against your pool, swivel and tangling. | Notes with the time and the battery level at the start and end, or a photo of a tangled cable. |
| Retrieval and docking | How the cleaner finishes: where it parks, how you get it out of the water, and whether you can lift it without leaning over the edge. Whether it returns to a dock or the wall, if it has that feature. | A clip of the end of a cycle and the way you lift it out. |
| App and connectivity | Pairing, schedules, modes, status and notifications. Radio signals generally don't pass through water well, so note when the app loses track of the cleaner and how it behaves when it reconnects. | A screen recording, plus a screenshot of any error with the time visible. |
| Safety systems | Out-of-water shut-off, and how the power supply and any cable are protected. Only the checks the manufacturer tells you to do, in the way it describes. | A short clip of each check. |
| Noise and wear | Noise at the pool edge, any grinding or vibration, and wear on the tracks, brushes, seals and cable after several weeks. | Dated notes, plus close photos of the wear points. |
Safety first. Water and electricity don't mix. Follow the manual for where the power supply goes and how it is plugged in, keep it well clear of the water, and disconnect the power before you touch the cleaner. Stay out of the water while it runs unless the manual says swimming alongside it is fine. Keep children and pets away from the pool and the cleaner while you test. A wet cleaner can be heavy, so use the retrieval method the manual describes and don't lean over the edge to reach it. Keep your phone dry and film from the deck. Check your beta agreement for anything else it requires.
A repeatable coverage and pickup test
"It cleaned well" can't be compared from one week to the next. This test can, so you can tell whether a firmware update, a mode or a new filter made things better or worse. It is a relative test, not a lab result.
- Choose four patches of floor about one square metre each: one in the deep end, one in the shallow end, one in a corner and one under or beside the steps.
- Pick a fixed mode, cycle length and starting point, and use the same ones every time.
- Photograph each patch from the deck, from the same spot every time. Then drop the same number of similar leaves over each (for example 20), soaked first so they sink, and let them settle.
- Run one cycle.
- Photograph each patch again. Count the leaves left on the floor, then the leaves in the filter.
- Work out the result for each patch: picked up % = (leaves placed − leaves left on the floor) ÷ leaves placed × 100. Note how many of the four patches were cleared.
- Note whether the pump, jets or any water features were running, because they move debris around.
- Repeat after every firmware update, filter change or settings change, and log the result in the same row each time.
Fine debris such as sand and silt is harder to count, so photograph it before and after and compare by eye.
Conditions to record every time
A fault you can't reproduce is just a story. Log the same details every time so engineers can line your problem up against everyone else's.
| Record | Why it matters |
|---|---|
| Date, time and time zone | Engineers can match your report to the cleaner's own logs, if the manufacturer collects them. |
| Water chemistry and pool type | Your own test-kit readings and whether the pool is chlorine, salt or something else. Chemistry can affect wear on tracks, brushes and seals. Record what you measured, not what you think it is. |
| Water temperature and weather | Very cold or very hot water, bright sun, wind and recent rain or storms change battery performance, debris loads and what you can see. |
| Pool shape and surface | Size, depth, shape, steps and fittings, and whether the surface is tile, vinyl, fibreglass or plaster. Grip and climbing vary by surface. |
| Debris load | What was in the pool (leaves, sand, pollen, algae, insects) and roughly how much, before the cycle. |
| Filter and maintenance state | Filter clean or full, brush or track wear, and how many cycles since the last service. |
| Other equipment | Pump, jets, heater, lights or a cover in use at the same time, which can move debris or get in the cleaner's way. |
| Mode and settings | Cycle mode and length, schedule and any custom settings. |
| Software | Cleaner firmware and app version. Note them again after every update. |
| Changes | Anything different since the last run: a new filter, a changed water level, a new router, a busy weekend of swimming. |
BetaSuite adds the date and time to every photo and video for you, and can add GPS, live weather, the season and the time zone, each of which you can turn on or off in Settings.
A first-week routine
Adjust this to your brief. The point is to do the same checks in the same order, so a change in the result means something changed in the cleaner.
| When | Focus |
|---|---|
| Day 1 | Set-up and a first cycle in the standard mode while you watch from the deck. Photograph the set-up and the pool floor. Note the firmware. |
| Days 2–3 | Full cycles in a clean pool. Check the floor from the deck afterwards, empty the filter and note how full it is. Watch the end of the cycle and how you get the cleaner out. |
| Days 4–5 | Walls and waterline, steps, ladders and corners. Try each mode. Test the schedule and the app controls. |
| Days 6–7 | Let it run on its schedule and check the result each day. Log every stuck event, error code and dropped connection. |
| After a windy day or a storm | Note how it copes with heavy debris and how quickly the filter fills. |
| Every week | Repeat the pickup test and the baseline photos from the same spots. After any firmware update, repeat the set-up, climbing and obstacle checks. |
A pass/fail checklist for a robot pool cleaner
Agree your thresholds with the manufacturer's brief first. If it doesn't give any, write your own before you start (for example, "no patch of floor left untouched bigger than a paving slab") so you aren't judging after the fact.
- Powers on and reaches a ready state at set-up, with no reset or call to support.
- Pairs with the app first time and keeps responding from beside the pool.
- Covers the whole floor, including corners and the area under steps, within your threshold.
- Climbs the walls it should, and cleans the waterline if the model claims it.
- Works past steps, ladders, drains and lights without getting stuck or snagged.
- Picks up at least your agreed share of the pickup test.
- Filter or basket comes out, empties and refits easily, and doesn't overflow in a normal cycle.
- Finishes its longest cycle: on one charge if it is cordless, or without tangling if it is corded.
- Parks, or can be retrieved, without anyone leaning over the edge.
- Stops its motors out of the water, if the manufacturer asks you to check this.
- Has no unexplained error codes, stops or stuck events in a week.
- Installs a firmware update without losing your schedule or settings.
A fail isn't a verdict on the cleaner. It's a finding. Write it up with the conditions and a clip.
How to capture evidence engineers can use
- Film the whole event, not just the fault. Ten seconds before, the fault itself, and ten seconds after.
- Film from the deck and keep your phone dry. Shoot from the edge or another safe spot, well away from the water. Glare off the water can hide the cleaner, so look down into the water at a steeper angle, with the sun behind you if you can.
- Start wide, then close. First the cleaner in the pool, then the detail: the filter, the tracks, the cable, the error on the app.
- Put the app and the cleaner in one clip. Dual recording in BetaSuite records your phone's screen and the rear camera into a single video, so what the app says and what the cleaner is doing sit side by side on one timeline.
- Use a reference. A tape measure or a tile of known size beside a missed patch shows how big it is. A coin beside a mark shows its size.
- Burn the time into the picture. A stamp that is part of the image travels with the file. File metadata is often stripped when a photo is shared or compressed. BetaSuite stamps every photo and video, with optional GPS, weather, season and time zone. More on that in the timestamp camera app guide.
- Catch the intermittent faults. Cleaners get stuck when you aren't looking. With Instant Capture, BetaSuite keeps a rolling buffer and saves the last 15–30 seconds when you tap, so you can work at the pool edge and still catch the moment. Without a tool like that, film the start of every cycle you can watch.
- Mark it as it happens. Add a one-tap marker or a quick voice note with the time, then write it up properly later.
- One issue per clip. Name it by what happened ("Stuck at the ladder, shallow end"), not "Video 14".
- Keep people out of the frame. Pools often have children, family or neighbours nearby. Film the water and the cleaner, not the people.
- Underwater footage needs the right camera. Don't take your phone into the pool. If you use a camera made for underwater use, BetaSuite can bring its clips in from your camera roll or Files and burn in their original date and time.
What to send the manufacturer
A good report is short at the top, specific in the middle and backed up at the bottom. See how to write a beta test report for the full structure.
| Section | What goes in it |
|---|---|
| At a glance | Model and serial number, firmware and app versions, test dates, whether it is cordless or corded, pool size and surface, and the water type (chlorine, salt or other). Three sentences on how it went. |
| Findings, worst first | Safety issues first (anything involving power, water or the cable), then anything that stops the cleaner working, then annoyances. For each one: what happened, what you expected, how to repeat it, how often it happens and the conditions. |
| Evidence | Time-stamped photos and clips matched to each finding by name or time. |
| What worked | A short list. Engineers also need to know what not to change. |
| Data files | A CSV or JSON of your findings if they ask, so they can load it into their own tools. BetaSuite reports can ship with both. |
Here is the format for one finding. The values are made up, to show the shape.
Title: Stops at the second step and reports an obstacle Severity: Medium (the cycle ends early) Expected: Works around the steps and finishes the cycle Actual: Stopped on the second step after about 25 minutes with an obstacle alert Frequency: 3 of the last 5 cycles, all with leaves in the pool Conditions: 26°C water, light debris, firmware 1.2.3, filter about half full Evidence: Wide clip then close-up of the filter, stamped 16:42:10
Want to see a finished report? Open the sample report (PDF). It has an At a Glance summary, findings sorted by severity and an image appendix.
Running it in BetaSuite
- Create a project for the cleaner, and note the model, serial number, whether it is cordless or corded, and the firmware in your first session.
- Import the free protocol below.
- Start a session each time you test. Capture the photos and clips the steps ask for, and drop a marker whenever something happens.
- At the end of the week, generate a PDF or Word report with an At a Glance summary and findings by severity, plus JSON and CSV data files if you want them.
AI summaries are optional and opt-in. For units under NDA, Confidential Mode blocks cloud AI and cloud uploads, and warns you before you share or export anything. Everything is stored on your device, with no account.
Free BetaSuite protocol: robot pool cleaner field test
This is a ready-made, 12-step protocol for the first week with a robot pool cleaner. It was built with the site's own Protocol Builder, so the link below is a real import link. It mixes pass/fail checks with choice, multiple-choice and rating questions, and asks for a photo or video where the evidence matters. Edit or delete any step that doesn't suit your model (for example, the cable check in the obstacle step on a cordless one). For the general idea, see the test protocol template.
Robot pool cleaner field test · 12 steps · free to import and edit
- Set-up: assemble and place the cleaner as the manual describes, and connect the power supply or charge the battery. It powers on and reaches a ready state with no error, and any power supply sits well clear of the water. Photo: the cleaner and its power supply as set up.
- App and pairing: the cleaner pairs with its companion app on the first attempt, and a cycle started from the app works from beside the pool.
- Floor coverage: after one full cycle in the standard mode, the whole pool floor is clean, with no missed strips or patches beyond your threshold. Photo: the worst missed patch, taken from the deck.
- Wall and waterline: it climbs the wall, cleans up to the waterline if the model claims that, and comes back down without slipping, stalling in a corner or leaving the water. Film from the deck.
- Obstacles: with a ladder, steps, a main drain, a light and a corner in the pool, it works past each one without snagging, getting stuck or losing its way. On a corded model, the cable doesn't knot or pull it off course. Film from the deck.
- Debris pickup: drop a measured handful of clean leaves or your usual debris on the floor, run one cycle and check what is left. How much did it pick up? Photo: what is left on the floor or in the filter.
- Filter: the filter or basket comes out, empties and refits easily, and it isn't clogged or overflowing after a cycle. Photo: the filter after a cycle.
- Retrieval: at the end of a cycle the cleaner parks where you can reach it from the deck, or can be lifted out the way the manual describes, without anyone leaning over the edge to reach it. A cordless model signals clearly when it is low on battery or stuck.
- Out-of-water safety: only if your beta brief covers it, and following the manufacturer's steps, the cleaner stops its motors when it leaves the water. Disconnect the power before you handle it, and keep the power supply away from the water.
- Power and cycle: did the cleaner finish its longest cycle? For a cordless model this is about the battery. For a corded one it is about the cable and swivel.
- Which problems did you see this week? Tick all that apply.
- How happy would you be to rely on this cleaner for your pool every week? 1 = not at all, 5 = very.
Show the raw code
How to import it
- On your phone, tap Open this protocol. If BetaSuite is installed, it opens its import sheet with the code filled in. If the link opens in your browser instead, tap Copy code there (or here).
- In BetaSuite, open Test protocols, choose Import and paste the code or link.
- Edit any step to suit your unit, then run it. Your answers are saved to a session and roll up into the report. Protocols, sessions and reports are part of Advanced mode; if you're in Basic mode, Test protocols offers to switch for you.
Want to change the steps first? Open the link in the free Protocol Builder. It runs in your browser with no account, and you can generate a fresh code from your edits. Running a whole beta programme? See how to give every tester the same protocol and merge the results.
When BetaSuite is the wrong choice
BetaSuite covers everything from a single stamped photo to a full report for the manufacturer. It is the wrong tool only when:
- You need data from the cleaner itself. BetaSuite doesn't connect to the cleaner, so it can't read its logs, route history or error history. You note what the app shows, or record the screen.
- You need lab numbers. A phone can't measure water chemistry, suction or noise to a standard. Use your own test kit and the pickup test to compare your own runs, not to certify anything.
- You want underwater footage from your phone. Don't put your phone in the pool. Use a camera made for it, then bring the clips in.
- You want to watch the pool while you're out. BetaSuite records the moments you start or save. A phone is not a pool monitor and is no substitute for supervising children around water.
- You want to test an app, not hardware. BetaSuite is not a TestFlight or Google Play testing tool. See what a hardware beta testing app needs to do.
Don't need a report? If the manufacturer takes findings through its own portal or form, or you only want stamped photos and video, switch BetaSuite to Basic mode (Settings > App mode). It keeps every capture tool, including dual recording and Instant Capture, and leaves out projects, sessions and reports, so it works as a plain timestamp camera. Attach the files wherever the manufacturer asks, and switch back to Advanced at any time without losing anything.
If you run the programme rather than test in it, see the programme page. For the longer case for the app, see why BetaSuite.
Frequently asked questions
- What is a robot pool cleaner beta test?
- A pre-release or early-access trial where a manufacturer lets real owners run an unfinished cleaner, firmware or app in real pools and report faults before general release. The tester's job is to find problems in real conditions and describe them so engineers can reproduce them.
- How long should a robot pool cleaner beta test run?
- As long as the programme asks. As a guide, run it long enough to see a clean pool and a dirty one, a few different cycle modes and at least one firmware update. For most pools that means several weeks rather than a few days.
- How do I test robot pool cleaner coverage fairly?
- Use the same starting point, mode and cycle length every time, and photograph the same patches of floor from the deck before and after. A repeatable pickup test with a fixed number of leaves in each patch shows change over time, not a lab result.
- Should I film a robot pool cleaner underwater?
- Only with a camera made for it. Don't take your phone into the pool. Film from the deck, and if you have underwater clips from a separate camera, BetaSuite can bring them in from your camera roll or Files and burn in their original date and time.
- Can I use BetaSuite for a pool cleaner under NDA?
- Yes. Everything is stored on your device with no account. Confidential Mode blocks cloud AI and cloud uploads, and warns you before you share or export anything. Photos and videos also go to your camera roll by default, so if your photo library syncs to the cloud, turn Camera Roll off in Settings → Save Destinations before you turn Confidential Mode on. Check your NDA first for what you may photograph and what you may send, and by what route.
- Is BetaSuite a TestFlight alternative?
- No. BetaSuite: Tester's Toolkit is not an app-beta tool and has no connection to TestFlight or Google Play testing tracks. It is for people who beta test physical products in the field and need timestamped evidence and a report for the manufacturer.
- Is BetaSuite a subscription?
- No. It is a one-time purchase from the App Store or Google Play, with no subscription, no ads and no account. Each store sets the price.
Related guides
Start testing with timestamped evidence
BetaSuite: Tester's Toolkit burns the date and time into every photo and video, then turns your sessions, markers and findings into a PDF or Word report, on iPhone, iPad and Android. Only need a timestamp camera? Use Basic mode.
One-off purchase · No subscription · No ads · No tracking · Yours for life