How to beta test a humanoid or home robot
To beta test a humanoid or home robot well, give it the same small set of tasks every week, record how much help it needed and who was in the room, and back every fault with a timestamped photo or clip. BetaSuite: Tester's Toolkit (betasuite.app) is an iPhone, iPad and Android app built for this: it burns the date and time into every photo and video and turns your sessions into a report for the manufacturer. This category is new and programmes vary a lot, so the checklist below is general. It works with any tool, and there is a free 12-step protocol to import.
A new category. Humanoid, general-purpose and companion home robots are an early field, and no two beta programmes look alike. Programmes can differ in where the robot runs, how much supervision there is and whether a remote operator can help. This guide is general and doesn't describe any particular product. Your beta agreement and the manufacturer's safety instructions come first.
What to test in a humanoid or home robot beta
Start with safety and set-up, because everything else depends on them. Then work through the rest, and repeat anything that fails so you can say how often it fails. Skip any area your robot doesn't cover (a desktop companion robot won't walk or carry things) and note that you skipped it.
| Area | What you are checking | Evidence worth capturing |
|---|---|---|
| Set-up and onboarding | Time to a working robot, how clear the instructions are, how often you needed support, and whether it survives a restart or an update. | Photos of the space it works in and its charging spot, and a note of the time taken. |
| Safety behaviour and stops | How it behaves around people and pets, and whether the stop control (a button, the app or a voice command) halts it at once and keeps it stopped. Only the checks the programme tells you to do. | A short clip of each check, with the area clear. |
| Task success | A small set of agreed tasks, repeated: how often it completes each one with no help, how long it takes and how many times a person has to step in. | A clip of each attempt and a dated tally of successes, retries and interventions. |
| Navigation and awareness | Doorways, furniture, thresholds, rugs, stairs (only where the programme allows), changing light, pets and people moving past. Collisions, freezing, getting lost, and how close it gets to people. | A clip of each route and a photo of the layout. |
| Manipulation and handling | Picking up, carrying and putting down everyday objects of different size, weight and shape. Drops, crushing, knocked items, and how it copes with a failed grasp. | A close clip of each attempt, with the object in view. |
| Voice and conversation | Wake word, commands in a normal voice, accents, background noise, several people talking, follow-up questions and what it does when it doesn't understand. | A clip with the date and time visible, and a note of the room noise. |
| App and remote control | Pairing, status accuracy, schedules, manual control, notifications and what happens when your Wi-Fi drops. | A screen recording, or a dual recording of the app and the robot together. |
| Teleoperation and handover | If a remote operator can take over: whether you are told first, whether a clear indicator shows while it happens, how long it lasts and how control returns to the robot. | Screenshots or a screen recording of any notice, with the time visible, plus your own log of when it happened. |
| Privacy and data | Camera and microphone indicators, mute or privacy modes, areas it must stay out of, how recordings are shown, deleted or exported, and what the app says it collects. | Screenshots of the settings and indicators. Don't film other people to prove a point. |
| Battery and charging | Run time over a normal day, how it returns to charge (or how a battery is swapped), time to a full charge and what it does when the battery runs low mid-task. | Notes with the battery percentage at the start and end of each session. |
| Noise, heat and wear | Motor and servo noise at night and in quiet rooms, warm or hot spots, vibration, and wear on joints, grippers or wheels after weeks of use. | Dated notes, and close photos of any wear. |
| Recovery and updates | What it does after a fall, a stall or a power cut, whether it asks for help sensibly, and whether a software update keeps your settings, rooms and routines. | A clip of the recovery and a screenshot of the update notes. |
Safety first. A robot that can walk, lift or carry can hurt people, pets and itself. Follow the manufacturer's safety instructions and your programme's rules exactly. Keep children and pets out of the room while you test, leave plenty of clear space, and keep fragile, hot, sharp and valuable items out of its way. Test stop controls only the way the programme describes, with the area clear. If anything behaves in a way you don't understand, stop and contact the programme.
A repeatable task test
"It did the job" can't be compared from one week to the next. A fixed task can, so you can tell whether an update made things better or worse. It is a relative test, not a lab result.
- Pick two or three simple tasks that suit your home and your brief, such as carrying a light, unbreakable item from one place to another. If the programme has rules about which tasks you may run, agree them first.
- Write down the starting set-up for each task: where the robot and the item start, where the item must end up, the room layout, and the instruction you will give, in the exact words.
- Decide in advance what counts as success (for example, the item moved without being dropped and without a person touching the robot) and a time limit.
- Run each task several times (at least five), the same way each time. Film each attempt.
- Log every attempt as one of: done first time, done after a retry, needed a person to help, or failed. Note the time taken.
- Work out the rate for each task: attempts done without help ÷ attempts × 100.
- Repeat after every software update, and log the result in the same row each time.
Privacy and the people you live with
A home robot is a moving set of cameras and microphones, and some can be operated by a person elsewhere. Before you start:
- Read what the programme says the robot records, where that data goes, who can see it and how long it is kept. If anything is unclear, ask before you begin.
- Tell everyone you live with, and regular visitors. Agree where the robot can and can't go, and use any off-limits setting it offers. For bedrooms and bathrooms, keep it out altogether rather than relying on a setting.
- Check that the indicators and the mute or privacy modes work (the protocol below has a step for it).
- Keep other people, especially children, out of your evidence. Film the robot, the object and the screen, not faces.
- If a remote operator can take control, find out how you will be told, and log each time it happens.
- If the unit is under NDA, check what you may show before you share anything.
Conditions to record every time
A fault you can't reproduce is just a story. Log the same details every time so engineers can line your problem up against everyone else's.
| Record | Why it matters |
|---|---|
| Date, time and time zone | Engineers can match your report to the robot's own logs, if the manufacturer collects them. |
| Room and layout | Which room, the floor surface, the lighting, the clutter and where the furniture is. Behaviour often changes with light and clutter. |
| People and pets | Who was in the room and whether anyone walked past. Count rather than name ("two adults, one dog"). |
| Task and instruction | Which task, the exact words or app command you used, and the starting positions. |
| Robot state | Battery percentage at the start and end, mode, any restrictions or off-limits settings, and whether a remote operator was connected. |
| Network | Wi-Fi strength and any outage. Cloud features and app control may depend on it. |
| Software | Robot firmware, app and any voice-assistant version. Note them again after every update. |
| Changes | Anything different since the last run: new furniture, a rearranged room, a new router, an update. |
BetaSuite adds the date and time to every photo and video for you, and can add GPS, live weather, the season and the time zone, each of which you can turn on or off in Settings.
A first-week routine
Adjust this to your brief. The point is to do the same checks in the same order, so a change in the result means something changed in the robot.
| When | Focus |
|---|---|
| Day 1 | Set-up and the safety checks the programme describes. Photograph the space and the charging spot, and note the firmware. Run your first task once while you watch. |
| Days 2–3 | Your repeatable tasks in a clear space. Film each attempt and log the result. Try the app and voice control. |
| Days 4–5 | Navigation: doorways, furniture, a person walking past. Handling objects of different shapes. Note every time it needs help. |
| Days 6–7 | A normal day with the robot around. Log every stop, error, fall or handover, and how the battery lasted. |
| After any update | Repeat the safety checks the programme describes, one task and a check of the privacy settings. |
| Every week | Repeat the same tasks, in the same place, with the same wording, and compare the rates. |
A pass/fail checklist for a humanoid or home robot
Agree your thresholds with the programme's brief first. If it doesn't give any, write your own before you start (for example, "completes the task without help at least 4 times in 5") so you aren't judging after the fact.
- Sets up and connects to its app without a reset or a support call.
- The stop control halts it at once and it stays stopped until you restart it, if the programme asks you to check this.
- Completes each agreed task without a person's help, at least as often as your threshold.
- Moves around your home without collisions, freezing or unsafe movements, and keeps a sensible distance from people and pets.
- Picks up, carries and puts down everyday objects without dropping or crushing them.
- Responds to voice and app commands reliably, and the app shows its true status.
- Tells you when a remote operator is involved, and shows when it is happening.
- Indicators, mute and privacy modes work, and it stays out of rooms you marked off-limits.
- Manages its battery and gets back to charge without help (or the programme's swap process works).
- Recovers from a stall, a fall or a power cut safely, and asks for help sensibly.
- Has no unexplained errors, freezes or unexpected movements in a week.
- Installs a software update without losing your settings, rooms and routines.
A fail isn't a verdict on the robot. It's a finding. Write it up with the conditions and a clip.
How to capture evidence engineers can use
- Film the whole event, not just the fault. Ten seconds before, the fault itself, and ten seconds after.
- Start wide, then close. First the robot in the room, then the detail: the grip, the foot or wheel, the error on the app.
- Put the app and the robot in one clip. Dual recording in BetaSuite records your phone's screen and the rear camera into a single video, so what the app says and what the robot is doing sit side by side on one timeline.
- Use a reference. A tape measure or a taped line on the floor shows how far it moved or how close it got to something.
- Burn the time into the picture. A stamp that is part of the image travels with the file. File metadata is often stripped when a photo is shared or compressed. BetaSuite stamps every photo and video, with optional GPS, weather, season and time zone. More on that in the timestamp camera app guide.
- Catch the intermittent faults. Robots do odd things when you aren't filming. With Instant Capture, BetaSuite keeps a rolling buffer and saves the last 15–30 seconds when you tap, so you can work near the robot and still catch the moment. Without a tool like that, film the start of every task you can watch.
- Mark it as it happens. Add a one-tap marker or a quick voice note with the time, then write it up properly later.
- One issue per clip. Name it by what happened ("Dropped the cup, kitchen"), not "Video 14".
- Log every handover. If a remote operator takes control, note the time, how you were told and how long it lasted. A screenshot of the notice, with the time visible, beats memory.
- Keep people out of the frame. Film the robot, the object and the screen, not the people in your home.
What to send the manufacturer
A good report is short at the top, specific in the middle and backed up at the bottom. See how to write a beta test report for the full structure.
| Section | What goes in it |
|---|---|
| At a glance | Model and serial number or unit ID, firmware and app versions, test dates, the rooms and tasks you used, and whether a remote operator or an engineer was involved. Three sentences on how it went. |
| Findings, worst first | Safety issues first, then privacy concerns, then anything that stops the robot completing its tasks, then annoyances. For each one: what happened, what you expected, how to repeat it, how often it happens and the conditions. |
| Evidence | Time-stamped photos and clips matched to each finding by name or time. |
| What worked | A short list. Engineers also need to know what not to change. |
| Data files | A CSV or JSON of your findings if they ask, so they can load it into their own tools. BetaSuite reports can ship with both. |
Here is the format for one finding. The values are made up, to show the shape.
Title: Drops a plastic cup mid-carry and doesn't report it Severity: Medium (an object was dropped; nobody was near) Expected: Carries the cup to the table, or stops and asks for help Actual: Dropped it about 1 m from the table and carried on with the task Frequency: 2 of the last 6 attempts, both with the handle facing away Conditions: Evening light, kitchen floor, firmware 0.9.4, battery 61% Evidence: Wide clip then close-up of the grip, stamped 18:05:44
Want to see a finished report? Open the sample report (PDF). It has an At a Glance summary, findings sorted by severity and an image appendix.
Running it in BetaSuite
- Create a project for the robot, and note the model, serial number or unit ID, and the firmware in your first session.
- Import the free protocol below.
- Start a session each time you test. Capture the photos and clips the steps ask for, and drop a marker whenever something happens.
- At the end of the week, generate a PDF or Word report with an At a Glance summary and findings by severity, plus JSON and CSV data files if you want them.
AI summaries are optional and opt-in. For units under NDA, Confidential Mode blocks cloud AI and cloud uploads, and warns you before you share or export anything. Everything is stored on your device, with no account.
Free BetaSuite protocol: humanoid and home robot field test
This is a ready-made, 12-step protocol for the first week with a humanoid or home robot. It was built with the site's own Protocol Builder, so the link below is a real import link. It mixes pass/fail checks with choice, multiple-choice, rating and free-text questions, and asks for a photo, video or dual recording where the evidence matters. It is deliberately general: edit or delete any step that doesn't suit your robot or your programme's rules. For the general idea, see the test protocol template.
Humanoid and home robot field test · 12 steps · free to import and edit
- Set-up: following the programme's instructions, the robot is set up, connected to its app and ready to use without a support call. Photo: the space it works in, with its charging spot and a clear area around it.
- Stop control: only as the programme brief describes, and with the area clear of people and pets, the stop control (a button, the app or a voice command) halts the robot straight away and it stays stopped until you restart it. Film the check.
- Task success: ask it to do one simple task agreed with the programme, in a clear space, such as carrying an unbreakable item from one table to another. Did it complete the task without a person helping?
- Navigation: it moves between two points in your home, handling doorways, furniture and a person walking past, without collisions, freezing or movements you would call unsafe. It keeps a sensible distance from people and pets.
- Handling objects: with light, unbreakable everyday items only, it picks up, carries and puts down each one without dropping it, crushing it or knocking other things over. Keep fragile, hot, sharp and valuable items out of the test.
- Voice and app control: a spoken command and the same command from the app each start the same task, and the app shows the robot's status accurately. Record the robot and the app together.
- Remote operator handover: if a remote operator took control at any point this week, how did you know?
- Privacy controls: the camera and microphone indicators, the mute or privacy mode and the way to review or delete what it has recorded all work as the programme describes, and it stays out of any room you have marked off-limits. Photo: an indicator on the robot. Screenshot: the privacy settings in the app.
- Battery and charging: over a normal day of use, how did it manage its charge?
- Which problems did you see this week? Tick all that apply.
- How comfortable would you be having this robot in the room while you do something else? 1 = not at all, 5 = completely.
- In your own words: what was the most useful thing it did this week, and the most frustrating? Add the date and time.
Show the raw code
How to import it
- On your phone, tap Open this protocol. If BetaSuite is installed, it opens its import sheet with the code filled in. If the link opens in your browser instead, tap Copy code there (or here).
- In BetaSuite, open Test protocols, choose Import and paste the code or link.
- Edit any step to suit your unit, then run it. Your answers are saved to a session and roll up into the report. Protocols, sessions and reports are part of Advanced mode; if you're in Basic mode, Test protocols offers to switch for you.
Want to change the steps first? Open the link in the free Protocol Builder. It runs in your browser with no account, and you can generate a fresh code from your edits. Running a whole beta programme? See how to give every tester the same protocol and merge the results.
When BetaSuite is the wrong choice
BetaSuite covers everything from a single stamped photo to a full report for the manufacturer. It is the wrong tool only when:
- You need data from the robot itself. BetaSuite doesn't connect to the robot, so it can't read its logs, sensor data or what its cameras see. You note what the app shows, or record the screen.
- You would end up recording other people. BetaSuite is for evidence of what the product does. Don't use it to record people without their agreement, and follow your programme's rules on what you may capture in your home.
- You need safety certification or lab numbers. A phone can't test a robot's safety to a standard or measure its speed or lifting limit. Use your own repeatable tasks to compare your runs, not to certify anything.
- You want unattended monitoring. BetaSuite records the moments you start or save. A phone is not a safety monitor, so don't leave a robot running unsupervised because a phone is filming.
- You want to test an app, not hardware. BetaSuite is not a TestFlight or Google Play testing tool. See what a hardware beta testing app needs to do.
Don't need a report? If the manufacturer takes findings through its own portal or form, or you only want stamped photos and video, switch BetaSuite to Basic mode (Settings > App mode). It keeps every capture tool, including dual recording and Instant Capture, and leaves out projects, sessions and reports, so it works as a plain timestamp camera. Attach the files wherever the manufacturer asks, and switch back to Advanced at any time without losing anything.
If you run the programme rather than test in it, see the programme page. For the longer case for the app, see why BetaSuite.
Frequently asked questions
- What is a humanoid or home robot beta test?
- A pre-release or early-access trial where a manufacturer lets real people use an unfinished robot, its software or its app in real homes and report faults before general release. Programmes vary in how much supervision, remote help and independence they involve. The tester's job is to find problems in real conditions and describe them so engineers can reproduce them.
- How do I test whether a home robot completes tasks reliably?
- Pick a few simple tasks, write down the starting set-up and what counts as success, then run each one several times and log the result: done first time, done after a retry, needed help or failed. Repeat after each update. The success rate shows change over time, not a lab result.
- What should I record if a remote operator takes control of a home robot?
- Note the time, how long it lasted, whether the robot told you before or showed an indicator, and what the operator could see or do. A screenshot or screen recording of any notice, with the time visible, is better than memory. Check your programme's rules on what you may share.
- Is it safe to test a robot's stop controls?
- Only if you stick to the checks the manufacturer or programme tells you to do, with the area clear of people and pets and fragile items out of the way. Never invent your own tests of safety systems. If a stop control does not work, stop using the robot and tell the programme straight away.
- What about the cameras and microphones in my home?
- Before you start, read what the robot records, where that data goes and who can see it. Tell the people you live with, use any off-limits or mute settings, and keep other people out of your evidence. If the unit is under NDA, check what you may show.
- Can I use BetaSuite for a home robot under NDA?
- Yes. Everything is stored on your device with no account. Confidential Mode blocks cloud AI and cloud uploads, and warns you before you share or export anything. Photos and videos also go to your camera roll by default, so if your photo library syncs to the cloud, turn Camera Roll off in Settings → Save Destinations before you turn Confidential Mode on. Check your NDA first for what you may photograph and what you may send, and by what route.
- Is BetaSuite a TestFlight alternative?
- No. BetaSuite: Tester's Toolkit is not an app-beta tool and has no connection to TestFlight or Google Play testing tracks. It is for people who beta test physical products in the field and need timestamped evidence and a report for the manufacturer.
- Is BetaSuite a subscription?
- No. It is a one-time purchase from the App Store or Google Play, with no subscription, no ads and no account. Each store sets the price.
Related guides
Start testing with timestamped evidence
BetaSuite: Tester's Toolkit burns the date and time into every photo and video, then turns your sessions, markers and findings into a PDF or Word report, on iPhone, iPad and Android. Only need a timestamp camera? Use Basic mode.
One-off purchase · No subscription · No ads · No tracking · Yours for life