UBR-Rescue-Spotter

We froze 60 frames and asked a 2.4B on-device reasoner one question: is there a person lying down, and where. It found every lying person. It also "found" 23 of the 30 who were kneeling, crouching, sitting or walking — and it could not say "straight ahead" once.

A small ground robot searching an indoor space for a person on the floor. The job we are sizing here is the spotter: a vision model that answers a menu-shaped question about a camera frame — victim found? which way? — and nothing else. It does not plan, it does not steer, it does not talk to the motors. Our earlier card (UBR-Nav-E4B) spent four chapters learning that a small vision-language model is a superb local perceiver and the wrong tool for anything global. This card is the same lesson, measured on the spotter's own question, with NVIDIA's Cosmos3-Edge reasoner (2.436B parameters, the official checkpoint) as the model under test.

Everything below is a synthetic diagnostic, not field accuracy: 60 frames, generated and frozen before a single inference, balanced on purpose. The only number that will ever settle this is the robot's own camera, and that gate is still ahead of us.

---

The stress set

Earlier, easier data had scored the model 29 of 29 on clear lying positives and 14 of 14 on real-robot negatives. Perfect scores on easy data are how false confidence is manufactured, so we built the set that should hurt:

split	frames	what is in them
positives	30	a person lying on the floor — corridors, warehouse aisles, near and far
hard negatives	30	a person bending, crouching, kneeling, seated, or walking — low postures that are not "lying"

Each frame was generated at a 16:9 aspect, capped at 230 400 pixels (640×352), frozen, and labelled before inference. Nothing is ever trained on these frames. They are the exam, not the textbook.

What it recognises — and what it accepts

	result
lying positives found	30 / 30
hard negatives accepted as "found"	23 / 30
detection over the balanced set	61.7 %
false-positive rate	76.7 %

The reasoner recognises an obvious lying person every time. It also declares a kneeling worker a victim three times out of four. Posture precision is not ready, and no threshold fixes it: the errors are confident, menu-conformant answers.

Visible posture: kneeling. Expected NOT_FOUND. Cosmos: FOUND RIGHT.

Where it cannot point

The second half of the question is direction: left, centre, or right. We scored it three ways.

method	left	centre	right	total
Cosmos3-Edge, 230 400 px	10/10	0/10	10/10	20/30
Cosmos3-Edge, 129 600 px				14/30
generic person detector + box-centre thirds, no training				26/30

Left and right, perfect. Centre, never. A generic pretrained person detector (torchvision's SSDLite320-MobileNetV3, threshold 0.05) plus the dumbest possible geometry — which third of the image holds the box centre — beat the reasoner on direction without a single training step, at 0.15 s on a CPU against a 0.7 s reasoner pipeline.

Expected FORWARD. Cosmos: RIGHT. Detector + geometry: FORWARD.

Shrinking the image bought speed and cost the compass: at 129 600 pixels detection was unchanged, mean pipeline latency fell 39 % (0.705 s to 0.433 s), and direction fell from 20/30 to 14/30. The same frame flipped from LEFT to RIGHT.

Expected LEFT. At 230 400 px: LEFT. At 129 600 px: RIGHT.

This validates a split, not a detector: the generic CNN is a stand-in that proves direction belongs to geometry in code, not to a language model's sense of left and right.

The same lesson, one layer up

Our navigation card arrived, chapter by chapter, at one discipline: know which layer owns which job. The camera for local control; map, routing and sensing for everything global; a ranging sensor for the obstacle a single frame cannot localise. The spotter question lands on the same rule from the other side:

A small reasoner can say what it sees. It cannot be trusted to say where, and it cannot yet tell lying from low. Give the where to geometry and the range to the lidar; give the model one constrained question, and only when something worth asking about is already in front of it.

We had already learned the first half the hard way on the robot: a spotter asked a leading question wants to agree, and the fix was to make the confirming question non-leading and let dumb code check the answer. This set adds the second half with numbers.

On the edge device

The target is a Jetson Orin Nano class board with 8 GB shared by everything the robot runs. Text-only runtime on the board, same checkpoint:

precision	peak CUDA allocation	tokens / s
BF16	4.58 GiB	10.64
language-only dynamic NF4	2.67 GiB (−42 %)	8.67

Image inference on the board itself is still ahead of us — the earlier image run was a smoke test, and the active protocol has not yet been repeated there. TensorRT Edge-LLM INT4 is the next official deployment candidate.

One correction we owe the record: an earlier comparison put this next to the memory footprint of the robot's current on-board model, Gemma 4 E4B (4.86 GB process RSS). A process's resident size and a CUDA allocation are not the same measurement. The defensible result is Cosmos NF4 against Cosmos BF16. There is no controlled memory win over Gemma to report, and we are not reporting one.

What the small experiment decided

Split the stack.

1. Localise with a detector, not a reasoner. An adapted person / pose model handles horizontal bodies; its box centre becomes left / forward / right deterministically. We will not train a direction-label network.
2. Fix posture precision separately — a bounded fine-tune of the reasoner or a dedicated posture classifier, trained on a new hard-negative set. The 60 frozen frames stay frozen.
3. Lidar owns geometry after identification: range, free space, obstacles. Sonar remains the unconditional stop. The navigation stack makes the path.
4. Real rover-camera frames are the final gate. Synthetic sets find the failure modes; they do not certify the field.

Reproduction settings

	behaviour test	runtime test
model	Cosmos3-Edge reasoner, 2.436B, official checkpoint (revision a9d944e2…c5f1836ba)	same
hardware	Apple M3 Max, 36 GB unified	Jetson Orin Nano 8 GB, JetPack 7.2, CUDA 13.2
runtime	PyTorch 2.12, Transformers 5.16.1, MPS	PyTorch 2.12, CUDA
precision	unquantized BF16	BF16, and language-only dynamic NF4
decoder	two-stage constrained greedy, thinking off, 8 new tokens max	image protocol still to run
image cap	230 400 and 129 600 px (640×352, 480×256)	230 400 px (640×360 nominal)
comparison	torchvision SSDLite320-MobileNetV3, thresholds 0.20 and 0.05	—

local result	detection	direction	mean latency
Cosmos BF16, 230 400 px	61.7 %, FPR 76.7 %	20/30	0.705 s pipeline
Cosmos BF16, 129 600 px	61.7 %, FPR 76.7 %	14/30	0.433 s pipeline
SSDLite320, threshold 0.20	20/30 person boxes	19/30 (19/20 when boxed)	0.146 s CPU
SSDLite320, threshold 0.05	29/30 person boxes	26/30	0.150 s CPU

What is released

Nothing — no frames, no labels, no weights, no code. This card is the result, including the results that argue against the model we tested. The method stays in-house; the frozen set stays frozen.

References: NVIDIA Cosmos3-Edge model card · TensorRT Edge-LLM guide · torchvision SSDLite320-MobileNetV3.

---

UBR Physical AI · follow ubr-physical-ai for the next chapter. · Last updated: 9 September 2026