Network · Path · Envelope — three predictive visualizations for delayed teleoperation
Every Move You Make
Visualizing near-future motion under delay for telerobotics. — when feedback lags by seconds, show operators where the robot is heading, not just how the delay works.

Driving a robot over a long-distance link means acting before you can see the result: a command issued now only shows up in the video seconds later. This paper splits that problem into three kinds of uncertainty — when commands take effect, how they map to motion, and what the terrain does to them — and tests a visualization for each. Across 24 operators under a fixed 2.56-second round-trip delay, only the one that projected the robot's committed trajectory (Path) actually made control faster and easier.

Dries Cardinaels · Raf Ramakers · Tom Veuskens · Thomas Pietrzak · Gustavo Rovelo Ruiz · Kris Luyten Digital Future Lab · UHasselt – Flanders Make · with Univ. Lille / Inria / CRIStAL · Diepenbeek, Belgium · CC BY-NC-ND 4.0
24 operators 2.56 s round-trip delay CHI 2026 · Barcelona
The gap

Delay breaks the link between what you do and what you see

Over a long link — a rover on the Moon, a vehicle across a flaky network — a command can take seconds to land, and the video confirming it arrives seconds later still. Operators cope by going "move-and-wait": nudge, freeze, wait for the picture to catch up, repeat. It's stable but slow and draining. Predictive displays have long tried to help, but most treat delay as one lump and just show a single guessed future state. This work instead asks: delay creates several distinct uncertainties — which one actually needs externalizing?

Reactive

Move-and-wait

  • Issue one input, then pause ~2.56 s for the video to confirm it
  • Task time and mental effort climb with every pause
  • Operators fall back on a fragile mental "ghost car" that breaks on rough terrain
EXTERNALIZE
Proactive

See the near future

  • Project the committed trajectory the robot will follow
  • Act now, before the feedback arrives
  • Fewer pauses, lower load, steadier control — even as terrain shifts

Key reframe: delay isn't only a communication defect to engineer away — it's operator uncertainty to design for. The paper separates it into three facets: communication (when), trajectory (how), and environmental (what).

Contributions

What the paper adds

C1

Delay, decomposed

Reframe delayed teleoperation as three facets of operator uncertainty — communication, trajectory, and environmental — instead of one monolithic problem.

C2

Three visualizations

A visualization that externalizes each facet: Network (command timing), Path (predicted motion), and Envelope (worst-case deviation from terrain and noise).

C3

A controlled study

24 participants drive a simulated rover under a fixed 2.56 s round-trip delay, with each visualization compared against a plain delayed-video baseline.

C4

What actually works

Trajectory feedforward (Path) cut task time and load and reduced reactive control; Envelope lowered load only; Network showed no measurable gain.

Try it · interactive

Drive a rover on a delay

You're steering the rover, but the view lags: the rover you see is where it was a moment ago, while the real one has already moved on your latest inputs. Reach the green target. Then switch visualizations and feel which one lets you stop guessing — Path draws the motion you've already committed but can't yet see.

Rover · lunar terrain
Click in · drive with ↑↓←→ (or buttons)
Active visualization
Baseline
Delayed video only — what the camera shows.
Round-trip delay
0.6 s1.6 s3.0 s
Options
Reveal actual position
Status
Driving…
0.0 s · steering 1.6 s into the future

Illustrative single-delay simulation — not the paper's UNITE data. The lag between the rover you see and where it actually is stands in for the 2.56 s round-trip; Path overlays the near-future motion you've already committed, Network shows in-flight command timing, and Envelope adds a worst-case spread. "Reveal actual position" is a teaching aid the real operator never gets.

The three visualizations

One facet of uncertainty each

All three are feedforward overlays — they show a likely future before it happens. They differ in what future they externalize, and that turns out to decide whether they help.

WHEN

Network

Four timelines — one per arrow key — show each command as a block sweeping toward execution, making the invisible round-trip delay visible and rhythmic.

Targets communication uncertainty: when will my input take effect?

command timingstatus visibility
HOW

Path

Projects the robot's intended trajectory from queued inputs — a center line plus a path for each wheel — using ideal differential-drive kinematics, capped at the 2.56 s lookahead.

Targets trajectory uncertainty: where will my inputs take the robot?

feedforwardcommitted path
WHAT

Envelope

Extends Path into a translucent cone marking the worst-case spread from terrain slip, motor noise and other disturbances — a guaranteed bound, not a probability.

Targets environmental uncertainty: how far might reality diverge?

worst-case boundsterrain & noise

All three run inside UNITE, a Unity-based simulation with a TurtleBot3 model, a six-source deterministic noise model, and a time-stamped command queue that enforces the fixed delay.

How they tested it

24 novices, four conditions, one delay

A within-subjects study where everyone drove a simulated rover to a target, as fast as they could, under each visualization and a delayed-video baseline.

WHO

24 participants

Novices in telerobotics (mean age 26). Gaming background varied but didn't affect performance.

DESIGN

4 conditions

Baseline, Network, Path, Envelope — order counterbalanced with a Williams Latin square; 96 trials total.

DELAY

2.56 s fixed

The theoretical minimum Earth–Moon round-trip. Held constant to isolate delay from network jitter.

MEASURES

Time, load, pauses

Completion time, NASA-TLX mental demand, and reactive "move-and-wait" pauses (gaps ≥ 2.56 s), plus rankings.

What they found

Only Path moved the needle

Compared to the delayed-video baseline, the trajectory projection made control faster, lighter, and far less reactive. Numbers compare Path to Baseline.

FASTER

0% shorter

Median task time dropped to 135 s with Path versus 210 s for Baseline — the only condition with a significant speedup.

LIGHTER

0% less load

NASA-TLX mental demand fell to 7.3 / 20 with Path, down from 13.2 — the lowest of all four conditions.

PREFERRED

0% ranked #1

19 of 24 operators ranked Path first for sense of control; it also led on helpfulness (75%) and lowest frustration.

PROACTIVE

Half the pauses

Path cut reactive "move-and-wait" pauses to a median of 1 per trial (Baseline: 5). Each extra pause added roughly 4 s.

NETWORK

No measurable gain

Knowing when commands land didn't help: Network matched Baseline on both time and cognitive load.

ENVELOPE

Lighter, but not better

The worst-case cone lowered perceived load, yet didn't reliably speed control — often too broad to steer by.

Design takeaways

Designing feedforward for delay Tap to expand.

Under the hood

The delayed control loop

Every input has to cross the gap and come back before the operator sees its effect. The same time-stamped queue that enforces the delay also feeds the lookahead overlays.

Honest limits

Where it stops short

L1

Fixed delay only

A constant 2.56 s isolates the cognitive effect cleanly, but real links jitter. How Path holds up under variable delay is left for future work.

L2

Static terrain

The world doesn't move during the delay here. With moving obstacles or shifting ground, reactive control fails harder — likely widening Path's advantage.

L3

Novices only

Experts build internal models tuned to a familiar delay, but prior work shows those break when conditions change — where an external reference should still help.

L4

Short-horizon measures

Time and workload capture immediate control, not longer-term trust, situational awareness, or attention over an extended mission.

In one breath

Show where it'll end up — not how the delay works

Visualizing uncertainty isn't enough on its own. Predictive cues only help when they match the decision the operator has to make: in spatial navigation that means projecting the committed trajectory the robot will follow. Timing (Network) and worst-case spread (Envelope) are accurate but force mental translation that competes for the very attention needed to plan. Externalize the future position, and operators shift from reactive move-and-wait to proactive control.