TISA:

Terrain- and Intent-Aware Shared Autonomy for Humanoid Teleoperation

Yintianrun Zhang*,1,2, Bingrui Zhao*,1,2, Yuxin Li1,2, Hongyan Dong1,
Haohuan Yuan1, Dingchen Zeng1, Xueyun Wang1, Ming Wang1, Hao Xu†,2

*: equal contribution, †: corresponding author

1Beihang University

2School of Intelligent Science and Technology, Nanjing University

TISA preserves direct whole-body teleoperation when it is predicted safe, then delegates only the executable portion of the operator's intent to a terrain-aware executor near risky contacts.

Release status. This project page is an early preview while the manuscript and public release package are being finalized. The paper, code, data, and trained models are under preparation and will be released after the review process. Demos and quantitative results may be updated in the public version.

Abstract.

Humanoid teleoperation over uneven terrain must balance rich operator authority with contact safety. Direct whole-body teleoperation preserves posture, arm motion, and task-specific body intent, but it can fail when the commanded motion is incompatible with local terrain. Perceptive hiking controllers traverse such terrain more robustly, yet they typically expose only compact locomotion intent and are therefore not replacements for teleoperation.

We present TISA, a policy-agnostic supervisory layer that frames assistance as intent-space authority allocation. TISA executes direct whole-body teleoperation when the proposed action is predicted safe, and delegates only the executable portion of the operator's intent to a reduced-authority terrain executor when risk rises. Its key learned component is an intent- and action-conditioned risk critic that estimates short-horizon failure probability from the current observation, projected intent, and candidate direct action, rather than classifying terrain difficulty alone. Instantiated with a perceptive hiking executor, TISA reduces failures relative to direct tracking while preserving direct authority on low-risk segments; a rich-intent variant further preserves selected upper-body references during assisted traversal.

Approach.

TISA framework with data engine, dual-executor training, risk critic, and authority allocator
Framework overview. The data engine constructs terrain-conditioned references and risk labels. A direct teleoperation policy and a perceptive hiking executor are trained separately, while a learned risk critic and hysteretic allocator decide whether to preserve direct authority or switch to intent-level terrain execution.

Intent-Space Allocation

TISA does not replace teleoperation with hiking. It reduces authority only to the richest intent space supported by the autonomy executor.

Action-Conditioned Risk

The critic estimates short-horizon failure probability from observation, projected intent, and the candidate direct action.

Hysteretic Switching

The allocator uses separate enter and exit thresholds, avoiding rapid mode changes while returning authority after risk subsides.

Experiments.

Across stairs, obstacles, height discontinuities, and cluttered transitions, TISA improves safety without collapsing the operator interface into pure hiking. In the paper's evaluation, pure tracking falls in 54.0% of episodes, while TISA reduces falls to 2.3%, executes 75.2% of low-risk steps in DIRECT mode, and routes 99.6% of risky steps to INTENTAUTO.

2.3%

fall rate with TISA

75.2%

low-risk steps kept direct

99.6%

risky steps assisted

1.44

switches per episode

Qualitative safety comparison across terrain challenges
Qualitative safety comparison. TISA routes risky terrain segments through the perceptive executor, while direct teleoperation keeps tracking through stairs, obstacles, and height discontinuities.
Intent following across terrain with green reference overlays
Intent following across terrain. Green overlays indicate commanded intent or reference motion; TISA changes execution authority only when unsafe contacts are predicted.
Allocation ablation bar chart
Allocation ablation. Learned-risk allocation retains high success compared with risk-only, terrain-only, and command-only triggers.
Risk trace plot showing mode switching over time
Risk trace. The same terrain can produce different authority profiles depending on operator intent.

Real-World Demo.

All videos show humanoid teleoperation with terrain-aware authority allocation.

Terrain Safety

Risky height transitions trigger assistance only around unsafe contacts.

Intent Recovery

Direct control returns after the contact transition has been cleared.

Box Interaction

The system preserves task-specific body intent while changing execution authority.

Clutter Traversal

Assistance is localized to cluttered and uneven contact regions.

Real-Robot Walking

TISA keeps operator authority visible throughout deployment.

Montage of real robot TISA deployments
Deployment montage. TISA operates across stairs, boxes, low-posture motion, and cluttered traversal while returning direct authority after risky contacts.

Citation.

Citation information will be added after the paper is publicly released.