Best Paper Runner-Up Award · MobiCom 2026

TAP3DThermal-Assisted 3D Human Point Clouds

Xie Zhang Chengxiao Li Xuan Liu Chenshu Wu^*

The University of Hong Kong

* Corresponding author

▤ Paper ↗</> Code ↗▷ Video { } BibTeX

From body heat to 3D human geometry.

A single low-cost thermal array. Fully passive sensing. No RGB input.

Seven single- and multi-person scenes with thermal inputs, TAP3D reconstructions and ground-truth point clouds. — **Seeing people through heat.** TAP3D reconstructs human point clouds across different poses and multi-person scenes. Rows show the reference scene, thermal input, our output, and ground truth.

01 / OVERVIEW

Human sensing,
through a thermal lens.

Recovering 3D structure from low-resolution heat signatures with physics-informed learning.

Abstract

Human body point clouds are a versatile representation for AI-enabled human sensing. However, existing methods using LiDAR, radar, and depth cameras suffer from inherent drawbacks in high cost, sparse reconstruction, and privacy concerns, etc. In this paper, we exploit low-cost thermal arrays and present TAP3D, the first system to reconstruct 3D human point clouds from body heat signatures, offering significant advantages in cost, density, human sensitivity, and privacy. To overcome major challenges in depth estimation, thermal interference, and multi-person separation, we propose a novel physics-informed design, which integrates a forward thermal physics model with two distinct modules: multi-primitive estimation for self-supervised joint recovery of depth and other thermal properties, and geometric perspective fusion for suppressing interference and disentangling multiple people. We implement TAP3D using a single commodity thermal array sensor and build a large-scale dataset (160K samples, 8 environments, 11 users) for evaluation. TAP3D achieves remarkable accuracy for dense point cloud generation, enabling downstream tasks like fall detection (91.46%), indoor tracking (21.86 cm MAE), and human mesh recovery (4.87 cm error). By transforming body heat into point clouds for the first time, TAP3D pioneers a new paradigm for privacy-first, fully passive human sensing for many applications.

02 / IN ACTION

Watch TAP3D in action.

Explore the system in the project demonstration.

Video not loading? Watch on YouTube ↗

03 / THE METHOD

Physics-informed learning model

A forward thermal physics model connects body heat to geometry. Two complementary modules recover depth, suppress interference, and separate people.

TAP3D pipeline: multi-primitive estimation with a forward physical model, followed by optical-axis and bird’s-eye-view refinement. — The TAP3D architecture. Click the figure to explore it at full resolution.

Recover thermal primitives

Jointly estimate depth, emissivity, temperature, and reflection. The forward thermal model provides physics-based self-supervision.

Fuse geometric perspectives

Optical-axis-view refinement suppresses thermal interference. Bird’s-eye-view refinement disentangles multiple people through differentiable index mapping.

Reconstruct people in 3D

Fuse the refined predictions into per-person point clouds for downstream human sensing tasks.

04 / EVALUATION

Small sensor. Rich geometry.

Evaluated on more than 160,000 thermal samples, across eight indoor environments and 11 participants.

4.60 cmDirected Chamfer distance ↓

0.953User detection F1 score ↑

160K+Thermal samples

8 / 11Environments / participants

Comparison with baselines adapted to thermal array input
Method	DCD (cm) ↓	F1 score ↑	Parameters (M) ↓	FLOPs (G) ↓
RGB2Point	10.15 ± 0.77	0.945	176.07	16.96
NeWCRF	19.26 ± 1.50	0.677	88.46	78.14
TAP3D Ours	4.60 ± 0.25	0.953	23.32	7.41

Results reported in the paper. F1 measures user detection at an IoU threshold of 0.5.

Experimental setup and dataset

The prototype uses an 80 × 62 Meridian MI0802M6S thermal array. A co-located Intel RealSense D455 provides ground truth during data collection. Thermal and depth frames are recorded at 8 Hz and spatially aligned.

Thermal array and ground-truth depth camera setup, with the eight indoor evaluation environments.

05 / ABLATION STUDY

What does each component contribute?

Controlled ablations evaluate perspective refinement, multi-primitive estimation, and self-supervised pretraining. Hover to enlarge; click or tap for a detailed view.

Bird’s-eye-view refinement: quantitative results from the TAP3D paper. — Bird’s-eye-view refinement
Effect of BEV refinement under varying numbers of users. Removing it has a larger impact in crowded scenes with four to six people.

Optical-axis-view refinement: quantitative results from the TAP3D paper. — Optical-axis-view refinement
OAV refinement improves detection under thermal interference: F1 rises from 0.935 to 0.986 for cooking appliances and from 0.903 to 0.922 for refrigerators.

Multi-primitive estimation: quantitative results from the TAP3D paper. — Multi-primitive estimation
Compared with estimating depth alone without the forward physical model, multi-primitive estimation reduces DCD by 5.59% overall.

Self-supervised pretraining: quantitative results from the TAP3D paper. — Self-supervised pretraining
Physics-guided self-supervised pretraining reduces DCD by 11.93% on average across label budgets. Labeled fine-tuning is still required for human detection.

06 / APPLICATIONS

Point clouds with a purpose.

A shared 3D representation supports activity understanding, indoor tracking, and human mesh recovery.

INDOOR TRACKING

Follow movement in 3D.

21.86 cm mean tracking error across 144 one-minute recordings.

Human meshes recovered from reconstructed TAP3D point clouds.

HUMAN MESH RECOVERY

From points to body shape.

4.87 cm minimum matching distance for recovered human meshes.

FALL DETECTION

Recognize safety-critical motion.

91.46%Fall detection accuracy

07 / LIMITATIONS & FUTURE WORK

Where TAP3D still struggles.

TAP3D is most reliable indoors with moderate thermal interference. The following examples show where overlapping people, reflected heat, and strong ambient radiation challenge reconstruction.

Seven failure-case examples showing multi-user overlap, reflected body heat near a metal door, sun-heated windows, and an outdoor scene. Each column shows the RGB scene, thermal input, and predicted point cloud. — **Representative failure cases.** Rows: reference RGB scene, thermal input, and predicted point cloud. Columns: (1) severe multi-user overlap with nearby interference; (2) false positives from a reflective metal door; (3–6) increasingly unstable detection near sun-heated windows; (7) outdoor failure under strong ambient thermal radiation. Click to enlarge.

01 / OVERLAPPING PEOPLE

Connected thermal signatures

Severe overlap can leave people connected even after bird’s-eye-view refinement. Nearby heat sources can further obscure individuals and cause missed detections.

02 / REFLECTION & SOLAR HEATING

Unstable thermal contrast

Metal surfaces can reflect body heat and produce false positives. Sun-heated windows can overwhelm body signatures, causing reconstruction to degrade or fail across frames.

03 / OUTDOOR CONDITIONS

Strong ambient radiation

Sun-heated surroundings can dominate the temperature map, leaving insufficient human–background contrast for reliable detection.

Directions for future work

These are proposed directions, rather than capabilities demonstrated by the current system.

Improve sensing resolution and sensitivity. Higher-quality arrays could help distinguish overlapping people and weak thermal contrasts. Current point density is limited to one 3D point per thermal pixel.
Use temporal and body-structure constraints. Temporal consistency, mesh constraints, and kinematic priors could stabilize reconstruction in difficult scenes.
Combine sensors and modalities. Multiple aligned thermal sensors, or fusion with LiDAR, radar, and depth cameras, could improve coverage and reconstruction robustness.

09 / BIBTEX

Cite TAP3D.

@inproceedings{zhang2026tap3d,
  author    = {Xie Zhang and Chengxiao Li and Xuan Liu and Chenshu Wu},
  title     = {{TAP3D}: Thermal-Assisted 3D Human Point Clouds},
  booktitle = {The 32nd Annual International Conference on Mobile Computing and Networking},
  year      = {2026},
  doi       = {10.1145/3795866.3844491}
}