0.117
Penetration ↓
Lowest self-penetration among all compared methods, a ~16–17% relative reduction over naive copy / HumanIK.
Computer Animation · Motion Retargeting
ReFM
Retargeting with Energy Flow and Motion Semantics
A source-mesh-agnostic framework that produces semantically faithful, penetration-aware, temporally smooth character motion.
Video results
Side-by-side qualitative comparison
Source motion alongside HumanIK, MeshRet, SAN, STaR, and ReFM-HumanIK across several retargeted clips.
Overview
Retargeting should refine, not regress.
ReFM reformulates motion retargeting as a progressive refinement process inspired by how animators work: starting from an initial retargeted motion, it iteratively corrects target-specific artifacts rather than directly regressing a final motion in one shot. Throughout refinement, preserving the semantic intent of the source motion remains the primary objective. A cross-character semantic encoder provides explicit semantic guidance, while target-aware physical and temporal constraints reduce self-penetration and motion artifacts with minimal modification to the initialization. ReFM requires only the source motion and skeleton together with the target skeleton and mesh, without requiring a source mesh or ground-truth target motion.
Contributions
Built for practical character transfer.
01
Source-mesh agnostic
Inference needs source motion and skeleton plus the target skeleton and mesh—without source geometry or target ground truth.
02
Semantics-aware
A frozen contrastive graph-transformer encoder for preserving character-invariant motion semantics.
03
Energy-based Refinement
Rather than generating from a one-step regression, ReFM refines a coarse retargeted motion progressively under the energy-guided trajectory, pushing the data to the low-energy domain.
Method
From a motion copy to a physically cleaner transfer
Three complementary stages deliver geometry-aware motion while retaining its original intent.
01
Motion Canonicalizer
Analytically removes global yaw to normalize input motion without learned parameters.
02
Semantic Motion Encoder
Maps motion to a 128-dimensional, character-invariant representation using contrastive learning.
03
Energy-Guided Flow
Refines the copy retarget by descending penetration, semantic, and temporal energies.
Evaluation
Leading results on the 222-pair Mixamo test set
Unified 65-joint evaluation protocol; Pen and Semsim are the primary metrics, Curv and MSE are auxiliary.
0.998
Semantic similarity ↑
Best semantic similarity for ReFM-Copy, tied with STaR and R2ET, among learning-based methods.
2.13
User-study avg. rank ↓
ReFM-HumanIK ranks best overall across all three criteria in a 20-participant user study.
| Method | Pen ↓ | Semsim ↑ | Curv ↓ | MSE ↓ |
|---|---|---|---|---|
| Reference motions | ||||
| Ground truth | 0.141 | — | 0.654 | — |
| Naive copy baseline | 0.140 | 0.999 | 0.657 | 0.051 |
| HumanIK | 0.141 | 0.999 | 0.657 | 0.049 |
| Skinned retargeting methods | ||||
| STaR | 0.137 | 0.998 | 0.702 | 0.074 |
| MeshRet | 0.135 | 0.977 | 0.702 | 0.102 |
| Skin-agnostic retargeting methods | ||||
| SAN | 0.137 | 0.955 | 1.497 | 0.161 |
| R2ET | 0.136 | 0.998 | 0.854 | 0.082 |
| Ours: skin-agnostic refinement | ||||
| ReFM-copy | 0.117 | 0.998 | 0.772 | 0.086 |
| ReFM-HumanIK | 0.117 | 0.996 | 0.772 | 0.091 |
Pen: self-penetration; Semsim: semantic similarity; Curv: temporal curvature; MSE: motion reconstruction error. Lower is better for Pen, Curv, and MSE; higher is better for Semsim. Pen and Semsim are the primary metrics for physical plausibility and semantic preservation, respectively; Curv and MSE are auxiliary measures with known limitations and increase for ReFM relative to its initializations, reflecting a trade-off between target-specific geometric correction and low-level motion correspondence. Semsim is not reported for Ground truth, since it is an independently recorded motion rather than a copy of the source rotations. Naive copy and HumanIK largely preserve source-motion semantics by construction (Semsim ≈ 0.999) but may still violate other constraints such as physical plausibility.
Training data
A shared 65-joint motion space
113characters across synthetic and real scans
11,973motion clips for pretraining and energy-flow learning
65joints: body, fingers, and root
222Mixamo GT test pairs (11 characters, 91 motions)
Qualitative results
Visual comparisons across characters and datasets
Source, ground truth, baseline methods, and ReFM refinements, on Mixamo (Body Jab Cross, Dribble) and ScanRet.
1 / 11
Citation
ReFM citation coming soon
A ReFM paper and public project links will be added when available.
Coming soon.