Apple's split-and-scale method increases the detail of 3D latents at inference without retraining

  • PublicPublic
  • 18 Nov, 2026
Apple's split-and-scale method increases the detail of 3D latents at inference without retraining

Apple machine learning researchers have introduced split-and-scale, a technique for increasing the capacity of latent 3D representations at inference time while keeping training costs manageable. Latent representations already encode 3D scenes compactly and underpin powerful 3D generative models. However, they struggle to preserve the fine details of complex objects because the latents can only hold so much information.

The obvious fix has serious drawbacks. Increasing latent capacity during training is computationally expensive and often impossible within memory limits. Adding more latents at inference does not help either, because the model fails to make effective use of the extra capacity.

The method relies on one key property. When a 3D scene is represented as a continuous function of 3D coordinates, a single pretrained tokenizer can be applied at inference to arbitrarily small sub-regions of that scene. Each sub-region receives the same latent capacity as a whole scene would, so the same budget is spent on a smaller spatial extent. As a result, the model captures far more detail, and no retraining is needed.

The team ran extensive ablations to measure the trade-offs between compute, memory, latent capacity, and reconstruction quality. The results show that this inference-time scaling reaches quality on par with much larger models that exceed the memory of a single GPU. The approach also works for generation: an autoregressive 3D generator trained on split-and-scale representations performs competitively with state-of-the-art baselines. This suggests a practical way to generate more detailed 3D assets on modest hardware.