Skip to content

API reference

The import surface is flat: everything below is available from splatreg directly (or the named submodule). Optional-dependency features (localize_camera needs splatreg[render]) degrade to None at import time rather than breaking the package.

Registration

splatreg.api.register

register(target, source, *, residuals=None, init=None, transform='se3', backend='builtin', max_iters=None, quality='full', refine=None, refine_kwargs=None, seed_gate=False)

Register source onto target over a list of residuals, returning the 4x4 transform.

Parameters:

Name Type Description Default
target the reference (a :class:`~splatreg.core.types.Gaussians`, usually).
required
source what is aligned to ``target``, a ``Gaussians`` (splat-to-splat) or a ``Frame``

(tracking); handed straight to each residual.

required
residuals sequence of :class:`~splatreg.residuals.base.Residual`, or ``None`` (default) to

build an ICP-dominant default set [ICP(point_to_plane=False, weight=1.0), SDF(sigma=auto, weight=0.3, n_points=<quality>)], the SDF sigma auto-derived from the target geometry (~2x its median Gaussian scale; see :func:_auto_sdf_sigma) and the sample size / chunking from quality. An explicit list is honoured unchanged (quality then only sets the autodiff row-chunk). None requires target to be a non-empty Gaussians.

None
init initial 4x4 transform, or ``None`` for identity, or one of the strings:
  • "fast", the recommended fast coarse-init: FPFH descriptors + GPU-batched 3-point RANSAC (:func:splatreg.align_features.feature_align), returning ONLY the seed for the LM to polish. ~20-35 ms vs the ~0.8-1.4 s blind sweep, with the feature path's full-rotation robustness. Falls back to "global" then identity if the feature module is unavailable.
  • "global", coarse-init from :func:splatreg.align.global_align (super-Fibonacci SO(3) sweep + batched trimmed ICP; robust to noise/outliers and near-symmetric clouds; assumes full overlap).
  • "features", a complete partial-overlap registrar from :func:splatreg.align_features.feature_align (FPFH descriptors + ratio-test mutual-NN + clique-prefiltered RANSAC + an overlap-aware target->source ICP refine, with an overlap-aware point-to-plane super-Fibonacci basin-sweep fallback that recovers the pose from geometry alone on smooth splat surfaces where the descriptors are non-discriminative). Designed for the case where the two clouds see different parts of the same object. It also returns honest diagnostics (info['ambiguous'] / info['confidence'] / info['feature']): when a crop removes the rotation-disambiguating geometry the pose is genuinely unrecoverable and the result is flagged rather than silently wrong. Because the default residual set assumes FULL overlap (its ICP would pull a good partial-overlap init off-pose), init="features" with the default residuals returns the feature registration DIRECTLY and skips that LM; pass an explicit overlap-safe residuals=[...] to run the LM seeded from the feature init.
  • "robust", scale-robust registrar (:func:splatreg.align_features.robust_feature_align): an Open3D FPFH+RANSAC seed (scale-correct, auto-voxelled, the ~77 % RR Open3D reports on real indoor scans like 3DMatch) refined by splatreg's overlap-aware ICP (+ Sim(3) scale). Like "features" it returns the registration DIRECTLY under the default residuals.
  • "learned", LEARNED registrar (:func:splatreg.align_features.learned_feature_align): the pretrained GeoTransformer 3DMatch correspondence model (CVPR 2022, ~92 % RR, past the classical FPFH ~77 % ceiling) supplies the coarse seed, then the SAME overlap-aware ICP (+ Sim(3) scale) refine as "robust". Falls back to "robust" (then identity) when GeoTransformer's module / built CUDA-ext / pretrained weights are unavailable.
  • "bufferx", ZERO-SHOT registrar (:func:splatreg.align_features.bufferx_feature_align): BUFFER-X (ICCV 2025, "Towards Zero-Shot Point Cloud Registration in Diverse Scenes", MIT-SPARK/BUFFER-X) supplies the coarse seed from the two Gaussian-center clouds — a single generalist model that generalises across sensors/scales with NO per-dataset training — then the SAME overlap-aware ICP (+ Sim(3) scale) refine as "robust". Falls back to "robust" (then identity) when BUFFER-X's module / built CUDA-ext / pretrained weights are unavailable (see splatreg/third_party_models/README-BUFFERX.md).
  • "mac", MAC maximal-clique registrar (:func:splatreg.mac.mac_feature_align; Zhang et al., CVPR 2023): instead of RANSAC minimal samples, hypotheses come from the maximal cliques of an SC²-weighted rigidity-compatibility graph over the FPFH correspondences, each clique gets a weighted-SVD pose, the inlier-count winner is refit on its consensus set, then the same overlap-aware ICP polish as "robust". Robust to heavily outlier-contaminated correspondence sets (including multi-consensus decoys that defeat a greedy prefilter + RANSAC); on an all-outlier set it returns an honest info['success']=False / ambiguous=True identity instead of a silent wrong pose. Like the other registrars it returns the registration DIRECTLY under the default residuals. Needs networkx (pip install "splatreg[mac]").

All string forms are guarded, fall back to identity with a logged note if the module is unavailable. For init="global" the chosen transform seeds the LM.

None
transform ``"se3"`` (dof 6) or ``"sim3"`` (dof 7; the scale DoF is solved, autodiffed).
'se3'
backend the solver engine. ``"builtin"`` (DEFAULT, fastest) is splatreg's closed-form-Jacobian

Levenberg-Marquardt core. "pypose" / "theseus" hand the whole nonlinear least-squares problem to that external engine instead, they optimise the same right-perturbation tangent through splatreg's own exp (so the recovered SE(3)/Sim(3) pose matches the builtin convention) and autodiff the Jacobian, so a user can bring their own solver without writing one. Both need the matching optional dependency (pip install splatreg[pypose|theseus]) and raise a clear ImportError otherwise. "gtsam" is recognised but raises NotImplementedError (a factor-graph backend needs hand-written analytic factor Jacobians).

'builtin'
max_iters maximum LM iterations. ``None`` (default) takes the value from ``quality``

(QualityConfig.max_iters); an explicit int always wins.

None
quality the quality / machine-adaptivity policy (see :func:`splatreg.quality.resolve_quality`):

"full" (DEFAULT, nothing capped, all source anchors, full fidelity), "balanced" / "low" (bounded sample + tighter chunks), a 0..1 float (scaled), or "auto" (detect free GPU/CPU memory and pick the largest sizes that fit, full on a big machine, scaled down on a small GPU / CPU so it runs without OOM). A :class:~splatreg.quality.QualityConfig may be passed for full manual control. The Sim(3) autodiff Jacobian is always row-chunked (per quality) so peak memory is bounded with no quality loss.

'full'
refine optional OPT-IN second refinement stage run AFTER the geometric solve. The only value

today is "photometric", a short extra LM whose residual renders the SOURCE splat under the current T from a small synthetic camera ring around the target and compares it against renders of the TARGET splat from the same cameras (:func:splatreg.residuals.photometric.refine_photometric; PhotoReg-style, arXiv 2410.05044, adapted splat-vs-splat so NO real images are needed). Geometric alignment leaves a visibly coloured seam that pure geometry cannot see; this stage fixes the part of it that is pose error. Requires BOTH splats to carry colors and gsplat to be installed (pip install "splatreg[render]") unless refine_kwargs['render_fn'] supplies a custom renderer, checked at call time with a clear ImportError, never at import. Works for both transform="se3" and "sim3"; iteration count defaults to the quality policy's refine_iters.

None
refine_kwargs optional dict of keyword overrides for the refine stage (see

:func:splatreg.residuals.photometric.refine_photometric: n_views, width / height, radius / radius_mult, dssim_weight, jac_mode, max_iters, render_fn, sh_degree, ...). Ignored when refine is None.

None
seed_gate opt-in (default ``False``) Decision-PCR-style confidence gate for ``init="learned"``.

When on, the learned seed is scored (mutual-NN inlier ratio + SC² spatial consistency) BEFORE LM refinement; a low-confidence seed is rejected and reseeded from the classical robust path (keeping whichever scores higher) instead of blindly refining a bad hypothesis. No-op for other init modes. The scores land in the result info under seed_gate.

False

Returns:

Type Description
class:`~splatreg.core.types.RegisterResult` whose ``T`` is the full transform (the similarity
``[[s*R, t], [0, 1]]`` for ``transform="sim3"``), ``scale`` the recovered ``s`` (``1.0`` for
SE(3)), and ``info`` carries ``cost`` / ``n_iters`` / ``rmse`` (filled by
func:`~splatreg.solvers.lm.run_lm`) plus the resolved ``quality`` label.

splatreg.api.merge

merge(gaussians_list, ref=0, *, residuals=None, transform='sim3', init='global', dedupe=True, dedupe_method='voxel', voxel=None, knn_radius=None, max_iters=None, quality='full', refine=None, refine_kwargs=None)

Register every splat onto gaussians_list[ref], fuse, and return one Gaussians.

This is the v0.1 headline, a merge that is not a naive cat. For each non-reference splat it :func:register\ s onto the reference (init="global" so large offsets recover; default ICP-dominant residuals; transform="sim3" so scale differences are absorbed), bakes the recovered Sim(3)/SE(3) into that splat's means / quats / scales, concatenates everything, then dedupes the overlap: a voxel-grid pass (:func:splatreg.fuse.voxel_dedupe) keeps the highest-opacity Gaussian per occupied voxel, so the double-density seam collapses to single density. The reference passes through unregistered. The result drops straight into :func:splatreg.io.save_ply.

Parameters:

Name Type Description Default
gaussians_list the splats to merge.
required
ref index of the reference splat (all others register onto it). Default 0.
0
residuals residual list per :func:`register` (``None`` -> the ICP-dominant default set).
None
transform ``"sim3"`` (default; recovers scale) or ``"se3"`` for each pairwise registration.
'sim3'
init initial transform / ``"global"`` (default coarse basin-finder) / a 4x4, per

:func:register. "global" makes the merge robust to large inter-capture offsets.

'global'
dedupe when ``True`` (default) run the overlap dedupe after concatenation; when ``False`` the

merge is a registered concatenation (still aligned, but double-density seam).

True
dedupe_method ``"voxel"`` (default) for the voxel-grid pass (:func:`splatreg.fuse.voxel_dedupe`)

or "knn" for the cross-splat radius pass (:func:splatreg.fuse.knn_dedupe). KNN is translation-invariant, so it also removes the boundary-straddling duplicates a voxel grid leaves (~16% residual overlap on a registered seam) at the cost of an O(N) chunked nearest-neighbour scan. Ignored when dedupe=False.

'voxel'
voxel voxel-pass edge in splat units; ``None`` auto-derives it from the anchor spacing (see

:func:splatreg.fuse.auto_voxel_size). Used only when dedupe_method="voxel".

None
knn_radius KNN-pass suppression radius in splat units; ``None`` auto-derives it (half the

anchor spacing, :func:splatreg.fuse.auto_knn_radius). Used only when dedupe_method="knn".

None
max_iters LM iterations per pairwise registration. ``None`` (default) takes the value from

quality; an explicit int wins.

None
quality quality / machine-adaptivity policy applied to every pairwise registration, ``"full"``

(DEFAULT), "balanced" / "low", a 0..1 float, or "auto" (fit detected memory). See :func:register / :func:splatreg.quality.resolve_quality.

'full'
refine optional opt-in per-pair refinement stage, forwarded to :func:`register`. The merge

seam is exactly where refine="photometric" earns its keep: after the geometric solve a short photometric LM renders each moving splat against the reference from a synthetic camera ring and polishes the pose so the seam looks aligned, not just measures aligned (PhotoReg-style; needs colors on the splats + gsplat or a custom render_fn).

None
refine_kwargs keyword overrides for the refine stage, forwarded to :func:`register`.
None

Returns:

Name Type Description
Gaussians the fused splat. Raises ``ValueError`` on an empty list or out-of-range ``ref``.

splatreg.api.apply_transform

apply_transform(gaussians, T, scale=1.0)

Bake an SE(3)/Sim(3) transform into a splat and return the transformed copy.

The align-without-merging workflow: :func:register two scans, apply the recovered transform to the source, and save each scan as its own PLY, aligned into a common frame but never fused::

result = register(target, source, transform="sim3")
save_ply(apply_transform(source, result.T, result.scale), "source_aligned.ply")
# target.ply stays untouched; the two files now sit registered in any viewer.

Equivalent to the CLI: splatreg align target.ply source.ply -o source_aligned.ply. T is the 4x4 from :class:RegisterResult (top-left block scale * R); pass result.scale for Sim(3) (1.0 for SE(3)). Colour is handled fully: DC is rotation-invariant, quats are composed, and higher-order SH bands (f_rest) are Wigner-rotated in the real basis (:mod:splatreg.sh) so view-dependent colour turns with the splat.

splatreg.api.Tracker

Tracker(target, residuals, *, transform='se3', max_iters=None, init=None, quality='full')

Stateful pose tracker: fixed target + residuals, warm-started across frames.

Hold the reference splat and the residual list once, then call :meth:track per frame; each call seeds the LM from the previously estimated pose so small inter-frame motion converges in a few iterations.

Parameters:

Name Type Description Default
target the reference (a :class:`~splatreg.core.types.Gaussians`, usually).
required
residuals sequence of :class:`~splatreg.residuals.base.Residual`.
required
transform ``"se3"`` or ``"sim3"``.
'se3'
max_iters LM iterations per :meth:`track` call. ``None`` takes the value from ``quality``.
None
init optional initial 4x4 pose (identity when ``None``).
None
quality quality / machine-adaptivity policy (see

:func:splatreg.quality.resolve_quality). For the tracker the residual stack is fixed by the caller, so quality sets the Sim(3) autodiff row-chunk (peak-memory bound, no quality loss) and the default max_iters; it is resolved against the target's device/anchors at construction. Default "full".

'full'

pose property

pose

The most recent estimated 4x4 pose (None before the first :meth:track).

reset

reset(init=None)

Drop the warm-start so the next :meth:track starts from init (or identity).

track

track(source_or_frame)

Estimate the pose of a new source (Gaussians or Frame), warm-started.

Returns the :class:~splatreg.core.types.RegisterResult; the estimated T is retained as the warm-start for the next call.

MAC maximal-clique seed (init="mac")

splatreg.mac.mac_feature_align

mac_feature_align(target, source, *, transform='se3', correspondences=None, inlier_tol=None, refine_iters=30, **mac_kwargs)

MAC registrar over splats: FPFH correspondences -> :func:mac_pose -> overlap-aware polish.

Mirrors :func:splatreg.align_features.feature_align but replaces the clique-prefiltered RANSAC hypothesis stage with MAC maximal-clique hypothesis generation (module docstring). The recovered seed is polished by the same overlap-aware trimmed ICP the feature path uses and gated on the overlap residual, so a degenerate polish can never worsen the seed.

Args: target / source: reference / to-align splats (only .means read). transform: "se3" or "sim3" (median-ratio scale seed + residual refit; see :func:mac_pose). correspondences: optional pre-matched (src_pts (M, 3), tgt_pts (M, 3)) pairs, the injection seam for learned correspondences (e.g. GeoTransformer's) or tests; when None the FPFH ratio-test mutual-NN matcher of the feature path supplies them. inlier_tol: hypothesis-scoring inlier distance; None -> the feature path default. refine_iters: overlap-ICP polish iterations on the MAC seed. **mac_kwargs: forwarded to :func:mac_pose (gamma, the caps, ...).

Returns: (T_4x4, info), info carries n_matches / n_inliers / n_cliques / n_hypotheses / truncated / success plus the honest ambiguous / confidence the other registrars report. success=False (all-outlier / no consensus) returns identity with ambiguous=True and confidence=0.0, never a silent wrong pose.

splatreg.mac.mac_pose

mac_pose(src_corr, tgt_corr, *, with_scale=False, inlier_tol=0.02, gamma=None, max_corr=_MAC_MAX_CORR, degree_cap=_MAC_DEGREE_CAP, max_cliques=_MAC_MAX_CLIQUES, max_hyps=_MAC_MAX_HYPS, time_budget_s=_MAC_TIME_BUDGET_S, min_inliers=_MAC_MIN_INLIERS)

MAC pose from a putative correspondence set: SC^2 graph -> maximal cliques -> weighted SVD.

Args: src_corr / tgt_corr: (M, 3) matched point pairs (src_corr[i] <-> tgt_corr[i]). with_scale: Sim(3), median-ratio scale seed first, SE(3) MAC on the de-scaled set, residual scale re-fit on the consensus inliers (see module docstring). inlier_tol: 3-D inlier distance for hypothesis scoring (target units). gamma: rigidity tolerance for the compatibility graph; None -> _MAC_GAMMA_MULT * inlier_tol. max_corr / degree_cap / max_cliques / max_hyps / time_budget_s: worst-case caps (module docstring). min_inliers: consensus floor, a winning hypothesis below this is reported as an honest failure (chance-compatible cliques score 3-5 inliers even on all-outlier sets; the same gate the feature path's _FS_MIN_INLIERS applies).

Returns: dict with T (4x4, src -> tgt, source dtype/device; for Sim(3) the top-left block is s*R), success (bool, False means the correspondences carried no consistent consensus: T is identity, do NOT trust it), n_inliers, n_matches, n_cliques (enumerated), n_hypotheses (evaluated), scale (1.0 for SE(3)), truncated (a cap/time budget cut the clique enumeration).

Multi-splat bundle registration

splatreg.bundle.bundle_register

bundle_register(splats, ref=0, pairs='auto', *, transform='se3', init='global', register_kwargs=None, max_iters=30, damping=1e-06, convergence_tol=1e-09, robust='huber', robust_scale='auto', reject_threshold=0.1, fuse=False, return_info=False, dedupe=True)

Jointly register N splats into one loop-consistent frame via a pose-graph solve.

Builds a relative-pose constraint T_ij for every edge in pairs (each from :func:splatreg.register), then optimises all N absolute poses T_i so every edge T_i @ T_ij ≈ T_j holds simultaneously, Gauss-Newton in the SE(3)/Sim(3) tangent, with the reference pose pinned to identity to fix the gauge. Unlike the sequential merge (which chains the edges and accumulates drift around a loop) the joint optimum spreads the closure error over the whole graph, so the max pairwise inconsistency drops.

Parameters:

Name Type Description Default
splats the ``N`` splats to register together.
required
ref index of the reference splat, held at identity (the global frame). Default 0.
0
pairs ``"auto"`` (chain ``0-1-…-(N-1)`` + loop-closure ``(N-1, 0)``) or an explicit

[(i, j), ...] edge list. Each pair becomes one relative-pose constraint via register(splats[i], splats[j]) (j aligned into i's frame).

'auto'
transform ``"se3"`` (6-DoF edges/poses) or ``"sim3"`` (7-DoF, scale included).
'se3'
init Union[str, Tensor, None]

to "global" so large inter-capture offsets recover; register_kwargs can set residuals / quality / max_iters of the pairwise solves).

'global'
max_iters int

(a small Levenberg damping keeps the normal equations SPD; the solve stops when the pose update norm drops below convergence_tol).

30
robust per-edge robust kernel applied in the joint solve (IRLS). ``"huber"`` (default) or

"cauchy" down-weight an edge whose tangent residual exceeds robust_scale so a single wrong pairwise :func:register result (a bad edge) cannot corrupt the global poses; None recovers the plain least-squares solve. The edge weight is folded into the normal equations (H += w·JᵀJ, b -= w·Jᵀe) and recomputed every iteration from the current residual.

'huber'
robust_scale the kernel scale ``c`` (the residual norm at which an edge starts to be

down-weighted). "auto" (default) sets c = 1.4826·median(‖e_ij‖) (a robust MAD estimate of the inlier spread) each iteration, so it adapts to the loop's noise level; pass a float to pin it.

'auto'
reject_threshold an edge whose final IRLS weight is below this is reported in

info.rejected_edges (it was effectively excluded from the solution). Diagnostic only, the down-weighting itself is continuous, not a hard cut.

0.1
fuse also return one merged :class:`~splatreg.core.types.Gaussians`, every splat baked into

its recovered absolute pose, concatenated, and (dedupe) overlap-deduped, like :func:splatreg.merge but with the jointly optimised poses.

False
return_info also return a :class:`BundleResult` with the consistency diagnostics.
False
dedupe voxel-dedupe the fused splat when ``fuse=True`` (default ``True``).
True

Returns:

Type Description
``list[Tensor]`` of the ``N`` absolute 4x4 poses, by default. With ``fuse=True`` a
``(poses, fused)`` tuple; with ``return_info=True`` a trailing :class:`BundleResult` is appended.

splatreg.bundle.pairwise_consistency

pairwise_consistency(poses, rel, transform='se3')

Max and mean per-edge tangent inconsistency of an absolute-pose set against the measurements.

rel maps (i, j) -> T_ij. Returns (max||e_ij||, mean||e_ij||), the metric the joint solve drives down and the headline "loop closes" number. The tangent norm mixes rotation (radians) and translation (splat units); for the synthetic loop both are small so it is a fair single scalar, but callers comparing regimes should look at the components.

Object pose

splatreg.object_pose.estimate_object_pose

estimate_object_pose(model, observation, *, init='fast', transform='se3', residuals=None, backend='builtin', max_iters=None, quality='full')

Estimate the 6-DoF pose T_SO of a known model splat from an observation.

Parameters:

Name Type Description Default
model the canonical object splat (the "CAD model" of the splat world), in its own object frame.
required
observation the observed instance, a :class:`~splatreg.core.types.Gaussians` (an observed

splat / crop) or a :class:~splatreg.core.types.Frame carrying a point_cloud, in the scene / camera frame.

required
init coarse-init mode, per :func:`splatreg.register`. Default ``"fast"`` (FPFH + GPU-batched

RANSAC) finds the rotation basin for an arbitrarily-rotated object; pass a 4x4 to warm-start from a prior pose, or "global" for the blind super-Fibonacci sweep.

'fast'
transform ``"se3"`` (default, rigid 6-DoF, a real object does not change size) or ``"sim3"``

when the observation is at an unknown metric scale.

'se3'
residuals forwarded to :func:`splatreg.register`.
None
backend forwarded to :func:`splatreg.register`.
None
max_iters forwarded to :func:`splatreg.register`.
None
quality forwarded to :func:`splatreg.register`.
None

Returns:

Type Description
class:`ObjectPose` whose ``T_SO`` maps model points into the observation frame.
Notes

Internally this is register(target=observation, source=model), the observation is the target so the recovered transform moves the model onto the observation, i.e. exactly the object pose T_SO the caller wants. The honest occlusion / partial-view limit of register carries over: a heavily cropped observation can leave the pose ambiguous, surfaced as info['ambiguous'].

splatreg.object_pose.ObjectPoseEstimator

ObjectPoseEstimator(model, *, transform='se3', init='fast', track_iters=4, quality='full')

Stateful 6-DoF object-pose tracker for a fixed known model (the FoundationPose regime).

Build once with the canonical model splat, then call :meth:estimate per frame. The first frame pays the global/feature init to find the pose basin; every subsequent frame warm-starts from the previous pose via the fast :func:splatreg.track path (skip-global-init + truncated-SDF closed-form LM), so a tracked object updates in a few LM iterations rather than re-searching SO(3).

Parameters:

Name Type Description Default
model the canonical object splat (held fixed for the estimator's life).
required
transform ``"se3"`` (default) or ``"sim3"``.
'se3'
init the FIRST-frame coarse init mode (per :func:`estimate_object_pose`); later frames ignore it.
'fast'
track_iters LM iterations per warm-started frame (default from :func:`splatreg.track`).
4
quality quality / machine-adaptivity policy for the first-frame :func:`register`.
'full'

pose property

pose

The most recent estimated T_SO (None before the first :meth:estimate).

reset

reset()

Drop the warm-start so the next :meth:estimate re-runs the cold init.

estimate

estimate(observation)

Estimate T_SO for a new observation, warm-started after the first frame.

splatreg.object_pose.add_metric

add_metric(model, T_pred, T_gt)

Hinterstoisser ADD: mean model-point distance between the predicted and GT pose.

ADD = mean_i || T_pred · x_i − T_gt · x_i || over the model points x_i. The standard non-symmetric pose-error metric (asymmetric objects). Returned in the model's units (metres).

splatreg.object_pose.adds_metric

adds_metric(model, T_pred, T_gt, *, max_pts=4000)

ADD-S: symmetric (closest-point) variant of ADD for symmetric objects.

ADD-S = mean_i min_j || T_pred · x_i − T_gt · x_j ||, each transformed-by-prediction point is matched to its nearest GT-transformed model point, so a rotation about a symmetry axis (which maps the model onto itself) is not penalised. This is the metric YCB-Video / FoundationPose use for symmetric objects. Both sides are deterministically strided to max_pts to bound the pairwise distance. Returned in model units (metres).

splatreg.object_pose.add_auc

add_auc(errors, *, max_threshold=0.1, n_steps=1000)

Area-under-curve of the ADD/ADD-S accuracy-threshold recall (the YCB-Video AUC).

Sweeps a distance threshold d from 0 to max_threshold (default 0.1 m = 10 cm, the YCB-Video convention), at each d measures the fraction of poses whose error ≤ d, and returns the normalised area under that curve in [0, 1] (×100 gives the reported AUC). A higher AUC means more poses are accurate at a tighter threshold.

Camera localization

splatreg.camera_loc.localize_camera

localize_camera(splat, frame, init_T_WC, *, iters=150, lr=0.01, refold_every=40, mask_to_rendered=True, huber_k=None, sh_degree=None, coarse_kwargs=None)

Localize a query camera in a splat by differentiable-render pose optimization.

Refines init_T_WC so the gsplat render of splat from that camera matches frame.rgb, optimizing the camera→world pose through gsplat's differentiable rasteriser (exact render gradient). Returns a :class:~splatreg.core.types.RegisterResult whose T is the refined T_WC and whose info carries the loss history.

Parameters:

Name Type Description Default
splat the world-fixed Gaussian splat (must carry ``colors``).
required
frame the query observation, needs ``rgb`` and ``K`` (optional ``mask``).
required
init_T_WC ``(4, 4)`` initial camera→world prior, OR the string ``"coarse"`` to first run the

prior-free :func:coarse_localize_camera viewpoint sweep and refine its seed. Direct image alignment has a limited basin (a few degrees / a few percent of depth), so without "coarse" a wide-baseline query (no good prior) falls outside it, that is exactly what the coarse seed bridges. coarse_kwargs is forwarded to the sweep.

required
iters Adam steps on the pose tangent.
150
lr Adam learning rate on the 6-vector right-perturbation tangent.
0.01
refold_every every this many steps, fold the accumulated tangent into ``T_WC`` and reset it to

zero (keeps exp near the origin where it is best-conditioned).

40
mask_to_rendered when ``True``, the photometric loss is restricted to pixels the splat actually

renders (rendered depth > 0), so empty background does not dominate the loss.

True
huber_k optional Huber threshold on the per-pixel RGB residual (robust to occluders / outliers);

None uses a plain L2 loss.

None
sh_degree SH degree if ``splat.colors`` are SH coefficients; ``None`` treats them as RGB.
None
coarse_kwargs forwarded to :func:`coarse_localize_camera` when ``init_T_WC == "coarse"``

(e.g. n_az / n_el / grid / radius).

None

Returns:

Type Description
class:`~splatreg.core.types.RegisterResult` with the refined ``T_WC``; ``info['mode'] ==
'camera_loc'``, ``info['loss']`` (final), ``info['loss_history']``.

splatreg.camera_loc.coarse_localize_camera

coarse_localize_camera(splat, frame, *, candidates=None, n_az=12, n_el=5, radius=None, grid=24, dilate=2, return_score=False)

Coarse, prior-free camera-pose seed by a project-and-compare viewpoint sweep (CPU, no gsplat).

:func:localize_camera refines a pose only within the narrow basin of direct image alignment, so it needs a decent prior. This provides that prior when none exists (a wide-baseline relocalise): it scores a sphere of candidate camera poses by how well the splat's projected occupancy overlaps the query frame's foreground silhouette (from frame.mask, or a luminance threshold of frame.rgb), and returns the best-scoring pose. Pure pinhole projection, it runs on CPU and needs no rasteriser, so it is a coarse seed, deliberately cheap, not a final pose. Feed its result as init_T_WC to :func:localize_camera for the fine refine.

Parameters:

Name Type Description Default
splat the world-fixed splat (only ``means`` are used here).
required
frame query observation; needs ``K`` and a foreground cue (``mask`` preferred, else ``rgb``).
required
candidates explicit list of ``(4, 4)`` ``T_WC`` to score; ``None`` auto-builds a look-at sphere

(n_az azimuths × n_el elevations) around the splat centroid at radius.

None
n_az int

⇒ ~2.5× the splat's bounding radius, a typical object-framing distance).

12
grid occupancy-bitmap resolution the IoU score is computed at (coarse on purpose).
24
dilate binary-dilation radius (in grid cells) applied to both the projected and the query

occupancy before scoring, closes the holes a sparse point splat leaves so the IoU compares connected silhouettes (higher and far more viewpoint-discriminative). 0 disables it.

2
return_score also return the best IoU score (diagnostic).
False

Returns:

Type Description
The best ``(4, 4)`` ``T_WC`` (camera→world); with ``return_score=True`` a ``(T_WC, score)`` tuple.

Core types

splatreg.core.types.Gaussians dataclass

Gaussians(means, quats, scales, opacities, colors=None, log_scales=False)

A 3D Gaussian Splat, gsplat-compatible. All tensors share a device.

Convention: quats are wxyz; scales are linear unless log_scales is True; colors are either (N,3) RGB or (N,K,3) SH coefficients.

splatreg.core.types.Frame dataclass

Frame(rgb=None, depth=None, K=None, mask=None, point_cloud=None)

A single observation (for the camera/object tracking modes).

For splat-to-splat registration the 'source' is another Gaussians, not a Frame, residuals receive whichever is relevant via the source argument.

splatreg.core.types.RegisterResult dataclass

RegisterResult(T, scale=1.0, converged=False, info=dict())

Output of register / Tracker.track.

T is the 4x4 transform aligning source to target (rotation*scale | translation for Sim(3); plain SE(3) when transform='se3'). scale is the recovered similarity scale (already baked into T's top-left block; 1.0 for SE(3)). converged is the solver's convergence flag (step/cost tolerance reached before the iteration cap). info carries diagnostics (per-iter cost, rmse, overlap, n_iters, timings, residual breakdown, and for the feature initializers the honest info["ambiguous"] / info["confidence"] flags). Builtin-LM solves also fill info["information"] (the undamped JᵀWJ at the final accepted linearisation, (6, 6) SE(3) / (7, 7) Sim(3), tangent order [t, r, (log_s)]) and info["covariance"] (its inverse scaled by the residual-variance estimate; None if singular, never faked) for pose-graph / loop-closure weighting, see :func:splatreg.solvers.lm.run_lm.

PLY + gsplat I/O

splatreg.io.load_ply

load_ply(path, device=None, dtype=torch.float32)

Load a standard 3D Gaussian Splatting .ply into a :class:Gaussians.

Recognises the canonical INRIA/gsplat layout (x y z, f_dc_0..2, f_rest_*, opacity, scale_0..2, rot_0..3). Stored values are raw, the returned Gaussians has log_scales=True, raw (pre-sigmoid) opacities, wxyz quats, and colors as SH coefficients shaped (N, K, 3) (or RGB (N, 3) if only DC is present, i.e. K == 1, kept 2-D for convenience).

Args: path: Path to the .ply file. device: Target device for the tensors (default: CPU). dtype: Floating dtype for the tensors (default: float32).

Returns: Gaussians: with log_scales=True and SH colours.

Raises: FileNotFoundError: if path does not exist. ValueError: if the required 3DGS properties are missing.

splatreg.io.save_ply

save_ply(gaussians, path)

Write a :class:Gaussians to a standard 3D Gaussian Splatting binary .ply.

The output is the canonical INRIA/gsplat layout consumable by SuperSplat, the antimatter15 viewer, gsplat, nerfstudio, etc. Parameters are stored raw: scale_* are log-scales (the input is log-transformed if log_scales is False), opacity is the stored logit, and the colour is written as SH (f_dc + f_rest). RGB colours are encoded to a DC-only SH; an (N, K, 3) SH input is written with full f_rest.

Args: gaussians: The splat to serialise. path: Destination .ply path (parent dirs are created).

splatreg.io.from_gsplat

from_gsplat(means, quats, scales, opacities, colors=None, log_scales=False)

Wrap gsplat-style rasteriser tensors as a :class:Gaussians (no copy of valid inputs).

This is the entry point for "I already have my splat as gsplat tensors." It performs only light normalisation (shape/contiguity), leaving values untouched.

Args: means: (N, 3) centres. quats: (N, 4) rotations, wxyz (gsplat's convention). scales: (N, 3). Linear by default; pass log_scales=True if these are log-scales. opacities: (N,) or (N, 1). Whatever activation state your gsplat call expects, splatreg treats this as opaque and passes it back unchanged in :func:to_gsplat. colors: optional (N, 3) RGB or (N, K, 3) SH coefficients. log_scales: set True if scales are already log-transformed.

Returns: Gaussians.

splatreg.io.to_gsplat

to_gsplat(g)

Unpack a :class:Gaussians into the keyword bundle gsplat's rasteriser consumes.

Scales are returned linear (exp applied if the Gaussians holds log-scales), since gsplat.rasterization expects linear scales. opacities and colors are passed through unchanged. The result is intended for gsplat.rasterization(..., **to_gsplat(g))::

from gsplat import rasterization
out = rasterization(viewmats=..., Ks=..., width=W, height=H, **to_gsplat(g))

Args: g: The splat to unpack.

Returns: dict: {"means", "quats", "scales", "opacities", "colors"} (colors omitted if g.colors is None).

Spherical-harmonic rotation (Wigner-D)

splatreg.sh.rotate_sh

rotate_sh(colors, R)

Rotate an (N, K, 3) SH coefficient tensor by R (3DGS convention; see module doc).

colors is coefficient-major / channel-last, exactly what :class:Gaussians carries and :mod:splatreg.io round-trips. The DC row passes through unchanged; every higher-degree band is multiplied by its Wigner-D block, so the view-dependent lobes turn WITH the splat.

Args: colors: (N, K, 3) SH coefficients (K a perfect square). R: (3, 3) pure rotation.

Returns: (N, K, 3) rotated coefficients (new tensor; input untouched).

splatreg.sh.sh_rotation_matrix

sh_rotation_matrix(R, n_coeffs)

Block-diagonal real-SH Wigner-D matrix for rotation R, in the 3DGS basis convention.

Args: R: (3, 3) pure rotation (orthonormal; de-scale a similarity block first). n_coeffs: total SH coefficient count K including DC, must be a perfect square (degree+1)² (1, 4, 9, 16, ..., the only counts standard 3DGS PLYs produce).

Returns: (K, K) matrix D (device/dtype of R) such that f' = D @ f are the coefficients of the rotated colour function: Σ f'_k Y_k(d) = Σ f_k Y_k(R⁻¹ d) in the 3DGS/gsplat real-SH basis. The DC block is 1 (rotation-invariant).

Raises: ValueError: if R is not 3×3 or n_coeffs is not a perfect square ≥ 1.

Gaussian-SDF field

splatreg.geometry.gaussian_sdf.gaussian_sdf

gaussian_sdf(gaussians, points, *, sigma, normals=None, trunc_sigmas=None, use_opacity=False, knn=50, chunk_size=2048, index=None)

Sample the Gaussian-derived signed-distance proxy and its gradient at points.

See the module docstring for the proxy definition and assumptions.

Args: gaussians: the splat providing the anchors (means, opacities). Only means (and opacities when use_opacity) are read. points: (N, 3) query positions, in the splat's own frame. sigma: Gaussian kernel bandwidth (influence radius), same units as means. Required, there is no universal default. Must be > 0. normals: optional (M, 3) per-anchor normals. If None they are estimated once via :func:estimate_anchor_normals (k-NN PCA, outward-oriented). trunc_sigmas: if set, only the anchors within trunc_sigmas * sigma of each query (via a per-query top-k gather) contribute; None uses every anchor. A speed knob for very large splats; does not change the proxy otherwise. use_opacity: multiply each kernel weight by its anchor opacity. knn: neighbourhood size passed to the normal estimator (ignored if normals given). chunk_size: rows of points processed per block, bounding the (chunk, M) weight matrix's memory. index: optional prebuilt :class:splatreg.spatial_index.SpatialIndex over the target anchors. When supplied with trunc_sigmas the per-query k-nearest gather is served by the voxel-hash grid (near-O(N) candidate lookup) instead of the full (chunk, M) distance matrix, the EXACT same truncated proxy, only the neighbour search is pruned, so a scene-scale target field stays cheap. Ignored when trunc_sigmas is None (the full-field path reads every anchor by definition).

Returns: (sdf, grad) where sdf is (N,) signed distances (> 0 outside) and grad is (N, 3) unit surface normals n~(p) (the spatial gradient of the proxy). Both live on points' device and dtype.

Raises: ValueError: empty splat, mismatched normals, malformed points, or sigma<=0.

splatreg.geometry.gaussian_sdf.gaussian_sdf_grad

gaussian_sdf_grad(gaussians, points, *, sigma, normals=None, use_opacity=False, knn=50, chunk_size=2048, trunc_sigmas=None)

Signed distance AND its EXACT spatial gradient ∇_p d, computed in closed form.

Same proxy as :func:gaussian_sdf, but the second return is the TRUE field gradient ∇_p d analytically, so the SE(3) SDF-residual path needs neither an autograd graph nor a second forward (this is the fast path; :func:gaussian_sdf + autodiff is the truncated fallback). Derivation (u = p - q~, a_i = p - q_i, c_i = q_i - q~)::

∂q~/∂p = (1/σ²) Cov_w,   Cov_w = (1/W) Σ_i w_i c_i c_iᵀ   (weighted anchor covariance)
∇_p d  = n~ - (1/σ²) Cov_w n~ - (1/(σ²‖S_n‖)) Σ_i w_i (n_i·x) a_i,   x = u - d n~

EXACT (~1e-8 vs central-difference numerical) wherever the field has anchor support, the entire regime where the SDF is meaningful and where the registration residual operates (residual audit tests/test_jacobians.py 8/8). At degenerate far queries (the weight-sum clamp activates, the field itself is undefined) it falls back to the bounded surface normal n~ rather than the blown-up un-clamped expression. Returns (sdf (N,), grad (N, 3)) where grad is ∇_p d, NOT the surface normal n~.

Truncation (the tracking fast path)

trunc_sigmas (default None = every anchor) restricts each query to its k nearest anchors via a per-query top-k gather, then zeros the weight of any beyond trunc_sigmas*sigma, exactly the support :func:gaussian_sdf uses. Because all the closed-form sums above (q~, S_n, Cov_w n~, the normal-derivative term) are computed over the SAME truncated anchor set, the gradient stays the EXACT analytic field gradient of the truncated proxy with NO autodiff, so warm-start tracking with a tight sigma costs N*k not N*M. k is sized from knn (clamped to the cloud), the same convention as :func:gaussian_sdf.

Fusion / dedupe

splatreg.fuse.voxel_dedupe

voxel_dedupe(g, voxel=None)

Collapse near-coincident Gaussians to one-per-voxel, keeping the highest-opacity survivor.

Args: g: the (typically concatenated) splat to dedupe. voxel: grid edge length in the splat's units; None derives it via :func:auto_voxel_size (a small multiple of the median Gaussian scale).

Returns: Gaussians: a subset of g with at most one Gaussian per occupied voxel. Fields, colors (if present), log_scales, device and dtype are preserved. An empty or single-Gaussian input is returned unchanged.

splatreg.fuse.knn_dedupe

knn_dedupe(g, radius=None, *, use_index=False)

Cross-splat radius dedupe: keep the highest-opacity survivor within every radius ball.

The translation-invariant complement to :func:voxel_dedupe, it removes the residual overlap a voxel grid leaves at cell boundaries (see this section's note). Use it (e.g. via merge(..., dedupe_method="knn")) when the voxel pass under-dedupes a registered seam.

Args: g: the (typically concatenated) splat to dedupe. radius: suppression ball radius in the splat's units; None derives it via :func:auto_knn_radius (half the median anchor spacing). use_index: when True route the neighbour search through the voxel-hash :class:splatreg.spatial_index.SpatialIndex (near-O(N) on scene-scale splats) instead of the default O(N^2) chunked cdist scan. The survivor set is IDENTICAL either way, only the neighbour search is pruned. Default False (the original brute-force path).

Returns: Gaussians: a subset of g with no two survivors closer than radius to a higher-priority neighbour. Fields, colors, log_scales, device and dtype are preserved. An empty / single-Gaussian input is returned unchanged.

splatreg.fuse.auto_voxel_size

auto_voxel_size(g)

Derive a dedupe voxel edge from the splat's anchor SPACING (median nearest-neighbour distance) -- _VOXEL_SPACING_MULT x it.

NOTE: call this on a single, duplicate-free splat (e.g. the merge reference). Running it on an already-concatenated splat would read the overlap duplicates themselves as the spacing and under-dedupe. Falls back to the median scale, then a bbox fraction, for degenerate inputs.

splatreg.fuse.auto_knn_radius

auto_knn_radius(g)

Derive a cross-splat KNN-dedupe radius from the anchor SPACING -- _KNN_RADIUS_MULT x it.

Same spacing basis (and the same "call on the clean reference, not the merged splat" caveat) as :func:auto_voxel_size; only the multiplier differs. A radius of half the anchor spacing suppresses a duplicate that landed anywhere within half a lattice step of a kept anchor, including the sub-voxel-boundary pairs a grid snap leaves behind, while never reaching a genuine one-spacing-away neighbour. Falls back to the median scale, then a bbox fraction, when degenerate.

Spatial index

splatreg.spatial_index.build_index

build_index(g, cell=None)

Build a :class:SpatialIndex over a splat's means (or a raw (M, 3) tensor).

Convenience constructor: accepts a :class:~splatreg.core.types.Gaussians (indexes its means) or a point tensor directly. cell None auto-sizes to the median anchor spacing.

splatreg.spatial_index.SpatialIndex

SpatialIndex(points, cell=None)

A voxel-hash grid over a point set supporting radius / knn / region queries.

Build once from a (M, 3) point tensor (or a :class:~splatreg.core.types.Gaussians via :func:build_index), then query repeatedly. All queries return results IDENTICAL to a brute-force scan, the grid only prunes which anchors are distance-tested, never the answer.

Parameters:

Name Type Description Default
points ``(M, 3)`` positions to index (the anchor means).
required
cell voxel edge in the points' units. ``None`` auto-derives it from the median anchor

spacing, so a radius up to one cell needs only the 27-cell neighbourhood. Must be > 0.

None

radius

radius(queries, r)

All anchors within distance r of each query, in flat (query, anchor) pair form.

Returns (pair_query_idx, pair_anchor_idx), two (P,) long tensors so that points[pair_anchor_idx[j]] is within r of queries[pair_query_idx[j]]. This flat form composes directly with the chunked SDF / dedupe code (no ragged padding). The result is EXACT: every in-radius anchor is returned and no out-of-radius anchor is, identical to a brute-force cdist <= r, the grid only limits which anchors are distance-tested.

radius_batch

radius_batch(queries, r)

Loop-free batch radius query, identical result to :meth:radius, no Python per-query loop.

Enumerates every query's candidate anchors in one vectorised pass (:meth:_candidate_pairs), then does a single batched exact distance test over all (query, candidate) pairs at once. The returned flat (pair_query_idx, pair_anchor_idx) pair set is identical (as a set) to :meth:radius / a brute-force cdist <= r, only the which-anchors-tested pruning is shared. Wins over the looped path when there are many queries on a small/moderate cloud (the Python loop overhead dominates there).

knn_batch

knn_batch(queries, k)

Loop-free batch knn, same (Q, k) (idx, dist) result as :meth:knn, no per-query loop.

Picks one ring wide enough to contain the k-th neighbour for every query (grown until each query has ≥ k candidates and the ring margin clears the k-th distance), enumerates all candidates vectorised, then does a single padded scatter + batched top-k. Falls back to growing the shared ring; for very non-uniform densities the ring may be larger than the per-query :meth:knn would use, but the result is identical (exact top-k over a superset).

knn

knn(queries, k)

The k nearest anchors to each query.

Returns (idx, dist) of shape (Q, k) (idx into points, dist Euclidean), sorted nearest-first per query, EXACTLY the brute-force cdist-topk result. Expands the searched cell ring until at least k candidates are found AND the ring radius safely exceeds the k-th distance (so no closer anchor in an unsearched outer cell is missed), then does the exact top-k over the candidates. k is clamped to the anchor count.

region

region(lo, hi)

Indices of every anchor inside the axis-aligned box [lo, hi] (inclusive).

lo / hi are length-3 (tensor or sequence). Gathers the box's covered cells, then does the exact per-axis bound test on those candidates, the EXACT set a brute-force ((points >= lo) & (points <= hi)).all(-1) would return.

Quality policy

splatreg.quality.resolve_quality

resolve_quality(quality, device=None, *, target_anchors=None, source_anchors=None)

Turn a user quality request into a concrete, memory-fitted :class:QualityConfig.

Two stages: (1) pick the ACCURACY knobs (n_points / knn / max_iters) from the requested policy, then (2) ALWAYS fit the quality-neutral CHUNK knobs (jac_row_chunk / sdf_chunk_size) to the available memory via :func:_fit_chunks so the Sim(3) autodiff peak can never OOM, for every mode, full included. Chunking is numerically lossless, so stage 2 changes footprint, not the result.

Parameters:

Name Type Description Default
quality Union[str, float, QualityConfig, None]
  • None or "full" -> full ACCURACY (all source anchors). This is the default. The chunk knobs are still memory-fitted so full runs even on a small GPU.
  • "balanced" / "low" -> named accuracy presets (bounded n_points).
  • a float in [0, 1] -> scaled accuracy (1.0 == full, smaller == fewer points).
  • "auto" -> detect free GPU/CPU memory and pick the largest n_points that fits (full on a roomy machine; capped on a small one), chunks fitted on top.
  • a :class:~splatreg.quality.QualityConfig -> returned UNCHANGED (advanced manual override; no fitting, so you own the memory budget).
required
device the device the run will execute on (read for memory in ``"auto"`` and in the chunk

fit). Defaults to CUDA-current when available else CPU.

None
target_anchors the target splat's Gaussian count, if known, sharpens both the ``"auto"``

n_points budget and the chunk fit (peak ~ jac_row_chunk x min(n_points,sdf) x M).

None
source_anchors the source splat's Gaussian count, if known, sharpens the chunk fit for

full (n_points=None samples ALL source anchors, so the true sample is the source size).

None

Returns:

Name Type Description
QualityConfig the resolved, memory-fitted sizing for this run.

splatreg.quality.QualityConfig dataclass

QualityConfig(n_points=_FULL_N_POINTS, jac_row_chunk=_FULL_JAC_ROW_CHUNK, sdf_chunk_size=_FULL_SDF_CHUNK, knn=_FULL_KNN, max_iters=_FULL_MAX_ITERS, refine_iters=_FULL_REFINE_ITERS, label='full')

Resolved per-run sizing knobs (the output of :func:resolve_quality).

Attributes:

Name Type Description
n_points source-anchor sample size for the SDF/ICP residuals, or ``None`` for *all* anchors

(full quality). An explicit residuals=[...] overrides this; it only fills the default set.

jac_row_chunk row-chunk for the Sim(3) autodiff Jacobian (``jacrev(chunk_size=...)``). Bounds

peak autodiff memory with no effect on the result.

sdf_chunk_size per-query row block for :func:`splatreg.geometry.gaussian_sdf.gaussian_sdf`.
knn neighbourhood size for anchor-normal estimation.
max_iters default LM iteration count (an explicit ``max_iters=`` to ``register`` wins).
refine_iters default LM iteration count for the opt-in photometric refinement stage

(register(..., refine="photometric")); an explicit max_iters in refine_kwargs wins. Each iteration renders the splats from the camera ring, so this is the knob that sizes the refine's render budget.

label human-readable provenance (e.g. ``"full"``, ``"auto:cuda 6.0GiB-free -> 0.50"``).

Extension points

splatreg.residuals.base.Residual

Residual(weight=1.0, robust=None)

Bases: ABC

residual abstractmethod

residual(T, target, source)

Return r with shape (..., dim).

target is the reference splat; source is what is being aligned to it, a Gaussians (splat-to-splat registration) or a Frame (camera/object tracking), depending on the residual. Use self.requires() to declare which.

jacobian

jacobian(T, target, source)

Optional analytic Jacobian, shape (..., dim, dof) wrt the se(3)/sim(3) tangent.

Return None to let splatreg autodiff it (functorch jacrev/vmap).

dim abstractmethod

dim()

Residual output dimension (last axis of residual).

requires

requires()

Inputs this residual needs (e.g. {'depth', 'K'} or {'source_gaussians'}).

splatreg.solvers.base.Solver

Bases: ABC

solve abstractmethod

solve(problem)

Solve the (damped) normal equations (JᵀWJ + λD) δ = −JᵀW r and return the step.

splatreg.testing.assert_residual_jacobian

assert_residual_jacobian(residual, T, target, source, *, atol=0.0001, eps=1e-06, max_mismatch=0.02)

Assert residual.jacobian matches the numerical Jacobian to atol for at least (1 - max_mismatch) of rows (a small NN-switch boundary fraction is tolerated for correspondence residuals). Returns the max per-row error. Raises AssertionError.

Command line

splatreg.cli.main

main(argv=None)

splatreg.cli.build_parser

build_parser()