Scripting API¶

Caliscope's calibration pipeline is available as Python functions. You can run intrinsic and extrinsic calibration from a script without the GUI.

The standard install includes the API:

uv pip install caliscope        # calibration library + scripting API
uv pip install caliscope[gui]   # adds desktop app, 3D visualization, and pose tracking

Everything below uses the caliscope.api module. A complete working script is in scripts/demo_api.py. For a complete markerless calibration recipe using a person and RTMPose, see Markerless Calibration.

For an alternate workflow, scripts/demo/demo_shared_video_calibration.py calibrates both intrinsics and extrinsics from one synchronized ChArUco recording. The script detects the board once, uses each camera's observations for intrinsic calibration, then uses the combined observations for extrinsic calibration. The recording must include varied board positions and tilts in every camera as well as simultaneous views across cameras.

Imports¶

from pathlib import Path
from caliscope.api import (
    CameraDistance, Charuco, CharucoTracker, CameraArray, CaptureVolume, ConstraintSet,
    extract_image_points, extract_image_points_multicam,
    calibrate_intrinsics, calibrate_extrinsics, estimate_vertical,
)
from caliscope.reporting import (
    print_intrinsic_report, print_extrinsic_report, print_camera_pair_coverage,
)

caliscope.api re-exports the domain classes and calibration functions. caliscope.reporting provides Rich terminal output for quality inspection.

Step 1: Define the calibration target¶

Create a Charuco board matching the physical board you recorded:

charuco = Charuco.from_squares(columns=4, rows=5, square_size_cm=3.0)
tracker = CharucoTracker(charuco)

square_size_cm sets the scale of all calibrated coordinates. A 3.0 cm square produces corners spaced 0.03 m apart in object space. Measure your printed board and use the actual size.

For a two-sided board on a real substrate, pass the measured thickness_cm so back-face detections are modeled at their true depth. See Two-Sided Boards and Thickness for mounting and coverage requirements:

charuco = Charuco.from_squares(columns=4, rows=5, square_size_cm=3.0, thickness_cm=0.6)

Printing the board¶

The GUI's board panel also collects a board width, height, and unit. Those dimensions never reach the calibration. Both the GUI and from_squares pin the geometry to the measured square size, so the width and height set only the aspect ratio of the printed image. A board built by from_squares and the same board configured in the GUI produce identical corner positions.

Use the full constructor when you want a printable board sized to your paper:

charuco = Charuco(
    columns=4, rows=5,
    board_width=8.5, board_height=11.0, units="inch",
    square_size_override_cm=5.4,
)
charuco.save_image("charuco.png")
charuco.save_mirror_image("charuco_mirror.png")

Print it, measure a square with calipers, and pass the measurement as square_size_override_cm. Printers rescale. The mirror image is the back face of a two-sided board.

Step 2: Build a camera array from video metadata¶

intrinsic_videos = {
    0: Path("intrinsic/cam_0.mp4"),
    1: Path("intrinsic/cam_1.mp4"),
    2: Path("intrinsic/cam_2.mp4"),
    3: Path("intrinsic/cam_3.mp4"),
}
cameras = CameraArray.from_video_metadata(intrinsic_videos)

This reads resolution and frame rate from each video and creates uncalibrated CameraData entries. The dictionary keys are camera IDs, which must stay consistent across intrinsic and extrinsic videos.

Step 3: Intrinsic calibration¶

For each camera, extract charuco corners from the intrinsic video, then solve for the camera matrix and distortion coefficients:

for cam_id, video in intrinsic_videos.items():
    points = extract_image_points(video, cam_id, tracker, frame_step=5)
    cal = calibrate_intrinsics(points, cameras[cam_id])
    cameras[cam_id] = cal.camera

    print_intrinsic_report(cal)

frame_step=5 processes every 5^th frame. Intrinsic calibration needs roughly 30 diverse frames, so skipping frames saves time without sacrificing quality.

extract_image_points shows a Rich progress bar by default. Pass progress=None to suppress it.

calibrate_intrinsics returns an IntrinsicCalibrationOutput containing the calibrated CameraData and an IntrinsicCalibrationReport with RMSE and coverage metrics. print_intrinsic_report prints these with color-coded quality badges.

This whole step is optional. The extrinsic pipeline can recover intrinsics during bundle adjustment; see Calibrating without intrinsics below. This path is experimental.

Step 4: Extract time-aligned points for extrinsic calibration¶

extrinsic_videos = {
    0: Path("extrinsic/cam_0.mp4"),
    1: Path("extrinsic/cam_1.mp4"),
    2: Path("extrinsic/cam_2.mp4"),
    3: Path("extrinsic/cam_3.mp4"),
}
ext_points = extract_image_points_multicam(extrinsic_videos, tracker, frame_step=3)

This function reads all videos concurrently (one thread per camera), aligns frames by timestamp, and runs the tracker on each time-aligned moment. The result is a single ImagePoints DataFrame with observations from all cameras.

If your cameras were not hardware-synchronized, pass a timestamps file:

ext_points = extract_image_points_multicam(extrinsic_videos, tracker, timestamps="timestamps.csv", frame_step=3)

frame_step works on time-aligned moments, not raw frames. frame_step=10 processes every 10^th synchronized moment.

If a camera recorded sideways, give it a quarter-turn count, the same value the GUI's rotate buttons set:

ext_points = extract_image_points_multicam(extrinsic_videos, tracker, rotation_counts={2: 1, 3: -1})

Rotation decides only what the tracker sees. Detected points come back in original image coordinates either way, so calibration results do not change. Charuco, ArUco, and chessboard detection is already rotation-invariant and ignores the setting. ONNX pose trackers need it because they were trained on upright people.

Check camera pair coverage¶

Before calibrating, verify that camera pairs share enough observations:

print_camera_pair_coverage(ext_points)

This prints a lower-triangle grid of shared observation counts and flags structural problems (e.g., a camera pair with zero overlap).

Step 5: Calibrate¶

calibrate_extrinsics runs the same pipeline as the GUI's Calibrate button: bootstrap camera poses from pairwise PnP, robust bundle adjustment, outlier filtering, and a final re-optimization.

constraints = ConstraintSet.from_charuco(charuco)
run = calibrate_extrinsics(ext_points, cameras, constraints)
volume = run.capture_volume

The constraints feed the board's known corner geometry into bundle adjustment as rigidity information, which stabilizes the solution and locks world scale. Build them from the same board definition you used for tracking.

Two keyword arguments matter most:

refine_intrinsics (default True): re-estimate each camera's focal length and leading distortion jointly with the poses. Refinement is subject to the depth-ratio gate: if any camera saw the target over too narrow a depth range, refinement is disabled for the whole rig.
filter_percentile (default 2.5): the worst percentage of observations removed per camera between the two optimization passes.

The returned CalibrationRun carries the calibrated volume plus diagnostics:

run.capture_volume              # the calibrated CaptureVolume
run.intrinsic_refinement_gated  # True if refinement was requested but gated off
run.synthesized_cam_ids         # cameras whose intrinsics started from a blind guess
run.intrinsic_estimates         # initial vs recovered f, k1, k2 per camera
run.dropped_static_markers      # static markers excluded for moving (ArUco path)

Per-camera near/far depth ratios are available via compute_depth_ratios(run.capture_volume) (from caliscope.core.scale_accuracy import compute_depth_ratios).

Check intrinsic_refinement_gated when you expected refinement: a True here with poor results usually means the target was not moved toward and away from the cameras enough.

calibrate_extrinsics accepts a progress callback (progress=lambda pct, msg: print(pct, msg)); it runs silently by default.

Manual pipeline control¶

The one-call pipeline is built from pieces you can also drive yourself:

volume = CaptureVolume.bootstrap(ext_points, cameras, constraints=constraints)
volume = volume.optimize(strict=False)
volume = volume.filter_by_percentile_error(2.5)
volume = volume.optimize(strict=False)

bootstrap estimates initial camera positions from pairwise PnP and triangulates 3D points. optimize runs bundle adjustment. Both return new CaptureVolume instances (the originals are unchanged). Reach for this when you want a nonstandard sequence, for example extra filter passes or intermediate inspection. Otherwise prefer calibrate_extrinsics, which adds the robust loss, the depth-ratio gate, and the static-marker guard.

Step 6: Align to object coordinates¶

sync_idx = volume.unique_sync_indices[len(volume.unique_sync_indices) // 2]
volume = volume.align_to_object(sync_idx)

This applies a similarity transform (Umeyama algorithm) that aligns the world coordinate frame to the board's position at the chosen sync index. Pick a frame where the board is at your desired origin.

Rotate the axes¶

The board rarely lands with its axes where your lab convention wants them. rotate turns the whole volume, cameras and points together. It is what the GUI's X, Y, and Z buttons call:

volume = volume.rotate("x", 90)
volume = volume.rotate("z", -90)

Positive angles follow the right-hand rule: counter-clockwise looking down the positive axis toward the origin. The GUI buttons are fixed at 90 degrees, but any angle works here.

Shift the origin¶

translate moves the whole volume by a fixed offset in meters:

volume = volume.translate(z=0.02)

Shape and scale do not change, only where the rig sits in world coordinates. Positive z raises everything. This is the primitive grounded and centered are built on, exposed for the shifts they do not cover.

Orienting to gravity without a board¶

A markerless calibration has no board to align to, so the orientation comes from the scene instead. Estimate the vertical direction from the extrinsic calibration videos before you orient the volume:

vertical = estimate_vertical(extrinsic_videos, cameras)
volume = volume.oriented(up=vertical.up_per_cam)   # +Z becomes vertical
volume = volume.scaled(CameraDistance(cam_a=0, cam_b=3, meters=4.2))
volume = volume.grounded(lowest_point_height_m=0.02)
volume = volume.centered()                          # XY origin at the camera centroid

Set the vertical direction and metric scale before grounding. You can call oriented and scaled in either order. grounded assumes Z is vertical and its height offset is in meters. centered changes only XY, so it can follow grounding without changing floor height. oriented takes a per-camera up vector, which estimate_vertical produces from the video.

Set metric scale¶

Use a measured camera baseline when you have one:

volume = volume.scaled(CameraDistance(cam_a=0, cam_b=3, meters=4.2))

Otherwise, a Halpe26 model card can create a stature cue from a measured stature. Load the same card used for pose tracking, then pass its cue as the only scale source:

from importlib.resources import files

from caliscope import MODELS_DIR
from caliscope.trackers.model_card import ModelCard

card_path = Path(str(files("caliscope.trackers.model_cards"))) / "rtmpose_l_halpe26.toml"
card = ModelCard.from_toml(card_path, models_dir=MODELS_DIR)
volume = volume.scaled(card.stature_cue(meters=1.78))

Select the primary person before pose estimation, then use the resulting reconstruction for the cue. Do not combine a stature cue with a camera-distance cue in this workflow.

For each complete frame, the stature cue sums heel-to-ankle, ankle-to-knee, knee-to-hip, and hip-to-shoulder distances on each side. It averages the two side totals, adds the distance from the shoulder midpoint to the head, and takes the median across frames. It applies no posture filter or correction factor. Frames missing any required point are skipped. If no complete frames remain, the cue raises ValueError. stature_cue also raises ValueError for model cards without a stature chain or whose chain names points absent from the card.

Aligning to the floor without a board¶

You may have one point that should align with the ground plane, such as a marker on the foot. The grounded method will shift the cameras and tracked subject such that the lowest point intersects with the ground plane. Supply lowest_point_height_m yourself when that point has a known height above the floor. Model cards do not provide a floor-height default.

volume = volume.grounded()

It may be the case that the lowest tracked point is not aligned with the floor (tracked foot points may for example float a centimeter or two above the floor). In this case, you can add an additional vertical shift to pad out the lowest point:

For example, a pose tracker might place its lowest point 2 cm above the bottom of a shoe:

volume = volume.grounded(lowest_point_height_m=0.02)

The above places the lowest point 2 cm above the ground plane.

Sometimes one bad 3D point appears far below the person. The default setting would use that point and place the rest of the reconstruction too high. Use this mode to ignore the extreme points.

volume = volume.grounded(mode="pooled_1st_percentile")

Use this option only when you can see bad points below the real floor.

Step 7: Inspect results¶

print_extrinsic_report(volume)

The report shows optimization status, reprojection error percentiles, per-camera breakdowns, and scale accuracy metrics (if origin was set).

Step 8: Save¶

volume.save("capture_volume")

This writes camera_array.toml, image_points.csv, and world_points.csv to the directory, plus constraints.toml when the volume carries constraints. To reload later:

volume = CaptureVolume.load("capture_volume")
cameras = CameraArray.from_toml("capture_volume/camera_array.toml")

Aniposelib export¶

Pose2Sim, anipose, and other aniposelib-compatible tools want their own TOML layout. The GUI writes one to the workspace root every time it saves. A script asks for it:

volume.camera_array.to_aniposelib_toml("camera_array_aniposelib.toml")

Only posed cameras are written, with rotations as Rodrigues vectors. save leaves this out on purpose. Its directory stays a self-contained snapshot of the calibration, not a mix of snapshot and handoff files.

Track a primary person before pose estimation¶

For human recordings, you can select one person's box track before estimating keypoints:

from caliscope.api import BoxTracks, estimate_keypoints, track_people
from caliscope.recording.synchronized_timestamps import SynchronizedTimestamps

timestamps = SynchronizedTimestamps.from_anchor_frames(videos, {cam_id: 0 for cam_id in videos})
box_tracks = track_people(videos, pose_tracker.detector, timestamps=timestamps, frame_step=4)
subject_ids = box_tracks.primary_track_ids()
subject_boxes = box_tracks.filter_to_tracks(subject_ids)
usable_boxes = subject_boxes.exclude_edge_boxes(margin_px=24)
image_points = estimate_keypoints(
    videos, pose_tracker, usable_boxes, min_confidence=pose_tracker.card.confidence_threshold
)

BoxTracks stores observed boxes, camera-local track IDs, source frame indices, image dimensions and inference rotations. The supplied SynchronizedTimestamps object determines sync_index and frame_time. frame_index remains the decoded source-frame index. With from_anchor_frames, a frame's time is (frame_index - anchor_frame) / FPS for its camera. Frames before the anchor have negative times. Choose anchors known to show the same instant across cameras. This workflow does not infer timing offsets or normalize video durations. primary_track_ids chooses the track whose boxes cover the most image area in each camera, counting each pixel once, and returns a camera-to-track-ID mapping. Filtering returns a new container, leaving the original available for inspection. exclude_edge_boxes takes a margin in original image pixels, with boxes touching that margin excluded. The keypoint pass uses the saved frame correspondence and rotations; it does not independently resynchronize the videos. The supplied detector and pose tracker are shared across camera workers and must support concurrent inference; each camera has its own box-association state.

Use box_tracks.to_csv(path) and BoxTracks.from_csv(path) to save and reload the box stage. Empty results retain their schema; these new entry points return empty containers when nothing is observed. No source gaps are filled. The generic extraction entry points retain their existing behavior. See markerless calibration for model setup, recording assumptions and the complete workflow.

Triangulate with calibrated cameras¶

To reconstruct detected image points with known camera poses:

world_points = image_points.triangulate(cameras, min_views=3)
world_points.to_csv("world_points.csv")

min_views is the minimum number of distinct posed cameras observing each keypoint at a synchronized moment. The default is two. Points below the minimum are omitted, leaving gaps in the reconstruction. This setting applies to this scripting call; it does not change the GUI's reconstruction settings.

For objects declared static with static_object_ids, observations are pooled across frames and the minimum counts distinct cameras over those pooled observations. Repeated observations from one camera count as one view.

Calibrating without intrinsics (experimental)¶

Experimental

This path passes synthetic tests but has not been validated on real-world data. The recommended workflow is to calibrate intrinsics first.

When a camera in the array has no intrinsics, calibrate_extrinsics synthesizes a starting guess from the resolution and recovers focal length and leading distortion during bundle adjustment. See Skipping Intrinsic Calibration for prerequisites.

from caliscope.api import (
    ArucoMarkerSet, ArucoTracker, CameraArray, ConstraintSet,
    calibrate_extrinsics, extract_image_points_multicam,
)

cameras = CameraArray.from_video_metadata(extrinsic_videos)
marker_set = ArucoMarkerSet.from_toml("calibration/targets/aruco_marker_set.toml")
tracker = ArucoTracker(dictionary=marker_set.dictionary, marker_set=marker_set)
constraints = ConstraintSet.from_marker_set(marker_set)

ext_points = extract_image_points_multicam(extrinsic_videos, tracker, frame_step=3)
result = calibrate_extrinsics(ext_points, cameras, constraints)

Calibrate fisheye cameras intrinsically first, then run the same extrinsic pipeline.

Chessboard extrinsics¶

A plain chessboard can drive the same extrinsic pipeline. The GUI does not offer it. The board must meet the symmetry condition in the warning below.

from caliscope.api import (
    Chessboard, ChessboardTracker, ConstraintSet,
    calibrate_extrinsics, extract_image_points_multicam,
)

chessboard = Chessboard(rows=6, columns=9, square_size_cm=3.0)
tracker = ChessboardTracker(chessboard)
constraints = ConstraintSet.from_chessboard(chessboard)

ext_points = extract_image_points_multicam(extrinsic_videos, tracker, frame_step=3)
run = calibrate_extrinsics(ext_points, cameras, constraints)

rows and columns count internal corners, not squares. square_size_cm is required here. ConstraintSet.from_chessboard raises without it, because the recovered world scale would otherwise be wrong. Detection is all-or-nothing, so a frame where any corner is cut off or covered contributes nothing.

Corner ordering and board symmetry

Use a board with one odd and one even inner-corner count, like the 6x9 example above. A board with both counts even or both odd looks identical after a half turn, so the detector cannot tell the two orientations apart. When two cameras see opposite orientations, corner ids reverse and triangulation silently pairs the wrong points. A ChArUco or ArUco target does not have this problem.

Matching the GUI¶

The two surfaces share their calibration code, so the same inputs give the same numbers. The GUI just does more of the surrounding steps for you.

GUI control	API equivalent
Board shape and square size	`Charuco.from_squares(columns, rows, square_size_cm)`
Board size and units	Print-only. Use the full `Charuco(...)` constructor.
Invert	`Charuco.from_squares(..., inverted=True)`
Save board PNG, mirror PNG	`charuco.save_image(path)`, `charuco.save_mirror_image(path)`
Camera rotate buttons	`rotation_counts={cam_id: turns}` at extraction
Calibrate	`calibrate_extrinsics(points, cameras, constraints)`
Refine intrinsics checkbox	`refine_intrinsics=True`
Filter outliers	`filter_percentile=2.5`
Set origin at frame	`volume.align_to_object(sync_index)`
X, Y, Z rotate buttons	`volume.rotate(axis, degrees)`
No GUI equivalent	`volume.translate(x, y, z)`
Saving a calibration	`volume.save(directory)`
`camera_array_aniposelib.toml`	`volume.camera_array.to_aniposelib_toml(path)`