LIBERO

LIBERO tests a Franka Panda arm on tabletop pick-and-place with a language instruction. Behind the scenes, Manifold runs Lifelong-Robot-Learning/LIBERO.

It contains four suites, each focusing on a different axis of variation:

• libero-spatial changes where the target object sits
• libero-object swaps which object is the target
• libero-goal rephrases the goal
• libero-10 is ten longer-horizon tasks

Each of the four runs 10 tasks with 50 episodes per task, so one suite is a 500 episode run.

Benchmark	Tasks	Episodes per task	Total episodes
libero-spatial	10	50	500
libero-object	10	50	500
libero-goal	10	50	500
libero-10	10	50	500

Contract

Observation

Each step, the benchmark sends an observation with these fields. If Manifold has adapters for the difference, it converts the observation to the form that your policy signature declares. Then it passes the observation to your policy.

Field	Values	Meaning
state["ee_pose"]	x, y, z	position in meters
	qx, qy, qz, qw	rotation as a quaternion, scalar last
	f0, f1	finger positions in meters, offset from the gripper centerline; 0 is closed, 0.04 is fully open; the second finger's value is negative
instruction		the task prompt, a string
sensors["agentview"]		256x256 color image, uint8
sensors["wrist"]		256x256 color image, uint8
sensors["agentview_depth"]		256x256 depth image, float32 meters
sensors["wrist_depth"]		256x256 depth image, float32 meters
extrinsics		one 4x4 camera-to-world pose per camera, keyed by camera name

Check the expected orientation of your policy

Manifold's LIBERO-based benchmarks serve camera images exactly as MuJoCo renders them.

True scene

FLIPPED_VERTICAL
Raw MuJoCo and Manifold

FLIPPED_HORIZONTAL
Common dataset orientation

Because of the way MuJoCo OpenGL buffers are emitted directly, the images will be FLIPPED_VERTICAL relative to the actual scene. The LIBERO signature in Manifold's SDK declares this to avoid confusion.

Unfortunately, common LIBERO training datasets (OpenVLA RLDS, HuggingFaceVLA/libero) apply a misleading 180-degree rotation of the raw MuJoCo output, which turns the images the right way up but horizontally mirrored relative to the real scene.

Manifold always declares image orientation relative to the actual scene.

• If your model was trained on frames that match the actual scene, declare UPRIGHT in your policy signature.
• If your model was trained on raw MuJoCo output, declare FLIPPED_VERTICAL in your policy signature.
• If your model was trained on the OpenVLA RLDS or HuggingFaceVLA/libero preprocessing (raw MuJoCo rotated 180 degrees), declare FLIPPED_HORIZONTAL in your policy signature. A 180-degree rotation of a vertically flipped image is a horizontal mirror relative to the actual scene.
• ROTATED_180 fits only frames that show the actual scene rotated 180 degrees, which is uncommon in published LIBERO datasets.

To learn more, read Your robot may be learning the world flipped.

Action

Return actions in the exact format that your policy signature declares. Manifold will attempt to convert your policy's output to match what the benchmark accepts.

Compatible policy signatures

Change the values to match your policy. If the policy signature below matches what your policy natively requires, then it is guaranteed to be compatible with LIBERO.

Action rotation

Action gripper

ProprioceptionFormat

End effector pose

CameraOrientationSize

widthheight

widthheight

Show policy signature

POLICY_SIGNATURE = PolicySignature(
    action_space=EEActionSpace(
        rotation=RotationFormat.AXIS_ANGLE,
        gripper=GripperFormat.SIGNED_OPEN_LOW,
        delta=True,
    ),
    proprioception=Proprioception(
        ee_pose=EEObservationSpec(
            rotation=RotationFormat.QUATERNION,
            gripper=GripperObservationSpec(dim=2),
        ),
    ),
    cameras=(
        Camera(
            name="agentview",
            shape=(256, 256, 3),
            mount=Mount.SCENE,
            orientation=CameraOrientation.FLIPPED_VERTICAL,
        ),
        Camera(
            name="wrist",
            shape=(256, 256, 3),
            mount=Mount.WRIST,
            orientation=CameraOrientation.FLIPPED_VERTICAL,
        ),
    ),
    instruction=True,
)

Depth and camera calibration

Running LIBERO on Manifold provides depth maps in addition to regular RGB images. Each observation returns a depth map and camera calibration data.

from manifold.benchmarks.libero import LIBERO

# The 3x3 intrinsic matrix, fixed for the whole run:
K = LIBERO.observation_space.camera("agentview_depth").calibration.intrinsics.matrix()

def infer(observation):
    depth = observation.sensors["agentview_depth"]    # (256, 256, 1) float32, meters
    pose = observation.extrinsics["agentview_depth"]  # (4, 4) camera-to-world pose
    ...

Depth is recorded in meters at the same resolution as the RGB image. Beyond the camera's range, the value is inf.

Debugging and testing

• The gripper opens at -1 and closes at +1. A flipped gripper can fail silently, and usually shows up as rapid opening and closing during the rollout.
• Observation quaternions are scalar last (x, y, z, w). Check the order your rotation library uses before converting.
• Actions are deltas. LIBERO applies them relative to the current pose.
• The rotation action is axis-angle, in radians.
• A policy that predicts joint angles needs forward kinematics. LIBERO only accepts end-effector deltas, so joint predictions must be converted with FK against a Franka model that matches the simulator. A mismatched model adds a pose bias to every action.

To check your policy end to end before a full suite run, submit it against debug-libero, a small debug version of this benchmark.

For the full walkthrough, see Serve your policy.