Get Started#

Here’s the full video for keyboard EE control

Prerequisites#

  • Ubuntu operating system

  • Basic familiarity with terminal commands

  • Miniconda (recommended)

Installation#

1. Create a Conda Environment#

If you already have a lerobot conda environment, you can use that. Otherwise, create a new environment:

conda create -y -n lerobot python=3.10
conda activate lerobot

Note: It’s recommended to have ManiSkill and lerobot code in the same environment for future sim2real deployment.

2. Install ManiSkill#

Follow the official installation instructions for ManiSkill.

# Basic installation
pip install mani-skill

# Download scene dataset
python -m mani_skill.utils.download_asset "ReplicaCAD"

2.1 If the dataset downloading goes wrong, use:#

git clone <https://huggingface.co/datasets/haosulab/ReplicaCAD>

to directly download the dataset from huggingface. And then folder contents should go to

~/.maniskill/data/scene_datasets/replica_cad_dataset

2.2 Familiarize yourself with ManiSkill using the quickstart guide and demo scripts.#

Try this command to test whether you have successfully installed Maniskill:

python -m mani_skill.examples.demo_random_action -e "ReplicaCAD_SceneManipulation-v1" \\
  --render-mode="human" --shader="rt-fast"

And you should be looking at something like this:

image

3. Additional Dependencies#

pip install pygame

4. Replace Robot Files#

Navigate to the ManiSkill package folder in your conda environment:

cd ~/miniconda3/envs/lerobot/lib/python3.10/site-packages/mani_skill

Replace the fetch robot code and assets with the XLeRobot files:

  1. Download simulation/Maniskill.zip

  2. Replace the files in /agents and /assets:

    image

    image

  3. Add control code to /examples:

    image

Usage#

The name of the files may be slightly different, check and change if you need.

Joint Control#

Run the joint control demo with:

python -m mani_skill.examples.XLeRobot_demo_joint_ctrl -e "ReplicaCAD_SceneManipulation-v1"   --render-mode="human" --shader="rt-fast" -c "pd_joint_delta_pos_dual_arm"

End Effector Control#

Run the end effector control demo with:

python -m mani_skill.examples.XLeRobot_demo_ee_ctrl -e "ReplicaCAD_SceneManipulation-v1"   --render-mode="human" --shader="rt-fast" -c "pd_joint_delta_pos_dual_arm"

⏫ Extra Parts for XLeRobot ⏫#

If you already have 2 SO100 Arms and 1 Lekiwi base, you’ll only need 3 additional parts for the XLeRobot hardware version 0.1.0.

(Please also check the Build From Scratch section below, as the parts list differs slightly)

Arm bases#

  • The bases are designed to rotate 15 degrees outward, expanding the workspace area on both sides while enabling smooth collaboration between the two arms.

  • The original forward facing base is also available if you want to keep things simple.

![image](https://github.com/user-attachments/assets/eb77aad3-4df3-45c1-93c2-1c2e278512b5)

  • The design features a hollow center to save filament. You can insert a used Bambulab filament cardboard spool for added structural support.

Lekiwi base connectors#

  • This component connects the Lekiwi base’s top plate to the IKEA cart bottom. It’s an enhanced version of the Lekiwi base motor mount, redesigned for better stability.

Note

If you’re using a not an authentic IKEA cart, you can adjust the z-axis scale in your slicer software (while maintaining the xy-axis scale) to ensure even pressure distribution between the Lekiwi base and cart wheels.

Thinner Lekiwi base top plate#

  • This is a thinner version of the standard Lekiwi base top plate, used to secure the connectors to the IKEA cart bottom.

  • While the current connection between the Lekiwi base and cart is stable, an upcoming second version will simplify installation on the cart’s metal mesh bottom.