Update README.md

README.md

@@ -1,62 +1,150 @@
 ---
-datasets: unknown
+language:
+- en
 library_name: lerobot
-license: apache-2.0
-model_name: xvla
 pipeline_tag: robotics
 tags:
-- xvla
-- robotics
+- vision-language-action
+- imitation-learning
 - lerobot
+inference: false
+license: apache-2.0
 ---
 
-# Model Card for xvla
+# X-VLA (LeRobot)
 
-<!-- Provide a quick summary of what the model is/does. -->
+X-VLA is a Vision-Language-Action foundation model that uses soft prompts to handle cross-embodiment and cross-domain robot control within a unified Transformer architecture.
 
+Checkpoint adapted for Google Robot platforms.
 
-_Model type not recognized — please update this template._
+**Original paper:** [X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model](https://arxiv.org/abs/2510.10274)
+**Reference implementation:** https://github.com/2toinf/X-VLA  
+**LeRobot implementation:** Follows the original reference code for compatibility.
 
 
-This policy has been trained and pushed to the Hub using [LeRobot](https://github.com/huggingface/lerobot).
-See the full documentation at [LeRobot Docs](https://huggingface.co/docs/lerobot/index).
+## Model description
 
----
+- **Inputs:** images (multi-view), proprio/state, optional language instruction
+- **Outputs:** continuous actions
+- **Training objective:** flow matching
+- **Action representation:** continuous
+- **Intended use:** Base model to fine tune on your specific use case
 
-## How to Get Started with the Model
 
-For a complete walkthrough, see the [training guide](https://huggingface.co/docs/lerobot/il_robots#train-a-policy).
-Below is the short version on how to train and run inference/eval:
+## Quick start (inference on a real batch)
 
-### Train from scratch
+### Installation
+
+```bash
+pip install "lerobot[xvla]"
+```
+For full installation details (including optional video dependencies such as ffmpeg for torchcodec), see the official documentation: https://huggingface.co/docs/lerobot/installation
+
+### Load model + dataset, run `select_action`
+
+```python
+import torch
+from lerobot.datasets.lerobot_dataset import LeRobotDataset
+from lerobot.policies.factory import make_pre_post_processors
+
+# Swap this import per-policy
+from lerobot.policies.xvla.modeling_xvla import XVLAPolicy
+
+# load a policy
+model_id = "lerobot/xvla-google-robot"  # <- swap checkpoint
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+policy = XVLAPolicy.from_pretrained(model_id).to(device).eval()
+
+preprocess, postprocess = make_pre_post_processors(
+    policy.config,
+    model_id,
+    preprocessor_overrides={"device_processor": {"device": str(device)}},
+)
+# load a lerobotdataset (we will replace with a simpler dataset)
+dataset = LeRobotDataset("lerobot/libero")
+
+# pick an episode
+episode_index = 0
+
+# each episode corresponds to a contiguous range of frame indices
+from_idx = dataset.meta.episodes["dataset_from_index"][episode_index]
+to_idx   = dataset.meta.episodes["dataset_to_index"][episode_index]
+
+# get a single frame from that episode (e.g. the first frame)
+frame_index = from_idx
+frame = dict(dataset[frame_index])
+
+batch = preprocess(frame)
+with torch.inference_mode():
+    pred_action = policy.select_action(batch)
+    # use your policy postprocess, this post process the action
+    # for instance unnormalize the actions, detokenize it etc..
+    pred_action = postprocess(pred_action)
+```
+
+
+## Training step (loss + backward)
+
+If you’re training / fine-tuning, you typically call `forward(...)` to get a loss and then:
+
+```python
+policy.train()
+batch = dict(dataset[0])
+batch = preprocess(batch)
+
+loss, outputs = policy.forward(batch)
+loss.backward()
+
+```
+
+> Notes:
+> 
+> - Some policies expose `policy(**batch)` or return a dict; keep this snippet aligned with the policy API.
+> - Use your trainer script (`lerobot-train`) for full training loops.
+
+
+## How to train / fine-tune
 
 ```bash
 lerobot-train \
   --dataset.repo_id=${HF_USER}/<dataset> \
-  --policy.type=act \
-  --output_dir=outputs/train/<desired_policy_repo_id> \
-  --job_name=lerobot_training \
+  --output_dir=./outputs/[RUN_NAME] \
+  --job_name=[RUN_NAME] \
+  --policy.repo_id=${HF_USER}/<desired_policy_repo_id> \
+  --policy.path=lerobot/[BASE_CHECKPOINT] \
+  --policy.dtype=bfloat16 \
   --policy.device=cuda \
-  --policy.repo_id=${HF_USER}/<desired_policy_repo_id>
-  --wandb.enable=true
+  --steps=100000 \
+  --batch_size=4
 ```
 
-_Writes checkpoints to `outputs/train/<desired_policy_repo_id>/checkpoints/`._
+Add policy-specific flags below:
 
-### Evaluate the policy/run inference
+- `-policy.chunk_size=...`
+- `-policy.n_action_steps=...`
+- `-policy.max_action_tokens=...`
+- `-policy.gradient_checkpointing=true`
 
-```bash
-lerobot-record \
-  --robot.type=so100_follower \
-  --dataset.repo_id=<hf_user>/eval_<dataset> \
-  --policy.path=<hf_user>/<desired_policy_repo_id> \
-  --episodes=10
-```
 
-Prefix the dataset repo with **eval\_** and supply `--policy.path` pointing to a local or hub checkpoint.
+## Real-World Inference & Evaluation
 
----
+You can use the `record` script from [**`lerobot-record`**](https://github.com/huggingface/lerobot/blob/main/src/lerobot/scripts/lerobot_record.py) with a policy checkpoint as input, to run inference and evaluate your policy. 
 
-## Model Details
+For instance, run this command or API example to run inference and record 10 evaluation episodes:
 
-- **License:** apache-2.0
+```
+lerobot-record  \
+  --robot.type=so100_follower \
+  --robot.port=/dev/ttyACM1 \
+  --robot.cameras="{ up: {type: opencv, index_or_path: /dev/video10, width: 640, height: 480, fps: 30}, side: {type: intelrealsense, serial_number_or_name: 233522074606, width: 640, height: 480, fps: 30}}" \
+  --robot.id=my_awesome_follower_arm \
+  --display_data=false \
+  --dataset.repo_id=${HF_USER}/eval_so100 \
+  --dataset.single_task="Put lego brick into the transparent box" \
+  # <- Teleop optional if you want to teleoperate in between episodes \
+  # --teleop.type=so100_leader \
+  # --teleop.port=/dev/ttyACM0 \
+  # --teleop.id=my_awesome_leader_arm \
+  --policy.path=${HF_USER}/my_policy
+```