Francesco Capuano
f3d931e1b2
Add direct access to action chunks ( #1020 )
...
* fix: sharing predicted chunk with user
* [pre-commit.ci] pre-commit autoupdate (#1011 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Revert "[pre-commit.ci] pre-commit autoupdate" (#1025 )
* fix(ci): Pin draccus (<0.10.0) and torch (<2.7) to fix pipeline (#1022 )
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
* fix(ci): Pin `torchcodec` (==0.2.1) to fix pipeline temporarly (#1030 )
* Update tutorial (#1021 )
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
* Add description motor order SO-101 leader (#1051 )
* feat(encoding): switching to PyAV for ffmpeg related tasks (#983 )
* feat(docs): Add new docs build process (#1046 )
Co-authored-by: Mishig Davaadorj <dmishig@gmail.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co>
* Docs: adapt text + fix video code (#1064 )
* Fix typos (#1070 )
* docs: minor corrections and clean-up (#1089 )
* Update 10_use_so100.md; use diff syntax (#944 )
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* Update 12_use_so101.md (#1081 )
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* bug fix for #1071 When --display_data=true, Failed running control_robot. (#1073 )
* Add editable -e for feetech install command (#1133 )
* Fix: emptying action queue between resets (#1117 )
* fix: typos and grammar (#1148 )
* Update README.md (#1160 )
* Update README.md (#1163 )
* [Fix] Unpin torch beyond 2.6.0 & torchcodec beyond 0.2.1 (#1127 )
* (hotfix): nightly CI by clipping pymunk version below 7.0.0 (#1182 )
* [pre-commit.ci] pre-commit autoupdate (#1048 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co>
* Add SmolVLA (#1175 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: fracapuano <francesco.capuano@huggingface.co>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
Co-authored-by: Dana Aubakirova <118912928+danaaubakirova@users.noreply.github.com>
Co-authored-by: Remi <remi.cadene@huggingface.co>
* Fix SmolVLA loss not sent to wandb (#1198 )
* Hardware API redesign (#777 )
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co>
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
Co-authored-by: Pepijn <pepijn@huggingface.co>
* fix(smolvla): update record.py, fix populate_queues and remove unused dependencies (#1208 )
* replaced OBS_ROBOT with OBS_STATE constant (#1211 )
* Fix test_teleoperate (#1216 )
* Fix LeKiwi example (#1217 )
* Fix smolVLA dependencies (#1218 )
* fix(pyserial): adding pyserial dependency to global ones (#1219 )
* Update SmolVLA README.md (#1228 )
* Fix unable to set camera width/height to non-default (#1225 )
* Update tutorial link (#1250 )
* update KochFollower.get_observation() so it returns same observation structure as SO101 (#1248 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [pre-commit.ci] pre-commit autoupdate (#1185 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
* Proposal for fix for enter_pressed on Windows (#1230 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
* fix: update pi0 dependency version constraint (#1247 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Match motor names with ids lekiwi (#1261 )
* fix issues: checkpoints keys mismatch and 'task' tokenisation in smolvla (#1256 )
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co>
* fix(docs): update realsense documentation (#1268 )
* Use HF Papers (#1120 )
* Skip normalization parameters in load_smolvla (#1274 )
* fix(record): no teleop needed when running with policy (#1284 )
* Port HIL SERL (#644 )
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>
Co-authored-by: Eugene Mironov <helper2424@gmail.com>
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com>
Co-authored-by: Ke Wang <superwk1017@gmail.com>
Co-authored-by: Yoel Chornton <yoel.chornton@gmail.com>
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co>
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co>
* fix(docs): SmolVLA fine-tuning getting started (#1201 )
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
Co-authored-by: Francesco Capuano <francesco_capuano@aol.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co>
* chore(teleop): print calibration path saved (#1286 )
* chore(dependencies): add gamepad support with pygame and hidapi (#1287 )
* Robot integration tutorial (#1285 )
* fix(docs): update send_feedback docstrings
* Add sim tutorial, fix lekiwi motor config, add notebook links (#1275 )
Co-authored-by: AdilZouitine <adilzouitinegm@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com>
Co-authored-by: Michel Aractingi <michel.aractingi@gmail.com>
Co-authored-by: Eugene Mironov <helper2424@gmail.com>
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
* Fixes on robot integration tutorial (#1290 )
* Add keyboard teleop device to control the end effector robot (#1289 )
* Improve type hints (#1293 )
* fix(record): no teleop arg in reset environment (#1294 )
* `learner.py` import so101_leader instead of so100 (#1295 )
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
* Fixing `PI0` Policy (#1297 )
* `gym_manipulator.py` Remove None value action_intervention of BaseLeaderTeleoperator (#1299 )
* (chore): incorrect resume parameter in recording documentation (#1301 )
* Update lekiwi.mdx (#1229 )
* bump `pi0` and `hil` transformers version (#1298 )
* docs: fix imitation learning robots docs command (#1308 )
* fix(benchmarks): remove .numpy() from frame in benchmark script (#1354 )
* add smolvla to the supported policies to run tests (:
* add: chunk-level access for the policy
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* add: smolvla in availables
* remove: smolvla from library supported policies
* fix: change env for training, xarm is broken as of now
* add: predict_action_chunk to all supported policies
* fix: add robot type constants
* add: predict action chunk in base policy class
* restore original Makefile
* fix: minor
* fix: dict keys come from lerobot/constants
* fix: improve act encapsulation, properly supporting temporal ensembling
* fix: smolvla action chunking
* fix: very minor, but very annoying
* fix: minor
* fix minor naming
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
Signed-off-by: Francesco Capuano <74058581+fracapuano@users.noreply.github.com>
* fix: refactoring inference for single actions and chunks into different components
* fix: minor
* fix: temporal ensembling
* fix: moving populate queues out of modular component for batch preparation
* fix: minor for CI
* fix: smovla debug
* fix: reward classifier, maybe the last policy lacking?
---------
Signed-off-by: Francesco Capuano <74058581+fracapuano@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co>
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
Co-authored-by: Caroline Pascal <caroline8.pascal@gmail.com>
Co-authored-by: Mishig Davaadorj <dmishig@gmail.com>
Co-authored-by: omahs <73983677+omahs@users.noreply.github.com>
Co-authored-by: CharlesCNorton <135471798+CharlesCNorton@users.noreply.github.com>
Co-authored-by: masato-ka <jp6uzv@gmail.com>
Co-authored-by: Ragnar <rodiondenmark@gmail.com>
Co-authored-by: mshukor <mustafa.shukor97@gmail.com>
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
Co-authored-by: Dana Aubakirova <118912928+danaaubakirova@users.noreply.github.com>
Co-authored-by: Remi <remi.cadene@huggingface.co>
Co-authored-by: Ben Zhang <5977478+ben-z@users.noreply.github.com>
Co-authored-by: Pepijn <pepijn@huggingface.co>
Co-authored-by: Dhruva <51377003+utterwqlnut@users.noreply.github.com>
Co-authored-by: Daisuke Sato <tiryoh@gmail.com>
Co-authored-by: Sarunas Kalade <sarunas.kalade@amd.com>
Co-authored-by: koenvanwijk <koenvanwijk@users.noreply.github.com>
Co-authored-by: Yushun Xiang <73413365+YushunXiang@users.noreply.github.com>
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz>
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>
Co-authored-by: Eugene Mironov <helper2424@gmail.com>
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com>
Co-authored-by: Ke Wang <superwk1017@gmail.com>
Co-authored-by: Yoel Chornton <yoel.chornton@gmail.com>
Co-authored-by: Michel Aractingi <michel.aractingi@gmail.com>
Co-authored-by: tidely <43219534+tidely@users.noreply.github.com>
Co-authored-by: David <17435126+DavidLMS@users.noreply.github.com>
2025-06-27 10:19:19 +02:00
Quentin Gallouédec
edfebd522c
Use HF Papers ( #1120 )
2025-06-12 09:58:59 +02:00
Simon Alibert
e23b41e79a
Hardware API redesign ( #777 )
...
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co>
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
Co-authored-by: Pepijn <pepijn@huggingface.co>
2025-06-05 17:48:43 +02:00
Simon Alibert
a1809ad3de
Add typos checks ( #770 )
2025-02-25 23:51:15 +01:00
Simon Alibert
c4c2ce04e7
Update pre-commits ( #733 )
2025-02-15 15:51:17 +01:00
Simon Alibert
90e099b39f
Remove offline training, refactor train.py and logging/checkpointing ( #670 )
...
Co-authored-by: Remi <remi.cadene@huggingface.co>
2025-02-11 10:36:06 +01:00
Simon Alibert
3c0a209f9f
Simplify configs ( #550 )
...
Co-authored-by: Remi <remi.cadene@huggingface.co>
Co-authored-by: HUANG TZU-CHUN <137322177+tc-huang@users.noreply.github.com>
2025-01-31 13:57:37 +01:00
Hirokazu Ishida
538455a965
feat: enable to use multiple rgb encoders per camera in diffusion policy ( #484 )
...
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com>
2024-10-30 11:00:05 +01:00
Alexander Soare
a60d27b132
Raise ValueError if horizon is incompatible with downsampling ( #422 )
2024-09-09 17:22:46 +01:00
NielsRogge
86bbd16d43
Improve discoverability on the hub ( #325 )
...
Co-authored-by: Lucain <lucainp@gmail.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
2024-08-19 15:16:46 +02:00
Halvard Bariller
7a3cb1ad34
Adjust the timestamps' description in Diffusion Policy ( #343 )
...
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com>
2024-07-26 12:47:03 +01:00
Alexander Soare
abbb1d2367
Make sure policies don't mutate the batch ( #323 )
2024-07-22 20:38:33 +01:00
Alexander Soare
cc2f6e7404
Train diffusion pusht_keypoints ( #307 )
...
Co-authored-by: Remi <re.cadene@gmail.com>
2024-07-09 12:35:50 +01:00
Jihoon Oh
b72d574891
fix Unet global_cond_dim to use state dim, not action dim ( #278 )
2024-06-17 15:17:28 +01:00
Alexander Soare
15dd682714
Add multi-image support to diffusion policy ( #218 )
2024-06-17 08:11:20 +01:00
Alexander Soare
cf15cba5fc
Remove redundant slicing operation in Diffusion Policy ( #240 )
2024-06-03 13:04:24 +01:00
Remi
d585c73f9f
Add real-world support for ACT on Aloha/Aloha2 ( #228 )
...
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com>
2024-05-31 15:31:02 +02:00
Alexander Soare
3d625ae6d3
Handle crop_shape=None in Diffusion Policy ( #219 )
2024-05-28 18:27:33 +01:00
Radek Osmulski
3b86050ab0
throw an error if config.do_maks_loss and action_is_pad not provided in batch ( #213 )
...
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com>
2024-05-27 09:06:26 +01:00
Akshay Kashyap
c9069df9f1
Port SpatialSoftmax and remove Robomimic dependency ( #182 )
...
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com>
2024-05-16 15:34:10 +01:00
Alexander Soare
68c1b13406
Make policies compatible with other/multiple image keys ( #149 )
2024-05-16 13:51:53 +01:00
Simon Alibert
f52f4f2cd2
Add copyrights ( #157 )
2024-05-15 12:13:09 +02:00
Akshay Kashyap
460df2ccea
Support for DDIMScheduler in Diffusion Policy ( #146 )
2024-05-08 18:05:16 +01:00
Alexander Soare
f5de57b385
Fix SpatialSoftmax input shape ( #150 )
2024-05-08 14:57:29 +01:00
Alexander Soare
a8e245fb31
Remove loss masking from diffusion policy ( #135 )
2024-05-06 07:27:01 +01:00
Alexander Soare
f3bba0270d
Remove EMA model from Diffusion Policy ( #134 )
2024-05-05 11:26:12 +01:00
Simon Alibert
c015252e20
Remove batch length restrictions in select_action ( #123 )
2024-05-04 15:33:55 +02:00
Alexander Soare
d1855a202a
Refactor TD-MPC ( #103 )
...
Co-authored-by: Cadene <re.cadene@gmail.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
2024-05-01 16:40:04 +01:00
Alexander Soare
a4891095e4
Use PytorchModelHubMixin to save models as safetensors ( #125 )
...
Co-authored-by: Remi <re.cadene@gmail.com>
2024-05-01 16:17:18 +01:00
Alexander Soare
01d5490d44
Follow transformers single file naming conventions ( #124 )
2024-05-01 13:09:42 +01:00
Alexander Soare
9d60dce6f3
Tidy up yaml configs ( #121 )
2024-04-30 16:08:59 +01:00
Quentin Gallouédec
508bd92d03
Remove update method from the policy ( #99 )
...
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com>
2024-04-29 12:27:58 +02:00
Alexander Soare
ccffa9e406
Fix policy defaults ( #113 )
2024-04-29 08:26:59 +01:00
Simon Alibert
791506dfb8
Remove warnings ( #111 )
...
- Replace `use_pretrained_backbone` with `pretrained_backbone_weights`
- Bump diffusers' minimum version `0.26.3` -> `0.27.2`
- Add ignore flags in CI's pytest
- Change Box observation spaces in simulation environments
- Set `version_base="1.2"` in Hydra initializations
- Bump einops' minimum version `0.7.0` -> `0.8.0`
2024-04-29 00:31:33 +02:00
Alexander Soare
45f351c618
Make sure targets are normalized too ( #106 )
2024-04-26 11:18:39 +01:00
Remi
e760e4cd63
Move normalization to policy for act and diffusion ( #90 )
...
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com>
2024-04-25 11:47:38 +02:00
Alexander Soare
2298ddf226
wip
2024-04-17 16:21:37 +01:00
Alexander Soare
a9496fde39
revision 1
2024-04-16 17:15:51 +01:00
Alexander Soare
9c2f10bd04
ready for review
2024-04-16 13:43:58 +01:00
Alexander Soare
03b08eb74e
backup wip
2024-04-16 12:51:32 +01:00
Alexander Soare
5608e659e6
backup wip
2024-04-15 19:06:44 +01:00
Alexander Soare
6d0a45a97d
ready for review
2024-04-12 11:36:52 +01:00
Alexander Soare
5666ec3ec7
backup wip
2024-04-11 18:33:54 +01:00
Alexander Soare
976a197f98
backup wip
2024-04-11 17:51:35 +01:00
Alexander Soare
863f28ffd8
ready for review
2024-04-08 13:10:19 +01:00
Alexander Soare
1e71196fe3
backup wip
2024-04-05 17:38:29 +01:00
Cadene
a420714ee4
fix: action_is_pad was missing in compute_loss
2024-04-05 11:33:39 +00:00
Cadene
5af00d0c1e
fix train.py, stats, eval.py (training is running)
2024-04-05 09:31:39 +00:00
Cadene
1cdfbc8b52
WIP
...
WIP
WIP train.py works, loss going down
WIP eval.py
Fix
WIP (eval running, TODO: verify results reproduced)
Eval works! (testing reproducibility)
WIP
pretrained model pusht reproduces same results as torchrl
pretrained model pusht reproduces same results as torchrl
Remove AbstractPolicy, Move all queues in select_action
WIP test_datasets passed (TODO: re-enable NormalizeTransform)
2024-04-04 15:31:03 +00:00
Alexander Soare
1a1308d62f
fix environment seeding
...
add fixes for reproducibility
only try to start env if it is closed
revision
fix normalization and data type
Improve README
Improve README
Tests are passing, Eval pretrained model works, Add gif
Update gif
Update gif
Update gif
Update gif
Update README
Update README
update minor
Update README.md
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
Update README.md
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
Address suggestions
Update thumbnail + stats
Update thumbnail + stats
Update README.md
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com>
Add more comments
Add test_examples.py
2024-03-26 10:10:43 +00:00
