82 lines
3.5 KiB
Markdown
82 lines
3.5 KiB
Markdown
# RoboTwin Code Generation
|
|
Branch: [gpt](https://github.com/TianxingChen/RoboTwin/tree/gpt) | <a href="https://hits.seeyoufarm.com"><img src="https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2FTianxingChen%2FRoboTwin&count_bg=%23184FFF&title_bg=%23E116E5&icon=&icon_color=%23E7E7E7&title=Repo+Viewers&edge_flat=true"/></a>
|
|
|
|
## Installation
|
|
> **Notice**: the model files of this branch are different from the main branch, which have different kinds of objects information !!!
|
|
|
|
Refer to [INSTALLATION.md](./INSTALLATION.md) for installation instructions. It takes approximately 20 minutes to complete the installation.
|
|
|
|
## Configure LLM API Key
|
|
|
|
Please configure the necessary API keys in the `./gpt_api/gpt_agent.py` file. Additionally, if the LLM you are utilizing does not support integration with the OpenAI API, you may need to make corresponding adjustments to the `generate()` function.
|
|
|
|
## Generate Your Task Code
|
|
|
|
### 1. **Add Task Description**
|
|
|
|
Add new task information, including the task name and natural language description, in `./gpt_api/task_info.py`.
|
|
|
|
### 2. **Add Basic Task Code**
|
|
|
|
Add the basic code file `${task_name}.py` in the `./envs/` directory, following the file structure as shown in [BASIC_TASK_FILE](./envs/README.md).
|
|
|
|
### 3. **Add Task Scene File**
|
|
|
|
Add the scene configuration file `${task_name}.json` in the `./task_config/scene_info/` directory, following the file structure as shown in [SCENE_CONFIG](./task_config/scene_info/README.md).
|
|
|
|
### 4. **Generate the Final Code**
|
|
|
|
Run the following script to generate task code:
|
|
```bash
|
|
python task_generation.py ${task_name}
|
|
```
|
|
The generated code file will be `./envs/gpt_${task_name}.py`. For example:
|
|
```bash
|
|
python task_generation.py apple_cabinet_storage
|
|
```
|
|
The generated code file will be `./envs/gpt_apple_cabinet_storage.py`
|
|
|
|
### 5. **Collect Data**
|
|
|
|
Run the script:
|
|
```bash
|
|
bash run_task.sh ${task_name} {gpu_id}
|
|
```
|
|
To collect expert data for the relevant tasks, where `${task_name}` corresponds to the task file name in the `./envs/` directory. For example:
|
|
```bash
|
|
bash run_task.sh gpt_apple_cabinet_storage 0
|
|
```
|
|
See [TASK_CONFIG](./task_config/README.md) for data collection configurations.
|
|
|
|
## Directory Structure
|
|
```
|
|
.
|
|
├── aloha_maniskill_sim Robot URDF and SRDF files
|
|
├── Code-generator Document.md
|
|
├── data Directory for expert data collection
|
|
├── envs
|
|
│ ├── base_task.py Base class for tasks
|
|
│ ├── TASK_NAME.py Task scene generation and success determination
|
|
│ ├── gpt_TASK_NAME.py Code file generated for the task
|
|
│ └── utils
|
|
├── gpt_api
|
|
│ ├── gpt_agent.py
|
|
│ ├── __init__.py
|
|
│ ├── prompt.py
|
|
│ └── task_info.py Task information
|
|
├── models
|
|
│ ├── MODEL_FILE Digital asset files
|
|
│ │ ├── base{id}.glb Model files
|
|
│ │ └── model_data{id}.json Model calibration data files
|
|
│ ├── ...
|
|
│ └── models.py
|
|
├── run_task.sh Script to run tasks
|
|
├── task_config
|
|
│ ├── scene_info Directory for each task scene data
|
|
│ ├── seeds Directory for random seeds of expert data for each task
|
|
│ ├── TASK_NAME.yml Task data collection parameters
|
|
│ └── ...
|
|
├── task_generation.py Code generation script for tasks
|
|
└── ...
|
|
```
|