Question regarding environment setup and version compatibility for AMP tasks (Aliengo)

[Ihtbfas]

Hi authors,

First of all, thank you for sharing this excellent repository!

I am currently trying to run the AMP training for the Aliengo robot (python scripts/amp_rl/train_amp.py --task=Locomotion-Aliengo-Flat --num_envs=4096 --headless), but I am running into a series of version compatibility issues regarding rsl-rl and amp-rsl-rl.

Here are the specific issues I encountered:

1. With rsl-rl==3.0.1 and the latest amp-rsl-rl : The training crashes with AttributeError: 'AliengoFlatEnvCfg' object has no attribute 'observations' and KeyError: 'key "amp" not found in TensorDict'. It seems the Aliengo environment configuration has not been fully updated to output the amp observation group expected by the new amp-rsl-rl architecture.
2. Standard PPO Training (train.py): When using rsl-rl==3.0.1, PPO.__init__() throws TypeError: unexpected keyword argument 'optimizer' and 'share_cnn_encoders'. I had to manually delete these keys from agent_dict in the training script to make it work.

To help me get the environment running smoothly, could you please clarify the exact dependency versions expected by this repository? Specifically:

• Which version/commit of IsaacLab?
• Which version of rsl-rl (2.2.4, 3.0.1, or another)?
• Which specific commit hash of amp-rsl-rl?

Thanks in advance for your time and help!

[giulioturrisi]

Hello,

we are using isaaclab v2.3.2, and i usually run my test with rsl-rl-lib==3.3.0.

Regarding the amp-rsl-rl integration, not sure at which rsl-rl-lib they are, and if the code is still on par with their latest release. I will check this in the next days!

Giulio

[Ihtbfas]

Hi Giulio, thanks for the update!

Good news! I set up the environment with isaaclab v2.3.2 and rsl-rl-lib==3.3.0 as you suggested, and the standard PPO training (train.py) runs perfectly now.

However, your suspicion about the amp-rsl-rl integration was spot on. I tried running the AMP training and encountered a few hurdles that confirm the code is currently out of sync with the 3.3.x architecture.

For your reference when you check this in the next days, here is a summary of the issues I found:

1. Import Error: ImportError: cannot import name 'store_code_state' from 'rsl_rl.utils'. It seems this function was removed in newer versions of rsl-rl, but amp_on_policy_runner.py still tries to import and call it.
2. Missing Config Keys: The AMP runner expects a dataset key in agent_cfg (KeyError: 'dataset'), which is missing in the older configurations.
3. The Main Blocker (Architecture mismatch): AttributeError: 'AliengoFlatEnvCfg' object has no attribute 'observations'. When I bypassed this to check further, it immediately hit KeyError: 'key "amp" not found in TensorDict'.

It seems the environment configurations (at least for Aliengo) are still using the older architecture and don't output the amp observation group that the new TensorDict-based amp-rsl-rl requires.

Take your time looking into it! I'll use the standard PPO for my experiments for now. Let me know if you need someone to test it once you get a chance to push an update for the AMP integration.

Thanks again for your work!

[giulioturrisi]

Hi, how do you bypassed AttributeError: 'AliengoFlatEnvCfg' object has no attribute 'observations'?

Regarding KeyError: 'key "amp" not found in TensorDict', you need in aliengo_env_cfg set use_amp = True probably

[Ihtbfas]

Hi Giulio,

To bypass the AttributeError, I temporarily modified amp_on_policy_runner.py inside the amp-rsl-rl repository. Around line 203, I wrapped the joint name extraction in a try...except block and hardcoded Aliengo's joint names as a fallback:

try:
    amp_joint_names = self.env.cfg.observations.amp.joint_pos.params["asset_cfg"].joint_names
except AttributeError:
    amp_joint_names = [
        "FL_hip_joint", "FR_hip_joint", "RL_hip_joint", "RR_hip_joint", 
        "FL_thigh_joint", "FR_thigh_joint", "RL_thigh_joint", "RR_thigh_joint",
        "FL_calf_joint", "FR_calf_joint", "RL_calf_joint", "RR_calf_joint"
    ]

Immediately after bypassing it, the execution failed at line 217 (num_amp_obs = observations["amp"].shape[1]) with KeyError: 'key "amp" not found in TensorDict'.
Regarding the KeyError, thanks for the tip! I will set use_amp = True in aliengo_env_cfg, revert my temporary bypass in the runner, and test it to see if it populates the TensorDict correctly.

[seyoungree]

Hi both. First of all, thank you for this amazing repo! I'm actually having the same issue, so I wanted to share. I am using IsaacLab v2.3.2, the latest version of amp-rsl-rl, and rsl-rl-lib==3.1.2.

I was able to bypass these errors by adding this to the EnvCfg (sorry this might be just a lazy fix.)

class observations:
        class amp:
            class joint_pos:
                class asset_cfg:
                    joint_names = ["FL_hip_joint", "FL_thigh_joint", "FL_calf_joint", "FR_hip_joint", "FR_thigh_joint", "FR_calf_joint", "RL_hip_joint", "RL_thigh_joint", "RL_calf_joint", "RR_hip_joint", "RR_thigh_joint", "RR_calf_joint"]
                params = {"asset_cfg": asset_cfg()}

I also had to uncomment self._robot.data.root_ang_vel_b in line 295 in locomotion_env.py. It seems that the discriminator expects the following for the amp observations:

self._robot.data.joint_pos,
self._robot.data.joint_vel,
self._robot.data.root_lin_vel_b,
self._robot.data.root_ang_vel_b,

Also, for the discriminator in rsl_rl_ppo_cfg.py I added the following to the DiscriminatorCfg class (I think you can have different arguments than me though):

loss_type: str = "BCEWithLogits"
empirical_normalization: bool = False

[giulioturrisi]

thanks for info @seyoungree. For the class observations, you put it exactly where?

[seyoungree]

Thanks for the reply! I put it in AliengoFlatEnvCfg in the aliengo_flat_cfg.py file.