AI Training Log (TienKung-Lab)
Reinforcement Learning — Dashboard Preview
Demo data · Training logs are not connected
The Reward and Loss charts use generated sample data to demonstrate the layout. They are not measured training results. This dashboard is ready to be connected to actual training logs.
Project Objectives
- Develop highly stable, natural, and robust locomotion policies for humanoid robots.
- Integrate AMP (Adversarial Motion Priors) techniques with complex reward shaping.
- Ensure accurate physics simulation via Isaac Lab to guarantee successful Sim-to-Real transfer.
Latest Experiment Logs
Current training phase data extracted from the system backend:
> Task Environment: "run" (การวิ่ง)
> Number of Envs: 4096 parallel simulators
> Latest Checkpoint: model_50900.pt
> Logger: Tensorboard
> Command Executed:
python legged_lab/scripts/train.py --task=run --headless --logger=tensorboard --num_envs=4096 --resume=True --load_run=2026-03-02_15-36-27 --checkpoint=model_50700.pt
System & Architecture Configuration
Background details of compute resources, neural network parameters, and reward function weights:
Compute Specs
| GPU | NVIDIA RTX 4090 (24GB VRAM) |
| CPU Core Count | 32 Cores |
| OS | Ubuntu 22.04 LTS (WSL2) |
| Simulator | Isaac Lab 4.0 |
| Physics Engine | PhysX 5 (GPU-Accelerated) |
| PyTorch Ver. | 2.1.0+cu121 |
PPO Network Arch
| Algorithm | PPO (Proximal Policy Opt) | |
| Actor Network | MLP [512, 256, 128] + ELU | |
| Critic Network | MLP [512, 256, 128] + ELU | |
| Learning Rate | 0.001 (Adaptive) | |
| Clip Range | 0.2 | |
| Discount Factor (Gamma) | 0.99 | |
| Entropy Coef | 0.01 |
| Term | Weight | Description |
|---|---|---|
| tracking_lin_vel | 1.5 | Reward for following target velocity command |
| tracking_ang_vel | 0.8 | Reward for maintaining angular heading |
| lin_vel_z_penalty | -2.0 | Penalty for bouncing up and down too much |
| ang_vel_xy_penalty | -0.05 | Penalty for rolling or pitching off-axis |
| dof_vel_penalty | -0.001 | Penalty for excessive joint speed (smoothness) |
| dof_acc_penalty | -0.0001 | Penalty for sudden jerks in actuator torque |
| action_rate_penalty | -0.01 | Penalty for changing actions too rapidly |
| base_height |
Reward Terms & Weights
The reward function is engineered to encourage forward movement while minimizing energy consumption and maintaining balance.
| Term | Weight | Description |
|---|---|---|
| tracking_lin_vel | 1.5 | Reward for following target velocity command |
| tracking_ang_vel | 0.8 | Reward for maintaining angular heading |
| lin_vel_z_penalty | -2.0 | Penalty for bouncing up and down too much |
| ang_vel_xy_penalty | -0.05 | Penalty for rolling or pitching off-axis |
| dof_vel_penalty | -0.001 | Penalty for excessive joint speed (smoothness) |
| dof_acc_penalty | -0.0001 | Penalty for sudden jerks in actuator torque |
| action_rate_penalty | -0.01 | Penalty for changing actions too rapidly |
| base_height | -5.0 | Penalty for lowering COM below target height |
Core Engine Source Code
Snippet of the actual Python code from legged_lab/envs/tienkung/run_cfg.py used to
compile the RL environment:
@configclass class LiteRewardCfg: # Exponential tracking terms track_lin_vel_xy_exp = RewTerm(func=mdp.track_lin_vel_xy_yaw_frame_exp, weight=1.0, params={"std": 0.5}) track_ang_vel_z_exp = RewTerm(func=mdp.track_ang_vel_z_world_exp, weight=1.0, params={"std": 0.5}) # Regularization / Penalties lin_vel_z_l2 = RewTerm(func=mdp.lin_vel_z_l2, weight=-1.0) ang_vel_xy_l2 = RewTerm(func=mdp.ang_vel_xy_l2, weight=-0.05) energy = RewTerm(func=mdp.energy, weight=-1e-3) dof_acc_l2 = RewTerm(func=mdp.joint_acc_l2, weight=-2.5e-7) action_rate_l2 = RewTerm(func=mdp.action_rate_l2, weight=-0.01) # Collision and Terminal states undesired_contacts = RewTerm( func=mdp.undesired_contacts, weight=-1.0, params={ "sensor_cfg": SceneEntityCfg( "contact_sensor", body_names=["knee_pitch.*", "shoulder_roll.*", "elbow_pitch.*", "pelvis"] ), "threshold": 1.0, }, ) termination_penalty = RewTerm(func=mdp.is_terminated, weight=-200.0)
Saved Checkpoints Pipeline
PyTorch model (.pt) files preserved during the latest training session:
Live Training Graphs (Interactive TensorBoard)
Interactive visualization of key metrics during parallel execution across 4096 environments: