CartPole

A foundational reinforcement learning environment where an agent balances a vertical pole on a motorized cart using discrete horizontal impulses.

CartPole is a classic control problem and the de facto Hello World for reinforcement learning (RL) practitioners. Originally described by Barto, Sutton, and Anderson, the task requires an agent to maintain a pole's vertical balance by applying a force of +1 or -1 to a cart on a frictionless track. The environment provides a four element observation vector: cart position, cart velocity, pole angle, and pole angular velocity. Success is defined by reaching a specific reward threshold (typically 475 points in the v1 environment) before the pole tips beyond 12 degrees or the cart moves past 2.4 units from the center. It remains the primary benchmark for testing the convergence and stability of algorithms like Deep Q-Networks (DQN) and Proximal Policy Optimization (PPO).

https://gymnasium.farama.org/environments/classic_control/cart_pole/