Batch (AWS)

AWS Batch automates the provisioning and scaling of compute resources to execute thousands of containerized batch jobs across Amazon ECS, EKS, and Fargate.

Batch eliminates the heavy lifting of manual cluster management by dynamically scaling EC2 instances based on job volume and resource requirements (vCPU, memory, and GPU). It manages the entire lifecycle of high-performance computing workloads, utilizing a priority-based queuing system to dispatch jobs to Spot or On-Demand instances. Developers define job definitions as Docker containers, allowing Batch to handle the underlying infrastructure orchestration for everything from genomic analysis to financial risk modeling. By integrating directly with AWS Step Functions and EventBridge, it transforms complex shell scripts into robust, distributed workflows without requiring a dedicated scheduler like Slurm or HTCondor.

https://aws.amazon.com/batch/