Skip to content
Source: InfoQ

⚡AWS Lambda Adds GPU Inference With 48 Concurrent Invocations Per Model

Your Lambda functions can finally run models on GPUs

TL;DR

AWS Lambda now runs GPU inference, letting up to 48 concurrent invocations share one model copy in GPU memory. Pricing is EC2 On-Demand plus a 9.75% fee. If you've been pushing inference to dedicated GPU boxes just to avoid cold-start pain, this changes the math.

AWS Lambda can now run GPU inference. Up to 48 concurrent invocations in one environment share a single copy of the model in GPU memory, so you're not paying to load weights 48 times. If you've been running inference on always-on GPU instances because serverless felt impossible for models, this is the first real alternative. You pick the EC2 capacity through a capacity provider, Lambda handles provisioning, patching, and scaling, and there are no CUDA versions or drivers to reconcile. Five base images ship ready to go, including vLLM and SGLang, and swapping models is a config change plus a new version. The numbers: pricing is the EC2 On-Demand rate plus a 9.75% management fee and $0.20 per million requests. Limits include 48 GB accelerator memory, 15,000 vCPUs per capacity provider, and up to 90-minute timeouts for async invocations. Savings Plans and Reserved Instances apply to the underlying instance. The catch: Managed Instances still support only Event Functions, so web-facing GPU workloads aren't covered yet. Watch for that gap closing.

AWS Lambda Adds GPU Inference With 48 Concurrent Invocations Per Model — InfoQ

Key Points

1

Up to 48 concurrent invocations in one environment share a single copy of the model in GPU memory, cutting per-request memory overhead dramatically for inference workloads.

Keep reading — it's free

Enter your email to keep reading — plus the best of AI & tech, daily. Free, forever.

Also get
or

Already a member? Sign in

awslambdagpuinferenceserverless

Comments

Subscribe to join the conversation...

Be the first to comment