Career roadmap
MLOps / AI Platform Engineer
Own the runway: training infrastructure, model serving, GPU cost, monitoring, and rollback for everything AI teams ship.
Before you start MLOps
- Linux and networking
- Docker
- Python
- One cloud and its IAM model
- CI/CD experience
Infrastructure fundamentals
Reproducible infrastructure, defined in code.
Clicking in a console is not a platform.
- State and locking
- Modules
- Plan review in CI
- Drift detection
The substrate most ML platforms run on.
- Deployments and services
- Requests and limits
- HPA and node pools
- GPU device plugins
Least privilege, and no credentials in a container image.
- Workload identity
- Secret managers
- Network boundaries
- Audit logging
Deploys must be boring and identical every time.
- Pipeline as code
- Environment promotion
- Artefact immutability
You cannot control a bill you cannot attribute.
- Tagging conventions
- Budget alerts
- Idle resource reaping
BuildA Terraform-defined cluster a teammate can recreate from scratch with one command.
Model lifecycle
Any training run must be reproducible six months later.
Runs, params, metrics, artefacts — in one place, automatically.
- MLflow or W&B
- Run metadata conventions
- Comparing runs
The contract between the people who train and the people who serve.
- Stages and promotion
- Approval gates
- Model cards
- Rollback to a prior version
Same code plus different data is a different model.
- DVC / LakeFS
- Immutable dataset snapshots
- Lineage from data to model
Pinned everything, seeds fixed, container digest recorded.
- Lockfiles and base images
- Deterministic seeds
- Recording the commit with the run
BuildA pipeline that reproduces a past training run from a commit hash.
Serving & GPUs
Serve models fast, and know the cost per thousand requests.
vLLM for LLMs, Triton or TorchServe for the rest.
- vLLM basics
- Triton model repositories
- Concurrency and queueing
- Health checks
The two levers that decide GPU throughput for LLM serving.
- Continuous batching
- KV cache memory maths
- Prefix caching
- Max sequence length trade-offs
Cheaper inference, measured against an eval set — never assumed.
- INT8 / FP8 / AWQ / GPTQ
- Quality regression testing
- Latency vs accuracy curves
GPUs are the most expensive idle resource you will ever own.
- Scale-to-zero and cold starts
- Queue-depth based scaling
- Spot and preemptible strategies
- Multi-tenancy / MIG
Answer this with a spreadsheet, not an opinion. You will be asked.
Ch 87 — Cost & Latency Optimization- Break-even volume
- Utilisation assumptions
- Engineering time as a cost
- Compliance drivers
BuildA self-hosted open model behind an autoscaling endpoint, with throughput, p95 latency, and cost per 1k requests reported.
Observability & safe release
Catch the regression before the customer writes in.
Model services need the same three pillars as any service, plus quality signals.
- Prometheus and Grafana
- Request tracing
- Token and cost metrics
- SLOs for an AI endpoint
Inputs change before outputs visibly break. Watch the inputs.
- Input distribution monitoring
- Prediction drift
- Proxy quality metrics
- Alert fatigue control
A quality bar the deploy pipeline enforces, with a tolerance you chose deliberately.
Ch 85 — Evaluation & Hallucination- Baseline comparison
- Tolerances that neither block everything nor nothing
- Blocking a merge
5% of traffic, a watched window, then promote or revert automatically.
- Traffic splitting
- SLO watch windows
- Automatic rollback
- Blue/green for weights
Model failures need runbooks too — and a written post-mortem.
- Severity levels
- Runbooks per service
- Post-mortems and guardrails added
BuildA canary deploy that rolls back automatically when an eval gate or latency budget fails.
Governance & cost
The questions finance and compliance will ask, answered in advance.
What the model is, what it was trained on, who approved it.
- Model card contents
- Approval records
- Change history
Prompts, datasets, and weights are all sensitive assets.
- Role-based access
- Prompt and dataset permissions
- Weight exfiltration risk
Per-team visibility, and a hard stop before the quarter is gone.
- Attribution by team and model
- Quota enforcement
- Chargeback reporting
Inference logs contain user data. Decide the window before legal asks.
- Retention windows
- PII scrubbing in logs
- Region constraints
BuildA cost dashboard broken down per model and per team, with an enforced quota.
MLOps tools on your CV
- Terraform
- Kubernetes
- MLflow
- vLLM or Triton
- Prometheus + Grafana
- GitHub Actions
- A cloud with GPU quota
What MLOps employers ask to see
- A repo that stands the platform up from zero
- A cost-per-request number you reduced, with the method
- An incident write-up with the guardrail you added
Demand follows AI adoption with a lag — companies hire this once models are in production and the bill arrives. The natural move for DevOps and SRE engineers; if you already run Kubernetes, you are most of the way there.
Content last reviewed 2026-08-31. Guidance only — no institute or paid placement is endorsed anywhere in this book.