AI Platform Engineer

About the Role We are seeking a passionate AI Platform Engineer to build and own the infrastructure layer that every AI use case in Kuok Group runs on —the LLM gateway, the deployment platform, CI/CD pipelines, model serving, observability, cost controls, and the eval pipeline infrastructure, end to end. This role will be reporting to the Principal AI Architect. This is a T-shaped role: broad cloud and DevOps foundations, with deep specialism in LLM infrastructure. The ideal candidate is equally comfortable provisioning environments and managing release pipelines as they are configuring a model gateway, wiring up LangSmith traces, and buildingan eval harness. Working closely with the Principal AI Architect on architecture direction and with the LLM Ops / MLOps Engineer on the observability and eval layer, this person will be the backbone of the platform that Applied AI Engineers depend on to ship confidently and at pace. . Key Responsibilities Deployment Platform & CI/CD Design, build, and maintain CI/CD pipelines for all AI use cases — from code commit through staging to production, with automated release gates and rollback capability Own environment provisioning and infra-as-code (Terraform or equivalent) — staging, UAT, and production environments should be reproducible, version-controlled, and auditable Manage the deployment platform end to end: release scheduling, environment promotion, incident response, and post-deployment validation Champion good deployment hygiene: automated pipelines, version-controlled configuration, and documented environment differences as standard practice LLM Gateway & Model Serving Build and operate the LLM gateway layer (LiteLLM or equivalent) — API access controls, rate limiting, model routing, and failover across Azure-backed endpoints Manage model serving configuration: endpoint management, load balancing, latency SLOs, and model switching without disrupting live use cases Own secrets and access management for all model API credentials and service accounts across environments Maintain a prompt and model version registry so that every production use case can be traced to a specific model version and prompt configuration Observability, Cost & Controls Instrument all deployed use cases with LLM observability tooling (LangSmith or equivalent)— traces, latency, token counts, and error rates as standard Build and maintain cost telemetry dashboards: per-use-case token consumption, compute spend, and alerting on cost anomalies Implement and maintain token budget controls and rate limits across BUs — keeping cost visible and predictable is a shared responsibility that starts at the platform layer Own general platform monitoring and reliability: uptime, alerting, on-call runbooks, and incident response for platform-layer issues Eval Pipeline Infrastructure Build the infrastructure layer for LLM evaluation pipelines — test harnesses, regression runners, and LLM-as-judge scaffolding used by Applied AI Engineers per use case Work with the LLM Ops / MLOps Engineer on eval pipeline design Ensure eval pipeline runs are logged, versioned, and traceable — eval results should be reproducible Support evals as a consistent deployment gate — working with the team to ensure every use case has a passing eval run on the current model version before moving to production Standards & Collaboration Maintain platform documentation — architecture diagrams, runbooks, environment specs, and onboarding guides — so institutional knowledge is shared and accessible across the team Work within the Principal AI Architect's engineering standards: all platform changes go through code review before deployment Support the QA / Dev Engineers (Applied AI cluster) on integration and regression testing where it touches the platform layer Proactively surface platform-layer risks and capacity constraints to the Principal AI Architect . Requirements Must-Have Solid cloud and DevOps engineering foundations — you have built and operated CI/CD pipelines, managed environments with IaC, and handled production deployments and rollbacks on at least one major cloud platform (Azure, AWS, or GCP);comfortable working across Linux and Windows Server, and familiar with core networking concepts — VPC/VNET, DNS, firewalls, and load balancers Hands-on experience with LLM infrastructure: you have configured and operated a model gateway or API proxy layer, managed multi-model routing, and dealt with rate limits and failover in a live environmen…