Senior ML Engineer | Kimchi (Inference Stack Optimization)

Job Title: Senior ML Engineer | Kimchi (Inference Stack Optimization) Company: CAST AI Job Category: Machine Learning Roles & Responsibilities: #### About the role Move those four numbers in the right direction and two things happen: customers get faster, cheaper inference, and our margins improve. That's the entire thesis of this role. Every loader you tune, every scheduler decision you land, every kernel you stare at shows up in a customer's p99 and on our P\&L. Your mandate is the inference stack's optimization, whatever it takes, and the technical direction of how we optimize it is yours to argue with data. The problem: the checkpoints run 0\.3\-1\.5 TB, and the "right" serving configuration depends on traffic shape, sequence\-length distribution, batch dynamics, GPU SKU, memory bandwidth, and quantization tolerance, all variables that shift week to week. Most teams pick a configuration once, over\-provision GPUs, and absorb the cost. We don't: we took Kimi\-K3 (1\.5 TB) from over an hour cold\-to\-serving to under twelve minutes with a shared filesystem, O\DIRECT weight loaders, and node\-local JIT caches, and we're nowhere near done. Prefill/decode disaggregation and tiered KV caches are next. * You've worked on LLM routing specifically: RouterBench\-class evals, cost/quality/latency Pareto fronts. * Calibration, abstention, or conformal methods in production decision systems. * You've run online A/B loops with drift monitoring on a model that touches real money. #### Responsibilities Work Location: Remote, UK Availability: Full time

#### Requirements Key Skills: Python, PyTorch, Model Training Salary: $50