Senior ML Infrastructure Engineer

Join us at EIT: At the Ellison Institute of Technology (EIT), we’re on a mission to translate scientific discovery into real world impact. We bring together visionary scientists, technologists, policy makers, and entrepreneurs to tackle humanity’s greatest challenges in four transformative areas: Health, Medical Science & Generative Biology Food Security & Sustainable Agriculture Climate Change & Managing CO₂ Artificial Intelligence & Robotics This is ambitious work - work that demands curiosity, courage, and a relentless drive to make a difference. At EIT, you’ll join a community built on excellence, innovation, tenacity, trust, and collaboration, where bold ideas become real-world breakthroughs. Together, we push boundaries, embrace complexity, and create solutions to scale ideas for lab to society. Explore more at www.eit.org Your Role: Join our SciComp team to build the cloud and compute foundation that enables scientific breakthroughs. Deliver reliable, secure platforms and self-service guardrails that accelerate experimentation and turn ideas into results - faster, at scale, and with confidence. Your Responsibilities: Build, operate, and continuously optimise our high-performance GPU training and inference clusters, focusing on robust, high-availability scheduling, isolation, and automated lifecycle management. Drive systems design and implementation for high-throughput data paths, optimising I/O, caching, and data locality across compute and storage (including our current Lustre implementation). Proactively benchmark, profile, and resolve performance bottlenecks across the compute, network, and orchestration layers to maximise efficiency for distributed training and inference. Establish comprehensive observability, resilience, and automated security controls to ensure compliance and robust operation of sensitive research environments. Partner with Research, Data, and Applied teams to forecast capacity and cost for GPU and storage needs, setting quotas and streamlining ML experimentation pipelines. Requirements Essential Skills, Qualifications & Experience: Proven experience leading the design, build, and operation of high-performance ML compute clusters at scale A proactive, autonomous approach to systems design and the proven ability and desire to ideate, co-create and implement optimal solutions Exposure to migrating or transforming ML infrastructure from traditional schedulers to modern, containerised systems Expertise with high-throughput storage systems for ML/HPC workloads Expert-level understanding of GPU architecture, high-speed networking for distributed training, and performance profiling to resolve bottlenecks A solid grasp of IaC and CI/CD practices (e.g., Terraform, Argo CD) Benefits We offer the following salary and benefits: Competitive salary (dependent on experience) + travel allowance + bonus Enhanced holiday. Our annual leave allowance is 25 days plus 8 bank holidays and an additional 3 days between Christmas and New Year. You will also have the opportunity to purchase an additional 5 days annual leave in January and July. Pension - Employer contribution 7.5%, minimum employee contribution 5% Life Assurance. Income Protection Private Medical Insurance as standard for you, your partner and any dependents. Including hospital Cash Plan Employee discounts Electric car scheme Nursery Salary Sacrifice scheme Cycle to Work Scheme Family Planning Neurodiversity support including advise and assessments Coaching & Therapy services Working together - what it involves: You must have the right to work permanently in the UK with a willingness to travel as necessary. In certain cases, we can consider sponsorship, and this will be assessed on a case-by-case basis. You will live in, or within easy commuting distance of, Oxford/London (or be willing to relocate) The SciComp team work to a hybrid working pattern of 3 days in the office, 2x in our Oxford Office and 1x in our London office. You must be able to commit to this should you be successfully appointed.