AI platform engineer — Roche

Descrizione completa

At Roche you can show up as yourself, embraced for the unique qualities you bring. Our culture encourages personal expression, open dialogue, and genuine connections, where you are valued, accepted and respected for who you are, allowing you to thrive both personally and professionally. This is how we aim to prevent, stop and cure diseases and ensure everyone has access to healthcare today and for generations to come. Join Roche, where every voice matters. The Position AI Platform Engineer Role Overview Own the full lifecycle of a production AI/ML platform spanning on-prem GPU clusters, multi-cloud infrastructure (AWS, Alibaba Cloud), and service delivery. This role bridges datacenter hardware, platform engineering, and AI service operations in a GxP-regulated pharmaceutical environment. You will work closely with global engineering teams, solution architects, and business stakeholders across time zones. Key Responsibilities Infrastructure Engineering (On-Prem & Cloud)

• Own OS baseline: REDHAT Satellite management, custom Base ISO lifecycle
• Integration with enterprise storage systems, which are managed by the Roche Storage team
• GPU server BOM selection,and hardware qualification
• Architect cloud resource strategy: Reserved Instance planning, cost optimization across AWS and Alibaba Cloud
• Cloud Accounts (AWS and Alibaba) Post Previsioning, configuration, and management, for Platform and Platform managed use case accounts Infrastructure as Code (IaC)
• Develop and maintain Ansible scripts for automated server management (Provision, Decommission, Configuration)
• Build and operate AMI Bakery pipelines for immutable image delivery
• Orchestrate multi-cloud server deployments (AWS, Alibaba Cloud) via IaC
• Automate Kubernetes cluster provisioning and management
• Develop and harden custom IAC Scripts MLOps Platform Engineering
• Manage full cluster lifecycle: provisioning, upgrades, scaling, disaster recovery
• Own 30+ platform components across the following domains: ◦ GPU & Device Management ◦ AI Workload Orchestration: Engineering for Kubernetes Scheduling and SLURM Scheduling ◦ Networking: Connectivity Engineering inside Kubernents Cluster ◦ Storage: Storage integration with multiple types, including object storage and block storage. ◦ Observability: Design and implement observability dashboards via Prometheus, Grafana, OpenTelemetry, etc. ◦ Security & PKI: PKI management for the entire platform. Implement dev-sec ops practice into dev-ops lifecycle ◦ Platform Engineering on Data configuration, API, Training/Influencing frameworks, Pipelines, and Toolsets ◦ Build and maintain CI/CD pipelines
• Build and maintain Github/Gitlab templates ◦ Support AI Use Cases on Engineering tasks. ◦ Accountable for troubleshooting platform related issues including leading the troubleshooting across different services that belong to different teams. AI Platform Services
• Deploy and operate AI Gateway (Portkey Data Plane) with full IaC coverage
• Execute on-prem model lifecycle management
• Develop and maintain workspace auto provisioning scripts
• Integrate AI safety guardrails
• Build and implement FinOps process
• Support AI Use Cases on Engineering tasks.
• Accountable for troubleshooting platform related issues including leading the troubleshooting across different services that belong to different teams. Compliance & Process
• Author and maintain system design documents
• Documentation and approval workflow management (via Veeva Quality doc, Markdown in Gitlab for project documents, runbooks, and user manuals.)
• Manage workloads in Jira Requirements Must Have — Technical
• 8+ years in production Linux systems engineering, with deep RHEL expertise (Satellite, Kickstart, custom ISO builds)
• 5+ years operating Kubernetes in production at scale (500+ nodes or 5,000+ pods), including cluster lifecycle management, Disaster recovery, and multi-tenant isolation
• Expert-level IaC proficiency: Ansible (cu