Head of ML Infrastructure Job at Hippocratic AI, Palo Alto, CA

NzlzL1hjZEZUS2cyRG5NU3Q3RUFmS0lNeHc9PQ==
  • Hippocratic AI
  • Palo Alto, CA

Job Description

About Us:

Hippocratic AI has developed a safety-focused Large Language Model (LLM) for healthcare. The company believes that a safe LLM can dramatically improve healthcare accessibility and health outcomes in the world by bringing deep healthcare expertise to every human. No other technology has the potential to have this level of global impact on health. 

Why Join Our Team:

  • Innovative Mission: We are developing a safe, healthcare-focused large language model (LLM) designed to revolutionize health outcomes on a global scale.

  • Visionary Leadership: Hippocratic AI was co-founded by CEO Munjal Shah, alongside a group of physicians, hospital administrators, healthcare professionals, and artificial intelligence researchers from leading institutions, including El Camino Health, Johns Hopkins, Stanford, Microsoft, Google, and NVIDIA.

  • Strategic Investors: We have raised a total of $278 million in funding, backed by top investors such as Andreessen Horowitz, General Catalyst, Kleiner Perkins, NVIDIA’s NVentures, Premji Invest, SV Angel, and six health systems.

  • World-Class Team: Our team is composed of leading experts in healthcare and artificial intelligence, ensuring our technology is safe, effective, and capable of delivering meaningful improvements to healthcare delivery and outcomes.

Position Overview:

We are seeking a highly skilled and innovative Head of ML Infrastructure to lead the design, development, and operation of our orchestration platform for a heterogeneous constellation of Large Language Models (LLMs). The ideal candidate will have deep expertise in infrastructure orchestration, multi-cloud environments, and tools such as Kubernetes and Terraform. This role is critical to ensuring that our AI systems are scalable, reliable, and seamlessly integrated into our broader technology ecosystem.

Key Responsibilities:

Orchestration Platform Development:

• Architect and implement an advanced orchestration platform to manage a diverse set of LLMs efficiently.
• Design solutions to optimize performance, scalability, and availability across various deployment environments.

Infrastructure Management:

• Utilize Kubernetes, Terraform, and other Infrastructure as Code (IAC) tools to automate and manage ML infrastructure.
• Collaborate with DevOps and cloud engineering teams to ensure seamless integration with CI/CD pipelines.
• Establish robust monitoring, logging, and alerting systems for ML infrastructure.

Multi-Cloud Strategy:

• Design and execute strategies to leverage multiple cloud providers for cost optimization, redundancy, and compliance.
• Manage cloud-native services to support model deployment and orchestration at scale.

Performance Optimization:

• Work closely with ML engineers to fine-tune model deployment strategies, focusing on latency, throughput, and fault tolerance.
• Conduct capacity planning and develop tools for model lifecycle management.

Leadership & Collaboration:

• Lead a team of infrastructure engineers, fostering a culture of innovation, collaboration, and excellence.
• Act as a bridge between ML research, engineering, and operations teams to align infrastructure capabilities with business needs.
• Stay abreast of emerging technologies and methodologies in ML infrastructure and orchestration.

Qualifications:

Technical Skills:

• Proven experience in building and managing ML infrastructure platforms, particularly for LLMs or other advanced AI systems.
• Expertise in Kubernetes, Terraform, and other IAC tools.
• Deep understanding of multi-cloud architectures (e.g., AWS, Azure, Google Cloud) and hybrid cloud solutions.
• Strong programming skills in Python, Go, or a similar language, with experience in building automation and orchestration tools.
• Familiarity with modern ML frameworks and tools (e.g., TensorFlow, PyTorch, Hugging Face).

Leadership & Communication:

  • Demonstrated success in leading infrastructure teams and managing large-scale projects

  • Excellent problem-solving and decision-making skills.

Strong communication skills, with the ability to convey complex technical ideas to non-technical stakeholders.

Education & Experience:

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field (or equivalent work experience).

  • 8+ years of experience in infrastructure engineering, with at least 3 years in a leadership

Job Tags

Work experience placement,

Similar Jobs

SMK Services, Inc.

Construction Project Manager Job at SMK Services, Inc.

 ...What We Do: SMK Services, Inc. (SMK) is a Construction Management and Engineering services...  ...environment where we take pride in the work we do and celebrate our collective success...  ...transit). The ideal candidate should have experience in Civil and Structural construction... 

US Federal Government

Legal Assistant Paralegal Job at US Federal Government

 ...Legal Assistant or Law Clerk/ Federal Government - On-Site- Arlington, VA ** 9-Month CONTRACT POSITION** Full-Time/40 hours per week General Counsel's office of a federal government agency needs an experienced Legal Assistant or Law Clerk for a 9-month contract... 

Delta Electronics Americas

Legal Counsel Job at Delta Electronics Americas

 ...Legal Counsel Delta Electronics Americas Fremont, CA, United States (On-site) Legal Counsel Company and BG Description : Founded...  ...legal forms relevant to the in-house context, such as leases, consulting and other independent contractor agreements, employee handbooks... 

First Title & Escrow

Title Clearance Specialist Job at First Title & Escrow

 ...Title Clearance Specialist 9:30am-6pm EST Help us clear the way for seamless real estate closings, one transaction at a time....  ...First Title First Title is a technology-forward leader in title insurance and closing services. Headquartered near Washington, D.C., with... 

ACL Digital

Video Editor / Animator Job at ACL Digital

Title: Video Editor / Animator Multimedia Designer (Sr. Staff) Location: San Diego, CA (92121) Type: Contract | On-call / As-needed We are seeking a highly creative and skilled Video Editor/Animator to support multimedia and video production projects for a...