Job Description
About The Role
Lead Inference Platform Engineer – specialized experience in machine learning/deep learning domains such as model compression, hardware‑aware model optimizations, hardware accelerators architecture, GPU/ASIC architecture, ML compilers, high‑performance computing, performance optimizations, numerics, or SW/HW co‑design.
Responsibilities
- Optimize LLMs and ML models for high‑performance inference using quantization, pruning, distillation, and hardware specific tuning.
- Deploy and scale inference workloads on GPUs across AWS, Azure, GCP and internal Kubernetes clusters, ensuring predictable performance during peak traffic.
- Implement routing and fail‑over strategies for OpenAI / Anthropic / Vertex AI traffic.
- Integrate models into production‑grade APIs supporting TR products and enterprise workflows.
- Develop highly optimized environments and eliminate performance bottlenecks to reduce latency.