Lead/Tech Expert - Global Live (LLM Model Serving)

About TikTok TikTok is the leading destination for short-form mobile video. At TikTok, our mission is to inspire creativity and bring joy. TikTok's global headquarters are in Los Angeles and Singapore, and we also have offices in New York City, London, Dublin, Paris, Berlin, Dubai, Jakarta, Seoul, and Tokyo.​ Why Join Us Inspiring creativity is at the core of TikTok's mission. Our innovative product is built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and bring joy - a mission we work towards every day.​ We strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. Every challenge is an opportunity to learn and innovate as one team. We're resilient and embrace challenges as they come. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our company, and our users. When we create and grow together, the possibilities are limitless. Join us.​ Diversity & Inclusion​ TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At TikTok, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.​ About Our Team Mission of Global Live Service Architecture team is Build Real-time Interactive Architecture, Safeguard Global LIVE. We are seeking highly skilled and experienced Expert/Senior Engineers to join our TikTok Live Architecture team. TikTok Live is a world-wide leader in live streaming, which occupies more than 50% of the market share. In the LLM team, you have the chance to understand the most advanced LLM models, and design architecture to apply LLM in the world 's largest businesses. We're people at the forefront of the world. Responsibilities: Model Service Deployment: Responsible for converting large-scale deep learning models into scalable services that meet the diverse needs of TikTok's live streaming business. Performance Optimization: Optimize the performance of model inference, including but not limited to efficient utilization of computing resources, minimizing response time, and maximizing throughput. Cross-Team Collaboration: Work closely with algorithm and business teams to facilitate the deployment of models into production and resolve issues that arise in the production environment. Technical Innovation: Continuously monitor and explore new technologies and methods in the AI field to drive technological advancement in model services. Qualifications Minimum Qualifications: Bachelor's degree or higher in Computer Science, Software Engineering, Artificial Intelligence, or related fields. 3+ years of relevant work experience, with experience in deploying and servicing large-scale machine learning models. Proficiency in mainstream deep learning frameworks (such as TensorFlow, PyTorch, DeepSpeed) and their deployment in production environments. Familiarity with model inference optimization techniques, such as quantization, distillation, distributed inference, ONNX, ZeRO, etc. Familiarity with online service tech stacks, such as RPC, Redis, Kafka, etc. Strong programming skills, proficient in Python, C++ or Golang, with a deep understanding of system performance optimization. Preferred Qualification: Have LLMs deployment and optimization experience