Hardware and Co-Design of MTIA at Meta

Overview

In this talk from AI Infra @ Scale 2024, Meta's software engineers Joel Colburn, Junqiang Lan, and Jack Montgomery discuss the second generation of MTIA, Meta's in-house training and inference accelerator. They delve into the co-design process involved in building Meta's custom silicon for AI workloads.

Co-Design Process

The talk covers how Meta employed the PyTorch software ecosystem and designed model architectures for key applications to create the second generation of MTIA. The goal was to achieve optimal performance, efficiency, and developer experience for launching models into production.

Achieving Performance and Efficiency

The team demonstrates how MTIA excels in performance and efficiency, contributing to the successful deployment of models. They showcase examples of co-design where special silicon features are leveraged to accelerate Meta's models.

Conclusion

Through meticulous hardware and co-design efforts, Meta has developed a powerful AI accelerator in MTIA. The second generation of this custom silicon promises enhanced performance and efficiency, making it a valuable asset for Meta's AI initiatives.