I led the Llama Mini foundation-model program at Meta, connecting architecture and pretraining with capability transfer and efficient deployment. The released research includes a 1.4T-token initial pretraining stage, simulation-guided data mixtures, specialist checkpoint merging, and quantization-aware training.
I also built on-device foundation models powering Ray-Ban Meta, jointly optimizing model quality, inference latency, memory footprint, and power consumption.
Program lead · Joint-first, joint-last and corresponding author of the released model report.