Extreme-scale Research Lead at LightOn.

🏫 > Education & work experience

  • 📈 | 2020 - Now | Extreme-Scale Project Lead, LightOn. LightOn is funding my Ph.D.

I am leading a team of 9 researchers, engineers, and interns working on furthering, developping, and deploying to the real-world extreme-scale models. We've developped tooling to scrape, filter, and clean trillions of words; trained models with dozens of billions of parameters with Megatron+DeepSpeed on world-class supercomputers; and our inference infrastructure serves millions of words every month to our customers. Our annual compute budget is of 1M V100-h, dedicated to both research and client-focused work. We manage our own research agenda (with papers at NeurIPS, as well as coverage in VentureBeat & ImportAI), and have contributed to the Big Science workshop.

  • 💡 | 2019 – 2020 | Machine Learning Research Scientist, LightOn.

I worked on expanding the applicability of beyond backpropagation methods to modern deep learning tasks and architectures (see our NeurIPS 2020 paper). I helped with the development of optical computing prototypes, achieving scalable optical training of neural networks of varied architectures. This work has lead to applications of Direct Feedback Alignment to adversarial robustness, as well as differential privacy.

  • 🎓 | 2019 – Now | Industrial Ph.D. in Applied Mathematics, École Normale Supérieure, Paris. "Beyond backpropagation: alternative training methods for neural networks".
  • 🌍 | 2018 – 2019 | M.Sc. in Climate Science, École Polytechnique, Palaiseau.
  • 🇭🇰 | 2017 – 2018 | Visiting research student, City University of Hong Kong, Kowloon. "Machine learning for solar engineering".
  • 🌡️ | 2015 – 2019 | M.Sc. in Thermal Engineering, École Normale Supérieure, Paris-Saclay.

📘 > Publications

See my publications page or my Google Scholar profile.

My research has been featured in Yannic Kilcher videos, in Import AI, and news outlets such as VentureBeat (here and there).

🤗 > Service

  • 🌸 | May 2021 - Now | BigScience. Architecture & Scaling Working Group Chair.

I am chairing the architecture & scaling working group for the Big Science workshop. Our goal is to empirically explore and validate architectural choices for the 200B+ parameters multilingual model that will be trained at the end of the project. We are studying considerations around model architecture & training objectives (encoder-decoder vs decoder-only, denoising vs language modelling), embeddings (rotary vs ALiBi), as well as multilinguality.

  • ✍️ | July/August 2021 | NeurIPS 2021. Reviewer. Outstanding Reviewer Award.
  • 🛥️ | June 2019 | MOOSE-GE scientific campaign, Mediterranean Sea, Thalassa vessel, 2 weeks. Science crew member. Double-Diffusive Processes in the Tyrrhenian Sea.

👨‍🏫 > Outreach

🎫 > Beyond academia

  • 🤿 | Diving. I am a passionate DIR diver, and I am working towards a GUE Fundamentals technical pass, with a strong interest for cave diving. I have a strong interest in cave and exploration diving. I am also preparing a FFESM N3 licence (down to 60m depth), and an E1 instructor licence (teaching down to 8m depth).
  • 🪁 | Paramotoring. I recently got into interested in paramotoring and paragliding. I am in the process of getting my pilot certification for paramotoring.
  • 👨🏻‍🍳 | Cooking. In particular sous-vide cooking, new cookery, and holistic cuisine. During 2020 lockdowns, I cooked my way through the Fat Duck Cookbook & the Eleven Madison Park cookbooks.
  • 🗺️ | Travel/adventures. In 2016, I drove 3,000km on a rickshaw in India, going from Shillong to Kochi, and raised 2,000€ for Cool Earth. In 2018, I took a motorbike around Java, Bali, and Lombok in Indonesia. I have also done roadtrips in Yucatán, Taiwan, Vietnam, Iceland, and Europe.
You’ve successfully subscribed to Julien Launay
Welcome back! You’ve successfully signed in.
Great! You’ve successfully signed up.
Success! Your email is updated.
Your link has expired
Success! Check your email for magic link to sign-in.