Trends in AI — April 2024

Big departures and transfers in the frontier AI lab market, new hardware announcements from Big Tech & startups, and the first AI software engineer contributing to open-source! Join us for an overview of the latest breakthroughs and developments in the industry, along with a curated list of the top 10 trending research papers of the month.

---

News Articles

• Nvidia unveils Blackwell B200, the “world’s most powerful chip” designed for AI

• Chip startup Cerebras launches new AI processor

• Introducing Devin, the first AI software engineer

• Stability AI CEO Emad Mostaque resigns

• Microsoft Hires DeepMind Co-Founder to Run Consumer A.I.

• Neuralink's first human patient able to control mouse through thinking, Musk says

• Amazon spends $2.75 billion on AI startup Anthropic in its largest venture investment yet

Model Releases

• Cohere: Command R, Command R+

• Stability AI: Stable Code Instruct 3B, Stable Audio 2.0

• Databricks: DBRX

• mixedbread: mxbai-embed-2d-large-v1, mxbai-embed-large-v1, mxbai-colbert-large-v1

• Unum: UForm

Trending AI papers for April 2024

1. Gecko: Versatile Text Embeddings Distilled from Large Language Models - Jinhyuk Lee et al. (Google DeepMind) - 29 Mar. 2024

2. FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions - Orion Weller et al. (John Hopkins University) - 22 Mar. 2024

3. RAFT: Adapting Language Model to Domain Specific RAG - Tianjun Zhang et al. (UC Berkeley) - 15 Mar. 2024

4. Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity - Soyeong Jeong et al. (Korea Advanced Institute of Science and Technology) - 21 Mar. 2024

5. MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training - Brandon McKinzie et al. (Apple) - 14 Mar. 2024

6. Jamba: A Hybrid Transformer-Mamba Language Model - Opher Lieber et al. (AI21 Labs) - 28 Mar. 2024

7. Mixture-of-Depths: Dynamically allocating compute in transformer-based language models - David Raposo et al. (Google DeepMind) - 2 Apr. 2024

8. Long-form factuality in large language models - Jerry Wei et al. (Google DeepMind) - 27 Mar. 2024

9. Humanoid Locomotion as Next Token Prediction - Ilija Radosavovicet al. (UC Berkeley) - 29 Feb. 2024

10. Stealing Part of a Production Language Model - Nicholas Carlini et al. (Google DeepMind) - 11 Mar. 2024

And a few runner-ups:

• Privacy Backdoors: Stealing Data with Corrupted Pretrained Models - Shanglun Feng & Florian Tramèr (ETH Zurich) - 30 Mar. 2024

• Logits of API-Protected LLMs Leak Proprietary Information - Matthew Finlayson et al. (USC) - 14 Mar. 2024

• AutoDev: Automated AI-Driven Development - Michele Tufano et al. (Microsoft) - 13 Mar. 2024

You can find an annotated collection of these papers in Zeta Alpha, allowing you to effortlessly discover relevant literature and dive deeper into any topic that interests you!

The full-length recording of our April 2024 Trends in AI webinar is available on our YouTube channel. For a short video overview describing the trending papers, check this out:

To join us live for the next installment of the Trends in AI webinar, sign up here.