Trends in AI — June 2024

OpenAI has released GPT-4 Omni, a new multimodal model that can process and produce text, image, and audio tokens, but the company is still embroiled in drama. Google and Microsoft also showed their cards, with multiple AI-powered goodies at their annual events, while other AI startups are furiously raising hundreds of millions to deliver the next breakthrough. Join us for an overview of the latest developments in AI R&D and a curated list of the month's top 10 trending research papers.

---

News Articles

• OpenAI’s Co-Founder and Chief Scientist Ilya Sutskever Is Leaving the Company

• ChatGPT suspends Scarlett Johansson-like voice as actor speaks out against OpenAI

• Microsoft announces Copilot Plus PCs with built-in AI hardware

• Unitree Robotics unveils G1 humanoid for $16k

• Elon Musk’s xAI raises $6 billion from Sequoia, Andreessen, and Saudi royals as it hails ‘significant strides’ in AI research

• Data-labeling startup Scale AI raises $1B as valuation doubles to $13.8B

• Ex-Deepmind scientists raise $220m to launch Paris-based agentic AI startup H

Model Releases

• Google: AlphaFold 3, PaliGemma, Gemini Flash

• Mistral AI: Codestral

• OpenAI: GPT-4o

• 01.AI: Yi 1.5

• TII UAE: Falcon 2

Trending AI papers for June 2024

1. Accurate structure prediction of biomolecular interactions with AlphaFold 3 - J. Abramson et al. (Google DeepMind) - 8 May 2024

2. xLSTM: Extended Long Short-Term Memory - M. Beck et al. (LIT AI, NXAI) - 7 May 2024

3. gzip Predicts Data-dependent Scaling Laws - R. Pandey (Reworkd) - 26 May 2024

4. The Platonic Representation Hypothesis - M. Huh et al. (MIT) - 13 May 2024

5. Faithful Logical Reasoning via Symbolic Chain-of-Thought - J. Xu et al. (NUS) - 28 May 2024

6. Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? - Z. Gekhman et al. (Technion, Google) - 9 May 2024

7. Chameleon: Mixed-Modal Early-Fusion Foundation Models - Chameleon Team (Meta) - 16 May 2024

8. Contextual Position Encoding: Learning to Count What's Important - O. Golovneva et al. (FAIR) - 28 May 2024

9. NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models - C. Lee et al. (NVIDIA) - 27 May 2024

10. DeeperImpact: Optimizing Sparse Learned Index Structures - S. Basnet et al. (NYU Abu Dhabi) - 27 May 2024

And a few runner-ups:

• Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality - T. Dao & A. Gu (Princeton University, CMU) - 31 May 2024

• Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities - V. Zayats et al. (Google DeepMind) - 28 May 2024

• Your Transformer is Secretly Linear - A. Razzhigaev et al. (AIRI, Skoltech) - 19 May 2024

You can find an annotated collection of these papers in Zeta Alpha, allowing you to effortlessly discover relevant literature and dive deeper into any topic that interests you!

The full-length recording of our June 2024 Trends in AI webinar is available on our YouTube channel! For a short video overview describing the trending papers, check this out:

To join us live for the next installment of the Trends in AI webinar, sign up here.