Top LLM Papers for the Week from 26/02 to 03/03

Stay Updated with Recent Large Language Models Research

Large language models (LLMs) have advanced rapidly in recent years. As new generations of models are developed, researchers and engineers need to stay informed on the latest progress. This article summarizes some of the most important LLM papers published during the First Week of March 2024.

The papers cover various topics shaping the next generation of language models, from model optimization and scaling to reasoning, benchmarking, and enhancing performance. Keeping up with novel LLM research across these domains will help guide continued progress toward models that are more capable, robust, and aligned with human values.

Table of Contents:

1. LLM Progress & Benchmarking

2. LLM Reasoning

3. LLM Training, Evaluation & Inference

4. LLM Fine-Tuning 

5. Transformers & Attention Based Models

---

---

1. LLM Progress & Benchmarking

1. Beyond Language Models: Byte Models are Digital World Simulators

2. StarCoder 2 and The Stack v2: The Next Generation

3. Orca-Math: Unlocking the Potential of SLMs in Grade School Math

4. Humanoid Locomotion as Next Token Prediction

5. Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models

6. Priority Sampling of Large Language Models for Compilers

7. The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

8. OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web

9. Nemotron-4 15B Technical Report

10. MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT

11. StructLM: Towards Building Generalist Models for Structured Knowledge Grounding

12. API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs

13. FuseChat: Knowledge Fusion of Chat Models

14. MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases

15. Genie: Generative Interactive Environments

16. Gen4Gen: Generative Data Pipeline for Generative Multi-Concept Composition

17. Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts

18. Watermarking Makes Language Models Radioactive

19. ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition

---

2. LLM Reasoning

1. Do Large Language Models Latently Perform Multi-Hop Reasoning?

2. Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models

---

3. LLM Training, Evaluation & Inference

1. AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning

2. Evaluating Very Long-Term Conversational Memory of LLM Agents

3. Towards Optimal Learning of Language Models

4. Training-Free Long-Context Scaling of Large Language Models

5. MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

6. Divide-or-Conquer? Which Part Should You Distill Your LLM?

7. GPTVQ: The Blessing of Dimensionality for LLM Quantization

---

4. LLM Fine-Tuning

1. DiffuseKronA: A Parameter Efficient Fine-tuning Method for Personalized Diffusion Model

2. When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method

---

5. Transformers & Attention Based Models

1. Simple linear attention language models balance the recall-throughput tradeoff

---

Are you looking to start a career in data science and AI and do not know how? I offer data science mentoring sessions and long-term career mentoring:

• Mentoring sessions: https://lnkd.in/dXeg3KPW

• Long-term mentoring: https://lnkd.in/dtdUYBrM