Top Important LLM Papers for the Week from 05/02 to 11/02

Stay Updated with Recent Large Language Models Research

Large language models (LLMs) have advanced rapidly in recent years. As new generations of models are developed, researchers and engineers need to stay informed on the latest progress. This article summarizes some of the most important LLM papers published during the Second Week of February 2024.

The papers cover various topics shaping the next generation of language models, from model optimization and scaling to reasoning, benchmarking, and enhancing performance. Keeping up with novel LLM research across these domains will help guide continued progress toward models that are more capable, robust, and aligned with human values.

Table of Contents:

1. LLM Progress & Benchmarking

2. LLM Reasoning

3. LLM Training & Evaluation

4. Transformers & Attention Based Models

---

---

1. LLM Progress & Benchmarking

1. StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback

2. PokéLLMon: A Human-Parity Agent for Pokémon Battles with Large Language Models

3. TravelPlanner: A Benchmark for Real-World Planning with Language Agents

4. Nomic Embed: Training a Reproducible Long Context Text Embedder

5. BlackMamba: Mixture of Experts for State-Space Models

6. OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models

7. Grandmaster-Level Chess Without Search

8. Direct Language Model Alignment from Online AI Feedback

9. Multi-line AI-assisted Code Authoring

10. Code Representation Learning At Scale

11. Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks

12. EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters

13. Scaling Laws for Downstream Task Performance of Large Language Models

14. CodeIt: Self-Improving Language Models with Prioritized Hindsight Replay

15. More Agents Is All You Need

16. An Interactive Agent Foundation Model

17. Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domains

18. In-Context Principle Learning from Mistakes

19. Memory Consolidation Enables Long-Context Video Understanding

20. Driving Everywhere with Large Language Model Policy Adaptation

21. Multilingual E5 Text Embeddings: A Technical Report

22. WebLINX: Real-World Website Navigation with Multi-Turn Dialogue

23. SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models

24. SpiRit-LM: Interleaved Spoken and Written Language Model

2. LLM Reasoning

1. K-Level Reasoning with Large Language Models

2. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

3. Self-Discover: Large Language Models Self-Compose Reasoning Structures

3. LLM Training, Inference, Evaluation & Optimization 

1. Specialized Language Models with Cheap Inference from Limited Domain Data

2. Rethinking Interpretability in the Era of Large Language Models

3. Rethinking Optimization and Architecture for Tiny Language Models

4. LiPO: Listwise Preference Optimization through Learning-to-Rank

5. Shortened LLaMA: A Simple Depth Pruning for Large Language Models

6. BiLLM: Pushing the Limit of Post-Training Quantization for LLMs

7. TP-Aware Dequantization

8. Hydrogen: High-Throughput LLM Inference with Shared Prefixes

9. Offline Actor-Critic Reinforcement Learning Scales to Large Models

4. Transformers & Attention Based Models

1. Repeat After Me: Transformers are Better than State Space Models at Copying

2. Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers

3. The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry

5. LLM Fine-Tuning 

1. Fine-Tuned Language Models Generate Stable Inorganic Materials as Text

---

Are you looking to start a career in data science and AI and do not know how? I offer data science mentoring sessions and long-term career mentoring:

• Mentoring sessions: https://lnkd.in/dXeg3KPW

• Long-term mentoring: https://lnkd.in/dtdUYBrM