Top Important LLM Papers for the Week from 08/01 to 14/01

Stay Updated with Recent Large Language Models Research

Large language models (LLMs) have advanced rapidly in recent years. As new generations of models are developed, researchers and engineers need to stay informed on the latest progress. This article summarizes some of the most important LLM papers published during the Second Week of January 2024.

The papers cover various topics shaping the next generation of language models, from model optimization and scaling to reasoning, benchmarking, and enhancing performance. Keeping up with novel LLM research across these domains will help guide continued progress toward models that are more capable, robust, and aligned with human values.

Table of Contents:

1. LLM Progress & Benchmarking

2. LLM Fine Tuning

3. LLM Reasoning

4. LLM Training & Evaluation

5. Transformers & Attention Based Models

---

---

1. LLM Progress & Benchmarking

1. DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

2. Mixtral of Experts

3. Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache

4. TOFU: A Task of Fictitious Unlearning for LLMs

5. DeepSeek LLM: Scaling Open-Source Language Models with Longtermism

6. LEGO: Language Enhanced Multi-modal Grounding Model

7. Patchscope: A Unifying Framework for Inspecting Hidden Representations of Language Models

8. DocGraphLM: Documental Graph Language Model for Information Extraction

9. A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism

10. Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk

11. Soaring from 4K to 400K: Extending LLM’s Context with Activation Beacon

12. MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

13. Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM

---

2. LLM Fine Tuning

1. Tuning LLMs with Contrastive Alignment Instructions for Machine Translation in Unseen, Low-resource Languages

2. AST-T5: Structure-Aware Pretraining for Code Generation and Understanding

---

3. LLM Reasoning

1. Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding

2. The Impact of Reasoning Step Length on Large Language Models

3. CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution

---

4. LLM Training & Evaluation

1. Narrowing the Knowledge Evaluation Gap: Open-Domain Question Answering with Multi-Granularity Answers

2. Secrets of RLHF in Large Language Models Part II: Reward Modeling

3. Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training

4. Efficient LLM inference solution on Intel GPU

---

5. Transformers & Attention Based Models

1. Transformers are Multi-State RNNs

2. Has Your Pretrained Model Improved? A Multi-head Posterior Based Approach

3. Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

4. Score Distillation Sampling with Learned Manifold Corrective

---

6. LLM Ethics and Trustworthiness

6.1. TrustLLM: Trustworthiness in Large Language Models

---

Are you looking to start a career in data science and AI and do not know how? I offer data science mentoring sessions and long-term career mentoring:

• Mentoring sessions: https://lnkd.in/dXeg3KPW

• Long-term mentoring: https://lnkd.in/dtdUYBrM