Top Important LLM Papers for the Week from 11/12 to 17/12

Stay Updated with Recent Large Language Models Research

Large language models (LLMs) have advanced rapidly in recent years. As new generations of models are developed, researchers and engineers need to stay informed on the latest progress. This article summarizes some of the most important LLM papers published during the third week of December.

The papers cover various topics shaping the next generation of language models, from model optimization and scaling to reasoning, benchmarking, and enhancing performance. Keeping up with novel LLM research across these domains will help guide continued progress toward models that are more capable, robust, and aligned with human values.

Table of Contents:

1. LLM Progress & Benchmarking

2. LLM Fine Tuning

3. LLM Reasoning

4. LLM Training & Evaluation 

5. Responsible AI & LLM Ethics

6. Transformers & Attention Models

---

---

1. LLM Progress & Benchmarking

1. Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

2. LLM360: Towards Fully Transparent Open-Source LLMs

3. Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models

4. Evaluation of Large Language Models for Decision Making in Autonomous Driving

5. Unlocking Anticipatory Text Generation: A Constrained Approach for Faithful Decoding with Large Language Models

6. “I Want It That Way”: Enabling Interactive Decision Support Using Large Language Models and Constraint Programming

7. Interfacing Foundation Models’ Embeddings

8. Rethinking Compression: Reduced Order Modelling of Latent Features in Large Language Models

9. Honeybee: Locality-enhanced Projector for Multimodal LLM

10. TinyGSM: achieving >80% on GSM8k with small language models

2. LLM Fine Tuning

1. Context Tuning for Retrieval Augmented Generation

2. Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes

3. Distributed Inference and Fine-tuning of Large Language Models Over The Internet

4. ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks

3. LLM Reasoning

1. Modeling Complex Mathematical Reasoning via Large Language Model-based MathAgentTigerBot: An Open Multilingual Multitask LLM

2. PathFinder: Guided Search over Multi-Step Reasoning Paths

4. LLM Training & Evaluation 

1. SparQ Attention: Bandwidth-Efficient LLM Inference

2. EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism

3. Order Matters in the Presence of Dataset Imbalance for Multilingual Learning

4. PromptBench: A Unified Library for Evaluation of Large Language Models

5. Steering Llama 2 via Contrastive Activation Addition

5. Responsible AI & LLM Ethics

1. Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations

2. Alignment for Honesty

3. Using Captum to Explain Generative Language Models

6. Transformers & Attention Models

1. TCNCA: Temporal Convolution Network with Chunked Attention for Scalable Sequence Processing

2. Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention

3. Invariant Graph Transformer

4. SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention

---

Are you looking to start a career in data science and AI and do not know how? I offer data science mentoring sessions and long-term career mentoring:

• Mentoring sessions: https://lnkd.in/dXeg3KPW

• Long-term mentoring: https://lnkd.in/dtdUYBrM