Top Important LLM Papers for the Week from 13/11 to 19/11

Stay Relevant to Recent Large Language Models Research

Large language models (LLMs) have advanced rapidly in recent years. As new generations of models are developed, it’s important for researchers and engineers to stay informed on the latest progress. This article summarizes some of the most important LLM papers published during the third week of November.

The papers cover various topics shaping the next generation of language models, from model optimization and scaling to reasoning, benchmarking, and enhancing performance. Keeping up with novel LLM research across these domains will help guide continued progress toward models that are more capable, robust, and aligned with human values.

Table of Contents:

1. LLM Progress & Benchmarking

2. LLM Training & Optimization

3. LLM Fine Tuning

4. LLM Reasoning

5. Responsible AI & LLM Ethics

---

1. LLM Progress & Benchmarking

• PEARL: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrievers

• Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities

• Language Models can be Logical Solvers

• ML-Bench: Large Language Models Leverage Open-source Libraries for Machine Learning Tasks

• Llamas Know What GPTs Don’t Show: Surrogate Models for Confidence Estimation

• A Survey on Language Models for Code

• MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks

• Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives

• ChatAnything: Facetime Chat with LLM-Enhanced Personas

• Towards General-Purpose Speech Abilities for Large Language Models Using Unpaired Data

• Fast Chain-of-Thought: A Glance of Future from Parallel Decoding Leads to Answers Faster

• Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs

• The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4

• Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer

---

2. LLM Training & Optimization & Evaluation 

• SelfEval: Leveraging the discriminative nature of generative models for evaluation

• The ART of LLM Refinement: Ask, Refine, and Trust

• Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models

• DiLoCo: Distributed Low-Communication Training of Language Models

• Instruction-Following Evaluation for Large Language Models

• Fusion-Eval: Integrating Evaluators with LLMs

• Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2

---

3. LLM Fine Tuning

• Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization

• Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying

• UT5: Pretraining Non autoregressive T5 with unrolled denoising

• Fine-tuning Language Models for Factuality

---

4. LLM Reasoning

• UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations

• Thread of Thought Unraveling Chaotic Contexts

• ADaPT: As-Needed Decomposition and Planning with Language Models

---

5. LLM Ethics & Safety 

• Testing Language Model Agents Safely in the Wild

• MART: Improving LLM Safety with Multi-round Automatic Red-Teaming

• Technical Report: Large Language Models can Strategically Deceive their Users when Put Under Pressure

• Trusted Source Alignment in Large Language Models

---

6. Transformers & Attention Models 

• Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers

• Frontier Language Models are not Robust to Adversarial Arithmetic, or “What do I need to say so you agree 2+2=5?

• PolyMaX: General Dense Prediction with Mask Transformer

• FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores

• SiRA: Sparse Mixture of Low-Rank Adaptation

• Hiformer: Heterogeneous Feature Interactions Learning with Transformers for Recommender Systems

---

7. Prompt Engineering 

• Contrastive Chain-of-Thought Prompting

•  Prompt Engineering a Prompt Engineer

• LayoutPrompter: Awaken the Design Ability of Large Language Models

---

Are you looking to start a career in data science and AI and do not know how? I offer data science mentoring sessions and long-term career mentoring:

• Mentoring sessions: https://lnkd.in/dXeg3KPW

• Long-term mentoring: https://lnkd.in/dtdUYBrM