LLM-Drop
Collection
Model weights of paper "What Matters in Transformers? Not All Attention is Needed" (https://arxiv.org/abs/2406.15786) • 15 items • Updated • 4
LLM-Drop
community
AI & ML interests
None defined yet.
Recent Activity
Papers
Demystifying When Pruning Works via Representation Hierarchies
Understanding and Harnessing Sparsity in Unified Multimodal Models
s1ghhh
updated a
collection about 5 hours ago
shwai-he
published 2
models 3 days ago
shwai-he
updated 2
models 3 days ago
shwai-he
submitted a
paper to Daily Papers 5 days ago
s1ghhh
updated a
collection over 1 year ago
s1ghhh
authored a
paper over 1 year ago
shwai-he
authored 9
papers over 1 year ago
Router-Tuning: A Simple and Effective Approach for Enabling Dynamic-Depth in Transformers
Paper • 2410.13184 • Published • 3
Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning
Paper • 2310.11716 • Published • 6
Merging Experts into One: Improving Computational Efficiency of Mixture of Experts
Paper • 2310.09832 • Published • 1
Vega-MT: The JD Explore Academy Translation System for WMT22
Paper • 2209.09444 • Published
MerA: Merging Pretrained Adapters For Few-Shot Learning
Paper • 2308.15982 • Published
Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning
Paper • 2402.00530 • Published • 2
Reformatted Alignment
Paper • 2402.12219 • Published • 17
Loki: Low-Rank Keys for Efficient Sparse Attention
Paper • 2406.02542 • Published
SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters
Paper • 2210.04284 • Published