Files

papers
- 2016
- 2017
- 2019
- 2020
- 2021
- 2022
- 2023
  - 230110 Scaling Laws for Generative Mixed-Modal Language Models.md
  - 230113 YOLOv6 v3.0.md
  - 230118 Learning-Rate-Free Learning by D-Adaptation.md
  - 230120 Is ChatGPT A Good Translator.md
  - 230121 SuperScaler.md
  - 230123 StyleGAN-T.md
  - 230123 Zorro.md
  - 230124 A Watermark for Large Language Models.md
  - 230126 DetectGPT.md
  - 230126 MusicLM.md
  - 230126 On the Importance of Noise Scheduling for Diffusion Models.md
  - 230126 simple diffusion.md
  - 230130 Alternating Updates for Efficient Transformers.md
  - 230131 Attend-and-Excite.md
  - 230131 Faithful Chain-of-Thought Reasoning.md
  - 230131 Grounding Language Models to Images for Multimodal Generation.md
  - 230131 In-Context Retrieval-Augmented Language Models.md
  - 230131 Large Language Models Can Be Easily Distracted by Irrelevant Context.md
  - 230131 The Flan Collection.md
  - 230201 mPLUG-2.md
  - 230202 Accelerating Large Language Model Decoding with Speculative Sampling.md
  - 230202 Dreamix.md
  - 230202 Dual PatchNorm.md
  - 230202 Factor Fields.md
  - 230202 Multimodal Chain-of-Thought Reasoning in Language Models.md
  - 230203 Measuring The Impact Of Programming Language Distribution.md
  - 230203 PyGlove.md
  - 230204 AV-NeRF.md
  - 230204 Representation Deficiency in Masked Language Modeling.md
  - 230205 Design Booster.md
  - 230206 Chain of Hindsight Aligns Language Models with Feedback.md
  - 230206 SmoothQuant.md
  - 230206 Zero-shot Image-to-Image Translation.md
  - 230207 Efficiently Upgrading Multilingual Machine Translation Models to Support More Languages.md
  - 230207 Exploring the Benefits of Training Expert Language Models over Instruction Tuning.md
  - 230207 FP8.md
  - 230207 Hard Prompts Made Easy.md
  - 230207 Long Horizon Temperature Scaling.md
  - 230208 Google Configuration System.md
  - 230208 Nerfstudio.md
  - 230208 Noise2Music.md
  - 230208 Q-Diffusion.md
  - 230209 Efficient Attention via Control Variates.md
  - 230209 In-Context Learning with Many Demonstration Examples.md
  - 230209 In-N-Out.md
  - 230209 Re-ViLM.md
  - 230209 Toolformer.md
  - 230210 The Wisdom of Hindsight Makes Language Models Better Instruction Followers.md
  - 230211 Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models.md
  - 230211 Generative AI에 대한 생각.md
  - 230212 I$^2$SB.md
  - 230213 3D-aware Blending with Generative NeRFs.md
  - 230213 Lossy Compression.md
  - 230213 Simple Hardware-Efficient Long Convolutions for Sequence Modeling.md
  - 230213 Symbolic Discovery of Optimization Algorithms.md
  - 230214 Adding Instructions during Pretraining.md
  - 230214 Score-based Diffusion Models in Function Space.md
  - 230215 Learning Performance-Improving Code Edits.md
  - 230215 PRedItOR.md
  - 230215 The Capacity for Moral Self-Correction in Large Language Models.md
  - 230216 3D-aware Conditional Image Synthesis.md
  - 230216 Aligning Language Models with Preferences through f-divergence Minimization.md
  - 230216 Decoupled Model Schedule for Deep Learning Training.md
  - 230216 MultiDiffusion.md
  - 230216 Pretraining Language Models with Human Preferences.md
  - 230216 Shared Microexponents.md
  - 230219 Scaling Laws for Multilingual Neural Machine Translation.md
  - 230219 Semantic Uncertainty.md
  - 230220 Composer.md
  - 230220 DSP.md
  - 230220 Poisoning Web-Scale Training Datasets is Practical.md
  - 230220 TA-MoE.md
  - 230221 Anthropic.md
  - 230221 ChatGPT.md
  - 230221 Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels.md
  - 230221 Hyena Hierarchy.md
  - 230221 On Calibrating Diffusion Probabilistic Models.md
  - 230222 AlpaServe.md
  - 230222 FlexGen.md
  - 230223 Colossal AI ChatGPT.md
  - 230223 Controlled and Conditional Text to Image Generation with Diffusion Prior.md
  - 230223 On the Generalization Ability of Retrieval-Enhanced Transformers.md
  - 230224 Check Your Facts and Try Again.md
  - 230224 World Models.md
  - 230227 ELITE.md
  - 230228 Large Language Models Are State-of-the-Art Evaluators of Translation Quality.md
  - 230228 SHP.md
  - 230301 StraIT.md
  - 230301 Unlimited-Size Diffusion Restoration.md
  - 230302 3D generation on ImageNet.md
  - 230302 Consistency Models.md
  - 230302 Dropout Reduces Underfitting.md
  - 230302 Google USM.md
  - 230302 Image as Set of Points.md
  - 230304 Prismer.md
  - 230306 PaLM-E.md
  - 230306.md
  - 230307 Flamingo.md
  - 230307 Is ChatGPT a Good NLG Evaluator.md
  - 230307 Larger language models do in-context learning differently.md
  - 230307 TRACT.md
  - 230307 The BigScience ROOTS Corpus.md
  - 230308 Visual ChatGPT.md
  - 230309 Cones.md
  - 230309 Scaling up GANs for Text-to-Image Synthesis.md
  - 230310 Towards MoE Deployment.md
  - 230311 A Novel Tensor-Expert Hybrid Parallelism Approach to Scale Mixture-of-Experts Training.md
  - 230311 Regularized Vector Quantization for Tokenized Image Synthesis.md
  - 230311 Resurrecting Recurrent Neural Networks for Long Sequences.md
  - 230311 Stabilizing Transformer Training by Preventing Attention Entropy Collapse.md
  - 230312 ChatGPT Asks, BLIP-2 Answers.md
  - 230313 High-throughput Generative Inference of Large Language Models with a Single GPU.md
  - 230313 Scaling Vision-Language Models with Sparse Mixture of Experts.md
  - 230314 ViperGPT.md
  - 230315 A Comprehensive Study on Post-Training Quantization for Large Language Models.md
  - 230315 GPT-4.md
  - 230316 $P+$.md
  - 230316 ART.md
  - 230316 Efficient Diffusion Training via Min-SNR Weighting Strategy.md
  - 230317 CoLT5.md
  - 230317 GPTs are GPTs.md
  - 230317 On the De-duplication of LAION-2B.md
  - 230320 Reflexion.md
  - 230320 SVDiff.md
  - 230322 MEGA.md
  - 230322 RepoCoder.md
  - 230322 Sparks of Artificial General Intelligence.md
  - 230323 The Quantization Model of Neural Scaling.md
  - 230324 Scaling Expert Language Models with Unsupervised Domain Discovery.md
  - 230327 EVA-CLIP.md
  - 230327 Sigmoid Loss for Language Image Pre-Training.md
  - 230327 unarXive 2022.md
  - 230328 Improving Code Generation by Training with Natural Language Feedback.md
  - 230331 Autoregressive Model.md
  - 230331 Choose Your Weapon.md
  - 230405 Generative Novel View Synthesis with 3D-Aware Diffusion Models.md
  - 230405 Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models.md
  - 230406 Diffusion Models as Masked Autoencoders.md
  - 230406 InstantBooth.md
  - 230406 Instruction Tuning with GPT-4.md
  - 230406 On the Pareto Front of Multilingual Neural Machine Translation.md
  - 230406 Quantization.md
  - 230407 Generative Agents.md
  - 230407 RLHF.md
  - 230410 Inference with Reference.md
  - 230411 RRHF.md
  - 230413 Zip-NeRF.md
  - 230414 DINOv2.md
  - 230414 OpenAssistant Conversations -- Democratizing Large Language Model Alignment.md
  - 230416 Open Assistant.md
  - 230416 Sabiá.md
  - 230417 A Comparative Study between Full-Parameter and LoRA-based Fine-Tuning on Chinese Instruction Data for Instruction Following Large Language Model.md
  - 230417 Low-code LLM.md
  - 230417 Tool Learning with Foundation Models.md
  - 230418 HCI.md
  - 230418 Hyperbolic Image-Text Representations.md
  - 230418 UniMax.md
  - 230419 A Theory on Adam Instability in Large-Scale Machine Learning.md
  - 230419 Scaling Transformer to 1M tokens and beyond with RMT.md
  - 230420 Stable LM.md
  - 230421 Can GPT-4 Perform Neural Architecture Search.md
  - 230421 Evaluating Transformer Language Models on Arithmetic Operations Using Number Decomposition.md
  - 230421 Inducing anxiety in large language models increases exploration and bias.md
  - 230424 Why we need RLHF.md
  - 230425 Stable and low-precision training for large-scale vision-language models.md
  - 230428 Are Emergent Abilities of Large Language Models a Mirage.md
  - 230428 CCpdf.md
  - 230428 Causal Reasoning and Large Language Models.md
  - 230428 Empirical Analysis of the Strengths and Weaknesses of PEFT Techniques for LLMs.md
  - 230428 ResiDual.md
  - 230428 Speak, Memory.md
  - 230501 In-Context Learning Unlocked for Diffusion Models.md
  - 230501 What Do Self-Supervised Vision Transformers Learn.md
  - 230502 Pick-a-Pic.md
  - 230502 RedPajama.md
  - 230502 Unlimiformer.md
  - 230503 3D Gaussian Splatting for Real-Time Radiance Field Rendering.md
  - 230503 Cheaply Evaluating Inference Efficiency Metrics for Autoregressive Transformer APIs.md
  - 230503 CodeGen2.md
  - 230504 BranchNorm.md
  - 230504 Can LLM Already Serve as A Database Interface.md
  - 230504 On the Expressivity Role of LayerNorm in Transformers' Attention.md
  - 230504 Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision.md
  - 230504 ZipIt! Merging Models from Different Tasks without Training.md
  - 230507 Vcc.md
  - 230507 X-LLM.md
  - 230509 Large Language Model Programs.md
  - 230509 MoT.md
  - 230510 Do LLMs Understand User Preferences.md
  - 230511 An Inverse Scaling Law for CLIP Training.md
  - 230511 Chain-of-Dictionary Prompting Elicits Translation in Large Language Models.md
  - 230511 CoMoSpeech.md
  - 230511 INGENIOUS.md
  - 230511 InstructBLIP.md
  - 230511 Musketeer (All for One, and One for All).md
  - 230511 Not All Languages Are Created Equal in LLMs.md
  - 230511 Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformers.md
  - 230511 Simple Token-Level Confidence Improves Caption Correctness.md
  - 230512 MEGABYTE.md
  - 230512 TinyStories.md
  - 230513 CodeT5+.md
  - 230513 On the Hidden Mystery of OCR in Large Multimodal Models.md
  - 230515 Common Diffusion Noise Schedules and Sample Steps are Flawed.md
  - 230515 Improved baselines for vision-language pre-training.md
  - 230515 Symbol tuning improves in-context learning in language models.md
  - 230516 Exploring the Impact of Layer Normalization for Zero-shot Neural Machine Translation.md
  - 230516 SoundStorm.md
  - 230516 SpecInfer.md
  - 230516 Towards Expert-Level Medical Question Answering with Large Language Models.md
  - 230517 Chain-of-Symbol Prompting Elicits Planning in Large Langauge Models.md
  - 230517 DoReMi.md
  - 230517 LeTI.md
  - 230517 SLiC-HF.md
  - 230517 Searching for Needles in a Haystack.md
  - 230517 Tree of Thoughts.md
  - 230518 Evidence of Meaning in Language Models Trained on Programs.md
  - 230518 LIMA.md
  - 230518 구글 달리기.md
  - 230519 Cross-Lingual Supervision improves Large Language Models Pre-training.md
  - 230521 A PhD Student's Perspective on Research in NLP in the Era of Very Large Language Models.md
  - 230522 GQA.md
  - 230522 How Language Model Hallucinations Can Snowball.md
  - 230522 To Repeat or Not To Repeat.md
  - 230522 Training Diffusion Models with Reinforcement Learning.md
  - 230523 Aligning Large Language Models through Synthetic Feedback.md
  - 230523 Goat.md
  - 230523 QLoRA.md
  - 230523 Sophia.md
  - 230523 ZeroSCROLLS.md
  - 230524 Mixture-of-Experts Meets Instruction Tuning.md
  - 230524 Model evaluation for extreme risks.md
  - 230525 Scaling Data-Constrained Language Models.md
  - 230525 The False Promise of Imitating Proprietary LLMs.md
  - 230526 Large Language Models as Tool Makers.md
  - 230526 MixCE.md
  - 230526 Training Socially Aligned Language Models in Simulated Human Society.md
  - 230527 Fine-Tuning Language Models with Just Forward Passes.md
  - 230529 Direct Preference Optimization.md
  - 230529 PaLI-X.md
  - 230529 RAPHAEL.md
  - 230530 Blockwise Parallel Transformer for Long Context Large Models.md
  - 230530 Grokking of Hierarchical Structure in Vanilla Transformers.md
  - 230531 Let's Verify Step by Step.md
  - 230531 The Impact of Positional Encoding on Length Generalization in Transformers.md
  - 230601 AWQ.md
  - 230601 Hiera.md
  - 230601 SnapFusion.md
  - 230601 StyleDrop.md
  - 230602 Fine-Grained Human Feedback Gives Better Rewards for Language Model Training.md
  - 230607 How Far Can Camels Go.md
  - 230608 SequenceMatch.md
  - 230611 LAMM.md
  - 230611 Language Versatilists vs. Specialists.md
  - 230612 Augmenting Language Models with Long-Term Memory.md
  - 230612 Gradient Ascent Post-training Enhances Language Model Generalization.md
  - 230613 Image Captioners Are Scalable Vision Learners Too.md
  - 230613 TART.md
  - 230614 WizardCoder.md
  - 230615 Exploring the MIT Mathematics and EECS Curriculum Using Large Language Models.md
  - 230615 Inverse Scaling.md
  - 230616 Demystifying GPT Self-Repair for Code Generation.md
  - 230616 Full Parameter Fine-tuning for Large Language Models with Limited Resources.md
  - 230616 Scaling Open-Vocabulary Object Detection.md
  - 230616 ZeRO++.md
  - 230619 BayLing.md
  - 230619 RepoFusion.md
  - 230620 Learning to Generate Better Than Your LLM.md
  - 230620 Textbooks Are All You Need.md
  - 230621 Constant Memory Attention Block.md
  - 230621 Deep Language Networks.md
  - 230621 Limits for Learning with Language Models.md
  - 230622 AudioPaLM.md
  - 230622 Quantizable Transformers.md
  - 230623 Bring Your Own Data! Self-Supervised Evaluation for Large Language Models.md
  - 230623 GKD.md
  - 230624 Beyond Scale.md
  - 230624 H$_2$O.md
  - 230625 Is RLHF More Difficult than Standard RL.md
  - 230626 DragDiffusion.md
  - 230626 Kosmos-2.md
  - 230626 Localized Text-to-Image Generation for Free via Cross Attention Control.md
  - 230626 Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression.md
  - 230626 Understanding In-Context Learning via Supportive Pretraining Data.md
  - 230627 CLIPA-v2.md
  - 230627 Extending Context Window of Large Language Models via Positional Interpolation.md
  - 230627 Free-style and Fast 3D Portrait Synthesis.md
  - 230627 IDOL.md
  - 230627 Length Generalization in Arithmetic Transformers.md
  - 230628 Towards Language Models That Can See.md
  - 230628 Towards Measuring the Representation of Subjective Global Opinions in Language Models.md
  - 230629 An Efficient General-Purpose Modular Vision Model via Multi-Task Heterogeneous Training.md
  - 230629 Benchmarking Large Language Model Capabilities for Conditional Generation.md
  - 230629 Generate Anything Anywhere in Any Scene.md
  - 230629 Generative AI for Programming Education.md
  - 230629 LLaVAR.md
  - 230630 Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting.md
  - 230630 Magic123.md
  - 230630 Preference Ranking Optimization for Human Alignment.md
  - 230701 Let Me Teach You.md
  - 230701 NTK Aware Scaled RoPE.md
  - 230704 Instruction Tuning Review.md
  - 230705 DragonDiffusion.md
  - 230705 Flacuna.md
  - 230705 Jailbroken.md
  - 230705 LongNet.md
  - 230705 Reasoning or Reciting.md
  - 230705 SkipDecode.md
  - 230706 Censored Sampling of Diffusion Models Using 3 Minutes of Human Feedback.md
  - 230706 Focused Transformer.md
  - 230706 Lost in the Middle.md
  - 230706 Style Over Substance.md
  - 230706 Superalignment.md
  - 230706 Training Models to Generate, Recognize, and Reframe Unhelpful Thoughts.md
  - 230707 SDXL.md
  - 230707 Teaching Arithmetic to Small Transformers.md
  - 230708 DDPO.md
  - 230710 About Anthropic.md
  - 230710 AnimateDiff.md
  - 230710 BeaverTails.md
  - 230710 FreeDrag.md
  - 230710 Large Language Models as General Pattern Machines.md
  - 230710 VampNet.md
  - 230711 GPT-4 FLOPS.md
  - 230711 Objaverse-XL.md
  - 230711 Secrets of RLHF in Large Language Models Part I.md
  - 230711 Self-consistency for open-ended generations.md
  - 230711 Stack More Layers Differently.md
  - 230712 Claude 2.md
  - 230712 Instruction Mining.md
  - 230712 No Train No Gain.md
  - 230712 Patch n' Pack.md
  - 230713 Hassabis.md
  - 230713 In-context Autoencoder for Context Compression in a Large Language Model.md
  - 230714 Code Interpreter.md
  - 230717 AlpaGasus.md
  - 230717 GEAR.md
  - 230717 Retentive Network.md
  - 230718 Flash Attention 2.md
  - 230718 How is ChatGPT's behavior changing over time.md
  - 230719 Llama 2.md
  - 230719 ZeroQuant-FP.md
  - 230720 FLASK.md
  - 230720 L-Eval.md
  - 230720 SciBench.md
  - 230724 RLCD.md
  - 230725 Retentive Network.md
  - 230727 Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback.md
  - 230727 PanGu-Coder2.md
  - 230727 Scaling TransNormer to 175 Billion Parameters.md
  - 230728 Exploring Format Consistency for Instruction Tuning.md
  - 230728 Skeleton-of-Thought.md
  - 230728 The Hydra Effect.md
  - 230729 Configuration System.md
  - 230731 KoBBQ.md
  - 230731 ToolLLM.md
  - 230801 Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models.md
  - 230802 From Sparse to Soft Mixtures of Experts.md
  - 230803 DETR Doesn't Need Multi-Scale or Locality Design.md
  - 230803 H100 Supply and Demand.md
  - 230803 Multimodal Neurons in Pretrained Text-Only Transformers.md
  - 230803 Scaling Relationship on Learning Mathematical Reasoning with Large Language Models.md
  - 230804 Retroformer.md
  - 230807 AgentBench.md
  - 230807 AlphaStar Unplugged.md
  - 230807 Intelligent Assistant Language Understanding On Device.md
  - 230807 LoRA-FA.md
  - 230807 TPTU.md
  - 230808 Continual Pre-Training of Large Language Models.md
  - 230808 Gentopia.md
  - 230808 Shepherd.md
  - 230809 StableCode.md
  - 230810 ReRoPE.md
- arXiv
- reviews
LICENSE
README.md

230209 Efficient Attention via Control Variates.md

Feb 22, 2023

cef0574 · Feb 22, 2023

https://arxiv.org/abs/2302.04542

Efficient Attention via Control Variates (Lin Zheng, Jianbo Yuan, Chong Wang, Lingpeng Kong)