---

Research interests

---

Publications & Preprints ✒️

DataDecide: How to Predict Best Pretraining Data with Small Experiments [code, models]

Ian Magnusson*, Nguyen Tai*, Ben Bogin*, David Heineman, Jena D. Hwang, Luca Soldaini, Akshita Bhagia, Jiacheng Liu, Dirk Groeneveld, Oyvind Tafjord, Noah A. Smith, Pang Wei Koh, Jesse Dodge
preprint, 2025

Establishing Task Scaling Laws via Compute-Efficient Model Ladders [code]

Akshita Bhagia*, Jiacheng Liu*, Alexander Wettig, David Heineman, Oyvind Tafjord, Ananya Harsh Jha, Luca Soldaini, Noah A. Smith, Dirk Groeneveld, Pang Wei Koh, Jesse Dodge, Hannaneh Hajishirzi
preprint, 2024

Improving Minimum Bayes Risk Decoding with Multi-Prompt [code]

David Heineman, Yao Dou, Wei Xu
EMNLP, 2024

Towards a Path Dependent Account of Category Fluency [code]

David Heineman, Reba Koenen, Sashank Varma
CogSci, 2024

Thresh 🌾: Unified, Customizable and Deployable Fine-Grained Text Evaluation [thresh.tools]

David Heineman, Yao Dou, Wei Xu
EMNLP Demo, 2023

Edit-level Simplification Evaluation using SALSA 💃 [code/data, metric]

David Heineman, Yao Dou, Mounica Maddela, Wei Xu
EMNLP, 2023

LENS 🔎: A Learnable Evaluation Metric for Text Simplification [code/data, metric]

Mounica Maddela*, Yao Dou*, David Heineman, Wei Xu
ACL, 2023

* = equal contribution

---

My past work 🌳

• Maintaining the Thresh 🌾 platform, an all-purpose tool for fine-grained text generation evaluation, including an annotation tool builder and integrated Python library.
• Built a search engine [code] for ML / NLP conferences, indexed with ColBERT.
• Wrote a LLM-based Rubiks cube solver as a demonstration of explore/exploit behavior for reasoning (🏆 2nd place at AGI House open source hackathon).
• Awarded the GT College of Computing Outstanding Undergraduate Research Award (1 of 3000+ CS students) for my undergradaute thesis work on fine-grained evaluation of LLMs.
• Designed new programming assignments for CS 4650, Natural Language Processing as a teaching assistant (sampling algorithms & LLaMA fine-tuning with LoRA).
• Built an air pollution complaint tracker and classifier [code] for the Georgia Environmental Protection Divison (part of a larger collaboration at GT).
• Awarded the PURA research grant to work on open problems in generation & evaluation (check out my Huggingface decoding vizualizer extension).
• Thoughts on approaching reasoning evaluation in LLMs using theories of human cognition.
• Integrated the LENS and LENS-SALSA models and PyPi library with HuggingFace to let anyone evaluate text simplification with LLMs in 5 lines of Python [demo].
• Interned in AWS EC2 Enterprise Services, developing a prototype language model service, addressing problems in inference cost and deployment of open-source LLMs.
• Designed the first rate-and-rank human evaluation framework for text generation as co-author of the LENS project.
• Earned 4th place in Georgia Tech's Wrek CTF (one of the largest greyhat hackathons in the southern US). See my answers here.
• Interned at AWS CloudWatch Application Insights, built infrastructure to monitor and group telemetry data from processes running on EC2 instances to identify the root causes of problems on customers' AWS infrastructure.
• Helped lead Georgia Tech's CS 3510, Design and Analysis of Algorithms as a teaching asssistant in Fall '21 and '22.
• Interned at Patientco (now part of Waystar), invented and deployed a new sequence-based prediction model to predict when a patient pays their healthcare bill using their payment history (used to customize ~5% of U.S. healthcare bills).
• Built an efficient, real-time data ingestion pipeline to 10x the volume of retreival data available to the fake news detection team at Big Data Big Impact.
• Deployed an efficient inference API to allow researchers to segment Twitter hashtags using a new segmentation model from Georgia Tech's NLP Lab.
• Developed new methods [code] for automatically grading student essays.