David Heineman

More about me

I grew up in Marietta, Georgia and now live a half-hour down in Midtown Atlanta where I'm currently in my final year at Georgia Tech. I thoroughly enjoy reading, hiking, and making homebrew nitrogen cold brew. ☕️ ⛰️

I'm exploring the field of natural language processing with a focus on evaluation, modeling and deployment of constrained text generation models. My research has focused on challenges in text simplification and fine-grained evaluation of LLMs. With new analytical tools and existing theories of cognition, I hope to build a rigorous top-down understanding of LLM behavior. 🛠️

Publications & Preprints ✒️

Thresh 🌾: A Unified, Customizable and Deployable Platform for Fine-Grained Text Evaluation [thresh.tools]
David Heineman, Yao Dou, Wei Xu
EMNLP Demo, 2023

Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSA 💃 [code/data]
David Heineman, Yao Dou, Mounica Maddela, Wei Xu
EMNLP, 2023

LENS 🔎: A Learnable Evaluation Metric for Text Simplification
Mounica Maddela*, Yao Dou*, David Heineman, Wei Xu
ACL, 2023

* = equal contribution

What I've been building 👷

Building the Thresh 🌾 platform, a multi-tool for fine-grained generation evaluation projects, including an annotation tool builder and integrated Python library.
Under the PURA research award, working on open problems in generation & evaluation (check out my new Huggingface decoding vizualizer extension). I'm very fortunate to be advised by Prof. Wei Xu and to have worked with Yao Dou and Dr. Mounica Maddela.
Reading up on cognitive science and question answering! Here's some of my thoughts on evaluating reasoning in LLMs.

My past work 🌳

Designed new programming assignments for NLP (CS 4650) as a teaching assistant (incl. sampling algorithms & LLaMA fine-tuning with LoRA!)
Interned in AWS EC2 Enterprise Services, developing a prototype language model service, addressing problems in inference cost and deployment of open-source LLMs.
Designed the first rate-and-rank human evaluation framework for text generation as co-author of the LENS project.
Earned 4th place in Georgia Tech's Wrek CTF (one of the largest greyhat hackathons in the south). See my answers here.
Interned at AWS CloudWatch Application Insights, built infrastructure to monitor and group telemetry data from processes running on EC2 instances to identify the root causes of problems on customers' AWS infrastructure.
Helped lead Georgia Tech's CS 3510, Design and Analysis of Algorithms as a teaching asssistant in Fall '21 and '22.
Interned at Patientco (now part of Waystar), invented and deployed a new sequence-based prediction model to predict when a patient pays their healthcare bill using their payment history (used to customize ~5% of U.S. healthcare bills).
Voulenteered with the fake news detection team at Big Data Big Impact to create a scraper to increase 10x the volume of data which could be queried for real-time predictions.
Deployed an efficient inference API to allow researchers to segment Twitter hashtags using a new segmentation model from Georgia Tech's NLP Lab.
Developed a state-of-the-art method [code] for grading student essays using bi-directional LSTMs. (My work has been surpassed post-2020)