University of Oxford Department of Computer Science, University of Oxford Wolfson College, Oxford ELLIS Society
Hazel Kim

Hazel Kim

Interpretable and Controllable Language Models Research →

{firstname}.kimh [AT] gmail

I am conducting research in Natural Language Processing and Machine Learning. I am grateful to be advised by Philip Torr and Yarin Gal from Oxford and Hinrich Schütze from LMU Munich as a student of the European Laboratory for Learning and Intelligent Systems (ELLIS Society). Full bio →

Portrait of Hazel Kim
Reasoning Acquisition from Limited Sources
  • Models reason well with quality, abundant data, and give biased perspectives with scarce, poor data.
  • I study how models acquire reasoning ability from a limited quantity or quality of resources.
Causal Inference, not Correlation
  • Correlations limit reasoning about causes not present in the text.
  • I study causal inference to control language models against correlation-biased data.
Information Quantification
  • Usable information varies with how input and output interact.
  • I study information quantification that interprets model behaviors.
Emergent Knowledge of Language Models
  • As scale grows, models emerge knowledge smaller models do not.
  • I study whether that knowledge is trustworthy for transparent, real-world use.
News
Received a Google Gemma Academic Program Award (Google Cloud Credit)
Received a Google Gemma Academic Program Award (Google Cloud Credit)
Visited LMU Munich — Hinrich Schuetze (July–September 2024)
Received an ELSA Mobility Grant, 2024
Received a G-Research Grant for Early Career Researchers, 2024 June
Started a DPhil in Computer Science at Oxford
Publications

  1. What Hallucinations Have We Measured? Semantic Illusion, Confabulation, and Expressed Uncertainty

    Hazel Kim, Jakub Binkowski, Ekaterina Fadeeva, Juan David Guerra, Yijie Hao, Zheng Yi Ho, Baixiang Huang, Buyun Liang, Siyuan Liang, Anh Tuan Luu, Potsawee Manakul, Niels Mündler-Sasahara, Aidar Myrzakhan, Shahrad Mohammadzadeh, Artem Shelmanov, Kai Shu, Ponhvoan Srey, Dacheng Tao, René Vidal, Haofei Yu, Lei Yu, Hongxiang Zhang, Yarin Gal, Hinrich Schütze, Philip Torr

    Preprint

    • Defines hallucination as a confidently incorrect response, split into semantic illusion and confabulation by whether a semantically equivalent prompt recovers the correct answer, and kept separate from expressed uncertainty.
    Details PDF

  2. FramingQA: Does the Question Shape the Answer? Measuring the Compositional Framing Effect

    Hazel Kim, Andrew M. Bean, Guilherme Affonso Ferreira de Camargo, Shanyu Chauhan, Felix Drinkall, Jade Kosché, Chenyang Ma, Glory Nwaugbala, Nabeel Seedat, Bradley Max Segal, Samuel Recht, Hinrich Schütze, Philip Torr

    Preprint

    • Introduces FramingQA, a benchmark of how question framing shifts LLM answers in law, medicine, finance, and robotics, even when the underlying facts stay fixed.
    Details PDF

  3. Single LLM Debate, MoLaCE: Mixture of Latent Concept Experts Against Confirmation Bias

    Hazel Kim, Philip Torr

    Preprint

    • Introduces MoLaCE, an inference-time method that mixes latent-concept experts so a single LLM can resist confirmation bias—emulating multi-agent debate more efficiently and reducing echo-chamber effects.
    Details PDF

  4. Measuring what Matters: Construct Validity in Large Language Model Benchmarks

    Andrew M. Bean, Ryan Othniel Kearns, Angelika Romanou, Franziska Sofia Hafner, Harry Mayne, Jan Batzner, Negar Foroutan, Chris Schmitz, Karolina Korgul, Hunar Batra, Oishi Deb, Emma Beharry, Cornelius Emde, Thomas Foster, Anna Gausen, María Grandury, Simeng Han, Valentin Hofmann, Lujain Ibrahim, Hazel Kim, Hannah Rose Kirk, Fangru Lin, Gabrielle Kaili-May Liu, Lennart Luettgau, Jabez Magomere, Jonathan Rystrøm, Anna Sotnikova, Yushi Yang, Yilun Zhao, Adel Bibi, Antoine Bosselut, Ronald Clark, Arman Cohan, Jakob Nicolaus Foerster, Yarin Gal, Scott A. Hale, Inioluwa Deborah Raji, Christopher Summerfield, Philip Torr, Cozmin Ududec, Luc Rocher, Adam Mahdi

    NeurIPS 2025 Datasets & Benchmarks

    • A systematic review of 445 LLM benchmarks with 29 experts, showing how weak construct validity undermines claims about safety and capability, and offering eight recommendations for more rigorous evaluation.
    Details PDF

  5. Detecting LLM Hallucination through Layer-wise Information Deficiency

    Hazel Kim, Tom A. Lamb, Adel Bibi, Philip Torr, Yarin Gal

    EMNLP 2025

    • Detects hallucinations at test time by tracking usable-information deficiencies across model layers—especially under ambiguous or unanswerable prompts—without extra training or architecture changes.
    Details PDF

  6. How Ambiguous Are the Rationales For Natural Language Reasoning?

    Hazel Kim

    COLING 2025

    • Studies how uncertain or ambiguous rationales affect reasoning performance, and proposes a simple way for models to choose among reasoning paths when rationale quality is inconsistent.
    Details PDF

  7. ATHENA: Mathematical Reasoning with Thought Expansion

    JB. Kim, Hazel Kim, Joonghyuk Hahn, Yo-Sub Han

    EMNLP 2023

    • Presents ATHENA, an attention-based architecture that expands candidate mathematical thoughts step by step, improving math-word-problem solving under limited or varied training signals.
    Details PDF

  8. ALP: Data Augmentation Using Lexicalized PCFGs for Few-Shot Text Classification

    Hazel Kim, Daecheol Woo, Seong Joon Oh, Jeong-Won Cha, Yo-Sub Han

    AAAI 2022

    • Generates syntactically diverse, label-preserving text with lexicalized PCFGs for few-shot classification, and pairs this with augmentation-aware train/validation splitting for stronger low-resource training.
    Details PDF

  9. LST: Lexicon-Guided Self-Training for Few-Shot Text Classification

    Hazel Kim*, Jaeman Son*, Yo-Sub Han

    Arxiv

    • Improves few-shot self-training by guiding pseudo-labels with a refined lexicon, reducing overconfident early errors when only a handful of labeled examples are available.
    Details PDF
Positions