prof_pic.jpg
Kolingasse 14-16, 1090 Wien

Hi! I’m Nick, a second-year PhD student at the University of Vienna supervised by Sebastian Schuster. I’m interested in evaluating and improving LLM agents as collaborators, currently focusing on coding agents. My work studies when agents should ask for clarification versus proceed autonomously (see Ask or Assume?), and how they handle users interrupting and correcting them mid-task in real-world developer-agent interactions. I’ve also worked on evaluating agents’ ability to implement AI research (RExBench).

Previously, I received a BA in Linguistics from the University of Cambridge, followed by an MSc in Speech & Language Processing from the University of Edinburgh. During my time at Edinburgh, I did research in emergent communication between neural networks with Hannah Rohde and Henry Conklin.

Get in touch here: nicholas.edwards[æt]univie.ac.at

news

Sep 25, 2026 🎉 Harbor Adapters and Harbor-Index accepted to the NeurIPS 2026 Evaluations & Datasets track!
Aug 21, 2026 🎉 Ask or Assume? accepted to EMNLP 2026 Findings and the REALM workshop! See you in Budapest!
Apr 17, 2026 RExBench has been integrated into the Harbor framework! See the announcement for more details.
Apr 07, 2026 🎉 RExBench accepted to the main ACL 2026 conference! Excited to share this work in San Diego!
Sep 15, 2025 Gave an invited talk on RExBench to Ludwig Schmidt’s group at Stanford/UW, together with Yukyung Lee! :t-rex:

selected publications

  1. NeurIPS ’26
    Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
    Lin Shi, …, Nicholas Edwards, …, Ludwig Schmidt, and Alex Shaw
    In Advances in Neural Information Processing Systems, Evaluations & Datasets Track, 2026
  2. EMNLP Findings ’26
    Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agents
    Nicholas Edwards and Sebastian Schuster
    In Findings of the Association for Computational Linguistics: EMNLP 2026, 2026
  3. ACL ’26
    RExBench: Can coding agents autonomously implement AI research extensions?
    Nicholas Edwards, Yukyung Lee, Yujun Audrey Mao, and 3 more authors
    In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics, 2026
  4. CogSci ’23
    Anaphoric structure emerges between neural networks
    Nicholas Edwards, Hannah Rohde, and Henry Conklin
    In 45th Annual Conference of the Cognitive Science Society: Cognition in Context, 2023