Wenyu Huang

School of Informatics

University of Edinburgh

Edinburgh, United Kingdom

I am a final year PhD student in Natural Language Processing at the School of Informatics, University of Edinburgh, supervised by Prof. Jeff Z. Pan and Prof. Mirella Lapata. My research focuses on conversational search, retrieval-augmented generation (RAG), large language models (LLMs), and multi-hop question answering.

My work spans several key areas in NLP:

Retrieval-Augmented Generation: Bridging the gap between RAG and LLMs, and exploring the capabilities of LLMs in RAG systems.
Large Language Models: Exploring the capabilities of large language models in conversational search systems and complex question answering / reasoning tasks.
Multi-hop Question Answering: Investigating and improving the capabilities of large language models in complex reasoning tasks. Including long context reasoning and RL-based solutions.

I have published papers in top-tier conferences including IJCNLP, SIGIR, EMNLP, and ACL. My research has received citations from the academic community, contributing to the advancement of conversational AI and knowledge-based systems.

I am currently seeking industry opportunities in NLP, including both internship and full-time positions.

news

Jun 24, 2025	Our paper “Masking in Multi-hop {QA}: An Analysis of How Language Models Perform with Context Permutation” is accepted by ACL 2025, and selected for oral presentation!
May 01, 2025	New Research on Memory in AI Systems
Oct 08, 2024	Our paper “Less is More: Making Smaller Language Models Competent Subgraph Retrievers for Multi-hop KGQA” has been accepted by EMNLP 2024!

selected publications

ACL 2025
Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation

Wenyu Huang, Pavlos Vougiouklis, Mirella Lapata, and 1 more author

In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2025

Oral Presentation

Abs Bib PDF

Multi-hop Question Answering (MHQA) adds layers of complexity to question answering, making it more challenging. When Language Models (LMs) are prompted with multiple search results, they are tasked not only with retrieving relevant information but also employing multi-hop reasoning across the information sources. Although LMs perform well on traditional question-answering tasks, the causal mask can hinder their capacity to reason across complex contexts. In this paper, we explore how LMs respond to multi-hop questions by permuting search results (retrieved documents) under various configurations. Our study reveals interesting findings as follows: 1) Encoder-decoder models, such as the ones in the Flan-T5 family, generally outperform causal decoder-only LMs in MHQA tasks, despite being significantly smaller in size; 2) altering the order of gold documents reveals distinct trends in both Flan T5 models and fine-tuned decoder-only models, with optimal performance observed when the document order aligns with the reasoning chain order; 3) enhancing causal decoder-only models with bi-directional attention by modifying the causal mask can effectively boost their end performance. In addition to the above, we conduct a thorough investigation of the distribution of LM attention weights in the context of MHQA. Our experiments reveal that attention weights tend to peak at higher values when the resulting answer is correct. We leverage this finding to heuristically improve LMs’ performance on this task. Our code is publicly available at https://github.com/hwy9855/MultiHopQA-Reasoning.
@inproceedings{huang-etal-2025-masking, title = {Masking in Multi-hop {QA}: An Analysis of How Language Models Perform with Context Permutation}, author = {Huang, Wenyu and Vougiouklis, Pavlos and Lapata, Mirella and Pan, Jeff Z.}, editor = {Che, Wanxiang and Nabende, Joyce and Shutova, Ekaterina and Pilehvar, Mohammad Taher}, booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, month = jul, year = {2025}, address = {Vienna, Austria}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2025.acl-long.869/}, pages = {17781--17795}, isbn = {979-8-89176-251-0}, note = {<b>Oral Presentation</b>} }

arXiv

Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions

Yiming Du^*, Wenyu Huang^*, Danna Zheng^*, and 5 more authors

May 2025

Bib PDF

@misc{du2025rethinkingmemoryaitaxonomy,
  title = {Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions},
  author = {Du, Yiming and Huang, Wenyu and Zheng, Danna and Wang, Zhaowei and Montella, Sebastien and Lapata, Mirella and Wong, Kam-Fai and Pan, Jeff Z.},
  year = {2025},
  eprint = {2505.00675},
  archiveprefix = {arXiv},
  primaryclass = {cs.CL},
  url = {https://arxiv.org/abs/2505.00675},
  month = may,
}

EMNLP 2024
Less is More: Making Smaller Language Models Competent Subgraph Retrievers for Multi-hop KGQA

Wenyu Huang, Guancheng Zhou, Hongru Wang, and 3 more authors

In Findings of the Association for Computational Linguistics: EMNLP 2024, Nov 2024

Abs DOI Bib PDF

Retrieval-Augmented Generation (RAG) is widely used to inject external non-parametric knowledge into large language models (LLMs). Recent works suggest that Knowledge Graphs (KGs) contain valuable external knowledge for LLMs. Retrieving information from KGs differs from extracting it from document sets. Most existing approaches seek to directly retrieve relevant subgraphs, thereby eliminating the need for extensive SPARQL annotations, traditionally required by semantic parsing methods. In this paper, we model the subgraph retrieval task as a conditional generation task handled by small language models. Specifically, we define a subgraph identifier as a sequence of relations, each represented as a special token stored in the language models. Our base generative subgraph retrieval model, consisting of only 220M parameters, achieves competitive retrieval performance compared to state-of-the-art models relying on 7B parameters, demonstrating that small language models are capable of performing the subgraph retrieval task. Furthermore, our largest 3B model, when plugged with an LLM reader, sets new SOTA end-to-end performance on both the WebQSP and CWQ benchmarks. Our model and data will be made available online: https://github.com/hwy9855/GSR.
@inproceedings{huang-etal-2024-less, title = {Less is More: Making Smaller Language Models Competent Subgraph Retrievers for Multi-hop {KGQA}}, author = {Huang, Wenyu and Zhou, Guancheng and Wang, Hongru and Vougiouklis, Pavlos and Lapata, Mirella and Pan, Jeff Z.}, editor = {Al-Onaizan, Yaser and Bansal, Mohit and Chen, Yun-Nung}, booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2024}, month = nov, year = {2024}, address = {Miami, Florida, USA}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2024.findings-emnlp.927/}, doi = {10.18653/v1/2024.findings-emnlp.927}, pages = {15787--15803}, }

arXiv

UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems

Hongru Wang, Wenyu Huang, Yang Deng, and 6 more authors

Jan 2024

Bib PDF

@misc{wang2024unimsragunifiedmultisourceretrievalaugmented,
  title = {UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems},
  author = {Wang, Hongru and Huang, Wenyu and Deng, Yang and Wang, Rui and Wang, Zezhong and Wang, Yufei and Mi, Fei and Pan, Jeff Z. and Wong, Kam-Fai},
  year = {2024},
  eprint = {2401.13256},
  archiveprefix = {arXiv},
  primaryclass = {cs.CL},
  url = {https://arxiv.org/abs/2401.13256},
  month = jan,
}

IJCNLP 2023

Retrieval Augmented Generation with Rich Answer Encoding

Wenyu Huang, Mirella Lapata, Pavlos Vougiouklis, and 2 more authors

In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Nov 2023

DOI Bib PDF

@inproceedings{huang-etal-2023-retrieval,
  title = {Retrieval Augmented Generation with Rich Answer Encoding},
  author = {Huang, Wenyu and Lapata, Mirella and Vougiouklis, Pavlos and Papasarantopoulos, Nikos and Pan, Jeff},
  editor = {Park, Jong C. and Arase, Yuki and Hu, Baotian and Lu, Wei and Wijaya, Derry and Purwarianti, Ayu and Krisnadhi, Adila Alfa},
  booktitle = {Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  month = nov,
  year = {2023},
  address = {Nusa Dua, Bali},
  publisher = {Association for Computational Linguistics},
  url = {https://aclanthology.org/2023.ijcnlp-main.65/},
  doi = {10.18653/v1/2023.ijcnlp-main.65},
  pages = {1012--1025},
}