Search
photo of Pranava Madhyastha

Dr Pranava Madhyastha

Senior Lecturer

School of Science & Technology Department of Computer Science

Contact details

Personal links

About

Overview

Dr. Pranava Madhyastha is an academic and researcher specializing in Artificial Intelligence, specifically in the fields of Natural Language Processing (NLP) and Multimodal Machine Learning. He is currently a Senior Lecturer (equivalent to Associate Professor) in Computer Science at City St George's, University of London . He also serves as a Principal Investigator at the Alan Turing Institute, the UK's national institute for data science and AI. 

Current Research Focus: Dr. Madhyastha's current work primarily explores how computers can understand and generate language by combining different types of data (across modalities). His research interests include: 

  • Multimodal Learning: Building and theorising models (and humans) that learn from lexical, visual (images/video), and auditory modalities simultaneously. 
  • Natural Language Understanding & Generation: Developing systems that can process and produce human language more effectively
  • Guaranteeing strict adherence to structure: Combining the learning capabilities and fluency of neural networks (like LLMs) with the reasoning power and strict logic of symbolic artificial intelligence

Research students

Maeve Hutchinson

Attendance: February 2023 - present, full-time

Role: 1st Supervisor

Chenxi Whitehouse

Attendance: October 2020 - present, full-time

Role: 2nd Supervisor

Nadine El Naggar

Attendance: September 2019 - present, full-time

Role: 2nd Supervisor

Hadeel Al-Negheimish

Attendance: October 2018 - present, full-time

Thesis title: Towards Numerical Reasoning in Machine Reading Comprehension

Role: 2nd Supervisor

Chiraag Lala

Attendance: September 2018 - September 2023, full-time

Thesis title: Multimodal Word Sense Translation

Role: 2nd Supervisor

1stsupervisor

2ndsupervisor

Publications

Publications by category

Conference papers and proceedings (56)

  • Visani Scozzi, M., Makri, S. and Madhyastha, P. "Although Powerful, it's not Infallible": Investigating Academic Researchers' Verification Challenges with LLMs. CHIIR '26: 2026 ACM SIGIR Conference on Human Information Interaction and Retrieval.doi:10.1145/3786304.3787865
  • Clay, A., Jiménez-Ruiz, E. and Madhyastha, P. (2025). Noise or Nuance: An Investigation Into Useful Information and Filtering For LLM Driven AKBC. 3rd Workshop on Knowledge Base Construction from Pre-Trained Language Models and the 4th Challenge on Language Models for Knowledge Base Construction (KBC-LM+LM-KBC 2025) 2 November, Nara, Japan.
  • Clay, A., Jiménez-Ruiz, E. and Madhyastha, P. (2025). Judge, Generator, Executioner: Utilizing an LLM for KBC. 3rd Workshop on Knowledge Base Construction from Pre-Trained Language Models and the 4th Challenge on Language Models for Knowledge Base Construction 2 November, Nara, Japan.
  • Hutchinson, M., Jianu, R., Slingsby, A., Wood, J. and Madhyastha, P. (2025). Capturing Visualization Design Rationale. 2025 IEEE Visualization and Visual Analytics (VIS) 1-7 November.doi:10.1109/vis60296.2025.00052
  • Hutchinson, M., Jianu, R., Slingsby, A., Wood, J. and Madhyastha, P. (2025). Chart Question Answering from Real-World Analytical Narratives. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop) July.doi:10.18653/v1/2025.acl-srw.50
  • Albinhassan, M., Madhyastha, P., Law, M. and Russo, A. (2025). Learning and Enforcing Context-Sensitive Control for LLMs. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop) July.doi:10.18653/v1/2025.acl-srw.59
  • Hutchinson, M., Jianu, R., Slingsby, A. and Madhyastha, P. (2024). LLM-Assisted Visual Analytics: Opportunities and Challenges. Computer Graphics & Visual Computing (CGVC) 2024 12-13 September, London, UK.doi:10.2312/cgvc.20241237
  • Hutchinson, M., Jianu, R., Slingsby, A. and Madhyastha, P. (2024). From LitVis to Reasoning in Data Visualisation: A Research Plan. Computer Graphics & Visual Computing (CGVC) 2024 12-13 September, London, UK.
  • Madhyastha, P., Zhang, Y., Vigliocco, G., Goldwater, M., Anggoro, F.K., Hayes, B.K.... Ong, D.C. (2023). Are words equally surprising in audio and audio-visual comprehension? 45th Annual Conference of the Cognitive Science Society. 26-29 July, Sydney, Australia.
  • El-Naggar, N., Ryzhikov, A., Daviaud, L., Madhyastha, P. and Weyde, T. (2023). Formal and Empirical Studies of Counting Behaviour in ReLU RNNs. The 16th International Conference on Grammatical Inference 10-13 July.
  • Hutchinson, M., Slingsby, A., Jianu, R. and Madhyastha, P. (2023). Towards Visualisation Specifications from Multilingual Natural Language Queries using Large Language Models. Eurographics 2023 8-12 May, Saarbrücken, Germany.doi:10.2312/evp.20231072
  • Whitehouse, C., Weyde, T. and Madhyastha, P. (2023). Towards a Unified Model for Generating Answers and Explanations in Visual Question Answering. Findings of the Association for Computational Linguistics: EACL 2023 May.doi:10.18653/v1/2023.findings-eacl.126
  • Al-Negheimish, H., Madhyastha, P. and Russo, A. (2023). Towards preserving word order importance through Forced Invalidation. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics May.doi:10.18653/v1/2023.eacl-main.187
  • El-Naggar, N., Madhyastha, P. and Weyde, T. (2023). Theoretical Conditions and Empirical Failure of Bracket Counting on Long Sequences with Linear Recurrent Networks. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop May.doi:10.18653/v1/2023.eacl-srw.15
  • Whitehouse, C., Weyde, T. and Madhyastha, P. Towards a Unified Model for Generating Answers and Explanations in Visual Question Answering. .
  • Al-Negheimish, H., Madhyastha, P. and Russo, A. Towards preserving word order importance through FORCED INVALIDATION. .
  • Madhyastha, P. Towards Integration of Embodiment Features for Prosodic Prominence Prediction from Text. ICMI '22: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION.doi:10.1145/3536220.3558540
  • Sabir, A., Moreno-Noguer, F., Madhyastha, P. and Padró, L. Belief Revision based Caption Re-ranker with Visual Semantic Information. 29th International Conference on Computational Linguistics 12-17 Oct 2022.
  • El-Naggar, N., Madhyastha, P. and Weyde, T. (2022). Experiments in Learning Dyck-1 Languages with Recurrent Neural Networks. 3rd Human-Like Computing Workshop (HLC 2022) 28-30 September, Windsor, UK.
  • Whitehouse, C., Weyde, T., Madhyastha, P. and Komninos, N. (2022). Evaluation of Fake News Detection with Knowledge-Enhanced Language Models. Proceedings of the Sixteenth International AAAI Conference on Web and Social Media, ICWSM 2022 6-9 June, Atlanta, Georgia, USA.
  • Al-Negheimish, H., Madhyastha, P. and Russo, A. (2021). Numerical reasoning in machine reading comprehension tasks: are we there yet? Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing November.doi:10.18653/v1/2021.emnlp-main.759
  • Mitzalis, F., Caglayan, O., Madhyastha, P. and Specia, L. (2021). BERTGen: Multi-task Generation through BERT. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) August.doi:10.18653/v1/2021.acl-long.503
  • López, J.A.D. and Madhyastha, P. (2021). A focused analysis of twitter-based disinformation from foreign influence operations. KnOD’21 Workshop 14 April-, Virtual event.
  • Ive, J., Li, A.M., Miao, Y., Caglayan, O., Madhyastha, P. and Specia, L. (2021). Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume April.doi:10.18653/v1/2021.eacl-main.281
  • Caglayan, O., Kuyu, M., Amac, M.S., Madhyastha, P., Erdem, E., Erdem, A.... Specia, L. (2021). Cross-lingual Visual Pre-training for Multimodal Machine Translation. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume April.doi:10.18653/v1/2021.eacl-main.112
  • Al-Negheimish, H., Madhyastha, P. and Russo, A. (2021). Discrete Reasoning Templates for Natural Language Understanding. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop April.doi:10.18653/v1/2021.eacl-srw.12
  • Caglayan, O., Ive, J., Haralampieva, V., Madhyastha, P., Barrault, L. and Specia, L. Simultaneous machine translation with visual context. .
  • Caglayan, O., Madhyastha, P. and Specia, L. Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale. .
  • Scarton Carolina, , Madhyastha Pranava, and Specia Lucia, Deciding When, How and for Whom to Simplify. .doi:10.3233/faia200342
  • Madhyastha, P. and Jain, R. (2019). On Model Stability as a Function of Random Seed. Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL) November.doi:10.18653/v1/k19-1087
  • Ive, J., Madhyastha, P. and Specia, L. (2019). Deep Copycat Networks for Text-to-Text Generation. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) November.doi:10.18653/v1/d19-1318
  • Chow, J., Specia, L. and Madhyastha, P. (2019). WMDO: Fluency-based Word Mover’s Distance for Machine Translation Evaluation. Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1) August.doi:10.18653/v1/w19-5356
  • Madhyastha, P., Wang, J. and Specia, L. (2019). VIFIDEL: Evaluating the Visual Fidelity of Image Descriptions. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics July.doi:10.18653/v1/p19-1654
  • Ive, J., Madhyastha, P. and Specia, L. (2019). Distilling Translations with Visual Awareness. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics July.doi:10.18653/v1/p19-1653
  • Caglayan, O., Madhyastha, P., Specia, L. and Barrault, L. (2019). Probing the Need for Visual Context in Multimodal Machine Translation. Proceedings of the 2019 Conference of the North June.doi:10.18653/v1/n19-1422
  • Holzenberger, N., Palaskar, S., Madhyastha, P., Metze, F. and Arora, R. (2019). Learning from Multiview Correlations in Open-domain Videos. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 12-17 May.doi:10.1109/icassp.2019.8683540
  • Chow, J., Madhyastha, P. and Specia, L. WMDO: Fluency-based word Mover's distance for machine translation evaluation. .doi:10.18653/v1/w19-5356
  • He, J., Madhyastha, P. and Specia, L. Deep copycat Networks for Text-to-Text Generation. .
  • Madhyastha, P.S., Wang, J. and Specia, L. (2018). End-to-end Image Captioning Exploits Distributional Similarity in Multimodal Space. Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP November.doi:10.18653/v1/w18-5455
  • Lala, C., Madhyastha, P.S., Scarton, C. and Specia, L. (2018). Sheffield Submissions for WMT18 Multimodal Translation Shared Task. Proceedings of the Third Conference on Machine Translation: Shared Task Papers October.doi:10.18653/v1/w18-6442
  • Madhyastha, P.S., Wang, J. and Specia, L. (2018). Defoiling Foiled Image Captions. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) June.doi:10.18653/v1/n18-2069
  • Wang, J., Madhyastha, P.S. and Specia, L. (2018). Object Counts! Bringing Explicit Detections Back into Image Captioning. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) June.doi:10.18653/v1/n18-1198
  • Madhyastha, P., Wang, J. and Specia, L. End-to-end image captioning exploits multimodal distributional similarity. .
  • Lala, C., Madhyastha, P., Scarton, C. and Specia, L. Sheffield Submissions for WMT18 Multimodal Translation Shared Task. .doi:10.18653/v1/W18-64069
  • Deena, S., Ng, R.W.M., Madhyastha, P., Specia, L. and Hain, T. (2017). Exploring the use of acoustic embeddings in neural machine translation. 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) 16-20 December.doi:10.1109/asru.2017.8268971
  • Madhyastha, P.S., Wang, J. and Specia, L. (2017). Sheffield MultiMT: Using Object Posterior Predictions for Multimodal Machine Translation. Proceedings of the Second Conference on Machine Translation September.doi:10.18653/v1/w17-4752
  • Madhyastha, P.S. and España-Bonet, C. (2017). Learning Bilingual Projections of Embeddings for Vocabulary Expansion in Machine Translation. Proceedings of the 2nd Workshop on Representation Learning for NLP August.doi:10.18653/v1/w17-2617
  • Madhyastha, P.S., Carreras, X. and Quattoni, A. Prepositional phrase attachment over word embedding products. .
  • Madhyastha, P.S., Bansal, M., Gimpel, K. and Livescu, K. (2016). Mapping Unseen Words to Task-Trained Embedding Spaces. Proceedings of the 1st Workshop on Representation Learning for NLP August.doi:10.18653/v1/w16-1612
  • Costa-jussà, M.R., España-Bonet, C., Madhyastha, P., Escolano, C. and Fonollosa, J.A.R. (2016). The TALP–UPC Spanish–English WMT Biomedical Task: Bilingual Embeddings and Char-based Neural Language Model Rescoring in a Phrase-based System. Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers August.doi:10.18653/v1/w16-2336
  • Quattoni, A., Ramisa, A., Madhyastha, P.S., Simo-Serra, E. and Moreno-Noguer, F. (2016). Structured Prediction with Output Embeddings for Semantic Image Annotation. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies June.doi:10.18653/v1/n16-1068
  • Ellebracht, L.D., Ramisa, A., Madhyastha, P.S., Cordero-Rama, J., Moreno-Noguer, F. and Quattoni, A. (2015). Semantic Tuples for Evaluation of Image to Sentence Generation. Proceedings of the Fourth Workshop on Vision and Language September.doi:10.18653/v1/w15-2806
  • Madhyastha, P.S., Carreras, X. and Quattoni, A. Tailoring word embeddings for bilexical predictions: An experimental comparison. .
  • Madhyastha, P.S., Carreras, X. and Quattoni, A. Learning task-specific bilexical embeddings. .
  • Deena, S., Ng, R.W.M., Madhyastha, P., Specia, L. and Hain, T. Semi-Supervised Adaptation of RNNLMs by Fine-Tuning with Domain-Specific Auxiliary Features. Interspeech 2017.doi:10.21437/interspeech.2017-1598
  • Madhyastha, P. Towards Holistic, Pragmatic and Multimodal Conversational Systems. .doi:10.1609/aaai.v38i20.30293

Journal articles (8)