Research

My research spans a wide range of topics around generative AI, with a particular focus on LLM research. Currently, I’m interested in (1) using foundational LLMs (especially SLMs) to solve both core research questions and real, user-facing problems, (2) aligning and integrating multiple modalities so models can reason across diverse inputs and contexts, (3) developing robust training methodologies: data-efficient objectives, distillation, refined pre-training and post-training techniques, (4) designing efficient architectures for language models, (5) improving inference efficiency for low-latency, low-memory execution on consumer hardware, and (6) ensuring robustness and alignment through rigorous evaluation, safety safeguards, and reliability testing.

Publications

Below is a list of my publications in reversed chronological order.


2025

  1. ACL
    Learning to Reason Over Time: Timeline Self-Reflection for Improved Temporal Reasoning in Language Models
    Adrián Bazaga, Rexhina Blloshmi, Bill Byrne, and Adrià Gispert
    In ACL 2025 (Association for Computational Linguistics) 2025
    arXiv Code

2024

  1. ICLR
    Unsupervised Pretraining for Fact Verification by Language Model Distillation
    Adrián Bazaga, Pietro Liò, and Gos Micklem
    In ICLR 2024 (International Conference on Learning Representations) 2024
    arXiv Code
  2. EMNLP
    HyperBERT: Mixing Hypergraph-Aware Layers with Language Models for Node Classification on Text-Attributed Hypergraphs
    Adrián Bazaga, Pietro Liò, and Gos Micklem
    In EMNLP 2024 (Empirical Methods in Natural Language Processing) 2024
    arXiv Code
  3. ICML
    TabMDA: Tabular Manifold Data Augmentation for Any Classifier using Transformers with In-context Subsetting
    Andrei Margeloiu,  Adrián Bazaga, Nikola Simidjievski, Pietro Liò, and Mateja Jamnik
    In ICML 2024 (International Conference on Machine Learning) 2024
    arXiv Code
  4. arXiv
    FLUID-LLM: Learning Computational Fluid Dynamics with Spatiotemporal-aware Large Language Models
    Max Zhu,  Adrián Bazaga, and Pietro Liò
    In arXiv:2406.04501 2024
    arXiv Code

2023

  1. arXiv
    SQLformer: Deep Auto-Regressive Query Graph Generation for Text-to-SQL Translation
    Adrián Bazaga, Pietro Liò, and Gos Micklem
    In arXiv:2310.18376 2023
    arXiv Code
  2. ICLR
    Language Model Knowledge Distillation for Efficient Question Answering in Spanish
    Adrián Bazaga, Pietro Liò, and Gos Micklem
    In ICLR 2024 (International Conference on Learning Representations) 2023
    arXiv Code
  3. Annals of Oncology
    Multi-site validation of a deep learning solution for HER2 profiling of breast cancer from H&E-stained pathology slides
    S Arslan, P Pandya, F Ntelemis, S Wolf, J Schmidt, A Geraldes,  A Bazaga, D Mehrotra, J Nyonyintono, S Singhal, and others
    Annals of Oncology 2023

2022

  1. Database
    HumanMine: advanced data searching, analysis and cross-species comparison
    Rachel Lyne,  Adrián Bazaga, Daniela Butano, Sergio Contrino, Joshua Heimbach, Fengyuan Hu, Alexis Kalderimis, Mike Lyne, Kevin Reierskog, Radek Stepan, and others
    Database 2022

2021

  1. Nature Scientific Reports
    Translating synthetic natural language to database queries with a polyglot deep learning framework
    Adrián Bazaga, Nupur Gunwant, and Gos Micklem
    Nature Scientific Reports 2021
    arXiv Code

2020

  1. Nature Scientific Reports
    Genome-wide investigation of gene-cancer associations for the prediction of novel therapeutic targets in oncology
    Adrián Bazaga, Dan Leggate, and Hendrik Weisser
    Nature Scientific Reports 2020

2019

  1. Applied Soft Computing
    A Convolutional Neural Network for the Automatic Diagnosis of Collagen VI related Muscular Dystrophies
    Adrián Bazaga, Mònica Roldán, Carmen Badosa, Cecilia Jiménez-Mallebrera, and Josep M. Porta
    Applied Soft Computing 2019
    arXiv Code
  2. WSOM
    Network Community Cluster-Based Analysis for the Identification of Potential Leukemia Drug Targets
    Adrián Bazaga, and Alfredo Vellido
    In International Workshop on Self-Organizing Maps 2019
  3. Neuromuscular Disorders
    Automated diagnosis of collagen VI related muscular dystrophies using advanced image analysis and machine learning
    Mónica Roldán,  Adrián Bazaga, Carmen Badosa, Josep M. Porta, and Cecilia Jimenez-Mallebrera
    In Neuromuscular Disorders 2019

2018

  1. arXiv
    Performance Evaluation of an Algorithm-based Asynchronous Checkpoint-Restart Fault Tolerant Application Using Mixed MPI/GPI-2
    Adrián Bazaga, and Michal Pitonak
    arXiv preprint 2018
  2. Bioinformatics
    BIOLITMAP: a web-based geolocated, temporal and thematic visualization of the evolution of bioinformatics publications
    Adrián Bazaga, Alfonso Valencia, and MJ Rementeria-Núñez
    Bioinformatics 2018