Baotong Lu Get in touch

Research at the intersection of AI and systems

Baotong Lu

AI for Systems. Systems for AI.

Senior Researcher at Microsoft Research Asia

I build systems that help AI use context, memory, and computation more effectively, and explore how AI can improve the systems underneath it.

Google Scholar GitHub Email
From context to memory to more capable agents A conceptual map of three related research directions. Sparse access selects from a large context; a hierarchy of memory tiers connects storage and computation; capability specialists contribute to an evolving agent memory program. Context, memory, and agents Three related research directions, arranged vertically: sparse attention for long-context inference, hierarchical memory for storage and retrieval, and capability-driven evolution of agent memory. Long-context inference Sparse access Hierarchical memory Storage & retrieval Agent memory Capability-driven evolution
A conceptual view of my research: making context accessible, memory scalable, and agents more capable.

Selected research

My current focus is on agentic AI and systems: how to serve longer contexts, manage growing memory, and help agents improve through experience.

Long-context serving

SPIN

Make sparsity work at the system level.

Sparse attention can reduce the KV state an LLM needs to access. But irregular transfers between GPU and CPU memory can erase those savings. SPIN co-designs sparse-attention execution and hierarchical memory management to turn algorithmic sparsity into more efficient serving.

Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving
Sparse attention accesses selected KV blocks across a memory hierarchy Sparse attention GPU memory CPU memory
Co-designing sparse access and memory movement.

Memory & retrieval

RetroInfer

Treat long-context inference as a storage problem.

As context grows, so does the memory behind inference. RetroInfer brings a vector-storage perspective to this problem, using retrieval to make a large KV state accessible without treating every part of it as equally relevant at every step.

RetroInfer: A Vector-Storage Engine for Scalable Long-Context LLM Inference
A query retrieves relevant blocks from a larger vector store for inference Inference Vector storage Retrieve
A large memory, accessed selectively.

Self-evolving agent memory

PrisMem

Improve capabilities, not just an overall score.

Improving one memory capability can hide regressions in another. PrisMem guides evolution along individual capability dimensions, refines specialists using task feedback, and integrates their complementary gains into a unified memory program.

Capability-Driven Self-Evolution of Agent Memory
Capability specialists contribute complementary improvements to a unified memory program Feedback Specialists Integrate
Capability refinement, followed by integration.

Systems for AI. Efficient inference, long-context serving, vector search, and memory for AI agents. AI Systems publications

AI for Systems. Learning-driven data structures and adaptive systems, building on my work in learned indexes and database systems. Database publications

About

Portrait of Baotong Lu

I am a Senior Researcher in the Systems and Networking Research Group at Microsoft Research Asia.

I received my PhD from The Chinese University of Hong Kong in 2023 and visited Simon Fraser University in 2019 and 2023. My research spans machine learning systems and database management systems, with a focus on emerging applications and hardware.

I received the 2021 ACM SIGMOD Research Highlight Award. Dash and APEX were also selected among the Most Influential ArXiv (Databases) Papers (top 30 per year).

  1. 2019 / 2023Simon Fraser UniversityResearch visits
  2. 2023The Chinese University of Hong KongPhD
  3. NowMicrosoft Research AsiaSenior Researcher

Selected publications

Google Scholar

AI Systems

* Students I advised

  1. PREPRINT

    Capability-Driven Self-Evolution of Agent Memory

    Yaoqi Chen, Yuru Feng, Qianxi Zhang, Baotong Lu, Jianan Lu, Zhirui Wang, Shusen Xu, Zewen Jin, Zengzhong Li, Cheng Li, Qi Chen

  2. EMNLP 2026

    Beyond Semantic Organization: Memory as Execution State Management for Long-Horizon Agents

    Yaoqi Chen, Haibin Lai, Yuru Feng, Chuyu Han, Qianxi Zhang, Baotong Lu, Menghao Li, Xinjiang Wang, Zewen Jin, Hao Wu, Cheng Li, Qi Chen

  3. PREPRINT

    Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving

    Zihan Zhao*, Baotong Lu, Shengjie Lin*, Yizou Chen, Jing Liu, Yanqi Zhang, Ziming Miao, Ming-Chang Yang, Haiying Shen, Qi Chen, Fan Yang

  4. PREPRINT

    DualSpec: Accelerating Deep Research Agents via Dual-Process Action Speculation

    Shuzhang Zhong*, Baotong Lu, Qi Chen, Chuanjie Liu, Fan Yang, Meng Li

  5. VLDB 2026

    RetroInfer: A Vector-Storage Engine for Scalable Long-Context LLM Inference

    Yaoqi Chen*, Jinkai Zhang*, Baotong Lu, Qianxi Zhang, Chengruidong Zhang, Jingjia Luo*, Di Liu, Huiqiang Jiang, Qi Chen, Jing Liu, Bailu Ding, Xiao Yan, Jiawei Jiang, Chen Chen, Mingxing Zhang, Yuqing Yang, Fan Yang, Mao Yang

  6. VLDB 2026

    Aker: Density-Aware Approximate Caching for Vector Search

    Sukjoon Oh*, Minki Kang, Dohyun Kim, Baotong Lu, Jing Liu, Qianxi Zhang, Qi Chen, Youjip Won

  7. DAC 2026

    KEEP: A KV-Cache-Centric Memory Management System for Efficient Embodied Planning

    Zebin Yang, Tong Xie, Baotong Lu, Shaoshan Liu, Bo Yu, Meng Li

  8. SIGMOD 2026

    Towards Efficient and Scalable Distributed Vector Search with RDMA

    Xiangyu Zhi, Meng Chen*, Xiao Yan, Baotong Lu, Hui Li, Qianxi Zhang, Qi Chen, James Cheng

  9. SIGMOD 2026

    Attribute Filtering in Approximate Nearest Neighbor Search: An In-depth Experimental Study

    Mocheng Li, Xiao Yan, Baotong Lu, Yue Zhang, James Cheng, Chenhao Ma

  10. PPoPP 2025

    SGDRC: Software-Defined Dynamic Resource Control for Concurrent DNN Inference on NVIDIA GPUs

    Yongkang Zhang, Haoxuan Yu, Chenxia Han, Cheng Wang, Baotong Lu, Yang Li, Xiaowen Chu, Huaicheng Li

  11. NeurIPS 2025

    RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval

    Di Liu*, Meng Chen*, Baotong Lu, Huiqiang Jiang, Zhenhua Han, Qianxi Zhang, Qi Chen, Chengruidong Zhang, Bailu Ding, Kai Zhang, Chen Chen, Fan Yang, Yuqing Yang, LiLi Qiu

    NeurIPS 2025 main track · NeurIPS 2024 ENLSP Workshop Best Paper Award

  12. AAAI 2020

Database Management Systems

  1. VLDB 2026

    Shard: A Scalable and Resize-optimized Hash Index on Disaggregated Memory

    Hantian Zha, Teng Ma, Baotong Lu, Yuansen Wang, Dongbiao He, Yuanhui Luo, Dafang Zhang, Yunpeng Chai, Yuxing Chen, Anqun Pan

  2. VLDB 2024

    DEX: Scalable Range Indexing on Disaggregated Memory

    Baotong Lu, Kaisong Huang, Mike Chieh-Jan Liang, Tianzheng Wang, Eric Lo

  3. VLDB 2022

    Are Updatable Learned Indexes Ready?

    Chaichon Wongkham, Baotong Lu, Chris Liu, Zhicong Zhong, Eric Lo, Tianzheng Wang

  4. VLDB 2022

    APEX: A High-Performance Learned Index on Persistent Memory

    Baotong Lu, Jialin Ding, Eric Lo, Umar Faroop Minhas, Tianzheng Wang

    Abstract at NVMW 2022 ↗

  5. SIGMOD RECORD 2021

    Scaling Dynamic Hash Tables on Real Persistent Memory

    Baotong Lu, Xiangpeng Hao, Tianzheng Wang, Eric Lo

    Volume 50, Issue 1 · Technical Perspective from Prof. Ken Ross

  6. VLDB 2020

    Dash: Scalable Hashing on Persistent Memory

    Baotong Lu, Xiangpeng Hao, Tianzheng Wang, Eric Lo

    Code ↗ · Abstract at NVMW 2021 ↗

    2021 ACM SIGMOD Research Highlight Award

    DragonflyDB's core hash table structure is based on Dash.

Academic service

Publications Chair

SoCC 2026

Program Committees

SIGMOD 2027
VLDB 2027 (PVLDB 20), 2026 (PVLDB 19), 2025 (PVLDB 18)
EuroSys 2027, 2026 · SoCC 2026
ICDE 2026 · EDBT 2026 · APSys 2025
ChinaSys 2025, 2024

Editorial & Reviewing

Reviewer: ICLR 2027
VLDB Journal Reviewer Board, 2025–2026
SIGMOD Availability Committee 2023, 2022
Reviewer: VLDBJ, ACM TOS, IEEE TKDE, IEEE TCAD

External reviewing

SIGMOD 2024, ICDE 2024, SIGMOD 2022, ICDE 2022, VLDB 2021, SIGMOD 2021, ICDE 2021, EDBT 2021, ATC 2020, DASFAA 2019, VLDB 2018 (Demo)

Selected awards

Best Paper Award, ENLSP Workshop @ NeurIPS
2024
ACM SIGMOD Research Highlight Award
2021
Mitacs Globalink Research Award
2022
China National Scholarship
2015, 2017