Research
Long-Horizon Autonomous Research Infrastructure
Infrastructure for reliable AI research loops: agent-driven code changes, isolated GPU experiments, candidate evaluation, and persistent experiment lineage. Execution protocols and recovery mechanisms support concurrent workers and process/storage failures, with fault-injection tests and TLA+ models used to check key invariants.
DScout: Diffusion-Drafted Speculative Decoding
Training-inference co-design for diffusion-based token drafting. The work combines prefix-aware drafter training with acceptance-latency analysis to improve speculative decoding efficiency.
Penelope: KV Cache Eviction for Agentic LLM Serving
KV cache eviction for multi-turn agents that preserves shared prefixes and cached suffixes, recomputing only missing history segments. Co-designed eviction, KV recomputation, and unified GPU-CPU cache management in SGLang, with evaluation on real agent traces.
Paper: Rethinking KV Cache Eviction for Agentic LLM Serving
dLLM-Serve: Memory-Efficient Diffusion LLM Serving
A serving system that addresses transient activation-memory peaks and changing compute/bandwidth demands during diffusion inference, co-optimizing memory budgets, request scheduling, and attention execution.
APEX: Asynchronous CPU-GPU Execution for LLM Inference
An asynchronous hybrid scheduler that overlaps CPU-offloaded work with GPU execution during memory-constrained decoding. A performance model guides heterogeneous resource allocation.
SLED: Distributed Speculative LLM Serving
Contributed to a distributed speculative decoding framework coordinating token drafting on heterogeneous edge devices with batched target-model verification on a shared server.