Accelerating Pre-processing in VLM Inference
- Level
- Bachelor
- Supervisor
- Sepideh Zohdi, Lin Wang
- Status
- Completed
- Date of Completion
- 2026.07
- Student
- Jonas Michael Cicholas
Description
Vision-Language Models (VLMs) are increasingly used for video tasks such as VideoQA, surveillance and robotics. While recent inference optimizations (PagedAttention, FlashAttention, continuous batching, speculative decoding) target GPU-side execution, our benchmarking work [1] shows that CPU-side video decoding dominates time-to-first-token (TTFT) for up to 80% of videos. vLLM currently decodes frames sequentially and transfers them to the GPU only after all frames are ready, leaving the GPU idle. Existing solutions either require multi-GPU NVDEC hardware or decode the whole video and sacrifice accuracy.
Research question: How much can TTFT and GPU idle time be reduced on single-GPU deployments by (a) decoding only the GOPs containing sampled frames in parallel and (b) overlapping CPU decoding with PCIe transfer through pieplining?
The thesis covers: (1) a parallel GOP-level decoder integrated into vLLM’s multimodal processor, (2) a pipelined GPU transfer with a batch-size trade-off study, (3) their combination, and (4) evaluation of TTFT, GPU idle time and throughput against unmodified vLLM.
Requirements
- Solid Python and PyTorch skills
- Basic understanding of video coding (GOPs, I/P/B-frames) and FFmpeg/PyAV, or willingness to learn
- Familiarity with concurrency (threads, queues, producer-consumer) and CPU–GPU data transfer
- Experience with profiling/benchmarking on Linux; interest in systems research; vLLM exposure is a plus
References
- S. Zohdi and L. Wang. Before the First Token: Benchmarking Data Preprocessing in Vision-Language Models. EuroMLSys 2026. https://doi.org/10.1145/3805621.3807641