My Projects

Key Contributions:

• Contributed as a core developer to SGLang, a high-performance serving framework for LLMs
• Co-lead the multi-modal language model (MLLM) team to integrate various MLLMs into the SGLang framework
• Investigated and resolved critical issues for the Qwen-VL model series, enhancing its stability within SGLang
• Refactored the multi-modal processing pipeline, improving robustness and maintainability
• Collaborated with the VeRL team to enable SGLang as a rollout backend for MLLMs
• Added support for new models including Qwen, Gemma, Mistral, etc.
• Refactored the OpenAI-compatible server for improved robustness and maintainability
• Established and maintained the MLLM benchmarking suite
• Maintained the Continuous Integration/Continuous Deployment (CI/CD) pipeline

Technologies Used:

Key Contributions:

• Contributed as a core developer to ServerlessLLM, an open-source serving system for cost-effective multi-LLM deployment on resource-constrained GPU environments
• Implemented a distributed profiling component for Ray workers to monitor and optimize performance
• Containerized the project using Docker to streamline cross-platform deployment
• Enhanced the auto-scaling component to enable elastic model instance scaling and efficient GPU multiplexing
• Developed a command-line interface (CLI) and comprehensive test suites to ensure system reliability and usability

Technologies Used:

Key Contributions:

• Developed a state-of-the-art Speech Emotion Recognition (SER) system by transitioning from traditional ML methods to fine-tuning self-supervised models
• Fine-tuned the cross-lingually pre-trained model to achieve SOTA performance
• Implemented extensive data augmentation and hyperparameter optimization techniques to enhance model robustness and generalization

Technologies Used:

Key Contributions:

• Engineered a C-based simulator that merges cache systems with virtual memory management, featuring TLB and Page Tables, to accurately simulate address translation from virtual to physical
• Introduced adjustable settings for cache sizes, TLB entries, and page replacement methods, offering the ability to mimic different computing environments. This adaptability is key for analyzing how system performance varies with configuration changes
• Established comprehensive error handling to detect, report, and resolve simulation issues, ensuring the simulator's reliability and producing precise outcomes while enhancing the user experience by minimizing disruptions

Technologies Used: