Grid Controls

Wilson Zheng

WILSON ZHENG

Inference Engineer • AI Infra Engineer
Game Lover • Astrophotographer

Hi, this is Wilson! I'm a second year masters student at CMU's MS in Artificial Intelligence and Innovation program. I was an intern at Google working on TPU firmware security. Recently, I've been spending most of my time doing open source work at SGLang-Omni.

Beyond academics, I've done a little bit of astrophotography, fully documented by the blog on my personal website. This hobby will continue when I start working and settle down and be willing to spend money on the equipment. A dream of mine is to have one of my images be on apod.nasa.gov!

Check out my illustrated notes on SGLang and my humble astrophotography blog.

Software Engineering Intern

Google

May 2026 – Aug 2026

Sunnyvale, CA

  • Engineered a virtual fault-injection framework in C++20 for security controllers, simulating instruction-skip attacks to uncover secure-boot bypass vulnerabilities
  • Authored hardened security patches mitigating glitch attacks via compiler-aware anti-glitch loop checks and reinforced verification primitives
  • Debugged bare-metal firmware and analyzed raw Arm disassembly to confirm both vulnerabilities and their fixes
  • Extended existing test suites to exhaustively evaluate Secure Boot verification logic under fault-injection scenarios

Software Development Engineer Intern

Amazon Web Services

Jun 2025 – Aug 2025

Beijing, China

  • Engineered Amazon Q-powered AI unit test generation tool, raising code coverage to up to 90% across services
  • Integrated into Amazon's CI/CD pipeline, enabling automatic test generation and saving ~10 hr/wk of manual testing
  • Designed event-driven, scalable architecture on AWS to support asynchronous test generation workflows
  • Developed multi-stage validation pipeline to detect and filter LLM hallucinations, significantly improving test reliability by eliminating invalid test cases before integration
SGLang-Omni Logo

SGLang-Omni

June 2026 – Present

Contributor; lead for PersonaPlex and speculative decoding

PersonaPlex-7B full-duplex speech-to-speech serving

  • Integrated NVIDIA's PersonaPlex-7B into SGLang-Omni with greedy parity against NVIDIA's reference
  • Co-designed the model-agnostic full-duplex session runtime: the model listens and speaks at the same time in 80 ms frames, with no VAD or turn boundaries, and keeps up in real time over 10-minute calls on one GPU
  • Led the optimization roadmap with ~10 contributors: 10× throughput at concurrency 16 and 74% lower p95 TTFA
  • Key optimizations: cross-request batching, CUDA-graph replay for the streaming Mimi decoder (5.6× faster decode step, 22% lower TTFA) and the 7B backbone decode (9% lower latency), and a 6.6× faster encoder→LM handoff

Speculative decoding

  • Integrated DFlash, DSpark and EAGLE3 drafts with lossless verification: 1.66× ASR throughput on Qwen3-Omni (WER unchanged) and 1.64× on Fun-CosyVoice3 TTS (1.41× end-to-end)
  • Built a multi-GPU training pipeline and trained DFlash drafts for Qwen3-ASR 0.6B and 1.7B on ~590k English, Chinese and Japanese clips, lifting the serving throughput gain from +69% to +79% with WER unchanged
  • Implemented DFlash in the native MLX C++ runtime and the Voxt Mac app: 3.0× decode throughput on Qwen3-ASR-1.7B (1.8× end-to-end) and 1.65× on 0.6B on an Apple M5

Fun-ASR-Nano serving optimization

  • Built a pre-LM audio encoder service that runs full-context encoding before LM admission, with single-flight dedup and an LRU embedding cache: 1.3× cold / 3.3× warm throughput at H100 concurrency 32
  • Profiled with Nsight Systems and built a bucketed CUDA-graph runner for the launch-bound 70-layer SANM encoder, delivering 2.3× throughput (56→130 samples/s) at concurrency 32 with no accuracy regression
Python C++ PyTorch CUDA MLX Speculative Decoding AI Infra SGLang

High-Performance CUDA Kernel for DeepSeek Sparse Attention

Feb 2026 – Apr 2026

Optimized a BF16 CUDA kernel on NVIDIA B200 for DSA, achieving a 22–50× speedup over the PyTorch reference across 23 trace workloads using WMMA m16n16k16 tensor cores, double-buffered cp.async KV gather, and a SplitK + online-softmax reduce pipeline that eliminated the small-batch SM occupancy bottleneck.

CUDA NVIDIA B200 Tensor Cores Kernel Optimization

NodeTorch – Visual ML Model Builder

Apr 2026 – Present

Educational drag-and-drop node-based environment that lets users compose neural networks visually and train them, giving users a sandbox to experiment with model architectures without writing code from scratch.

Designed a layered, registry-driven execution engine separating graph runtime, type system, and ML domain. Built rich interactive visualizations of every layer's behavior (forward activations, parameter distributions, attention patterns, loss surfaces). Integrated an in-app AI assistant with a backend agent loop allowing any LLM to interact with the app through a schema-defined tool interface.

PyTorch React FastAPI Node-based UI AI Assistant

Hypoll – Interactive Storytelling Social Platform

Oct 2025 – Apr 2026

AI-powered social platform using React Native and FastAPI, enabling users to create interactive audio content through conversations with an AI host. Architected end-to-end AI content pipeline: GPT-4o mini script generation, Whisper transcription, Gemini image generation and ElevenLabs TTS with word-level subtitle alignment.

Designed scalable backend with async PostgreSQL using psycopg3, deployed on GCP. Implemented semantic search with Qdrant and OpenAI embeddings for content discovery. Engineered secure infrastructure (JWT, rate limiting, moderation) and stress-tested with Locust simulating hundreds of concurrent users.

React Native FastAPI GPT-4o mini ElevenLabs TTS Qdrant PostgreSQL GCP
Keyboard controller demo

Hand Gesture-Based Keyboard Controller

Mar 2025 - May 2025

Developed a real-time computer vision-powered hand gesture recognition system with PyTorch and OpenCV, achieving 95% confidence for gesture classification across 12 distinct hand poses captured via webcam feed.

Built gesture-to-keyboard mapping application motivated by accessibility and hands-free computing needs.

PyTorch OpenCV Computer Vision Real-time Processing Accessibility

Carnegie Mellon University

Master of Science in Artificial Intelligence and Innovation

Expected May 2027

Pittsburgh, PA

Relevant Courses:

Computer Architecture, Compilers, Operating Systems, Cloud Computing, Machine Learning, Deep Learning, LLM Systems, AI Engineering, Generative AI for Music, NLP

The University of Virginia

Bachelor's Degree in Computer Science & Physics

August 2021 - May 2025

Charlottesville, VA

Echols Scholar College Science Scholar Dean's List

Relevant Courses:

Data Structures & Algorithms, Software Development Methods, Computer Architecture, Machine Learning, Cybersecurity, Compilers, Operating System, Database Systems, Cloud Computing, Quantum Computing