Skip to content
View shiloz-stack's full-sized avatar
❄️
Focusing
❄️
Focusing
2 followers · 3 following

Block or report shiloz-stack

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
shiloz-stack/README.md

🚀 GPU Systems & Inference Engineering

Building custom CUDA, Triton, and TIRx kernels for LLM inference — speculative decoding, MoE scheduling, and the hardware boundary.

⬛ blackwell-moe-kernel-scheduler

Workload-aware MoE kernels on NVIDIA Blackwell

Project-owned BF16 MoE grouped-GEMM family in TIRx: double-buffered → persistent → warp-specialized → dynamic / chunked / hybrid → CLC.

TMA · tcgen05 · TMEM · Cluster Launch Control

🎯 Routing-aware dispatch under Zipfian / heavy-hitter skew · Nsight on B200

⚡ VeloSpec

Grammar-aware speculative decoding

3 CUDA + 1 Triton kernel. Adaptive K via bitmask popcount.

popcount_density 111× · grammar_argmax fused · fused_sample online softmax

🔥 2.7× throughput on enum-heavy schemas · A100-SXM4

🛠️ CUDA · TIRx · Triton · PyTorch · C++ · Nsight Compute · Blackwell · A100

Popular repositories Loading

  1. VeloSpec VeloSpec Public

    Grammar-aware adaptive speculative decoding for structured LLM output — custom CUDA/Triton kernels, 2.7× throughput on enum-heavy schemas

    Python 3 2

  2. Flare Flare Public

    From-scratch Triton kernels for modern attention variants — FlashAttention v2, MLA (DeepSeek), and Kimi Delta Attention. Unified benchmark suite on A100.

    Python 2

  3. OverlapRL OverlapRL Public

    A minimal async GRPO training pipeline with staleness-aware scheduling and compute overlap for single-GPU RL post-training.

    Python 1

  4. blackwell-moe-kernel-scheduler blackwell-moe-kernel-scheduler Public

    Workload-aware persistent expert-tile scheduling for irregular MoE inference kernels on NVIDIA Blackwell GPUs

    Cuda 1

  5. shiloz-stack shiloz-stack Public

  6. sglang sglang Public

    Forked from sgl-project/sglang

    SGLang is a high-performance serving framework for large language models and multimodal models.

    Python