Machine Learning Infrastructure

Demystifying PyTorch Performance: From Standard nn.Linear to High-Performance Fused MLPs

An in-depth exploration of PyTorch profiling, identifying memory-bandwidth bottlenecks in standard neural network layers, and leveraging kernel fusion via Triton and torch.compile to optimize MLP performance.