Implementing a Transformer Block
InterviewCodingtransformersimplementationarchitecture
Related module: Sequence models and attention
Problem statement
Implement a single Transformer decoder block in PyTorch with:
- Multi-head self-attention
- Feed-forward network
- Layer normalization (Pre-LN style)
- Residual connections
Include causal masking for autoregressive generation.