โฮตsam (@Hesamation)
2025-08-09 | โค๏ธ 715 | ๐ 106
The perfect weekend article just dropped by @rasbt on gpt-oss architectures.
Itโs wild to see how fast these architectures have leveled up since GPT-2 in 2019.
abs positional emb โ RoPE GELU MLP โ Swish + SwiGLU one dense feed-forward โ MoE Multi-head attention โ GQA โฆ https://x.com/Hesamation/status/1954237350512619868/photo/1
๋ฏธ๋์ด
