Awni Hannun (@awnihannun)
2024-09-26 | โค๏ธ 2732 | ๐ 240
Llama 3.2 1B in 4-bit runs at ~60 toks/sec with MLX Swift on my iPhone 15 pro.
Itโs quite good and easily runs on-device: https://x.com/awnihannun/status/1839330067039887622/video/1
๋ฏธ๋์ด
