The Kaitchup – AI on a Budget
Subscribe
Sign in
Share this discussion
Marlin: Nearly Ideal Inference Speed for 4-bit Models with vLLM (1k+ tokens/sec)
newsletter.kaitchup.com
Copy link
Facebook
Email
Note
Other
Marlin: Nearly Ideal Inference Speed for…
Benjamin Marie
Mar 28
5
Share this post
Marlin: Nearly Ideal Inference Speed for 4-bit Models with vLLM (1k+ tokens/sec)
newsletter.kaitchup.com
Copy link
Facebook
Email
Note
Other
20
This thread is only visible to paid subscribers of The Kaitchup – AI on a Budget
Subscribe to view →
Comments on this post are for paid subscribers
Subscribe
Already a paid subscriber?
Sign in
Share
Copy link
Facebook
Email
Note
Other
This site requires JavaScript to run correctly. Please
turn on JavaScript
or unblock scripts
Marlin: Nearly Ideal Inference Speed for 4-bit Models with vLLM (1k+ tokens/sec)
Marlin: Nearly Ideal Inference Speed for…
Marlin: Nearly Ideal Inference Speed for 4-bit Models with vLLM (1k+ tokens/sec)
This thread is only visible to paid subscribers of The Kaitchup – AI on a Budget
Comments on this post are for paid subscribers