The Kaitchup – AI on a Budget

Home
Notes
AI Notebooks
The Kaitchup Pro
Table of Contents
The Kaitchup's Book
Tutorials
Models
Archive
About

Tutorials

Run and Serve Faster VLMs Like Pixtral and Phi-3.5 Vision with vLLM
Understanding how much memory you need to serve a VLM
 • 
Benjamin Marie
2
Multimodal RAG with ColPali and Qwen2-VL on Your Computer
Retrieve and exploit information from PDFs without OCR
 • 
Benjamin Marie
3
GuideLLM: Is Your Server Ready for LLM Deployment?
Simulate real-world inference workloads with GuideLLM
 • 
Benjamin Marie
Falcon Mamba, Jamba, RWKV... Can You Use Them on Your Computer?
A close look at quantization and parameter-efficient fine-tuning (LoRA/QLoRA) for SSMs, RWKV, and hybrid models
 • 
Benjamin Marie
Run Qwen2-VL on Your Computer with Text, Images, and Video, Step by Step
Your local multimodal chat model
 • 
Benjamin Marie
4
Run Llama 3.1 70B Instruct on Your GPU with ExLlamaV2 (2.2, 2.5, 3.0, and 4.0-bit)
Is ExLlamaV2 Still Good Enough?
 • 
Benjamin Marie
3
Mistral-NeMo: 4.1x Smaller with Quantized Minitron
How Pruning, Knowledge Distillation, and 4-Bit Quantization Can Make Advanced AI Models More Accessible and Cost-Effective
 • 
Benjamin Marie
4
Fine-tuning Phi-3.5 MoE and Mini on Your Computer
With code to quantize the models with bitsandbytes and AutoRound
 • 
Benjamin Marie
11
QLoRA with AutoRound: Cheaper and Better LLM Fine-tuning on Your GPU
Bitsandbytes is not your only option
 • 
Benjamin Marie
1
SmolLM: Full Fine-tuning and Aligning Tiny LLMs on Your Computer
With supervised fine-tuning and distilled DPO
 • 
Benjamin Marie
9
Multi-GPU Fine-tuning for Llama 3.1 70B with FSDP and QLoRA
What you can do with only 2x24 GB GPUs, and a lot of CPU RAM
 • 
Benjamin Marie
18
Serve Multiple LoRA Adapters with vLLM
Without any increase in latency
 • 
Benjamin Marie
7
© 2024 The Kaitchup
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great culture