A datapoint on budgeting. On quickpod going rate ATM for an RTX4090 is $0.23/hr. 6500 hours to get to the $1700 price new. 9 months or so, 24hrs a day.
In practice, I don't think this is possible to achieve with quickpod. The machine are not reliable enough and we can't rent the GPU for a long time. Also, this doesn't seem to include the cost of storage.
But indeed, for quick fine-tuning, quickpod can be an option. For long, reproducible, and reliable fine-tuning, buying the GPU is the best option.
CPU memory, CPU model and RAM are other variables. That number came with an underconfigured 16GM of RAM and 4 processors. Is your container on an overconfigured motherboard with too many GPUs for the number of PCI lanes, etc.?
Further up the food chain you get out of the marketplace of repurposed *coin mining gear and old gaming rigs to the all datacenter systems and recently released GPU models, it gets better. I hope to get far enough along that I'm driven up there by requirements. So far I'm on a tight budget.
My final "yes, but". :-) The big boys have MLOps teams. Out here, some of us will be drawn to that.
A datapoint on budgeting. On quickpod going rate ATM for an RTX4090 is $0.23/hr. 6500 hours to get to the $1700 price new. 9 months or so, 24hrs a day.
In practice, I don't think this is possible to achieve with quickpod. The machine are not reliable enough and we can't rent the GPU for a long time. Also, this doesn't seem to include the cost of storage.
But indeed, for quick fine-tuning, quickpod can be an option. For long, reproducible, and reliable fine-tuning, buying the GPU is the best option.
CPU memory, CPU model and RAM are other variables. That number came with an underconfigured 16GM of RAM and 4 processors. Is your container on an overconfigured motherboard with too many GPUs for the number of PCI lanes, etc.?
Further up the food chain you get out of the marketplace of repurposed *coin mining gear and old gaming rigs to the all datacenter systems and recently released GPU models, it gets better. I hope to get far enough along that I'm driven up there by requirements. So far I'm on a tight budget.
My final "yes, but". :-) The big boys have MLOps teams. Out here, some of us will be drawn to that.