Combine Multiple LoRA Adapters for Llama 2

[baconnier loic]

And LORAX, i made a container with it work fine, just have to make a proxy to have open air api like

[Benjamin Marie]

I didn't know lorax. It looks like S-LoRA but I might be wrong.

[baconnier loic]

Must have a look at Dare

https://huggingface.co/papers/2311.03099

[Tryppy]

How about keeping the adaptor for different base models?

[Tryppy]

Any need to fine tune again? Or as base models update, just apply the adaptor?

[Benjamin Marie]

An adapter is fine-tuned for a specific base model. If we change the base model, we have to fine-tune again the adapter.

[Tryppy]

Thank you