At last: Russian LLM fine-tune
Feb. 22nd, 2024 08:50 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
izardFor a long time, there was no decent local LLM that understood and spoke Russian well. I've tested many fine-tunes, and they all behaved significantly dumber when answering in Russian than their respective English originals.
Russian internet has a huge potential training corpus second only to English; but smart French, German, Spanish and Arabic fine-tunes for local LLMs existed for months now, and there was no Russian one.
Until Serge Gotsuliak released his finetune few days ago.
Highly recommend it, works great with llama.cpp. I was missing RAG on my Russian books, will set it up now.
There is a drawback: 4-bit quant that I tested needs GPU[s] with ~48GB VRAM, or Apple silicon machine with 64GB+ RAM to run at decent speed.
Russian internet has a huge potential training corpus second only to English; but smart French, German, Spanish and Arabic fine-tunes for local LLMs existed for months now, and there was no Russian one.
Until Serge Gotsuliak released his finetune few days ago.
Highly recommend it, works great with llama.cpp. I was missing RAG on my Russian books, will set it up now.
There is a drawback: 4-bit quant that I tested needs GPU[s] with ~48GB VRAM, or Apple silicon machine with 64GB+ RAM to run at decent speed.
