fine-tuning-open-source-models.mdx - Visual Studio Code
When off-the-shelf AI isn't enough: a business guide to fine-tuning open-source modelsAI & ML

When off-the-shelf AI isn't enough: a business guide to fine-tuning open-source models

When fine-tuning an open-source model pays off, how it differs from RAG, how to pick a model (Qwen, Mistral, Gemma, Llama, Bielik), and what to prepare (2026).

Author: Published Updated 10 min read

Summary

  • Fine-tuning means further training an existing model on your examples so it adopts the format, style, and vocabulary of your domain. You are not training a model from scratch.
  • Fine-tuning teaches behavior, not facts. For answers grounded in current company documents, RAG works better, and the two can be combined.
  • LoRA and QLoRA mean that adapting a large model no longer requires a supercomputer: in the QLoRA paper, a 65B model was fine-tuned on a single 48 GB GPU.
  • Choosing a model in 2026 comes down to license and language first: Gemma 4, Bielik, Mistral Large 3 and Small 4, and the smaller Qwen 3.8 models use Apache 2.0, but the flagship Qwen 3.8 and Mistral Medium 3.5 use their vendors' own licenses, and Llama 4 carries restrictions for EU-based companies.
  • Start with evaluation and a small set of good examples. If a few dozen examples improve nothing, the problem usually lies elsewhere.

Quick answer

Fine-tuning is further training of an existing language model, most often an open one (Qwen, Mistral, Gemma, Bielik), on examples from your company so it responds in your format, style, and vocabulary. It pays off when API models keep failing at a specialized task, when data cannot leave your company, or when a large, stable query volume makes a model of your own cheaper to run. It does not replace RAG: for answers grounded in current documents, retrieval from sources works better (Ovadia et al., 2023).

When does standard AI stop being enough?

When the model sounds generic, confuses specialized terms, or ignores the format you need despite good instructions. Most companies start with off-the-shelf tools such as ChatGPT, Claude, or Gemini, and rightly so, because deployment is fast and the capabilities are broad.

The problem shows up later. The model doesn't know your procedures, internal abbreviations, or report layouts, and every answer needs editing. At that point the question shifts from "what can we use?" to "what can we adapt?"

Before reaching for fine-tuning, try two cheaper options: better instructions with examples in the prompt, and RAG, which connects the model to your documents. Fine-tuning makes sense when both have failed or when you need a model that runs on your own infrastructure.

Adapting vs. building from scratch: what's the difference?

Fine-tuning adapts an existing model rather than training a new one. Training from scratch takes thousands of GPUs and huge datasets. For scale: Mistral Large 3 was trained from scratch on 3,000 NVIDIA H200 GPUs (Mistral AI, 2025). That is a path for AI labs, not for companies that want to improve a process.

Fine-tuning takes a model that already understands language and the world and teaches it the specifics of your context. Think of furnishing a well-built house to fit your needs instead of pouring the foundation yourself.

Fine-tuning or RAG: which should you choose?

Fine-tuning teaches the model behavior; RAG gives it knowledge. This is the most common misunderstanding: companies want to "teach the model their documents" through fine-tuning, when retrieval from sources does that job better.

A study comparing the two found that RAG consistently outperformed unsupervised fine-tuning, both for knowledge the model had seen in training and for entirely new knowledge, and that models struggled to learn new facts through fine-tuning alone (Ovadia et al., 2023).

Need Fine-tuning RAG
Consistent answer format, style, brand voice Yes Partly (instructions)
Specialized vocabulary and classification Yes Partly
Answers based on current documents No (knowledge goes stale) Yes
Citation to a source with each answer No Yes
Frequent content changes Requires retraining Just update the index
A smaller, cheaper model in production Yes Not directly

In practice the best systems combine both: a model fine-tuned for the format and language of the domain, with knowledge pulled in from documents through RAG.

Qwen, Mistral, Gemma, Llama, or Bielik: which model should you choose?

License first, then quality on your task and in your language. The open model market changes every few months, so the table below reflects the state as of October 2026. Read the license of the specific version before you choose.

Model family License Strengths Watch out for
Qwen 3.8 (Alibaba) Mixed: smaller models such as 27B under Apache 2.0 (model card), the flagship and Flash variants under Qwen's own licenses (model card) Latest generation of open Qwen models, the first open Max-class model Check the license of the specific variant: the flagship license requires a separate license only from Model as a Service or AI Work Assistant businesses with revenue above USD 50 million over 12 consecutive months (license); the flagship needs multiple GPUs
Mistral Large 3, Small 4, Ministral 3 (Mistral AI) Apache 2.0 (Mistral AI, Small 4 card) European vendor, Mistral 3 supports 40+ languages, small 3B, 8B, and 14B models Test quality in your language on your own examples
Mistral Medium 3.5 (Mistral AI) Modified MIT license (model card) Dense 128B model No rights for companies with monthly revenue above USD 20 million
Gemma 4 (Google) Apache 2.0 (Google, 2026) Models from edge devices up to 31B parameters, easy to self-host Smaller models carry less general knowledge
Bielik v3 (SpeakLeash, ACK Cyfronet AGH) Apache 2.0 (model card) Optimized for Polish, 32 European languages Smaller model, no built-in moderation
PLLuM (consortium of Polish research institutions) Some Apache 2.0 and Llama 3.1, some non-commercial only (PLLuM) Polish data and Polish use cases Use only the commercial versions in a business
Llama 4 (Meta) Llama 4 Community License (license) Large community and tooling ecosystem License restrictions, including for EU companies (below)

A note on Llama 4. Its acceptable use policy in Meta's official GitHub repository withholds the license rights to the multimodal models from EU-based individuals and companies ("are not being granted to you if you are an individual domiciled in, or a company with a principal place of business in, the European Union"), except as end users of a finished product (Llama 4 Acceptable Use Policy). In addition, companies whose products had more than 700 million monthly active users must request a separate license from Meta. For an EU company that wants to fine-tune and self-host a model, Apache 2.0 models are usually the simpler choice.

"Open source" does not mean "free to run." The model itself costs nothing, but you need compute and expertise. "Open" means you control the model, know where it runs, and don't depend on an API vendor's pricing.

How does fine-tuning work in practice?

You take a base model and continue training it on "input, expected output" pairs from your company. From them the model learns:

  • the terminology and abbreviations your company uses,
  • the structure of your reports, notes, and replies,
  • your tone and communication style,
  • how you classify cases and documents.

Today almost nobody updates all of a model's parameters. LoRA (Low-Rank Adaptation) freezes the original weights and trains only small additional matrices. In the original paper, for GPT-3 175B this cut the number of trainable parameters by 10,000 times and GPU memory requirements by 3 times (Hu et al., 2021). QLoRA combines LoRA with 4-bit quantization: its authors fine-tuned a 65B model on a single 48 GB GPU in 24 hours (Dettmers et al., 2023).

In practice teams use open-source libraries such as Hugging Face TRL and PEFT, and the resulting LoRA adapter can be served together with the base model on your own server.

What do you need?

  1. Good examples. Past reports, approved replies, correctly classified cases. Quality matters more than quantity. OpenAI's fine-tuning documentation lists a minimum of 10 examples and reports improvements from 50 to 100 well-crafted examples, noting that the right number depends on the use case (OpenAI).
  2. An evaluation before training. A set of test cases and criteria that shows whether the fine-tuned model beats the base model. Without it you can't tell whether fine-tuning helped.
  3. Compute. Rented GPUs or your own server are usually enough for training. For daily use the model has to be hosted somewhere, in the cloud or on-premise. The trade-offs are covered in our comparison of on-premise vs. cloud AI.
  4. Someone to maintain it. Quality monitoring, retraining, base model updates.

What nobody tells you about fine-tuning

Fine-tuning solves specific problems, but it creates new responsibilities. Before you start, factor in these realities:

  • Models go stale. New products and changes in procedures and regulations require retraining. This is not a one-off project.
  • Data preparation is most of the work. Data must be clean, consistent, and representative, or the model will learn the errors too.
  • Infrastructure is an ongoing cost. Training is a one-time expense, but a production model runs every day. Calculate the cost per query and compare it with the API.
  • Personal data in the training set. If examples contain personal data, GDPR applies, and removing a specific piece of information from a trained model is hard. Anonymize data before training.
  • The EU AI Act. European Commission guidelines explain that general-purpose AI model provider obligations apply only to those who significantly modify a model, not to those making minor changes (European Commission). Typical fine-tuning on company examples usually falls in the second group, but for large projects it is worth checking with a lawyer.

5 signs it's time to fine-tune

  1. Off-the-shelf models fail in the same place. Despite good instructions and in-prompt examples, the model keeps getting the format, terminology, or classification wrong.
  2. You have valuable data you can't send outside. Your edge comes from knowledge and examples that should not go to external APIs.
  3. Privacy or regulation matters. Finance, healthcare, law: a self-hosted model simplifies the data conversation.
  4. Volume is high and stable. A smaller fine-tuned model running on your infrastructure can cost less per query than a large model through an API. Check it against your own traffic numbers.
  5. You have the resources to maintain it. A budget for infrastructure and an in-house team or a trusted partner.

Don't fine-tune just because open source is trendy. Do it when the business clearly benefits from control, customization, and predictable cost.

How do we approach this at Mandala?

We start with the simplest solution that works and reach for fine-tuning only when evaluation shows we need it. We run language models, embeddings, a reranker, and speech recognition and synthesis on our own GPU infrastructure, so client documents and recordings don't go to external APIs.

Our agentic knowledge base is a good example. Where answers must come from documents, we don't fine-tune the model on their content. We build retrieval instead: a document pipeline with OCR, hybrid search that handles Polish inflection, reranking, and answers with citations, all on local models. Knowledge changes with every new document, and the index updates without retraining.

How to start: a step-by-step plan

  1. Name a specific problem that off-the-shelf models don't handle well, as concretely as the first pilot in AI transformation, and check whether better instructions or RAG would be enough.
  2. Build an evaluation: a set of test cases and scoring criteria.
  3. Prepare a small, carefully reviewed set of examples, without unnecessary personal data.
  4. Choose a model with a suitable license and compare several candidates on your data.
  5. Run a pilot with LoRA or QLoRA and compare the result with the base model.
  6. Scale only when the benefit is measured, and plan for retraining.

Open source in AI isn't about being free. It's about control. For the right business problems, that control is worth the investment.

Sources

Frequently asked questions

How much does fine-tuning a custom AI model cost?

The cost has three parts: data preparation (usually the largest, because it is human work), compute for training, and infrastructure to run the model day to day. Thanks to LoRA and QLoRA, training itself is relatively cheap today, but the model then has to be hosted, monitored, and retrained from time to time. An honest estimate is only possible after assessing your data and query volume.

Is fine-tuning the same as RAG?

No. Fine-tuning changes the model itself, teaching it format, style, and vocabulary. RAG leaves the model unchanged and, before answering, retrieves passages from your documents and gives them to the model as context. For facts that change, RAG is better; for a fixed format and specialized language, fine-tuning is better; in practice the two are often combined.

How much data do I need for fine-tuning?

Less than people usually assume, but it has to be good. OpenAI's fine-tuning documentation lists a minimum of 10 examples and reports improvements from 50 to 100 well-crafted examples, noting that it depends on the use case. Start with a small, carefully reviewed set and an evaluation before collecting thousands of examples.

Which open-source model should I fine-tune?

Check the license first, then quality on your task and language. Gemma 4, Mistral Large 3 and Small 4, and the smaller Qwen 3.8 models (such as 27B) are licensed under Apache 2.0, but the flagship Qwen 3.8 and Mistral Medium 3.5 use their vendors' own licenses, so check the specific variant. Bielik is developed in Poland and optimized for Polish. Llama 4 uses Meta's license with restrictions, including no rights to the multimodal models for companies based in the EU.

Do I need an in-house ML team for fine-tuning?

Not from day one, but someone has to prepare the data, run the training, build the evaluation, and maintain the model in production. Many companies start with a technical partner and build in-house skills once the model becomes part of a core process.