Can Small, Language-Specialised AI Models Replace Large LLMs for Coding? A Research Review

Abstract

General-purpose code models handle many languages and tasks, but they are expensive to serve. This article asks whether compact models that specialise in one programming language and its ecosystem, placed inside an agent framework that supplies tools, memory and escalation, could deliver competitive coding performance at lower cost. We review primary studies on language-specific fine-tuning, language-expert architectures, model scaling, mixture-of-experts (MoE), routing and agentic software engineering benchmarks. The evidence shows that specialisation improves in-language scores on function-level benchmarks by roughly 5 to 7 points at equal size [1], that sharing most parameters while specialising a thin adapter layer outperformed fully separate per-language models in one controlled study [3], and that routing between cheap and strong models can cut cost substantially on general benchmarks [12, 13]. However, we found no published study that measures language-specialised small models on full repository-level agent tasks against general models at equal cost. We set out four architectural options, a novelty matrix, five falsifiable hypotheses and a reproducible evaluation protocol centred on cost per successfully completed task. This is a literature-based investigation. No new experiments were run.

Keywords: small language models, code generation, language specialisation, mixture-of-experts, adapters, LLM routing, coding agents, SWE-bench, inference cost

Status of this work: a review of published papers and official model documentation, read and checked on 16 October 2026. All numbers in charts and text come from the cited sources as reported by their authors. Diagrams marked "proposed" or "conceptual" describe ideas, not implemented or measured systems.

1. Introduction

Modern coding assistants are mostly built on large general models. They are trained on dozens of programming languages, natural-language text and reasoning data, so one model can explain a SQL query, refactor a C# service and write a Python test in the same session. That breadth comes at a price. Large models need more memory, respond more slowly per token, and cost more per call, whether you pay an API provider or run the hardware yourself.

Most developers, though, do not work in dozens of languages at once. A Python developer spends most of the day inside Python, its libraries, its packaging and its test tools. A .NET developer lives inside C#, ASP.NET, Entity Framework and NuGet. That raises a simple question: could a compact model that knows one ecosystem deeply do most of that developer's routine work, more cheaply, while a larger model is called only when needed?

The idea rests on a separation. Language knowledge (syntax, idioms, libraries, error messages) might live in a small specialist. Agent capability (reading a repository, running tests, searching documentation, planning, recovering from errors) might be supplied by a shared framework around the model. This article examines what published evidence says about each half of that idea, where the evidence stops, and how the remaining questions could be tested.

2. Key terms

• Language specialisation: further training a model mainly on one programming language and its ecosystem, beyond the broad code training it already had.
• Adapter (LoRA): a small set of extra trainable weights added to a frozen base model, so one base can carry several cheap specialisations [14].
• Mixture-of-experts (MoE): a model with many sub-networks where a router activates only a few for each token.
• Total vs active parameters: total parameters set the memory needed to host a model. Active parameters set the compute spent on each token. For dense models they are the same. For MoE models they differ greatly.
• Agentic coding: an AI system that works over many steps with tools, such as opening files, running tests and reading errors, rather than answering one prompt.
• Routing or cascading: choosing, per request, which model handles it, usually a cheap one first and a strong one when needed.
• Cost per successful task: all inference and tool costs for a set of tasks, retries and escalations included, divided by the number of tasks actually completed.

3. Research questions

• RQ1. Does specialising a code model in one programming language improve its performance in that language, and at what cost to other languages?
• RQ2. Can smaller models match larger ones on well-defined coding tasks?
• RQ3. Which architecture best delivers specialisation: one general model, separate specialists, a shared backbone with adapters, or a routed hybrid?
• RQ4. Does evidence exist that specialists can sustain long, multi-step agentic workflows on real repositories?
• RQ5. Does specialisation reduce the total cost of completing software tasks, not just the cost of each model call?

4. How we reviewed the evidence

We searched arXiv, the ACL Anthology, conference proceedings and official model documentation for work on language-specific fine-tuning, language-expert and MoE architectures for code, model scaling in code families, routing and cascades, long-context use, and repository-level agent benchmarks. For each included study we read the primary paper or official page, recorded the exact numbers from its tables, and noted its stated limitations. We excluded secondary summaries as sources of numbers.

This is a focused, structured review, not a formal systematic review. The search was not exhaustive, so the absence of a study in our results does not prove that no such study exists. Results from different papers use different models, data and settings, so we compare results within a study, not across studies.

5. The proposal

Figure 1 shows the system this article evaluates. A router reads the task and sends it to a language specialist. Every specialist works through the same shared agent harness, which provides repository access, a terminal, tests, documentation search and project memory. If the tests pass, the work is done. If they keep failing, the task escalates to a general model, carrying the history of what was tried.

Figure 1. Proposed language-specialised agent system. Conceptual design, not an implemented or measured system.

5.1 Why it might be beneficial

Each benefit below is a hypothesis to test, not a demonstrated result.

• Lower cost per call. A small model needs less compute per token, so routine requests cost less.
• Lower memory and faster responses. Smaller models fit on cheaper hardware, including a single consumer GPU or a laptop, and generate tokens faster.
• Local and private use. A compact open-weight specialist can run on a developer's own machine, keeping code off third-party servers.
• Cheaper updates. With a shared backbone and adapters, improving C# support means retraining a small adapter, not a whole model.
• Better use of a large model's budget. If specialists absorb routine work, the expensive model is used only where it adds the most.

6. Evidence

6.1 Specialising on one language improves in-language scores

The cleanest controlled test is Code Llama [1]. The authors took their general code model and trained it on an additional 100 billion tokens of Python-heavy data, producing Code Llama - Python at the same three sizes. Architecture and size are held constant, so the difference isolates the effect of specialisation.

Figure 2. HumanEval pass@1 for Code Llama and Code Llama - Python. Source: [1], Table 2. HumanEval contains 164 Python programming problems [15]. Scores as reported by the authors, not reproduced.

The Python specialist scored higher at every size, by 4.9, 7.3 and 4.9 points. It also scored higher on MBPP, a second Python benchmark, at every size. But specialisation did not overturn scale: the specialised 13B model (43.3%) stayed below the general 34B model (48.8%).

6.2 The effect on other languages is small

Figure 3. MultiPL-E average pass@1 across non-Python languages. MultiPL-E translates HumanEval and MBPP into 18 additional languages [5]. Source: [1], Table 4.

On other languages, the Python models were about level with the general ones: 1.2 and 0.9 points higher at 7B and 13B, and 1.3 points lower at 34B. The authors describe the results as comparable. So in this case, specialisation added in-language skill without clearly damaging other languages, but it did not transfer gains to them either.

6.3 On short tasks, returns to scale flatten

Qwen2.5-Coder [2] was released at six sizes from 0.5B to 32B, all trained on the same data of over 5.5 trillion tokens covering 92 programming languages. That makes it a clean view of how size alone affects a function-level benchmark.

Figure 4. HumanEval scores of Qwen2.5-Coder base models. Source: [2], Table 5.

Moving from 0.5B to 7B adds 33.6 points. Moving from 7B to 32B, more than four times the parameters, adds 4.3 points. On short, self-contained problems, a mid-sized model captures most of the available score. This supports RQ2 for narrow tasks. It says little about repository-level work, where tasks are longer and depend on reasoning across files.

6.4 Sharing most of the model beat separate specialists

Zong, Deng and Nie [3] tested how best to specialise a 1.3B DeepSeek Coder base across eight languages (C, C++, C#, Go, Java, JavaScript, Python and Rust). They compared one adapter shared by all languages, a separate adapter per language, full fine-tuning, and their Mix-of-Language-Experts (MoLE) design: a shared adapter used by every language plus a small expert adapter per language, selected by the language of the code.

Figure 5. Average pass@1 by training setup on summarisation, synthesis and translation built on HumanEvalPack. Per-language models do not apply to translation. Source: [3], Table II.

MoLE beat both the shared model and the separate per-language models on every task where they could be compared. The margins are small, between 0.4 and 1.3 points, and per-language results were mixed. Full fine-tuning was still best on synthesis. The important result for this article is directional: fully separate specialists did not win. Programming languages share structure, and a design that keeps that shared knowledge while specialising a thin layer did better than isolating each language. The authors note that only one small model was tested and that some language adapters were under-trained.

6.5 Specialisation helps most where data is scarce

MultiPL-MoE [4] added a hybrid of token-level and segment-level experts to an existing model to extend it to more programming languages. On HumanEval translated into Rust, Go and Ruby, it reported about 2 points higher pass@1 than the next-best method, while staying within about 2 points of the strongest baselines on Python, Java and C++. Together with Section 6.2, this suggests specialisation adds the most where the base model is weakest, in lower-resource languages. Python, which every code model sees in vast quantities, has less headroom.

6.6 Agentic software engineering is a different test

Everything above measures isolated code generation. Agentic work means navigating a repository, running tests, reading failures and iterating. SWE-bench Verified [7] is the most widely used test for this: 500 real GitHub issues from 12 open-source Python repositories, each screened by human developers, where the model must produce a patch that passes the project's tests.

The SWE-smith project [6] generated 50,000 training task instances from 128 repositories and trained SWE-agent-LM-32B, which resolved 40.2% of SWE-bench Verified at pass@1. This shows that a mid-sized open model trained on software engineering tasks can be competitive on repository work. Its specialisation is in the job, not in a programming language. We found no study that evaluates a language-specialised model against a general model on this kind of benchmark at matched cost. That is the central gap this article identifies.

6.7 A long context window is not enough

Liu et al. [8] showed that language models use information best when it appears at the start or end of a long input, and noticeably worse when it sits in the middle, including in models designed for long inputs. For agents, this means the choice of what enters the context matters as much as window size. Retrieval and context management belong to the system design, whichever model is used. It also means a small specialist with good context management is not automatically disadvantaged against a large model with a bigger window.

6.8 Do not compare total parameters with active ones

Figure 6. Total versus per-token active parameters. Sources: Mixtral [10], Qwen3-Coder model card [11], DeepSeek-V3 [9].

Mixtral 8x7B has 47B parameters but uses 13B per token, routing each token to 2 of 8 experts [10]. Qwen3-Coder-480B-A35B uses 35B of 480B, with 8 of 160 experts active [11]. DeepSeek-V3 uses 37B of 671B [9]. Compute per token follows the active count, while hosting memory follows the total. A "huge" MoE model can therefore be cheaper per token than its headline size suggests, which makes comparisons against small dense specialists closer than they first appear. Any cost comparison must state which quantity it uses.

6.9 Routing and cascades already show large savings

RouteLLM [12] trained routers that decide per query whether a weaker model suffices, and reported over 2 times cost reduction on public benchmarks without sacrificing response quality. FrugalGPT [13] cascaded several models and reported matching GPT-4 performance with up to 98% cost reduction, or 4% higher accuracy at the same cost. Neither study focuses on agentic coding, and the savings depend heavily on the task mix. But both show that sending easy work to cheap models and hard work to strong ones is an effective, demonstrated strategy, which is the core of Architecture D below.

6.10 Evidence summary

7. Four architectural options

Figure 7. Four ways to deliver coding ability. Conceptual comparison.

Rows on memory, cost and updates follow from model size and design, so they are expected properties rather than measured results. The quality row reflects the cited studies only.

8. What is new and what is not

Much of this idea already exists. Language-specific fine-tuning [1], language adapters [3], language-aware MoE [4] and routing [12, 13] are all published. The novelty matrix below separates what is established from what remains open. A gap is not proven just because our searches did not find a paper, so the confidence column is deliberately cautious.

The most defensible contribution is therefore narrow: measuring cost per successfully completed repository task for language specialists inside a shared agent harness, with escalation, against general models under the same budget.

9. Testable hypotheses

• H1. In-domain gain. A language-specialised model resolves more tasks in its own ecosystem than its base model at the same size. Supported if the paired difference in pass rate is positive with a confidence interval above zero. Contradicted if it is zero or negative on repository tasks, even if function-level scores improve.
• H2. Cost per success. A specialist inside an agent harness has lower cost per successful task than a general model, at a pass rate within an agreed margin. Contradicted if retries and escalations erase the per-call saving.
• H3. Hybrid beats either alone. A routed specialist-plus-general system sits on a better quality-cost frontier than either model used for every task. Contradicted if the router's mistakes cost more than it saves.
• H4. Cross-language penalty. Specialists lose more on mixed-language tasks (for example C# plus TypeScript) than on single-language ones. Supported if the gap between single- and mixed-language pass rates is larger for specialists than for general models.
• H5. Shared beats separate at scale. The MoLE result [3] holds at 7B or larger and inside an agent loop. Contradicted if separate specialists match or beat a shared backbone with adapters at equal parameter budget.

10. A proposed evaluation protocol

Figure 8. Proposed evaluation pipeline. Not yet run.

Tasks. Start with Python and C#/.NET. Use SWE-bench Verified [7] for Python repository issues, and assemble a comparable, human-checked set of C# issues from open-source .NET repositories, since no equivalent standard exists that we verified. Add function-level checks from HumanEval and MultiPL-E [5] only as a secondary signal.

Configurations. Test all four architectures from Section 7 with the same harness, tools, time limit, token budget and number of attempts. Fix the repository snapshot, system prompt, sampling settings, quantisation and hardware for every run, and record them.

Metrics. Report these together, never one alone:

pass rate              = tasks solved / tasks attempted
cost per success       = total inference and tool cost / tasks solved
escalation rate        = tasks escalated / tasks attempted
median time to finish  = median wall-clock time over solved tasks

Cost per success must always appear next to pass rate. A model that fails most tasks should not look efficient because its few successes were cheap. API costs and self-hosted costs should be reported separately, with hardware, utilisation and electricity assumptions stated.

Statistics. Because every configuration attempts the same tasks, use paired comparisons: McNemar's test for pass or fail outcomes and bootstrap confidence intervals for differences in pass rate and cost. Run each configuration several times to capture variation from sampling. Report sample sizes, failed runs and effect sizes, and correct for multiple comparisons across the five hypotheses.

11. A decision framework for escalation

Until such experiments exist, teams can still apply the routing logic that the evidence supports. Figure 9 shows one practical policy.

Figure 9. Proposed routing and escalation decision flow.

Two design choices matter most. First, the retry budget k: too low and the system escalates tasks a specialist could have solved, too high and retries burn the savings. Second, escalation should pass the full history, including diffs, logs and failing tests, so the stronger model starts from what was already learned rather than from scratch.

12. Why this matters for cost-sensitive developers

The question is global, but it is sharper wherever developers pay per use and watch every rupee or dollar. When billing is per token, the difference between cost per call and cost per finished task becomes visible quickly: a cheap model that needs three attempts can cost more than a stronger one that succeeds once. Open-weight models at 7B and below can run on modest local hardware, which matters for students, small teams and anyone who cannot send code to third-party servers.

We have not verified data on the hardware, budgets or preferences of Indian developers, so we make no claims about them here. Whether specialised small models genuinely widen access to coding assistance in India is itself an empirical question worth studying, alongside API pricing, GPU availability and local deployment barriers.

13. Limitations and threats to validity

• No new experiments. Every result is as reported by the original authors, and we reproduced none of them.
• Function-level benchmarks dominate. Most specialisation evidence uses HumanEval-style problems, which are short and widely used in training data, so contamination may inflate scores.
• Cross-study comparisons are unsafe. Different papers use different models, data and settings. We therefore compare only within each study.
• Small or narrow studies. The MoLE result rests on one 1.3B model. The Code Llama result rests on one language and one model family.
• Routing results are not from coding agents. RouteLLM and FrugalGPT savings come from general benchmarks and may not carry over.
• Search coverage. Our search was structured but not exhaustive. A relevant study may exist that we did not find.

14. Conclusion

The evidence supports a qualified version of the idea. Specialising a code model in one language reliably adds a few points on that language's function-level benchmarks, and costs little elsewhere. Mid-sized models capture most of the score on short tasks. Sharing most parameters while specialising a thin adapter layer beat fully separate specialists in the one controlled study we found. Routing between cheap and strong models already delivers large savings on general workloads.

What the evidence does not yet show is the claim that matters most in practice: that language-specialised small models, inside a shared agent harness with escalation, complete real repository tasks at lower total cost than general models. That is the defensible research contribution, and Section 10 sets out how to test it. Until then, the practical advice is simple: use small models for routine, single-language work, escalate deliberately, and measure cost per solved task rather than cost per call.

Try it yourself

You can run your own version of this comparison on Velona: call small and large coding models through one OpenAI-compatible API, pay per use from a rupee wallet with UPI, and compare which model actually finishes your tasks and at what cost. Start with our guides to building a model router in Python and the real cost per task of coding agents, or browse the models.

FAQ

Can a small AI model beat a large one at coding?

On narrow tasks, sometimes. Code Llama - Python 7B beat the general Code Llama 7B on HumanEval (38.4% against 33.5%) [1]. But the general 34B model still beat the specialised 13B one. For full repository tasks, there is no public evidence yet that small specialists match large models.

Is a Python-specific model better than a general coding model for Python?

In the best-documented case, yes, by about 5 to 7 points on HumanEval at the same size [1]. Whether that holds for newer models and for agent tasks has not been shown.

Do specialised coding models get worse at other languages?

Not much in the evidence we found. Code Llama - Python scored about the same as the general model on other languages, slightly lower at the largest size [1].

Should I train one model per programming language?

The available evidence suggests not. In the MoLE study, a shared base with small per-language adapters beat fully separate per-language models [3].

What is a mixture-of-experts model?

A model with many sub-networks where only a few run for each token. DeepSeek-V3 has 671B parameters but uses 37B per token [9]. That cuts compute per token, though the whole model must still fit in memory.

Is a bigger context window enough for long coding tasks?

No. Models use information in the middle of long inputs less reliably [8]. Choosing what goes into the context matters as much as the window size.

References

1. B. Rozière et al., Code Llama: Open Foundation Models for Code, arXiv:2308.12950, 2023.
2. B. Hui et al., Qwen2.5-Coder Technical Report, arXiv:2409.12186, 2024.
3. Y. Zong, Y. Deng and P. Nie, Mix-of-Language-Experts Architecture for Multilingual Programming, arXiv:2506.18923, 2025.
4. Q. Wang et al., MultiPL-MoE: Multi-Programming-Lingual Extension of Large Language Models through Hybrid Mixture-of-Experts, arXiv:2508.19268, 2025.
5. F. Cassano et al., MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generation, arXiv:2208.08227, 2022.
6. J. Yang et al., SWE-smith: Scaling Data for Software Engineering Agents, arXiv:2504.21798, 2025.
7. OpenAI, Introducing SWE-bench Verified, 2024.
8. N. F. Liu et al., Lost in the Middle: How Language Models Use Long Contexts, Transactions of the ACL, vol. 12, 2024.
9. DeepSeek-AI, DeepSeek-V3 Technical Report, arXiv:2412.19437, 2024.
10. A. Q. Jiang et al., Mixtral of Experts, arXiv:2401.04088, 2024.
11. Qwen Team, Qwen3-Coder-480B-A35B-Instruct model card, Hugging Face.
12. I. Ong et al., RouteLLM: Learning to Route LLMs with Preference Data, arXiv:2406.18665, 2024.
13. L. Chen, M. Zaharia and J. Zou, FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance, arXiv:2305.05176, 2023.
14. E. J. Hu et al., LoRA: Low-Rank Adaptation of Large Language Models, arXiv:2106.09685, 2021.
15. M. Chen et al., Evaluating Large Language Models Trained on Code, arXiv:2107.03374, 2021.