Awesome Testing

Models · reviewed · reviewed Sep 3, 2026 · 4 min

Which language should you use in a prompt?

Choose instruction, evidence, terminology, and output languages from the task and audience, then evaluate the exact combination. English is not universally best: model, tokenizer, domain, language resources, and output requirements can all change quality, cost, and risk.

Prompt language is a versioned task variable to evaluate, not a universal English-versus-local-language rule.

One request can contain four language decisions

“Should I prompt in English?” compresses several choices into one. An engineering request can use different languages for:

Part of the requestUseful default
InstructionsThe language in which the task and constraints can be stated most precisely
EvidencePreserve the source language unless an authorized translation is part of the pipeline
Domain termsUse the vocabulary recognized by the people, code, API, or regulation being discussed
OutputThe language required by the reader or downstream system

A Polish user can therefore give Polish instructions, quote an English API specification, preserve JavaScript identifiers, and request a Polish explanation. Forcing every part through English can erase distinctions; forcing every technical term into Polish can make the result less aligned with the repository and its documentation.

State the language contract explicitly when it matters: “Read the English source verbatim, answer in Polish, keep identifiers unchanged, and flag terms whose translation is ambiguous.” This is more useful than assuming that the model will infer which text may be translated.

Why results differ across languages

Models are trained on language mixtures that are neither equal nor fully disclosed. A language, dialect, domain, or script with less or poorer training data may receive weaker coverage. Post-training can also reward instruction following differently across languages.

Tokenization adds another difference. The same meaning can require different numbers of tokens across languages, affecting the available context, latency, and sometimes price. Fragmentation does not directly measure model competence, but it changes the numerical sequence the model processes.

Prompt wording matters within one language, too. Translation changes word choice, politeness, ambiguity, examples, legal meaning, and cultural assumptions. It is therefore a new prompt variant, not a lossless transport layer.

Research has found sizeable language differences and improvements from particular cross-lingual prompt templates on selected benchmarks. Those studies are evidence that language can matter—not proof that translating reasoning into English is always superior. The measured models, tasks, languages, and prompt patterns are bounded, and current models can behave differently.

Choose the natural task language first

Start with the language used by the person who owns the requirement and the artifacts that define correctness. This usually reduces translation effort and makes review easier. Keep code, selectors, schemas, product copy, error messages, and quoted requirements in their authoritative form.

Try an English or bilingual variant when there is a concrete reason:

  • authoritative documentation and technical vocabulary are primarily English;
  • the model repeatedly misunderstands a lower-resource language;
  • a shared prompt must serve reviewers in several languages;
  • the output must match an English repository or API contract;
  • a controlled comparison shows a material improvement.

Do not translate private evidence through another service merely to improve a prompt. Data handling, retention, residency, and authorization still apply.

A bilingual prompt needs boundaries

Mark which passages are instructions and which are evidence. Identify the requested output language and the terms that must remain unchanged. If translation is required, preserve both the original and translated version with provenance rather than silently replacing the evidence.

For example:

Goal: Find missing acceptance cases.
Instructions and answer: Polish.
Authoritative evidence: the English user story below.
Keep API fields and quoted UI labels unchanged.
If a requirement is ambiguous in translation, show both readings.

This structure also limits prompt injection and review mistakes: a sentence in a quoted document does not become a higher-priority instruction because it appears in English.

Evaluate language as a product slice

Freeze the model version, prompt structure, evidence, tools, and decoding settings. Run representative cases for each supported language and for realistic mixed-language requests. Compare task success, unsupported claims, preserved identifiers, format validity, tone, refusal behaviour, latency, token use, and human correction effort.

Include accented text, inflection, mixed scripts, locale-specific dates and numbers, untranslated identifiers, code-switching, and ambiguous domain terms. Use qualified native speakers for judgement that depends on meaning or tone. Machine translation or a second model can help screen outputs, but it is not an independent oracle for the same linguistic failure.

The decision may be different by task. English may work better for one coding workflow while Polish works better for explaining a Polish requirement. Record that result as a scoped finding tied to a model and prompt version—not as a timeless property of LLMs.

Sources and further reading

  1. 01
    Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought PromptingHuang et al. · research · published Dec 6, 2023 · source checked Sep 3, 2026

    A multilingual benchmark study demonstrating that prompt language and structure can change results, with model-, task-, and language-specific limitations.

  2. 02
    SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text ProcessingKudo and Richardson · research · published Aug 19, 2018 · source checked Aug 30, 2026

    Primary description of a language-independent tokenizer trained directly from raw text with a fixed subword vocabulary.

  3. 03
    Holistic Evaluation of Language ModelsLiang et al. · research · published Nov 16, 2022 · source checked Aug 30, 2026

    A primary framework connecting scenarios, adaptations, metrics, and transparent raw results.

  4. 04
    Prompting best practicesAnthropic · documentation · source checked Aug 31, 2026

    Current vendor guidance illustrating clear instructions, examples, prompt structure, output control, tool use, and model-specific validation rather than universal prompt incantations.