Interview Questions: LLMs: Training and Inference

40 questions on “LLMs: Training and Inference” — with short answers and a link to an in-depth explanation of each topic.

How to use it: cover the answer, answer out loud within a minute, then compare. All sections are on the interview questions page. Topics marked “in Russian” are not translated yet.

• Pretraining and Scaling Laws · 4
• Fine-tuning: SFT, LoRA, PEFT · 4
• Alignment: RLHF and DPO · 4
• Reasoning Models and Test-Time Compute · 4
• Sampling: Temperature, Top-k, Top-p · 4
• Inference: KV-Cache, Quantization, Speculative Decoding · 4
• Distillation and Model Compression · 4
• Evaluating LLMs: Benchmarks and LLM-as-Judge · 4
• Working with Context and Message Markup · 4
• Prompt Patterns and Structured Outputs · 4

Pretraining and Scaling Laws (in Russian)

• “Why is pretraining called self-supervised?” — labels are not created by humans: the next token is the label, so all the text in the world is usable.
• “What is the Chinchilla finding?” — at fixed compute, N and D should grow together (~20 tokens per parameter); GPT-3 was undertrained on data.
• “Why deduplication?” — repeats make the model memorize, distort the loss and contaminate benchmarks that leaked into the training set.
• “Why isn't a base model usable as an assistant?” — its goal is to plausibly continue text, not to fulfil a request; the “helpful answer” format is given to it only in post-training.

Fine-tuning: SFT, LoRA, PEFT (in Russian)

• “Prompting, RAG or fine-tuning?” — in order of cost: first the prompt and few-shot, RAG for facts, fine-tuning when you need a stable style/format/domain and have the data and budget for quality evaluation.
• “How does LoRA work and why doesn't it slow down inference?” — ΔW = A·B of rank r is trained separately and after training is added to W — the architecture and latency don't change.
• “How much memory does a full fine-tune of a 7B model need?” — ~16 bytes per parameter with Adam and mixed precision: around 112 GB, hence the popularity of PEFT.
• “What is catastrophic forgetting and how do you mitigate it?” — loss of general skills during narrow fine-tuning; a small learning rate, mixing in general data, and LoRA instead of full FT help.

Alignment: RLHF and DPO (in Russian)

• “Why the KL penalty in RLHF?” — the RM is imperfect; without a leash the policy finds its holes (reward hacking) and degenerates. KL keeps the model close to the reference.
• “How is DPO different from RLHF?” — the same objective (preferences + KL), but a direct supervised loss on pairs: no separate RM, no PPO, no online generation — simpler and more stable.
• “What is sycophancy and where does it come from?” — the model learns to agree with the user because labelers more often prefer agreeable answers; it is a bias in the preference data, not the model's “character”.
• “Why is RLVR used for math and code?” — the reward is verifiable (the answer matches, the tests pass), so it can't be gamed with a nice-looking but wrong text.

Reasoning Models and Test-Time Compute (in Russian)

• “How is a reasoning model different from a regular one with a CoT prompt?” — a CoT prompt merely asks an already trained model to write out steps; a reasoning model is trained via RL (RLVR) to develop strategies — decomposition, self-checking, backtracking — and its reasoning actually finds and fixes its own mistakes.
• “When do you use a reasoning model and when a regular one?” — by task difficulty versus cost and latency: math, code, planning — yes; extraction, classification, chat that requires an instant reply — no.
• “What is RLVR and why no labelers?” — RL with verifiable rewards: the task's answer is checked, code is run against tests; the reward is objective, and human comparisons of answers aren't needed.
• “How else can you spend test-time compute besides long reasoning?” — in parallel: self-consistency (voting over N chains) and best-of-N with a scorer.

Sampling: Temperature, Top-k, Top-p (in Russian)

• “What does temperature do mathematically?” — it divides the logits before softmax: T < 1 sharpens the distribution, T > 1 flattens it; in the limits you get argmax and uniform.
• “Why is top-p better than top-k?” — the cutoff size adapts to the model's confidence instead of being fixed in advance.
• “Why does greedy decoding degenerate?” — a locally probable token ≠ globally good text; the typical symptom is repetition loops, fixed with sampling and penalties.
• “What parameters would you set for SQL generation?” — low temperature (≈0–0.2), possibly a lower top-p; diversity is harmful here.

Inference: KV-Cache, Quantization, Speculative Decoding (in Russian)

• “Why is decode slower than prefill?” — prefill is parallel across tokens and compute-bound; decode is one token per step and is bound by memory traffic (weights + cache for every token).
• “What does the KV-cache store and how big is it?” — keys and values of all past tokens across all layers and heads; its size is linear in context length (≈0.5 MB/token for a 7B model in fp16).
• “Does speculative decoding hurt quality?” — no: the acceptance rule preserves the large model's distribution; the speedup comes from parallel verification.
• “How do you evaluate serving?” — TTFT, tokens/s per request and total throughput; continuous batching and PagedAttention increase throughput specifically.

Distillation and Model Compression (in Russian)

• “How is distillation different from quantization?” — quantization compresses the numbers of the same model (fp16 → int4), distillation transfers knowledge into a different, smaller architecture by training on the teacher's predictions; they are combined.
• “Why temperature in distillation?” — to soften the teacher's softmax: at T = 1 the distribution is almost one-hot, while at T = 3–5 the relations between classes become visible — that very dark knowledge.
• “How do you transfer reasoning into a small model?” — sequence-level distillation: the teacher generates reasoning chains, the student is fine-tuned on them with SFT (this is how the R1 distills were made).
• “Can the student outperform the teacher?” — not on the teacher's distribution, that is the ceiling; the win is in cost and latency, and on a narrow domain the gap is minimal.

Evaluating LLMs: Benchmarks and LLM-as-Judge (in Russian)

• “How can you tell a benchmark is contaminated?” — a suspiciously high score with weakness on fresh/paraphrased tasks of the same type; checking n-grams against the training corpus.
• “What biases does an LLM judge have and how do you catch them?” — position (rerun with A/B swapped), length (correlation of the verdict with length), self-preference (compare judges from different model families).
• “How do you evaluate an LLM feature in production?” — a golden set of real queries, rubrics, an LLM judge calibrated against humans, regression runs for every release.
• “Model A beat B in 6 out of 10 comparisons — is it better?” — no, the sample is tiny: the confidence interval covers 50/50.

Working with Context and Message Markup (in Russian)

• “Why does role markup matter more than plain text?” — roles set priority and boundaries: the model is trained to treat system as rules and user as a request; without roles this information is lost.
• “What is a chat template?” — the format that wraps messages in special tokens (for example, ChatML with <|im_start|>/<|im_end|>), marking turn boundaries the way the model saw them in training.
• “Where do you put the important stuff in a long context?” — at the beginning and at the end: because of the lost-in-the-middle effect, the middle is used worse.
• “How is context engineering broader than prompt engineering?” — prompting is about wording the instruction, context is about assembling and marking up the whole input: roles, block order, delimiters, token budget.

Prompt Patterns and Structured Outputs (in Russian)

• “How do you get valid JSON from an LLM?” — a ladder: ask in the prompt → few-shot with examples → the provider's JSON mode → schema-constrained decoding; only the last level gives a guarantee, because it masks invalid tokens during sampling.
• “Why is few-shot stronger than a verbal instruction?” — the model continues patterns: examples set the format in exactly the form it must be reproduced, with no “words → behavior” translation.
• “What's wrong with a giant prompt for many tasks?” — instructions compete and contradict each other, errors can't be localized, eval is uninformative; fixed by decomposing into a chain of prompts.
• “Does a schema guarantee correct data?” — no: it guarantees the shape (parsing, field types), while correctness of the values is checked only by evaluation on a reference set.