Awesome Testing / maintained reference
57 concepts · 122 sources · reviewed 2026-09-03
AI Systems Wiki
Plain-English, source-linked notes and small interactive models for understanding AI, agents, testing, and the systems built around them.
model→harness→agent→evaluation
/
Teaching paths
Five useful sequences
From text to the next token
Use this sequence to explain what happens inside an LLM without starting from matrix notation.
From a prompt to useful context
Use this sequence to move from writing instructions to designing the complete evidence available for one call.
From model proposal to controlled action
Use this sequence to separate generated data and tool proposals from parsing, permission, and hostile instructions.
From one turn to a durable run
Use this sequence to explain repository context, external memory, lifecycle hooks, isolation, and delegated work.
From a demo to release evidence
Use this sequence to assemble cases, trials, graders, agent traces, and production feedback into one evaluation loop.
Index
Concepts
Foundations
- What is an AI system?The model is one probabilistic component inside a larger engineered and governed system.
- How do AI, machine learning, deep learning, and LLMs fit together?AI is the broad field; machine learning, deep learning, and large language models are progressively narrower families inside it.
- What is a perceptron?The perceptron is a learned linear threshold: useful for seeing both the basic unit of a network and the limit of one straight boundary.
- How does a neural network work?A neural network learns a layered function: linear mixing plus nonlinearity produces reusable intermediate representations.
- What happens to data before model training?Training data is a governed transformation pipeline, not a folder of raw text fed directly into a model.
- How are models trained?From examples and loss to gradients, parameter updates, and evidence of generalization.
- What is the difference between training and inference?The same network can participate in two very different processes: parameter learning and prediction.
- What are generalization and overfitting?Training loss shows fit, not transferable capability; generalization needs protected data, relevant shifts, slices, and complete-system evaluation.
- What happens after pretraining?Fine-tuning changes model weights; supervised and preference stages shape behaviour without replacing system-level controls.
- What should a model or system card contain?Model and dataset cards describe components; a system card connects them to users, effects, evaluations, controls, and the operating environment.
Models
- What are tokens and tokenization?Tokenization is a versioned encoding contract between human-readable text and a model vocabulary.
- What are embeddings?Embeddings turn tokens, sentences, images, or other items into coordinates learned for a particular task.
- How does a transformer work?Attention moves information between token positions; repeated residual updates build the representation used for prediction.
- How does a large language model work, end to end?Two loops connect the whole system: training changes weights; generation reuses fixed weights to extend a token sequence.
- How does a language model choose the next token?Generation is a repeated engineering policy over model scores, not another training step.
- What is a context window?Context is a temporary, application-assembled input—not the model's training data and not durable memory.
- How should engineers prompt a model?Treat prompts as testable task contracts: state the outcome, context, constraints, evidence, and definition of done.
- Which language should you use in a prompt?Prompt language is a versioned task variable to evaluate, not a universal English-versus-local-language rule.
- What is context engineering?Curate the smallest trustworthy working set that lets the model perform the current step.
Agents and harnesses
- What is an agent harness?The deterministic and operational boundary around probabilistic model proposals.
- What is an AI agent?Agency appears when model generation is placed inside a stateful observe–act loop with tools, policy, and stopping rules.
- How does a coding agent work?A coding agent is a general agent loop whose context, tools, environment, and completion rules are designed for software work.
- What is an agent skill?Skills package procedural knowledge for on-demand use while the harness still owns discovery, context, tools, permissions, and execution.
- How does function calling work?A model proposes a typed call; the application owns execution and returns the observation.
- What are structured outputs?A schema can make generated data parseable; it cannot make the data true or grant an action permission.
- How should agent tool calls be controlled?Schemas describe a proposed action; harness policy decides whether that action may affect the world.
- What is prompt injection?Natural-language data can look like instructions; security depends on limiting authority and effects outside the model.
- What is an agent sandbox?Isolation limits the blast radius of execution; it does not replace permission checks, careful tool design, or a threat model.
- How does agent memory work?Agent memory is an application-managed read and write policy over durable data, not recollection hidden inside the model.
- What are agent hooks?Hooks attach ordinary code to explicit agent lifecycle events; their power and risk come from where and how that code runs.
- What is a subagent?Delegation can create focused context or safe parallelism, but it multiplies coordination, cost, and failure boundaries.
- What is the Model Context Protocol?A versioned interoperability boundary for tools, resources, and prompts—not an agent, permission system, or guarantee of safety.
- How do browser agents use accessibility trees?Accessibility snapshots give browser agents a compact semantic view of controls, names, states, and relationships.
- What is AI-assisted engineering?AI-assisted engineering combines model proposals with repository context, small changes, executable feedback, and human ownership.
Testing and evaluation
- How do you test a model?From model claims and datasets to repeatable evidence, uncertainty, and known limits.
- How does an AI evaluation loop work?Cases, trials, graders, diagnosis, and regression form a loop that supports an engineering decision.
- Which metrics make an AI evaluation useful?A metric is a measurement lens for one claim; no single score captures quality, reliability, safety, and cost.
- How do you make an AI-assisted session reproducible?Reproduce the system conditions and observable effects; do not expect a stochastic model to repeat the same transcript.
- How do you know whether AI assistance is worth it?Measure quality-adjusted cost per accepted task, not generated tokens, code volume, or perceived speed.
- How do you build an evaluation dataset?A credible evaluation dataset is versioned evidence with explicit coverage, labels, provenance, separation, and exposure history.
- How do AI graders work?A grader is a versioned measurement instrument; its score is evidence with error modes, not ground truth.
- How do you test an agent harness?Separate runtime correctness from model quality and challenge every path from proposal to effect.
- How do you test an AI agent?Agent evaluation treats the model, harness, tools, environment, and graders as one versioned system.
- Where do AI agents fail?An agent failure is a broken transition in a layered stateful system, not simply a bad final sentence.
- How should AI-generated tests and test data be used?Generated tests are hypotheses to execute and challenge with boundaries, coverage, realistic faults, and independent oracles.
- How should a tester pair with AI?AI proposes and transforms; the tester owns risk, evidence, oracles, permissions, and acceptance.
- What is agentic testing?An agentic tester adapts an investigation through tools; its value depends on evidence, coverage, state control, and safe effects.
- How should an AI system handle uncertainty?Token probability, fluent language, self-reported confidence, and real-world correctness are different signals; none is a universal truth meter.
- What does responsible AI use mean for a tester?Responsible AI turns affected people, rights, data, impacts, oversight, and accountability into observable system obligations.
- How do you red-team an AI system?AI red teaming is threat-driven system testing with explicit adversaries, assets, impacts, safeguards, and stop conditions—not a jailbreak contest.
Production systems
- How should an application call an LLM API?Reliable model calls need distributed-systems discipline plus explicit handling for variable generation, token budgets, structured output, and tool proposals.
- How does vector search work?Vector search is a measured candidate-retrieval pipeline, not a database that understands meaning or permission.
- What is retrieval-augmented generation?RAG is a versioned evidence pipeline joining ingestion, retrieval, context assembly, generation, and citation.
- Should you use prompting, RAG, tools, or fine-tuning?Prompting, retrieval, tools, code, and fine-tuning solve different failure classes and often work together.
- How does prompt caching work?Prompt caching reuses recent prefix computation to reduce input cost or latency; it does not remember facts or reuse answers.
- What is a KV cache?KV caching trades growing inference memory for much less repeated computation during token-by-token generation.
- How do you monitor an AI system in production?Production monitoring connects real requests to reproducible traces and delayed outcomes, then turns confirmed failures into privacy-safe evaluation cases.