AI and digital tools

Understand practical uses of artificial intelligence, choose appropriate tools and manage risks involving data, reliability and automation.

111 articles · Latest article updated

Search covers every article in this topic, including the archive and article text.

Understand the topic

A practical guide to this topic

AI should be judged in a real task: what information enters the system, which decision it helps make and which errors would be costly. This category connects models, assistants and tools to their data, operating conditions and human oversight. For an AI agent, text quality and action validity are separate checks. Useful evaluation follows the retrieved document, resource permissions, tool call and resulting effect. It also records correct refusals and human recovery, which disappear from averages limited to successful responses.

What to examine
  • Evaluate a use case with representative examples rather than a single demonstration.
  • Trace an assistant’s sources, answers, actions and human interventions.
  • Compare providers through data handling, operation, change management and exit options.
  • Compare versions on the same tasks, find local losses and make a decision supported by cases rather than an average.
  • Test retrieval, citations, caches and revocation so a document assistant respects each user’s permissions.
  • Bind human approval to a specific operation, limit its lifetime and account for effects already executed.
  • Separate parsing, schema checks, documentary evidence and business rules when turning documents into usable records.
  • Evaluate generated records before sharing them or using them to train a model.
  • Check omissions, numbers and reading order before transcription feeds search or decisions.
  • Bind claims to a document version and passage, then handle stale citations without rewriting history.
Useful questions
  • What should the system refuse or escalate?
  • How will a plausible but unsupported answer be detected?
  • Who can pause or change the use?
  • Which component changed?
  • Is a separate vector namespace enough?
  • Does Stop undo every action?
  • Can valid JSON contain an error?
  • Does synthetic mean anonymous?
  • Is a high score enough?
  • Does a citation found in the new version confirm the answer?

Illustrative decision example

Situation
Fictional example: these inputs describe no customer or observed result. A classifier improves its overall score but becomes worse at identifying requests needing urgent handling.
Check
Replay identical cases, retain results by request type and review critical errors using a common definition.
Act
Decide using severity, volume and recovery capability. The best average score is not automatically the best service choice.

Deep Learning: Transforming the Capabilities of AI Models

Evaluate AI regressions before changing versions

NIST — AI Risk Management Framework 1.0, Measure

Test source authority

A relevant corpus can contain adversarial instructions. Separate evidence for an answer from authority to act; check denial in the tool rather than only in the agent’s text.

Can a retrieved source trigger sending or changes outside the task?

Explore the methods

Read data · verify · decide

AI bias: examine who bears the errors

A global rate can hide weaker performance for a subgroup. Make counts, errors and consequences visible before calling a system fair.

Group A error rate
5%5 / 100 documents
Group B error rate
20%10 / 50 documents
Observed gap
15 points20% − 5%

Fictional teaching case — figures describe neither CIE nor a client.

Questions to support your decisions

Find answers and practical checks in the analyses in this category.

Explore the archive

Further analysis in this topic beyond the recent articles listed below.

References: data, uses and limitations

Documentation reviewed on .

References: data, uses and limitations
Source and purposeVerified data or benchmarkHow to use itLimitation to retain
MITRE ATLAS

Examine documented tactics, techniques and cases involving AI systems.

The official repository publishes ATLAS tactics, techniques and case-study data. They support analysis of the actual deployed system. Select scenarios matching your application’s data, interfaces and tools. Test permissions and consequences, not only model responses.

Explore the method →

A technique catalogue is neither an exhaustive coverage guarantee nor a test result for your system.
NIST · AI Risk Management Framework

Organise management of AI-related risks.

The AI RMF Core is organised around Govern, Map, Measure and Manage. It is a voluntary risk-management framework. Define intended use, affected people and error consequences. Connect each risk to an evaluation, owner and stop or recovery mechanism.

Explore the method →

The framework certifies neither a model nor an application. Using it does not remove the need to examine applicable requirements.
OWASP · LLM Top 10

Prepare tests for language-model application risks.

The 2025 edition includes Prompt Injection, Sensitive Information Disclosure and Excessive Agency. Turn a risk into an observable scenario: hostile document, restricted data or out-of-scope tool. Record expected behaviour and evidence of refusal.

Explore the method →

A risk list is not a complete assessment. Input filtering alone does not establish authorisation for every action.

RAG: access rights, citations and caches

Compare AI architectures through their access boundaries

Beyond model choice, examine who retrieves documents, under which identity and with which caching policy. These decisions define separation tests between teams.

  • Identify each component seeing a passage or citation.
  • Replay the same question under several groups.
  • Check that a reading tool cannot become a sending tool.

Explore the RAG access matrix and replay cases →

References consulted on · OWASP · LLM08:2025 Vector and Embedding Weaknesses OWASP · API1:2023 Broken Object Level Authorization

RAG relevance and ranking

Compare retrieval before evaluating the answer

Keep the same needs, judgments and depth k when comparing configurations. Inspect regressing queries before drawing conclusions from an average.

  • Separate window precision, relevant-record coverage and ordering quality.
  • Document missing judgments and the no-relevant-record convention.
  • Then evaluate the answer, citations and refusals using a separate protocol.

Explore ranking measures and relevance judgments →

Stanford CS276 · Retrieval evaluation · 2019 course archive · PDF pp. 3, 5–6 · consulted 2026-10-23

NIST TREC · Reciprocal rank definition · Repository snapshot 35cf096ba6d5ae08ca4803c056b31ed61a56a6d4 · consulted 2026-10-23

Reference material

Primary guidance published by the organizations responsible for these methods and standards.