LLM application engineering

See the system

Choose models and prompts through task evidence, validate outputs, and design for uncertainty.

Understand the model boundary

A language model generates outputs conditioned on its input and learned parameters. It is not an authoritative database or an authorization engine. Context limits, sampling behavior, provider changes, and ambiguous instructions affect outputs.

Make the task small and explicit. Supply the necessary evidence and expected response shape. Decide which claims require external verification and which tasks can tolerate variation. A fluent response is not evidence that a fact is true.

Use structured output carefully

Structured generation can make outputs easier to consume, but syntax and domain validity are different. A valid JSON object can still contain a wrong customer ID or an unauthorized action. Validate types, allowed values, references, and business rules before use.

Define behavior for refusal, missing fields, truncation, and invalid output. Avoid endless “repair” loops. A bounded retry or a clear fallback is easier to operate and evaluate.

Treat prompts as versioned application code

A prompt encodes task instructions, output requirements, and assumptions. Version it with the application configuration and evaluation results. Changes that look stylistic can alter task behavior.

Separate trusted instructions from untrusted source material. Delimiters help organization but do not by themselves prevent prompt injection. Explain when the assistant should abstain and ensure consequential actions remain controlled by trusted code.

Select with a representative workload

Compare candidate models or configurations on the same held-out cases. Measure correctness, refusal/abstention behavior, latency, and cost. Include difficult and unsupported cases, not only examples that demonstrate the intended feature.

The largest model is not automatically the best operational choice. Verify model availability, data handling, supported interfaces, and limits in the chosen environment. Retain a rule or conventional software baseline to test whether model complexity is justified.

Choose the right adaptation strategy

Prompting is useful for clear tasks with adequate context. Retrieval supplies changing external knowledge. Tools perform bounded actions or queries. Fine-tuning may adapt behavior, but introduces data, evaluation, and maintenance requirements; it does not automatically make private knowledge current.

Use the smallest intervention that addresses a measured failure. The posting mentions DBRX; treat it as a model example to understand, while choosing supported models based on current access and task evidence.

Worked scenario

A ticket classifier returns valid JSON with category “urgent-refund,” but the application only supports billing, technical, and account. Schema validation should reject the unsupported category rather than silently introducing a new routing path. Compare a rules baseline with the model on ambiguous cases.

A small example

This example isolates one concept. Read its boundary conditions before adapting it to an application.

ALLOWED = {"billing", "technical", "account"}

def validate_category(result):
    if not isinstance(result, dict):
        raise ValueError("Expected an object")
    if result.get("category") not in ALLOWED:
        raise ValueError("Unsupported category")
    return result["category"]

print(validate_category({"category": "billing"}))

Practical assignment

1. Define a classification or summarization contract.
2. Create normal, ambiguous, and unsupported fixtures.
3. Implement a deterministic baseline.
4. Design or run a budgeted model comparison if credentials are available.
5. Validate outputs and record failure behavior.
6. Label live measurements separately from simulated results.

What to submit

Submit the artifacts named above, a short explanation of your decisions, and evidence of the checks you performed. Distinguish measured results from estimates and designs from executed integrations.

Knowledge check

References & next step

Platform examples are environment-dependent. Start with the official documentation in the reference library and verify the exact cloud, region, privileges, and versions you use.

Open the official reference library

Editorial edition: 5 October 2026. The local reference lab is executed locally; this course does not claim a live Databricks deployment.