Build knowledge-grounded LLM applications

A documentation assistant is asked whether an integration supports a particular authentication method. It retrieves a preview announcement and confidently says yes. The text is real and the citation works, but the feature is unavailable in the user's installed release.

Grounding an answer requires more than including related documents in a prompt. The application needs evidence that is appropriate for the question and rules for what to do when that evidence is incomplete.

Treat the knowledge base as a versioned application input. With Polyaxon, DAG operations can prepare source material, build candidate indexes, and run qualification queries. Tracking and artifacts retain the configuration and evidence used to decide whether a knowledge update is ready.

Define the answer contract

For the integration assistant, establish the information required before answering: product, installed version, authentication method, and any environment constraint that changes support.

Classify the question before selecting its evidence path:

This classification prevents a historical document from answering a live-state question. It also keeps a roadmap statement from being presented as a shipped capability.

Build a small, explicit evidence bundle

Retrieve candidate material, apply the required access and version checks, then assemble the passages used for generation. Give every passage an identifier, source revision, section reference, and relevant product metadata.

Keep source authority explicit. A release-specific reference page, an archived announcement, and a community discussion may all mention the same feature. They should not be treated as interchangeable evidence merely because their wording is similar.

For the authentication question, include the supported-method table and any exception applying to that release. If the exception lives in a neighboring section, preserve that relationship during extraction. A fragment that omits its conditions can support a misleading answer even when retrieval similarity is high.

The verified data pipeline guide covers source identity, freshness, and access metadata in greater depth. Here, those fields become inputs to the answer decision.

Make the index build an inspectable operation

For the documentation assistant, start with a source manifest containing approved page IDs, product versions, content digests, and extraction rules. Record its revision separately from the embedding model and chunking settings. A changed source corpus and a changed embedding model are different interventions.

The following operation template assumes your team has registered a build-docs-index component with corpus_revision, embedding_revision, and chunk_size inputs. Its program reads the selected corpus, builds a new candidate index, and returns the index identifier and source manifest URI as tracked outputs. The component name and connection below are illustrative and must exist in your environment.

version: 1.1
kind: operation
name: build-candidate-docs-index
hubRef: your-team/build-docs-index:1.0
params:
  corpus_revision: {value: product-docs-snapshot-24}
  embedding_revision: {value: embedding-candidate-v2}
  chunk_size: {value: 512}
termination:
  timeout: 3600
  maxRetries: 0
runPatch:
  connections: [approved-docs-store]
  container:
    resources:
      requests:
        cpu: "2"
        memory: 8Gi
      limits:
        cpu: "4"
        memory: 16Gi
        nvidia.com/gpu: 1

This uses the documented operation patch mechanism and connections. The resource values illustrate a local embedding workload; choose them for your model and hardware. Remove the GPU request for an API-only embedding client, and manage provider request limits in that client. The component must define whether chunk_size counts tokens or another unit; in this example it is a token budget under its recorded tokenizer revision.

Write the candidate to a distinct index namespace or revision. Do not replace the serving index while an experiment is still building it. Your search backend and indexing code implement those operations; requesting resources in Polyaxon does not configure a vector database automatically.

Make missing and conflicting evidence visible

Treat “no relevant document,” “document inaccessible,” “wrong version,” and “sources conflict” as distinct outcomes. They call for different user responses and different operational fixes.

If the installed release is unknown, ask for it. If the reference and release notes disagree, apply the documented source-precedence rule or send the conflict for review. If access is denied, avoid revealing restricted document titles or snippets while explaining that the application could not verify the answer.

A retrieval score is not a universal probability that an answer is correct. Choose any sufficiency threshold using reviewed examples from your corpus, and reassess it when the retrieval configuration changes.

Include cases where the correct response is partial. The assistant might confirm that a feature exists while being unable to verify its availability in one deployment environment. Preserve that distinction rather than forcing every request into a complete answer or a generic refusal.

Generate claims that can be checked

Ask the application to return the answer with references to evidence identifiers. Validate that those identifiers were actually supplied and that their links resolve to the intended source revision.

Then check support at the claim level. A citation to a page that mentions authentication does not establish that it supports the specific configuration being recommended. Evaluate conditions, exceptions, and version qualifiers as part of the claim.

Keep retrieved instructions separate from application policy. A document may contain examples of commands or even malicious text. It cannot grant permission to execute those commands, change the assistant's access scope, or override the answer contract.

For live-state questions, use a narrowly authorized API response with a timestamp. Explain the observation window. A documentation index cannot establish current service health, regardless of how recently it was refreshed.

Release knowledge changes as application changes

A new document set can alter behavior without a code deployment. Record the corpus revision, extraction rules, chunking configuration, embedding model, index revision, and answer prompt used by a candidate.

Build a candidate index separately from the active one when the search infrastructure permits it. Compare the same questions against both. Include changed topics, unchanged topics, removed features, and questions whose answer should remain unavailable.

Promotion needs an explicit decision and a rollback target. Permission revocations and required deletions need a separate, timely enforcement path; they should not wait for the next full index evaluation. Reverting an index must not restore access to revoked material.

Connect indexing and evaluation with Polyaxon

Connect the build operation to an evaluation component using output parameter references. Pass the returned candidate index identifier and manifest URI explicitly. The evaluator should query that exact index, not the currently active alias, and record the query-set and answer-rubric revisions.

Use metadata logging for corpus, embedding, and query-set revisions and for numeric measurements such as answered questions, unsupported answers, and clarification rate. Save permitted source manifests, evidence bundles, generated answers, and case-level reviews as controlled artifacts. The SDK example shows how to attach a detailed report to the run.

This record helps separate a content change from a model or retrieval change. It also makes a reported wrong answer easier to reproduce under the corresponding corpus revision.

Select baseline and candidate in the comparison dashboard, then inspect changed questions in their reports. Keep index publication in a separate operation with an explicit approval policy, following the qualification and release pattern. The search service still owns live retrieval, authorization, and index cutover.

Evaluate the decision to answer

Measure correctness and support among answered questions, along with the proportion answered, clarification rate, and escalation rate. An application can improve apparent correctness by declining nearly everything, so report coverage alongside quality.

Inspect unsupported confidence and unnecessary abstention separately. The first risks misleading users; the second makes useful knowledge difficult to access. The RAG evaluation guide explains how to diagnose retrieval and generation failures independently.

Start with a narrow corpus and a clear question class. Expand when the team can show which sources support an answer, how updates affect that answer, and how the application behaves when the evidence is missing.