Designing document retrieval and access for AI systems

An AI application answering questions from documents must select evidence that supports the task and control where that information can go. Those requirements extend from source ingestion to search, model inputs, citations, cached answers, and downstream actions. A relevant passage can be outdated, incomplete, restricted, or unsuitable for the intended recipient.

This resource develops a practical design for choosing retrieval methods and enforcing access throughout that path. It uses one fictional supplier-review workflow, with reusable source records, expected outputs, and tests. The recommendations are design guidance. The example’s organizations, identities, documents, amounts, and access rules are hypothetical.

1. Define the task and its evidence requirements

Begin with the output the user needs. “Find this agreement,” “explain the cancellation procedure,” and “determine the fee applicable on September 1” require different evidence. A complete list of agreements requiring review adds an enumeration obligation: the application must establish which agreements belong in the population before applying the rule.

For the running example, Alder Facilities wants an assistant to prepare an internal routing recommendation:

For supplier case F17, determine the recurring monthly fee applicable on September 1, 2026, calculate the annualized amount, and identify whether procurement review is required. Show the governing sources and any unresolved gaps.

Alder’s document custodian maintains a case register independently of search results. In the ordinary example, it identifies these records:

Source	Stipulated content	Role in the answer
A1, version 1, Schedule 1, page 8	USD 2,400 per month from January 1, 2026.	Establishes the original recurring fee and agreement context.
M1, version 1, section 2, page 2	Executed June 20; effective July 1, 2026. Replaces the recurring fee with USD 2,800 per month.	Changes the fee from its effective date; other agreement provisions remain in force.
P3, version 1, section 4.2, page 3	Effective January 1, 2026. Annualized recurring fees of at least USD 30,000 require procurement review.	Establishes the internal routing rule. Annualization means monthly fee multiplied by 12 in this example.

Table 1. Fictional source fixtures. These are stipulated inputs for explaining and testing the design, not observations from a deployment.

Maya, an internal reviewer, may read all three sources. Ben, a facilities coordinator, may read A1 and M1 but not P3. The assistant may draft a recommendation for an authorized reviewer. Recording a disposition, approving an agreement, making a payment, and sharing externally are separate operations.

Write those distinctions into the application’s task contract:

• The case register defines the expected document family and its revision.
• The requested business date determines which provisions apply.
• Current permissions determine what the requester and processing services may receive.
• Completion requires sufficient evidence for each requested conclusion.
• Missing, conflicting, or inaccessible evidence produces a bounded unresolved result.
• Delivery identifies the recipient and destination, including the audience of any shared workspace.

Keep the business date separate from the access-check time. Asking in November what fee applied in June does not restore permissions revoked in October. If the task instead asks what the organization could have known in June, it also needs a historical record of source availability. That is a different question from applying today’s available evidence to a past business date.

The register bounds the coverage claim. It can establish which documents are expected within its stated scope and revision; it cannot establish that no undisclosed amendment exists elsewhere.

2. Choose retrieval methods by the evidence they must recover

Retrieval-augmented generation selects source material for a generated answer. Its retrieval component can use identifiers, database queries, keyword search, embeddings, or combinations of these. A vector database is one implementation option.

Method	A suitable starting condition	Failure to test
Exact lookup or structured query	The user supplies a case, document, policy, or product identifier; an authoritative system exposes the required record.	Wrong tenant, identifier normalization, missing linked records, or selection of the wrong revision.
Read a complete document or bounded packet	The required material is known, small enough to process, and authorized for the task.	Missing attachments, unreadable pages, truncation, overlooked exceptions, or incomplete interpretation.
Keyword retrieval	Distinctive phrases, clause names, codes, and terminology identify relevant material.	Tokenization breaks identifiers; wording differs from the query; relevant qualifications occur elsewhere.
Embedding retrieval	Users and documents express the same subject in different language.	Similar topics outrank governing evidence; identifiers, dates, or negation are mishandled.
Hybrid retrieval	Lexical and semantic methods recover complementary evidence on the application’s cases.	A second branch adds irrelevant candidates, violates scope, or displaces required evidence after fusion.
Reranking	Required evidence reaches the candidate set but is ordered poorly for the available context budget.	The reranker drops a governing passage, truncates a qualifier, or processes material it may not receive.

Table 2. Conditional method choices and their diagnostic tests. The rows are alternatives and combinations, not stages of organizational maturity.

For Alder, start with exact case lookup and registered agreement-to-amendment relationships. Read the resulting authorized packet in full if practical. If the packet is large, search within it for the relevant provisions while retaining the inventory and coverage checks. Searching a whole repository for “current supplier fee” should not determine whether all amendments have been considered.

For a procedure question with no known identifier, keyword retrieval is a reasonable baseline. Add embedding retrieval when paraphrases cause material misses. The 2020 Dense Passage Retrieval study provides a useful historical comparison: combining its dense retriever with BM25 improved some dataset results and reduced others. That supports testing the combination against the actual task rather than assuming a universal gain. aclanthology.org

Hybrid retrieval needs a defined fusion rule. Reciprocal rank fusion combines positions in ranked lists, avoiding direct addition of unlike lexical and vector scores. Its original paper establishes the method and reports results on its studied collections; those results do not prescribe the best configuration for Alder. cormack.uwaterloo.ca

A cross-encoder reranker jointly processes the query and each candidate passage, as illustrated by Passage Re-ranking with BERT. It can improve ordering within the candidate pool. It cannot select an amendment that the first stage never supplied. Measure required evidence both before reranking and after the final context is assembled. arxiv.org

Query expansion, relationship traversal, and model-selected tool calls are additional choices. Preserve the original identifiers, date, tenant, and task constraints across every reformulation. Generated search text is a search aid, not evidence. A known sequence such as “fetch agreement, enumerate amendments, fetch policy” can be ordinary application logic. Dynamic planning is justified when the next information need genuinely depends on an earlier result.

Choose storage using these requirements: supported queries, filter selectivity, update and deletion behavior, isolation, rebuild time, operational skills, latency, and cost. A universal vector-count threshold cannot settle that choice. Fine-tuning on document contents also does not provide the source linkage and current access checks this task requires.

3. Register sources and preserve meaning through ingestion

An ingestion service may need broader access than any individual user to maintain an index. Give that service an explicit scope and permitted processing route. Its ability to read a file for indexing must not become every user’s ability to retrieve it.

Before parsing, establish the source’s identity, policy reference, and allowed processors. A hosted optical character recognition service, embedding endpoint, or document converter receives source information during ingestion. Access controls introduced only before answer generation arrive too late for that transfer.

Queries and user attachments can also contain protected information. Apply the permitted processing route before query rewriting or query embedding, as well as when handling retrieved documents.

A practical source record should distinguish the following:

Record group	Fields to preserve	Purpose
Identity and provenance	Tenant, source system, source identifier, immutable revision, content digest, original locator.	Connect derived material to the exact acquired source. A digest identifies bytes, not authority.
Business meaning	Document role, executed or approved status, effective dates, and validated amendment or supersession relationships.	Determine which source can establish which proposition. Supersession may affect one clause rather than the whole document.
Observation and processing	Acquisition time, register revision, parser/configuration version, processing status, failed pages or attachments.	Distinguish late arrival and processing failure from an absence of evidence.
Access and permitted use	Policy reference, classification, allowed processing destinations, retention category.	Resolve current access and handling rules without treating copied permissions as permanently valid.
Derived structure	Chunk and parent identifiers, page/region anchors, table headers, units, and dependency links.	Preserve context and propagate changes to dependent records.

Table 3. A source-record specification to adapt. Access metadata and the register itself require protection.

Quarantine records with missing access metadata or unresolved provenance. Do not treat an absent access-control list as permission for general use. Where a connector cannot represent the source system’s rules faithfully, narrow its scope or exclude the affected material until the discrepancy is resolved.

Parsing creates a representation that later stages can examine. Check whether it preserves text, reading order, headings, tables, units, footnotes, and page boundaries. Successful conversion and a valid schema establish that a process produced an output of the expected form. Correctness requires comparison with the source for the properties that matter to the task.

In the fictional M1, suppose the fee cell contains 2.8 under “Monthly fee, USD thousands.” The normalized value is 2.8 × 1,000 = 2,800. Keep the cell, heading, period, normalization, and page anchor together. Extracting 2.8 correctly while losing the heading is inadequate for the fee determination.

Chunking should follow those evidence relationships. A small matching passage can link to a larger section, a definition, or a continued table. Expansion must preserve the correct source revision and recheck access to everything added. An allowed paragraph does not automatically authorize its parent document or neighboring restricted sections.

What PDF-Bench and PDFsmith can establish

PDF-Bench develops guidance for selecting parsing and extraction components against application requirements, supported by retained comparisons of text and structural outputs. PDFsmith exposes PDF extraction through a common Python interface and command-line interface across parsing backends. These are relevant resources for the ingestion stage. GitHub

A parser evaluation can provide evidence about the tested extraction properties on a specified corpus, configuration, and metric. It does not establish that an application retrieved every governing document, selected the applicable version, enforced current permissions, or delivered a supported answer to an authorized recipient. A wrapper that successfully invokes a parser establishes still less about those downstream requirements.

Use such resources to structure parser comparisons and inspect failure cases. Parser rankings, accuracy figures, and cost comparisons need the exact corpus and reference outputs, parser and backend versions, configuration, failed-run accounting, and recorded results. Successful API execution alone does not establish extraction accuracy or readiness for this workflow.

4. Enforce eligibility before information reaches a consumer

Authorization should be an explicit application decision, independent of relevance scores and model instructions. OWASP’s authorization guidance recommends least privilege, default denial, server-side enforcement, and permission validation on every request. Those principles also apply to search results, source downloads, and derived answers. OWASP Cheat Sheet Series

For a user-initiated task, carry both the authenticated requester and the executing service identity. Bind tenant, task scope, allowed operation, and processing destination in trusted application code. Do not accept an identity, group membership, or access decision invented by the model or supplied as an unchecked request parameter.

An autonomous background job can operate under its own explicitly scoped authority. That does not authorize it to disclose its results to every user who can invoke or view the job.

Before releasing a source unit to a consumer, establish that:

1. The unit belongs to the allowed tenant and task scope.
2. The requester or explicitly authorized job may use it for the requested operation.
3. The executing service and receiving processor may handle it.
4. Its policy and source status are current enough for the defined access contract.
5. Its lineage and content revision are identifiable.

The consumer might be an external reranker, the answer model, a user-visible search result, a log processor, or an export destination. Eligibility is therefore more specific than “the user can open the original file.”

Keep authorization separate from applicability. A readable amendment may be future-dated. A superseded provision may be necessary for a historical question. First establish what may be processed, then determine what each source establishes for the requested date.

Where filtering belongs

Some systems apply access restrictions during candidate search. Others search a broader index inside a trusted retrieval service and authorize candidates before releasing content. Both designs need an explicit trust boundary. Unauthorized text, titles, counts, snippets, or other protected metadata must not cross that boundary into a consumer that lacks permission.

A trusted search engine may examine ineligible records internally. That differs from sending those records to a model and asking it to ignore them. Output screening is an additional check, not the mechanism that authorizes the model’s input.

Apply the same restrictions to every branch of hybrid search, direct fetches, relationship traversal, parent expansion, retries, and fallback paths. An unrestricted fallback can undo the protection on the main path.

Filtering can also affect adequacy. The pgvector v0.8.7 documentation explains that filters are applied after approximate index scans, potentially leaving too few matching results. Its iterative scans can search further, subject to limits. A system can therefore withhold all forbidden results and still miss required permitted evidence. raw.githubusercontent.com

Test selective access scopes explicitly. If permitted candidates are insufficient, retrieve further within the same scope, use an appropriate exact search, fetch known dependencies directly, or return an unresolved result. Never relax permissions to fill a result quota. Exact nearest-neighbor search means exactness under the chosen distance function; it does not establish semantic completeness.

Database policy features require correct configuration. PostgreSQL 18 documents that superusers and roles with BYPASSRLS bypass row-level security, and table owners normally do as well. Test the actual application role and pooled-connection behavior. Enabling a vector extension supplies no application authorization policy by itself. postgresql.org

A retrieval contract that exposes the checks

The following is a component-contract sketch, not a runnable security implementation. Each function represents behavior that needs implementation and integration tests.

request = bind_authenticated_identity_task_and_destination()
prior_context = load_and_reauthorize_retained_context(request)
inventory = protected_catalog.snapshot(request.scope)

candidates = choose_lookup_or_search(request, inventory)
evidence = []

for candidate in candidates:
    permit = authorize_current(candidate, request, next_consumer)
    if permit.allowed:
        evidence += fetch_exact_revision(candidate, permit)
    else:
        record_protected_reason(candidate, permit)

evidence = expand_context_with_the_same_checks(evidence, request)
coverage = compare_with_required_evidence(inventory, evidence, request)

if coverage.supports_requested_conclusions:
    draft = interpret_and_calculate(evidence, prior_context, request.business_date)
else:
    visible_gaps = project_permitted_gap_information(coverage, request)
    draft = prepare_supported_partial_result(visible_gaps, evidence)

dependencies = record_content_inputs(evidence, prior_context)
control_dependencies = record_inventory_and_policy_state(inventory, request)
validate_claims_citations_and_calculations(draft, dependencies)

release_through_authorization_and_freshness_gate(
    draft, dependencies, control_dependencies,
    request.recipient, request.destination
)

The coverage check must account for denied or unavailable requirements without exposing their protected identity. Optional irrelevant material should not block an otherwise supported answer. A missing governing source should.

Coverage assessment and the projection into permitted gap information run inside the trusted service. The answer model receives only the permitted explanation. Full inventory details and denial reasons remain in protected diagnostics.

5. Assemble an answer whose scope and dependencies are inspectable

For Maya’s September 1 request, the registered packet supports this result:

Applicable monthly fee: USD 2,800, from M1 version 1, section 2, page 2, effective July 1, 2026, amending A1’s fee provision.
Annualized amount: USD 2,800 × 12 = USD 33,600.
Routing: Procurement review required under P3 version 1, section 4.2, page 3, because USD 33,600 is at least USD 30,000.
Coverage: The agreement, registered amendment, and governing policy were available and examined within the stipulated case inventory.
Status: Recommendation prepared for authorized review.

These source references are fictional fixture locators, not links to public documents. In an application, each should open the exact authorized version and the relevant passage, including governing headers and footnotes.

For June 25, A1’s USD 2,400 monthly fee applies. Its annualized amount is USD 28,800, below P3’s threshold. M1 has been executed but is not yet effective. The output can identify the upcoming change if relevant and permitted.

Ben can receive the supported fee calculation from A1 and M1. His answer should leave the internal routing determination unresolved because his permitted evidence does not include the rule needed to establish it. It should not reveal P3’s threshold or silently use that restricted rule. An organization could create a separately approved routing service or releasable policy view; that would be an additional authorization design.

Missing or conflicting sources change the permitted conclusion

In a separate September 1 variant, the register records M2, an executed amendment effective August 15, but its content is unavailable. A1 and M1 still establish what those documents said. They no longer establish the current fee within the registered packet.

Where the user may see the missing record’s metadata, identify the gap and its owner. Otherwise use a message such as:

The available material is insufficient to establish the fee applicable on September 1 or the resulting review route. An authorized reviewer needs to resolve the source coverage before that determination can be completed.

Do not reveal a restricted title, document count, or inferred amendment merely to explain abstention. The protected diagnostic record can retain more detail for the authorized custodian.

If even a change in answer availability would reveal protected activity, define a uniform permitted response for that question class and test paired cases that differ only in the hidden information. A generic error message alone does not establish protection against such inference.

Conflicting executed copies require an authority decision. Upload time, semantic similarity, or an extraction confidence score cannot resolve which copy governs. Likewise, a cached policy summary cannot supersede its authoritative source. Preserve the competing evidence and hold the affected conclusion.

Citations and dependency records serve different purposes

A citation should support the claim beside it. The ALCE study evaluates answer correctness separately from citation quality; its citation-recall measure assesses support for statements that were generated. That does not establish that the answer considered an omitted governing amendment. aclanthology.org

Maintain two records. The reader-facing answer links consequential claims to supporting passages and shows calculations. The protected dependency record separates content inputs from control dependencies. Content inputs include every source and retained context item supplied to interpretation, including uncited passages and intermediate summaries. Control dependencies include inventory revisions, policy decisions, and configuration identifiers used to check freshness and invalidate results.

Control dependencies stay within the trusted service. Ben need not receive or have permission to read the full protected catalog to obtain an answer derived from A1 and M1. His permitted answer content and safe gap explanation are checked separately from the internal records used to decide whether to release them.

A model-generated list of citations is insufficient as the dependency record. Removing a citation or paraphrasing a fact does not remove its information sensitivity.

6. Make revocation and source changes govern every reuse path

An index, a cache, and a conversation history are additional stores of source information. They need rules for changes in both content and authority.

A permission change can make unchanged text unavailable. A new amendment can make an answer obsolete even though every source it originally used is unchanged. Track those events separately.

Define when revocation becomes effective

For the illustrative workflow, use this contract: after the access service acknowledges a revocation as effective, it admits no new processing or answer release that depends on the revoked material. The service must order revocations and admission decisions consistently. A final permission check followed by an unprotected send is not, by itself, an atomic guarantee.

There can also be delay between the source system’s permission change and the access service learning about it. Specify and measure that interval. If the requirement is to stop access at the source change itself, the integration must support that consistency requirement. Calling a periodically synchronized index “current” does not satisfy it.

Choose a mechanism appropriate to the requirement: an authoritative online check, a shared policy service with consistent decisions, or synchronized permissions with a defined maximum lag and a deny-on-stale rule. A valid authentication token can still contain outdated group claims. Session and entitlement refresh must participate in the revocation design.

Check permissions before content admission and again before answer release or a downstream action. For this example, buffer the answer until the release decision. If generation began before revocation, it may finish internally, but the affected result is withheld. Material already sent to a processor or recipient cannot be recalled by deleting an index entry. Cancellation and deletion requests have their own completion semantics.

The release gate also checks the captured source and inventory state against the freshness contract. A newly governing document can require reassessment even when access remains unchanged.

An unavailable authorization service should produce an operationally unresolved result for the affected material. It should not trigger fallback to broad ingestion credentials or silently become “no matching documents.”

Revalidate caches against content, permissions, and scope

A cache entry needs more than the query text. Record tenant, requester or justified shared entitlement scope, task, business date, source dependencies and revisions, register revision, relevant configuration, and permitted destination. Similar wording does not establish that two queries have the same date or evidence requirements.

On reuse, validate current permissions and whether relevant source or inventory changes invalidate the result. Per-user caching prevents some cross-user mistakes but still needs to handle that user’s revoked access.

Stored artifact	What can invalidate reuse	Required response
Search results or passage cache	Changed permissions, source withdrawal, revised text, stale index references.	Reauthorize and resolve current status before returning content.
Generated answer	Any changed input dependency, a newly registered governing source, changed task date, or changed destination.	Reassess coverage and regenerate where necessary.
Cached “no answer” result	New documents, restored access, repaired parsing, or a corrected register.	Reevaluate the negative result.
Summary, extracted field, or graph relation	Changed underlying source, lost lineage, or a narrower source policy.	Restrict use, recompute, or withdraw the derivative.
Conversation history and saved reports	Revoked access, a new viewer, or movement into a broader audience.	Reauthorize before replay, display, model reuse, or sharing.

Table 4. Invalidation follows both existing dependencies and changes to the relevant collection. A new document need not appear in an old dependency list to invalidate an answer.

For a derived artifact combining sources, a conservative default requires the recipient and processor to have access to every content dependency. Protected control dependencies remain internal. Broader release requires a separately approved transformation and disclosure policy. Redaction, aggregation, or sanitization must be assessed for the information it still reveals; deleting citations does not change the policy.

Protect embeddings, extracted records, summaries, traces, and evaluation samples as retained derivatives. Minimize copied personal or confidential data where the task permits. Keep detailed evidence needed for investigation in a controlled store with its own access and retention policy, rather than in generally accessible application logs.

Update indexes without restoring withdrawn information

Keep a source catalog and change record independently of the searchable representation. A current catalog can reveal that M2 is required even while its text is absent from an older index. The application can fetch it through an authorized route or hold the determination.

For a rebuild, identify the source snapshot, parser, chunking, embedding model, index configuration, and policy state used. Apply changes arriving during the rebuild before cutover, or use a current gate that rejects stale references. Check deletions and revocations explicitly. Rolling back the search index must not roll back current access decisions.

A withdrawal should first prevent ordinary retrieval and reuse, then propagate to chunks, embeddings, caches, and other derivatives according to their retention rules. A separately authorized historical archive can remain available for its defined purpose. Restoring a backup must reapply later withdrawals and permission changes before serving it.

7. Control source opening and downstream use

Opening a citation is a new access request. Authorize the source revision, rendered page, thumbnail, attachment, and download endpoint. An obscure identifier or a previously issued link is not a substitute for permission. Sharing an answer must also account for the audience of the destination.

For Alder, Maya’s permission to read the packet supports preparing the internal recommendation. It does not authorize an external email containing the packet, a payment, or approval of the supplier. If an application can record a disposition, the action endpoint should check the actor, allowed operation, target case, current evidence state, and destination before accepting it.

Document text must not modify those rules. An agreement might contain a passage instructing an AI reader to export files or ignore prior instructions. AgentDojo studies this attack surface through instructions inserted into untrusted tool-returned data. Its experiments establish a reason to test the boundary; they do not validate a particular defense for this workflow. arxiv.org

Treat retrieved instructions as source content. Restrict available tools to the task, authorize each invocation, and constrain output destinations. A prompt warning or injection detector may help, but it should not grant, revoke, or enlarge access. The same restrictions apply when an apparently relevant document suggests following a link or fetching an additional attachment.

8. Test evidence adequacy and access enforcement separately

Build references from the original source fixtures and independently defined policies. Do not let the system under test determine which documents were required or what its own caller was allowed to see.

For the following matrix, use the base A1/M1/P3 packet, Maya’s full access, Ben’s access to A1/M1, and a second tenant containing a different case also named F17. Add variants as specified. Keep M2 in its separate missing-source variant.

Instrument the boundaries: candidate release, external parsing and reranking requests, model inputs, retained context, user-visible results, source opening, logs, and downstream actions. Assertions should inspect both source references and actual payloads. A unique test string can detect copying, but paraphrases, amounts, titles, and existence information require additional checks.

Evidence and answer tests

ID	Setup or perturbation	Required behavior
E1	Maya requests the September 1 fee and route from the base packet.	Return USD 2,800 monthly, USD 33,600 annualized, and procurement review required, with the correct source versions and passages.
E2	Change the business date to June 25, then to July 1.	Use USD 2,400 and USD 28,800 before the boundary; USD 2,800 and USD 33,600 from July 1. Preserve current access checks.
E3	Ben submits the same September 1 request.	Return supported fee information; leave routing unresolved. No P3 content, threshold, or protected metadata reaches his context or response.
E4	The register lists effective M2, but its content is absent from the index and cannot be fetched.	Mark current fee and route unresolved. Explain the gap only at a permitted disclosure level.
E5	Two executed copies conflict; a later upload has no established precedence.	Preserve the conflict and withhold the affected determination. Do not resolve it by upload time or ranking score.
E6	Drop M1’s unit header, effective date, or a required page during parsing.	Detect inadequate evidence or produce an unresolved result. Do not silently normalize or apply the fee.
E7	Nearby unauthorized vectors dominate an approximate search, or lexical and semantic searches find different required evidence.	Release only permitted results; measure missing permitted evidence; refill or use a scoped fallback without broadening access.
E8	M1 appears among candidates but is removed by reranking or context truncation.	Detect the missing governing evidence at context assembly. Do not produce a completed current-fee determination.

Table 5. Functional acceptance cases. Expected values are hypothetical fixture results, not measured model performance.

Access and disclosure tests

ID	Setup or perturbation	Required behavior
S1	Another tenant requests its own F17, or a caller forges a tenant/user parameter.	Bind identity and scope from authenticated authority. No Alder content or protected metadata crosses tenants.
S2	A tool hop loses requester identity and retains only the broad service credential.	Reject the delegated request or confine it to a separately authorized job scope. Do not substitute ingestion access.
S3	Exercise the deployed database role and alternate users on pooled connections.	Policies apply under the real role; no owner/bypass privilege or retained session identity exposes another user’s records.
S4	Omit a filter from one hybrid branch, then expand an allowed child into a restricted parent.	Independently enforce both branch and expansion boundaries before content reaches any unauthorized consumer.
S5	Guess source, chunk, page, thumbnail, and download identifiers.	Enforce object-specific access; return no protected title, snippet, count, or content in failures.
S6	Maya may read a document, but the chosen hosted parser, embedding service, reranker, or model may not process it.	Send no protected payload to that processor. Use an allowed route or return unresolved.
S7	Maya warms an answer cache; Ben repeats the same question and a paraphrase.	Reauthorize cached dependencies. No restricted routing information leaks through exact or semantic cache reuse.
S8	A document tells the assistant to export the packet; separately, the user requests an unauthorized approval or broader share.	No forbidden tool call or destination receives information. Read permission does not confer action or sharing authority.

Table 6. Access tests apply to intermediate payloads and metadata as well as final answers.

Revocation and change tests

ID	Setup or perturbation	Required behavior
L1	Revoke Maya’s access to M1 while its index permissions still allow her.	The current gate blocks new use after the specified effective point despite the stale index.
L2	Repeat L1 with a cached answer, a saved summary, and prior conversation context.	Prevent replay or redisclosure of revoked material; do not regenerate from unauthorized history.
L3	Pause before context admission, revoke, resume. Repeat with revocation after generation but before release.	Enforce the stated ordering at both gates. Withhold affected output; record the decision in protected diagnostics.
L4	Make the policy service unavailable, exceed the freshness bound, or ingest a record without access metadata.	Deny affected use. Distinguish operational unavailability from an ordinary search miss.
L5	Withdraw a source while keeping its chunks, cached summary, and an old source link.	Block ordinary retrieval, derivative reuse, and source opening; verify cleanup or separately authorized retention.
L6	Register M2 after caching a successful answer; add needed evidence after caching “no answer.”	Invalidate both results through relevant inventory changes, even though their old dependencies did not change.
L7	Change parser, chunking, or embedding configuration during an index rebuild.	Preserve source/version mappings and retest required-evidence recovery. Reject incompatible or untracked mixtures; preserve deliberate versioned routing.
L8	Rebuild or restore an old index whose snapshot predates a revocation or deletion.	Current access and withdrawal state still governs. Cutover or rollback must not resurrect access.

Table 7. Lifecycle tests should use controlled change events and execution barriers so their timing is reproducible.

For L3, pause the request at a known boundary, apply the revocation, wait for the access service’s effective-revocation acknowledgment, and resume. Inspect every subsequent admission and release. Repeat with a policy decision already in progress to test the chosen ordering semantics. An arbitrary sleep cannot establish that the revocation reached the relevant service.

Interpret the results without hiding failure modes

For answerable requests, define the required evidence units before testing. Measure their presence in the initial candidates and in the final context:

$$\text{Required-evidence coverage}(q) = \frac{|G_q \cap C_q|}{|G_q|},$$

where $G_q$ is the reference set of required evidence units and $C_q$ is the assembled evidence. Use source/version/passage identities or explicitly accepted equivalent evidence, rather than literal string overlap.

Also report whether every required unit was recovered. Finding two of Alder’s three stipulated units gives coverage of $2/3$, but fails the complete-evidence condition if M1 is missing. The proportion alone can conceal the decisive omission. For legitimately unanswerable or restricted conclusions, use the expected unresolved outcome instead of penalizing the retriever for withholding forbidden evidence.

Report answer correctness, citation support, wrong effective dates, missed governing sources, unnecessary abstentions, and unauthorized boundary crossings separately. Include all assigned cases, including ingestion failures, timeouts, and malformed outputs. A refusal-only system cannot satisfy the positive cases. Correct handling of a missing amendment also leaves the underlying business determination open.

A failed access test blocks the affected use until corrected. Zero observed violations is the acceptance requirement for this suite; it is not proof that every possible violation has been excluded. OWASP similarly treats unit and integration tests as useful checks that do not replace dedicated security testing. OWASP Cheat Sheet Series

9. Use failures to select changes and assign responsibility

Start with the simplest design that meets the task contract. For Alder, that can be authenticated case lookup, a versioned register, controlled full-packet reading, explicit calculations, and authorized delivery.

Add complexity in response to a diagnosed failure. Missing paraphrases can motivate embedding retrieval. Complementary lexical and semantic misses can motivate fusion. Poor candidate ordering can motivate reranking. Missing registered amendments require a coverage or synchronization fix. Unauthorized reuse requires an access or invalidation fix. Changing the answer model addresses none of those mechanisms automatically.

Compare alternatives on the same scoped cases and permissions. Record permitted-evidence recovery, correct and unresolved outcomes, reviewer corrections, processing latency, time to accepted disposition, and operating cost. Include ingestion, updates, policy checks, storage, retries, and human review. Reuse may reduce processing while increasing the work needed to keep derivatives current.

Assign named people to four decisions, even when one person holds several roles:

Responsibility	Decision and evidence
Source custodian	Defines the case inventory, authoritative versions, and unresolved relationships; resolves missing or conflicting records.
Access-policy owner	Defines who may read, process, share, and act; sets the revocation and freshness contract.
Retrieval/application owner	Maintains ingestion, indexing, context assembly, dependencies, and change tests; diagnoses missing evidence.
Workflow owner	Defines acceptable outputs and downstream actions; ensures unresolved cases reach someone with the access and expertise to finish them.

Table 8. Responsibilities concern concrete decisions. Technical access checks cannot resolve an undocumented business rule or an unknown source’s authority.

Before a material change is released, rerun the affected functional cases and the critical access and lifecycle cases under the proposed configuration. Preserve the test fixtures, source and policy versions, configuration, boundary observations, failures, and disposition of unresolved issues. These records make the release decision assessable and subsequent regressions diagnosable.

References

The references below support method descriptions and specific documented mechanisms. The workflow, access rules, and tests are proposed designs.

1. Karpukhin et al. (2020). Dense Passage Retrieval for Open-Domain Question Answering. EMNLP, especially Table 2 and its comparison method.
2. Cormack, Clarke, and Büttcher (2009). Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods. SIGIR.
3. Nogueira and Cho. Passage Re-ranking with BERT. Version 5, April 14, 2020.
4. OWASP. Authorization Cheat Sheet. Living guidance, inspected November 2, 2026.
5. pgvector. README, version 0.8.7, especially Indexing, Filtering, and Iterative Index Scans.
6. PostgreSQL. Version 18 documentation, Row Security Policies.
7. Gao et al. (2023). Enabling Large Language Models to Generate Text with Citations. EMNLP, especially section 3.3.
8. Debenedetti et al. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. Version 3, November 24, 2024.
9. PDF-Bench repository. Project README, inspected November 2, 2026.
10. PDFsmith repository. Project README, inspected November 2, 2026; the inspected main-branch description identifies an unpublished development version.