Read the assurance pillar alongside the adoption language
The memorandum sets four pillars: adoption, adaptation, assurance and accountability. It calls for security and functionality testing, evaluation, validation and verification; it also addresses government control over mission-dependent AI and protection against unauthorized disablement or material modification.
Its deadlines include a 90-day update to DoD Directive 3000.09 with annual review, and a 120-day review and update of procurement processes for rapid onboarding from multiple vendors. Those are directed actions, not proof that all resulting guidance has been issued. Accountability remains with commanders, directors and agency heads, including obligations involving lawful use, privacy and civil liberties.
This broad direction builds on existing requirements. The January 2023 version of DoDD 3000.09 already required appropriate human judgment and verification, validation and testing for covered autonomous weapon systems. Programs should map their actual obligations instead of assuming assurance began with this memorandum or that one identical approval gate applies everywhere.
The useful deliverable is an evidence package
An assurance claim needs a defined system and a defined use. Saying a model is accurate says little until the reader knows the task, data, failure consequence and tested operating conditions. A release can improve average performance while becoming worse on a small set of consequential cases.
Build a package that answers:
- What was tested? Identify the model, supporting software, retrieval data, tools and permissions.
- Where did it work? Describe representative tasks, measured results and the conditions under which those results apply.
- Where did it fail? Include misleading confidence, missing information, hostile inputs and cases requiring human intervention.
- What can change? Document supplier updates, administrative access, dependency changes and the government's approval controls.
- Who can act on the evidence? Name the owner of release, restriction, incident response and reevaluation decisions.
The supplier-control question deserves engineering attention. A contractual commitment has to be supported by the architecture: access controls, update mechanisms, dependency inventory and recoverable configurations. A system that cannot identify what changed is difficult to assure after a vendor refresh.
Translate policy into a bounded release decision
A policy requirement can become difficult to use when it arrives as a broad demand for trustworthiness without a defined application. The program needs to connect the requirement to a specific function, the information involved, and the person responsible for accepting its use. Otherwise, teams can generate large amounts of documentation without resolving the decision in front of them.
Consider a hypothetical internal document assistant. Its evaluation might need to establish whether it cites the authorized source, preserves important qualifications, and makes unsupported answers recognizable. A change that gives the same application access to an action-taking tool creates a different assessment problem. The existing evidence remains useful, but it does not automatically cover the expanded permissions.
The useful question is what the proposed release changes about the accepted use. It may change the model, the available data, the users, or what the surrounding software can do. Each deserves an assessment proportionate to its effects. Calling both releases the same product should not obscure the difference.
Government control needs a supportable implementation
A government customer can require control over its mission-dependent capability while still relying on supplier expertise and infrastructure. The practical work is to identify the dependencies and the arrangements needed to operate, maintain, and change the delivered service within the approved scope.
The customer should understand who can introduce updates, how changes are communicated, what records remain available, and what happens if the service relationship changes. Contract terms need to match the technical and support arrangements. A broad promise of control is difficult to use if the receiving organization lacks the necessary access, documentation, or skills.
There are tradeoffs. Retaining greater local responsibility may improve independence while adding staffing and maintenance burdens. Relying on a managed service may reduce those burdens while making the service agreement and transition provisions more consequential. The program should choose deliberately and identify which responsibilities it can realistically sustain.
Make uncertainty actionable
An assurance package should allow a responsible official to distinguish a known limitation from an unexamined question. A known limitation may support a restricted use; an unanswered question may require more evidence before any use is justified. Treating both as a generic risk score can hide the available choices.
For each material issue, state the consequence, the supporting evidence, and the available response. The response could be further testing, a narrower use, an additional review step, or deferring the release. These are recommended decision practices, not a claim that NSPM-11 prescribes one universal procedure.
The program should also state what would change its conclusion. New information, a supplier update, or a different user community can make an earlier decision worth revisiting. Recording those conditions gives monitoring a purpose and makes later review more focused than repeatedly asking whether the entire system is trustworthy.
Generate evidence while the system is being built
CDAO's evaluation frameworks cover models, human interaction, integration and operational performance. That breadth matters: a model can pass an isolated test while users misinterpret its outputs or its surrounding application gives it excessive authority.
- Define important use cases and unacceptable outcomes before integration.
- Preserve versioned test inputs, expected behavior and observed results.
- Run regression checks when a model, data source or connected tool changes.
- Test how users recognize uncertainty and recover from an incorrect recommendation.
- Keep the responsible official's decision, restrictions and monitoring plan with the release record.
This approach makes TEVV an ongoing engineering activity. It also gives the program a way to explain why evidence transfers to a new configuration—or why additional testing is needed. Logging should support reproducibility and oversight while respecting classification, access and retention requirements.
Accountability works best when officials receive a clear decision rather than an undifferentiated stack of reports. Show the mission benefit, remaining uncertainty and available restrictions. Vendors that deliver that clarity make their capability easier to evaluate, integrate and sustain.
Sources and further reading
Spartan X’s AI and cybersecurity practices connect these policy obligations to system design, testing and release decisions, giving mission owners a clearer basis for adopting capability with confidence.
