Is microsoft/JARVIS production-ready?

Name: microsoft/JARVIS agent governance score
Creator: Shadow
License: https://opensource.org/licenses/MIT

Mechanical governance score from agent-contracts scan against a clean clone. No vibes, no testimonials — reproducible from the exact commit below.

65/100 D no change since the previous scan

Weakest dimension: tests_and_ci (0/20). This measures a test suite plus CI that runs it. A low score here means an agent built on JARVIS can fail in this dimension without the repo signalling it.

Dimension breakdown (100 points)

tests_and_ci — a test suite plus CI that runs it0/20

tool_governance — permission / allowlist / approval / human-in-loop gating on tool calls13/20

secret_safety — secrets git-ignored, no obvious hardcoded keys15/15

eval_harness — an eval / benchmark / labeled-corpus surface15/15

dependency_pinning — a lockfile or pinned dependency manifest7/10

observability — logging / audit / tracing10/10

resilience — retry / backoff / fallback / escalation5/10

Reproduce this score

pip install "agent-contracts @ git+https://github.com/impartshadow/agent-contracts.git"
git clone --depth 1 https://github.com/microsoft/JARVIS.git
git -C JARVIS checkout 7624cf388b
agent-contracts scan --root JARVIS --json

Scored commit: 7624cf388b47334ff8a0868e7d862dde18cfda86. The score is the output of a deterministic scanner, not a judgment call — a clean clone reproduces it.

What this score can and can't tell you

This index measures observable governance surface — the guardrails a reliable agent needs. It does not certify that agents built on JARVIS behave correctly at runtime. The most expensive agent failure — confidently claiming a task is "done" with nothing to back it — has no surface to grep, so no static score (including this one) can see it. Here's the proof, on the #1-ranked framework.

Get the weekly scan delta — $7/mo See the full index

Every framework is re-scanned on a schedule. The weekly delta — what moved, on which commit, and what it means for teams building on it — goes out at echofromshadow.substack.com.