Agent Framework Reliability Leaderboard
A mechanical governance score for the 27 most-used autonomous-agent frameworks. Every score is the output of agent-contracts scan against a clean clone — no vibes, no testimonials, no self-reported numbers.
| # | Project | Score | Grade | Trend | Strongest | Weakest |
|---|---|---|---|---|---|---|
| 1 | langchain-ai/langgraph ↗ | 100 | A | – | tests_and_ci | — |
| 2 | microsoft/autogen ↗ | 100 | A | – | tests_and_ci | — |
| 3 | run-llama/llama_index ↗ | 100 | A | – | tests_and_ci | — |
| 4 | stanfordnlp/dspy ↗ | 100 | A | – | tests_and_ci | — |
| 5 | Significant-Gravitas/AutoGPT ↗ | 93 | A | – | tests_and_ci | secret_safety |
| 6 | crewAIInc/crewAI ↗ | 93 | A | – | tests_and_ci | secret_safety |
| 7 | langchain-ai/langchain ↗ | 93 | A | – | tests_and_ci | secret_safety |
| 8 | microsoft/semantic-kernel ↗ | 93 | A | ▼ -7 | tests_and_ci | secret_safety |
| 9 | camel-ai/camel ↗ | 93 | A | – | tests_and_ci | secret_safety |
| 10 | pydantic/pydantic-ai ↗ | 93 | A | – | tests_and_ci | secret_safety |
| 11 | openai/openai-agents-python ↗ | 93 | A | – | tests_and_ci | secret_safety |
| 12 | mem0ai/mem0 ↗ | 93 | A | – | tests_and_ci | secret_safety |
| 13 | agno-agi/agno ↗ | 93 | A | – | tests_and_ci | secret_safety |
| 14 | geekan/MetaGPT ↗ | 90 | A | – | tests_and_ci | secret_safety |
| 15 | princeton-nlp/SWE-agent ↗ | 90 | A | – | tests_and_ci | dependency_pinning |
| 16 | huggingface/smolagents ↗ | 90 | A | – | tests_and_ci | dependency_pinning |
| 17 | google/adk-python ↗ | 90 | A | – | tests_and_ci | secret_safety |
| 18 | All-Hands-AI/OpenHands ↗ | 85 | B | – | tests_and_ci | eval_harness |
| 19 | assafelovic/gpt-researcher ↗ | 85 | B | – | tool_governance | dependency_pinning |
| 20 | Aider-AI/aider ↗ | 84 | B | – | tests_and_ci | secret_safety |
| 21 | OpenBMB/ChatDev ↗ | 80 | B | – | tool_governance | eval_harness |
| 22 | TransformerOptimus/SuperAGI ↗ | 78 | C | – | tests_and_ci | eval_harness |
| 23 | reworkd/AgentGPT ↗ | 78 | C | – | tests_and_ci | eval_harness |
| 24 | openai/swarm ↗ | 70 | C | – | tool_governance | dependency_pinning |
| 25 | yoheinakajima/babyagi ↗ | 65 | D | – | tool_governance | tests_and_ci |
| 26 | microsoft/JARVIS ↗ | 65 | D | – | secret_safety | tests_and_ci |
| 27 | letta-ai/letta ↗ | 20 | F | ▼ -58 | tool_governance | tests_and_ci |
Scores are intentionally harsh: a missing dimension earns zero. The scanner measures observable governance surface (tests/CI, tool gating, secret hygiene, dependency pinning, eval harness, observability, resilience) — not runtime safety. A high score means the guardrails a reliable agent needs are present, not that the agent is provably safe.
Reproduce any score
pip install "agent-contracts @ git+https://github.com/impartshadow/agent-contracts.git" git clone --depth 1 https://github.com/<owner>/<repo>.git agent-contracts scan --root <repo> --json
Shallow clone is sufficient — the scanner reads the working tree at HEAD and skips .git, so depth-1 and a full clone produce identical scores.
Embed your live badge
Every indexed framework has a self-updating governance-score badge — a static shields.io endpoint served from this page. Embed it once; it re-renders the current score on every scan, no re-publish on your side.
[](https://impartshadow.github.io/agent-contracts/leaderboard/)
Submit your agent
Open an issue with your repo URL, or run agent-contracts scan --root . --output-markdown AGENT_GOVERNANCE_SCORE.md and open a PR adding the row. Self-scores are accepted only when reproducible from a clean clone.