An in-the-wild benchmark for AI agents in the production harness.
-
Updated
Aug 17, 2026 - Python
#
agentic-evaluation
Here are 14 public repositories matching this topic...
Gen AI Evaluation Toolkit on AWS is a flexible, cloud-native accelerator built on AWS serverless architecture that enables comprehensive evaluation of generative AI applications.
agent evaluation bedrock evaluation-framework llm agentic-ai agentcore agentic-evaluation agentic-evals
-
Updated
Sep 8, 2026 - TypeScript
DataClawEval: A Benchmark for Engineering Data Agents in Real Industrial Harness
-
Updated
Aug 5, 2026 - Python
A SnitchBench-style benchmark inverted for the dark-forest problem: does a listener AI alert humans about an alien signal when alerting may doom humanity?
-
Updated
Apr 27, 2026 - TypeScript
🚀 AI Evolution Factory - From evaluation tool to continuous AI self-improvement platform. Agentic evaluation, auto-finetuning, global P2P testing, and hardware telemetry for local LLMs
python benchmark arena ai evaluation p2p elo fine-tuning huggingface byzantine-consensus llm vllm local-llm ollama unsloth ai-evolution openai-compatible hardware-telemetry agentic-evaluation crypto-proof
-
Updated
Jun 23, 2026 - Python
A small benchmark for agent skills, verification artifacts, and fresh-session resumability.
benchmarking verification developer-tools software-engineering-research ai-engineering coding-agents agent-benchmark agentic-evaluation workflow-evaluation
-
Updated
Jul 10, 2026 - Python
Pipeline to investigate structured reasoning and instruction adherence in multimodal LLMs
benchmark robustness grounding out-of-distribution neuro-symbolic robustness-verification instruction-following trustworthy-ai large-language-models faithfulness hallucination-detection agentic-ai llm-alignment agentic-evaluation agentic-reasoning deterministic-eval
-
Updated
May 28, 2026 - Python
Runnable lab measuring invalidation/staleness in agent memory — paper: Are We Ready For An Agent-Native Memory System? (arXiv:2606.24775)
benchmark invalidation mlx memory-systems bi-temporal rag paper-reproduction llm kuzu llm-agents agent-memory agentic-evaluation build-with-paper
-
Updated
Jun 26, 2026 - Python
Deterministic synthetic fixtures and hard-gate scoring for agentic biosafety safeguard routing.
-
Updated
Sep 6, 2026 - Python
Two-tier honest-evaluation harness for agentic RTL design (research, WIP)
benchmark eda rtl verilog formal-verification hardware-design llm-agents agentic-evaluation build-with-paper
-
Updated
Jul 2, 2026 - Python
Proof-bound evaluator stress testing with oracle-witnessed reward-hacking exploits and replayable evidence.
-
Updated
Sep 3, 2026 - Python
Measure AI agents’ performance with standardized tests across 314 tasks, 33 domains, and 4 difficulty levels for clear, reproducible comparison.
agent benchmark ai discovery openai benchmarks codex claude ai4science llm ai-scientist clawdbot openclaw agentic-evaluation auto-research research-claw
-
Updated
Sep 9, 2026 - Python
A Harbor-format RL environment for pass@k estimation, with a verifier built to resist reward hacking. 17 tests, 10 adversarial solutions rejected, amd64 CI.
docker pytest harbor rl-environments llm-evaluation reward-hacking terminal-bench agentic-evaluation
-
Updated
Sep 8, 2026 - Python
A GPU-backed Harbor/Terminal-Bench RL environment for data-poisoning defense: containerized task, oracle solution, and a 25-check verifier that rejects 17 reward-hacking solutions.
docker pytest harbor adversarial-machine-learning rl-environments data-poisoning llm-evaluation reward-hacking terminal-bench agentic-evaluation
-
Updated
Sep 8, 2026 - Python
Add this topic to your repo
To associate your repository with the agentic-evaluation topic, visit your repo's landing page and select "manage topics."