The Cognitive Pentad for Autonomous AI Agents ā Local-first Model Context Protocol (MCP) infrastructure providing persistent state memory, perceptual vision caching, 3D/2D spatial world models, strategic BDI reasoning, and high-frequency behavior execution.
-brightgreen.svg?style=flat-square){kind=icon}
PuterVision engineers high-performance, zero-telemetry, local-first infrastructure and Model Context Protocol (MCP) servers for autonomous AI agents, coding assistants, and browser automation runtimes.
Modern frontier LLMs possess extraordinary semantic reasoning but suffer from fundamental architectural bottlenecks:
- Stateless Ephemeral Contexts: "Agent amnesia" across tool steps, compaction cycles, and restarts.
- Visual Token Bloat & Hallucinations: Re-ingesting raw screen pixels costs thousands of tokens per step and degrades reasoning fidelity.
- Absence of Spatial & Object Permanence: Inability to track unobserved entities, physical bounds, or 3D view projections.
- Uncalibrated Deliberation: Ad-hoc tool execution without formal multi-objective utility scoring or risk quantification.
- Sluggish, Non-Deterministic Action Loops: High-latency LLM roundtrips incapable of handling reactive events or ~60Hz browser/game controls.
The PuterVision Cognitive Pentad solves these limitations through a modular, synergistic local-first architecture. Backed by local SQLite databases in WAL mode, sub-millisecond dispatch, SHA-256 Merkle audit trails, and zero external network calls, each server handles a distinct cognitive tier while seamlessly interoperating across shared data structures.
flowchart TD
subgraph Perception ["1. Perception & Grounding"]
VM["šļø vision-memory-mcp (v1.2.1)<br/>Perceptual Cache & Element Grounding"]
end
subgraph Spatial ["2. Spatial World Model"]
WM["š world-model-mcp (v0.4.1)<br/>3D/2D Spatial Memory & Frustum"]
end
subgraph State ["3. Workflow State Memory"]
SM["š¾ state-memory-mcp (v1.2.1)<br/>Task DAGs, Decisions & Graph RAG"]
end
subgraph Cognition ["4. Strategic Deliberation"]
AR["š§ agent-reasoning-mcp (v0.2.1)<br/>BDI Planning, Utility & Risk"]
end
subgraph Execution ["5. High-Frequency Runtime"]
BM["ā” behavior-mcp (v0.2.1)<br/>~60Hz Behavior Trees & Safety Stack"]
end
VM -->|Visual Grounding & AX Detections| WM
VM -->|Visual Proof & State Hashes| SM
WM -->|Spatial Map & Entity Proximity| AR
SM -->|Active Goals, Plans & Blockers| AR
AR -->|Action Directives & Intentions| BM
BM -->|Execution Telemetry & Outcomes| SM
BM -->|Action Feedback & Obstacles| AR
| Project | Version | Badges | Focus & Capabilities | Links |
|---|---|---|---|---|
state-memory-mcp |
1.2.1 |
  |
Persistent Workflow State Memory (15 Tools) Zero-infrastructure, deterministic property graph backed by local SQLite in WAL mode. Tracks tasks, architectural decisions, artifacts, plans, and blockers. Features FTS5 full-text search, DAG cycle detection, multi-turn session attribution, SHA-256 Merkle audit chains, and WebGL 3D graph visualization. |
GitHub ā¢ npm ā¢ Website |
vision-memory-mcp |
1.2.1 |
  |
Visual Memory & Perceptual Grounding (15 Tools) Local-first perceptual cache using perceptual hashing (dHash/pHash), local CLIP vector embeddings via LanceDB, and Accessibility (AX) tree element grounding. Eliminates up to 90% of redundant vision LLM calls while predicting precise click/type coordinates and verifying Visual SDD specs. |
GitHub ā¢ npm ā¢ Website |
world-model-mcp |
0.4.1 |
  |
3D/2D Spatial World Model (15 Tools) Deterministic spatial internal world model for AI agents. Delivers persistent entity tracking, object permanence across occlusions with confidence decay, spatial topological relations ( on, inside, near), AABB collision prediction, expected view frustum projection, Three.js bridge, and Playwright 3D automation. |
GitHub ā¢ npm ā¢ Website |
agent-reasoning-mcp |
0.2.1 |
  |
Strategic BDI Cognitive Engine (10 Tools) Formal Belief-Desire-Intention cognitive deliberation framework. Decomposes complex objectives into dependency DAGs, computes multi-objective expected utility scores ((E[U] = \sum w_i u_i)), assesses quantitative risk, decays belief confidences, and triggers adaptive replanning upon obstacles. |
GitHub ā¢ npm ā¢ Website |
behavior-mcp |
0.2.1 |
  |
~60Hz Behavior Tree Execution Engine (10 Tools) High-frequency in-browser behavior tree tactical runtime. Delivers deterministic execution with priority reactive triggers, cooldown guards, a 5-layer fail-closed safety stack, telemetry capture, deterministic action sequence replay, stuck recovery, and SHA-256 Merkle audit verification. |
GitHub ā¢ npm ā¢ Website |
The 5 servers expose a unified surface of 65 specialized Model Context Protocol tools:
āāā state-memory-mcp (15 tools)
ā āāā manage_nodes - Create, update, query, and batch-process workflow entities
ā āāā manage_edges - Build and query typed DAG dependency and semantic links
ā āāā manage_sessions - Multi-turn workflow tracking and agent change attribution
ā āāā manage_tasks - Priority queue, runnable task extraction, and blocker resolution
ā āāā manage_snapshots - Checkpoint and time-travel rollback with diff inspection
ā āāā manage_specs - SDD design contract baseline verification and compliance
ā āāā manage_database - SQLite maintenance, diagnostics, and SHA-256 Merkle audits
ā āāā manage_data - Interleaved multi-modal trajectory and dataset export
ā āāā query_graph - Depth-bounded DAG traversal, shortest paths, and cycles
ā āāā get_analytics - Summary statistics, velocity, and blocker analytics
ā āāā get_events - Event-sourced changelog and session-level mutation feeds
ā āāā run_diagnostics - Graph health validation, orphan detection, and cycle checks
ā āāā use_blackboard - Shared key-value state blackboard for multi-agent workflows
ā āāā link_visual_proof - Connect workflow tasks to perceptual vision state IDs
ā āāā register_milestone - Project milestone tracking and release boundary anchors
ā
āāā vision-memory-mcp (15 tools)
ā āāā analyze_screenshot - Perceptual hash lookup, vector search, and AX tree grounding
ā āāā recall_memory - Semantic text & image similarity search over visual states
ā āāā record_outcome - Record UI action transitions and log visual blocker states
ā āāā get_navigation_paths - BFS shortest-path graph traversal between visual states
ā āāā predict_next_action - Predict optimal UI action and target screen coordinates
ā āāā compare_states - Structural perceptual diffing and video keyframe comparison
ā āāā get_session_context - Aggregated visual metrics, cache hit ratios, and state timeline
ā āāā manage_snapshot - Save, restore, and diff visual memory checkpoints
ā āāā manage_visual_spec - Visual SDD contract baseline registration and live verification
ā āāā manage_video - Ingest and search video recordings with keyframe extraction
ā āāā create_evidence_pack - Cryptographic multimodal evidence packages with SHA-256 hashes
ā āāā export_trajectories - Export training datasets in JSON, LLaVA, and Qwen2-VL formats
ā āāā undo_visual_mutation - Revert accidental state or transition edge ingestions
ā āāā forget_state - Purge sensitive or secret visual states for privacy compliance
ā āāā wait_for_visual_state - Polling barrier waiting until target UI state renders
ā
āāā world-model-mcp (15 tools)
ā āāā update_entity - Create/update 3D/2D entities with positions, bounds, and properties
ā āāā query_entities - FTS5 keyword, spatial proximity radius, and tag filtering
ā āāā set_relation - Record spatial relations (on, inside, next_to, above, contains)
ā āāā get_spatial_map - Export JSON, GeoJSON, topological graph, glTF, OBJ, or summary
ā āāā simulate_movement - Predict trajectories, test AABB obstacle collisions, plan waypoints
ā āāā ingest_observation - Merge vision detections into world model with re-identification
ā āāā get_expected_view - Compute entities visible within observer pose and FOV cone
ā āāā link_to_goal - Associate spatial entities with state-memory workflow task IDs
ā āāā record_outcome - Persist action outcomes and entity mutations in spatial log
ā āāā manage_spatial_spec - Spatial SDD physical constraint baseline registration and checks
ā āāā create_evidence_pack - Generate cryptographic SHA-256 spatial evidence bundles
ā āāā use_spatial_blackboard- Shared spatial intent board with spatial mutex locks
ā āāā manage_snapshot - Spatial time-travel snapshots, diffs, and state restore
ā āāā generate_game_inputs - Generate Playwright automation inputs and screen-to-3D unprojection
ā āāā wait_for_spatial_state- Block until an entity reaches specified spatial conditions
ā
āāā agent-reasoning-mcp (10 tools)
ā āāā set_goal - Manage hierarchical goal DAGs and decompose complex intents
ā āāā evaluate_situation - Score and rank candidate actions via multi-attribute utility
ā āāā replan - Dynamically reconstruct subgoals upon obstacles or failures
ā āāā assess_risk - Quantitative threat scoring and risk-adjusted probability calculation
ā āāā query_knowledge - Search heuristics, historical decisions, and strategic patterns
ā āāā set_utility_weights - Tune agent priorities (aggression, caution, greed, exploration)
ā āāā get_decision_trace - Chain-of-thought rationale playback and audit verification
ā āāā manage_beliefs - Structured belief state tracking with exponential confidence decay
ā āāā manage_intentions - Dispatch actionable directives queue to runtime behavior engine
ā āāā manage_reasoning_db - Diagnostics, snapshots, diffs, and SHA-256 Merkle audits
ā
āāā behavior-mcp (10 tools)
āāā load_behavior - Inject, initialize, or hot-swap behavior tree execution loops
āāā set_parameters - Dynamically update runtime behavior tree execution variables
āāā get_status - Query active traversal node path, tick count, duration, and errors
āāā abort_behavior - Immediately halt, pause, resume execution, or disengage inputs
āāā register_trigger - Configure priority reactive interrupts with cooldown guards
āāā replay_recording - Deterministic frame sequence capture and adaptive timing replay
āāā get_metrics - Execution telemetry, tick duration histograms, and stuck events
āāā manage_behaviors - Register and version behavior tree definitions with SHA-256 verification
āāā manage_blackboard - Read, write, delete, lease, and list shared behavior variables
āāā manage_runtime_db - Database maintenance, diagnostics, and SHA-256 Merkle audits
Beyond the Core Pentad MCP servers, PuterVision develops and maintains production-grade security, cryptographic, static analysis, and utility libraries:
| Project | Description | Links |
|---|---|---|
spc (Space Proof Code) |
High-performance, zero-dependency static analysis tool enforcing NASA Power of Ten safety-critical coding standards and security invariants across 20+ programming languages. | GitHub ā¢ npm |
WebCrypt |
Zero-dependency cryptographic vault suite for browser & Node.js Web Crypto API. Powers AES-256-GCM symmetric encryption, RSA-4096 hybrid public-key encryption, ECDH, digital signatures, and post-quantum security. | GitHub ā¢ npm ā¢ Demo |
ScreenChunk |
High-resolution screen capture, spatial chunking, and visual layout partitioning engine for dense browser and desktop interfaces. | Website |
BassMusic.ai |
Client-side, browser-native algorithmic and neural music generator demonstrating zero-server Web Audio processing. | Website |
- š 100% Local-First & Zero Telemetry: All tasks, decisions, visual caches, 3D entity models, and reasoning beliefs reside strictly in local SQLite and LanceDB files on your machine. Zero cloud dependencies, zero external analytics collection, and zero data leakage.
- ā” Sub-Millisecond Retrieval & Execution: Eliminates massive context window token burn. High-frequency indexing, perceptual hashing, and local graph traversal deliver deterministic responses in
<2ms. - š”ļø Fail-Closed Safety & Tamper-Proof Audit: Engineered with 5-layer safety stacks, watchdog circuit breakers, and SHA-256 Merkle audit chains for reproducible, tamper-evident agent trajectories.
- š§Ŗ Exhaustive Automated Verification: Backed by 1,253 automated unit, integration, and stress tests across 268 test suites with zero version drift verified across all configurations, manifests, docs, and releases.
- š Universal MCP Compatibility: Natively supported by Claude Code, Cursor, Gemini CLI / Antigravity, Windsurf, VS Code, and any Model Context Protocol compliant client.
Add the PuterVision Pentad servers to your favorite MCP-enabled editor or agent environment:
{
"mcpServers": {
"state-memory-mcp": {
"command": "npx",
"args": ["-y", "@putervision/state-memory-mcp"]
},
"vision-memory-mcp": {
"command": "npx",
"args": ["-y", "@putervision/vision-memory-mcp"]
},
"world-model-mcp": {
"command": "npx",
"args": ["-y", "@putervision/world-model-mcp"]
},
"agent-reasoning-mcp": {
"command": "npx",
"args": ["-y", "@putervision/agent-reasoning-mcp"]
},
"behavior-mcp": {
"command": "npx",
"args": ["-y", "@putervision/behavior-mcp"]
}
}
}# Install all 5 Pentad servers globally
npm install -g \
@putervision/state-memory-mcp \
@putervision/vision-memory-mcp \
@putervision/world-model-mcp \
@putervision/agent-reasoning-mcp \
@putervision/behavior-mcp
# Initialize project databases in your working directory
state-memory-mcp init
vision-memory-mcp init
world-model-mcp init
agent-reasoning-mcp init
behavior-mcp init- š Official Website: putervision.com
- š¦ npm Organization: @putervision
- š GitHub Organization: github.com/putervision
- š License: MIT Open Source
