1. Skill Overview
This skill is deployed when OpenClaw encounters test performance bottlenecks — including slow full test suite execution, heavy Plugin Inspector loading, excessive RSS memory consumption, or imminent OOM risks. It delivers a systematic performance engineering workflow: establish baselines → locate root causes → resolve fundamental issues → re-benchmark and quantify performance gains, ensuring all optimizations are backed by measurable data.
Core Principle: Evidence-first, fix root causes instead of mitigating symptoms. Avoid speculative runner parameter tweaks; eliminate performance bottlenecks at the source through precise metric collection and root-cause analysis.
2. Core Capabilities
2.1 Establish Performance Baselines
Reliable performance baselines must be captured before any code modifications.
| Command | Purpose |
|---|---|
| pnpm test:perf:groups --full-suite --allow-failures --output | Rank test groups across the full suite to identify the slowest modules |
| /usr/bin/time -l pnpm test <file> --maxWorkers=1 --reporter=verbose | Capture wall-clock time and peak RSS for single-file test runs (Linux/macOS) |
| OPENCLAW_VITEST_IMPORT_DURATIONS=1 OPENCLAW_VITEST_PRINT_IMPORT_BREAKDOWN=1 pnpm test | Decompose import latency per test file to pinpoint barrel/runtime loading hotspots |
| pnpm test:extensions:batch <group> | Batch plugin scope testing to isolate performance defects within specific plugin groups |
2.2 Multi-Dimensional Metric Collection
The skill defines a complete performance metric matrix:
| Metric | Purpose | Collection Method |
|---|---|---|
| Wall Time | End-to-end suite latency perceived by end users | /usr/bin/time -l, Testbox runtime metrics |
| Internal Vitest Duration | Overhead of test case execution and module imports | Per-file/shard output from Vitest |
| Import Latency | Expansive barrel file / runtime module loading overhead | Environment variable OPENCLAW_VITEST_IMPORT_DURATIONS=1 |
| Peak RSS | Memory pressure and OOM risk assessment | /usr/bin/time -l, test:extensions:memory script |
| CPU / User / Sys Time | Differentiate CPU-bound workloads vs I/O waiting | /usr/bin/time -l, Testbox job timing logs |
| Heap Snapshots | Distinguish genuine memory leaks vs retained module graphs | openclaw-test-heap-leaks dedicated workflow |
2.3 Built-in Root Cause Pattern Library
Predefined known performance anti-patterns specific to the OpenClaw codebase for rapid diagnosis:
- Full packaged channel/plugin runtime loaded solely to retrieve static metadata (fix: adopt lightweight manifests or static helper utilities)
- Fallback calls to getChannelPlugin() used where preloaded fixtures or pure parsers would suffice (fix: inject dependencies directly)
- Overly broad barrel files (api.ts, runtime-api.ts, test-api.ts, plugin-sdk) pulled into hot test execution paths (fix: use granular subpath imports)
- SDK root aliases or package barrels pull wide plugin dependency graphs into focused workloads (fix: refactor export structure)
- Plugin-Inspector loads full runtime code merely to render metadata, reports, or CI policy scoring (fix: decouple static data from runtime logic)
- Packaged plugins reuse raw production config/home states instead of synthetic, sanitized, isolated test fixtures (fix: deploy test-specific fixtures)
- vi.resetModules() paired with repeated fresh imports in every test iteration (fix: reuse module cache or isolated containerization)
- Gateway/server/client instances instantiated per test when simple state resets would suffice (fix: reuse suite-level singleton instances)
- Parallel Vitest workers share the default node_modules/.experimental-vitest-cache without separate OPENCLAW_VITEST_FS_MODULE_CACHE_PATH assignments (fix: allocate independent cache paths per worker)
2.4 Actionable Remediation Strategies
Targeted fixes mapped to each root cause category:
- Static metadata / parsing logic: Extract into lightweight helpers or manifest artifacts, shared by full runtime and fast execution paths
- Excessive runtime loading: Apply dependency injection; only load plugin lookup logic, explicit fixtures, and pure stateless helpers instead of broad mocks
- Repeated server/client instantiation: Reuse suite-level server/client instances when fresh handshake validation is irrelevant to test assertions
- Scheduler/background loops: Keep disabled unless tests explicitly validate scheduling behavior
- Plugin import paths: Migrate static metadata to lightweight manifest artifacts; defer full runtime plugin loading to explicit execution boundaries
2.5 Specialized Plugin Suite Optimization
Apply this workflow when optimizing packaged plugins, Plugin Inspector, SDK barrels, package boundary tests, or extension suites:
- Suite scope mapping: First run pnpm test extensions/, pnpm test:extensions:batch, package boundary canary/compile checks, and plugin import memory analysis
- Narrow-to-wide testing order: Validate single-plugin test slices and Plugin-Inspector subsets first, then incrementally expand scope to SDKs, channels, memory workloads, and feature plugins
- Crabbox/Testbox validation: Maintainers default to Blacksmith Testbox for broad or package-heavy plugin benchmarking
- Package artifact sensitivity: Switch to release-openclaw-plugin-testing and Package Acceptance workflows instead of relying on source-level timing data when compiled artifacts impact performance
2.6 Coverage Retention Rules
Optimizations must never reduce test coverage:
- Do not delete slow integration tests unless equivalent production logic is extracted into named, independently validated helper functions
- Retain minimal lightweight integration smoke tests when cross-component connectivity is critical
- Explicitly document any incidental coverage lost during refactoring (if applicable)
2.7 Post-Fix Verification Checklist
Mandatory validation steps after completing performance refactors:
- Source code lint & static checks: pnpm check:changed
- Targeted regression tests: pnpm test:changed or granular test files matching modified logic
- Changes touching lazy loading, build artifacts, package boundaries, dynamic imports, or build outputs: Run full pnpm build
- Modifications to Plugin SDK, barrels, or runtime layers: Execute pnpm plugin-sdk:api:check or pnpm plugin-sdk:api:gen
- Plugin suite performance fixes: Validate at minimum one representative plugin batch + the modified gate test cases
- Missing/outdated dependencies: Re-run all workflows after pnpm install
3. Primary Use Cases
- Full test suite performance optimization – Diagnose and resolve bottlenecks when pnpm test exhibits slow execution or excessive RSS consumption
- Plugin suite performance troubleshooting – Systematically locate root causes for slow Plugin Inspector runs, batch extension tests, or packaged plugin validation workflows
- Memory leak investigation – Diagnose sustained RSS growth, worker OOM crashes, and suspected object retention via heap snapshots and the openclaw-test-heap-leaks workflow
- Import hotspot identification – Detect bloated barrel file and runtime module loading overhead using OPENCLAW_VITEST_IMPORT_DURATIONS=1
- CI performance regression prevention – Compare baseline metrics before PR merge to block performance degradation from entering the main branch
- Remote Crabbox/Testbox benchmarking – Run isolated, reproducible performance validation on maintainer workstations via Blacksmith Testbox
4. Standard End-to-End Workflow
1. Review relevant AGENTS.md documentation (agent import hotspots, plugin/channel lazy loading, gateway lifecycle, test helper specs)
2. Capture performance baselines via test:perf:groups or scoped /usr/bin/time -l measurement
3. Prioritize optimization targets by impact: High ROI > High leverage > Low risk > High risk
4. Resolve root causes (migrate static data to lightweight artifacts, apply dependency injection, reuse suite-level server instances, etc.)
5. Preserve test coverage integrity (avoid deleting slow integration validations)
6. Re-run identical baseline commands to quantify latency reduction and percentage performance gains
7. Compile full performance report: before/after metrics, generated artifacts, coverage notes, verification steps, next optimization targets
8. Stage and push only files modified in the current optimization pass upon stakeholder request5. Mandatory Final Report Format
All performance optimization deliverables must include these sections:
- Root cause analysis
- Affected suite/plugin scope
- List of modified source files
- Quantified before/after metrics: wall time, Vitest/import latency, CPU utilization, peak RSS
- Leak classification (if memory-related: genuine leak, retained module graph, or inconclusive)
- Test coverage retention summary
- Full verification command inventory
- Testbox ID or remote benchmark workflow URL (where applicable)
- Target commit hash and push deployment status
6. Core Guiding Principles
- Evidence-first: All optimizations require validated baseline data; no speculative tuning without measurable proof
- Fix root causes, not symptoms: Do not mask bottlenecks via runner parameter tweaks (e.g. --maxWorkers)
- Preserve coverage: Never remove slow integration tests without equivalent named helper test coverage
- Narrow-to-wide testing sequence: Validate single-file / single-plugin workloads before expanding scope
- Plugin-Inspector failures are product-level signals: JSON outputs must remain parseable; warnings and errors require categorization, not suppression
- Do not label RSS growth as a memory leak without supporting heap snapshot / retainer graph analysis
Summary
OpenClaw Test Performance is a systematic performance engineering skill built around a closed-loop workflow: baseline capture → root cause diagnosis → fundamental remediation → post-optimization re-benchmarking. It delivers tangible improvements to execution speed and memory efficiency for OpenClaw’s core test suites and plugin validation pipelines while maintaining full test coverage. Equipped with a built-in library of common performance anti-patterns, a complete metric collection framework, and standardized post-fix verification checklists, it serves as the core toolset for sustaining fast, stable, healthy OpenClaw test infrastructure.
