Overview
Build an ongoing monitoring and testing program for SOC 2 audit readiness. Creates a control testing plan with method, frequency, and tester assignments; an evidence collection matrix mapping controls to TSC criteria; and an exception tracker with root cause analysis and remediation. Validates testing coverage, evidence strength, and exception management.
SOC 2 Type II auditors test whether controls operated effectively throughout the audit period — not just at a point in time. This requires a testing program with defined methods (inquiry, observation, inspection, reperformance), sampling methodology, evidence collection, and exception management. This skill builds all three components and validates them with deterministic tools.
Getting Started
Add your source materials to the project workspace — organization profile, controls inventory, risk register, and any existing testing results — then activate the SOC 2 Monitoring & Testing skill. The agent builds the testing plan, evidence matrix, and exception tracker, validating each with built-in tools.
Example Conversation
Below is an example of how a typical monitoring and testing session unfolds. The agent builds a testing plan, validates coverage, checks evidence strength, and tracks exceptions.
You: Build the control testing plan for our Type II audit period (July 2025 – June 2026). We have 22 controls across Security (CC) and Availability (A).
Agent: I’ve loaded the controls inventory and organization profile. Building the testing plan for 22 controls across 36 in-scope TSC criteria. Each control gets a testing method, frequency, tester assignment, and expected evidence.
For Type II, most controls need quarterly testing with period-spanning evidence. Critical controls (CC7.1–CC7.2 system operations, A1.1 availability monitoring) need continuous monitoring evidence.
[Runs test_plan_validator] Results:
- TSC criteria coverage: 36/36 criteria have at least one tested control ✓
- Test methods specified: 22/22 controls ✓
- One flag: CTRL-AC-003 (access reviews) is tested only by inquiry. Inquiry alone is insufficient — upgrading to inspection + reperformance.
- Tester assignments: 22/22 controls have assigned testers ✓
- Type II period coverage: All tests span the full audit period ✓
You: Good. Now check our evidence status — we’ve been collecting evidence since July.
Agent: [Runs evidence_coverage_checker] Results:
- 19/22 controls have evidence collected (86%)
- 3 controls pending: CTRL-DR-001 (DR test — Q3 was skipped), CTRL-VM-002 (vendor reassessment — due Q4), CTRL-CM-003 (infrastructure change log — collection in progress)
- Evidence strength: 60% inspection/reperformance (strongest), 30% observation, 10% inquiry (always paired)
- No stale evidence detected (all within 12 months) ✓
- Warning: CTRL-SEC-001 relies solely on policy document as evidence — recommend adding Okta configuration export
You: We had two exceptions this period. The Q2 access review was 5 days late, and the Q3 DR test was skipped.
Agent: [Runs exception_tracker] I’ve logged both:
- EXC-001 (Minor): Q2 access review completed 5 days late. Root cause: reviewer on PTO without backup. Remediation: backup reviewer assigned for all future reviews. Owner: Sarah Chen. Target: immediate.
- EXC-002 (Moderate): Q3 DR test not performed. Root cause: infrastructure team transition — 2 SREs left in August. Remediation: DR test scheduled for November, documented procedure for future runs. Owner: Michael Torres. Target: 30 Nov 2025.
Both exceptions have complete remediation plans. No overdue remediations. Exception rate: 2/22 controls (9%) — within acceptable range for Type II.
All three documents saved. Dashboard updated: 22 controls tested, 86% evidence coverage, 2 open exceptions, 0 remediated.
Sample Output Excerpt
Below is a representative excerpt from the control testing plan and evidence collection matrix.
Control Testing Plan (excerpt)
| Control ID | Control | TSC Criteria | Method | Frequency | Tester | Expected Evidence |
|---|---|---|---|---|---|---|
| CTRL-AC-001 | MFA enforcement via Okta | CC6.1, CC6.2 | Inspection + Reperformance | Quarterly | Lisa Park | Okta MFA config export, sample login logs |
| CTRL-AC-003 | Quarterly access review | CC6.3 | Inspection + Reperformance | Quarterly | External (QA team) | Access review completion report, revocation tickets |
| CTRL-SO-001 | 24/7 system monitoring | CC7.1, CC7.2 | Inspection | Continuous | Lisa Park | Datadog dashboard screenshots, alert history |
| CTRL-DR-001 | Annual DR test | A1.3 | Reperformance | Annual | Michael Torres | DR test report, recovery time logs |
| CTRL-CM-001 | PR review + CI gates | CC8.1 | Inspection | Quarterly | Sarah Chen | GitHub PR merge history, CI pipeline logs |
Evidence Collection Matrix (excerpt)
| Control ID | Evidence Type | Collection Schedule | Storage | Status |
|---|---|---|---|---|
| CTRL-AC-001 | Okta MFA configuration export | Quarterly | evidence/access/ | Collected (Q1, Q2) |
| CTRL-AC-001 | Login audit logs (sample) | Quarterly | evidence/access/ | Collected (Q1, Q2) |
| CTRL-SO-001 | Datadog alert history export | Monthly | evidence/monitoring/ | Collected (Jul–Dec) |
| CTRL-DR-001 | DR test execution report | Annual | evidence/continuity/ | Pending (Q3 missed) |
| CTRL-CM-001 | GitHub PR merge audit | Quarterly | evidence/change-mgmt/ | Collected (Q1, Q2) |
Built-in Validation
The skill includes three validation tools that check testing coverage, evidence strength, and exception management. The agent runs all three after completing the testing plan and iterates until all checks pass.
What the validators check
Test Plan Validator:
| Check Category | What It Validates | Severity |
|---|---|---|
| TSC criteria coverage | Every in-scope criterion has at least one tested control | ERROR per uncovered criterion |
| Test method specified | Each test defines method (inquiry/observation/inspection/reperformance), frequency, tester | ERROR if missing |
| Period coverage (Type II) | Tests cover the entire audit period, not just point-in-time | WARNING if insufficient |
| Inquiry-only tests | Controls tested only by inquiry (insufficient alone per AICPA guidance) | WARNING per control |
| Tester assignment | Each control has an assigned tester independent of control operation | WARNING if missing |
Evidence Coverage Checker:
| Check Category | What It Validates | Severity |
|---|---|---|
| Evidence exists | Each implemented control has at least one evidence artifact planned or collected | ERROR if missing |
| Evidence strength | Ranks by strength: automated_report > system_log > configuration > ticket > screenshot > policy | WARNING if relying solely on policy |
| Category coverage | Coverage percentage per TSC category | INFO |
| Stale evidence | Evidence older than 12 months flagged as potentially stale for Type II | WARNING per item |
Exception Tracker:
| Check Category | What It Validates | Severity |
|---|---|---|
| Complete record | Each exception has: control_id, description, root_cause, severity, remediation_plan, owner, target_date | ERROR if incomplete |
| Remediation plans | All exceptions have remediation plans with owners and dates | ERROR if missing |
| Overdue remediations | Target date in the past with status not “resolved” | WARNING per overdue item |
| Severity classification | Minor (isolated), Moderate (pattern), Significant (systematic) — consistent with impact | INFO |
| Management acknowledgment | Significant exceptions without management acknowledgment | WARNING |
Example validation output
========================================================================
SOC 2 Monitoring & Testing — Test Plan Validation
========================================================================
Controls: 22
In-scope TSC criteria: 36
[PASS] All 36 criteria have at least one tested control
[PASS] All 22 controls have method, frequency, and tester
[PASS] All tests cover the full Type II audit period
[WARNING] CTRL-AC-003 tested only by inquiry — upgrade to
inspection + reperformance
Result: PASS (1 warning)
========================================================================
========================================================================
SOC 2 Monitoring & Testing — Evidence Coverage
========================================================================
Controls with evidence: 19/22 (86%)
Pending:
CTRL-DR-001: DR test report — Q3 missed, scheduled Q4
CTRL-VM-002: Vendor reassessment — due Q4
CTRL-CM-003: Infrastructure change log — collection in progress
Evidence strength:
Reperformance/Inspection: 60%
Observation: 30%
Inquiry (paired): 10%
[WARNING] CTRL-SEC-001: Relies solely on policy document.
Recommend adding system configuration export.
Result: 86% coverage (3 controls pending)
========================================================================
The agent runs all three validators automatically and surfaces findings inline, so you always know the testing status and evidence gaps before audit fieldwork begins.