Module: S04 — CTF Harness Engineering · Duration: ~60 minutes
Format: Verbatim transcript. Cues [SLIDE N] map to 03-slide-deck.html.
[SLIDE 1 — Title]
This is Module S Zero Four: CTF Harness Engineering. Sixty minutes. The fourth and final module of Pillar One before we build the meta-harness in S Zero Five. Three sub-sections: CTF domain architecture, speed and parallelism and human-in-the-loop tradeoffs, and flag extraction and submission and scoring. By the end you can build a harness that solves CTF challenges at competition speed — and you will know exactly which safety constraints to keep and which to remove to get there.
[SLIDE 2 — The speed inversion]
Before anything else, set the frame. A CTF is not a smaller pentest. It is a different problem with a different optimization target. S Zero Two and S Zero Three optimized for thoroughness — full coverage, repeatable evidence, client-ready reports. CTF optimizes for speed to flag. A competition ranks by flag count, then by time. A harness that takes forty minutes to solve what a human solves in ten is not in the running, regardless of how clean its evidence chain is.
This speed target inverts several of the constraints we built in S Zero Two and S Zero Three. The persistent engagement memory that justifies a multi-day bug bounty engagement is overhead in a sixty-minute competition. The hash-chained evidence log that protects you legally against a client is unnecessary when the target is a container you spun up ten minutes ago. The model-judged triage that filters two hundred nuclei findings down to one reportable is not the loop when the deliverable is a thirty-two-character flag string.
But — and this is the whole content of S Zero Four point two — it does not invert all of them. The engineering discipline is knowing which inversions are safe and which are catastrophic. Remove the wrong gate and you have a harness that solves challenges quickly and then gets its operator escorted out of the competition venue for attacking infrastructure that was not part of the game. Pentest is thorough. CTF is fast. The rest of the module is the difference.
[SLIDE 3 — S04.1: CTF Domain Architecture]
Sub-section one. CTF domain architecture. CTF challenges partition into six recognized domains. Web, Pwn, Crypto, Forensics, Reversing, Misc. Each has a distinct reasoning pattern, a distinct tool stack, and a distinct context budget. The rightmost column on this table — context burn — is not decoration. It drives the engineering.
A Reversing challenge can saturate a context window with decompiler output in a handful of tool calls. The harness must aggressively summarize and retain only structural facts — function signatures, cross-references, suspicious comparisons. A Forensics challenge rarely saturates context; the bottleneck there is tool selection and artifact triage. Treating them the same is the first mistake.
Walk the reasoning patterns. Web is request manipulation — send a payload, inspect the response, escalate from error-based to blind. Pwn is memory layout reasoning — protections, offsets, gadget chains, syscall sequences. Crypto is mathematical weakness detection — find the oracle, find the bias, exploit it with math, not brute force. Forensics is artifact carving — triage the file, extract the interesting bits, search for the flag pattern. Reversing is disassembly reasoning — decompile, locate the comparison, invert the logic. And Misc is everything else — steganography, scripting puzzles, encoding chains, esoteric languages. Six different mental modes. A model instructed to be good at all six will be excellent at none of them.
[SLIDE 4 — Why a single harness fails]
A single undifferentiated harness loaded with all six tool stacks fails for three compounding reasons. First, context pollution. A Web challenge does not need ROPgadget in its tool manifest. Loading every tool into every session inflates the system prompt, dilutes the model's tool-selection signal, and consumes context the challenge itself needs. Empirically, models with broad tool manifests call the wrong tool more often than models with narrow manifests. The signal-to-noise on tool selection is measurable, and broader manifests degrade it.
Second, reasoning drift. The mental mode for Pwn — low-level memory reasoning, careful hypothesis testing — is the opposite of the mode for Web, which is broad fuzzing and rapid iteration. A model instructed to be good at both will be excellent at neither. Specialized system prompts per domain, written by domain experts, encoding the domain's actual heuristics — measurably outperform a generic prompt that tries to cover all six. The Web prompt encodes "test parameter by parameter, watch response codes, escalate from error-based to blind." The Pwn prompt encodes "run checksec first, look for a win function, then determine the offset to RIP." These are not generic security prompts.
Third, token waste. Each tool's schema and examples is paid every turn. Six domains of tools means six domains of overhead every turn, every challenge. The cure is the meta-agent-plus-specialists pattern: a small classifier routes, a specialized agent solves with a narrow manifest.
[SLIDE 5 — The domain router]
The harness's first job on a new challenge is to identify the domain and route. This is the meta-agent. It reads the challenge description and the attached artifacts, emits a domain label, and dispatches to the corresponding specialized agent with its own prompt and tool subset.
Note that the router uses both signals — text and file type. Text-only classification misroutes a Pwn challenge described as "analyze this binary" into Reversing. File-type signals correct that. An ELF binary with checksec showing NX disabled and a win symbol is Pwn, regardless of how the description is worded. A pcap file is Forensics. A URL is Web. The file-type signal is often the more reliable of the two, because challenge authors write descriptions to mislead and the artifacts do not lie.
And critically, the router is allowed to be uncertain. When the meta-agent cannot route with confidence, the challenge goes to a TriageAgent that runs cheap probes — file, checksec, strings frequency, HTTP response code — and re-routes with the additional signal. Two-stage routing catches the ambiguous cases. The router is allowed to be uncertain. The agents are not. Once a challenge lands in a specialist, that specialist is expected to commit to the domain's reasoning loop and execute it. A specialist that second-guesses its domain is a specialist that wastes turns.
This is CAI's architecture — the one worth studying. Specialized prompts authored per domain. Tool subsets loaded per domain, not globally. A router that is allowed to admit uncertainty and recover via cheap probes. The engineering decisions are not exotic. The discipline is in executing them: every specialist gets a narrow manifest, every prompt encodes the domain's actual heuristics, every router fallback is cheap and fast.
[SLIDE 6 — S04.2: Speed, Parallelism, HITL]
Sub-section two. Speed, parallelism, and the human-in-the-loop tradeoff. Four properties, in combination, produce benchmark-level speed. No approval gates — every tool call executes immediately, no round-trip through a human. In a sixty-minute competition window, an approval gate that costs thirty seconds per call across forty calls burns twenty of those minutes. The math is brutal and unforgiving.
Pre-loaded tool context — schemas, examples, and domain heuristics are in the system prompt at session start. The agent does not spend turns discovering what it can do. Narrow domain scope — the router has already restricted the agent to one domain's tools and one domain's reasoning mode. There is no decision paralysis across a broad tool surface. And deterministic tool routing — common workflows like checksec-before-reasoning, file-before-binwalk, parameter-fuzzing-before-sqlmap — are baked into the agent's prompt as ordered checklists, not left for the model to rediscover each session.
The first — no approval gates — is the most consequential and the most dangerous. It is where the speed comes from, and it is where the catastrophic failure lives if you get the safety reasoning wrong.
[SLIDE 7 — When is HITL removal safe?]
The criterion for safe HITL removal is three conditions, all of which must hold. The target is owned or sandboxed by the operator. The target is isolated from production and third-party systems at the network layer. And the worst-case action the agent can take is restart the target or waste competition time. When all three hold, approval gates add latency without adding safety. When any one fails, the gate stays.
Walk the table. Owned Docker container with a CTF binary — remove the gate. Owned, isolated, restartable. Competition-provided sandboxed VM — remove. Sandboxed by the organizer, isolated. Remote CTF service with rate limits — partial. The service is fair game, but credential reuse against other services is the unbounded risk, so that gate stays. Live bug bounty target, which is not CTF — keep the gate. S Zero Two rules apply. Production network adjacent to the sandbox — keep. Isolation is not verified, blast radius is unbounded.
[SLIDE 8 — The speed comparison]
Here is what the gate costs you. Same challenge, same three tool calls — checksec, ROPgadget, exploit payload. point to the left column With HITL enabled, you wait twenty-five seconds for human approval on checksec, thirty on ROPgadget, twenty-eight on the payload. Eighty-three seconds of pure wait. Plus eight seconds of execution. Roughly ninety-one seconds wall-clock.
point to the right column Autonomous mode. Same three calls. Eight seconds total. That eighty-three-second gap is the budget that decides whether your harness is competitive.
And the exception that proves the rule, at the bottom: the credential-reuse gate stays even in autonomous mode. Discovered credentials are never auto-reused against other services. That is the one unbounded-risk action in a CTF context. Remove the speed-killing gates. Keep the blast-radius gate.
[SLIDE 9 — Parallel solve attempts]
The second speed multiplier is parallelism. Two forms. Intra-challenge: multiple exploit hypotheses against the same challenge, concurrently. A Pwn challenge might present as either a stack overflow or a format string vulnerability. The meta-agent spawns two PwnAgents, one pursuing each hypothesis. The first to find a flag wins. The other session is killed. You trade compute for wall-clock time, which is the right trade in a timed competition. The engineering point is that the sessions are isolated — they do not share memory, they do not coordinate, they just race. Coordination overhead would defeat the purpose.
Inter-challenge: three to five challenges running concurrently, each in its own session with its own router dispatch, its own agent, its own flag extractor. The scoreboard moves in parallel. The constraint is compute and rate limits — most competitions rate-limit per team, and five concurrent sqlmap instances against one service will trip those limits. Three to five is the practical ceiling on most teams' hardware and most competitions' tolerance.
And the rule that is easy to violate by accident: shared memory across parallel attempts is forbidden. Two PwnAgents working the same challenge, sharing a memory store, would overwrite each other's intermediate state and reintroduce the context pollution that domain routing was designed to eliminate. Isolated sessions, isolated tool state, isolated flag sinks. Winner takes all on flag capture.
[SLIDE 10 — S04.3: Flag Extraction & Scoring]
Sub-section three. Flag extraction, submission, and scoring. A solved challenge produces a flag — a string in a known format, typically flag open-brace close-brace, CTF open-brace close-brace, or a competition-specific prefix. The harness's final job is to detect flags in tool output, deduplicate them, submit them to the scoreboard, and track which challenges are solved.
The key design decision: flag detection is middleware on every tool output. It is not a separate step the agent remembers to call. The agent does not "submit a flag." The agent runs tools, and the middleware notices when a tool's output contains a flag pattern. This is the S Zero Two point three evidence logger pattern, repurposed: every tool output is scanned, matches are extracted, the agent is notified that a flag was captured. If the agent has to remember to call the flag detector, it will not — on the critical turn where it matters most. Automatic, not optional.
The patterns are configurable per competition. Most competitions publish their flag format in the rules. Default patterns cover the common prefixes; competition-specific prefixes load from a rules file.
[SLIDE 11 — CTFd submission]
Most modern CTFs run on CTFd — open-source scoreboard, documented REST API. Submission is a single POST to the challenges attempt endpoint with the token, the challenge ID, and the flag. HackTheBox uses a similar pattern with a different endpoint and auth scheme; the harness abstracts both behind a ScoreboardClient interface so the agent and flag extractor do not care which platform they are talking to.
The response comes back in one of three states, and you must handle all three correctly. Correct — mark the challenge solved, kill any parallel attempts on that challenge, free the slot. Incorrect — log the attempt, let the agent continue solving. Already solved — treat as correct. Another team member or a parallel session got there first. Do not re-submit; that burns attempts and trips rate limits.
And rate-limit the submission client. Two to three seconds between submissions, per-challenge lockout after N wrong attempts. Most scoreboards throttle or temporarily ban teams that submit too fast. Self-inflicted throttling ends competitions. The submission client is the one place where the harness can sabotage itself, and it is the easiest place to add a guardrail that pays for itself the first time the verifier is wrong.
[SLIDE 12 — The scorecard]
With five challenges running concurrently across three to five sessions, the only way to know what is happening is the scorecard. It tracks, per challenge: state — unattempted, in progress, solved, abandoned. The sessions running against it. Attempt count. First blood timestamp. And globally: flags captured with source challenge and source session, flags submitted with CTFd response and attempt count.
This scorecard is the operator's view into a parallel autonomous harness. Without it you are watching terminals scroll. With it you have a structured view that answers the operational questions: how many solved, how many in progress, where are we spending time, where are we stuck, which domain is the router misrouting.
And it is measurable. Challenges per hour. Mean time to flag, per domain — Pwn will be slower than Web, that is expected, and the per-domain breakdown tells you whether your specialists are calibrated. Router accuracy — did the meta-agent pick the right domain on first dispatch, and how often did the TriageAgent recover an uncertain route. Flag-extraction precision — true flags versus decoys, which tells you whether your verifier is too loose or too tight. These metrics are the CTF equivalent of S Zero Two point four's precision and recall. They tell you whether the harness is competing or just burning compute. A harness that solves three challenges an hour with sixty percent router accuracy is not in the same league as one that solves eight an hour with ninety percent router accuracy. The scorecard is how you know which one you have.
[SLIDE 13 — Anti-patterns]
Four anti-patterns to close on. The single undifferentiated agent — one agent, all six domains of tools, attempting every challenge. Context pollution, reasoning drift, tool-selection errors. Cure: meta-agent plus specialists with narrow per-domain manifests.
HITL off against unowned targets — treating "CTF context" as a blanket license to remove approval gates, then pointing the harness at a remote service with unknown isolation. Cure: apply the three-criterion test — owned, isolated, bounded blast radius. Keep the gate when any criterion fails. This is the anti-pattern that gets people in legal trouble, because CTF feels safe and the boundary between a CTF sandbox and an adjacent production network is not always obvious.
Skipping checksec or file — a Pwn agent that starts building a ROP chain without reading protections, or a Forensics agent that runs binwalk without file. Cure: ordered checklists in the prompt, first step always the cheap probe that constrains the rest. Checksec is two seconds and tells you whether ROP is even viable. Skipping it is the most common Pwn failure mode in the field.
And auto-submitting every regex match — the extractor finds flag open-brace example close-brace in the challenge description and submits it. Burned attempt, rate limit tripped. Cure: verifier before submission, with an allowlist of flag sources. The flag extractor surfaces candidates; a small verifier model or the operator confirms plausibility before the submission client touches the scoreboard.
These four anti-patterns account for most of the failure modes you will see in a real CTF harness. Recognize them, and you have eliminated eighty percent of the field-engineering problems before they happen.
[SLIDE 14 — What you take into S05]
S Zero Five builds the meta-harness — the layer that orchestrates bug bounty, pentest, and CTF harnesses into a single system that selects the right harness for the engagement type. The CTF harness you built in this module becomes one of three orchestrated backends. The routing pattern from S Zero Four point one — meta-agent plus specialists — scales up to the meta-harness level. Same architecture, one layer up.
[End of script]
# Teaching Script — Module S04: CTF Harness Engineering **Module**: S04 — CTF Harness Engineering · **Duration**: ~60 minutes **Format**: Verbatim transcript. Cues `[SLIDE N]` map to `03-slide-deck.html`. --- [SLIDE 1 — Title] This is Module S Zero Four: CTF Harness Engineering. Sixty minutes. The fourth and final module of Pillar One before we build the meta-harness in S Zero Five. Three sub-sections: CTF domain architecture, speed and parallelism and human-in-the-loop tradeoffs, and flag extraction and submission and scoring. By the end you can build a harness that solves CTF challenges at competition speed — and you will know exactly which safety constraints to keep and which to remove to get there. [SLIDE 2 — The speed inversion] Before anything else, set the frame. A CTF is not a smaller pentest. It is a different problem with a different optimization target. S Zero Two and S Zero Three optimized for thoroughness — full coverage, repeatable evidence, client-ready reports. CTF optimizes for speed to flag. A competition ranks by flag count, then by time. A harness that takes forty minutes to solve what a human solves in ten is not in the running, regardless of how clean its evidence chain is. This speed target inverts several of the constraints we built in S Zero Two and S Zero Three. The persistent engagement memory that justifies a multi-day bug bounty engagement is overhead in a sixty-minute competition. The hash-chained evidence log that protects you legally against a client is unnecessary when the target is a container you spun up ten minutes ago. The model-judged triage that filters two hundred nuclei findings down to one reportable is not the loop when the deliverable is a thirty-two-character flag string. But — and this is the whole content of S Zero Four point two — it does not invert all of them. The engineering discipline is knowing which inversions are safe and which are catastrophic. Remove the wrong gate and you have a harness that solves challenges quickly and then gets its operator escorted out of the competition venue for attacking infrastructure that was not part of the game. Pentest is thorough. CTF is fast. The rest of the module is the difference. [SLIDE 3 — S04.1: CTF Domain Architecture] Sub-section one. CTF domain architecture. CTF challenges partition into six recognized domains. Web, Pwn, Crypto, Forensics, Reversing, Misc. Each has a distinct reasoning pattern, a distinct tool stack, and a distinct context budget. The rightmost column on this table — context burn — is not decoration. It drives the engineering. A Reversing challenge can saturate a context window with decompiler output in a handful of tool calls. The harness must aggressively summarize and retain only structural facts — function signatures, cross-references, suspicious comparisons. A Forensics challenge rarely saturates context; the bottleneck there is tool selection and artifact triage. Treating them the same is the first mistake. Walk the reasoning patterns. Web is request manipulation — send a payload, inspect the response, escalate from error-based to blind. Pwn is memory layout reasoning — protections, offsets, gadget chains, syscall sequences. Crypto is mathematical weakness detection — find the oracle, find the bias, exploit it with math, not brute force. Forensics is artifact carving — triage the file, extract the interesting bits, search for the flag pattern. Reversing is disassembly reasoning — decompile, locate the comparison, invert the logic. And Misc is everything else — steganography, scripting puzzles, encoding chains, esoteric languages. Six different mental modes. A model instructed to be good at all six will be excellent at none of them. [SLIDE 4 — Why a single harness fails] A single undifferentiated harness loaded with all six tool stacks fails for three compounding reasons. First, context pollution. A Web challenge does not need ROPgadget in its tool manifest. Loading every tool into every session inflates the system prompt, dilutes the model's tool-selection signal, and consumes context the challenge itself needs. Empirically, models with broad tool manifests call the wrong tool more often than models with narrow manifests. The signal-to-noise on tool selection is measurable, and broader manifests degrade it. Second, reasoning drift. The mental mode for Pwn — low-level memory reasoning, careful hypothesis testing — is the opposite of the mode for Web, which is broad fuzzing and rapid iteration. A model instructed to be good at both will be excellent at neither. Specialized system prompts per domain, written by domain experts, encoding the domain's actual heuristics — measurably outperform a generic prompt that tries to cover all six. The Web prompt encodes "test parameter by parameter, watch response codes, escalate from error-based to blind." The Pwn prompt encodes "run checksec first, look for a win function, then determine the offset to RIP." These are not generic security prompts. Third, token waste. Each tool's schema and examples is paid every turn. Six domains of tools means six domains of overhead every turn, every challenge. The cure is the meta-agent-plus-specialists pattern: a small classifier routes, a specialized agent solves with a narrow manifest. [SLIDE 5 — The domain router] The harness's first job on a new challenge is to identify the domain and route. This is the meta-agent. It reads the challenge description and the attached artifacts, emits a domain label, and dispatches to the corresponding specialized agent with its own prompt and tool subset. Note that the router uses both signals — text and file type. Text-only classification misroutes a Pwn challenge described as "analyze this binary" into Reversing. File-type signals correct that. An ELF binary with checksec showing NX disabled and a win symbol is Pwn, regardless of how the description is worded. A pcap file is Forensics. A URL is Web. The file-type signal is often the more reliable of the two, because challenge authors write descriptions to mislead and the artifacts do not lie. And critically, the router is allowed to be uncertain. When the meta-agent cannot route with confidence, the challenge goes to a TriageAgent that runs cheap probes — file, checksec, strings frequency, HTTP response code — and re-routes with the additional signal. Two-stage routing catches the ambiguous cases. The router is allowed to be uncertain. The agents are not. Once a challenge lands in a specialist, that specialist is expected to commit to the domain's reasoning loop and execute it. A specialist that second-guesses its domain is a specialist that wastes turns. This is CAI's architecture — the one worth studying. Specialized prompts authored per domain. Tool subsets loaded per domain, not globally. A router that is allowed to admit uncertainty and recover via cheap probes. The engineering decisions are not exotic. The discipline is in executing them: every specialist gets a narrow manifest, every prompt encodes the domain's actual heuristics, every router fallback is cheap and fast. [SLIDE 6 — S04.2: Speed, Parallelism, HITL] Sub-section two. Speed, parallelism, and the human-in-the-loop tradeoff. Four properties, in combination, produce benchmark-level speed. No approval gates — every tool call executes immediately, no round-trip through a human. In a sixty-minute competition window, an approval gate that costs thirty seconds per call across forty calls burns twenty of those minutes. The math is brutal and unforgiving. Pre-loaded tool context — schemas, examples, and domain heuristics are in the system prompt at session start. The agent does not spend turns discovering what it can do. Narrow domain scope — the router has already restricted the agent to one domain's tools and one domain's reasoning mode. There is no decision paralysis across a broad tool surface. And deterministic tool routing — common workflows like checksec-before-reasoning, file-before-binwalk, parameter-fuzzing-before-sqlmap — are baked into the agent's prompt as ordered checklists, not left for the model to rediscover each session. The first — no approval gates — is the most consequential and the most dangerous. It is where the speed comes from, and it is where the catastrophic failure lives if you get the safety reasoning wrong. [SLIDE 7 — When is HITL removal safe?] The criterion for safe HITL removal is three conditions, all of which must hold. The target is owned or sandboxed by the operator. The target is isolated from production and third-party systems at the network layer. And the worst-case action the agent can take is restart the target or waste competition time. When all three hold, approval gates add latency without adding safety. When any one fails, the gate stays. Walk the table. Owned Docker container with a CTF binary — remove the gate. Owned, isolated, restartable. Competition-provided sandboxed VM — remove. Sandboxed by the organizer, isolated. Remote CTF service with rate limits — partial. The service is fair game, but credential reuse against other services is the unbounded risk, so that gate stays. Live bug bounty target, which is not CTF — keep the gate. S Zero Two rules apply. Production network adjacent to the sandbox — keep. Isolation is not verified, blast radius is unbounded. [SLIDE 8 — The speed comparison] Here is what the gate costs you. Same challenge, same three tool calls — checksec, ROPgadget, exploit payload. *point to the left column* With HITL enabled, you wait twenty-five seconds for human approval on checksec, thirty on ROPgadget, twenty-eight on the payload. Eighty-three seconds of pure wait. Plus eight seconds of execution. Roughly ninety-one seconds wall-clock. *point to the right column* Autonomous mode. Same three calls. Eight seconds total. That eighty-three-second gap is the budget that decides whether your harness is competitive. And the exception that proves the rule, at the bottom: the credential-reuse gate stays even in autonomous mode. Discovered credentials are never auto-reused against other services. That is the one unbounded-risk action in a CTF context. Remove the speed-killing gates. Keep the blast-radius gate. [SLIDE 9 — Parallel solve attempts] The second speed multiplier is parallelism. Two forms. Intra-challenge: multiple exploit hypotheses against the same challenge, concurrently. A Pwn challenge might present as either a stack overflow or a format string vulnerability. The meta-agent spawns two PwnAgents, one pursuing each hypothesis. The first to find a flag wins. The other session is killed. You trade compute for wall-clock time, which is the right trade in a timed competition. The engineering point is that the sessions are isolated — they do not share memory, they do not coordinate, they just race. Coordination overhead would defeat the purpose. Inter-challenge: three to five challenges running concurrently, each in its own session with its own router dispatch, its own agent, its own flag extractor. The scoreboard moves in parallel. The constraint is compute and rate limits — most competitions rate-limit per team, and five concurrent sqlmap instances against one service will trip those limits. Three to five is the practical ceiling on most teams' hardware and most competitions' tolerance. And the rule that is easy to violate by accident: shared memory across parallel attempts is forbidden. Two PwnAgents working the same challenge, sharing a memory store, would overwrite each other's intermediate state and reintroduce the context pollution that domain routing was designed to eliminate. Isolated sessions, isolated tool state, isolated flag sinks. Winner takes all on flag capture. [SLIDE 10 — S04.3: Flag Extraction & Scoring] Sub-section three. Flag extraction, submission, and scoring. A solved challenge produces a flag — a string in a known format, typically flag open-brace close-brace, CTF open-brace close-brace, or a competition-specific prefix. The harness's final job is to detect flags in tool output, deduplicate them, submit them to the scoreboard, and track which challenges are solved. The key design decision: flag detection is middleware on every tool output. It is not a separate step the agent remembers to call. The agent does not "submit a flag." The agent runs tools, and the middleware notices when a tool's output contains a flag pattern. This is the S Zero Two point three evidence logger pattern, repurposed: every tool output is scanned, matches are extracted, the agent is notified that a flag was captured. If the agent has to remember to call the flag detector, it will not — on the critical turn where it matters most. Automatic, not optional. The patterns are configurable per competition. Most competitions publish their flag format in the rules. Default patterns cover the common prefixes; competition-specific prefixes load from a rules file. [SLIDE 11 — CTFd submission] Most modern CTFs run on CTFd — open-source scoreboard, documented REST API. Submission is a single POST to the challenges attempt endpoint with the token, the challenge ID, and the flag. HackTheBox uses a similar pattern with a different endpoint and auth scheme; the harness abstracts both behind a ScoreboardClient interface so the agent and flag extractor do not care which platform they are talking to. The response comes back in one of three states, and you must handle all three correctly. Correct — mark the challenge solved, kill any parallel attempts on that challenge, free the slot. Incorrect — log the attempt, let the agent continue solving. Already solved — treat as correct. Another team member or a parallel session got there first. Do not re-submit; that burns attempts and trips rate limits. And rate-limit the submission client. Two to three seconds between submissions, per-challenge lockout after N wrong attempts. Most scoreboards throttle or temporarily ban teams that submit too fast. Self-inflicted throttling ends competitions. The submission client is the one place where the harness can sabotage itself, and it is the easiest place to add a guardrail that pays for itself the first time the verifier is wrong. [SLIDE 12 — The scorecard] With five challenges running concurrently across three to five sessions, the only way to know what is happening is the scorecard. It tracks, per challenge: state — unattempted, in progress, solved, abandoned. The sessions running against it. Attempt count. First blood timestamp. And globally: flags captured with source challenge and source session, flags submitted with CTFd response and attempt count. This scorecard is the operator's view into a parallel autonomous harness. Without it you are watching terminals scroll. With it you have a structured view that answers the operational questions: how many solved, how many in progress, where are we spending time, where are we stuck, which domain is the router misrouting. And it is measurable. Challenges per hour. Mean time to flag, per domain — Pwn will be slower than Web, that is expected, and the per-domain breakdown tells you whether your specialists are calibrated. Router accuracy — did the meta-agent pick the right domain on first dispatch, and how often did the TriageAgent recover an uncertain route. Flag-extraction precision — true flags versus decoys, which tells you whether your verifier is too loose or too tight. These metrics are the CTF equivalent of S Zero Two point four's precision and recall. They tell you whether the harness is competing or just burning compute. A harness that solves three challenges an hour with sixty percent router accuracy is not in the same league as one that solves eight an hour with ninety percent router accuracy. The scorecard is how you know which one you have. [SLIDE 13 — Anti-patterns] Four anti-patterns to close on. The single undifferentiated agent — one agent, all six domains of tools, attempting every challenge. Context pollution, reasoning drift, tool-selection errors. Cure: meta-agent plus specialists with narrow per-domain manifests. HITL off against unowned targets — treating "CTF context" as a blanket license to remove approval gates, then pointing the harness at a remote service with unknown isolation. Cure: apply the three-criterion test — owned, isolated, bounded blast radius. Keep the gate when any criterion fails. This is the anti-pattern that gets people in legal trouble, because CTF feels safe and the boundary between a CTF sandbox and an adjacent production network is not always obvious. Skipping checksec or file — a Pwn agent that starts building a ROP chain without reading protections, or a Forensics agent that runs binwalk without file. Cure: ordered checklists in the prompt, first step always the cheap probe that constrains the rest. Checksec is two seconds and tells you whether ROP is even viable. Skipping it is the most common Pwn failure mode in the field. And auto-submitting every regex match — the extractor finds flag open-brace example close-brace in the challenge description and submits it. Burned attempt, rate limit tripped. Cure: verifier before submission, with an allowlist of flag sources. The flag extractor surfaces candidates; a small verifier model or the operator confirms plausibility before the submission client touches the scoreboard. These four anti-patterns account for most of the failure modes you will see in a real CTF harness. Recognize them, and you have eliminated eighty percent of the field-engineering problems before they happen. [SLIDE 14 — What you take into S05] S Zero Five builds the meta-harness — the layer that orchestrates bug bounty, pentest, and CTF harnesses into a single system that selects the right harness for the engagement type. The CTF harness you built in this module becomes one of three orchestrated backends. The routing pattern from S Zero Four point one — meta-agent plus specialists — scales up to the meta-harness level. Same architecture, one layer up. --- [End of script]