Fix ia-candidate marker HTML-comment wrapper loss (kotkan/claude-plugin-inference-arbitrage#23) #27

Merged
oleks merged 3 commits from fix/issue-23-marker-wrapper into main 2026-07-30 19:54:28 +03:00
5 changed files with 110 additions and 13 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "inference-arbitrage",
"version": "0.7.0",
"version": "0.7.1",
"description": "Audits a Claude Code plugin's definitions and usage transcripts to find steps that should be a deterministic script instead of raw LLM inference, and files the well-evidenced ones as issues on the target repo.",
"author": {
"name": "oleks"
+23 -4
View File
@@ -85,10 +85,29 @@ def marker(candidate_id):
def marker_re(candidate_id):
"""Tolerates whitespace drift inside the comment, nothing else. The id
itself must match exactly — a fuzzy id match would let two genuinely
different candidates collapse onto one issue."""
return re.compile(r"<!--\s*ia-candidate:\s*" + re.escape(candidate_id) + r"\s*-->")
"""Tolerates whitespace drift inside the comment, and tolerates the
HTML-comment wrapper (`<!-- -->`) being absent entirely, nothing else. The
id itself must match exactly — a fuzzy id match would let two genuinely
different candidates collapse onto one issue.
The bare form (no `<!-- -->`) is a backward-compatibility case, not the
contract: kotkan/claude-plugin-inference-arbitrage#23 found that issues
filed on 2026-07-29 (oleks/claude-plugin-cluster#56, #57) lost the wrapper
somewhere between `filing-plan`'s render and the Gitea write — the
template and this script always render the wrapped form (see `marker()`
below and references/issue-template.md). Matching the bare line here is
purely so a *later* audit still recognizes those already-filed issues and
doesn't duplicate-file them; it does not license filing new issues without
the wrapper.
"""
wrapped = r"<!--\s*ia-candidate:\s*" + re.escape(candidate_id) + r"\s*-->"
# End-of-line anchored (not \b): candidate_id contains "/" and "-", so a
# word-boundary check would treat "…label-derivation" as a match inside
# "…label-derivation-longer" (the "n"/"-" transition IS a \b). Anchoring
# to end-of-line (re.MULTILINE) instead requires an exact id, matching
# what marker() actually renders as the line's sole content.
bare = r"^ia-candidate:\s*" + re.escape(candidate_id) + r"[ \t]*$"
return re.compile(wrapped + "|" + bare, re.MULTILINE)
def find_existing(candidate_id, issues):
+20
View File
@@ -38,6 +38,26 @@ edited by hand on a filed issue. Removing it from an issue causes the next audit
to file a duplicate — which the spec names as the single most likely way for
this plugin to become hated.
**Known failure mode (kotkan/claude-plugin-inference-arbitrage#23).** Two
issues filed on 2026-07-29 (oleks/claude-plugin-cluster#56, #57) reached Gitea
with the `<!-- -->` wrapper stripped — the body opened with the bare text
`ia-candidate: <id>` instead. `bin/filing-plan` always *renders* the wrapped
form (verified: a direct `issue_write` probe with a literal `<!-- -->` in the
body round-trips through the Gitea MCP write path unchanged, so the API/MCP
layer is not the cause). The loss happens on the **relay hop**
`Agent(anxious:issuer-agent)` composing the actual Gitea call from the prompt
it's handed — because an HTML comment is, by design, invisible when the prompt
text is read as markdown; an agent relaying "what the issue should say" rather
than pasting the string byte-for-byte can drop a line it never visually saw.
That is why `skills/offload-audit/SKILL.md` §5d now fences the rendered body
in the delegation prompt instead of inlining it as bare markdown — a fenced
block reads as literal text, not as renderable markup, to the relaying agent.
`bin/filing-plan`'s `find_existing()` also now matches the bare, unwrapped
form as a backward-compatibility fallback, so issues already filed without the
wrapper (like cluster#56/#57) are still found — but that fallback exists only
to avoid duplicate-filing on *already-broken* issues, not as a second
acceptable format going forward.
## issue-title
A proposal. `issuer` may rewrite it.
+29 -8
View File
@@ -261,14 +261,31 @@ rendered. **Execute the plan verbatim — do not rewrite a body, and never turn
One `Agent` call per action (or one call carrying all of them):
```text
**The body MUST be embedded inside its own fenced code block in the prompt, not
inlined as bare markdown.** kotkan/claude-plugin-inference-arbitrage#23 found
that two issues filed this way (oleks/claude-plugin-cluster#56, #57) reached
Gitea with the leading `<!-- ia-candidate: ... -->` marker's HTML-comment
wrapper stripped. The write path itself was verified clean (a literal `<!--
-->` round-trips through the Gitea MCP write unchanged); the loss happens on
the relay hop, because an HTML comment is invisible when its surrounding text
is read as markdown — `issuer-agent` can only paste a line byte-for-byte if it
was actually shown one, not markdown that silently ate it. A fenced block
(` ``` `) forces the body to be read and relayed as literal text.
````text
Agent(subagent_type="anxious:issuer-agent", prompt=...)
# for action == "file"
"File this on <owner/repo>. It is an inference-arbitrage audit finding.
Proposed title: <proposed_title> (yours to rewrite per your taxonomy)
Body — file VERBATIM, the first line is an idempotency marker that a later
audit searches for and MUST NOT be altered or reformatted:
Body — copy the fenced block below byte-for-byte as the issue body. Do not
retype, paraphrase, reformat, or 'clean up' it — the first line is an
idempotency marker (an HTML comment, `<!-- ia-candidate: ... -->`) that a
later audit's marker search depends on, and it renders invisible on Gitea
by design, so you will not visually see it in the rendered issue — that is
expected, do not treat it as accidental syntax to strip:
```
<body>
```
Labels: 'token-offload' is required — create it on this repo if absent
(colour #8b5cf6, 'A step paying for inference where a deterministic script
would do'). Add the four-axis taxonomy labels and any milestone per your own
@@ -278,11 +295,15 @@ Agent(subagent_type="anxious:issuer-agent", prompt=...)
# for action == "comment"
"Add this comment to <issue>, an existing inference-arbitrage finding being
re-measured. Do not open a new issue. Post the body VERBATIM (the first line
is an idempotency marker). If the issue is closed, use your judgement about
reopening; the audit does not require it.
<body>"
```
re-measured. Do not open a new issue. Copy the fenced block below
byte-for-byte as the comment body — do not retype or reformat it, the first
line is an invisible-by-design HTML-comment idempotency marker (see the
'file' case above for why). If the issue is closed, use your judgement
about reopening; the audit does not require it.
```
<body>
```"
````
`issuer` owns repo, final title, labels beyond `token-offload`, and milestone.
You own the body.
+37
View File
@@ -112,6 +112,43 @@ check "found via the comment" "$(q "$tmp/plan3.json" "d['actions'][0]['action']"
check "closed state surfaced for issuer" \
"$(q "$tmp/plan3.json" "d['actions'][0]['issue_state']")" "closed"
echo "== kotkan/claude-plugin-inference-arbitrage#23: a marker that lost its"
echo " HTML-comment wrapper on write (oleks/claude-plugin-cluster#56, #57)"
echo " is still found — backward compat, not a second accepted format =="
python3 - "$tmp/plan1.json" "$tmp/bare.json" <<'PY'
import json, sys
plan = json.load(open(sys.argv[1]))
filed = next(a for a in plan["actions"] if a["action"] == "file")
# Same body, but the first line has lost its "<!-- " / " -->" wrapper — the
# exact shape observed on the live tracker.
bare_body = filed["body"].replace(
"<!-- ia-candidate: fixture-plugin/skill-a/label-derivation -->",
"ia-candidate: fixture-plugin/skill-a/label-derivation",
)
json.dump([{"number": 56, "state": "closed", "title": filed["proposed_title"],
"body": bare_body, "comments": []}], open(sys.argv[2], "w"))
PY
plan "$tmp/bare.json" "$tmp/plan-bare.json"
check "bare marker still found, no duplicate filed" \
"$(q "$tmp/plan-bare.json" "d['summary']['to_file']")" "0"
check "recognized as a comment on the existing issue" \
"$(q "$tmp/plan-bare.json" "d['actions'][0]['action']")" "comment"
check "on issue 56, matching the observed failure" \
"$(q "$tmp/plan-bare.json" "d['actions'][0]['issue_number']")" "56"
echo "== a bare marker for a DIFFERENT id still does not false-match =="
python3 - "$tmp/bare.json" "$tmp/bare-other.json" <<'PY'
import json, sys
issues = json.load(open(sys.argv[1]))
issues[0]["body"] = issues[0]["body"].replace(
"fixture-plugin/skill-a/label-derivation", "fixture-plugin/skill-a/label-derivation-longer",
)
json.dump(issues, open(sys.argv[2], "w"))
PY
plan "$tmp/bare-other.json" "$tmp/plan-bare-other.json"
check "different id (even as a prefix) still files fresh" \
"$(q "$tmp/plan-bare-other.json" "d['summary']['to_file']")" "1"
echo "== real Gitea API shapes parse, and a DIFFERENT id does not false-match =="
plan tests/fixtures/gitea-issues.json "$tmp/plan4.json"
check "unrelated markers -> files" "$(q "$tmp/plan4.json" "d['summary']['to_file']")" "1"