VerifierResearchresearchacademic writingjournalism
Claim Verifier
Review a claim and its citations without reducing evidence to a truth badge: source identity and status, exact claim support, citation coverage, and consistency with the wider record.
Select a sentence in any doc and choose Is it true?, or pick it in the orange tab on the right to check every fact in the doc. Each fact gets a short answer beside it, with the sources in its thread. It won't change your doc unless you ask.
What’s inside
- Skills
skills/claim-verification/SKILL.mdclaim-verification · 7.3 KBUse when the user asks to verify, fact-check, validate, audit, or review a claim or citation. Checks bibliographic identity and status, exact claim-to-source support, citation coverage, and consistency with workspace and external evidence while making uncertainty explicit.
--- name: claim-verification description: > Use when the user asks to verify, fact-check, validate, audit, or review a claim or citation. Checks bibliographic identity and status, exact claim-to-source support, citation coverage, and consistency with workspace and external evidence while making uncertainty explicit. --- # Claim Verifier Verification is an evidence review, not a truth badge. Keep these questions separate throughout the investigation: 1. **Reference integrity:** Does the cited object exist, and do its author, title, date, DOI, version, and publication status match the document? 2. **Claim support:** Does an exact passage in that source support this exact claim, with the same population, conditions, magnitude, and certainty? 3. **Citation coverage:** Are checkable statements cited, and are citations placed on the claims they support rather than merely nearby? 4. **External consistency:** What do relevant primary or authoritative sources say beyond the cited source? ## Workflow A **one-click Verify** (the card beside a sentence) skips this workflow: see §5. What follows is the full evidence review, for when the reader asks for it. ### 1. Atomize and scope - Break the selection into independently checkable claims. Separate facts from forecasts, opinions, definitions, and recommendations. - Read enough surrounding workspace context to resolve pronouns, qualifiers, citation markers, dates, and units. Do not silently broaden the claim. - State what kind of check is possible. A private, inaccessible, or uncited source limits the result; it does not make the claim false. ### 2. Inspect workspace evidence first - Find the cited entry and any attached source using `Glob`, `Grep`, and `Read`. Check in-text marker ↔ bibliography consistency before searching. - For quotations, confirm the wording and locator. For paraphrases, compare meaning rather than keyword overlap. - Record the shortest decisive evidence passage, page/section when available, source version, and access date. Read the original source; snippets and search-result summaries are discovery aids only. ### 3. Check source identity and status - For Markdown or LaTeX citation structure, optionally run `python3 <this-skill-folder>/scripts/audit_citations.py --document '<path>' --format json`, resolving `<this-skill-folder>` from the `SKILL.md` path you read. The script is offline: it checks citation keys, bibliography entries, duplicate identifiers, DOI strings, and URLs, but never retrieves evidence or decides whether a source supports the claim. For Markdown, `--markdown-spans multiline` (the default) permits inline code spans across nonblank lines, as in Pandoc manuscripts. Select `--markdown-spans line` for the line-local span interpretation used when reviewing text in Sundial. The report records this choice; the offline scanner covers a Markdown subset, and neither option guarantees full renderer equivalence. LaTeX ignores this option. - For cross-file self-consistency, optionally run `python3 <this-skill-folder>/scripts/audit_consistency.py --path '<project>' --format json`. The script is offline and reports CANDIDATES only: places where a quantity, an acronym expansion, or a LaTeX macro is asserted with more than one value across the workspace. Two different numbers may be two legitimate measurements; investigate before treating any candidate as a contradiction. - Use Crossref first for publication metadata, publisher-deposited updates, and Retraction Watch coverage; DataCite for datasets/software and other DataCite objects; OpenAlex for discovery/citation graph context; Unpaywall only to locate a lawful open copy. - Registry metadata may be incomplete. A clean lookup means no issue was found in the records checked, not that the source is valid in every respect. - Read `references/providers.md` when choosing or interpreting an external provider, especially optional Scite, Elicit, Consensus, or Fact Check lookup. ### 4. Test exact support - Compare each atomic claim to exact source passages. Check entity/population, intervention or condition, outcome, time period, direction, magnitude, and uncertainty. Watch for causal language derived from correlational evidence. - Prefer primary sources for factual findings and original attribution; authoritative official sources for laws, statistics, and current policy; reviews for context, not as automatic replacements for originals. - Search externally with the agent's web search/fetch tools when workspace evidence is absent, ambiguous, stale, or contested. Use multiple independent search formulations and follow results to primary sources. Do not use Bash as a substitute for web access. If recency matters, verify dates and status. - Google Fact Check results only show that a publisher previously reviewed a similar public claim. No result means “no prior review found,” never “true.” ### 5. Report, then stop Begin with this exact two-part contract so the app can show one small inline result while retaining the full review in its durable thread: ```markdown ## Claim Verifier result Contradicted — <one short reason> ## Details <complete evidence review> ``` The result line must be exactly one line, keep its reason to at most 12 words, and begin with either `Contradicted —` or `Not contradicted —`. Use **Contradicted** only when reliable checked evidence directly conflicts with at least one material checkable claim in the selected passage. Otherwise use **Not contradicted**, including when the evidence is incomplete or inaccessible; state that limitation in the short reason. “Not contradicted” does not mean supported or verified true. Do not use any other overall verdict in the result line. **A one-click Verify** is quick, because a card is waiting beside the text. Write the verdict first. Do at most 2 web searches. Read a workspace file only when the passage cites one: the message already gives the paragraph around it and says whether the document has a bibliography. Keep `## Details` to about 120 words: the reason in a sentence or two, then 1 or 2 linked sources with the key quote. Stop there. **A full evidence review** (the reader asks for it in chat) follows the workflow above. Under `## Details`, use one finding per atomic claim: - **Supported** — the checked evidence directly supports the material claim. - **Partially supported** — a narrower or qualified form is supported. - **Contradicted** — reliable checked evidence directly conflicts with it. - **Insufficient evidence** — available evidence cannot resolve it. - **Source inaccessible** — the necessary source or passage could not be read. - **Needs human review** — interpretation, domain expertise, or conflicting high-quality evidence prevents a responsible automated conclusion. For each finding include: - the atomic claim and verdict; - reference integrity/status separately from support; - a brief exact evidence excerpt or precise paraphrase with link and locator; - reasoning about material scope differences; - sources/providers checked, access date, and unresolved limitations. Finish with a compact citation-consistency checklist for the selected passage. Say “No issue found in the sources checked” when appropriate, never “verified true.” Do not modify the document unless the user explicitly asks for edits.- Starter files
skills/claim-verification/references/providers.md2.7 KB
# Evidence provider guide No provider covers every verification layer. Use the smallest relevant set and retain the exact provider response or source passage in the audit trail. ## Baseline public services | Service | Use it for | Do not infer | | --- | --- | --- | | [Crossref](https://www.crossref.org/documentation/retrieve-metadata/rest-api/) | DOI identity, publisher-deposited metadata, relations and updates | Full-text support or source quality | | [Retraction Watch via Crossref](https://www.crossref.org/documentation/retrieve-metadata/retraction-watch/) and [Crossmark](https://www.crossref.org/services/crossmark/) | Retractions and participating publishers' current-version notices | Complete correction/expression-of-concern coverage | | [DataCite](https://support.datacite.org/docs/rest-api) | DOI metadata for datasets, software, and non-Crossref objects | Peer review or claim support | | [OpenAlex](https://developers.openalex.org/) | Work discovery, citation graph context, and some open full text | A definitive publication-status verdict | | [Unpaywall](https://unpaywall.org/products/api) | A lawful open-access location and visible manuscript version | Quality, correctness, or support | | [Google Fact Check Tools](https://developers.google.com/fact-check/tools/api/reference/rest/v1alpha1/claims/search) | Prior reviews of similar public claims | Truth from a match, or truth from no match | Use the agent's web search/fetch tools to open registry records, publisher pages, full text, and primary evidence. Do not use Bash, Python HTTP, `curl`, or `wget` as a substitute for unavailable web tools. The bundled `scripts/audit_citations.py` is intentionally offline and structural only. ## Optional connected services - **Scite:** later-paper citation contexts and supporting/contrasting/mentioning signals. These are useful leads, not proof that the user's cited passage supports their sentence. - **Elicit:** evidence extraction with an exact supporting quote. Re-read the original and preserve the quote/locator; its AI output remains reviewable. - **Consensus:** literature discovery and evidence landscapes. Use it to find studies, then verify the original sources. - **Zotero:** library identity, linked citations, locators, bibliography refresh, and retraction warnings. It does not judge claim support. Only use an optional service when its connector or credentials are already available in the workspace. Never request secrets in chat or invent access. ## Retrieval order 1. Attached workspace source and cited passage. 2. DOI registry and publisher/current-version page. 3. Lawful full text or authoritative primary source. 4. Independent primary/context sources. 5. Prior fact checks or citation-network signals as supplemental context.skills/claim-verification/scripts/audit_citations.py52 KB
#!/usr/bin/env python3 """Offline Markdown/LaTeX citation consistency audit; no network access.""" from __future__ import annotations import argparse import json import re import sys from pathlib import Path from typing import Any CITE_RE = re.compile( r"\\(?:cite|citep|citet|citealp|citeauthor|citeyear|autocite|parencite|textcite|footcite)\w*" r"\s*(?:\[[^\]]*\]\s*){0,2}\{([^}]+)\}" ) PANDOC_CITE_RE = re.compile(r"(?<![\w@])@([A-Za-z0-9_:.+/-]+)") DOI_RE = re.compile(r"(?i)\b10\.\d{4,9}/[-._;()/:A-Z0-9]+") URL_RE = re.compile(r"https?://[^\s<>\]\[{}\"']+") LATEX_SUFFIXES = {".tex", ".ltx", ".sty", ".cls"} # The masking below does not delegate to lib/markdown/parser.mjs, the repo's one # markdown grammar: these files ship as a standalone skill INTO a user workspace, # where a plain `python3` is all that runs and the repo's lib/ is absent, and what # they audit is Pandoc -- `~~~` fences, simple tables and `@key` citations, none of # which the editor parser models (its MD_CODE_FENCE is backtick-only, so deriving # regions from it would leave every `~~~` sample scannable and report the keys in # it as missing). Where the dialects do overlap the readings here mirror the # canonical ones, each named at the rule it follows. # Markdown code, which the scanners must not read: a `@key`-shaped token in a # sample is code, not a citation, and reporting it as missing fails `--strict` # over something the bibliography was never meant to carry. Both delimiters are # variable-length, so neither can be hardcoded at three backticks or one: # `~~~python` is as standard a fence as ```python, and an inline span opens on # a run of N backticks (``@example``) and closes on the next run of N. DELIM_RUN_RE = re.compile(r"(?P<char>[`~])(?P=char)*") # A quote prefix changes the content column, not the paragraph boundary. # Unmarked indented lines may still be lazy Pandoc paragraph continuations. BLOCKQUOTE_MARKER = r"(?:>[ \t]?)+" BLOCKQUOTE_MARKER_RE = re.compile(BLOCKQUOTE_MARKER) QUOTED_LINE_RE = re.compile(r" {0,3}" + BLOCKQUOTE_MARKER) # A line that is nothing but a run of fence characters, the only thing that can # close a fenced block. The three-space limit CommonMark puts on fence # indentation is load-bearing on the opening side: a `~~~` indented four spaces # is an indented code line rather than a document-level fence, and reading it as # one would blank the whole rest of the document and lose every citation below # it. What such a line does delimit is bounded by NESTED_FENCE_LINE_RE instead. FENCE_LINE_RE = re.compile(r"(?m)^ {0,3}(?P<fence>(?P<char>[`~])(?P=char){2,})[ \t]*$") # Pandoc also accepts an unmarked closer on a lazy line of the quote. Keep # this search inside quote_limit so blank-separated prose or an admitted outer # fence cannot be consumed by pairing unrelated delimiter lines. QUOTED_FENCE_LINE_RE = re.compile( rf"(?m)^(?: {{0,3}}{BLOCKQUOTE_MARKER} {{0,3}}|[ \t]*)(?P<fence>(?P<char>[`~])(?P=char){{2,}})[ \t]*$" ) # The closer for a fence opened at four or more columns -- inside a list item, # where the container's indentation sits in front of both delimiters, so the # closer is indented past what FENCE_LINE_RE accepts. The search for it is # bounded by `nested_block_end`, which is what keeps the looser indent from # running away with the document when nothing closes the fence. NESTED_FENCE_LINE_RE = re.compile(r"(?m)^[ \t]*(?P<fence>(?P<char>[`~])(?P=char){2,})[ \t]*$") # The rest of the line after a backtick fence opener, which CommonMark forbids # from carrying another backtick -- the same rule the canonical MD_CODE_FENCE # in lib/markdown/parser.mjs encodes as `^(`{3,})([^`]*)$`. BACKTICK_INFO_RE = re.compile(r"[^`\n]*(?:\n|\Z)") # Pandoc accepts a bare language (no backticks/braces/space), optional braced # attributes, or both. Quoted attribute values may contain spaces/backticks. ATTRIBUTE_ID = r"[^\W\d_][\w:.-]*" ATTRIBUTE_VALUE = r'''(?:"(?:\\.|[^"\\\n])*"|'(?:\\.|[^'\\\n])*'|(?:\\.|[^\s}])*)''' ATTRIBUTE = rf"(?:\#[\w:.-]+|\.{ATTRIBUTE_ID}|{ATTRIBUTE_ID}={ATTRIBUTE_VALUE}|-)" TILDE_INFO_RE = re.compile( rf"[ \t]*(?:\{{[ \t]*=[\w-]+[ \t]*\}}|" rf"(?:[^\s`{{}}]+[ \t]*)?(?:\{{[ \t]*(?:{ATTRIBUTE}[ \t]*)*\}})?)[ \t]*(?:\n|\Z)" ) # A thematic break may use solid or spaced delimiters (`***`, `- - -`). # It can precede an indented block but is never a list container. THEMATIC_BREAK = r"(?P<rule>[-_*])(?:[ \t]*(?P=rule)){2,}[ \t]*$" THEMATIC_BREAK_RE = re.compile(THEMATIC_BREAK) # A line that ends whatever block came before it, so a four-column-indented line # below it starts one of its own: an ATX heading, a list marker whose item holds # the indented content, or a thematic break. Setext underlines require preceding # heading text; fence boundaries come from completed code blocks instead. BLOCK_BREAK_RE = re.compile( r"(?:#{1,6}|[-+*]|\d{1,9}[.)])(?:[ \t]|$)|" + THEMATIC_BREAK ) ATX_HEADING_RE = re.compile(r"#{1,6}(?:[ \t]|$)") SETEXT_RULE_RE = re.compile(r"={3,}[ \t]*$") # A line whose content is a list marker: the container that puts its own # indentation in front of every block the item holds, so four columns there are # the item's content rather than code. `MD_BULLET` / `MD_ORDERED` in # lib/markdown/parser.mjs, plus CommonMark's `1)` form, and without their # empty-marker rule, which only decides whether a marker interrupts a paragraph. # A spaced thematic break (`- - -`) has this shape but opens no container: a # rule ends a list rather than starting one, which is why `parseListAt` tests # MD_HR first -- and so does `list_container`. LIST_MARKER_RE = re.compile(r"(?:[-+*]|\d{1,9}[.)])(?:[ \t]|$)") ORDERED_MARKER_RE = re.compile(r"\d{1,9}[.)](?:[ \t]|$)") TICK_RUN_RE = re.compile(r"`+") BLANK_LINE_RE = re.compile(r"\n[ \t]*\n") VERBATIM_RE = re.compile(r"\\begin\{(verbatim\*?|Verbatim\*?|lstlisting)\}") VERB_RE = re.compile(r"\\verb\*?([^A-Za-z\s])") LITERAL_ARG_RE = re.compile(r"\\(?:url|nolinkurl|path|href)\s*(?=\{)") def blank(text: str) -> str: """Replace every character but newlines with a space, so a region can be hidden from the scanners while every later offset keeps its line number.""" return re.sub(r"[^\n]", " ", text) def literal_region(text: str, index: int) -> tuple[int, bool] | None: """`(end, hide)` for a region LaTeX reads literally, in which a `%` is data and not the start of a comment; `None` when `index` opens no such region, or when it never closes -- which LaTeX itself rejects, so the safe reading is to keep scanning rather than swallow the rest of the file. `hide` marks code that is shown rather than written, the counterpart of a Markdown code span; a `\\url` argument stays visible because the URL it holds is part of the evidence this audit reports.""" if match := VERBATIM_RE.match(text, index): closer = f"\\end{{{match.group(1)}}}" end = text.find(closer, match.end()) return (end + len(closer), True) if end >= 0 else None if match := VERB_RE.match(text, index): end = text.find(match.group(1), match.end()) newline = text.find("\n", match.end()) if end < 0 or 0 <= newline < end: # `\verb` never spans a line return None return end + 1, True if match := LITERAL_ARG_RE.match(text, index): depth, cursor = 0, match.end() while cursor < len(text): if text[cursor] == "{": depth += 1 elif text[cursor] == "}": depth -= 1 if not depth: return cursor + 1, False cursor += 1 return None def strip_latex_comments(text: str) -> str: """Blank out `%` comments. A commented-out `\\cite` is not a citation, and reporting one sends the reviewer hunting for a key the paper never uses. Verbatim and `\\url`-style arguments are stepped over whole: a `%` there is literal, and blanking from it to the end of the line would drop the real citations sharing that line.""" out = list(text) index = 0 while index < len(text): if text[index] == "\\": region = literal_region(text, index) if region is None: index += 2 # an escaped char (\% or the \\ linebreak) continue end, hide = region if hide: out[index:end] = list(blank(text[index:end])) index = end elif text[index] == "%": while index < len(text) and text[index] != "\n": out[index] = " " index += 1 else: index += 1 return "".join(out) def fence_end(text: str, opener: int, char: str, size: int, closer: re.Pattern[str], limit: int, maximum_indent: int | None = None) -> int | None: """Offset just past the line closing the fence opened at `opener`, or `None` when nothing closes it before `limit`. Pandoc reads an unterminated fence as prose, including citations below its apparent opener. The closer must be at least as long as the opener and use the same character, so a ``` inside a ````-fenced block is sample text rather than the end of it.""" newline = text.find("\n", opener) if newline < 0 or newline + 1 >= limit: # the opener ends the region return None for match in closer.finditer(text, newline + 1, limit): if (match.group("char") == char and len(match.group("fence")) >= size and (maximum_indent is None or measure_indent(text, match.start())[0] <= maximum_indent)): return match.end() return None def measure_indent(text: str, start: int) -> tuple[int, int]: """`(columns, offset)` for the whitespace opening the line at `start`: how far the line is indented, counting a tab as the four columns `getIndent` in lib/markdown/parser.mjs gives it, and where its content begins.""" columns, index = 0, start while index < len(text) and text[index] in " \t": columns += 4 if text[index] == "\t" else 1 index += 1 return columns, index def line_end(text: str, index: int) -> int: """Offset just past the newline ending the line holding `index`.""" newline = text.find("\n", index) return len(text) if newline < 0 else newline + 1 def at_line_start(text: str, index: int) -> bool: """Whether `index` sits at the head of a line. A masked region can end on either side of a newline -- a fence ends on its closer, an indented block on the line past its last line -- and the scanner has to resume in the state the resumption point is really in: reading a line head as mid-prose misses the fence opener on it, which then blanks the rest of the document from the closer down and hides the citations there.""" return not index or text[index - 1] == "\n" def indented_block_end(text: str, opener: int, minimum_indent: int = 4, quoted: bool = False) -> int: """Offset past the code run at `minimum_indent` columns: four at top level, or four past a list item's content. A blank line stays in the run only when more indented text follows it, the reading lib/markdown/parser.mjs gives an indented code block. This bounds the hunt for the closer of a fence no container holds, so an unclosed one costs its own block rather than every citation below it.""" end = cursor = line_end(text, opener) while cursor < len(text): stop = line_end(text, cursor) content = cursor if quoted: marker = QUOTED_LINE_RE.match(text, cursor) if marker is None: break # unmarked lazy prose does not inherit quote-relative code indentation content = marker.end() if text[content:stop].strip(): if measure_indent(text, content)[0] < minimum_indent: break end = stop cursor = stop return end def item_content_indent(indent: int, marker: re.Match[str]) -> int: """The column the content of the list item opened by `marker`, written at column `indent`, sits at: past the marker and the spaces after it, which count as part of the marker only up to four -- a wider gap, or an empty item, puts the content one column past the marker instead. Only a line indented at least that far is the item's own content; one indented less sits outside the item however deeply the marker itself is indented.""" width = len(marker.group(0).rstrip(" \t")) rest = marker.string[width:] gap = 0 for char in rest: if char not in " \t": break gap += 4 if char == "\t" else 1 if not 1 <= gap <= 4 or not rest.strip(): gap = 1 return indent + width + gap def empty_dash_starts_list(text: str, start: int, last_block_end: int) -> bool: """A bare dash may underline heading text instead of starting an item. Keep real empty items after blanks, completed code blocks and lists.""" if not start or (0 < last_block_end <= start and not text[last_block_end:start].strip()): return True previous_start = text.rfind("\n", 0, start - 1) + 1 previous = text[previous_start:start].strip() return not previous or bool( (THEMATIC_BREAK_RE.fullmatch(previous) and thematic_boundary(text, previous_start, last_block_end)) or (LIST_MARKER_RE.match(previous) and (previous == "-" or opens_block(text, previous_start, last_block_end=last_block_end))) ) def list_container( text: str, start: int, last_block_end: int = 0, completed_fence: tuple[int, int, tuple[int, int] | None] | None = None, allow_lazy: bool = False, ) -> tuple[int, int] | None: """The marker/content columns of the list item holding the indented line at `start`, or `None` when no container holds it and the indentation is an indented code block of its own. The run is walked back the way `opens_block` walks it, past its own indented and blank lines, and lands either on that marker or on the prose, break or start of document an indented code block sits under.""" opener_indent = measure_indent(text, start)[0] previous = text.rfind("\n", 0, start - 1) + 1 lazy = allow_lazy and bool(text[previous:start].strip()) cursor = start while cursor: line = text.rfind("\n", 0, cursor - 1) + 1 if completed_fence and completed_fence[0] <= line < completed_fence[1]: # An outdented sample inside completed code is not an item boundary. container = completed_fence[2] return container if container and container[1] <= opener_indent else None indent, body = measure_indent(text, line) content = text[body:cursor].strip() marker = LIST_MARKER_RE.match(content) if content else None if ( marker and (item_content_indent(indent, marker) <= opener_indent or lazy) and not THEMATIC_BREAK_RE.match(content) and opens_block(text, line, last_block_end=last_block_end) and (content != "-" or empty_dash_starts_list(text, line, last_block_end)) ): return indent, item_content_indent(indent, marker) if content and indent < 4: previous = text.rfind("\n", 0, line - 1) + 1 if not line or not text[previous:line].strip(): return None cursor = line return None def container_end(text: str, opener: int, indent: int) -> int: """Offset just past the last line of the list item whose marker sits at column `indent` and whose content holds `opener`. The item keeps every line indented past its marker, and an outdented one too when no blank line precedes it -- the lazy continuation Pandoc reads as more of the item, and what lets a fence nested in a list carry an outdented body. The item ends at the next marker at that level, or where a blank line hands the document back to an outdented one.""" end = cursor = line_end(text, opener) gap = False while cursor < len(text): stop = line_end(text, cursor) content = text[cursor:stop].strip() if not content: gap = True elif measure_indent(text, cursor)[0] <= indent and (gap or LIST_MARKER_RE.match(content)): break else: end, gap = stop, False cursor = stop return end def nested_block_end( text: str, opener: int, last_block_end: int = 0, completed_fence: tuple[int, int, tuple[int, int] | None] | None = None, ) -> int: """The bound on the hunt for the closer of a fence opened four columns in, which is what keeps an unclosed one from swallowing every citation below it. Which block bounds it depends on what owns the indentation: the list item around the fence when a container does, so an outdented line inside the item stays code, and the indented run itself at top level, where the same outdented line is a block of its own and the citations on it are real.""" container = list_container(text, opener, last_block_end, completed_fence) if container is None: return indented_block_end(text, opener) return container_end(text, opener, container[0]) def setext_heading_before(text: str, underline: int, last_block_end: int) -> bool: """An equals run ends a block only when it underlines preceding heading text, not when it follows a blank, completed code or a list marker.""" if not underline or (last_block_end and underline <= last_block_end): return False previous_start = text.rfind("\n", 0, underline - 1) + 1 previous = text[previous_start:underline].strip() return bool( previous and (not LIST_MARKER_RE.match(previous) or THEMATIC_BREAK_RE.fullmatch(previous)) and opens_block(text, previous_start, last_block_end=last_block_end) ) def thematic_boundary(text: str, start: int, last_block_end: int) -> bool: """Rule-looking lines cannot interrupt a Pandoc paragraph. Solid dashes may instead finish a single-line setext heading.""" content = text[start:line_end(text, start)].strip() return (opens_block(text, start, allow_list_paragraph=False, last_block_end=last_block_end) or (bool(re.fullmatch(r"-{3,}", content)) and setext_heading_before(text, start, last_block_end))) def setext_follows(text: str, start: int, quoted: bool) -> bool: """Pandoc gives a single heading line priority over indented code.""" next_line = line_end(text, start) if quoted and (marker := QUOTED_LINE_RE.match(text, next_line)): next_line = marker.end() return (bool(re.fullmatch(r" {0,3}(?:=+|-+)[ \t]*\n?", text[next_line:line_end(text, next_line)])) and not line_span_crosses(text, start)) def line_span_crosses(text: str, start: int) -> bool: """Whether inline parsing of this line consumes a later physical line. Respect row-local spans and the same shorter-run fallback as the scanner.""" stop = line_end(text, start) cursor = start while (run := TICK_RUN_RE.search(text, cursor, stop)) is not None: cursor = run.end() if escaped(text, run.start()): continue for opener in range(run.start(), run.end()): end = span_end(text, opener, run.end() - opener) if end is not None: if end > stop: return True cursor = end break return False def simple_table(text: str, start: int, quoted: bool = False) -> tuple[int, tuple[int, ...]] | None: """Find observed simple-table rows in their physical or quote container. Separator columns bound each cell, including an indented first column.""" def body(position: int) -> int: marker = QUOTED_LINE_RE.match(text, position) if quoted else None return marker.end() if marker else position first_end = line_end(text, start) first = text[start:first_end].rstrip() border = r" {0,3}-+(?:[ \t]+-+)*" following = text[body(first_end):line_end(text, first_end)].rstrip() header = (not re.fullmatch(border, first) or first.count("-") < 3 or bool(re.fullmatch(border, following))) separator = following if header else first if not re.fullmatch(border, separator): return None cursor = first_end if header: # Complete fenced blocks and admitted ATX headings win over a table. run = DELIM_RUN_RE.match(text, start) if (run and len(run.group()) >= 3 and opens_fence(text, run.end(), run.group("char")) and fence_end(text, start, run.group("char"), len(run.group()), FENCE_LINE_RE, len(text)) is not None): return None if ATX_HEADING_RE.match(first): return None if (len(re.findall(r"-+", separator)) == 1 and not separator.startswith(" ") and not line_span_crosses(text, start)): return None cursor = line_end(text, cursor) columns = tuple(match.start() for match in re.finditer(r"-+", separator)) rows = 0 while cursor < len(text): stop = line_end(text, cursor) content = text[body(cursor):stop].rstrip() if not content.strip(): return (cursor, columns) if header and rows else None if re.fullmatch(border, content): return (stop, columns) if rows else None rows += 1 cursor = stop return (cursor, columns) if rows and (header or separator.startswith(" ")) else None def line_block_continuation(text: str, start: int, last_block_end: int) -> bool: """Indented lines continue a Pandoc line block; they are not code.""" cursor = start while cursor: previous = text.rfind("\n", 0, cursor - 1) + 1 content = text[previous:cursor].strip() if not content: return False if content.startswith("| "): return opens_block(text, previous, last_block_end=last_block_end) if not measure_indent(text, previous)[0]: return False cursor = previous return False def opens_block( text: str, start: int, allow_list_paragraph: bool = True, last_block_end: int = 0, minimum_indent: int = 4, ) -> bool: """Whether the indented line at `start` really begins a block. Indentation opens code only where a block can start: Pandoc reads the ` ~~~` in `A paragraph\\n ~~~\\n [@missing]` as more of that paragraph, so masking a fence there hides a real citation and lets `--strict` pass over a missing bibliography key. The run of indented lines is walked back to the line that precedes it, which has to be blank, absent, or a block break for the indentation to mean code. A completed code block also ends a paragraph; an unmatched fence-looking line does not.""" if 0 < last_block_end <= start and not text[last_block_end:start].strip(): return True cursor = start while cursor: line = text.rfind("\n", 0, cursor - 1) + 1 content = text[line:cursor].strip() if not content: # a blank line ends whatever ran above it return True if measure_indent(text, line)[0] < minimum_indent: if content.startswith("| "): # A real Pandoc line block ends before a following list. A # pipe in ordinary paragraph text is not this boundary. return opens_block(text, line, last_block_end=last_block_end) if ATX_HEADING_RE.match(content): # Pandoc headings cannot interrupt an existing paragraph. return opens_block(text, line, allow_list_paragraph=allow_list_paragraph, last_block_end=last_block_end, minimum_indent=minimum_indent) if SETEXT_RULE_RE.fullmatch(content): return setext_heading_before(text, line, last_block_end) if THEMATIC_BREAK_RE.fullmatch(content): return thematic_boundary(text, line, last_block_end) if content == "-": # A dash within a continuing paragraph is neither an item nor # a heading underline, so it cannot admit quote/indented code. return (opens_block(text, line, allow_list_paragraph=allow_list_paragraph, last_block_end=last_block_end, minimum_indent=minimum_indent) or setext_heading_before(text, line, last_block_end)) marker = LIST_MARKER_RE.match(content) if (not allow_list_paragraph and marker and content[marker.end():].strip() and not THEMATIC_BREAK_RE.fullmatch(content)): # List paragraph text needs a blank before indented code; # its continuation may instead contain an inline code span. return False if marker and content != "-" and not THEMATIC_BREAK_RE.fullmatch(content): # Marker-shaped paragraph continuations need their own block # boundary. A bare dash can instead complete a setext heading. cursor = line continue return bool(BLOCK_BREAK_RE.match(content)) cursor = line return True def escaped(text: str, index: int) -> bool: """Whether the delimiter at `index` is a backslash-escaped literal, which Pandoc prints rather than reads. Such a backtick opens no code span, and treating it as one masks through to the next backtick and hides the citations in between. Parity decides it: an odd run of backslashes escapes the delimiter, while an even run escapes only itself and leaves a live delimiter behind.""" start = index while start and text[start - 1] == "\\": start -= 1 return (index - start) % 2 == 1 def tilde_block_start(text: str, info: int, last_block_end: int) -> bool: """Tildes cannot interrupt paragraph/list text. A blank, document start, or completed block can precede them; a list marker alone is not a break.""" start = text.rfind("\n", 0, info) + 1 if not text[last_block_end:start].strip(): return True previous_start = text.rfind("\n", 0, start - 1) + 1 previous = text[previous_start:start].strip() if ATX_HEADING_RE.match(previous): return opens_block(text, previous_start, last_block_end=last_block_end) if SETEXT_RULE_RE.fullmatch(previous): return setext_heading_before(text, previous_start, last_block_end) if THEMATIC_BREAK_RE.fullmatch(previous): return thematic_boundary(text, previous_start, last_block_end) return not previous or bool( BLOCK_BREAK_RE.match(previous) and not LIST_MARKER_RE.match(previous) and not DELIM_RUN_RE.match(previous) ) def opens_fence(text: str, info: int, char: str, last_block_end: int = 0) -> bool: """Whether a line-initial run of three or more fence characters really opens a block. An info string may not contain a backtick after a ``` run, so such a line is prose holding a code span; reading it as an opener leaves the fence unterminated and blanks the rest of the document, hiding every citation below it -- including the missing keys `--strict` exists to catch. Tildes additionally require a block boundary and a Pandoc info string; backticks retain their distinct paragraph-interrupting behavior.""" if char == "~": return tilde_block_start(text, info, last_block_end) and bool(TILDE_INFO_RE.match(text, info)) return bool(BACKTICK_INFO_RE.match(text, info)) def quote_context(text: str, index: int, last_block_end: int = 0, allow_unmarked: bool = False) -> tuple[str, int] | None: """Read the quote's prior paragraph without its container prefixes. Lazy unmarked lines remain prose; a physical blank starts a new context.""" start = text.rfind("\n", 0, index) + 1 marker = QUOTED_LINE_RE.match(text, start) indent, content = measure_indent(text, start) if marker is None and indent < 1 and not allow_unmarked: return None content = marker.end() if marker else content if content > index: return None lines = [] cursor = start while cursor: previous = text.rfind("\n", 0, cursor - 1) + 1 if previous < last_block_end or not text[previous:cursor].strip(): break lines.append((previous, cursor)) cursor = previous lines.reverse() first_quote = next((a for a, _ in lines if QUOTED_LINE_RE.match(text, a)), start if marker else None) if first_quote is None: return None if setext_follows(text, first_quote, quoted=False): return None # an outer underline makes the quote-shaped header literal container = list_container(text, first_quote, last_block_end, allow_lazy=True) minimum_indent = container[1] + 4 if container else 4 if not opens_block(text, first_quote, allow_list_paragraph=False, last_block_end=last_block_end, minimum_indent=minimum_indent): return None # Pandoc does not admit a quote marker inside an existing paragraph before = "".join(text[m.end():b] if (m := QUOTED_LINE_RE.match(text, a)) else text[a:b].lstrip(" \t") for a, b in lines) return before + text[content:line_end(text, index)], len(before) + index - content def quote_limit(text: str, opener: int) -> int: """Do not pair delimiters across a separate quote or a real outer fence. Unmarked lazy lines without a blank can remain inside a quoted fence.""" cursor = line_end(text, opener) while cursor < len(text): stop = line_end(text, cursor) if not text[cursor:stop].strip(): return cursor # an unmarked blank separates Pandoc quote blocks if not QUOTED_LINE_RE.match(text, cursor): run = DELIM_RUN_RE.match(text, cursor) if (run and len(run.group()) >= 3 and opens_fence(text, run.end(), run.group("char")) and fence_end(text, cursor, run.group("char"), len(run.group()), FENCE_LINE_RE, len(text)) is not None): return cursor cursor = stop return len(text) def span_end(text: str, opener: int, size: int, markdown_spans: str = "multiline", limit: int | None = None, in_list: bool = False) -> int | None: """Offset just past the run of exactly `size` backticks closing the inline span opened at `opener`, or `None` when nothing closes it and the backticks are ordinary prose. The closing run must be the SAME length as the opening one: a longer run (``a``` ) closes nothing. The explicit span policy bounds the hunt at a physical newline or a blank line. Block-looking lines inside a matched multiline span are literal text, including fence delimiters.""" bound = len(text) if limit is None else limit if markdown_spans == "line": limit = min(bound, line_end(text, opener)) else: boundary = BLANK_LINE_RE.search(text, opener) limit = min(bound, boundary.start() if boundary else len(text)) if in_list: cursor = line_end(text, opener) while cursor < limit: stop = line_end(text, cursor) content = text[cursor:stop].strip() if LIST_MARKER_RE.match(content) and not THEMATIC_BREAK_RE.fullmatch(content): limit = cursor # a next or nested item ends the current item's inline paragraph break cursor = stop for run in TICK_RUN_RE.finditer(text, opener + size, limit): if run.end() - run.start() == size: return run.end() return None def mask_markdown_code(text: str, markdown_spans: str = "multiline") -> str: """Blank every code region a citation cannot live in. Fences and spans have to be found in one pass, because each hides the other's delimiters: a ``` line inside a span that crosses lines (``a\\n```\\n``) opens no block, and a lone backtick inside a fenced block closes no span. Matching fences first instead pairs that inner line with the next real opener and blanks the prose between them, dropping the citations it carries. A delimiter only counts where Markdown reads one: a backslash-escaped backtick is printed text, and indentation decides whether a run of three opens a block here or in the indented code the container around it holds.""" out = list(text) index, line_start = 0, True last_block_end = 0 completed_fence = None table_end, table_columns, table_quoted = 0, (), False while index < len(text): if table_end and index >= table_end: last_block_end, completed_fence, table_end = table_end, None, 0 char = text[index] if char == "\n": index, line_start = index + 1, True continue if line_start: indent, content = measure_indent(text, index) if not table_end: table_context = quote_context(text, index, last_block_end, allow_unmarked=True) table_text, table_start = table_context or (text, index) if opens_block(table_text, table_start, last_block_end=0 if table_context else last_block_end): table_end, table_columns = simple_table(text, index, quoted=table_context is not None) or (0, ()) table_quoted = table_context is not None if index < table_end: row_start = text.rfind("\n", 0, index) + 1 if table_quoted and (marker := QUOTED_LINE_RE.match(text, row_start)): row_start = marker.end() edges = [row_start + column for column in table_columns] out[row_start:edges[0]] = list(blank(text[row_start:edges[0]])) # A token split between cells is not the unsplit citation. # Keep offsets stable; truncated-cell keys remain outside # this bounded lexical audit rather than inventing a key. for cite in PANDOC_CITE_RE.finditer(text, row_start, line_end(text, index)): if any(cite.start() < edge < cite.end() for edge in edges): out[cite.start():cite.end()] = list(blank(text[cite.start():cite.end()])) line_start = False elif indent and line_block_continuation(text, index, last_block_end): index, line_start = content, False continue elif indent >= 4: context = quote_context(text, index, last_block_end) if context is None and quote_context(text, content, last_block_end) is not None: # Lazy quoted indentation is not outer indented code; # a delimiter still leads its quote-relative line. run = DELIM_RUN_RE.match(text, content) index, line_start = content, bool(run and len(run.group()) >= 3) continue block_text, block_start = context or (text, index) container = list_container(block_text, block_start, 0 if context else last_block_end, None if context else completed_fence) code_indent = container[1] + 4 if container else 4 if (indent >= code_indent and not setext_follows(text, index, context is not None) and opens_block(block_text, block_start, allow_list_paragraph=False, last_block_end=0 if context else last_block_end, minimum_indent=code_indent)): # An indented code block can contain a lone backtick or a # list-looking sample. Inside a list, measure the four # code columns from the item's content, not the page edge. end = indented_block_end(text, index, code_indent, quoted=context is not None) out[index:end] = list(blank(text[index:end])) if not context: last_block_end = end index, line_start = end, at_line_start(text, end) continue # Four columns of indentation make this an indented code line at # top level and a list item's own content inside a container, so # a fence opened where a block can start is code under either # reading. Its closer carries the same indentation, which # FENCE_LINE_RE will not match, and reading the line as prose # instead leaves the block scannable and reports the `@key` # samples in it as missing. Under an open paragraph the same # indentation is a lazy continuation of it, prose either way. run = DELIM_RUN_RE.match(text, content) size = run.end() - content if run else 0 fence = ( size >= 3 and indent < code_indent and opens_fence(text, run.end(), text[content], last_block_end) and (bool(container and indent == container[1]) or opens_block(text, index, allow_list_paragraph=False, last_block_end=last_block_end)) ) end = ( fence_end( text, content, text[content], size, NESTED_FENCE_LINE_RE, nested_block_end(text, index, last_block_end, completed_fence), code_indent - 1 if container else None, ) if fence else None ) if end is not None: out[content:end] = list(blank(text[content:end])) completed_fence = (content, end, container) if container else None last_block_end = end index, line_start = end, at_line_start(text, end) else: index, line_start = content, False continue if content > index: # three columns or fewer still open a fence index = content continue if ((marker := BLOCKQUOTE_MARKER_RE.match(text, index)) and quote_context(text, marker.end(), last_block_end) is not None): # The quote's marker stands in front of the block it holds, so # the rest of the line still leads a line: a fence opener there # opens, and four columns past the marker are its indented code. index = marker.end() continue if (run := DELIM_RUN_RE.match(text, index)) and not escaped(text, index): size = run.end() - index context = quote_context(text, index, last_block_end) block_text, block_start = context or (text, index) block_line = block_text.rfind("\n", 0, block_start) + 1 container = list_container(block_text, block_line, 0 if context else last_block_end, None if context else completed_fence, allow_lazy=True) indent = measure_indent(block_text, block_line)[0] relative_indent = indent - container[1] if container else indent limit = quote_limit(text, index) if context else len(text) boundary = (relative_indent == 0 or bool(container and indent <= container[0]) or opens_block( block_text, block_line, allow_list_paragraph=False, last_block_end=0 if context else last_block_end, minimum_indent=container[1] + 4 if container else 4)) if line_start and size >= 3 and bounskills/claim-verification/scripts/audit_consistency.py11 KB
#!/usr/bin/env python3 """Offline cross-file consistency audit for a project; no network access. Finds places where the project contradicts ITSELF: one quantity asserted with two values, one acronym given two expansions, one LaTeX macro defined two ways. Like the citation audit, this reports CANDIDATES for a human or agent to judge, never verdicts -- two different numbers can be two legitimate measurements. """ from __future__ import annotations import argparse import json import re import sys from pathlib import Path from typing import Any from audit_citations import LATEX_SUFFIXES, blank, line_number, scannable PROSE_SUFFIXES = {".md", ".markdown", ".mdx", ".txt"} | LATEX_SUFFIXES SKIP_DIRS = {".git", "node_modules", ".venv", "venv", "__pycache__", "_minted", "build", "dist", ".next"} # Rows of a table legitimately carry different values for the same label (one # per model, per year, per condition), so they are hidden before scanning -- # the single largest source of false "conflicts" in a real paper. MD_TABLE_DELIMITER_RE = re.compile(r"[ \t]*:?-+:?[ \t]*") TABULAR_RE = re.compile(r"(?s)\\begin\{(tabular\*?|tabularx|longtable|array)\}.*?\\end\{\1\}") NUMBER = r"\d{1,3}(?:,\d{3})+|\d+(?:\.\d+)?" MAGNITUDE = r"k|thousand|m|mm|million|b|bn|billion" VALUE_RE = rf"(?P<num>{NUMBER})\s*(?P<mag>{MAGNITUDE})?\b\s*(?P<pct>\\?%|percent)?" # "accuracy of 91.2%", "the sample size is 1,200", "F1 = 0.83" LABEL_FIRST_RE = re.compile( rf"(?i)(?P<label>[A-Za-z][A-Za-z0-9_+\-]*(?:[ \t]+[A-Za-z][A-Za-z0-9_+\-]*){{0,2}})" rf"[ \t]*(?:of|is|was|are|were|reached|equals?|=|:)[ \t]*{VALUE_RE}" ) # "1.2M examples", "12 participants" VALUE_FIRST_RE = re.compile(rf"(?i){VALUE_RE}[ \t]+(?P<label>[A-Za-z][A-Za-z0-9_-]{{2,}})") # "large language model (LLM)" ACRONYM_RE = re.compile( r"(?P<expansion>(?:[A-Za-z][\w'-]*[ \t]+){1,6}[A-Za-z][\w'-]*)[ \t]*\((?P<acronym>[A-Z][A-Za-z0-9]{1,9})s?\)" ) MACRO_RE = re.compile( r"\\(?:re)?newcommand\*?[ \t]*\{?\\(?P<name>[A-Za-z@]+)\}?(?:\[\d+\])?(?:\[[^\]]*\])?[ \t]*\{|" r"\\def[ \t]*\\(?P<defname>[A-Za-z@]+)[^{}]*\{" ) STOPWORDS = { "the", "a", "an", "and", "or", "but", "if", "then", "than", "that", "this", "these", "those", "it", "its", "we", "our", "they", "their", "he", "she", "there", "here", "which", "who", "of", "in", "on", "at", "to", "for", "by", "with", "from", "as", "about", "into", "over", "is", "was", "are", "were", "be", "been", "being", "has", "have", "had", "all", "each", "one", "two", "three", "first", "second", "third", "next", "last", "other", "some", "any", "figure", "table", "section", "chapter", "equation", "appendix", "page", "line", "step", "eq", "fig", "tab", "sec", "ref", "cite", "item", "note", "see", "et", "al", } MAGNITUDES = {"k": 1e3, "thousand": 1e3, "m": 1e6, "mm": 1e6, "million": 1e6, "b": 1e9, "bn": 1e9, "billion": 1e9} def normalize_label(raw: str) -> str | None: """A comparable subject key, or None when the words carry no subject. Trailing plural `s` is dropped so `12 participants` and `15 participant` meet, and leading filler is trimmed so `the sample size` keys on `sample size`.""" words = [w.lower() for w in re.split(r"[ \t]+", raw.strip()) if w] while words and words[0] in STOPWORDS: words.pop(0) while words and words[-1] in STOPWORDS: words.pop() if not words: return None singular = [w[:-1] if len(w) > 3 and w.endswith("s") and not w.endswith("ss") else w for w in words] if all(w in STOPWORDS or len(w) < 3 for w in singular): return None return " ".join(singular) def normalize_value(match: re.Match[str]) -> tuple[float, str] | None: try: number = float(match.group("num").replace(",", "")) except ValueError: return None if magnitude := (match.group("mag") or "").lower(): number *= MAGNITUDES[magnitude] return number, "%" if match.group("pct") else "" def mask_pipe_tables(text: str) -> str: """Hide delimiter-confirmed pipe tables without changing source offsets.""" def cells(line: str) -> list[str]: parts = re.split(r"(?<!\\)\|", line.strip()) if len(parts) < 2: return [] if not parts[0].strip(): parts.pop(0) if not parts[-1].strip(): parts.pop() return parts lines = text.splitlines(keepends=True) for index in range(1, len(lines)): delimiter = cells(lines[index]) if not delimiter or not all(MD_TABLE_DELIMITER_RE.fullmatch(cell) for cell in delimiter): continue if len(cells(lines[index - 1])) != len(delimiter): continue end = index + 1 while end < len(lines) and cells(lines[end]): end += 1 for row in range(index - 1, end): lines[row] = blank(lines[row]) return "".join(lines) def prose(text: str, latex: bool) -> str: """Scannable prose with tables hidden, offsets preserved.""" text = scannable(text, latex) text = TABULAR_RE.sub(lambda m: blank(m.group(0)), text) return text if latex else mask_pipe_tables(text) def excerpt(text: str, match: re.Match[str]) -> str: start = text.rfind("\n", 0, match.start()) + 1 end = text.find("\n", match.end()) return text[start : end if end >= 0 else len(text)].strip()[:200] def scan_file(path: Path, root: Path) -> dict[str, list[dict[str, Any]]]: text = prose(path.read_text(encoding="utf-8"), path.suffix.lower() in LATEX_SUFFIXES) source = str(path.relative_to(root)) if path.is_relative_to(root) else str(path) found: dict[str, list[dict[str, Any]]] = {"quantity": [], "acronym": [], "macro": []} claimed: set[tuple[int, int]] = set() for form, pattern in (("label-first", LABEL_FIRST_RE), ("value-first", VALUE_FIRST_RE)): for match in pattern.finditer(text): # `label-first` wins an overlap: it names the subject explicitly. if any(start < match.end() and match.start() < end for start, end in claimed): continue label = normalize_label(match.group("label")) value = normalize_value(match) if label is None or value is None: continue claimed.add((match.start(), match.end())) found["quantity"].append({ "key": label, "value": value[0], "unit": value[1], "form": form, "source": source, "line": line_number(text, match.start()), "text": excerpt(text, match), }) for match in ACRONYM_RE.finditer(text): expansion = normalize_label(match.group("expansion")) if expansion is None: continue found["acronym"].append({ "key": match.group("acronym"), "value": expansion, "source": source, "line": line_number(text, match.start()), "text": excerpt(text, match), }) for match in MACRO_RE.finditer(text): depth, index = 1, match.end() while index < len(text) and depth: if text[index] == "\\": index += 2 continue depth += {"{": 1, "}": -1}.get(text[index], 0) index += 1 if depth: continue found["macro"].append({ "key": match.group("name") or match.group("defname"), "value": " ".join(text[match.end() : index - 1].split()), "source": source, "line": line_number(text, match.start()), "text": excerpt(text, match), }) return found def conflicts(occurrences: list[dict[str, Any]], group: tuple[str, ...]) -> list[dict[str, Any]]: """Groups whose occurrences disagree, i.e. carry more than one value.""" buckets: dict[tuple[Any, ...], list[dict[str, Any]]] = {} for item in occurrences: buckets.setdefault(tuple(item[field] for field in group), []).append(item) out = [] for key, items in sorted(buckets.items(), key=lambda kv: str(kv[0])): values = {item["value"] for item in items} if len(values) < 2: continue out.append({ "key": key[0], "unit": key[1] if len(key) > 1 else None, "values": sorted(values, key=str), "occurrences": sorted(items, key=lambda i: (i["source"], i["line"])), }) return out def main() -> int: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--path", action="append", default=[], help="file or directory to scan (repeatable; defaults to the working directory)") parser.add_argument("--format", choices=("json", "text"), default="json") parser.add_argument("--strict", action="store_true", help="exit 1 when any conflict is reported") args = parser.parse_args() roots = [Path(p) for p in (args.path or ["."])] files: list[Path] = [] for root in roots: if not root.exists(): parser.error(f"{root}: no such file or directory") candidates = [root] if root.is_file() else sorted(root.rglob("*")) files.extend( path for path in candidates if path.is_file() and path.suffix.lower() in PROSE_SUFFIXES and not SKIP_DIRS.intersection(path.parts) ) base = roots[0] if roots[0].is_dir() else roots[0].parent collected: dict[str, list[dict[str, Any]]] = {"quantity": [], "acronym": [], "macro": []} warnings: list[str] = [] for path in dict.fromkeys(files): try: for kind, items in scan_file(path, base).items(): collected[kind].extend(items) except (OSError, UnicodeDecodeError) as error: warnings.append(f"{path}: {error}") report = { "schema_version": 1, "files_scanned": len(dict.fromkeys(files)), "quantity_conflicts": conflicts(collected["quantity"], ("key", "unit")), "acronym_conflicts": conflicts(collected["acronym"], ("key",)), "macro_conflicts": conflicts(collected["macro"], ("key",)), "parse_warnings": warnings, "scope": "internal consistency candidates only; nothing fetched and no claim judged true or false", } findings = any(report[key] for key in ("quantity_conflicts", "acronym_conflicts", "macro_conflicts")) if args.format == "json": json.dump(report, sys.stdout, indent=2, ensure_ascii=False) sys.stdout.write("\n") else: print(f"{report['files_scanned']} files scanned") for kind in ("quantity_conflicts", "acronym_conflicts", "macro_conflicts"): for conflict in report[kind]: where = ", ".join(f"{o['source']}:{o['line']}" for o in conflict["occurrences"]) print(f"{kind[:-10]} {conflict['key']!r}: {conflict['values']} ({where})") for warning in warnings: print(f"warning: {warning}") return 1 if args.strict and findings else 0 if __name__ == "__main__": raise SystemExit(main())