Documentation

PARALLAX is a correctness gate for self-healing scrapers: it proves a heal still extracts the right value, not just a non-null one. Terminal-first; every command exits with a CI-friendly code.

Quickstart

git clone https://github.com/vinayaksonthalia/parallax && cd parallax npm install npm test # 919 tests node bin/parallax.mjs check https://example-store.com/product/x

Pages are fetched through Bright Data's Web Unlocker when the bdata CLI is authenticated (bdata login); without it, PARALLAX falls back to a direct fetch so you can develop offline. No configuration file is required to start.

parallax check

Reads one live page three diverse ways and flags disagreement — the signature of a silent wrong-but-plausible value.

parallax check <url> [--json]
◉ parallax check · https://www.chesshouse.com/products/chessup-chess-computer fetched via bright-data structural eye $399.00 [class*="price"] semantic eye $399.00 label "Sale price" json-ld oracle $399.00 structured data ✓ CONSISTENT — all reads agree on $399.00.

That is a real, unedited run against a live store. The three reads: a structural eye (CSS priority list), a semantic eye (label-anchored text search that uses zero CSS — it reports whether it truly anchored on a label, so a coincidental match can't pose as verification), and the site's own JSON-LD structured data.

parallax verify

The metamorphic promotion gate. Give it the selector a heal proposes; it pins the semantic eye's node, mutates a copy of the page six ways, and requires the selector to keep tracking the pinned value. Verdicts are explained, not just colored.

parallax verify <url> --selector <css> [--json] [--html-file <path>]
◉ parallax verify · https://demo.local/p healed selector .buybox p:nth-of-type(2) pinned oracle $1299.00 label "Deal Price" identity ................... tracks reorder-siblings ........... tracks inject-decoy-no-label ...... diverged latched decoy token 900001 wrap-in-div ................ diverged read 19.99 · pinned 1299 duplicate-non-target ....... diverged latched decoy token 900002 value-sentinel ............. tracks ✗ QUARANTINED — under inject-decoy-no-label the heal read the decoy token (900001): it is anchored to position, not meaning. (consistency with the semantic eye's pinned node — not proof the value is right)

Each decoy carries a distinct token, so a failure names the exact wrong node the heal latched onto. --html-file runs the gate offline against a saved page — useful for demos and CI fixtures.

Fields

Every command takes --field (API: field). Nine built-ins ship, each with its own selector priorities, label vocabulary, JSON-LD path, and normalizer — graded honestly, because v1 quality is not uniform:

FieldReadsHonest grade
priceselectors · label tiers (sale > now > generic > regular) · offers.pricethe mature one — locale-aware, fail-closed
stockbadge/attribute · "Availability:" labels · schema.org statesgood — normalized to in_stock / out_of_stock
ratingrating nodes · "Rating: 4.5 out of 5" shapes · aggregateRatingv1 — scales aren't normalized across sites; same-page reads compare fine
skusku nodes · "SKU / Item number" labels · skugood — opaque tokens make wrong reads obvious
titleitemprop=name / h1 · longest headingweakest diversity — no label exists; structural-vs-heading, stated plainly
datedelivery/ships/published labelsv1 — only unambiguous shapes parse; 01/02/2026 fails closed
brandbrand/seller/"by" labels · brand.namev1 — "by" is loose by nature; shortest-element rule keeps it honest
discount"Save / % off" labelsv1 — percent vs amount reported, never converted
currencyexplicit code, else derived from the price symbolweakest of the nine — symbol derivation is a stated guess ($ ≠ always USD)

Custom fields — verify anything labelled

parallax check <url> --label "Warranty" --type number parallax verify <url> --label "Delivery by" --type date --selector ".ship-date"

The custom engine compiles a field definition at runtime from any label plus a type (number · text · date · state). The four newest built-ins are this same engine with a pre-filled vocabulary — nothing special-cased. A custom field is honestly a two-eye check: the site never declared it in structured data, so there is no third read, and the output says so instead of letting a null look like agreement.

parallax replay — blast radius

watch archives every page it fetches into a snapshot store. replay runs any selector across that history against the semantic read, so a verdict becomes visible prevented-damage:

parallax replay <url> --selector ".col .price" [--field <name>] [--snapshots-dir <dir>]
oldest → newest ███▁▁▁ ✗ would have differed from the semantic read on 3 of 6 snapshots. (what this selector WOULD have read, against a cache of past fetches — not a claim about the future.)

verify appends the same line after its verdict when history exists — a heal that passes on today's page still gets confronted with what it would have read yesterday. Honest edges: a run from --html-file stores nothing (one file re-read is not a timeline), an unparseable selector exits 1 rather than reading as zero differences, and "couldn't run" is never conflated with "no history".

MCP server

The 2026 loop closed: the AI agent that heals a scraper can consult the gate before trusting its own heal. PARALLAX ships a zero-dependency Model Context Protocol server over stdio.

npm run mcp # or: node mcp/server.mjs # register in Claude Code: claude mcp add parallax -- node ./mcp/server.mjs
ToolArgumentsReturns
check_pageurl, optional field or labelthe three reads + CONSISTENT / DISAGREE / INCOMPLETE
verify_healurl or inline html, selector, optional field or label/typeoutcome, per-mutation results, explanation, suggestion
explain_verdicta verdict objectthe one-sentence explanation (pure, offline)

Full registration and schema details in mcp/README.md.

GitHub Action

The gate as CI: fail a pull request when a selector change would ship wrong-but-plausible data.

- uses: vinayaksonthalia/parallax@main with: html_file: fixtures/product.html # or url: for live pages selector: ".buybox .money"

The job fails on QUARANTINED, passes with a warning on NEEDS SCOPE (the suggested narrowed selector is surfaced as an output and in the warning), and writes a step summary with the verdict badge, the per-mutation table — latched decoy tokens named — and the explanation. Inputs reach the shell via env, so a PR-controlled selector can't inject commands.

Alerting

Gate events (QUARANTINED, NEEDS_SCOPE, DISAGREE, zero-rows) can POST to any webhook; Slack webhooks get a formatted blocks payload automatically. Alerting is engineered to never take down the pipeline: 5-second timeout, one retry with backoff, and a never-throws contract.

Verdicts & exit codes

VerdictMeaningExit
CONSISTENTAll diverse reads agree. (check)0
PROMOTEDThe healed selector tracked the pinned value through every mutation, including the value-sentinel echo. (verify)0
DISAGREEThe eyes read different values — a silent wrong-but-plausible candidate. (check)2
QUARANTINEDThe heal diverged under mutation: position-locked, wrong-node, or baseline mismatch. The explanation names the failing mutation and evidence. (verify)2
INCOMPLETEAn eye could not read the field at all. Fail-closed: investigate, don't trust. (check)3
NEEDS SCOPEThe selector reads a different legitimate copy of the same value (multi-copy page). Not wrong today, but unscoped — narrow it to the canonical container. (verify)4

Non-zero exits make every verdict directly usable as a CI gate.

How the gate works

1 — Pin. The semantic eye locates the true value node by its label ("Sale price: $399") on the original page, and the gate tags that node. Decoys injected later can never re-poison which node is "true".

2 — Mutate. Six meaning-preserving mutations run on fresh copies: identity, reorder-siblings, inject-decoy-no-label, wrap-in-div, duplicate-non-target, value-sentinel. A selector that only got lucky by position diverges here.

3 — Sentinel. The pinned node's value is rewritten to a unique token, and the selector must echo it. This upgrades the check from "same value" to "same node" — a selector reading a different node that coincidentally shares the value (a "Was $1,299" strikethrough) cannot echo the token and is caught.

The verdict is rendered as one factual sentence. The gate certifies consistency with the pinned semantic node under the mutation suite — deliberately not "truth" (see limits).

Tamper-evidence for free: sentinel and decoy tokens are unique values (900001+, 910000+). If one ever appears in your downstream dataset, your pipeline ingested a mutated verification page — the gate was wired wrong or bypassed. Treat it as an alarm.

Bright Data integration

PARALLAX is built around Scraper Studio's lifecycle, driven from the terminal:

StageCommandPARALLAX's role
createbdata scraper create <url> "<desc>"The generated collector emits both eyes per field.
runbdata scraper run / POST /dca/triggerThe conductor scans rows: zero-rows sentinel, per-row agreement.
healbdata scraper heal <id> "<prompt>"Break detected → heal requested on the same collector.
gateparallax verifyThe metamorphic gate decides: promote or quarantine.
approvebdata scraper approve --auto-saveOnly a promoted heal is saved to production.

Page fetching for check/verify uses the Web Unlocker (bdata scrape -f html) — rendering, unblocking, retries, and geo-routing come from Bright Data, so the gate sees the page a real user sees.

Testing, honestly

The suite currently passes 919 tests across 23 files — but we'd rather you know what that number is made of than be impressed by it:

GradeCountWhat they are
behavioral / scenario~399The real spine — each asserts a distinct behavior: the gate and its verdicts (metamorphic + parity + qa-hardening ≈ 129), the live watch loop with injected effects, replay, MCP driven over real stdio, the SSRF guard, the 9-field + custom-field engines, alerting failure paths.
table-driven sweeps~520Mechanical cross-products guarding the parser and agreement engine (comprehensive_adversarial, property_fuzz, error_paths, golden_fixtures). Every one runs real code, but the insight density is closer to 40–60 distinct equivalence classes than 520 hand-crafted cases. We say so here so the total can't mislead.

Plus a strict typecheck and the independence lint, all on every push in CI.

Honest limits

A correctness tool that hides its own boundaries isn't trustworthy. These are inherent, stated, and tested:

Consistency, not truth. If both eyes read the same wrong value (a mislabeled page), they agree and the gate promotes. It cannot distinguish "both right" from "both wrong."

Bounded by the semantic eye. Every verdict inherits its correctness from the label anchor. Where a page has no labels, the semantic eye honestly reports it isn't independent — and the gate fails closed rather than pretending.

Diverse, not independent. The eyes share the snapshot and DOM neighborhood; their failures are correlated (Knight & Leveson, 1986). We never claim statistical independence.

Multi-copy values. When the same value legitimately appears in several nodes (buybox + sticky bar), an unscoped selector reads "a" copy, not "the" copy. The gate detects this and reports NEEDS SCOPE with a narrowing suggestion instead of false-vetoing — but it still can't know which copy is canonical without the semantic pin. Found live on our first real target page, and fixed honestly.

JS-rendered pages need browser rendering — and the tool does it. A static fetch of a client-side storefront contains no values; pass --render and PARALLAX fetches the page through Bright Data's cloud browser (executes JS, applies CSS) before reading it. This also catches CSS-class-hidden nodes that static parsing can't see.

vs. the field

From a ~60-source survey of the domain (open source, academic, commercial):

SystemWhat it verifies after a heal / extractionPer-extraction correctness?
Bright Data self-healField is non-null (preview + human approval)no
Scrapling / anansiA structurally similar element still resolvesno
crawl4ai / Firecrawl / ScrapeGraphAIOutput matches the declared JSON schema (shape)no
Great Expectations / Soda / dbtColumn shape & aggregate distribution, post-hocno
Monte Carlo / AnomaloWarehouse-level drift vs. historical baselinesno
PARALLAXThe healed selector tracks the meaning-pinned node under mutation, with an explained verdictyes — this gap is the product

Fair statement of ancestry: dual-read verification and metamorphic testing are decades-old techniques (N-version programming, Chen et al. 1998). PARALLAX's contribution is the fusion — metamorphic testing as an automated post-heal promotion gate for web extraction — which our prior-art search did not find anywhere else.

Roadmap

Shipped: MCP server, GitHub Action, webhook/Slack alerting, the NEEDS_SCOPE verdict, the autonomous watch loop (check → heal → gate → approve on a schedule), blast-radius replay over cached snapshots, and multi-field support beyond price (stock, rating, sku, title, date, brand, discount, currency, plus custom labelled fields). Next: broaden field coverage and normalizers, richer snapshot analytics, and more first-class collector integrations.

Deliberately not building: a hosted dashboard (a static per-run report serves the need) and claims of truth-certification (see limits). The /api service that powers the playground is intentionally thin — the CLI, Action, and MCP server remain the primary consumers.

AI assistance disclosure: built with substantial AI assistance under the author's direction, review, and verification. MIT © 2026 Vinayak Sonthalia.