Documentation
PARALLAX is a correctness gate for self-healing scrapers: it proves a heal still extracts the right value, not just a non-null one. Terminal-first; every command exits with a CI-friendly code.
Quickstart
Pages are fetched through Bright Data's Web Unlocker when the bdata CLI is authenticated (bdata login); without it, PARALLAX falls back to a direct fetch so you can develop offline. No configuration file is required to start.
parallax check
Reads one live page three diverse ways and flags disagreement — the signature of a silent wrong-but-plausible value.
That is a real, unedited run against a live store. The three reads: a structural eye (CSS priority list), a semantic eye (label-anchored text search that uses zero CSS — it reports whether it truly anchored on a label, so a coincidental match can't pose as verification), and the site's own JSON-LD structured data.
parallax verify
The metamorphic promotion gate. Give it the selector a heal proposes; it pins the semantic eye's node, mutates a copy of the page six ways, and requires the selector to keep tracking the pinned value. Verdicts are explained, not just colored.
Each decoy carries a distinct token, so a failure names the exact wrong node the heal latched onto. --html-file runs the gate offline against a saved page — useful for demos and CI fixtures.
Fields
Every command takes --field (API: field). Nine built-ins ship, each with its own selector priorities, label vocabulary, JSON-LD path, and normalizer — graded honestly, because v1 quality is not uniform:
| Field | Reads | Honest grade |
|---|---|---|
| price | selectors · label tiers (sale > now > generic > regular) · offers.price | the mature one — locale-aware, fail-closed |
| stock | badge/attribute · "Availability:" labels · schema.org states | good — normalized to in_stock / out_of_stock |
| rating | rating nodes · "Rating: 4.5 out of 5" shapes · aggregateRating | v1 — scales aren't normalized across sites; same-page reads compare fine |
| sku | sku nodes · "SKU / Item number" labels · sku | good — opaque tokens make wrong reads obvious |
| title | itemprop=name / h1 · longest heading | weakest diversity — no label exists; structural-vs-heading, stated plainly |
| date | delivery/ships/published labels | v1 — only unambiguous shapes parse; 01/02/2026 fails closed |
| brand | brand/seller/"by" labels · brand.name | v1 — "by" is loose by nature; shortest-element rule keeps it honest |
| discount | "Save / % off" labels | v1 — percent vs amount reported, never converted |
| currency | explicit code, else derived from the price symbol | weakest of the nine — symbol derivation is a stated guess ($ ≠ always USD) |
Custom fields — verify anything labelled
The custom engine compiles a field definition at runtime from any label plus a type (number · text · date · state). The four newest built-ins are this same engine with a pre-filled vocabulary — nothing special-cased. A custom field is honestly a two-eye check: the site never declared it in structured data, so there is no third read, and the output says so instead of letting a null look like agreement.
parallax replay — blast radius
watch archives every page it fetches into a snapshot store. replay runs any selector across that history against the semantic read, so a verdict becomes visible prevented-damage:
verify appends the same line after its verdict when history exists — a heal that passes on today's page still gets confronted with what it would have read yesterday. Honest edges: a run from --html-file stores nothing (one file re-read is not a timeline), an unparseable selector exits 1 rather than reading as zero differences, and "couldn't run" is never conflated with "no history".
MCP server
The 2026 loop closed: the AI agent that heals a scraper can consult the gate before trusting its own heal. PARALLAX ships a zero-dependency Model Context Protocol server over stdio.
| Tool | Arguments | Returns |
|---|---|---|
| check_page | url, optional field or label | the three reads + CONSISTENT / DISAGREE / INCOMPLETE |
| verify_heal | url or inline html, selector, optional field or label/type | outcome, per-mutation results, explanation, suggestion |
| explain_verdict | a verdict object | the one-sentence explanation (pure, offline) |
Full registration and schema details in mcp/README.md.
GitHub Action
The gate as CI: fail a pull request when a selector change would ship wrong-but-plausible data.
The job fails on QUARANTINED, passes with a warning on NEEDS SCOPE (the suggested narrowed selector is surfaced as an output and in the warning), and writes a step summary with the verdict badge, the per-mutation table — latched decoy tokens named — and the explanation. Inputs reach the shell via env, so a PR-controlled selector can't inject commands.
Alerting
Gate events (QUARANTINED, NEEDS_SCOPE, DISAGREE, zero-rows) can POST to any webhook; Slack webhooks get a formatted blocks payload automatically. Alerting is engineered to never take down the pipeline: 5-second timeout, one retry with backoff, and a never-throws contract.
Verdicts & exit codes
| Verdict | Meaning | Exit |
|---|---|---|
| CONSISTENT | All diverse reads agree. (check) | 0 |
| PROMOTED | The healed selector tracked the pinned value through every mutation, including the value-sentinel echo. (verify) | 0 |
| DISAGREE | The eyes read different values — a silent wrong-but-plausible candidate. (check) | 2 |
| QUARANTINED | The heal diverged under mutation: position-locked, wrong-node, or baseline mismatch. The explanation names the failing mutation and evidence. (verify) | 2 |
| INCOMPLETE | An eye could not read the field at all. Fail-closed: investigate, don't trust. (check) | 3 |
| NEEDS SCOPE | The selector reads a different legitimate copy of the same value (multi-copy page). Not wrong today, but unscoped — narrow it to the canonical container. (verify) | 4 |
Non-zero exits make every verdict directly usable as a CI gate.
How the gate works
1 — Pin. The semantic eye locates the true value node by its label ("Sale price: $399") on the original page, and the gate tags that node. Decoys injected later can never re-poison which node is "true".
2 — Mutate. Six meaning-preserving mutations run on fresh copies: identity, reorder-siblings, inject-decoy-no-label, wrap-in-div, duplicate-non-target, value-sentinel. A selector that only got lucky by position diverges here.
3 — Sentinel. The pinned node's value is rewritten to a unique token, and the selector must echo it. This upgrades the check from "same value" to "same node" — a selector reading a different node that coincidentally shares the value (a "Was $1,299" strikethrough) cannot echo the token and is caught.
The verdict is rendered as one factual sentence. The gate certifies consistency with the pinned semantic node under the mutation suite — deliberately not "truth" (see limits).
Bright Data integration
PARALLAX is built around Scraper Studio's lifecycle, driven from the terminal:
| Stage | Command | PARALLAX's role |
|---|---|---|
| create | bdata scraper create <url> "<desc>" | The generated collector emits both eyes per field. |
| run | bdata scraper run / POST /dca/trigger | The conductor scans rows: zero-rows sentinel, per-row agreement. |
| heal | bdata scraper heal <id> "<prompt>" | Break detected → heal requested on the same collector. |
| gate | parallax verify | The metamorphic gate decides: promote or quarantine. |
| approve | bdata scraper approve --auto-save | Only a promoted heal is saved to production. |
Page fetching for check/verify uses the Web Unlocker (bdata scrape -f html) — rendering, unblocking, retries, and geo-routing come from Bright Data, so the gate sees the page a real user sees.
Testing, honestly
The suite currently passes 919 tests across 23 files — but we'd rather you know what that number is made of than be impressed by it:
| Grade | Count | What they are |
|---|---|---|
| behavioral / scenario | ~399 | The real spine — each asserts a distinct behavior: the gate and its verdicts (metamorphic + parity + qa-hardening ≈ 129), the live watch loop with injected effects, replay, MCP driven over real stdio, the SSRF guard, the 9-field + custom-field engines, alerting failure paths. |
| table-driven sweeps | ~520 | Mechanical cross-products guarding the parser and agreement engine (comprehensive_adversarial, property_fuzz, error_paths, golden_fixtures). Every one runs real code, but the insight density is closer to 40–60 distinct equivalence classes than 520 hand-crafted cases. We say so here so the total can't mislead. |
Plus a strict typecheck and the independence lint, all on every push in CI.
Honest limits
A correctness tool that hides its own boundaries isn't trustworthy. These are inherent, stated, and tested:
Consistency, not truth. If both eyes read the same wrong value (a mislabeled page), they agree and the gate promotes. It cannot distinguish "both right" from "both wrong."
Bounded by the semantic eye. Every verdict inherits its correctness from the label anchor. Where a page has no labels, the semantic eye honestly reports it isn't independent — and the gate fails closed rather than pretending.
Diverse, not independent. The eyes share the snapshot and DOM neighborhood; their failures are correlated (Knight & Leveson, 1986). We never claim statistical independence.
Multi-copy values. When the same value legitimately appears in several nodes (buybox + sticky bar), an unscoped selector reads "a" copy, not "the" copy. The gate detects this and reports NEEDS SCOPE with a narrowing suggestion instead of false-vetoing — but it still can't know which copy is canonical without the semantic pin. Found live on our first real target page, and fixed honestly.
JS-rendered pages need browser rendering — and the tool does it. A static fetch of a client-side storefront contains no values; pass --render and PARALLAX fetches the page through Bright Data's cloud browser (executes JS, applies CSS) before reading it. This also catches CSS-class-hidden nodes that static parsing can't see.
vs. the field
From a ~60-source survey of the domain (open source, academic, commercial):
| System | What it verifies after a heal / extraction | Per-extraction correctness? |
|---|---|---|
| Bright Data self-heal | Field is non-null (preview + human approval) | no |
| Scrapling / anansi | A structurally similar element still resolves | no |
| crawl4ai / Firecrawl / ScrapeGraphAI | Output matches the declared JSON schema (shape) | no |
| Great Expectations / Soda / dbt | Column shape & aggregate distribution, post-hoc | no |
| Monte Carlo / Anomalo | Warehouse-level drift vs. historical baselines | no |
| PARALLAX | The healed selector tracks the meaning-pinned node under mutation, with an explained verdict | yes — this gap is the product |
Fair statement of ancestry: dual-read verification and metamorphic testing are decades-old techniques (N-version programming, Chen et al. 1998). PARALLAX's contribution is the fusion — metamorphic testing as an automated post-heal promotion gate for web extraction — which our prior-art search did not find anywhere else.
Roadmap
Shipped: MCP server, GitHub Action, webhook/Slack alerting, the NEEDS_SCOPE verdict, the autonomous watch loop (check → heal → gate → approve on a schedule), blast-radius replay over cached snapshots, and multi-field support beyond price (stock, rating, sku, title, date, brand, discount, currency, plus custom labelled fields). Next: broaden field coverage and normalizers, richer snapshot analytics, and more first-class collector integrations.
Deliberately not building: a hosted dashboard (a static per-run report serves the need) and claims of truth-certification (see limits). The /api service that powers the playground is intentionally thin — the CLI, Action, and MCP server remain the primary consumers.
AI assistance disclosure: built with substantial AI assistance under the author's direction, review, and verification. MIT © 2026 Vinayak Sonthalia.