Webhook-Driven Screenshot Pipelines: Batch, Callback, Store

Single screenshot requests are perfect for a user waiting on one image. For anything bigger, such as a nightly capture of 400 pages, a monitoring run or a client report, the reliable shape is asynchronous: submit batches, let a webhook tell you when they finish, store the results, and retry only what failed. This guide builds that pipeline and covers the details that decide whether it survives a bad night.

The shape of the pipeline

scheduler ──> submit (POST /v1/screenshot/batch, up to 50 URLs) ──> 202 + jobId
                                                                       │
               rendering happens on SnapRender's side, fresh, banners removed
                                                                       │
your endpoint <── POST screenshot.completed (signed) <─────────────────┘
      │
      ├─> GET /v1/screenshot/batch/{jobId}   (items, statuses, download links)
      ├─> copy each completed capture into your storage
      └─> queue failed URLs for one retry

Every arrow is a short HTTP call. Nothing waits on an open connection while pages render, so the pipeline behaves the same for 50 URLs or 5,000.

Facts from the SnapRender batch API that shape the design:

• Up to 50 URLs per request; larger lists become several jobs.
• Each job renders its pages one after another on SnapRender's side, always fresh.
• Completed items carry a downloadUrl valid for 24 hours. Copy results into your own storage when the webhook arrives.
• Failed items carry an error and are refunded to your quota.
• The screenshot.completed webhook fires once per job, with the job id and counts.

1. Register the webhook

curl -s -X POST "https://app.snap-render.com/v1/webhooks" \
  -H "X-API-Key: $SNAPRENDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://pipeline.example.com/hooks/snaprender", "events": ["screenshot.completed"]}'

Keep the secret from the response. An account can have up to five webhooks, which is enough for separate staging and production endpoints. POST /v1/webhooks/{id}/test sends a test delivery when you want to check the endpoint.

2. Submit in chunks of 50

# submit.py
import os, requests

API = "https://app.snap-render.com"
HEADERS = {"X-API-Key": os.environ["SNAPRENDER_API_KEY"]}

def submit(urls, **options):
    job_ids = []
    for i in range(0, len(urls), 50):
        r = requests.post(f"{API}/v1/screenshot/batch", headers=HEADERS, timeout=30,
                          json={"urls": urls[i:i + 50], **options})
        r.raise_for_status()
        job_ids.append(r.json()["jobId"])
    return job_ids

urls = [line.strip() for line in open("urls.txt") if line.strip()]
print(submit(urls, format="png", full_page=True))

Options apply to the whole job: format, size, full_page, device, dark_mode, hide_selectors and the rest. If you need desktop and mobile, submit the list twice with different device values.

Batch requests reserve quota for every URL up front and refund failures when the job finishes, so a job never overruns your plan halfway through.

3. Verify, acknowledge, then work

The handler has one rule above all others: answer fast. Deliveries time out after 10 seconds. Verify the signature, return 200, and do the downloading after the response is sent or on a queue.

Python with Flask:

# hooks.py
import hashlib, hmac, json, os, threading
from flask import Flask, request, abort

app = Flask(__name__)
SECRET = os.environ["SNAPRENDER_WEBHOOK_SECRET"].encode()

@app.post("/hooks/snaprender")
def snaprender_hook():
    raw = request.get_data()  # the exact bytes that were signed
    expected = "sha256=" + hmac.new(SECRET, raw, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(expected, request.headers.get("X-SnapRender-Signature", "")):
        abort(401)
    job_id = json.loads(raw)["jobId"]
    threading.Thread(target=process_job, args=(job_id,), daemon=True).start()
    return "", 200

Node.js with Express is the same idea; use express.raw({ type: 'application/json' }) on the route so the body is not parsed and re-serialized before you hash it. Hashing a re-serialized JSON object is the most common reason signature checks fail.

The event type also arrives in the X-SnapRender-Event header, which is useful if one endpoint receives several kinds of events.

4. Make processing idempotent

Retries mean the same job can be delivered more than once: if your endpoint returned a 500 or took longer than 10 seconds, SnapRender tries again after 1, 5, 15 and 60 minutes. Your handler must do no harm the second time.

import pathlib, requests

API = "https://app.snap-render.com"
HEADERS = {"X-API-Key": os.environ["SNAPRENDER_API_KEY"]}
DONE = pathlib.Path("jobs-done")
DONE.mkdir(exist_ok=True)

def process_job(job_id: str):
    marker = DONE / job_id
    if marker.exists():
        return  # already handled: a retry of a delivery we processed
    job = requests.get(f"{API}/v1/screenshot/batch/{job_id}", headers=HEADERS, timeout=30).json()
    failed = []
    for index, item in enumerate(job["items"]):
        if item["status"] != "completed":
            failed.append(item["url"])
            continue
        data = requests.get(item["downloadUrl"], timeout=60).content
        store(f"{job_id}/{index:03d}.png", data, source_url=item["url"])
    if failed:
        queue_retry(failed)
    marker.touch()  # only after everything is stored

Two choices make this safe. Object keys derived from the job id and item index mean a second pass overwrites identical files instead of creating duplicates. The "done" marker is written last, so a crash halfway leaves the job unmarked and a retry finishes it. In production, keep the marker in your database instead of on disk.

5. Retry failures once, then report them

Failures cluster into two kinds. Transient ones, such as a timeout or a site briefly down, often pass on a second try. Persistent ones, such as a site that blocks automated browsers, fail every time. Retry each failed URL once in a new batch, then report whatever fails twice instead of looping. Failed items are refunded either way, so a retry costs nothing unless it succeeds.

6. Do not let a dead endpoint go unnoticed

After five failed delivery attempts the webhook is switched off. A deploy that breaks your endpoint on a Friday can therefore silently stop the pipeline. Three cheap safeguards:

• Check webhook health. GET /v1/webhooks lists your webhooks with an isActive field; alert if the production one is false.
• Keep a polling fallback. Store every job id at submission time. A job that has not been processed an hour later can be fetched with GET /v1/screenshot/batch/{jobId} while the job and its links are still available.
• Store results quickly. Job details and download links last 24 hours, so a fallback that runs within a few hours loses nothing.

When single requests are the better fit

Batches add moving parts. If a user is waiting for one image, or you capture a few pages per run, a plain GET /v1/screenshot that returns the image in the response is simpler and just as fast. Batches pay off once runs are scheduled, lists are long, or you want the work to happen without holding connections open.

Related guides

• Daily screenshots of 100 URLs with GitHub Actions, the polling version of this pipeline.
• Storing website screenshots in S3 and R2, for the store() side.
• Competitor pricing monitoring in Node.js, a complete webhook handler in Express.
• The batch screenshot API and webhooks reference pages.

To try a list of URLs before writing any code, the bulk screenshot tool captures five without an account.