Browser Workflow integrations
Stream large batch manifests and verify Browser Workflow webhooks.
This guide covers two integration patterns for the Browser Workflow Data API: streaming a large NDJSON manifest and validating webhook deliveries.
Stream a 100,000-item manifest with Node.js
The first line is the batch header. Each following line is an item containing the variables for one workflow run. The request is streamed instead of building a 100,000-item array in memory.
import { randomUUID } from "node:crypto";
import { Readable } from "node:stream";
const baseUrl = process.env.BROWSER_WORKFLOW_API_URL ?? "https://api.runharvester.com";
const apiKey = process.env.BROWSER_WORKFLOW_API_KEY;
const projectId = process.env.BROWSER_WORKFLOW_PROJECT_ID;
const environmentId = process.env.BROWSER_WORKFLOW_ENVIRONMENT_ID;
const workflowId = process.env.BROWSER_WORKFLOW_ID;
const versionId = process.env.BROWSER_WORKFLOW_VERSION_ID;
const idempotencyKey = process.env.RUNHARVESTER_IDEMPOTENCY_KEY ?? randomUUID();
if (!apiKey || !projectId || !environmentId || !workflowId || !versionId) {
throw new Error("Browser Workflow configuration is incomplete");
}
console.log("Idempotency-Key:", idempotencyKey);
function* manifestLines() {
yield JSON.stringify({
type: "batch",
workflow_version_id: versionId,
priority: 1,
configured_concurrency: 4,
}) + "\n";
for (let page = 1; page <= 100_000; page += 1) {
yield JSON.stringify({ type: "item", variables: { page } }) + "\n";
}
}
const response = await fetch(
`${baseUrl}/api/v1/browser/projects/${projectId}/environments/${environmentId}/workflows/${workflowId}/batches/manifest`,
{
method: "POST",
headers: {
"content-type": "application/x-ndjson",
"x-api-key": apiKey,
"idempotency-key": idempotencyKey,
},
body: Readable.toWeb(Readable.from(manifestLines())),
duplex: "half",
},
);
if (!response.ok) throw new Error(`Manifest submission failed: ${response.status}`);
console.log(await response.json());The manifest endpoint validates each line as it arrives and accepts at most 100,000 items. Keep the first line as the batch header and use item lines for workflow variables. For an uncertain submission, set RUNHARVESTER_IDEMPOTENCY_KEY to the printed key and rerun with the same header and items. Generate a new stream for each attempt; a consumed request stream cannot be reused.
Verify a webhook in Python
This Python 3.9+ example checks the signature against the raw request body and enforces a five-minute freshness window before you parse JSON or dispatch the event. The timestamp header is Unix time in milliseconds, not seconds.
import hashlib
import hmac
import time
TIMESTAMP_TOLERANCE_MS = 5 * 60 * 1000
def verify(headers: dict[str, str], body: bytes, signing_secret: str) -> bool:
# HTTP header names are case-insensitive; reject ambiguous duplicates.
def single_header(name):
values = [value for key, value in headers.items() if key.lower() == name]
return values[0] if len(values) == 1 else None
timestamp = single_header("x-browser-workflow-timestamp")
received = single_header("x-browser-workflow-signature")
if not timestamp or not received or not timestamp.isascii() or not timestamp.isdecimal():
return False
if len(timestamp) > 16 or not received.isascii():
return False
now_ms = time.time_ns() // 1_000_000
if abs(now_ms - int(timestamp)) > TIMESTAMP_TOLERANCE_MS:
return False
expected = "v1=" + hmac.new(
signing_secret.encode(), timestamp.encode() + b"." + body, hashlib.sha256
).hexdigest()
return hmac.compare_digest(received, expected)After verification, parse the signed body and check that its id, event, and version match x-browser-workflow-event-id, x-browser-workflow-event, and x-browser-workflow-event-version. Deduplicate using the verified body id, not an unchecked header: deliveries are at-least-once, and a replay retains the same event ID. Persist that ID with the processing outcome before acknowledging a delivery. Store the signing secret securely; the create-webhook response returns it only once.
Last updated on