Structured extraction
Return typed fields from a page instead of parsing raw HTML yourself.
Structured extraction lets a Data API request return a typed data object alongside the page result. Define the fields you need, or reuse an existing recipe for a repeatable extraction.
Choose an extraction mode
Reuse a recipe
Pass an owned, active recipe when the same extraction should run repeatedly:
{
"url": "https://example.com/catalog",
"mode": "browser",
"extraction": {
"recipe_id": "<recipe-uuid>"
}
}Define fields in the request
Automatic extraction accepts a list of scalar fields and object collections. Optional instructions provide context for ambiguous pages; recipe_name gives the generated recipe a readable name; auto_repair_enabled defaults to true.
{
"url": "https://example.com/catalog",
"mode": "browser",
"proxy": "random",
"extraction": {
"fields": [
{ "name": "title", "type": "string", "required": true },
{ "name": "price", "type": "number", "required": true },
{ "name": "tags", "type": "string", "multiple": true, "required": false },
{
"name": "offers",
"type": "object",
"multiple": true,
"fields": [
{ "name": "seller", "type": "string" },
{ "name": "url", "type": "url" }
]
}
],
"instructions": "Read the product details shown on the page.",
"recipe_name": "Product catalog",
"auto_repair_enabled": true
}
}Field rules
| Field | Meaning |
|---|---|
string, number, integer, boolean, date, url | Scalar output types. |
required | Whether the field must be present; defaults to true. |
multiple | Whether a scalar returns an array; defaults to false. |
type: "object" | A collection of objects. It must set multiple: true and provide fields. |
description | Optional field-specific guidance, up to 1,000 characters. |
Collection fields may contain scalar fields only. Nested object collections are not supported. The combined number of top-level and nested fields must not exceed 200; see limits.
Field names must be unique within their level. The request body is strict: unknown extraction properties are rejected instead of being ignored.
Read the result
Poll the request as usual and fetch /requests/{id}/result only after the request is complete. An extraction result includes:
data: the requested structured shape;extraction: the extraction run state, recipe identity, and any public error information;- the normal
html,text, and result metadata when available.
Validate required fields and types in your application before storing or using data. A completed page acquisition does not remove the need to validate the extracted payload.
Structured extraction is supported on an individual Data API request. Data API batches accept raw scrape jobs only; put extraction requests through separate submissions. For sources inside reusable recipes, see JSON sources.
Last updated on