Product data API: a practical buyer and builder guide
Learn what a product data API should return, how to model variants and offers, and how to benchmark coverage, freshness, latency, reliability, and cost.
An endpoint that returns a title, price, and image is easy to call. A production product data API must preserve identity across normalization: a group stays linked to its variants, variants to their offers, and volatile values to source and observation time.
Catalog provides a structured product-data layer for AI and agentic commerce. You get the record model, contract questions, and a repeatable evaluation scorecard. For source mechanics, see our product data extraction and price monitoring API explainers.
A product data API is a source-to-record contract
A product data API accepts a product URL, catalog identifier, feed, or search request and returns a typed product record. The record should be safe to store, compare, join, and pass to another system without parsing page markup again.
“Typed” means that a price has an amount and currency, availability uses a documented value set, timestamps have a stated time zone, and an array is different from a free-form string. “Identity-preserving” means the response keeps stable IDs and relationships while it normalizes names, units, and values.
The API contract covers the whole interaction. It includes authentication, request parameters, response schemas, pagination, asynchronous execution, null behavior, errors, rate limits, versioning, retries, and cost. An endpoint description that says only “send a URL and get JSON” leaves the parts that break integrations undefined. An OpenAPI description is a useful baseline because it makes operations, parameters, content types, and schemas inspectable.
A product data API can sit between several kinds of source:
| Source or interface | What it usually returns | When it fits |
|---|---|---|
| First-party catalog API | A merchant’s canonical products, variants, inventory, and offers | You control the store or have an authorized integration |
| Product feed or export | A scheduled snapshot in a file or feed schema | You need bulk loads and can tolerate scheduled updates |
| Product data API | Normalized records from one or many source systems | You need a stable shape across merchants, platforms, or public URLs |
| HTML or scraper output | Page markup and parser-specific fields | You have no structured source and own the parser and its upkeep |
A product data API earns its place when the record is more useful than the page that produced it. The test is whether a downstream developer can rely on field types, identity links, provenance, and failure states without reverse-engineering the response.
Model the record at the right grain
The word product can mean a product family, a sellable option combination, or a seller’s offer. Pick a grain before you compare APIs. A practical model has three levels:
| Level | Meaning | Fields that belong here |
|---|---|---|
| Product group or family | The shared concept, such as a running shoe model | Group ID, title, brand, shared description, category, and the dimensions that vary |
| Variant | One exact option combination, such as size 9 in black | Variant ID, parent group ID, option values, SKU, GTIN or MPN, variant media, and variant-specific attributes |
| Offer | A seller’s proposition for one variant in a market | Offer ID, seller, channel, country, price, currency, availability, condition, fulfillment, and observation time |
The group describes what the item is. The variant describes which sellable configuration it is. The offer describes who is selling that configuration, where, and under what commercial terms.
A group can have one default variant. A variant can have several offers. A product data API may flatten these objects for convenience, but it should still expose parent IDs and offer-to-variant links so you can reconstruct the relationships.
This separation matches the shape of established commerce vocabularies and APIs. Shopify models a Product with ProductVariant records, while Google Merchant uses an offerId and an itemGroupId to identify an offer and link related variants. Schema.org separates ProductGroup, hasVariant, variesBy, and offers. Each model is platform-specific. Together, they show why a parent title with a bag of option strings is insufficient. See the Shopify Product, Shopify ProductVariant, Google Merchant product resource, and Schema.org ProductGroup definitions for the underlying semantics.
A canonical response can preserve those relationships without forcing every provider into the same vocabulary:
{
"schema_version": "1.0",
"record_id": {
"value": "rec_123",
"namespace": "provider"
},
"source": {
"url": "https://shop.example/products/runner",
"observed_at": "2026-09-30T12:04:05Z"
},
"product_group": {
"id": "grp_456",
"title": "Example Runner",
"brand": "Example"
},
"variants": [
{
"id": "var_789",
"group_id": "grp_456",
"options": {"color": "black", "size": "9"},
"identifiers": {
"sku": "RUN-BLK-9",
"gtin": "00012345678905"
},
"offers": [
{
"id": "off_001",
"variant_id": "var_789",
"seller": "Example Shop",
"price": {"amount": 129.0, "currency": "USD"},
"availability": "in_stock",
"observed_at": "2026-09-30T12:04:05Z"
}
]
}
],
"provenance": {
"method": "source_observation",
"processed_at": "2026-09-30T12:04:08Z"
},
"warnings": []
}This is an illustrative canonical shape, not a vendor schema. The important properties are explicit relationships, typed money, namespaced identifiers, and separate source and processing timestamps.
Keep identifier namespaces explicit
Identifiers answer different identity questions. Store the value, its namespace, the issuer or source, and the scope in which it is unique.
| Identifier | Usually owned by | What it answers | Safe use |
|---|---|---|---|
| Provider or database ID | API provider or merchant platform | Which record does this system store? | Joins inside that provider’s dataset |
| Source URL | Storefront or marketplace | Where was this record observed? | Traceability and re-fetching, with redirect history |
| SKU | Merchant or seller | Which sellable item does this merchant manage? | Merchant-scoped inventory and operations |
| GTIN | Brand owner under GS1 rules | Which trade item is this? | Cross-seller matching when valid and available |
| MPN | Manufacturer | Which model or part is this? | Manufacturer-level matching with brand context |
| Offer ID | Merchant, feed, or marketplace | Which seller/channel offer is this? | Offer-level updates and deduplication |
| Group or parent ID | Catalog or normalization layer | Which variants belong together? | Variant grouping and shared attributes |
A SKU can be reused by two sellers. A URL can change. A provider ID can be meaningful only inside one tenant. A GTIN identifies a trade item and is not a substitute for a store SKU. GS1’s GTIN definition and Google’s identifier guidance are useful guardrails: preserve manufacturer-assigned identifiers and never invent a missing value.
Do not join records on title, URL, or a bare SKU across merchants. Keep each identifier in a typed namespace and make the matching decision explicit. A good API returns the original values alongside any normalized value so you can audit a merge.
Read the API contract before writing code
The response shape is only one part of the integration. Ask for the following behavior in the contract and test fixtures.
| Contract area | What to demand | Why it changes your implementation |
|---|---|---|
| Authentication and scope | Header or token scheme, scopes, key rotation, tenant and environment boundaries | Determines secret storage, least privilege, and deployment separation |
| Schema and types | OpenAPI or JSON Schema, required fields, enums, units, currency, time zone, and compatibility rules | Prevents silent parsing and unit errors |
| Nulls and omission | Meaning of null, missing fields, empty arrays, empty strings, and disabled features | Lets you distinguish “not observed” from “not applicable” or “failed” |
| Pagination | Cursor or page token, stable ordering, page size, continuation expiry, and duplicate behavior | Prevents dropped or repeated records in large results |
| Asynchronous jobs | Accepted response, execution ID, states, polling or callback rules, cancellation, result retention, and partial results | Separates submission from completion and gives workers a safe state machine |
| Errors | HTTP status, machine-readable code, request ID, item-level errors, and retryability | Makes failures observable and retry logic safe |
| Rate limits | Quotas by key, tenant, or route; burst and concurrency rules; limit headers; Retry-After | Lets you throttle before work fails and budget capacity |
| Versioning | Version location, backward-compatibility policy, deprecation window, and schema-change notice | Keeps generated clients and stored records usable over time |
| Idempotency | Idempotency key support, retention, deduplication scope, and replay result | Prevents duplicate jobs and duplicate charges after a timeout |
| Cost | Unit of billing, treatment of failed or empty records, refresh cost, and enrichment cost | Gives you cost per usable record rather than cost per request |
For pagination, prefer a cursor or page token tied to a stable snapshot. An offset can shift when a catalog changes between calls. For asynchronous work, model states such as queued, running, completed, completed_with_errors, failed, and expired. Store the execution ID and request ID with every result.
HTTP semantics help set sensible defaults. 429 Too Many Requests indicates rate limiting, and the Retry-After header can communicate when a client should try again. Read the HTTP semantics and 429 guidance, then use the provider’s documented policy for the final retry decision.
A timeout does not prove that a POST failed. Unless the API documents idempotency keys or another deduplication mechanism, treat a POST retry as potentially unsafe. If idempotency is supported, send a stable key derived from your logical job, persist it, and confirm whether a replay returns the original result or creates a new execution.
Contract questions to answer before integration
- Can the current schema generate a client without hand-written field guesses?
- Which fields are required for a usable record, and which are optional by category or source?
- How are money, units, locale, time zone, and availability values represented?
- Can a response contain a valid record and item-level failures at the same time?
- How do you retrieve a large result, and what happens if a page token expires?
- Which status codes and error codes are safe to retry?
- How do you correlate a request, execution, source URL, and stored record?
- What changes require a new API version, and how are deprecations announced?
- What is billable when a job returns no usable record or only a partial result?
Catalog’s public API page shows a concrete extraction interface with POST /v2/extract, x-api-key authentication, URL input, country selection, and explicit enrichment, review, and image-tag flags. That level of explicitness is useful for downstream builders because a feature flag changes the meaning of a missing field. Treat batch support, rate limits, latency targets, freshness, uptime, and accuracy as contract questions rather than assumptions from a sample request.
Freshness has two clocks
A record can arrive quickly and still describe an old page. It can also take time to process a source change and eventually return a current value. Track these clocks separately:
- Source age: how long since the value was observed at the source.
- Processing duration: how long the provider spent fetching, parsing, normalizing, and enriching.
- Output lag: how long between a source change and the first usable API result that contains it.
- Serving age: how long since the API stored or published the result you are reading.
Use timestamps at the field or offer level when a value changes independently. A record-level updated_at cannot tell you whether a price, image, or availability value is current. A source-reported update time also has provider-specific semantics. For example, Shopify exposes a product updatedAt field, but a platform timestamp can change for inventory or administrative reasons. Keep the provider timestamp and your own observation timestamp instead of treating one as universal truth.
A simple measurement set is:
source_age = api_read_time - source_observed_at
processing_duration = output_ready_at - fetch_started_at
output_lag = output_ready_at - controlled_source_change_atMeasure price and availability on a shorter acceptable-age window than stable descriptions or dimensions. If you cannot create a controlled source change, report observed source age and processing duration separately. Do not turn a sample response duration into a freshness promise. Catalog’s product data quality guidance has adjacent advice on freshness lag and field-level quality checks.
Separate source facts, normalization, and enrichment
A normalized value can preserve the source fact while changing its representation. An enriched value adds a classification, attribute, relationship, or description that was not directly present in the source. Those layers need different trust and review rules.
| Layer | Example | Minimum provenance |
|---|---|---|
| Source-observed | The page says water resistant and shows $129.00 | Source URL, observed time, raw or captured evidence |
| Normalized | water resistant maps to water_resistance: true; $129.00 becomes amount 12900 in USD cents | Source field, transformation rule or unit, processor version |
| Enriched or inferred | A category model assigns trail_running; an AI system proposes a material | Method or model, production time, evidence, confidence, and review state |
Keep source and enriched values addressable separately. A downstream agent may accept an inferred category for discovery while requiring source evidence for a safety, compatibility, or regulated claim. Catalog’s product data enrichment guide covers the broader enrichment workflow; the contract question here is whether each value tells you how it was produced.
Null behavior needs the same discipline. A documented null might mean “source did not provide this value.” In another API, it might mean an option was disabled. An omitted field might mean the schema version does not support it, while an empty array might mean an observed collection is empty. An error state can look identical if the API does not expose per-field or per-item status.
Require a documented distinction among at least these states:
- Not observed: the source did not expose the field.
- Not applicable: the field does not apply to the record.
- Disabled: the request omitted or disabled the feature.
- Unsupported: the source or provider cannot produce the field.
- Failed: processing attempted the field and did not complete.
- Empty: the source explicitly returned an empty collection or value.
Never treat null as a safe default for zero, false, or an empty list. Preserve the status so consumers can choose whether to hide, retry, or review the field.
Evaluate a product data API with a reproducible scorecard
A demo proves that one URL works. A benchmark should show how an API behaves across the catalog you intend to operate.
Start with a representative set of 100 to 300 URLs or identifiers, adjusted to your budget and catalog size. Stratify the set by source platform, category, country and language, simple and multi-variant products, seller count, and unavailable or out-of-stock cases. This range is a practical starting point, not an external standard.
Build the test
- Define success. Write the required fields for each category and what counts as a usable record. Mark price, currency, availability, and identity fields as hard gates if a downstream workflow depends on them.
- Capture ground truth. Manually record the source value and timestamp for the fields you will use. Capture each variant and offer that matters, not only the parent title.
- Run the same configuration. Use the same input set, region, locale, flags, output version, concurrency, and retry policy for every candidate.
- Repeat after a controlled change. Change a test price or availability value where you control the source, then measure when the correct field appears.
- Retain raw evidence. Store request IDs, response bodies, status codes, job states, page tokens, timestamps, and billable units. Redact secrets.
- Report slices. Publish results by platform, category, market, variant complexity, and outcome. A single average hides the failures your production catalog will expose.
Score the dimensions
| Dimension | Measure | Decision signal |
|---|---|---|
| Coverage | Eligible inputs that produce a usable record, split by source and category | Shows where the API works and where it returns unsupported or empty results |
| Accuracy | Exact or normalized correctness for price, currency, brand, specs, image, and availability | Separates plausible text from values safe to use |
| Completeness | Required-field and category-attribute coverage per record | Shows whether records can power the intended workflow |
| Identity and variants | Stable group and variant IDs, identifier preservation, parent links, option values, and variant-specific offer fields | Catches merges, splits, and parent-level data incorrectly copied to variants |
| Freshness | Source change to correct output, plus source age and serving age by field | Sets refresh cadence from observed behavior |
| Enrichment | Precision and coverage on labeled inferred values, with evidence and confidence | Shows whether enrichment adds usable facts or review work |
| Submit and end-to-end latency | p50, p95, and p99 from submit to completion, segmented by batch and source | Exposes tail behavior that an average conceals |
| Reliability | HTTP errors, job completion, timeouts, schema validation, retry outcomes, and request-ID availability | Shows whether the pipeline can operate and debug failures |
| Partial failures | Per-input status, error code, retry scope, and duplicate behavior in mixed-result jobs | Determines whether one bad URL blocks a whole batch |
| Cost | Total spend divided by usable records and by correct usable records, including refreshes, enrichment, and retries | Connects price to data that actually reaches production |
Use hard gates before a weighted score. If a shopping agent cannot identify a variant or trust its currency and availability, a strong description score does not compensate. For the remaining dimensions, report rates and distributions with the test conditions beside them.
A small set of formulas keeps the benchmark reproducible:
usable_record_rate = usable_records / eligible_inputs
required_field_accuracy = correct_required_fields / labeled_required_fields
correct_record_rate = records passing all hard gates / eligible_inputs
cost_per_correct_record = total_cost / correct_recordsReport p50 and tail percentiles for latency rather than an average alone. Service-level objective guidance explains why a high percentile captures the slow requests that shape user experience. The same logic applies to extraction and async completion.
Choose the right product-data layer
The right choice depends on source authority, source count, and how much operational behavior you want to own. When a structured layer is the need, Catalog is our recommended fit for a product-data workflow because it focuses on normalized records, synchronization, distribution to AI surfaces, and outcome measurement.
| Option | Best fit | What you own |
|---|---|---|
| Catalog | You need a structured product-data layer across heterogeneous product sources and AI shopping surfaces | Your source policy, required fields, acceptance gates, and downstream use of the records |
| First-party API or feed | You own the catalog or have an authorized merchant integration | Source credentials, platform-specific semantics, and feed or webhook operations |
| Build and maintain extraction | You have a narrow source set and want full control of parsers and storage | Fetching, parser changes, identity resolution, freshness, retries, and data quality |
The Catalog API shows the pattern plainly: a versioned extraction route that accepts product URLs and explicit options, then returns machine-ready product data. The value is the layer around extraction. Your application can keep identity and provenance with the record, apply the scorecard above, and publish the accepted data to AI commerce workflows.
Use product data quality for operating field-level checks, product data syndication for distribution concerns, and trusted data sources for the broader source and agent architecture. The focus here is making the API contract and returned record testable.
FAQ
Is a product data API the same as a product catalog API?
A product catalog API usually exposes one merchant’s own catalog and may support creating or updating products. A product data API can normalize records from several sources, including public product pages, and often emphasizes read access, identity resolution, provenance, and cross-source consistency. The names overlap, so compare the input sources and response semantics rather than the endpoint label.
Should price live on the product, variant, or offer?
Put price on the lowest level where it can change independently. A single canonical price can live on a variant. A seller-, market-, currency-, or fulfillment-specific price belongs on an offer. Keep the currency and observation time with the amount, and avoid copying an offer price to every parent record without the seller and market context.
What does a successful HTTP response prove?
It proves transport-level success. It does not prove that every input produced a usable record, that a price is current, or that an asynchronous job has completed. Track service status and data status separately, including per-input failures inside a successful batch response.
How fresh should product data be?
Set the age window from the decision the data supports. Price, availability, inventory, delivery, and promotions usually need tighter windows than descriptions, dimensions, or stable specifications. Measure source age, processing duration, and source-change-to-output lag separately so a fast response does not hide stale data.
Can enriched attributes be treated as source truth?
Only when the provider gives you evidence and your own validation accepts the field for that use. Keep source-observed, normalized, and enriched values separate. Store the method, production time, confidence, and review state for inferred values, then apply stricter gates to compatibility, safety, and regulated claims.
How large should an evaluation set be?
Begin with a stratified set large enough to include your major platforms, categories, markets, variant patterns, and unavailable cases. One hundred to 300 inputs is a practical first pass for many teams. Expand the set when a long tail, high failure cost, or new source segment changes the decision.
If you need a structured product-data layer that your developers and AI commerce workflows can build on, see how Catalog works.
