Data freshness is a product-data operating metric

A product can be current in your commerce platform and already stale where a shopper sees it. A price change may reach the product page while an ad feed still carries the old value. An inventory update may enter a warehouse system but miss a variant-level API response.

Data freshness gives merchants and ecommerce data teams a way to see that gap. Define it at the field and destination level, measure the handoffs, and trace a change from source to destination.

What data freshness means for a product record#

Data freshness is how current a product field is for the system or person using it. IBM's data freshness definition ties the concept to how recently information was updated. For a product-data team, the useful question is specific:

How old is the value for this product, variant, or offer at this destination, compared with the source state it should represent?

That question has three parts:

• Field: price, availability, inventory, variant structure, title, image, or another attribute.
• Entity: product group, sellable variant, or offer in a market and channel.
• Destination: storefront, checkout, feed, marketplace, API, or AI shopping surface.

A parent product can be fresh while one size or color is stale. A product page can show the right price while an accepted feed row shows yesterday's price. Freshness belongs to the field and destination, not to a catalog-wide status badge.

A useful product record carries identity and time context. Keep stable product and variant IDs, the source's effective or updated timestamp, and the timestamps recorded as the value moves through your product-data layer. Shopify's InventoryItem model is one example: it represents a product variant, has inventory levels by location, and exposes an updatedAt timestamp. Shopify's inventory documentation shows why a single product-level stock number can hide the state that matters. A scheduled job can publish zero changed rows, and a destination can accept an old payload, so freshness measures the state downstream systems receive and use.

Freshness, latency, and data quality are different signals#

These terms describe related parts of the same flow, so teams often use them as synonyms. Keep them separate when you instrument product data.

Freshness is a standing property of the current record. Latency is an event measurement. A feed can have low latency for every update it receives and still be stale because the connector stopped receiving updates. A batch export can have high latency and still be fresh enough for a weekly content field.

Fresh data can also be wrong. If a source writes $9.99 instead of $99.99, the wrong price can reach every destination quickly. Data-quality checks still need to test accuracy, completeness, consistency, validity, uniqueness, and usefulness. Our product data quality guide lays out those dimensions and the checks that sit beside freshness.

Freshness age needs context too. A product description that has been unchanged for six months may be healthy. A quantity that has not been observed for six hours during a busy promotion may be risky. “Stale” is a decision against an objective, not a property of every old timestamp.

Set freshness objectives by field and destination#

A single refresh cadence for an entire catalog creates two problems. It makes low-risk content expensive to refresh, and it leaves high-risk offers under-instrumented. Set an objective for the field, entity grain, destination, and business window that create the customer promise.

Start with the field's risk#

Google's product data specification requires price and availability to match the landing page, structured data, and checkout. It also defines variants through attributes such as item group, color, and size. Google's product data requirements are a useful destination contract, even when Google is only one of your channels.

Then add the destination's behavior#

The same field may need different measurements at different destinations:

• Storefront and checkout: Verify the exact variant a shopper can select and purchase. A cache or pricing rule can make the storefront differ from the source.
• Feeds and marketplaces: Record when the output was generated, fetched, processed, and accepted. A file sitting in storage is not a delivered offer.
• Product APIs: Expose source and processing timestamps with stable IDs, typed money, and availability values. A caller needs to know whether a response is current enough for its use.
• AI shopping surfaces: Keep live, normalized product data and offer context available to machine readers. A product-data layer should preserve variant relationships and commercial state as it publishes typed product objects.
• Warehouse and analytics systems: Measure arrival against the decision they support. A reporting table may need a load watermark, while an inventory decision needs the current variant and location state.

Use service-level indicators (SLIs) to measure these behaviors, service-level objectives (SLOs) to set an internal target, and a service-level agreement (SLA) only when you need a formal promise between parties. Google's SRE guidance defines an SLO as a target measured by an SLI, and points out that the right value depends on the service and its users. The SRE explanation of SLOs is a useful guardrail against choosing a number first and a business reason later.

There is no universal ecommerce freshness SLA. The right objective depends on the field, destination, change pattern, and cost of a stale customer promise.

A field-and-destination objective can look like this:

For availability on variant × market in destination D, at least X% of accepted source changes reach the destination within W during the operating window, and the stale-offer rate stays below R.

Choose X, W, and R with the teams that own the customer promise. A campaign, flash sale, or fulfillment constraint can change the objective for a defined window. That is more useful than labeling every field “real time.”

Measure freshness age, propagation lag, and target attainment#

Instrument the clocks before you choose alerts. At minimum, preserve these timestamps for each meaningful field change:

1. source_changed_at: when the source made the value effective.
2. ingested_at: when your integration received the source version.
3. normalized_at: when the product-data layer validated and mapped it.
4. published_at: when a feed row or API object was made available to the destination.
5. destination_accepted_at: when the destination confirmed processing or acceptance.
6. observed_at: when you read back the destination or customer-facing page.

Use one clock and timezone, preferably UTC. Attach a stable product or offer ID and a source version to every event. A timestamp without identity cannot tell you which variant is stale.

Freshness age#

At a destination read time, calculate the age of the value the destination currently exposes:

freshness_age = observed_at - source_changed_at

If the source did not change, age the value from its last effective source timestamp and compare it with the field's expected update pattern. Do not substitute a file's modified time and call it source freshness. A job can rewrite an unchanged file, or a source can change a value without changing a container's metadata.

For dataset-style monitoring, a load watermark is useful:

dataset_age = now - latest_successful_source_or_load_timestamp

The timestamp must match what you are trying to prove. dbt's freshness configuration uses a loaded_at_field or query to identify load time and apply warning or error thresholds. dbt's freshness documentation is an implementation example, not a universal ecommerce target.

Propagation lag#

For a single change, measure the time between the source event and each handoff:

end_to_end_lag = destination_accepted_at - source_changed_at
stage_lag      = next_stage_timestamp - prior_stage_timestamp

Track a distribution, such as p50, p95, and the maximum over a defined window. The average hides the few price or availability changes that wait behind a failed retry. Keep rejected, retried, and out-of-order events in the same trace so a successful later update does not erase the incident.

A worked example:

At 14:10, the destination value has a freshness age of 10 minutes, and accepted propagation lag is 7 minutes 40 seconds. The 2 minutes 20 seconds between acceptance and observation belongs to the destination's serving or cache behavior. Keep that delay visible instead of charging it to ingestion.

Stale-offer rate#

Measure stale offers at the grain shoppers use. For a defined destination, market, and time window:

stale_offer_rate = stale eligible offers / eligible offers × 100

An offer is stale when its destination value is older than the field objective, fails to match the current source value, or has no accepted update after a qualifying change. Compare the same variant, market, currency, and inventory location. Comparing a parent product to a variant can make a bad join look like a freshness problem.

For example, if 14 of 2,000 eligible variant offers exceed the agreed objective, the stale-offer rate is 0.7%. Report the rate by field and destination, then show the affected IDs. A catalog-wide average can hide one channel or one high-volume SKU family with a serious problem.

Target attainment#

Target attainment answers whether the system met its objective over a window:

target_attainment = qualifying changes within objective / eligible changes × 100

If 9,850 of 10,000 eligible availability changes reached an accepted destination within the objective, attainment is 98.5%. Pair this percentage with the objective, the operating window, and the exclusions. A number without its denominator or clock is a dashboard decoration.

AWS Glue's DataFreshness rule evaluates a date column against a time expression and handles nulls explicitly. The AWS rule reference is a useful implementation example; your product fields and destination contracts determine what a pass means.

Trace a price or availability change from source to destination#

A freshness metric becomes operational when it can lead you to the failing handoff. Use a single variant and follow its identity through every representation.

1. The source records the change#

Suppose the black, size 9 variant RUN-BLK-9 changes from $129.00 to $119.00, or from in_stock to out_of_stock, at 14:00 UTC. The commerce platform, ERP, or inventory system writes the new value, currency or availability state, variant ID, and source_changed_at. If stock is location-specific, include the inventory location and the rule that rolls it up to a sellable offer.

The source owns the business value. A downstream feed should not become a second price authority. Set a version or sequence number so a delayed $129.00 event cannot overwrite the newer value.

2. The integration detects and validates it#

A webhook, connector, poll, or scheduled file brings the change into the product-data layer. Catalog keeps the parent, variant, and offer relationships intact while it normalizes fields for downstream use. It validates the currency, accepted availability value, identifier, and required context before publishing.

If ingestion never happens, inspect the source connector, permissions, schedule, and event cursor. If the source version arrived but the normalized record is old, inspect mapping, validation, and retry state. Each clock narrows the owner of the failure.

3. The feed or API publishes an output#

Catalog emits the destination representation, such as a feed row or typed API object. The output keeps the same variant identity and carries the new offer value plus processing and publication timestamps. A destination-specific mapping can change field names or enum values, but it should not change which variant the offer belongs to.

Our product data integration guide covers ownership, mappings, identifiers, validation, and incremental updates. Our product data API guide goes deeper on typed money, offer relationships, and source versus processing timestamps.

4. The destination accepts and serves it#

A successful upload is an intermediate event. Record the destination's acceptance or processing result, then read the value back from the feed, API, marketplace, product page, and checkout path that matter.

For Google Merchant Center, price and availability must agree across the product data, landing page, structured data, and checkout. Google's automatic updates can detect some mismatches from a product page, yet they are a safety net for limited fields. Google's automatic-update guidance still requires accurate source data and structured markup where applicable. Do not use automatic correction as your freshness monitor.

Our Google Merchant Center feed guide covers the separate stages of fetch, processing, approval, and freshness when you operate that destination.

If the destination accepted $119.00 but the product page still shows $129.00, the offer is inconsistent even though the feed pipeline is fresh. If the page is current but the destination still exposes $129.00, investigate destination processing, cache behavior, or an ID mismatch. If the size 9 offer shows size 8's stock, investigate the variant join before changing the cadence.

5. Verify, alert, and replay#

Close the trace only after a read-back confirms the expected value at the expected grain. Store the evidence with the source version:

• expected value and source timestamp;
• destination value and observed timestamp;
• acceptance or rejection response;
• affected market, currency, and location;
• owner and remediation status.

Then replay the corrected event or feed row. Recalculate propagation lag, stale-offer rate, and target attainment after the destination catches up. A retry count tells you that work happened. A read-back tells you that the customer promise was restored.

A practical freshness operating model#

Use this sequence to turn freshness into a repeatable product-data practice:

1. Inventory the paths. List each source, field, entity grain, transformation, feed or API, destination, page, and checkout path.
2. Assign owners and objectives. Name owners for values, mappings, and incidents. Choose an age limit, propagation window, stale-offer tolerance, operating hours, and attainment target for each high-risk field and destination.
3. Record lineage and clocks. Carry stable IDs, versions, source and processing timestamps, acceptance results, and read-back timestamps through the path.
4. Alert and review. Alert on missed source changes, stage lag, rejected outputs, parity failures, and stale-offer rate. Break results down by destination, field, variant family, country, and location.
5. Replay and verify. Make corrections recoverable. Confirm the value where a shopper, channel, or shopping agent reads it, then recalculate the metrics.

Catalog is built for the middle of this operating model. We help brands structure product information into live, normalized product data, keep records synchronized as source values change, and publish machine-readable product objects to AI shopping surfaces. Your storefront, PIM, ERP, DAM, warehouse, and channel systems can keep their ownership while Catalog gives the product-data layer one place to preserve identity, provenance, field context, and downstream delivery signals.

Frequently asked questions#

How often should ecommerce product data refresh?#

Set the cadence from the field and destination risk. Price, availability, inventory, sale windows, and delivery promises need an objective tied to how quickly they can change and how much a stale value costs. Descriptions and technical attributes can follow approval or launch workflows. Measure event-driven updates where changes are irregular, and scheduled freshness where the source is batch-based.

Can product data be fresh and still low quality?#

Yes. Freshness says when a value arrived. It does not prove that the value is accurate, complete, valid, or joined to the right variant. Run freshness checks alongside data-quality checks, then verify the customer-facing page and destination contract.

Which timestamp should a data team use for freshness?#

Use the source's effective or updated timestamp to measure how old a destination value is, then keep separate timestamps for ingestion, normalization, publication, acceptance, and observation. A file modification time or successful job time can show delivery activity, yet it cannot prove that the value represents the current source state.

If your team can see a change in the source but cannot prove when the right offer reached every destination, talk to Catalog about building a product-data layer you can measure.