To find out more about product data feeds, have a go at this article in the knowledge base. Please note that advertisers do not necessarily supply product data feeds and if they do, they might not grant all partners access to them. If you should find that – after following these instructions – you cannot find the product data feed, please contact your advertiser.
Navigate toCreatives -> Product data feedsin the main menu.
Select anAdspace. You will then see the product feeds you are allowed to download for that particular feed.
Configure your feed
To start working with a product feed, click on the icon next to the product feed you need. If you do so, you will see a panel looking more or less like this:
This panel allows you to configure the data feed and export a file that is completely adjusted to your needs.
First, select the columns of the products you require: you can add and remove columns using the buttons in the middle of the panel. The window on the left represents the available data fields, the window on the right represents the fields you have selected. By using the buttons on the far right (first, move up, move down, last), you can define the order in which the data will be displayed in the file.
If you do not select any column, then the export will automatically contain all columns of the data feed.
Next, there is the section that allows for additional configuration of the tracking URL. Here you can specify subIds appended to your tracking Link. These subIds will be added as parameter to the tracking URLs. By doing this you can easily sub-segment traffic that originates from the product data feed (e.g. to distinguish traffic from connected public networks).
Further below, you can define other characteristics of your file:
the format (CSV, or XML)
the csv-delimiter (SEMICOLON, COMMA, or TAB)
Setting parameter preview to true allows you to download a feed with a vastly reduced file. Use this for testing purposes. File downloads with the parameter preview set to true will not consume any quotas.
the compression type (ZIP or GZ)
Note: The Ingenious Platform does not support uncompressed feeds. Feeds quickly become large and exceed hundreds of megabytes and more. Because the content of feeds is always textual (csv, xml), compression algorithms can reduce the file size significantly. Compressed feeds save bandwidth, but also are faster to download.
Your configuration will affect the URL you see at the bottom of the page. When you are done, you can just copy this URL and paste it into your own application, or download the product data directly using the download button.
Download your feed efficiently and with low latency
To retrieve the newest products their pricing from your advertisers, you have to consider the following topics:
quotas and rate-limiting: “You should not load a feed too often.”
conditional requests: “You should load the feed only if it’s changed.”
The following sections describe how you deal with both topics:
Quotas and Rate-Limiting
Different types of API requests to Ingenious are subject to different rate limits. To ensure the quality and to better protect our Product Data Feed system integrity, we have the following limitations
Maximum of 5 concurrent download requests per 1 minute per partner
Maximum of 3 download requests per 1 hour per feed per adspace for compressed feeds
Maximum of 1 download requests per 1 hour per feed per adspace for uncompressed feeds
$ curl -I "YOUR-DOWNLOAD-LINK"
> HTTP/1.1 429 Too Many Requests
> Date: Tue, 15 Mar 2022 14:55:57 GMT
In this case you have the following options how to react:
your implementation should back-off and reduce its frequency of loading data. However, this will lead to a higher latency for getting new product data.
you implement the “conditional request” logic describe below
The Ingenious Feed response returns an ETag header, and also a Last-Modified header. You can use the values of these headers to make subsequent requests to this feed using the If-None-Match and If-Modified-Since headers, respectively. If the resource has not changed, the server will return a 304 Not Modified HTTP status code.
Note: Making a conditional request and receiving a 304 response does not count against your Rate Limit, so we encourage you to use it whenever possible.
Once the feed has been updated by the advertiser, the feed download will return the compressed feed file with an HTTP status code 200.
Check for changed feeds via API
You can check for updates on all your feeds regularly via the API point /creatives/productdata/findFeeds. Here are the important fields of information in the response regarding update status of the feed:
lastFileHash - Hash of the last version of the feed. Save this in your database. If this hasn’t changed since you last updated the feed, an update is not necessary.
lastUpdatedAt - The time when the feed last changed. If this date is older than your last download date,an update is not necessary.
Responses are paged. Make sure to process all the pages to not miss out on a feed.
Debugging and Testing
When you want to integrate a feed for the first time, you may want to download it multiple times without having to concern yourself with rate-limits. For this use-case, you can use the preview mode to get a version of your feed that is vastly reduced in file size. Using preview mode will not consume any quota.
If your request cannot be processed, you typically receive an error response like this:
"description": "Requested columns that do not exists in processed file of feed (123): MY_COLUMN"
Here is a list of the most common error codes and how to best react to them:
You requested a column that doesn’t exist. Go to the UI and check the feed mapping again in the download page.
This problem typically occurs when the Advertiser changed the available columns on the feed.