Before you start

To find out more about product data feeds, have a go at this article in the knowledge base. Please note that advertisers do not necessarily supply product data feeds and if they do, they might not grant all partners access to them. If you should find that – after following these instructions – you cannot find the product data feed, please contact your advertiser.

Getting started

Navigate to Creatives -> Product data feeds in the main menu.

Select an Adspace. You will then see the product feeds you are allowed to download for that particular feed.

Configure your feed

To start working with a product feed, click on the icon next to the product feed you need. If you do so, you will see a panel looking more or less like this:

This panel allows you to configure the data feed and export a file that is completely adjusted to your needs.

Columns

• First, select the columns of the products you require: you can add and remove columns using the buttons in the middle of the panel. The window on the left represents the available data fields, the window on the right represents the fields you have selected. By using the buttons on the far right (first, move up, move down, last), you can define the order in which the data will be displayed in the file.

• If you do not select any column, then the export will automatically contain all columns of the data feed.

Sub-IDs

• Next, there is the section that allows for additional configuration of the tracking URL. Here you can specify subIds appended to your tracking Link. These subIds will be added as parameter to the tracking URLs. By doing this you can easily sub-segment traffic that originates from the product data feed (e.g. to distinguish traffic from connected public networks).

Format

• Further below, you can define other characteristics of your file:

  • the format (CSV, or XML)

  • the csv-delimiter (SEMICOLON, COMMA, or TAB)

Preview

• Setting parameter preview to true allows you to download a feed with a vastly reduced file. Use this for testing purposes. File downloads with the parameter preview set to true will not consume any quotas.

Compression

• the compression type (ZIP or GZ)

Your configuration will affect the URL you see at the bottom of the page. When you are done, you can just copy this URL and paste it into your own application, or download the product data directly using the download button.

Download your feed efficiently and with low latency

To retrieve the newest products their pricing from your advertisers, you have to consider the following topics:

• quotas and rate-limiting: “You should not load a feed too often.”

• conditional requests: “You should load the feed only if it’s changed.”

The following sections describe how you deal with both topics:

Quotas and Rate-Limiting

Different types of API requests to Ingenious are subject to different rate limits. To ensure the quality and to better protect our Product Data Feed system integrity, we have the following limitations

• Maximum of 5 concurrent download requests per 1 minute per partner

• Maximum of 3 download requests per 1 hour per feed per adspace for compressed feeds

• Maximum of 1 download requests per 1 hour per feed per adspace for uncompressed feeds

In case you exceed these quotas, the response of the feed download will return a 429 Too Many Requests HTTP status code.

$ curl -I "YOUR-DOWNLOAD-LINK"
> HTTP/1.1 429 Too Many Requests
> Date: Tue, 15 Mar 2022 14:55:57 GMT

In this case you have the following options how to react:

• your implementation should back-off and reduce its frequency of loading data. However, this will lead to a higher latency for getting new product data.

• you implement the “conditional request” logic describe below

Conditional requests

The Ingenious Feed response returns an ETag header, and also a Last-Modified header. You can use the values of these headers to make subsequent requests to this feed using the If-None-Match and If-Modified-Since headers, respectively. If the resource has not changed, the server will return a 304 Not Modified HTTP status code.

$ curl -I "YOUR-DOWNLOAD-LINK"
> HTTP/1.1 200 OK
> Date: Tue, 15 Mar 2022 14:59:17 GMT
> ETag: "b9bb4a36d73f69f44ae8588b15950651"
> Last-Modified: Tue, 15 Mar 2022 12:39:44 GMT
> Cache-Control: no-cache

Store the ETag (in the example "b9bb4a36d73f69f44ae8588b15950651") and/or Last-Modified (in the example Tue, 15 Mar 2022 14:59:17 GMT) value(s) on your side.

You now can check/poll regularly if the file changed:

Using If-None-Match and providing the ETag value you stored:

$ curl -I "YOUR-DOWNLOAD-LINK" -H 'If-None-Match: "b9bb4a36d73f69f44ae8588b15950651"'
> HTTP/1.1 304 Not Modified
> Date: Tue, 15 Mar 2022 15:01:01 GMT
> ETag: "b9bb4a36d73f69f44ae8588b15950651"
> Last-Modified: Tue, 15 Mar 2022 12:39:44 GMT
> Cache-Control: no-cache

Or using If-Modified-Since and providing the Last-Modified value you stored:

$ curl -I "YOUR-DOWNLOAD-LINK" -H 'If-Modified-Since: Tue, 15 Mar 2022 14:59:17 GMT'
> HTTP/1.1 304 Not Modified
> Date: Tue, 15 Mar 2022 15:02:18 GMT
> ETag: "b9bb4a36d73f69f44ae8588b15950651"
> Last-Modified: Tue, 15 Mar 2022 12:39:44 GMT
> Cache-Control: no-cache

Once the feed has been updated by the advertiser, the feed download will return the compressed feed file with an HTTP status code 200.

Check for changed feeds via API

You can check for updates on all your feeds regularly via the API point /creatives/productdata/findFeeds. Here are the important fields of information in the response regarding update status of the feed:

Example Request:

curl -X POST "{YOUR_API_DOMAIN}/creatives/productdata/findFeeds?adspaceId={YOUR_ADSPACE_ID}&page=0&rowsPerPage=100" \
 -H "Accept: application/json" \
 -H "x-api-key: {YOUR_API_KEY}" \
 -H "Content-Type: text/plain" 

Example Response:

{
  "page": 0,
  "totalPages": 1,
  "entries": [
    {
      "feedId": "1234",
      "name": "Example Feed",
      "advertiserId": "9876",
      "advertiserName": "Example Advertiser",
      "lastUpdatedAt": "2022-03-17T11:55:59.761+00:00",
      "lastModifiedAt": "2022-01-17T10:12:44.121+00:00",
      "numberOfProducts": 12844,
      "lastFileHash": "637df52981d762c284d3db9a588carc7",
      "updateInterval": "0 0,12 * * *",
      "nextImportScheduledAt": "2022-03-18T11:00:00.000+00:00",
      "importUrl": null,
      "contentType": null,
      "compression": null,
      "status": null,
      "partnerRestrictionEnabled": null,
      "allowedPartners": null,
      "lastAction": null,
      "lastActionStatus": null,
      "lastActionMessage": null
    }
  ],
  "totalEntries": 1
}

Debugging and Testing

When you want to integrate a feed for the first time, you may want to download it multiple times without having to concern yourself with rate-limits. For this use-case, you can use the preview mode to get a version of your feed that is vastly reduced in file size. Using preview mode will not consume any quota.

Error handling

If your request cannot be processed, you typically receive an error response like this:

{
	"errors": [
		{
			"code": "CRE_UNSUPPORTED_COLUMN",
			"description": "Requested columns that do not exists in processed file of feed (123): MY_COLUMN"
		}
	]
}

Here is a list of the most common error codes and how to best react to them: