Magnite Data Integration Spec

This document outlines the step-by-step process for Data Partners to onboard their data to the Magnite platform.

Considerations

• File Types: Magnite expects two file types for onboarding:

• Taxonomy File: A JSON payload containing segment IDs, names, owners, and CPM values.

• Membership File: A JSON payload mapping user identifiers to their respective segments.

• Delivery Frequency: Files can be delivered daily, weekly, or monthly.

• Data Retention: User-to-segment mappings expire after 28 days unless refreshed.

Important Note: If you only utilize data on our DV+ platform, let your account lead know, and we can provide an API solution.

Taxonomy File Format

The taxonomy file defines the segments available for targeting.

• File Type: NDJSON.

• Compression: Gzipped (.gz).

• Structure: A valid NDJSON object containing a segments array.

Taxonomy Schema

Example Row

{"id": "123abc","name":"Cat Owners","price": {"default": {"type": "cpm","value": 1.00}},"owner": "YourCompanyName","description": "Cat owners"}\n

Membership File Format

The membership file maps users to specific segment IDs.

• File Type: NDJSON.

• Compression: Gzipped (.gz).

Membership Schema

Example Rows

{"uuids": [{"id": "EA7583CD-A667-48BC-8806-42EC82848606","idType": "ctvid"},{"id": "127.0.0.1","idType": "ip"}],"updateType": "partial","segments": [{"id":"123abc"}]}\n
{"uuids": [{"id": "EA7583CD-A667-48BC-8806-42EC82848606","idType": "ctvid"},{"id": "127.0.0.1","idType": "ip"}],"updateType": "partial","segments": [{"id":"123abc"}]}\n
{"uuids": [{"id": "EA7583CD-A667-48BC-8806-42EC82848606","idType": "ctvid"},{"id": "127.0.0.1","idType": "ip"}],"updateType": "partial","segments": [{"id":"123abc"}]}\n

File Constraints & Naming

Size Restrictions

• Total File Size: Must be less than 2GB. If larger, split the file using a numeric postfix, e.g.:

• Membership-TestData-1680626358-1.ndjson.gz (part 1, <2GB).

• Membership-TestData-1680626358-2.ndjson.gz (part 2, <2GB).

• Row Size: Individual NDJSON rows must be less than 4MB (uncompressed). Split data into multiple rows if necessary.

Naming Convention

Files should follow this format: [Type]-[Owner]-[UnixTimestamp].ndjson.gz.

• Examples: 

• Taxonomy-CompanyName-1680626358.ndjson.gz

• Membership-CompanyName-1680626358.ndjson.gz

Supported Identifiers

Magnite supports any valid string for user IDs, though IP addresses undergo specific validation.

• Mobile: IDFA, AAID.

• OTT/CTV: Roku RIDA, Amazon Fire AFAI, Samsung TIFA, LG, Vizio, and IP addresses.

• DV+: IDFA, AAID, and Cookies are accepted (Cookie sync is required).

Delivery Transfer

Magnite uses Amazon S3 for batch delivery. During the bucket setup process, the client will be required to set up a role for bucket access. Further permissioning will be coordinated.

Directory Structure:

s3://<bucket_name>/YYYYMMDD/<partner_identifier>/

• Bucket: Magnite will provide this. Each bucket will be partner-specific

• Partner Identifier:

• Publisher 1st Party: Use the Magnite Supply Seat ID.

• Advertiser Data: Use the advertiser entity name.

• 3rd Party Data: Use the string 3PD

If you are interested in other onboarding paths, please reach out to your account manager.

Integration Workflow

1. Permissioning: Magnite and the partner coordinate S3 bucket access and roles.

2. Testing: Partner sends a test file to the S3 location.

3. Verification: Magnite confirms receipt and validates file contents.

4. Ingestion: Magnite ingests the data into the expected platforms for use.

5. Confirmation: Magnite confirms when the integration is complete.

Checklist

To ensure your first S3 test upload is successful and follows our spec, use this checklist to verify your files before delivery. 

• File Formatting & Compression

• Format: Ensure both Taxonomy and Membership files are in NDJSON format.

• Compression: All files must be gzipped with the .gz extension.

• Line Breaks: Each JSON object must be on its own line ending with a newline character (\n). 

• Taxonomy File Content

• Segment IDs: Each id must be a string that matches the IDs you will use in your membership files.

• Price Object: Ensure the price object contains a default object with a type ("cpm" or "revshare") and a numerical value.

• CPM Values:

• Publisher 1st Party: Set to $0 or leave blank if Magnite isn't billing.

• 3rd Party/Advertiser: Populated with the correct value to be charged/paid back.

• Membership File Content

• User IDs: Each entry must include an id and a valid idType (e.g., "ctvid", "ip", "maid", or "cookie").

• Update Type: Use "partial" to add users to segments or "remove" to opt them out.

• Segment Mapping: Ensure the segments array contains existing IDs from your taxonomy file.

• Size & Naming Constraints

• Total Size: Each file must be under 2GB.

• Row Size: Individual uncompressed rows must be under 4MB.

• Naming Convention: Follow the format [Type]-[Owner]-[Timestamp].ndjson.gz.

• Regex & Directory Validation

• Your files must match the following regex patterns to be successfully ingested from the S3 bucket:

• Membership Files:
  ^([[:digit:]]{1,})/([[:alnum:]_-]{1,})/Membership-([[:alnum:]]{1,})-([[:digit:]]{1,})(-([[:digit:]]{1,}))?.ndjson.gz$

• Taxonomy Files:
  ^([[:digit:]]{1,})/([[:alnum:]_-]{1,})/Taxonomy-([[:alnum:]]{1,})-([[:digit:]]{1,})(-([[:digit:]]{1,}))?.ndjson.gz$

Note: These patterns validate the directory structure YYYYMMDD/partner_identifier/ followed by the specific filename components (Type, Owner, and Timestamp). The regex checks for the structure: YYYYMMDD/PartnerName/Type-Owner-Timestamp(-Part).ndjson.gz.

Notes and FAQs

Does the taxonomy need to be flat?

Yes, the specification requires a flat taxonomy. However, you can represent hierarchies within the segment name using a "Category > Tier > Segment Name" format.

Can I use unique Unix timestamps instead of part numbers for large files?

Yes. You may use different Unix timestamps for each file to avoid large inputs. Please note that if you choose this method over part numbers, you are responsible for ensuring no file name collisions occur.

How are user opt-outs and removals handled?

To remove a user, send an update with the updateType set to "remove" for the relevant segments. The user will expire in Magnite after 28 days. Magnite will move processed files from the original location to the new location after it's been processed: s3://<bucket_name>/YYYYMMDD/<partner_identifier>/processed/<FILE>*.

Must a taxonomy file be fully processed before sending membership files?

No. You can send taxonomy files before or after membership files. While memberships cannot be targeted without a corresponding taxonomy, Magnite will maintain the membership data and link it once the taxonomy is received. Note that sending the taxonomy first is the preferred method.

Do taxonomies support incremental updates?

Yes, taxonomy updates are incremental and can be sent at any time. Files are generally processed in the order they are received.

What feedback is provided regarding delivery and processing?

• S3 Feedback: Magnite creates an output file for every input file in the /processed/ folder. Input files remain in the original location until manually deleted or reach their TTL (Time to Live).

• UI Visibility: Onboarded data will be visible in the user interface, typically within 1-2 days after the initial platform configuration and data integration.

How do I perform a full refresh of membership data? To refresh the complete state of your segments, use the updateType "partial" (which functions as an "add" operation) and include all current membership-to-segment ID associations.