Egress Data Transfer
Export event-level data to Amazon S3 with scheduling and field transformations.
Overview
The Data transfer-Exports feature in Netcore CE allows you to extract large volumes of behavioral and engagement event data. This feature is best for your data teams for control, flexibility, and automation in how customer engagement data flows into your ecosystem.
Netcore CE enables CDP-level data exchange with customizable exports, which is unique in the martech industry.
You can:
- Select your destination. For now, only S3 is supported.
- Select the format (CSV, JSON, Parquet, Avro).
- Select all/or event types and schemas.
- Apply filters and transformations such as selecting events from a specific campaign, renaming fields, masking PII, and so on.
- Schedule one-time or recurring exports.
Let us understand this with a simple example:
A bank exports application step events to S3 to analyze user drop-offs during credit card applications. Data is filtered, transformed, and delivered hourly to match their CRM, enabling the identification and resolution of drop-off points.
Use Cases
Refer to the given table to know the use cases for this feature for different industries.
Industry | Use Case | Example Fields | Transformations |
---|---|---|---|
Finance | Export real-time transaction events and loan application touchpoints for fraud analytics and regulatory audits. | transaction_id , event_timestamp , user_id , credit_score | Mask PII fields (e.g., phone numbers) before exporting to S3 for audit teams. |
E-Commerce | Export product views, add-to-cart, and purchase events to build conversion funnels in a data warehouse. | user_id , product_id , event_type , device_type | Rename product_id to sku_id and reformat timestamps. |
Marketing | Export email clicks, notification opens, and campaign engagement data to optimize targeting strategy. | campaign_id , event_name , click_time , user_segment | Convert timestamps to YYYY-MM-DD and apply filters for campaign types. |
How to Enable Egress
-
Contact your account manager.
-
Share required details with your CSM (fields, formats, event types).
- Panel Name: (e.g.,
client_cee
)
Connection Name: The S3 export connection configured on your CE dashboard.
Export Format: Select fromCSV
,JSON
,Parquet
, orAvro
.
- Panel Name: (e.g.,
-
Specify Data Selection Criteria
- Event Type: Select
Behavioral
orEngagement
. - Events: Select to export all events or specific events. The available event formats are:
- Standard: Generates one file per event type. For example, if you configure exports for email open, click, and unsubscribe events, you will receive separate files for each event.
For behavioral reports, each event payload field is added as a new column. - Consolidated: Combines all selected events into a single file.
- Standard: Generates one file per event type. For example, if you configure exports for email open, click, and unsubscribe events, you will receive separate files for each event.
- Event Type: Select
-
Define the fields to be exported.
- Engagement Fields: Standard fields are listed in the sheet. Additional or optional fields such as, User attributes, Campaign Details, Device Details can be included upon request by your CSM.
- Campaign Details: If requested, campaign details are included as individual columns after the default columns. These details can be found in the detailed reports section on the Netcore CE dashboard.
Channel-specific details will be available (e.g., for Email: Campaign details > Subject). For channels without such data, these columns will be empty. - Behavioral Fields: All user attributes are supported for export.
-
Specify required data transformations. The options include:
- Date Format: Supported formats include
DDMMYYYY
,MMDDYYYY
. - Field Conversion: Convert specified fields to string format.
- Masking: Apply data masking where necessary.
- Field Renaming: Rename fields as required.
- Date Format: Supported formats include
-
Set export frequency and delivery timing. Supported options are: Hourly, Daily, Weekly, or Monthly.
After you submit these details, our team will validate your configuration. Once validated, we will send a sample report for your approval. Upon approval, we will begin sending data files to your specified S3 destination.
Data Customization
Customize your export by transforming fields or adding new ones.
The transformation types available are:
- Mask field: Hide sensitive data by replacing it with symbols or dummy values.
- Rename column: Change the name of a data column to something more meaningful.
- Field to string: Convert a field’s value into text format.
- Date format: Change how dates are shown, like from
MM/DD/YYYY
toYYYY-MM-DD
.
- Define the transformation using the From and To fields.
Example: To rename a column, setevent_id
as From andevent_identity
as To.
Schedule Export
Select how and when your data export runs. The options include:
-
One-time: Export all event data for a selected time range.
-
Recurring: Automate exports using a frequency (Hourly, Daily, Weekly, Monthly) and define a start and end date.
Note: You can run up to 10 exports at once.
Your export is now active.
Updated 1 day ago