Overview

The egress flow lets you send high-volume behavioural and engagement event data from Netcore CE, like page views, purchases, or campaign clicks, into your Amazon S3 bucket, where your data team can run it through analytics, warehousing, or downstream pipelines. Once configured, each export writes files to your bucket on the schedule you define, in the format and structure you choose.

End-to-end, the egress flow looks like this:

1. Connect: set up an Amazon S3 connection under Settings > Integrations, so it's available to pick from during export setup.
2. Configure: create a data transfer specifying which events, schema, and fields to export, and in which file format.
3. Customise (optional): apply transformations, such as renaming fields, masking PII, or reformatting dates, before the file is written.
4. Go live: choose a historic or recurring schedule and create the export; files begin landing in your S3 bucket automatically.

Example: E-Commerce Conversion Funnels

An online retailer exports product view, add-to-cart, and purchase events to S3 on an hourly schedule. The data team renames fields to match their warehouse schema and builds conversion funnels without waiting on a manual data pull.

The export is built in four steps: Setup, Data selection, Data customisation, and Schedule.

Step 1: Open Data Transfer

1. Go to Settings in the left menu.
2. Click Data transfer.
3. You land on the Export tab. Here you see a list of your past and current exports, each with a status such as Running, Draft, or Completed.

Step 2: Start a New Export

1. Click CREATE TRANSFER at the top right.
2. A window opens: Select a destination to export your data.
3. Click Amazon S3.

The export builder opens on the Setup step.

Step 3: Setup

Fill in the basic details of the export.

1. Export title: Type a name for this export. Use a name you will recognise later.
2. Select export mode: Choose Batch. Stream is greyed out because Amazon S3 does not support it.
3. Connection name: Pick your Amazon S3 connection from the list. You can type in the box to search for it.
4. Export file format: Choose the file type. The options are JSON, CSV, Parquet, and Avro.
5. Click NEXT.

Step 4: Data Selection

Here you choose what data goes into the file. The left side is where you make choices. The right side, Data preview, shows the fields that will appear in your file. The preview updates as you make changes.

1. Pick the events

• Choose Engagement or Behavioural.
• Choose all events or specific events. For specific events, open the dropdown, search, and tick the ones you want.

2. Pick the schema

• Standard: One file per event.
• Consolidated: One file for all the events you selected.

3. Pick the fields

• Standard fields: These are the default fields. Open the list to tick or untick fields. A Select all option is available.
• Additional fields: Click + ADD to include more.
  • For Engagement, you can add Attribute details (contact attributes) and Channel details (Email, SMS, App Push).
  • For Behavioural, you can add Attribute details (user attributes).
  • Open the dropdown and tick the attributes you want.

4. Apply filters (optional)

• Turn on the Filters toggle.
• For Engagement, filter by events, campaigns, journeys, or tags.
• For Behavioural, filter by event and source (website or app).

When you are done, click NEXT.

Step 5: Data Customisation (Optional)

This step lets you change how fields look in the file. You can skip it if you do not need any changes.

1. Turn on the Transformation toggle.
2. Click + ADD to add a change. For each one, pick a Transformation type and the field it applies to:
   • Rename column: Change a field name. Set the old name in From and the new name in To.
   • Mask field: Hide a field value. Pick the field.
   • Field to string: Turn a field value into text. Pick the field.
   • Date format: Change how a date looks. Pick the date field, then pick the new format in To.
3. Add as many changes as you need. Click + ADD again for each one.

Send a test file (optional)

• Click SEND TEST DATA at the top right of the Data preview.
• This sends a small sample to your Amazon S3 connection so you can check the output.
• The test uses the latest 100 events.
• If you turned on transformations, add at least one valid transformation before you send the test.

When you are done, click NEXT.

Step 6: Schedule

Choose when and how often the export runs.

1. Pick the type:
   • Historic: A one-time export for a date range you choose.
   • Recurring: Runs again and again. Pick a Frequency (Hourly, Daily, Weekly, or Monthly) and an Interval (for example, every 2 hours).
2. Event duration: Pick the date range for the data. You can use a preset such as Today, Yesterday, Last 7 Days, or Last 30 Days, or set a custom range.
3. Click NEXT.

Step 7: Review and Create

You now see a summary of all four steps: Setup, Data selection, Schedule, and Data customisation.

1. Check each section.
2. To change anything, click EDIT on that section. This takes you back to that step.
3. When everything is correct, click CREATE EXPORT.

A green message confirms the export was scheduled. Your new export appears in the list on the Data transfer page.

After You Create the Export

Back on the Data transfer list, you can track and manage your export.

The list shows these columns:

• Data transfer info: The export name, ID, and status.
• Last transfer data: When the export last ran.
• Next transfer date: When it will run next.
• Destination: Where the data goes, such as Amazon S3.
• Total transfer: How many times it has run.

To manage an export, click the three-dot menu on its row. The actions you see depend on the status. They can include Edit, Duplicate, Delete, Pause, Resume, Stop, and Retry.