Overview

A/B Testing and Split Campaigns for In-App Messages enable you to create and compare multiple message variants within a single campaign. By measuring performance across variants, you can identify the content that drives the best results and automatically send the winning message to the rest of your audience.

This feature is available in the In-App Message channel of the Netcore CE dashboard.

Campaign Types

When creating a new In-App campaign, refer to this document to learn about the available campaign types.

Prerequisites

Ensure the following conditions are met before creating an A/B or Split In-App campaign:

• Your Netcore CE account has In-App Message enabled.
• Admin-level Conversion Goal (CG) settings have been configured in Admin Configuration if you plan to use a Control Group.
• The in-app templates you intend to use are built with the Unified Content Editor (UCE) or Custom HTML. Legacy templates without UCE support are not compatible with A/B campaigns.
• Please update your app to the latest SDK versions - Android 3.8.5 and iOS 3.8.3 or above - to enable split campaign rendering for your users. Users on the older SDK versions will not receive AB or split in-app campaigns.

Create an A/B or Split In-App Campaign

Campaign creation follows a four-step process: Setup → Audience → Content → Schedule. Refer to the steps below for each page.

1. Navigate to Campaigns > In-App Messages and click Create Campaign.
2. Select In-App Message as the channel.
3. Under Campaign Type, choose one of the following:
   • A/B Campaign: To run a test with automatic winner declaration.
   • Split Campaign: To run a test with a fixed audience split and no automatic winner.
4. The campaign name header will automatically display an [A/B] or [Split] tag to identify the type.
5. Configure the Conversion Goal (Optional but Recommended)
6. Set the Control Group (Optional)
7. Select the Audience.

Content

The Content page is where you configure your message variants, set the winning criteria, define audience distribution, and set the test window. This page is divided into a left-hand configuration panel and a right-hand template panel.

A/B Configuration

Winning Criteria (A/B Campaign only)

Select the metric that determines the winning variant:

• Clicks: the variant with the highest click-through rate wins.
• Conversions: the variant with the highest conversion rate wins.

Variants

Each campaign starts with two variants (Variant A and Variant B) by default.

Tips Panel

A Tips section is displayed above the variant scroller to provide contextual guidance:

• Control Group tip: Shows current control group % which is set in the Setup section.
• Test window and winner declaration tip: visible for A/B campaigns only (not shown for Split campaigns).

Audience Distribution

Controls what percentage of the campaign audience sees each variant during the test window.

Test Window (A/B Campaign only)

The test window is the period during which the platform collects data before declaring a winner.

• Default: 2 hours.
• Duration range: 1–23 hours, or 1–30 days.
• Unit: Hours or Days.
• In case of a tie at the end of the test window, the winner is selected at random by Netcore.

Template Selection

Use the right-hand panel to select or create the in-app message template for each variant.

Once a template is selected, the preview panel shows three action buttons:

• Change Template: swap the current template for a different one.
• Edit Template: open the template in the UCE or Custom HTML editor for modifications.
• Live Preview: view a rendered preview of the message on a simulated device.

8. Schedule: Define the campaign's delivery time and date.
9. Preview your campaign. The Preview page allows you to review the full campaign configuration before publishing.

On the Preview page, you can see:

• Winning Criteria (A/B campaigns only) , Clicks or Conversions.
• Test Window: The configured duration for the A/B test.
• Variant Previews: Toggle between Variant A, Variant B, and any additional variants to review each message design and content.

Review all details carefully. Once you publish the campaign, some configuration fields become locked.

Campaign Listing Page

The campaign listing page provides an at-a-glance view of all your campaigns. A/B and Split campaigns have the following additional features.

Identifying A/B and Split Campaigns

• Each A/B campaign and Split campaign displays a unique icon in the listing to differentiate it from Regular campaigns.
• A downward arrow icon on a campaign row expands to reveal the individual variants.
• Note on Campaign IDs: The first variant (Variant A) shares the same Campaign ID as the parent campaign. Subsequent variants are assigned consecutive IDs.

Viewing Variants and Control Group

• Expand a campaign row to see all variants listed below the parent campaign.
• If a Control Group was configured, it appears at the top of the expanded list, above the variants.
• Control Group metrics show only Conversions (total and unique) and Revenue. All other metrics are displayed as N/A.

Winner Indicator

• Once the A/B test window ends and a winner is declared, a Crown icon appears next to the winning variant in the list.
• The winning variant moves to a Running state and continues delivering to the remaining audience.
• The non-winning variants move to a Completed state.
• The parent campaign continues to show a Running status until the campaign end date is reached.

Filtering Campaigns

Use the Campaign Type filter to narrow the campaign list. After selecting the In-App channel, the Campaign Type dropdown offers:

• Regular Campaign
• A/B Campaign
• Split Campaign

Editing a Live Campaign

The fields available for editing depend on the campaign's current state. The table below summarises the editing rules.

Duplicating a Campaign

You can duplicate an existing A/B or Split In-App campaign to quickly create a copy with the same configuration.

• All variants and their template content are cloned identically.
• The Admin Configuration defaults are not re-applied. The duplicate respects the settings of the original campaign, including Conversion Goal, Control Group, Winning Criteria, and Test Window.
• After duplicating, you can modify any configuration before publishing the new campaign.

Reporting

A/B and Split campaigns include enhanced reporting to help you analyse performance across variants.

Summary Report

The Summary Report CSV includes two additional columns for A/B and Split campaigns:

Additional reporting behaviour:

• All variants share the same Campaign Name but have different Campaign IDs.
• Control Group data appears as a separate row showing Conversion (total and unique) and Revenue metrics only. The Campaign Name and ID for the Control Group row match those of Variant A.

Detailed Report

The Detailed Report includes additional fields for A/B and Split campaigns:

• Two new fields are available under the Campaign Details dropdown: Campaign Type and Message Type.
• If selected, these fields are included as columns in the exported CSV.
• Control Group users appear in the Detailed Report with only the Conversion Event populated. All other event fields will be empty.

Winner Declaration Logic (A/B Campaigns)

For A/B Campaigns, the platform automatically evaluates all variants at the end of the configured test window and selects a winner based on the chosen Winning Criteria.

How It Works

1. The test window begins as soon as the campaign enters the Running state.
2. During the test window, the audience is split equally among variants and performance data is collected.
3. At the end of the test window, the platform compares variants using the selected Winning Criteria (Clicks or Conversions).
4. The variant with the highest metric value is declared the winner.
5. The winning variant is automatically delivered to the remaining audience for the rest of the campaign duration.
6. Non-winning variants are marked Completed.

Tie-Breaker

If two or more variants are statistically tied at the end of the test window, the winner is selected at random by Netcore's system. The tie-breaker uses a secure random selection method to ensure fairness.

Audience Behaviour for Split Campaigns

Split Campaigns are designed for long-running, parallel tests where you want full control over the audience split and do not need automatic winner declaration.

• Sticky assignment: Each user is consistently assigned to the same variant across all sessions. A user who sees Variant A in their first session will continue to see Variant A in subsequent sessions.
• All variants run for the full campaign duration. No variant is stopped early.
• The audience distribution is fully customisable; adjust the sliders to allocate any percentage to each variant, as long as all percentages total 100%.
• Audience sizes of up to 10 million users are supported without degradation in delivery speed or platform performance.

For SDK integration details and upgrade guides, refer to the Netcore CE Developer Documentation or contact your Netcore integration support team.