A/B Testing with Resync

Learn how to run sophisticated A/B tests and experiments with Resync's campaign system.

What You'll Build

By the end of this tutorial, you'll be able to:

• Create campaigns with multiple variants
• Target specific user audiences
• Track conversions and measure results
• Declare statistical winners

Prerequisites

• Resync SDK installed and initialized
• Basic understanding of A/B testing concepts
• A Resync account with campaign access

Step 1: Create Content Variants

First, create the different versions of your content:

In the Resync Dashboard:

1. Go to Content Blocks
2. Create Variant A: CheckoutFlowA
   • Single-page checkout
   • Prominent "Buy Now" button
   • Trust badges at the top
3. Create Variant B: CheckoutFlowB
   • Two-page checkout
   • "Continue to Payment" button
   • Trust badges at the bottom

Step 2: Create a Campaign

In the Dashboard:

1. Navigate to Campaigns → Create New

2. Set campaign details:

   • Name: "Checkout Flow Test"
   • Start Date: Today
   • End Date: 30 days from now
   • Goal Event: evt_purchase_completed

3. Add variants:

   • Control: CheckoutFlowA (50% traffic)
   • Treatment: CheckoutFlowB (50% traffic)

4. (Optional) Set audience targeting:

   • Target: "All Users" or specific audience

5. Click Save as Draft

Step 3: Implement in Your App

React Native Example

import React from 'react';
import { View, StyleSheet } from 'react-native';
import { ResyncCampaignView } from 'resync-react-native';
import Resync from 'resync-react-native';

export default function CheckoutScreen() {
  const handlePurchaseComplete = async (orderId) => {
    // Track conversion event
    await Resync.logEvent({
      eventId: 'evt_purchase_completed',
      logId: orderId,
      metadata: {
        amount: 99.99,
        currency: 'USD',
      },
    });
  };

  const functionRegistry = {
    completePurchase: handlePurchaseComplete,
  };

  return (
    <View style={{ flex: 1 }}>
      <ResyncCampaignView
        name="checkout_flow_test"
        functionRegistry={functionRegistry}
      />
    </View>
  );
}

That's it! ResyncCampaignView automatically:

• Calls getVariant() and tracks impressions
• Fetches the correct variant's content
• Renders it natively
• No manual variant logic needed!

Step 4: Track Conversion Events

Track when users complete the goal action:

// When purchase is completed
async function completePurchase(orderId, amount) {
  // Process payment...
  const success = await processPayment();

  if (success) {
    // Track conversion event
    Resync.logEvent({
      eventId: 'evt_purchase_completed',
      logId: orderId,
      metadata: {
        amount: amount,
        currency: 'USD',
        orderId: orderId,
      },
    });
  }
}

Step 5: Launch the Campaign

1. Review your campaign setup
2. Click Activate to start the test
3. Monitor initial performance for any issues

Step 6: Monitor Results

In the Dashboard:

Navigate to Campaigns → Checkout Flow Test to view:

• Impressions: How many users saw each variant
• Conversions: How many completed purchases
• Conversion Rate: Percentage who converted
• Statistical Significance: Is there a clear winner?

Metrics to Track:

Metric	Variant A	Variant B
Impressions	5,000	5,000
Conversions	150	200
Conversion Rate	3.0%	4.0%
P-value	-	0.02 ✅

Step 7: Declare a Winner

When the campaign ends or reaches statistical significance:

1. Review results in the dashboard
2. If significant: Winner is automatically declared
3. Webhook campaign.winner_declared is triggered
4. Implement the winning variant permanently

Handle Winner via Webhook

// Your webhook endpoint
app.post('/webhook', (req, res) => {
  const { event, data } = req.body;

  if (event === 'campaign.winner_declared') {
    console.log('Winner:', data.winner);
    console.log('P-value:', data.pValue);
    console.log('Confidence:', data.confidenceLevel);

    // Update your config to use the winning variant
    updateProductionConfig({
      CHECKOUT_FLOW: data.winner,
    });
  }

  res.sendStatus(200);
});

Advanced: Multi-Variant Testing (A/B/C)

Test more than two variants:

React Native

// Simply use the campaign name - ResyncCampaignView handles all variants
<ResyncCampaignView name="checkout_flow_test" />

The campaign in your dashboard can have as many variants as you want. ResyncCampaignView will automatically assign and render the correct one.

JavaScript

const variant = await Resync.getVariant('checkout_flow_test');

switch(variant) {
  case 101:
    renderSinglePageCheckout();
    break;
  case 102:
    renderTwoPageCheckout();
    break;
  case 103:
    renderThreePageCheckout();
    break;
  default:
    renderDefaultCheckout();
}

Advanced: Audience Targeting

Test different variants for different user segments:

Create Audiences:

1. New Users - Signed up < 7 days ago
2. Returning Users - Signed up > 7 days ago

Create Separate Campaigns:

• Campaign 1: "Checkout Test (New Users)"

  • Audience: New Users
  • Variants: A (simple) vs B (detailed)

• Campaign 2: "Checkout Test (Returning Users)"

  • Audience: Returning Users
  • Variants: A (quick) vs B (upsell)

Best Practices

1. Set Clear Goals

Define success metrics before starting:

• Primary: Conversion rate
• Secondary: Average order value, time to checkout

2. Ensure Statistical Significance

• Minimum sample size: 100 conversions per variant
• Run for at least 1-2 weeks
• Don't stop early (even if winning)

3. Test One Variable at a Time

Bad: Testing button color AND checkout flow Good: Test button color, then test checkout flow

4. Consider User Experience

• Keep consistent experience per user
• Don't switch variants mid-session
• Respect user preferences

5. Document Results

Keep a record of:

• What was tested
• Why it was tested
• Results and learnings
• Next steps

Troubleshooting

Users Not Seeing Variants

Check:

• Campaign is active
• Start date has passed
• User is in target audience
• Variant assignment is working

Debug:

const variant = await Resync.getVariant('checkout_flow_test');
console.log('Assigned variant:', variant);

Conversion Not Tracking

Check:

• Event ID matches campaign goal
• Event is being logged after user action
• Event metadata includes relevant data

Debug:

// Log when event is tracked
Resync.logEvent({
  eventId: 'evt_purchase_completed',
  metadata: { debug: true, orderId: '123' },
});
console.log('Conversion event tracked');

No Statistical Significance

Possible reasons:

• Sample size too small
• Variants too similar
• High variance in results

Solutions:

• Run test longer
• Increase traffic to test
• Try more different variants

Real-World Example

Scenario: Mobile App Onboarding

Goal: Increase signup completion rate

Variants:

• A (Control): 5-screen onboarding with all features
• B (Treatment): 2-screen onboarding with core features only

Implementation:

function OnboardingFlow() {
  const handleComplete = () => {
    // Track conversion
    Resync.logEvent({
      eventId: 'evt_onboarding_completed',
    });

    // Navigate to home
    navigation.navigate('Home');
  };

  return (
    <ResyncCampaignView
      name={'Onboard Premium'}
      functionRegistry={{
        handleComplete,
      }}
    />
  );
}

Results (after 2 weeks):

• Variant A: 45% completion rate
• Variant B: 67% completion rate
• P-value: 0.001 ✅ (highly significant)

Winner: Variant B (shorter onboarding)

Next Steps

• Learn about Audiences →
• Set up Webhooks →
• View campaign examples →
• Explore analytics →

Resources

• Statistical Significance Calculator
• A/B Testing Best Practices