Building Integrations for a First-Party Data World: Connecting Twilio SendGrid with Twilio Segment

Data privacy concerns and regulatory compliance demands are compelling today’s businesses to shift their approach to customer data. They’re moving away from third-party customer data, instead depending more heavily on first-party customer data. With that dependence comes new strategies that leverage modern tools for collecting, analyzing, and activating customer data.

In this extended tutorial, we’ll walk through how a business can leverage first-party customer data by integrating Twilio SendGrid with Twilio Segment. In this tutorial, you will learn how to send Sendgrid data to Segment and learn how to activate that data in Segment to personalize your emails to users.

After a brief review of why companies are turning to first-party data and introductions to Twilio SendGrid and Twilio Segment, our tutorial will guide you through the following:

• Connecting SendGrid as a source in Segment
• Sending SendGrid data to a data warehouse
• Sending SendGrid events to Segment with webhooks
• Using Segment source functions to track SendGrid engagement
• Using Segment destination functions to call the SendGrid API

Before our deep dive into the SendGrid-Segment integration, let’s first consider why today’s companies are turning to first-party data.

Why Companies Are Turning to First-Party Data

Even though today’s businesses have access to a massive amount of customer data, not all customer data is equally reliable. Changing winds regarding customer data sources and reliability are causing businesses to reduce their dependence on third-party data in favor of first-party data.

Third-party data is collected through a variety of sources, ranging from web browser cookies to demographic data to behavioral data. Some of this data is actively provided by users, while some of it is collected in the background without users’ awareness. Online advertisers and data brokers collect this data, which is then sold to businesses for use in marketing and sales.

First-party data, on the other hand, is collected directly from customers in the everyday course of business. Sources include social media interactions, online purchases, customer service interactions, marketing campaign engagements, and survey participation.

The drawbacks of third-party data

A primary example of third-party data is web browser cookies. Cookies capturing user browsing behavior on one site are then used for targeted advertising on other sites. Users are becoming increasingly wary of this highly-targeted advertising, causing an erosion of customer trust.

In addition, governments and regulators are increasing efforts to protect the data privacy of consumers. Whether it is because of GDPR in the EU or consumer privacy laws in the United States (such as in California or Connecticut), businesses need to tread carefully when it comes to the collection and usage of third-party data.

The benefits of first-party data

On the other hand, first-party data is collected by businesses with the customer’s knowledge and consent. Of course, businesses still need to comply with data privacy regulations when it comes to the collection and use of first-party data. However, first-party data provides more accurate pictures of customers. The result is products, services, and marketing that are more relevant to customers.

A first-party data strategy requires the use of effective tools. Among those tools are Twilio SendGrid and Twilio Segment.

Twilio SendGrid and Twilio Segment prerequisites

Before we begin, you'll need to register accounts (if you haven't yet) with SendGrid and Segment. You'll also need to perform a few initial setup steps.

First, sign up for a free Twilio SendGrid account if you haven’t yet. There are some initial setup steps you’ll need to take if this is your first time. The single most important step in being trusted as a sender of email is completing the Domain Authentication process. Once your domain is authenticated, meaning you've given SendGrid permission to send email on behalf of your domain, you'll be ready to start delivering your messages. You can find additional setup steps to improve deliverability and secure your account in the Account Setup section of this guide.

Second, sign up for a new Segment account if you haven't yet.

SendGrid and Segment

Twilio SendGrid is an email delivery service that provides programmable Email API capabilities as well as an application-based Marketing Campaigns platform. SendGrid processes over 130 billion emails each month through their proprietary mail transfer agent.

• Twilio SendGrid Email API is built for developers looking to build reliable, scalable, email delivery into their product experience.
• Twilio SendGrid Marketing Campaigns enables businesses to manage customer lists, schedule complex email campaigns, and design beautiful email content for delivery to customers.

Users across both products can utilize dashboards, visualizations, and event webhook data to determine which emails are being opened, which links are being clicked, and which customers are unsubscribing.

As a customer communication solution that tracks direct user engagement with email, Twilio SendGrid is an excellent source of first-party customer data.

Twilio Segment is a customer data platform (CDP) with pre-built integrations – and the flexibility to create custom integrations – that help businesses quickly connect their customer data sources (such as Twilio SendGrid, Facebook Ads, or Braze) with destinations for activating customer data. Those destinations might include Salesforce, Bento, or HubSpot. By providing a central database to capture all customer interactions across all of a business’s touchpoints, Twilio Segment helps businesses build accurate and complete profiles of their customers and audiences.

As a CDP, Twilio Segment provides a secure means for your business to collect, govern, synthesize, and activate first-party customer data.

Now that we’ve laid the foundation, let’s begin our walkthrough and learn how to integrate these two tools.

Connecting SendGrid as a Source in Segment

SendGrid is one of dozens of sources available on the Twilio Segment Platform. This integration streamlines sending SendGrid data to Segment, which in turn can be sent to a data warehouse. Setting up the SendGrid source is straightforward.

We will start start by adding some initial customer data (after you log in). We’ll be using the SendGrid Marketing Campaigns product to manage contacts for this part of the tutorial.

Add initial data to SendGrid

We navigate to Marketing, Contacts to add a list with two customers. On the Contact Lists page, we click on Add Contacts, Manual Add.

We will Add Contacts to a new list. We choose a name for our list (“Customer Leads”), and we add information for our first contact.

We add our second contact to the same list.

We now have a SendGrid contact list called “Customer Leads” with two contacts. Next, we’ll prepare an email for SendGrid to deliver to our contact list. We navigate to Marketing, Single Sends. On the Single Sends page, we click on Create a Single Send.

We use a blank template and the Design Editor to craft our email. We choose a name for our campaign, provide an email subject, and choose the contact list with the recipients for this email.

Next, we craft the body of our email. Our email will contain two different images with links. This will allow us to test click tracking in Segment. Our email body looks like this:

Finally, we Save our email draft and exit back to the Dashboard. We want to set up some things on the Segment side before we send our email.

Add SendGrid as a source in Segment

Next, we will add SendGrid as a customer data source in Segment. Segment will pull information about our SendGrid objects for use in downstream destinations. However, to access that SendGrid data, we will need to provide Segment with a SendGrid API key.

Generate SendGrid API key

In SendGrid, we navigate to Settings, API Keys. On the API Keys page, we click on Create API Key.

We will give our API key a name (“Segment: Pull Marketing Objects”) and create a key with Restricted Access. We do not need an API key with full access to our SendGrid account. For use as a Segment source, we only need to authorize access to marketing-related data.

After creating the API key, we copy the value of the key for use in Segment:

Add and configure source

Just as we did with SendGrid, we sign up for a free account with Segment and sign in. The initial Connections page shows that we need to start by adding a source and a destination.

We click on Add Source. Although there are many sources to choose from, we search for the SendGrid source, select it, and click Next.

We provide a name for our source and paste in our SendGrid API key. Then, we click Create Source.

The next step for setting up a source deals with adding a Segment webhook URL to SendGrid. We won’t do this step yet, so that we can see first what kind of data Segment pulls from SendGrid apart from the webhook.

There is an optional step to test the connection with SendGrid, which we’ll skip. We click on Next and then Done to complete setting up this source. We now have one source set up for our Segment Hub.

Segment pulls SendGrid object data regularly

When we click on our source ("My SendGrid Data"), we see an overview with the sync status for that source.

Segment tells us that it performed a data sync recently, pulling SendGrid object data from our SendGrid account. When we click on the table for details, we see more granularly the kind of data that was pulled.

These numbers look correct. We created a single contact list (“Customer Leads”) with two contacts (John and Sam), and we also created a Single Send email (which we haven’t sent yet).

The Segment documentation for the SendGrid source tells us that Segment will pull data from SendGrid every three hours. This data is stored in the Segment Hub until it is sent to the downstream destination. For the SendGrid source, that downstream destination is a data warehouse.

Sending SendGrid Data to a Data Warehouse

Each source in Segment has its set of options for available destinations. For the SendGrid source, however, the only option for a destination is a data warehouse. (Later, in the section on source functions, we’ll show a workaround for additional ways to leverage SendGrid data.)

For our data warehouse, we will use a PostgreSQL database.

Set up a PostgreSQL database

While any PostgreSQL database in the cloud is supported by Segment, we set up an AWS RDS database running PostgreSQL 14.4. For AWS, we also need to ensure that our database is part of a Security Group with an inbound rule accepting connections from Segment by using the CIDR 52.25.130.38/32.

Add data warehouse as a destination in Segment

In Segment, we click Add Destination and search for the Postgres destination.

Next, we are asked to choose a data source for this warehouse. We see our (only) source, called “My SendGrid Data.” We select it and click Next.

Finally, we provide a name for our destination (“My PostgreSQL Data Warehouse”) along with credentials and connection information.

We click Connect to complete the destination setup.

Segment pushes SendGrid data regularly

Once your destination is created, Segment will perform an initial sync, forwarding any source data stored in Segment Hub to the destination. After this, Segment will sync data to your data warehouse once or twice per day, depending on your Segment plan.

After the first sync completes, we can inspect our database to see what data was sent over. Keep in mind that all data related to a specific source is accessible in the schema based on that source’s name. For our example, with our SendGrid source called “My SendGrid Data,” the schema in our PostgreSQL database will be named my_sendgrid_data. The corresponding schema name for each source can be found on the Warehouse Overview page for your destination. (Note: The schema name will use underscores instead of the displayed hyphens.)

Using a PostgreSQL client to connect to our instance, we can see the data that has been sent from Segment to our data warehouse.