Redpoint DM + Google BigQuery + AEP

Architecture summary

In this reference solution architecture document, we present a strategic framework for integrating Redpoint Data Management (RPDM) with Google BigQuery's cloud-based data warehousing and analytics platform along with Adobe's customer engagement platform, Adobe Experience Platform (AEP). This integration is designed to leverage the strengths of each platform, facilitating a seamless flow of unified customer data from Redpoint Data Management using Google BigQuery’s data warehousing and integrating AEP for targeted and personalized customer engagement.

By outlining the technical and strategic considerations, this document serves as a comprehensive guide for our internal users, partners, and clients, empowering them to effectively unify customer data, derive actionable insights, and make data-driven decisions with unparalleled speed and accuracy. The architecture aims to optimize data utilization across platforms, enhance data governance and security, and provide a single source of truth for customer data to the Adobe platform ecosystem.

With Google BigQuery's scalable and high-performance data warehousing capabilities, organizations can efficiently store, manage, and analyze massive volumes of customer data, enabling real-time insights and personalized customer experiences. By integrating RPDM with Google BigQuery, businesses can unlock the full potential of their customer data, drive innovation, and gain a competitive edge in today's data-driven landscape. Additionally, by working in tandem with AEP, businesses can enhance interactions across multiple channels, including web, mobile, email, SMS, and mobile apps, further enriching the customer experience.

Example use cases

• Identity resolution and profile unification across any data source or structure

• Unified customer view for personalized campaigns and experiences across multiple channels

• Real-time customer insights for immediate action

• Simplified data governance and compliance management

• Enhanced data security and privacy controls

• Scalable data storage for growing customer data volumes

• Customer lifecycle management

Logical diagram

Key components and roles

• Redpoint Data Management

  • Unified profile of individuals

  • Identity resolution with key persistence

  • API-accessible single customer views

• Google BigQuery

  • Data warehouse and storage

  • Secure data sharing

  • Data governance

  • Data modeling

  • Scalable point of data integration

• Adobe Experience Platform

  • Audience definition / segmentation

  • Orchestration

  • Personalized messaging

  • Real-time engagement - triggered messages

  • Real-time engagement - on-site personalization

Data flows

Why does this architecture/solution make sense?

In this architecture, Redpoint Data Management acts as the central point of coordination for customer data collection, unification, and enrichment, serving as the single point of control for customer data, both consuming data from and persisting data within Google BigQuery. Google BigQuery functions as the robust and scalable data storage and processing powerhouse, enabling efficient data warehousing, advanced analytics, and secure data sharing. By integrating these two platforms, organizations can create a seamless data pipeline that leverages the strengths of each system.

RPDM's ability to collect and refine customer data from various sources, coupled with its advanced identity resolution and data quality management capabilities, ensures that high-quality, unified customer profiles are readily available. Google BigQuery's cloud-based architecture and near-infinite scalability allow for the storage and processing of massive volumes of customer data, enabling real-time analytics and insights.

This integration empowers organizations to break down data silos, achieve a holistic view of their customers, and make data-driven decisions with unparalleled speed and accuracy. By leveraging Google BigQuery's data sharing capabilities, organizations can securely share customer insights across departments and with external partners, fostering collaboration and innovation. Moreover, the combined solution helps organizations maintain data governance and compliance with global data privacy regulations, as both platforms prioritize data security and privacy.

The RPDM + Google BigQuery architecture creates a powerful, end-to-end customer data management and analytics solution that drives personalized customer experiences, optimizes marketing strategies, and fuels business growth in today's data-driven landscape.

Furthermore, integrating AEP streamlines outbound communications by supporting robust marketing campaigns, personalized customer journeys, and uniform messaging across diverse channels. This approach ensures all communications are cohesive, enhancing customer engagement. Moreover, AEP captures essential marketing disposition data during these interactions and relays it to RPDM via Google BigQuery as well as other data transfer mechanisms. This process enriches the customer data profile, offering a holistic view of customer behaviors and enabling more precise adjustments to engagement strategies. Collectively, Google BigQuery, AEP, and RPDM forge a seamless, data-driven communication ecosystem that meets the demands of complex customer engagement environments.

Business benefits

• 360-Degree Customer View: By integrating RPDM, Google BigQuery, and AEP, organizations can achieve a comprehensive, unified view of their customers, enabling them to deliver highly personalized experiences, improve customer satisfaction, and increase customer lifetime value.

• Enhanced Data-Driven Decision Making: The combination of RPDM's data unification and enrichment capabilities with Google BigQuery's advanced analytics and real-time data processing, along with AEP’s outbound multi-channel execution and disposition data feedback, empowers businesses to make informed, data-driven decisions that drive growth, optimize marketing strategies, and improve operational efficiency.

• Scalability and Performance: Google BigQuery's cloud-based architecture and near-infinite scalability, coupled with RPDM's robust data management capabilities, allow organizations to efficiently store, process, and analyze massive volumes of customer data, ensuring optimal performance and the ability to grow with the business. Integrating AEP for outbound delivery and personalized experiences further increases the resources of the platform and by proxy extends the scalability and performance.

• Secure Data Sharing and Collaboration: Google Cloud Platform and Adobe support various integration methods that support secure data sharing and collaboration between platforms, helping teams foster collaboration, innovation, and data-driven decision-making.

• Reduced Data Silos and Improved Efficiency: By integrating RPDM, Google BigQuery, and AEP, businesses can break down data silos, streamline data management processes, and reduce the time and effort required to derive valuable customer insights, ultimately improving operational efficiency and reducing costs.

Technical benefits

• Unified and Enriched Customer Data: RPDM's advanced data unification and enrichment capabilities consolidate customer data from various sources, creating a single, comprehensive view of each customer and enhancing data quality for accurate analysis and targeting.

• Scalable and High-Performance Data Warehousing: Google BigQuery's cloud-based architecture and near-infinite scalability enable organizations to store, manage, and process massive volumes of customer data efficiently, ensuring optimal performance and the ability to handle growing data needs.

• Real-Time Data Processing and Analytics: Google BigQuery's real-time data processing capabilities, combined with RPDM's robust data processing features, allow organizations to analyze customer data and derive actionable insights in real-time, enabling faster decision-making and more responsive customer engagement.

• Seamless Data Integration and Connectivity: The integration between RPDM and Google BigQuery is streamlined, leveraging Google BigQuery's ODBC and JDBC drivers for efficient querying and data transfer. AEP offers various connectivity approaches, including APIs and direct connection methods, further supporting the goal of providing seamless data integrations. This simplifies the process of connecting the platforms and reduces the complexity of managing data pipelines, saving time and resources.

• Robust Data Security and Compliance: Both RPDM and Google BigQuery prioritize data security and privacy, offering advanced security features such as encryption, access controls, and data governance tools to protect sensitive customer data and ensure compliance with global data privacy regulations.

• Collaborative Data Sharing and Accessibility: Google BigQuery's secure data sharing capabilities allow organizations to easily share customer data and insights across departments and with external partners, such as AEP.

• Simplified Data Governance and Lineage: The combined solution simplifies data governance by providing tools for data lineage tracking, data cataloging, and metadata management. This ensures that data is consistently and accurately defined, making it easier to maintain data quality, comply with regulations, and track data usage across the organization.

Advantages of Redpoint Data Management

There are numerous advantages to incorporating Redpoint Data Management into your marketing technology ecosystem, including functionalities such as data management operations, ETL (Extract, Transform, Load), and Identity Resolution, among others. Next, we will examine Redpoint’s and Adobe’s approaches to Identity Resolution.

Identity Resolution

The goal of Identity Resolution (IDR) is to take disparate data sources and unify them in a way that provides a complete view of the customer (PII) and behavioral (event) data from which you can generate a unified profile. There are different approaches to perform IDR that include standardized processing along with algorithms to establish a link between the data, whereas other platforms capture a set of IDs and use the capturing of those source system IDs to stitch together a profile.

Redpoint

Redpoint’s approach to Identity Resolution (IDR) uses sophisticated and comprehensive hygiene procedures and probabilistic techniques to match individuals and households using a variety of available data. This enables the unique identification of individuals and the detection of their corresponding associations. IDR processes new and updated records against previously identified matches, and multiple disparate sources of data can be processed simultaneously.

Refer to the documentation topic on Redpoint Matching Functionality and Capabilities.

The Redpoint IDR process flow involves:

• Data collection

• Hygiene, enhancement, and standardization of Personally Identifiable Information (PII) data

• Match process

• Match ID assignment

• Output / storage

Redpoint’s IDR process is non-destructive, meaning that it:

• Maintains the original source input data

• Supplements and enhances the PII data

• Assigns a persistent INDIVIDUAL_ID and HOUSEHOLD_ID to each customer account

Adobe

Adobe uses a combination of tools/services to define what fields are designated as identity fields. These identities are defined as primary and secondary identity fields. When a record comes into the system and it contains more than one identity field, then the two IDs are mapped to each other in the identity graph based on the hierarchy of the IDs found in the record. The identity graph can then be used to define a policy on how to merge the data to generate what is referenced as a Real-Time Customer Profile.

Identity Service

You can use Identity Service to generate and maintain the identity graph that brings together the disparate identities of an individual customer. To use Identity Service, you must ingest record data or time series events that have at least two fields that are marked as identity. The fields that you mark as identity are then ingested into Identity Service.

Real-Time Customer Profile

You can use Real-Time Customer Profile to bring together disparate profile fragments and create a merged profile. This process requires the use of the identity graph.

High-level comparison table

There are primary differences in the way that IDR works between Redpoint and Adobe.

• Redpoint uses PII data and algorithms to determine individuals and households and assign unique IDs and then associates those IDs with the inbound records that were provided, like customer account records. From there, additional aggregations are generated based on rules to define identity profiles.

• Adobe uses input fields designated as identity fields (primary and secondary) to associate provided identity fields to each other. From there, profile merge rules are defined to create profiles from the linked data.

The following table delineates some features of IDR and what each platform supports.

Overall architectural considerations

The Adobe stack is vast, and AEP contains a fundamental set of tools, which populates the data layer that supports its marketing applications. The integration patterns for Google BigQuery are well known and very straightforward to set up and maintain. As RPDM leverages Google BigQuery as both a data source and the target for unified customer profiles, using this data to hydrate Adobe and fuel Adobe-powered customer experience provides a robust combined ecosystem.