Admin: Configuring Snowflake Warehouse Overrides

Overview

If you run RPI on Snowflake, not every process RPI runs needs the same amount of horsepower. Some processes are lightweight but chatty; others are resource-intensive but infrequent. Snowflake warehouse overrides let you split those processes across different warehouses instead of running everything on one, which helps control Snowflake consumption cost while keeping RPI responsive as your environment scales.

This configuration allows you to dynamically assign specific Snowflake warehouses to different actions. By overriding the default warehouse, you can optimize credit usage:

• Scale Up: use larger warehouses for performance-heavy tasks like Interactions or Audience Tests.

• Scale Down: use smaller, cost-effective warehouses for routine System Tasks or Data Synchronizations.

What should I override?

Look at where most of your database resourcing and processing time is being spent.

• Processes with a heavier, more consistent load are good candidates for isolation onto their own warehouse.

• Smaller, frequent, "chatty" processes can often be offloaded together onto a separate warehouse.

Step 1: Identify the target process

To trigger an override, you must identify the CommandType and/or the TaskType of the process.

• CommandType: maps to the rpi_ExecutionSchedule.Type field. It identifies if the process is a Client Job, or Audience, etc.

  • Task Command

  • Client Job

  • Delay

  • Broadcast

  • Wait for event

  • Subscription Group

  • Rollback

  • Workflow Tidyup

  • Audience

  • Fulfillment

• TaskType: provides granular identification:

  • Tasks: use the specific Task Name, mapping to the rpi_tasks table.

    • Housekeeper

    • Web events stream analytics importer

    • Web form processor

    • Channel <insert channel name> QL data synchronization

    • Client health monitor

    • Fulfillment state flow count updates

    • Audience snapshot requests

    • Web events importer

    • Export backfill state data

    • Web cache data importer

    • Event alerting

    • Channel <insert channel name> data synchronization

    • Callback service processor

    • Channel <insert channel name> with File Import data synchronization

    • System health monitor

    • Attribute value catalog

    • Goal smart asset monitor

    • Channel <insert channel name> data synchronization

  • Client Jobs: use the job type (from op_ClientJobs).

    • Analysis panel refresh

    • Analyze

    • Broadcast Response

    • Cache attribute values

    • Catalog re-sync

    • Configuration item export

    • Connectivity test

    • Dashboard execution

    • Data Import Data Load

    • Data viewer

    • Database Test

    • Delete staged email offer content

    • Diagnostics information

    • Download asset from ECP

    • Email delivery test

    • Email link test

    • Export Tenant Items

    • File download

    • File export

    • File external copy

    • File import

    • Fulfillment test

    • Get Single Customer View

    • Joins audit

    • Joins refresh

    • NoSQL catalog

    • Offline Configuration

    • Perform Single Customer View Search

    • Publish to web

    • Re-subscribe contacts

    • Refresh value list

    • Render offer content

    • Rule export

    • Rule NLP creation

    • Rule Visualization

    • Sample Database Table

    • Scheduled triggers report

    • Selection rule waterfall

    • Telemetry report

    • Unpublish

    • Update attributes in NLP model

    • Update cached attribute list table

    • Upload asset to ECP

    • Upload file to ECP

    • Validate audience definitions

    • Validate channels

    • Validate SQL Expression

    • Workflow diagnostics

    • Workflow waterfall report

  • Workflows: use the specific workflow type (e.g., Interaction, Audience Test from op_InteractionWorkflows).

    • Interaction

    • Audience Test

    • Audience Snapshot

Step 2: Create a value list

Create a new value list to serve as your mapping table. The system evaluates these from most specific to most general.

• Label: set to CommandType (general) or CommandType:TaskType (specific).

• Value: set to the exact name of the Snowflake warehouse.

Sample Mapping Entry

Step 3: Update System Configuration

Once the Value List is defined, link it within the System Configuration settings:

• SnowflakeDefaultWarehouse: enter the name of your baseline warehouse. This is used if no match is found in your override list.

• SnowflakeWarehouseOverrides: select the Value List you created in Step 2.

How the System Resolves Warehouses

The system looks for a warehouse in this specific order:

1. Specific Match: CommandType:TaskType (e.g., Audience:Interaction)

2. General Match: CommandType (e.g., Audience)

3. Global Default: SnowflakeDefaultWarehouse