Data Warehouse Manager

The Data Warehouse Manager, accessed by clicking Manage Data > Data Warehouse, is the portal to your Adobe Commerce Intelligence Data Warehouse. Using the Data Warehouse Manager, you can manage table and column sync settings, drill down into a table’s schema, and create calculated columns to use in reports.

This topic covers:

Learning your way around

The left side of Data Warehouse Manager page contains the tables list, allowing you to easily toggle between tables. When you select a table from the list, the table management area populates with the table’s schema where you can modify the selected table.

Within the table list, tables are grouped by their connection source. These sources are added under Manage Data > Integrations and may be either a database, an API, or a third-party connector. At the top of the table list, there is a search box that enables you to easily find desired tables.

Underneath the search box, you see two options: All Tables and Synced Tables. The All Tables option lists all the tables that you have made available to your Data Warehouse, which includes both synced and unsynced tables.

The Synced Tables option shows all tables that have already been added into your Data Warehouse and have data being replicated from the selected columns.

Do not see the table that you are looking for in the All Tables list? There are a few possible reasons for this:

Syncing tables and columns

Syncing New Tables and Native Columns

The Data Warehouse Manager not only gives you the ability to easily view and manage your data sources, you also have the freedom to select the individual tables and columns you want to sync.

A Success! message displays and the status changes to Pending for the selected columns. After the next full update completes, the newly synced tables and columns will be available for use in reports. You can also set new replication methods after the initial sync.

Here is a quick look at the whole process:

Syncing New Tables in the Background

When you a sync a large table for the first time, your Data Warehouse needs to retroactively capture all data points in the table before capturing new data on an ongoing basis. If your table is large, you may not want to have that initial sync run in sequence with your update cycle. In this situation, you want the initial sync to occur in the background, in parallel with any currently running update.

To make sure that occurs, you should select the Save and Sync Data Immediately option syncing that table for the first time.

Checking for new tables and columns

Your Data Warehouse does not automatically detect new sources, tables, or columns the moment they’re added. A synchronization process runs throughout the week to find new additions and make them available, but you can force a structure synchronization if you want to access newly added tables and columns before the process runs.

Below the search bar in the table list is a Check for new tables and columns link. Clicking this link will force-start the structure sync process; new additions are typically available after 10 minutes. Refresh the page to see the new source, table, or column.

Creating Calculated Columns

Simply being able to see and manage data from all your sources makes gaining insights into your business that much easier. But within the Data Warehouse Manager, you can go a step further by creating calculated columns inside your tables. Calculated columns derive new information from your existing data.

Say you want to add user's lifetime revenue to your users table to find high value users. Or, if you want to segment revenue by gender, you can add customer's gender to your orders table.

For more information, check out this tutorial.

Dropping Tables and Removing Columns

Just as you can select tables and columns to sync to your Data Warehouse, you can also drop or remove them.

Do not worry if you click Delete by accident. A dependency check runs before anything is deleted, so you have the chance to review everything before confirming.

To remove columns, click the table that the column belongs to. Check the columns that you want to remove and click the button.

To remove a synced table, select all columns in the table, and again click the button. This removes all native and calculated columns that use this table from your Data Warehouse.

Confirming Changes

Whether you are dropping a table or removing columns, a dependency check runs before the deletion process completes. Dependencies are calculated columns, metrics, filter sets, and reports that use the table or column(s) being removed. Any discovered dependencies display - at this point, you can either cancel the process or click Confirm Changes to drop the table/remove the column(s).

While deleted dependencies cannot be restored, the tables and columns will still be available if you need to resync any native columns in the future.

Here is a quick look at removing a column:

So, when can I use my new columns?

New synced columns and new/updated calculated columns will be ready for use after the next full update completes. If an update is not already in progress, you can force an update by clicking Force update shown at the top of the Data Warehouse or Integrations page. You can also schedule an email notification upon completion of the update by clicking Email me when complete.

When you are ready to use your new columns in reports, you need to add them to metrics first. Although data is not available until an update completes, you can still use new columns in reports. Data within the report displays when the update is finished.

Wrapping up

This article covered a lot of material. By now, you should have a solid understanding of what a database is, how data is organized, how tables relate to each other, and what you can do with the Data Warehouse Manager.

Go test your knowledge by creating a calculated column or making some interesting reports.