Challenge: Our client wanted to build a data product that would enable companies in their ecosystem to efficiently quantify demand.
Our client, a leading investment firm, wanted to build a data product to enable companies in their ecosystem to quantify demand for B2B products and services and generate marketing leads.
Our client identified an external data source ZoomInfo, that produces B2B product demand data. They wanted to integrate this third-party data and prove the value and fit for users in their ecosystem. It was also a priority for our client to demonstrate near-instantaneous result delivery, ensuring an ideal user experience.
We initially built the data transformation via Snowflake - the existing DWH environment of our client. With our clients' guidance, we iteratively developed flexible and concise dashboards in looker. As a last step, we migrated the solution to Firebolt to take advantage of their focus and differentiation on the speed of the reporting and visualization layer.
Since the data provider was also on Snowflake, the data extraction is straightforward: The data provider exposes the data via a Snowflake share.
Following our standards, we have worked with the provider on implementing data quality steps detecting schema changes, unique test failures, and freshness and recency issues.
We quickly examined the data, decided on the transformation steps needed to bring the data in shape for the required granularity, and integrated it with other sources where required. With the data being heavy, optimizing the dashboard delivery speed had to start with the data transformation. It was a priority for us to design the final data mart tables accordingly. This was our main focus during the transformation phase.
We finalized the Looker dashboards achieving informative, flexible, and concise dashboards. However, the dashboard load speeds were not at the desired level. Our client wanted to take advantage of Firebolt. One of the differentiation points of firebolt is its focus on visualization speed.
To validate the claimed speed of Firebolt, we built the mart tables in the firebolt environment. Optimization at the Looker and firebolt level achieved the targeted dashboard load speed improvement. As a result, we decided to serve the dashboard data from Firebolt.
We used dbt cloud to unload the data to an S3 bucket that Firebolt reads from. When we automated the project with dbt cloud, in addition to transforming the data in snowflake, dbt unloads the data in an S3 bucket. Firebolt then reads the data and serves to the looker dashboard.
The Lead Generation product has enabled our client to offer tangible value from its data to the companies in its ecosystem. The ecosystem participants now have an engine they can use to perform strategic market analysis, size the addressable market, and identify potential clients for their products and services and connect with them.
Additional notes: While developing the GTM product:
git, dbt, snowflake, firebolt, looker
%203-svg%20(2).png)