Please note that this feature is currently in beta and parts of the dbt metric specification may not yet be fully supported.
Overview
With the integration of dbt, you can create metric definitions in code within your dbt core project and effortlessly synchronize them with your Steep workspace. By doing so, you can maintain a unified source of truth for your metric definitions within version-controlled code. Additionally, metrics generated in dbt can be combined with manually defined metrics in Steep, allowing for flexibility when organizing your workspace.
Prerequisites
Before you can import your dbt metrics to Steep you need to connect Steep to your database, the same database that is used by your dbt project. See Connect database.
Metric definitions in dbt
As a starting point you need to define your metrics in your dbt core project according to the dbt metrics specification. Definitions can be placed in any .yml file under the metrics key. For the full specification, read more in the dbt docs.
Example yml definition below.
metrics: - name: new_customers label: New Customers model: ref('dim_customers') description: "Count of new paying customers using the product" calculation_method: count_distinct expression: user_id timestamp: signup_date time_grains: [day, week, month, quarter, year] dimensions: - plan - country filters: - field: is_paying operator: "is" value: "true"
Manual import of metrics
You can manually upload your dbt metrics to Steep, which is a good choice as a starting point or for metric definitions that rarely change. To manually import metric definitions, you can run your dbt project locally and import the resulting manifest.json file into Steep.
1. Run your dbt project
Run your dbt project to produce a manifest.json eg. using dbt run
or dbt build
.
2. Import manifest.json file in Steep
Go to the dbt integration page in your Steep workspace and upload the manifest.json file. Steep will then provide feedback on any metrics that were created, updated, or skipped during the import process.
3. Explore your metrics
Once the import process is complete, explore your metrics in the sidebar and begin analyzing the metric data right away. Explore metrics.
Automatic sync of metrics via API
To ensure that your metrics are always up to date in Steep, you can sync your dbt metrics automatically via the Steep API as part of your continuous integration process or scheduled production run.
1. Create API key
To get started, you need to create an API key. You can create a key on the dbt integration page in your Steep workspace. Click the "Create API Key" button to create a new key.
2. Post manifest.json to API
Run your dbt project in the environment of your choice, and then upload the manifest.json by using POST
to https://api.steep.app/dbt-metrics
, attaching your API key as an authorization header. See example below using curl.
curl -H "Content-Type: application/json" \ -H "Authorization: ApiKey API_KEY_HERE" \ -X POST \ -d @/path/to/manifest.json \ https://api.steep.app/dbt-metrics
The API will return 200 with a response body indicating the outcome of the operation.
If validation of the manifest fails the response will include an error message.
{ outcome: "failed"; error: string; }
If successul the response will contain references to the dbt unique_id field for each metric created, updated or that failed with an error message.
{ outcome: "completed"; created: string[]; updated: string[]; errors: Record<string, string>; }
3. Explore your metrics
Once the sync has successfully completed, your Steep metric definitions will have been updated accordingly and you can explore and analyze your metric data within Steep. Explore metrics.