dbt-duckdb - Meltano Hub

dbt DuckDB (EDK preview)

dbt-duckdb from jwills

The dbt-duckdb utility

is an adapter-specific dbt transformer for running SQL-based transformations on data stored in your warehouse.

EDK Preview

This utility is in a preview stage of readiness. It is based upon our Extension Developer Kit (EDK) which is the new way to build and add plugins to Meltano Hub. This preview of dbt will replace the native transformer once it comes out of preview. In the meantime, we would appreciate any feedback in the EDK repo and in the dbt Extension repo.

Getting Started

Prerequisites

If you haven't already, follow the initial steps of the Getting Started guide:

Installation and configuration

Add the dbt-duckdb utility to your project using
```
meltano add
```
:

meltano add utility dbt-duckdb

Configure the dbt-duckdb settings using
```
meltano config
```
:

meltano config dbt-duckdb set --interactive

Next steps

If you're running dbt for the first time in a new environment:

# create a starter dbt_project.yml file, a profiles.yml file, and models directory
meltano invoke dbt-duckdb:initialize

If you run into any issues, learn how to get help.

Capabilities

This plugin currently has no capabilities defined. If you know the capabilities required by this plugin, please contribute!

Settings

Settings for dbt itself can be configured through dbt_project.yml as usual, which can be found at transform/dbt_project.yml in your Meltano project.

The dbt-duckdb settings that are known to Meltano are documented below. To quickly find the setting you're looking for, click on any setting name from the list:

skip_pre_invoke
type
project_dir
profiles_dir
path
schema

You can override these settings or specify additional ones in your meltano.yml by adding the settings key.

Please consider adding any settings you have defined locally to this definition on MeltanoHub by making a pull request to the YAML file that defines the settings for this plugin.

Skip Pre-invoke (skip_pre_invoke)

Environment variable: DBT_DUCKDB_SKIP_PRE_INVOKE
Default Value: false

Whether to skip pre-invoke hooks which automatically run dbt clean and deps

dbt Profile type (type)

Environment variable: DBT_DUCKDB_TYPE
Default Value: duckdb

[No description provided.]

Projects Directory (project_dir)

Environment variable: DBT_DUCKDB_PROJECT_DIR
Default Value: $MELTANO_PROJECT_ROOT/transform

[No description provided.]

Profiles Directory (profiles_dir)

Environment variable: DBT_DUCKDB_PROFILES_DIR
Default Value: $MELTANO_PROJECT_ROOT/transform/profiles/duckdb

[No description provided.]

Path (path)

Environment variable: DBT_DUCKDB_PATH

The path on your local filesystem where you would like the DuckDB database file and it's associated write-ahead log to be written.

Schema (schema)

Environment variable: DBT_DUCKDB_SCHEMA
Default Value: main

Specify the schema to write into.

Something missing?

This page is generated from a YAML file that you can contribute changes to.

Edit it on GitHub!

Looking for help?

If you're having trouble getting the dbt-duckdb utility to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the

#plugins-general

channel.

Install

meltano add utility dbt-duckdb

Homepage

Maintenance Status

Repo

https://github.com/jwills/dbt-duckdb

Maintainer

Meltano Stats

Keywords