Metabase Drivers handle various things we need to do with connected data warehouse databases, including things like introspecting their schemas and processing and running MBQL queries. Drivers must implement some or all of the multimethods defined below, and register themselves with a call to [[metabase.driver/register!]].

SQL-based drivers can use the :sql driver as a parent, and JDBC-based SQL drivers can use :sql-jdbc. Both of these drivers define additional multimethods that child drivers should implement; see [[metabase.driver.sql]] and [[metabase.driver.sql-jdbc]] for more details.

Send notification that all Databases should immediately release cached resources (i.e., connection pools).

Currently only used below by [[report-timezone]] setter (i.e., only used when report timezone changes). Reusing pooled connections with the old session timezone can have weird effects, especially if report timezone is changed to nil (meaning subsequent queries will not attempt to change the session timezone) or something considered invalid by a given Database (meaning subsequent queries will fail to change the session timezone).

Current report timezone abbreviation

Current report timezone string

+----------------------------------------------------------------------------------------------------------------+ | Current Driver | +----------------------------------------------------------------------------------------------------------------+

Current driver (a keyword such as :postgres) in use by the Query Processor/tests/etc. Bind this with with-driver below. The QP binds the driver this way in the bind-driver middleware.

Impl for with-driver.

Bind current driver to driver and execute body.

(driver/with-driver :postgres ...)

+----------------------------------------------------------------------------------------------------------------+ | Driver Registration / Hierarchy / Multimethod Dispatch | +----------------------------------------------------------------------------------------------------------------+

Is this driver available for use? (i.e. should we show it as an option when adding a new database?) This is true for all registered, non-abstract drivers and false everything else.

Note that an available driver is not necessarily initialized yet; for example lazy-loaded drivers are registered when Metabase starts up (meaning this will return true for them) and only initialized when first needed.

Like [[clojure.core/the-ns]]. Converts argument to a keyword, then loads and registers the driver if not already done, throwing an Exception if it fails or is invalid. Returns keyword. Note that this does not neccessarily mean the driver is initialized (e.g., its full implementation and deps might not be loaded into memory) -- see also [[the-initialized-driver]].

This is useful in several cases:

;; Ensuring a driver is loaded & registered (isa? driver/hierarchy (the-driver :postgres) (the-driver :sql-jdbc)

;; Accepting either strings or keywords (e.g., in API endpoints) (the-driver "h2") ; -> :h2

;; Ensuring a driver you are passed is valid (t2/insert! Database :engine (name (the-driver driver)))

(the-driver :postgres) ; -> :postgres (the-driver :baby) ; -> Exception

Add a new parent to driver.

Dispatch function to use for driver multimethods. Dispatches on first arg, a driver keyword; loads that driver's namespace if not already done. DOES NOT INITIALIZE THE DRIVER.

Driver multimethods for abstract drivers like :sql or :sql-jdbc should use [[dispatch-on-initialized-driver]] to ensure the driver is initialized (i.e., its method implementations will be loaded).

Like [[the-driver]], but also initializes the driver if not already initialized.

Like [[dispatch-on-uninitialized-driver]], but guarantees a driver is initialized before dispatch. Prefer [[the-driver]] for trivial methods that should do not require the driver to be initialized (e.g., ones that simply return information about the driver, but do not actually connect to any databases.)

+----------------------------------------------------------------------------------------------------------------+ | Interface (Multimethod Defintions) | +----------------------------------------------------------------------------------------------------------------+

Methods a driver can implement. Not all of these are required; some have default implementations immediately below them.

SOME TIPS:

To call the Clojure equivalent of the superclass implementation of a method, use get-method with the parent driver:

(driver/register-driver! :my-driver, :parent :sql-jdbc)

(defmethod driver/describe-table :my-driver [driver database table] (-> ((get-method driver/describe-table :sql-jdbc) driver databse table) (update :tables add-materialized-views)))

Make sure to pass along the driver parameter-as when you call other methods, rather than hardcoding the name of the current driver (e.g. :my-driver in the example above). This way if other drivers use your driver as a parent in the future their implementations of any methods called by those methods will get used.

DO NOT CALL THIS METHOD DIRECTLY. Called automatically once and only once the first time a non-trivial driver method is called; implementers should do one-time initialization as needed (for example, registering JDBC drivers used internally by the driver.)

'Trivial' methods include a tiny handful of ones like [[connection-properties]] that simply provide information about the driver, but do not connect to databases; these can be be supplied, for example, by a Metabase plugin manifest file (which is supplied for lazy-loaded drivers). Methods that require connecting to a database dispatch off of [[the-initialized-driver]], which will initialize a driver if not already done so.

You will rarely need to write an implentation for this method yourself. A lazy-loaded driver (like most of the Metabase drivers in v1.0 and above) are automatiaclly given an implentation of this method that performs the init-steps specified in the plugin manifest (such as loading namespaces in question).

If you do need to implement this method yourself, you do not need to call parent implementations. We'll take care of that for you.

VERY IMPORTANT: Unlike all other driver multimethods, we DO NOT use the driver hierarchy for dispatch here. Why? We do not want a driver to inherit parent drivers' implementations and have those implementations end up getting called multiple times. If a driver does not implement initialize!, always fall back to the default no-op implementation.

initialize-if-needed! takes care to make sure a driver's parent(s) are initialized before initializing a driver.

A nice name for the driver that we'll display to in the admin panel, e.g. "PostgreSQL" for :postgres. Default implementation capitializes the name of the driver, e.g. :oracle becomes "Oracle".

When writing a driver that you plan to ship as a separate, lazy-loading plugin (including core drivers packaged this way, like SQLite), you do not need to implement this method; instead, specifiy it in your plugin manifest, and lazy-loaded-driver will create an implementation for you. Probably best if we only have one place where we set values for this.

The contact information for the driver

Dispatch on initialized driver, except checks for classname, subprotocol, connection-uri in the details map in order to prevent a mismatch in spec type vs driver.

Check whether we can connect to a Database with details-map and perform a simple query. For example, a SQL database might try running a query like SELECT 1;. This function should return truthy if a connection to the DB can be made successfully, otherwise it should return falsey or throw an appropriate Exception. Exceptions if a connection cannot be made. Throw an ex-info containing a truthy ::can-connect-message? in ex-data in order to suppress logging expected driver validation messages during setup.

Return a map containing information that describes the version of the DBMS. This typically includes a :version containing the (semantic) version of the DBMS as a string and potentially a :flavor specifying the flavor like MySQL or MariaDB.

Some drivers like BigQuery or Snowflake cannot provide a meaningful stable version.

Return a map containing information that describes all of the tables in a database, an instance of the Database model. It is expected that this function will be peformant and avoid draining meaningful resources of the database. Results should match the [[metabase.sync.interface/DatabaseMetadata]] schema.

Return a map containing a single field :fields that describes the fields in a table. database will be an instance of the Database model; and table, an instance of the Table model. It is expected that this function will be peformant and avoid draining meaningful resources of the database. The value of :fields should be a set of values matching the [[metabase.sync.interface/TableMetadataField]] schema.

Returns a reducible collection of maps, each containing information about fields. It includes which keys are primary keys, but not foreign keys. It does not include nested fields (e.g. fields within a JSON column).

Takes keyword arguments to narrow down the results to a set of schema-names or table-names.

Results match [[metabase.sync.interface/FieldMetadataEntry]]. Results are optionally filtered by schema-names and table-names provided. Results are ordered by table-schema, table-name, and database-position in ascending order.

Returns a set of map containing information about the indexes of a table. Currently we only sync single column indexes or the first column of a composite index. Results should match the [[metabase.sync.interface/TableIndexMetadata]] schema.

Returns a reducible collection of maps, each containing information about the indexes of a database. Currently we only sync single column indexes or the first column of a composite index. We currently only support indexes on unnested fields (i.e., where parent_id is null).

Takes keyword arguments to narrow down the results to a set of schema-names or table-names.

Results match [[metabase.sync.interface/FieldIndexMetadata]]. Results are optionally filtered by schema-names and table-names provided.

escaping for when calling .getColumns or .getTables on table names or schema names. Useful for when a database driver has difference escaping rules for table or schema names when used from metadata.

For example, oracle treats slashes differently when querying versus when used with .getTables or .getColumns

Return information about the foreign keys in a table. Required for drivers that support :metadata/key-constraints but not :describe-fks. Results should match the [[metabase.sync.interface/FKMetadata]] schema.

Returns a reducible collection of maps, each containing information about foreign keys. Takes optional keyword arguments to narrow down the results to a set of schema-names and table-names.

Results match [[metabase.sync.interface/FKMetadataEntry]]. Results are optionally filtered by schema-names and table-names provided. Results are ordered by fk-table-schema and fk-table-name in ascending order.

Required for drivers that support :describe-fks.

this is no longer used but we can leave it around for not for documentation purposes. Maybe we can actually do something useful with it like write a test that validates that drivers return correct connection details?

Return information about the connection properties that should be exposed to the user for databases that will use this driver. This information is used to build the UI for editing a Database details map, and for validating it on the backend. It should include things like host, port, and other driver-specific parameters. Each property must conform to the [[ConnectionDetailsProperty]] schema above.

There are several definitions for common properties available in the [[metabase.driver.common]] namespace, such as default-host-details and default-port-details. Prefer using these if possible.

Like display-name, lazy-loaded drivers should specify this in their plugin manifest; lazy-loaded-driver will automatically create an implementation for you.

Execute a native query against that database and return rows that can be reduced using transduce/reduce.

Pass metadata about the columns and the reducible object to respond, which has the signature

(respond results-metadata rows)

You can use [[metabase.query-processor.reducible/reducible-rows]] to create reducible, streaming results.

respond MUST BE CALLED SYNCHRONOUSLY!!!

Example impl:

(defmethod reducible-query :my-driver [_ query context respond] (with-open [results (run-query! query)] (respond {:cols [{:name "my_col"}]} (qp.reducible/reducible-rows (get-row results) (context/canceled-chan context)))))

Optional. Efficiently calculate metadata about the columns that would be returned if we were to run a query (hopefully without actually running), for example:

(query-results-metadata :postgres {:lib/type :mbql/query :stages [{:lib/type :mbql.stage/native :native "SELECT * FROM venues WHERE id = ?" :args [1]}] ...}) => [{:lib/type :metadata/column :name "ID" :database-type "BIGINT" :base-type :type/BigInteger} {:lib/type :metadata/column :name "NAME" :database-type "CHARACTER VARYING" :base-type :type/Text} ...]

Metadata should be returned as a sequence of column maps matching the :metabase.lib.schema.metadata/column shape.

This is needed in certain circumstances such as saving native queries before they have been run; metadata for MBQL-only queries can usually be determined by looking at the query itself without any driver involvement.

If this method does need to be invoked, ideally it can calculate this information without actually having to run the query in question; it that is not possible, ideally we'd run a faster version of the query with the equivalent of LIMIT 0 or LIMIT 1.

A naive default implementation of this method lives in [[metabase.query-processor.metadata]] that runs the query in question with a LIMIT 1 added to it. Drivers that can infer result metadata in a more performant way (i.e., without actually running the query) should implement this method.

The :sql-jdbc parent driver provides a default implementation for JDBC-based drivers in [[metabase.driver.sql-jdbc.metadata/query-result-metadata]], so you shouldn't need to implement this yourself if your driver derives from :sql-jdbc. For other drivers, please use this implementation as a reference when working on your own one.

There is no guarantee that query is already fully compiled from MBQL to the appropriate native query language (e.g. SQL), so you should call [[metabase.query-processor.compile/compile]] to get a fully-compiled native query.

Set of all features a driver can support.

Does this driver and specific instance of a database support a certain feature? (A feature is a keyword, and can be any of the ones listed above in driver-features. Note that it's the same set of driver-features with respect to both database-supports? and [[supports?]])

Database is guaranteed to be a Database instance.

Most drivers can always return true or always return false for a given feature (e.g., :left-join is not supported by any version of Mongo DB).

In some cases, a feature may only be supported by certain versions of the database engine. In this case, after implementing [[dbms-version]] for your driver you can determine whether a feature is supported for this particular database.

(database-supports? :mongo :set-timezone mongo-db) ; -> true

By default a driver supports :native-parameter-card-reference if it supports :native-parameters AND :nested-queries.

Escape a column-or-table-alias string in a way that makes it valid for your database. This method is used for existing columns; aggregate functions and other expressions; joined tables; and joined subqueries; be sure to return the lowest common denominator amongst if your database has different requirements for different identifier types.

These aliases can be dynamically generated in [[metabase.query-processor.util.add-alias-info]] or elsewhere (usually based on underlying table or column names) but can also be specified in the MBQL query itself for explicit joins. For :sql drivers, the aliases generated here will be quoted in the resulting SQL.

The default impl of [[escape-alias]] calls [[metabase.driver.impl/truncate-alias]] and truncates the alias to [[metabase.driver.impl/default-alias-max-length-bytes]]. You can call this function with a different max length if you need to generate shorter aliases.

That method is currently only used drivers that derive from :sql and for drivers that support joins. If your driver is/does neither, you do not need to implement this method at this time.

Return a humanized (user-facing) version of an connection error message. Generic error messages provided in [[metabase.driver.util/connection-error-messages]]; should be returned as keywords whenever possible. This provides for both unified error messages and categories which let us point users to the erroneous input fields. Error messages can also be strings, or localized strings, as returned by [[metabase.util.i18n/trs]] and metabase.util.i18n/tru.

Transpile an MBQL query into the appropriate native query form. query will match the schema for an MBQL query in [[metabase.legacy-mbql.schema/Query]]; this function should return a native query that conforms to that schema.

If the underlying query language supports remarks or comments, the driver should use [[metabase.query-processor.util/query->remark]] to generate an appropriate message and include that in an appropriate place; alternatively a driver might directly include the query's :info dictionary if the underlying language is JSON-based.

The result of this function will be passed directly into calls to [[execute-reducible-query]].

For example, a driver like Postgres would build a valid SQL expression and return a map such as:

{:query "-- Metabase card: 10 user: 5 SELECT * FROM my_table"}

In 0.51.0 and above, drivers should look the value of [[compile-with-inline-parameters]] and output a query with all parameters inline when it is truthy.

Pretty-format native form presumably coming from compiled query. Used eg. in the API endpoint /dataset/native, to present the user with a nicely formatted query.

How to use and extend this method?

At the time of writing, this method acts as identity for nosql drivers. However, story with sql drivers is a bit different. To extend it for sql drivers, developers could use [[metabase.driver.sql.util/format-sql]]. Function in question is implemented in a way, that developers, implemnting this multimethod can: - Avoid implementing it completely, if their driver keyword representation corresponds to key in [[metabase.driver.sql.util/dialects]] (eg. :postgres). - Ignore implementing it, if it is sufficient to format their drivers native form with dialect corresponding to :standardsql's value from the dialects map (eg :h2). - Use [[metabase.driver.sql.util/format-sql]] in this method's implementation, providing dialect keyword representation that corresponds to to their driver's formatting (eg. :sqlserver uses :tsql). - Completly reimplement this method with their special formatting code.

Whether to compile an MBQL query to native with parameters spliced inline (as opposed to using placeholders like ? and passing the parameters separately.) Normally we want to pass parameters separately to protect against SQL injection and whatnot, but when converting an MBQL query to SQL it's nicer for people to see

WHERE bird_type = 'cockatiel'

instead of

WHERE bird_type = ?

so we bind this to true.

Drivers that have some notion of parameterized queries (e.g. :sql-jdbc-based drivers) should look at the value of this dynamic variable in their implementation of [[metabase.driver/mbql->native]] and adjust query compilation behavior accordingly.

Deprecated and unused in 0.51.0+; multimethod declaration left here so drivers implementing it can still compile until we remove this method completely in 0.54.0 or later.

Instead of implementing this method, you should instead look at the value of [[metabase.driver/compile-with-inline-parameters]] in your implementation of [[metabase.driver/mbql->native]] and adjust behavior accordingly.

Notify the driver that the attributes of a database have changed, or that `database was deleted. This is specifically relevant in the event that the driver was doing some caching or connection pooling; the driver should release ALL related resources when this is called.

TODO -- shouldn't this be called notify-database-updated!, since the expectation is that it is done for side effects? issue: https://github.com/metabase/metabase/issues/39367

Drivers may provide this function if they need to do special setup before a sync operation such as sync-database!. The sync operation itself is encapsulated as the lambda f, which must be called with no arguments.

(defn sync-in-context [driver database f] (with-connection [_ database] (f)))

Return a sequence of all the rows in a given table, which is guaranteed to have at least :name and :schema keys. (It is guaranteed to satisfy the DatabaseMetadataTable schema in metabase.sync.interface.) Currently, this is only used for iterating over the values in a _metabase_metadata table. As such, the results are not expected to be returned lazily. There is no expectation that the results be returned in any given order.

This method is currently only used by the H2 driver to load the Sample Database, so it is not neccesary for any other drivers to implement it at this time.

Return the system timezone ID name of this database, i.e. the timezone that local dates/times/datetimes are considered to be in by default. Ideally, this method should return a timezone ID like America/Los_Angeles, but an offset formatted like -08:00 is acceptable in cases where the actual ID cannot be provided.

This is currently used only when syncing the Database (see [[metabase.sync.sync-metadata.sync-timezone/sync-timezone!]]) -- the result of this method is stored in the timezone column of Database.

In theory this method should probably not return nil, since every Database presumably assumes some timezone for LocalDate(Time)s types, but in practice implementations of this method return nil for some drivers. For example the default implementation for :sql-jdbc returns nil unless the driver in question implements [[metabase.driver.sql-jdbc.sync/db-default-timezone]]; the :h2 driver does not for example. Why is this? Who knows, but it's something you should keep in mind.

This method should return a [[String]], a [[java.time.ZoneId]], or a [[java.time.ZoneOffset]].

For drivers that support :native-parameters. Substitute parameters in a normalized 'inner' native query.

{:query "SELECT count(*) FROM table WHERE id = {{param}}" :template-tags {:param {:name "param", :display-name "Param", :type :number}} :parameters [{:type :number :target [:variable [:template-tag "param"]] :value 2}]} -> {:query "SELECT count(*) FROM table WHERE id = 2"}

Much of the implementation for this method is shared across drivers and lives in the metabase.driver.common.parameters.* namespaces. See the :sql and :mongo drivers for sample implementations of this method.`Driver-agnostic end-to-end native parameter tests live in [[metabase.query-processor-test.parameters-test]] and other namespaces.

Return how fields should be sorted by default for this database.

Return the day that is considered to be the start of week by driver. Should return a keyword such as :sunday.

TODO -- this can vary based on session variables or connection options Issue: https://github.com/metabase/metabase/pull/39386

A multimethod for driver-specific behavior required to incorporate details for an opened SSH tunnel into the DB details. In most cases, this will simply involve updating the :host and :port (to point to the tunnel entry point, instead of the backing database server), but some drivers may have more specific behavior.

WARNING! Implementations of this method may create new SSH tunnels, which need to be cleaned up. DO NOT USE THIS METHOD DIRECTLY UNLESS YOU ARE GOING TO BE CLEANING UP ANY CREATED TUNNELS! Instead, you probably want to use [[metabase.util.ssh/with-ssh-tunnel]]. See #24445 for more information.

A multimethod for driver specific behavior required to incorporate response of an auth-provider into the DB details. In most cases this means setting the :password and/or :username based on the auth-provider and its response.

Normalizes db-details for the given driver. This is to handle migrations that are too difficult to perform via regular Liquibase queries. This multimethod will be called from a :post-select handler within the database model. The full database model object is passed as the 2nd parameter, and the multimethod implementation is expected to update the value for :details. The default implementation is essentially identity (i.e returns database unchanged). This multimethod will only be called if :details is actually present in the database map.

TODO:

1. We definitely should not be asking drivers to "update the value for :details". Drivers shouldn't touch the application database.
2. Something that is done for side effects like updating the application DB NEEDS TO END IN AN EXCLAMATION MARK! Issue: https://github.com/metabase/metabase/issues/39392

Returns the driver that supersedes the given driver. A non-nil return value means that the given driver is deprecated in Metabase and will eventually be replaced by the returned driver, in some future version (at which point any databases using it will be migrated to the new one).

This is currently only used on the frontend for the purpose of showing/hiding deprecated drivers. A driver can make use of this facility by adding a top-level superseded-by key to its plugin manifest YAML file, or (less preferred) overriding this multimethod directly.

Execute a writeback query e.g. one powering a custom QueryAction (see [[metabase.models.action]]). Drivers that support :actions/custom must implement this method.

Processes a sample of rows produced by driver, from the table's fields using the query result processing function rff. The default implementation defined in [[metabase.db.metadata-queries]] runs a row sampling MBQL query using the regular query processor to produce the sample rows. This is good enough in most cases so this multimethod should not be implemented unless really necessary. opts is a map that may contain additional parameters: :truncation-size: size to truncate text fields to if the driver supports expressions.

Sets the database role used on a connection. Called prior to query execution for drivers that support connection impersonation (an EE-only feature).

+----------------------------------------------------------------------------------------------------------------+ | Upload | +----------------------------------------------------------------------------------------------------------------+

The number of rows to insert at a time when uploading data to a database. This can be bound for testing purposes.

Return the maximum number of bytes allowed in a table name, or nil if there is no limit.

Return the maximum number of bytes allowed in a column name, or nil if there is no limit.

Create a table named table-name. If the table already exists it will throw an error. args is an optional map with an optional entry primary-key. The primary-key value is a vector of column names that make up the primary key.

Drop a table named table-name. If the table doesn't exist it will not be dropped. table-name may be qualified by schema e.g.

schema.table

Delete the current contents of table-name. If something like a SQL TRUNCATE statement is supported, we use that, but may otherwise fall back to explicitly deleting rows, or dropping and recreating the table. Depending on the driver, the semantics can vary on whether triggers are fired, AUTO_INCREMENT is reset etc. The application assumes that the implementation can be rolled back if inside a transaction.

Insert values into a table named table-name. values is a lazy sequence of rows, where each row's order matches column-names.

The types in values may include: - java.lang.String - java.lang.Double - java.math.BigInteger - java.lang.Boolean - java.time.LocalDate - java.time.LocalDateTime - java.time.OffsetDateTime

Add columns given by column-definitions to a table named table-name. If the table doesn't exist it will throw an error. args is an optional map with an optional key primary-key. The primary-key value is a vector of column names that make up the primary key. Currently only a single primary key is supported.

Alter columns given by column-definitions to a table named table-name. If the table doesn't exist it will throw an error. Currently we do not currently support changing the the primary key, or take any guidance on how to coerce values.

Returns the set of syncable schemas in the database (as strings).

Returns the database type for a given metabase.upload type as a HoneySQL spec. This will be a vector, which allows for additional options. Sample values:

• [:bigint]
• [[:varchar 255]]
• [:generated-always :as :identity]

Returns true if the driver should create an auto-incrementing primary key column when appending CSV data to an existing upload table. This is because we want to add auto-pk columns for drivers that supported uploads before auto-pk columns were introduced by metabase#36249. It should return false if the driver supported the uploads feature in version 48 or later.

Returns the rows of data as arrays needed to populate the table_privileges table with the DB connection's current user privileges. The data contains the privileges that the user has on the given database. The privileges include select, insert, update, and delete.

The rows have the following keys and value types: - role :- [:maybe :string] - schema :- [:maybe :string] - table :- :string - select :- :boolean - update :- :boolean - insert :- :boolean - delete :- :boolean

Either: (1) role is null, corresponding to the privileges of the DB connection's current user (2) role is not null, corresponding to the privileges of the role