TODO: Consider refacor this namespace by defining custom schema with [[mr/def]] instead. For example the PositiveInt can be defined as (mr/def ::positive-int pos-int?) | (ns metabase.util.malli.schema (:require [clojure.string :as str] [malli.core :as mc] [metabase.legacy-mbql.normalize :as mbql.normalize] [metabase.legacy-mbql.schema :as mbql.s] [metabase.lib.schema.common :as lib.schema.common] [metabase.lib.schema.temporal-bucketing :as lib.schema.temporal-bucketing] [metabase.util :as u] [metabase.util.date-2 :as u.date] [metabase.util.i18n :as i18n :refer [deferred-tru]] [metabase.util.json :as json] [metabase.util.malli :as mu] [metabase.util.malli.registry :as mr] [metabase.util.password :as u.password] [toucan2.core :as t2])) |
(set! *warn-on-reflection* true) | |
-------------------------------------------------- Utils -------------------------------------------------- | |
Helper for creating a schema to check whether something is an instance of (ms/defn my-fn [user :- (ms/InstanceOf User)] ...) TODO -- consider renaming this to | (def ^{:arglists '([model])} InstanceOf
(memoize
(fn [model]
(mu/with-api-error-message
[:fn
{:error/message (format "value must be an instance of %s" (name model))}
#(t2/instance-of? model %)]
(deferred-tru "value must be an instance of {0}" (name model)))))) |
Helper for creating schemas to check whether something is an instance of a given class. | (def ^{:arglists '([^Class klass])} InstanceOfClass
(memoize
(fn [^Class klass]
[:fn
{:error/message (format "Instance of a %s" (.getCanonicalName klass))}
(partial instance? klass)]))) |
Given a schema of a sequence of maps, returns a schema that does an additional unique check on key | (def ^{:arglists '([maps-schema k])} maps-with-unique-key
(memoize
(fn [maps-schema k]
(mu/with-api-error-message
[:and
[:fn (fn [maps]
(= (count maps)
(-> (map #(get % k) maps)
distinct
count)))]
maps-schema]
(deferred-tru "value must be seq of maps in which {0}s are unique" (name k)))))) |
Returns an enum schema that accept both keywords and strings. (enum-keywords-and-strings :foo :bar) ;; => [:enum :foo :bar "foo" "bar"] | (defn enum-keywords-and-strings [& keywords] (assert (every? keyword? keywords)) (vec (concat [:enum] keywords (map u/qualified-name keywords)))) |
Returns an enum schema that decodes strings to keywords. (enum-decode-keyword :foo :bar) ;; => [:enum {:decode/json keyword} :foo :bar] | (defn enum-decode-keyword
[keywords]
(into [:enum {:decode/json keyword}] keywords)) |
-------------------------------------------------- Schemas -------------------------------------------------- | |
Schema for a string that cannot be blank. | (def NonBlankString (mu/with-api-error-message ::lib.schema.common/non-blank-string (deferred-tru "value must be a non-blank string."))) |
Schema representing an integer than must also be greater than or equal to zero. | (def IntGreaterThanOrEqualToZero
(let [message (deferred-tru "value must be an integer greater or equal to than zero.")]
[:int
{:min 0
:description (str message)
:error/fn (fn [_ _]
(str message))
:api/regex #"\d+"}])) |
Schema representing an integer. | (def Int
(let [message (deferred-tru "value must be an integer.")]
[:int
{:description (str message)
:error/fn (fn [_ _]
(str message))
:api/regex #"-?\d+"}])) |
Schema representing an integer than must also be greater than zero. | (def PositiveInt
(let [message (deferred-tru "value must be an integer greater than zero.")]
[:int
{:min 1
:description (str message)
:error/fn (fn [_ _]
(str message))
:api/regex #"[1-9]\d*"}])) |
Schema representing an integer than must be less than zero. | (def NegativeInt
(let [message (deferred-tru "value must be a negative integer")]
[:int
{:max -1
:description (str message)
:error/fn (fn [_ _]
(str message))
:api/regex #"-[1-9]\d*"}])) |
Schema representing a numeric value greater than zero. This allows floating point numbers and integers. | (def PositiveNum (mu/with-api-error-message [:and number? pos?] (deferred-tru "value must be a number greater than zero."))) |
Schema for something that can be either a | (def KeywordOrString (mu/with-api-error-message [:or :string :keyword] (deferred-tru "value must be a keyword or string."))) |
Schema for a valid Field base or effective (data) type (does it derive from | (def FieldType (mu/with-api-error-message [:fn #(isa? % :type/*)] (deferred-tru "value must be a valid field type."))) |
Schema for a valid Field semantic type deriving from | (def FieldSemanticType (mu/with-api-error-message [:fn #(isa? % :Semantic/*)] (deferred-tru "value must be a valid field semantic type."))) |
Schema for a valid Field relation type deriving from | (def FieldRelationType (mu/with-api-error-message [:fn #(isa? % :Relation/*)] (deferred-tru "value must be a valid field relation type."))) |
Schema for a valid Field semantic or Relation type. This is currently needed because the | (def FieldSemanticOrRelationType (mu/with-api-error-message [:fn (fn [k] (or (isa? k :Semantic/*) (isa? k :Relation/*)))] (deferred-tru "value must be a valid field semantic or relation type."))) |
Schema for a valid Field coercion strategy (does it derive from | (def CoercionStrategy (mu/with-api-error-message [:fn #(isa? % :Coercion/*)] (deferred-tru "value must be a valid coercion strategy."))) |
Like | (def FieldTypeKeywordOrString (mu/with-api-error-message [:fn #(isa? (keyword %) :type/*)] (deferred-tru "value must be a valid field data type (keyword or string)."))) |
Like | (def FieldSemanticTypeKeywordOrString (mu/with-api-error-message [:fn #(isa? (keyword %) :Semantic/*)] (deferred-tru "value must be a valid field semantic type (keyword or string)."))) |
Like | (def FieldRelationTypeKeywordOrString (mu/with-api-error-message [:fn #(isa? (keyword %) :Relation/*)] (deferred-tru "value must be a valid field relation type (keyword or string)."))) |
Like | (def FieldSemanticOrRelationTypeKeywordOrString
(mu/with-api-error-message
[:fn (fn [k]
(let [k (keyword k)]
(or (isa? k :Semantic/*)
(isa? k :Relation/*))))]
(deferred-tru "value must be a valid field semantic or relation type (keyword or string)."))) |
Schema for a valid legacy | (def LegacyFieldOrExpressionReference
(mu/with-api-error-message
[:fn (fn [k]
((comp (mr/validator mbql.s/Field)
mbql.normalize/normalize-tokens) k))]
(deferred-tru "value must an array with :field id-or-name and an options map"))) |
Like | (def CoercionStrategyKeywordOrString (mu/with-api-error-message [:fn #(isa? (keyword %) :Coercion/*)] (deferred-tru "value must be a valid coercion strategy (keyword or string)."))) |
Validates entity type derivatives of | (def EntityTypeKeywordOrString (mu/with-api-error-message [:fn #(isa? (keyword %) :entity/*)] (deferred-tru "value must be a valid entity type (keyword or string)."))) |
Schema for a valid map. | (def Map (mu/with-api-error-message :map (deferred-tru "Value must be a map."))) |
Schema for a valid email string. | (def Email
(mu/with-api-error-message
[:and
:string
[:fn {:error/message "valid email address"} u/email?]]
(deferred-tru "value must be a valid email address."))) |
Schema for a valid URL string. | (def Url (mu/with-api-error-message [:fn u/url?] (deferred-tru "value must be a valid URL."))) |
Schema for a valid password of sufficient complexity which is not found on a common password list. | (def ValidPassword
(mu/with-api-error-message
[:and
:string
[:fn {:error/message "valid password that is not too common"} (every-pred string? #'u.password/is-valid?)]]
(deferred-tru "password is too common."))) |
Schema for a string that can be parsed as an integer.
Something that adheres to this schema is guaranteed to to work with | (def IntString
(mu/with-api-error-message
[:and
:string
[:fn #(u/ignore-exceptions (Integer/parseInt %))]]
(deferred-tru "value must be a valid integer."))) |
Schema for a string that can be parsed as an integer, and is greater than zero.
Something that adheres to this schema is guaranteed to to work with | (def IntStringGreaterThanZero
(mu/with-api-error-message
[:and
:string
[:fn #(u/ignore-exceptions (< 0 (Integer/parseInt %)))]]
(deferred-tru "value must be a valid integer greater than zero."))) |
Schema for a string that can be parsed as an integer, and is greater than or equal to zero.
Something that adheres to this schema is guaranteed to to work with | (def IntStringGreaterThanOrEqualToZero
(mu/with-api-error-message
[:and
:string
[:fn #(u/ignore-exceptions (<= 0 (Integer/parseInt %)))]]
(deferred-tru "value must be a valid integer greater than or equal to zero."))) |
Schema for a string that can be parsed by date2/parse. | (def TemporalString
(mu/with-api-error-message
[:and
:string
[:fn #(u/ignore-exceptions (boolean (u.date/parse %)))]]
(deferred-tru "value must be a valid date string"))) |
Schema for a string that is valid serialized JSON. | (def JSONString
(mu/with-api-error-message
[:and
:string
[:fn #(try
(json/decode %)
true
(catch Throwable _
false))]]
(deferred-tru "value must be a valid JSON string."))) |
(def ^:private keyword-or-non-blank-str-malli
(mc/schema
[:or {:json-schema {:type "string" :minLength 1}} :keyword NonBlankString])) | |
Schema for a valid representation of a boolean
(one of | (def BooleanValue
(-> [:enum {:decode/json (fn [b] (contains? #{"true" true} b))
:json-schema {:type "boolean"}}
"true" "false" true false]
(mu/with-api-error-message
(deferred-tru "value must be a valid boolean string (''true'' or ''false'').")))) |
Same as above, but allows distinguishing between | (def MaybeBooleanValue
(-> [:enum {:decode/json (fn [b] (some->> b (contains? #{"true" true})))
:json-schema {:type "boolean" :optional true}}
"true" "false" true false nil]
(mu/with-api-error-message
(deferred-tru "value must be a valid boolean string (''true'' or ''false'').")))) |
Schema for valid source_options within a Parameter | (def ValuesSourceConfig
;; TODO: This should be tighter
(mc/schema
[:map
[:values {:optional true} [:* :any]]
[:card_id {:optional true} PositiveInt]
[:value_field {:optional true} LegacyFieldOrExpressionReference]
[:label_field {:optional true} LegacyFieldOrExpressionReference]])) |
Has two components:
1. | (def RemappedFieldValue [:tuple :any :string]) |
Has one component: | (def NonRemappedFieldValue [:tuple :any]) |
Schema for a valid list of values for a field, in contexts where the field can have a remapped field. | (def FieldValuesList [:sequential [:or RemappedFieldValue NonRemappedFieldValue]]) |
Schema for a value result of fetching the values for a field, in contexts where the field can have a remapped field. | (def FieldValuesResult [:map [:has_more_values :boolean] [:values FieldValuesList]]) |
Schema for a valid Parameter. We're not using [[metabase.legacy-mbql.schema/Parameter]] here because this Parameter is meant to be used for Parameters we store on dashboard/card, and it has some difference with Parameter in MBQL. | #_(def ParameterSource
(mc/schema
[:multi {:dispatch :values_source_type}
["card" [:map
[:values_source_type :string]
[:values_source_config
[:map {:closed true}
[:card_id {:optional true} IntGreaterThanZero]
[:value_field {:optional true} Field]
[:label_field {:optional true} Field]]]]]
["static-list" [:map
[:values_source_type :string]
[:values_source_config
[:map {:closed true}
[:values {:optional true} [:* :any]]]]]]]))
(def Parameter
;; TODO we could use :multi to dispatch values_source_type to the correct values_source_config
(mu/with-api-error-message
[:map
[:id NonBlankString]
[:type keyword-or-non-blank-str-malli]
;; TODO how to merge this with ParameterSource above?
[:values_source_type {:optional true} [:enum "static-list" "card" nil]]
[:values_source_config {:optional true} ValuesSourceConfig]
[:slug {:optional true} :string]
[:name {:optional true} :string]
[:default {:optional true} :any]
[:sectionId {:optional true} NonBlankString]
[:temporal_units {:optional true} [:sequential ::lib.schema.temporal-bucketing/unit]]]
(deferred-tru "parameter must be a map with :id and :type keys"))) |
Schema for a valid Parameter Mapping | (def ParameterMapping
(mu/with-api-error-message
[:map [:parameter_id NonBlankString]
[:target :any]
[:card_id {:optional true} PositiveInt]]
(deferred-tru "parameter_mapping must be a map with :parameter_id and :target keys"))) |
Schema for a valid map of embedding params. | (def EmbeddingParams
(mu/with-api-error-message
[:maybe [:map-of
:keyword
[:enum "disabled" "enabled" "locked"]]]
(deferred-tru "value must be a valid embedding params map."))) |
Schema for a valid ISO Locale code e.g. | (def ValidLocale
(mu/with-api-error-message
[:and
NonBlankString
[:fn
{:error/message "valid locale"}
i18n/available-locale?]]
(deferred-tru "String must be a valid two-letter ISO language or language-country code e.g. ''en'' or ''en_US''."))) |
Schema for a 21-character NanoID string, like "FReCLx5hSWTBU7kjCWfuu". | (def NanoIdString
(mu/with-api-error-message
[:re #"^[A-Za-z0-9_\-]{21}$"]
(deferred-tru "String must be a valid 21-character NanoID string."))) |
Schema for a UUID string | (def UUIDString (mu/with-api-error-message [:re u/uuid-regex] (deferred-tru "value must be a valid UUID."))) |
Helper for creating schemas to check whether something is an instance of a collection. | (defn CollectionOf
[item-schema]
[:fn
{:error/message (format "Collection of %s" item-schema)}
#(and (coll? %) (every? (partial mr/validate item-schema) %))]) |
Helper for creating a schema that coerces single-value to a vector. Useful for coercing query parameters. | (defn QueryVectorOf
[schema]
[:vector {:decode/string (fn [x] (cond (vector? x) x x [x]))} schema]) |
Helper for creating a schema to check if a map doesn't contain kebab case keys. | (defn MapWithNoKebabKeys
[]
[:fn
{:error/message "Map should not contain any kebab-case keys"}
(fn [m]
;; reduce-kv is more efficient that iterating over (keys m). But we have to extract the underlying map from
;; Toucan2 Instance because it doesn't implement IKVReduce (yet).
(let [m (if (instance? toucan2.instance.Instance m)
(.m ^toucan2.instance.Instance m)
m)]
(reduce-kv (fn [_ k _]
(if (str/includes? k "-")
(reduced false)
true))
true m)))]) |
Schema for a file coming in HTTP request from multipart/form-data | (def File
[:map {:closed true}
[:content-type string?]
[:filename string?]
[:size int?]
[:tempfile (InstanceOfClass java.io.File)]]) |