Implementation for the | (ns metabase.cmd.endpoint-dox (:require [clojure.java.classpath :as classpath] [clojure.java.io :as io] [clojure.string :as str] [clojure.tools.namespace.find :as ns.find] [metabase.config :as config] [metabase.plugins.classloader :as classloader] [metabase.util :as u])) |
API docs intro | |
Exists just so we can write the intro in Markdown. | (defn- api-docs-intro [] (str (slurp "src/metabase/cmd/resources/api-intro.md") "\n\n")) |
API docs page title | |
Some paid endpoints have different formatting. This way we don't combine the api/table endpoint with sandbox.api.table, for example. | (defn- handle-enterprise-ns
[endpoint]
(if (str/includes? endpoint "metabase-enterprise")
(str/split endpoint #"metabase-enterprise.")
(str/split endpoint #"\."))) |
Used to format initialisms/acronyms in generated docs. | (def initialisms '["SSO" "SAML" "GTAP" "LDAP" "SQL" "JSON" "API" "LLM" "SCIM"]) |
Converts initialisms to upper case. | (defn capitalize-initialisms
[name initialisms]
(let [re (re-pattern (str "(?i)(?:" (str/join "|" (map #(str % "\\b") initialisms)) ")"))]
(str/replace name re u/upper-case-en))) |
Creates a name for endpoints in a namespace, like all the endpoints for Alerts. Handles some edge cases for enterprise endpoints. | (defn- endpoint-ns-name
[endpoint]
(-> (:ns endpoint)
ns-name
name
handle-enterprise-ns
last
u/capitalize-first-char
(str/replace #"(.api.|-)" " ")
(str/replace ".api" "") ; account for `serialization.api` namespace
(capitalize-initialisms initialisms)
(str/replace "SSO SSO" "SSO"))) |
Used for formatting YAML string punctuation for frontmatter descriptions. | (defn- handle-quotes
[s]
(-> s
(str/replace #"\"" "'")
str/split-lines
(#(str/join "\n " %)))) |
Formats description for YAML frontmatter. | (defn- format-frontmatter-description [desc] (str "|\n " (handle-quotes desc))) |
Used to grab namespace description, if it exists. | (defn- get-description
[ep ep-data]
(let [desc (-> ep-data
first
:ns
meta
:doc
u/add-period)]
(if (str/blank? desc)
(u/add-period (str "API endpoints for " ep))
desc))) |
Formats frontmatter, which includes title and summary, if any. | (defn- endpoint-page-frontmatter
[ep ep-data]
(let [desc (format-frontmatter-description (get-description ep ep-data))]
(str "---\ntitle: \"" ep "\""
"\nsummary: " desc "\n---\n\n"))) |
Creates a page title for a set of endpoints, e.g., | (defn- endpoint-page-title [ep-title] (str "# " ep-title "\n\n")) |
API endpoint description | |
If there is a namespace docstring, include the docstring with a paragraph break. | (defn- endpoint-page-description
[ep ep-data]
(let [desc (get-description ep ep-data)]
(if (str/blank? desc)
desc
(str desc "\n\n")))) |
API endpoints | |
Creates a name for an endpoint: VERB /path/to/endpoint. Used to build anchor links in the table of contents. | (defn- endpoint-str
[endpoint]
(-> (:doc endpoint)
(str/split #"\n")
first
str/trim)) |
Decorates endpoints with strings for building API endpoint pages. | (defn- process-endpoint
[endpoint]
(assoc endpoint
:endpoint-str (endpoint-str endpoint)
:ns-name (endpoint-ns-name endpoint))) |
Regular expression to match endpoints. Needs to match namespaces like: - metabase.api.search - metabase-enterprise.serialization.api - metabase.api.api-key | (def api-ns (re-pattern "^metabase(?:-enterprise\\.[\\w-]+)?\\.api(?:\\.[\\w-]+)?$")) |
(defn- api-namespaces []
(for [ns-symb (ns.find/find-namespaces (classpath/system-classpath))
:when (and (re-find api-ns (name ns-symb))
(not (str/includes? (name ns-symb) "test")))]
ns-symb)) | |
Gets a list of all API endpoints. | (defn collect-endpoints
[]
(for [ns-symb (api-namespaces)
[_sym varr] (do (classloader/require ns-symb)
(sort (ns-interns ns-symb)))
:when (:is-endpoint? (meta varr))]
(meta varr))) |
Builds a list of endpoints and their parameters.
Relies on docstring generation in | (defn- endpoint-docs [ep-data] (str/join "\n\n" (map #(str/trim (:doc %)) ep-data))) |
Is the endpoint a paid feature? | (defn- paid?
[ep-data]
(or (str/includes? (:endpoint-str (first ep-data)) "/api/ee")
;; some ee endpoints are inconsistent in naming, see #22687
(str/includes? (:endpoint-str (first ep-data)) "/api/mt")
(= 'metabase-enterprise.sandbox.api.table (ns-name (:ns (first ep-data))))
(str/includes? (:endpoint-str (first ep-data)) "/auth/sso")
(str/includes? (:endpoint-str (first ep-data)) "/api/moderation-review"))) |
Adds a footer with a link back to the API index. | (defn endpoint-footer
[ep-data]
(let [level (if (paid? ep-data) "../../" "../")]
(str "\n\n---\n\n[<< Back to API index](" level "api-documentation.md)"))) |
Build API pages | |
Builds a page with the name, description, table of contents for endpoints in a namespace, followed by the endpoint and their parameter descriptions. | (defn endpoint-page
[ep ep-data]
(apply str
(endpoint-page-frontmatter ep ep-data)
(endpoint-page-title ep)
(endpoint-page-description ep ep-data)
(endpoint-docs ep-data)
(endpoint-footer ep-data))) |
Creates a filepath from an endpoint. | (defn- build-filepath
[dir endpoint-name ext]
(let [file (-> endpoint-name
str/trim
(str/split #"\s+")
(#(str/join "-" %))
u/lower-case-en)]
(str dir file ext))) |
Creates a link to the page for each endpoint. Used to build links
on the API index page at | (defn build-endpoint-link
[ep ep-data]
(let [filepath (build-filepath (if (paid? ep-data) "api/ee/" "api/") ep ".md")]
(str "- [" ep (when (paid? ep-data) "*") "](" filepath ")"))) |
Creates a string that lists links to all endpoint groups, e.g., - Activity. | (defn- build-index [endpoints] (str/join "\n" (map (fn [[ep ep-data]] (build-endpoint-link ep ep-data)) endpoints))) |
Creates a sorted map of API endpoints. Currently includes some endpoints for paid features. | (defn- map-endpoints
[]
(->> (collect-endpoints)
(map process-endpoint)
(group-by :ns-name)
(into (sorted-map-by (fn [a b] (compare
(u/lower-case-en a)
(u/lower-case-en b))))))) |
Page generators | |
Creates an index page that lists links to all endpoint pages. | (defn- generate-index-page!
[endpoint-map]
(let [endpoint-index (str
(api-docs-intro)
(build-index endpoint-map))]
(spit (io/file "docs/api-documentation.md") endpoint-index))) |
Takes a map of endpoint groups and generates markdown pages for all API endpoint groups. | (defn- generate-endpoint-pages!
[endpoints]
(doseq [[ep ep-data] endpoints]
(let [file (build-filepath (str "docs/" (if (paid? ep-data) "api/ee/" "api/")) ep ".md")
contents (endpoint-page ep ep-data)]
(io/make-parents file)
(spit file contents)))) |
Is it a markdown file? | (defn- md?
[file]
(= "md"
(-> file
str
(str/split #"\.")
last))) |
Used to clear the API directory for rebuilding docs from scratch so we don't orphan files as the API changes. | (defn- reset-dir
[file]
(let [files (filter md? (file-seq file))]
(doseq [f files]
(try (io/delete-file f)
(catch Exception e
(println "File:" f "not deleted")
(println e)))))) |
Builds an index page and sub-pages for groups of endpoints.
Index page is | (defn generate-dox!
[]
(when-not config/ee-available?
(println (u/colorize
:red (str "Warning: EE source code not available. EE endpoints will not be included. "
"If you want to include them, run the command with"
\newline
\newline
"clojure -M:ee:doc api-documentation"))))
(let [endpoint-map (map-endpoints)]
(reset-dir (io/file "docs/api"))
(generate-index-page! endpoint-map)
(println "API doc index generated at docs/api-documentation.md.")
(generate-endpoint-pages! endpoint-map)
(println "API endpoint docs generated in docs/api/{endpoint}."))) |