Install a library at runtime

• Set where the library keeps files. :project_path is the project root and :extensions_path is the folder that holds installed packages. Without :extensions_path the folder is deployment/<env>/extensions under the project path. The PROJECT_PATH environment variable, when set, wins over :project_path. Set one of them even when you set :extensions_path: install/1 ends by changing the working directory back to the project path, and without one it raises at the end, after the application has started and its record is written.
• Build the library with the same Erlang/OTP and Elixir versions as the node that loads it.
• Make sure the library's own dependencies are already available on the node. The installer puts only the new application's ebin on the code path, and the application fails to start if a dependency is missing.
• Call the install functions from trusted code. A .beam file runs with full node privileges, and the BEAM has no sandbox.
• Upgrading from an older release? The changes documentation for 0.1.6 covers the move to pre-built packages and the allowlist for remote installs. The changes documentation for 0.1.14 covers semantic versions and the single record for each application.

config :mishka_installer,
  project_path: "/data",
  extensions_path: "/data/extensions"

A package is a folder with an ebin directory. ebin holds the .beam files and the <app>.app file. Mix writes both to _build/<env>/lib/<app>/ebin. To publish it for a URL or a release, pack the folder as a tar.gz with ebin at the top level, or one directory below it:

tar -czf demo-0.1.0-ebin.tar.gz -C _build/prod/lib/demo ebin

Put the package folder inside the extensions directory, then call Installer.install/1:

alias MishkaInstaller.Installer.Installer

Installer.install(%{
  app: "demo",
  version: "0.1.0",
  path: "/data/extensions/demo-0.1.0"
})

The default type is :path. The installer refuses a path outside the extensions directory. If the folder is not already named <app>-<version>, it is moved there, and a folder that already has that name is deleted first. The app name starts with a lowercase letter and holds only lowercase letters, digits and underscores.

The version must be a semantic version: MAJOR.MINOR.PATCH, with an optional pre-release and build, such as 0.1.0, 1.0.0-rc.1 or 1.0.0+build.5. The installer refuses anything else, such as 1.0, before it installs anything. It also compares the version with the one already running, and refuses one that is not higher.

Installer.install(%{app: "demo", version: "1.0", path: "/data/extensions/demo-1.0"})
# {:error, [%{action: :semver, field: :version, message: "Invalid semantic version format in the version field (expected MAJOR.MINOR.PATCH)"}]}

On success the call returns {:ok, %{extension: record, dir: dir}}. The application is loaded and started, and the record is stored in the registry.

Installing a higher version of an application that is already installed upgrades it. The record of the application is updated, so the registry holds one record for each application. The new application is started, but a module that is already loaded keeps its old code until the node restarts, and the folder of the earlier version stays on disk:

Installer.install(%{app: "demo", version: "0.2.0", path: "/data/extensions/demo-0.2.0"})

Enum.count(Installer.get(), &(&1.app == "demo"))
# 1

Remote installs are closed until you allow their source. List the hosts in the allowlist, then pass the artifact address as path:

config :mishka_installer, :allowlist, url_hosts: ["cdn.example.com"]

Installer.install(%{
  app: "demo",
  version: "0.1.0",
  type: :url,
  path: "https://cdn.example.com/demo-0.1.0-ebin.tar.gz"
})

The installer compares the host of the address with the list, exactly. An empty or missing list blocks every remote install.

Allow the repository, then use :github_tag with a tag, or :github_latest_release:

config :mishka_installer, :allowlist, github_repos: ["owner/demo"]

Installer.install(%{app: "demo", version: "0.1.0", type: :github_tag, path: "owner/demo", tag: "0.1.0"})
Installer.install(%{app: "demo", version: "0.1.0", type: :github_latest_release, path: "owner/demo"})

The installer reads the release through the GitHub API and downloads the first asset. To pick another asset, pass its file name as asset:. The allowlist checks owner/repo. The address of the asset comes from the GitHub response, and it is not checked against url_hosts.

The API call carries no token. GitHub counts it against the limit for unauthenticated requests, which is 60 an hour for each IP address (see GitHub's rate limit documentation). Nodes and CI runners behind one address share that limit. A request over the limit gets a 403 or 429, and an unauthenticated request cannot read a private repository. Any status other than 200 ends the install with a :download error.

Three things help:

• Skip the API. Install with type: :url and the address of the asset, such as https://github.com/owner/demo/releases/download/0.1.0/demo-0.1.0-ebin.tar.gz, and list github.com in url_hosts. A :url install makes no API call.
• Send a token. The :downloader_req_options setting is a list of Req options that is merged into every download request. auth: sets the authorization header, so the API call is authenticated. GitHub's limit for a personal access token is 5,000 an hour. Read the token when the node starts, in config/runtime.exs.
• Use a proxy. The :proxy setting is a second list of Req options, merged after the first one. Req does not read HTTP_PROXY or HTTPS_PROXY, so the proxy goes here.

config :mishka_installer, :downloader_req_options,
  auth: {:bearer, System.fetch_env!("GITHUB_TOKEN")}

config :mishka_installer, :proxy,
  connect_options: [proxy: {:http, "proxy.example.com", 3128, []}]

The proxy value is Req's {scheme, address, port, options} tuple. Both settings are merged as whole lists, so a key that both set takes its value from :proxy.

Pass checksum:, the SHA-256 of the downloaded archive as lowercase hex. The installer checks it before extracting and installs nothing on a mismatch. Local installs have no download to check. Without checksum:, nothing checks the downloaded archive.

:crypto.hash(:sha256, File.read!("demo-0.1.0-ebin.tar.gz")) |> Base.encode16(case: :lower)

The installer refuses to install over or uninstall an application on the protected_apps list. The default list is ["mishka_installer"]. Setting the key replaces the default, so keep "mishka_installer" in the list you set:

config :mishka_installer, :allowlist, protected_apps: ["mishka_installer", "my_app"]

Installer.async_install/1 takes the same map and queues the install behind others. It returns :ok, or the validation error, at once. The result is not returned. Watch the [:mishka_installer, :installer, :install] telemetry event, which carries result: :ok or result: :error, and the log.

Only the validation of the fields, the version included, runs before the call returns. The name, allowlist and download checks run later, so a blocked source still returns :ok and fails in the log and in the event, with result: :error and app: nil. A connection that cannot be made raises instead of returning an error. In a background install that stops the install handler. Its supervisor starts it again with an empty queue, so installs queued behind the failed one are dropped and no event is emitted.

Installer.uninstall(%{app: "demo", version: "0.1.0"})
# :ok

The installer stops and unloads the application, removes its ebin from the code path, deletes the demo-0.1.0 folder and deletes the record. After an upgrade there is still one record, so an uninstall leaves none. It does not remove the application's dependencies or the folders of other versions. It returns :ok also when the application is not running or has no record. It does not purge the application's modules, which stay loaded until the node restarts, and it does not touch the plugin registry.

A library that registers plugins needs more than this call. Read the documentation on plugins in installed libraries before you uninstall or upgrade it.

A failed install is undone only in part. When the code path, unload or start step fails, the installer stops and unloads the application and takes its ebin off the code path. Two things are not undone:

• The package folder. A local folder has already been moved to <app>-<version>, or an archive has been extracted there, and a folder that had that name was deleted first. This happens before the version is compared with the running one, so an install refused as :compare_version_with_installed_app has already replaced the folder of that name.
• The record. When the record cannot be stored, install/1 returns a :database error, and the application keeps running on the code path with no record. The next restart does not load it. Call Installer.uninstall/1, which also deletes the folder, put the package back and install again.

A failed install returns {:error, [%{action: action, field: field, message: message}]}. The action says which check failed:

A connection that cannot be made, such as a refused one, does not return an error tuple. The call raises a Req.TransportError after Req has retried it three times. To fail at once, set retry: false in :downloader_req_options.

Compute the checksum of the archive you attached to the release, then pass it with the install:

config :mishka_installer, :allowlist, github_repos: ["owner/demo"]

checksum =
  :crypto.hash(:sha256, File.read!("demo-0.1.0-ebin.tar.gz")) |> Base.encode16(case: :lower)

Installer.install(%{
  app: "demo",
  version: "0.1.0",
  type: :github_tag,
  path: "owner/demo",
  tag: "0.1.0",
  asset: "demo-0.1.0-ebin.tar.gz",
  checksum: checksum
})

After the node restarts, the application is loaded again from the stored record and the folder on disk.

• How the runtime installer works: the order of an install and what happens at boot.
• Installer reference: the record fields and every function.
• Ship and remove plugins with a runtime-installed library: what an upgrade or an uninstall leaves behind.
• Deploy to production: keep installs across restarts.
• Changes and upgrade notes: what each release changed in the installer.