Deploy to production

• Mount a persistent volume that your application can write to. The steps use /data.
• Keep the node name the same across restarts. The Mnesia schema belongs to the node that created it. If the node name changes, the library logs node mismatch and refuses to start.
• Build the release with the MIX_ENV you deploy, normally prod. The library decides at compile time which of its own processes to start. For MIX_ENV=test it starts only the PubSub, and none of its store and handler processes.
• The library takes over the node's Mnesia when it boots. If your application has Mnesia tables of its own, read "Share Mnesia with your own tables" below before you deploy.

Set the paths in config/runtime.exs, so a release reads them at boot:

config :mishka_installer,
  project_path: "/data",
  extensions_path: "/data/extensions"

config :mishka_installer, MishkaInstaller.MnesiaRepo,
  mnesia_dir: "/data/mnesia",
  essential: [MishkaInstaller.Event.Event, MishkaInstaller.Installer.Installer]

• :extensions_path is where installed packages are written.
• mnesia_dir is where the records live.
• :essential lists the tables created on boot, the plugin store and the install store. Keep both.
• :project_path is the directory the installer returns to after each install.

The tables use disc copies outside the test environment, so the records reach the volume.

The library starts its own supervision tree when your application starts: the install handler, the event handler and the Mnesia store. Your plugins register themselves, so list each plugin module in a supervisor of your application:

defmodule MishkaProdTest.Application do
  use Application

  @impl true
  def start(_type, _args) do
    Supervisor.start_link([MishkaProdTest.DurablePlugin],
      strategy: :one_for_one,
      name: MishkaProdTest.Supervisor
    )
  end
end

This is the host application of the library's own release test. On a boot, the event handler starts each stored plugin again. A plugin an operator stopped stays stopped, and a plugin whose dependencies are not started yet is held.

Mnesia replicates the plugin and install tables across a cluster. Form the Erlang cluster first, with libcluster or Node.connect/1. Then choose one seed node and point the others at it:

# On the seed node, the default
config :mishka_installer, MishkaInstaller.MnesiaRepo, cluster_nodes: :auto

# On every other node
config :mishka_installer, MishkaInstaller.MnesiaRepo, cluster_nodes: [:"seed@host-a"]

A node with a list of peers wipes its local schema, joins them, becomes a disc copy member and copies the essential tables. If no peer is reachable yet, it boots empty with the status :pending and joins when a configured peer connects. Each node builds its own event modules from the replicated table, and a rebuild on one node is announced so the others rebuild too.

When a peer node boots, or the Mnesia store of a node announces that it is ready again, its :synchronized message reaches every node of the cluster. Each event handler then starts all the events again, and [:mishka_installer, :event, :synchronized] fires again. The install handler replays every stored library once more. Each library that is already running logs [mishka_installer.installer] skipped re-loading ... on boot as an error and emits [:mishka_installer, :installer, :replay] with result: :error, while the libraries keep running, so these lines need no action. The documentation on PubSub messages and readiness lists when the ready signal repeats, and the Telemetry and log reference lists the events and the log lines.

An install belongs to one node

Mnesia replicates tables only. Files stay on the node that wrote them. Installer.install/1 writes the package under the :extensions_path of the node that runs it, puts its ebin on that node's code path and starts the application there. uninstall/1 is local in the same way: it stops the application and deletes the files on its own node, and it deletes the shared record. The other nodes get the install record and the plugin records, and nothing else.

On every boot a node replays all install records. A node that has the record but not the files logs skipped re-loading followed by the app name, and the application stays unloaded there. A plugin of that library stays registered on that node and cannot be called. The library does not unregister it, because its application is not loaded. The event is built without that plugin, and Hook.call/3 answers its cannot_run option, or the data it was given.

To load the library on another node, run the install on that node too. There is one record for each app, so a second install updates it, and the prepend_paths written by the last install are what every node reads on its next boot. Give every node the same :extensions_path. The documentation on plugins in installed libraries covers upgrades and removals.

A node runs one Mnesia, and the library takes it over when its store starts. The :mnesia application is already running by then, because the library lists it as an extra application. The store compares what runs with what you configured:

• If Mnesia runs on :mnesia_dir and every module in :essential already has a local table, the store only announces that it is ready.
• Otherwise it stops Mnesia, sets the :dir of the :mnesia application to :mnesia_dir, creates the schema when that folder has none, starts Mnesia and creates the missing essential tables.

The log shows the OTP line Application mnesia exited: :stopped each time the store stops Mnesia.

Keep your tables in the same folder

Set :mnesia_dir to the folder your own tables already use. When the node boots, the store restarts Mnesia on that folder, and tables with disc copies load from it as before. A table with ram_copies is empty after that restart. If :mnesia_dir is another folder, Mnesia moves there and your tables are not loaded. Their files stay in the old folder.

Let the store create your tables

A module in :essential that defines database_config/0 gets its table created on boot, with the options of :mnesia.create_table/2. The table is named after the module, and its records are tuples that start with the module name. A node that joins a cluster copies each essential table from the seed.

defmodule MyApp.Settings.Table do
  def database_config,
    do: [attributes: [:key, :value], disc_copies: [node()]]
end

config :mishka_installer, MishkaInstaller.MnesiaRepo,
  mnesia_dir: "/data/mnesia",
  essential: [
    MishkaInstaller.Event.Event,
    MishkaInstaller.Installer.Installer,
    MyApp.Settings.Table
  ]

Wait for the store

Your application starts after the library, and the store finishes its boot while the library starts. Mnesia is therefore ready when your own start/2 runs, except on a joiner that boots :pending. The [:mishka_installer, :mnesia, :synchronized] telemetry event marks the moment the tables are ready. Code that starts later can read it as a value with MishkaInstaller.MnesiaRepo.state/0: the status is :synchronized, :pending on a joiner that has not found a seed, or :inconsistent after a split brain. The configuration documentation lists the fields.

Restart the node and confirm that the state came back:

MishkaInstaller.Installer.Installer.get()     # installed libraries, with their prepend_paths
MishkaInstaller.Event.Event.get()             # every plugin with its stored status
MishkaInstaller.Event.Hook.health()           # a report for every event

The log shows [mishka_installer.event] plugins synchronized once the event handler has started the stored plugins. It shows [mishka_installer.installer] skipped re-loading followed by the app name when a stored library could not be loaded again. Both lines come back when a peer node boots or the store announces itself again, as the section on running more than one node explains. The Telemetry and log reference lists the messages.

The library repository proves this path with a test that builds a real release, installs a package, disables a plugin, ends the VM without a clean shutdown, and starts it again. Run it with mix test --only production_release in a checkout of the library.

• Configuration reference: every key and its default.
• How the runtime installer works: what is replayed on boot.
• Telemetry and log reference: watch boots and cluster changes.
• PubSub messages and readiness: the messages of a boot and when the ready signal repeats.
• Ship and remove plugins with a runtime-installed library: upgrades, uninstalls and clusters.