Installing MISO on baremetal

This installation guide is intended to be used if you cannot use Docker and Docker compose, and is not trivial to set up. We recommend using Docker compose if possible by following the Docker compose installation guide.

Refer here in the event that you run into issues during installation.

Prerequisites

For each service, which may be put on the same machine, the following tools are required:

Application Server:

  • JDK 17
  • Tomcat 10

Database Server:

  • MySQL 9.7
  • Docker

Important Note: MISO requires SSL (HTTPS) to function correctly over a network. It is usually ideal to configure SSL using a proxy server. This guide will not cover SSL configuration, but you can see an example configuration for NGINX here. SSL is not required to access MISO over a localhost connection (development purposes)

Downloading the latest release

Use the GitHub interface to download the following files for the latest release:

  • ROOT.war
  • sqlstore.jar
  • setup_files.tar.gz (for new installations only)

Setting Up the Database Server

The database server needs to have MySQL 9.7. Docker must also be present to migrate the database as the application is developed, but it can be installed on a different server so long as it can access the database server.

It is best to set a default timezone for MySQL. You can configure this in my.cnf. The simplest and recommended option is to set it to UTC by adding the following lines:

[mysqld]
default-time-zone='+00:00'

You could use a named timezone instead if you've populated the timezone tables. See the MySQL docs for more information.

The default password in the following IDENTIFIED BY clauses should be changed.

Once installed, start the MySQL console and create the database:

CREATE DATABASE lims;
USE lims;

Then add a user that has all grant access on the 'lims' db:

CREATE USER 'tgaclims'@'localhost' IDENTIFIED BY 'tgaclims';
GRANT ALL ON `lims`.* TO 'tgaclims'@'localhost';

If your database and Tomcat install are on different machines, then you will need to add a grant privilege to the MISO database from your remote machine:

GRANT ALL ON `lims`.* TO 'tgaclims'@'your.tomcat.install.server';
GRANT ALL ON `lims`.* TO 'tgaclims'@'your.tomcat.install.server' IDENTIFIED BY 'tgaclims';

Setting Up the Application Server

The application server needs Tomcat 10.

Create a file called ROOT.xml in the following directory $CATALINA_HOME/conf/Catalina/localhost, creating the directory if necessary, and populate it with the following information:

<Context path="/ROOT" docBase="${catalina.home}/webapps/ROOT" reloadable="false">
  <Resources allowLinking="true"/>
  <Resource name="jdbc/MISODB" type="javax.sql.DataSource"
  driverClassName="com.mysql.cj.jdbc.Driver"
  initialSize="32"
  maxIdle="10"
  maxActive="100"
  maxWait="1000"
  removeAbandoned="true"
  removeAbandonedTimeout="120"
  logAbandoned="true"
  testWhileIdle="true"
  testOnBorrow="true"
  testOnReturn="true"
  validationQuery="select 1"
  url="jdbc:mysql://localhost:3306/lims?autoReconnect=true&amp;characterEncoding=UTF-8&amp;allowPublicKeyRetrieval=true&amp;sslMode=DISABLED&amp;connectionTimeZone=SERVER&amp;cacheDefaultTimeZone=false"
  username="tgaclims"
  password="tgaclims"/>
  <Parameter name="miso.propertiesFile" value="file:${catalina.home}/conf/Catalina/localhost/miso.properties" override="false"/>
</Context>

Make sure the database path in ROOT.xml is correct for your install:

url="jdbc:mysql://your.database.server:3306/dbname"

Note that the connection parameters suggested above disable SSL between the web and database server, meaning that the communications will not be encrypted. You should consider the security implications of this -- especially if the connection is over a public network -- and configure SSL properly if required.

Extract the setup_files.tar.gz file and copy miso.properties to $CATALINA_HOME/conf/Catalina/localhost/miso.properties. This file contains site-specific configuration that you should review and modify as appropriate. See Site Configuration for more information.

Append the following line to $CATALINA_HOME/bin/setenv.sh or, if you installed Tomcat through apt, /etc/default/tomcat10:

JAVA_OPTS="$JAVA_OPTS -Dsecurity.method=jdbc -Xmx768M"

(Update the security method if you are using LDAP or Active Directory LDAP.)

Create the directory /storage/miso and subdirectories /storage/miso/log and /storage/miso/files. Note that these directories can be changed in miso.properties, so if you changed them, make sure to create the directories you specified instead.

mkdir -p /storage/miso/log
mkdir -p /storage/miso/files

Ensure that the user that Tomcat runs as has write permission to the storage directory. For example, if the user is in the 'tomcat' group:

chgrp -R tomcat /storage/miso/

Copy the following configuration files from setup_files.tar.gz into the /storage/miso/ directory:

File Purpose
security.properties properties to set the security environment (see below).
submission.properties properties to set the submission environment.

Security Environment (updating /storage/miso/security.properties)

MISO can use LDAP (ldap), Active Directory LDAP (ad), JDBC (jdbc), or SAML single sign-on (saml) as an authentication mechanism. This is set by the -Dsecurity.method noted in the previous section, and the same value must be set in security.properties.

If you are using JDBC (i.e. storing usernames and passwords in the MISO database), set the security method to jdbc. No additional configuration is necessary.

For using LDAP, set the security method to ldap. Additional settings are needed for LDAP in the security.properties. Talk to your LDAP administrator.

To use Active Directory, a specific kind of LDAP, set the security method to ad. Some active directory settings are needed in addition to the LDAP settings in the security.properties file.

The search for a user is done against userPrincipalName which takes the form of an email address. To log in, the user will type their username and to do the lookup their username will be added to the domain specified in the property security.ad.emailDomain.

Use the security.ad.url property to indicate the url for the Active Directory. Some valid examples are: ad.oicr.on.ca, ldap://ad.oicr.on.ca:389 and ldaps://ad.oicr.on.ca:636.

The groups used by MISO are ROLE_INTERNAL for regular users and ROLE_ADMIN for administrators. If you find these names too general you may wish to add a prefix before adding these groups to your Active Directory. For example MISO_ROLE_INTERNAL gives a clearer indication as to what the group is used for. In this case you will need to set the property security.ldap.stripRolePrefix to the value MISO_ to allow MISO to ignore the prefix.

For using SAML single sign-on, set the security method to saml. Copy the example security.properties file and update the SAML settings there for your environment.

The identity provider metadata URL should point to the SAML metadata published by your identity provider. The SP private key and certificate are used by MISO to sign SAML messages. The attribute properties tell MISO which SAML assertion attributes to use for the user's login name, full name, email, and roles.

MISO requires one role for regular users and optionally one for administrators. These are configured using security.saml.internalRoleName and security.saml.adminRoleName. The SAML assertion must contain those exact role names in the attribute configured by security.saml.rolesAttribute.

When MISO is running, you can access the MISO service provider metadata at:

https://<miso-base-url>/saml2/service-provider-metadata/<registrationId>

Use that metadata when configuring the SAML client in your identity provider so that the assertion consumer and single logout service URLs match the running MISO instance.

If using JDBC, once running, you should change the passwords of the admin and notification accounts.

Setting Up the Run Scanner

Run Scanner is a webservice that scans the paths containing sequencer output. It is not required for a functioning MISO install, but without it, sequencer runs must be added manually.

Please see the Run Scanner Installation and Setup Guide for setup instructions.

Once complete, edit $CATALINA_HOME/conf/Catalina/localhost/miso.properties of the MISO Tomcat server and set miso.runscanner.urls to the URL of the Run Scanner instance. It is possible to set up multiple run scanners managing different sequencers and add all the URLs to miso.properties. If you are adding Run Scanner to a previously-established MISO environment, restart MISO.

Installing and Upgrading

Prior to the installation, ensure that you have followed the instructions in the above and have WAR files for both MISO (ROOT.war) and, if desired, Run Scanner(runscanner-*.war). Docker must also be available on either the database server, or another machine that can access the database.

To install or upgrade, perform the following steps:

  1. If performing an upgrade, make sure to read all release notes since the previous version you were running. Some releases indicate additional upgrade instructions, which may include
    • Configuration changes (in miso.properties or elsewhere)
    • Database preparation or cleanup steps
    • Other necessary steps or warnings
  2. Backup your existing database.
  3. Stop MISO's Tomcat.
  4. Remove $CATALINA_HOME/webapps/ROOT directory and $CATALINA_HOME/webapps/ROOT.war file if they exist.
  5. Copy the new ROOT.war to $CATALINA_HOME/webapps.
  6. Migrate the database to the newest version. (Described below.)
  7. If also releasing Run Scanner, deploy it:
    1. Stop Run Scanner's Tomcat.
    2. Deploy the Run Scanner WAR.
    3. Restart Run Scanner's Tomcat.
  8. Restart MISO's Tomcat.

Migrating the database

Flyway is used to apply patches to your database to make it compatible with the new MISO version. A Docker image is provided to simplify setup and running the migrations.

A MySQL user with several privileges must be used to run Flyway and create and restore backups. GRANT ALL PRIVILEGES ON *.* TO 'username'@'host'; will grant all the necessary privileges. Only a subset of these is required, but we have not investigated specifics. Using a host of '%' for this user is simplest since a different Docker container will be used each time. The user specified in ROOT.xml, which the web server will use to communicate with the database, should NOT be granted global (*.*) privileges, so this should be a different user.

  1. If you have not already done so, create an environment file defining the parameters required for Flyway to connect to your database: ``` # Database server hostname and port separated by a colon # use "host.docker.internal" to refer to the host if you're running this on the database server # 3306 is the default port for MySQL MISO_DB_HOST_PORT=host.docker.internal:3306

    Database name

    MISO_DB=lims

    Same path set for miso.fileStorageDirectory in miso.properties

    MISO_FILES_DIR=/storage/miso/files

    MySQL user that Flyway will use

    FLYWAY_USER=admin 2. Create a file containing just the password for the MySQL user that Flyway will use. 3. Run Flyway migrate via the Docker container. * use your environment file created above for `${ENV_FILE}` * mount the password file to `/run/secrets/flyway_password`. `${PASSWORD_FILE}` is the absolute path to the file on the host machine * `${MISO_VERSION}` must match the version of MISO that you are deploying docker run --rm --env-file "${ENV_FILE}" -v "${PASSWORD_FILE}:/run/secrets/flyway_password:ro" --add-host=host.docker.internal:host-gateway ghcr.io/miso-lims/miso-lims-migration:${MISO_VERSION} migrate ```

If you run into an issue with migration V0611, ensure that the user running Flyway has read and write permissions on MISO_FILES_DIR.

Server Deployment Troubleshooting

  • Whenever unexpected behaviour arises, be sure to check the build logs, located at ${CATALINA_BASE}/logs. Read through the output of catalina.out, catalina.<date>.log, and localhost.<date>.log.
  • Ensure you are using the correct version of Java with Tomcat. This can be checked in catalina.out. In case you are not, append/modify the value of $JAVA_HOME in setenv.sh or /etc/default/tomcat10.
  • If file access for /storage/miso is causing an error, this may help.