minIO

Appliance Description

MinIO is an object storage solution that provides an API compatible with Amazon Web Services S3. The HyperCX MinIO appliance deploys a MinIO running on Ubuntu.

Main Features

Based on MinIO release on Ubuntu 22.04 LTS
TLS configuration with self-generated certificates. Enabled by default
Single-Drive and Multi-Drive deployment
Multi-Node deployment, optional as a OneFlow service template

Requirements

HyperCX version: >= 10.9
Recommended Memory: 32GB
Minimum disks for Single-Node Multi-Drive: 4 disks, 8 disks recommended

Quick Start

The default template will instantiate a Single-Node Single-Drive deployment, recommended only for local testing and evaluation or small-scale data workloads that do not have availability or performance requirements.

Steps to deploy a Single-Node instance:

Download the MinIO appliance from the Appliances Marketplace. This will download the VM template and the image for the OS.
Create empty datablock for storage disks.
Adjust the VM template as desired (i.e. CPU, MEMORY, number of disks).
Instantiate MinIO template.
Attach a new NIC to the VM.
Access your new MinIO instance on https://<vm-ip-address>:9001 and login with the defined user and password.

To instantiate a Single-Node Multi-Drive deployment instead, it is necessary to add extra disks to the template file. A template for a Single-Node Multi-Drive with 4 disks for MinIO storage should look like this:

  CONTEXT=[
    NETWORK="YES",
    ONEAPP_MINIO_HOSTNAME="$ONEAPP_MINIO_HOSTNAME",
    ONEAPP_MINIO_OPTS="$ONEAPP_MINIO_OPTS",
    ONEAPP_MINIO_ROOT_PASSWORD="$ONEAPP_MINIO_ROOT_PASSWORD",
    ONEAPP_MINIO_ROOT_USER="$ONEAPP_MINIO_ROOT_USER",
    ONEAPP_MINIO_TLS_CERT="$ONEAPP_MINIO_TLS_CERT",
    ONEAPP_MINIO_TLS_KEY="$ONEAPP_MINIO_TLS_KEY",
    SSH_PUBLIC_KEY="$USER[SSH_PUBLIC_KEY]" ]
  CPU="1"
  DISK=[
    DEV_PREFIX="vd",
    IMAGE="service-MinIO",
    IMAGE_UNAME="oneadmin" ]
  DISK=[
    DEV_PREFIX="vd",
    IMAGE="minio-disk",
    IMAGE_UNAME="oneadmin" ]
  DISK=[
    DEV_PREFIX="vd",
    IMAGE="minio-disk",
    IMAGE_UNAME="oneadmin" ]
  DISK=[
    DEV_PREFIX="vd",
    IMAGE="minio-disk",
    IMAGE_UNAME="oneadmin" ]
  DISK=[
    DEV_PREFIX="vd",
    IMAGE="minio-disk",
    IMAGE_UNAME="oneadmin" ]
  GRAPHICS=[
    LISTEN="0.0.0.0",
    TYPE="VNC" ]
  HOT_RESIZE=[
    CPU_HOT_ADD_ENABLED="NO",
    MEMORY_HOT_ADD_ENABLED="NO" ]
  HYPERVISOR="kvm"
  LOGO="images/logos/minio.png"
  MEMORY="2048"
  MEMORY_UNIT_COST="MB"
  USER_INPUTS=[
    ONEAPP_MINIO_HOSTNAME="O|text|MinIO hostname for TLS certificate| |localhost,minio-*.example.net",
    ONEAPP_MINIO_OPTS="O|text|Additional commandline options for MinIO server| |--console-address :9001",
    ONEAPP_MINIO_ROOT_PASSWORD="O|password|MinIO root user password for MinIO server",
    ONEAPP_MINIO_ROOT_USER="O|text|MinIO root user for MinIO server| |myminioadmin",
    ONEAPP_MINIO_TLS_CERT="O|text64|MinIO TLS certificate (.crt)| |",
    ONEAPP_MINIO_TLS_KEY="O|text64|MinIO TLS key (.key)| |" ]

Contextualization

The contextualization parameters Context Section in the VM template controls the configuration of the service, see the table below:

Parameter	Default	Description
`ONEAPP_MINIO_ROOT_USER`	`myminioadmin`	MinIO server root account.
`ONEAPP_MINIO_ROOT_PASSWORD`	`minio-secret-key-change-me`	MinIO server root account password.
`ONEAPP_MINIO_OPTS`	`--console-address :9001`	MinIO server additional command-line options. If `--console-address` is omitted, MinIO generates a dynamic port at startup and writes it to the system log.
`ONEAPP_MINIO_HOSTNAME`	`localhost,minio-\*.example.net`	Hostnames used to auto-generate certificates when no certificate or key are provided.
`ONEAPP_MINIO_TLS_ENABLED`	`YES`	Enable TLS configuration.
`ONEAPP_MINIO_TLS_CERT`		TLS certificate. If `ONEAPP_MINIO_TLS_CERT` or `ONEAPP_MINIO_TLS_KEY` is empty, certificates are auto-generated with MinIO certgen.
`ONEAPP_MINIO_TLS_KEY`		TLS key. If `ONEAPP_MINIO_TLS_CERT` or `ONEAPP_MINIO_TLS_KEY` is empty, certificates are auto-generated with MinIO certgen.
`ONEAPP_MINIO_MULTI`	`NO`	MinIO Multi-Node configuration. To be used with the OneFlow service template; see topic MinIO Multi-Node.

Warning

Do not add the option --certs-dir on the ONEAPP_MINIO_OPTS, since it is hardcoded in the scripts. The configuration scripts will append the configuration --certs-dir /opt/minio/certs to the ONEAPP_MINIO_OPTS parameter, so on the MinIO configuration file (/etc/default/minio) the MINIO_OPTS environment variable will look like this: MINIO_OPTS="${ONEAPP_MINIO_OPTS} --certs-dir /opt/minio/certs".

TLS Configuration

When the parameter ONEAPP_MINIO_TLS_ENABLED="YES" is set, the appliance will configure the MinIO deployment to use TLS by appending --certs-dir /opt/minio/certs to the MINIO_OPTS in the configuration file /etc/default/minio.

The appliance will create the folder /opt/minio/certs and two files in that location: public.crt with the contents of ONEAPP_MINIO_TLS_CERT and private.key with the contents of ONEAPP_MINIO_TLS_KEY. If either of those variables is empty, the scripts will autogenerate new certificates using the MinIO certgen tool.

Note

On recontextualization the scripts will skip the certificate creation if public.crt or private.key are present in /opt/minio/certs. In order to update the certificates it would be necessary to first manually delete the existing ones and then recontext the VM.

Storage volumes

The contextualization scripts will detect how many additional disks are attached to the VM on instantiation and create a partition on each of those disks, formatting it with XFS.

Warning

Do NOT attach additional disks to the VM once instantiated. The contextualization scripts is not prepared to handle disks reconfiguration. If you would like to resize the disks of the VM you have to do it manually under your own resposibility.

Note

There could be performance and / or availability constraints when deploying Multi-Drive instances, for instance if all virtual disks are created on the same phisical disk.

Persistent storage

It is possible to configure persistent images to preserve storage after MinIO instance is terminated. To achieve this it would be necessary to create a persistent image for each disk on the MinIO deployment and add them to the template.

For example, to configure a Single-Node Single-Drive deployment with persistent storage first create a persistent Datablock first.

Then update the template to use the persistent image. This is an example of how would looks like the template with this modification:

    CONTEXT=[
        NETWORK="YES",
        ONEAPP_MINIO_HOSTNAME="$ONEAPP_MINIO_HOSTNAME",
        ONEAPP_MINIO_OPTS="$ONEAPP_MINIO_OPTS",
        ONEAPP_MINIO_ROOT_PASSWORD="$ONEAPP_MINIO_ROOT_PASSWORD",
        ONEAPP_MINIO_ROOT_USER="$ONEAPP_MINIO_ROOT_USER",
        ONEAPP_MINIO_TLS_ENABLED = "$ONEAPP_MINIO_TLS_ENABLED",
        ONEAPP_MINIO_TLS_CERT="$ONEAPP_MINIO_TLS_CERT",
        ONEAPP_MINIO_TLS_KEY="$ONEAPP_MINIO_TLS_KEY",
        SSH_PUBLIC_KEY="$USER[SSH_PUBLIC_KEY]" ]
    CPU="1"
    DISK=[
        DEV_PREFIX="vd",
        IMAGE="service-MinIO",
    IMAGE_UNAME="oneadmin" ]
    DISK=[
        IMAGE="MinIO-disk-1",
        IMAGE_UNAME="oneadmin" ]
    GRAPHICS=[
        LISTEN="0.0.0.0",
        TYPE="VNC" ]
    HOT_RESIZE=[
    CPU_HOT_ADD_ENABLED="NO",
    MEMORY_HOT_ADD_ENABLED="NO" ]
    HYPERVISOR="kvm"
    LOGO="images/logos/ubuntu.png"
    MEMORY="2048"
    MEMORY_UNIT_COST="MB"
    USER_INPUTS=[
        ONEAPP_MINIO_HOSTNAME="O|text|MinIO hostname for TLS certificate| |localhost,minio-*.example.net",
        ONEAPP_MINIO_OPTS="O|text|Additional commandline options for MinIO server| |--console-address :9001",
        ONEAPP_MINIO_ROOT_PASSWORD="O|password|MinIO root user password for MinIO server",
        ONEAPP_MINIO_ROOT_USER="O|text|MinIO root user for MinIO server| |myminioadmin",
        ONEAPP_MINIO_TLS_ENABLED = "M|boolean|Enable TLS configuration| |YES",
        ONEAPP_MINIO_TLS_CERT="O|text64|MinIO TLS certificate (.crt)| |",
        ONEAPP_MINIO_TLS_KEY="O|text64|MinIO TLS key (.key)| |" ]

To configure a Single-Node Multi-Drive instance with persistent storage, create a persistent image for each disk on the deployment and add them to the VM template. For example, a Single-Node Multi-Drive deployment using persistent disks could be like this:

    CONTEXT=[
        NETWORK="YES",
        ONEAPP_MINIO_HOSTNAME="$ONEAPP_MINIO_HOSTNAME",
        ONEAPP_MINIO_OPTS="$ONEAPP_MINIO_OPTS",
        ONEAPP_MINIO_ROOT_PASSWORD="$ONEAPP_MINIO_ROOT_PASSWORD",
        ONEAPP_MINIO_ROOT_USER="$ONEAPP_MINIO_ROOT_USER",
        ONEAPP_MINIO_TLS_ENABLED = "$ONEAPP_MINIO_TLS_ENABLED",
        ONEAPP_MINIO_TLS_CERT="$ONEAPP_MINIO_TLS_CERT",
        ONEAPP_MINIO_TLS_KEY="$ONEAPP_MINIO_TLS_KEY",
        SSH_PUBLIC_KEY="$USER[SSH_PUBLIC_KEY]" ]
    CPU="1"
    DISK=[
        DEV_PREFIX="vd",
        IMAGE="service-MinIO",
    IMAGE_UNAME="oneadmin" ]
    DISK=[
        IMAGE="MinIO-disk-1",
        IMAGE_UNAME="oneadmin" ]
    DISK=[
        IMAGE="MinIO-disk-2",
        IMAGE_UNAME="oneadmin" ]
    DISK=[
        IMAGE="MinIO-disk-3",
        IMAGE_UNAME="oneadmin" ]
    DISK=[
        IMAGE="MinIO-disk-4",
        IMAGE_UNAME="oneadmin" ]
    GRAPHICS=[
        LISTEN="0.0.0.0",
        TYPE="VNC" ]
    HOT_RESIZE=[
    CPU_HOT_ADD_ENABLED="NO",
    MEMORY_HOT_ADD_ENABLED="NO" ]
    HYPERVISOR="kvm"
    LOGO="images/logos/ubuntu.png"
    MEMORY="2048"
    MEMORY_UNIT_COST="MB"
    USER_INPUTS=[
        ONEAPP_MINIO_HOSTNAME="O|text|MinIO hostname for TLS certificate| |localhost,minio-*.example.net",
        ONEAPP_MINIO_OPTS="O|text|Additional commandline options for MinIO server| |--console-address :9001",
        ONEAPP_MINIO_ROOT_PASSWORD="O|password|MinIO root user password for MinIO server",
        ONEAPP_MINIO_ROOT_USER="O|text|MinIO root user for MinIO server| |myminioadmin",
        ONEAPP_MINIO_TLS_ENABLED = "M|boolean|Enable TLS configuration| |YES",
        ONEAPP_MINIO_TLS_CERT="O|text64|MinIO TLS certificate (.crt)| |",
        ONEAPP_MINIO_TLS_KEY="O|text64|MinIO TLS key (.key)| |" ]

MinIO Multi-Node OneFlow Service

The Multi-Node deployment mode is delivered as a OneFlow Service template, using the same disk image as the one used for the Single-Node deployment, check the Contextualization topic for the list of available parameters. The service template consists of two roles:

vr: Uses the Virtual Router appliance with a custom VM template configured to enable Load Balancing to the deployed MinIO instance (as recommended in the MinIO Multi-Node Networking and Firewalls documentation).
minio: MinIO backend servers to be configured in a Multi-Node deployment. Adjust this role’s cardinality to set the number of MinIO backends (minimum recommended is 4).

Downloading and Deploying MinIO Multi-Node

Download the Service MinIO Multi-Node appliance from the Appliances Marketplace.
Adjust MinIO Multi-Node VM template to your needs. Add the necessary additional drives to the template as documented in the topic Quick Start. You can also set the CPU and RAM for the VMs, according to your needs and following the recommended MinIO requirements.
Configure networks for the Service MinIO Multi-Node, by selecting the OpenNebula existing private and public networks that the service will use.
Configure Role MinIO parameters to match your needs. In this step you can enable the TLS configuration, in which case the TLS certificate and key parameters have to be provided, in plain text, so the configuracion succeeds.

Note

If you want to disable the TLS configuration, remember to change the “LB port for MinIO Console parameter” to the port 80 -or the port of your choice- instead of 443 in both roles.

Important

The MinIO hostname parameter is necessary for the MinIO instances to be able to communicate internally, please configure a valid hostname (e.g. minio.example.local). The provided certificate for the TLS connection must match this hostname.

Instantiate the service.
Access the MinIO instance connecting through the public NIC of the vr role VM.

Requirements

HyperCX version: >= 10.9
OneFlow and OneGate for multi-node orchestration.
Minimal Memory per VM: 512 MB (VR), 2 GB (MinIO).
Minimal Cores (VCPU) per VM: 1 (VR), 1 (MinIO).
Recommended specs for MinIO VMs: Check MinIO Hardware Recommendations.

Main References

MinIO in the OpenNebula one-apps project
Full documentation for the MinIO appliance