miniwdl run
container runtimes¶
miniwdl’s default Docker task runtime is recommended for production use. Users unable to employ the Docker daemon may configure miniwdl to use Podman, Singularity, or udocker instead (locally on Linux hosts only). Podman is the most compatible with Docker, but also typically requires root privileges. Singularity and udocker don’t need to run as root, but impose a few other limitations.
Podman (BETA)¶
Podman closely resembles Docker, to the extent that it’s usually operated as root. If miniwdl is configured to use Podman (and isn’t running as root itself), then it attempts to use sudo podman
internally, which therefore must not require interactive password entry.
To configure miniwdl to use sudo podman
instead of Docker:
- Allow relevant user(s) to
sudo podman
without password entry using this procedure. - Run the following command to sanity-check Podman:
sudo -kn podman run ubuntu:20.04 cat /etc/issue
- Set the environment variable
MINIWDL__SCHEDULER__CONTAINER_BACKEND=podman
or the equivalent configuration file option[scheduler] container_backend=podman
- Test the configuration with
miniwdl run_self_test
If your podman installation is configured such that you needn’t/can’t run it with sudo
, then override MINIWDL__PODMAN__EXE='["podman"]'
or set [podman] exe = ["podman"]
, and proceed with step 3 above.
If podman seems unable to pull Docker Hub images by “short-name”: see this thread
Limitations:
- Without the aforementioned procedure to enable passwordless
sudo podman
, typical defaults require password entry after a five-minute timeout since the lastsudo
operation. This can cause confusion when test runs succeed because they turn over containers rapidly, but lengthy workflows fail in the middle due to this timeout.- Alternatives to the passwordless configuration include increasing the timeout, using a background script to refresh it temporarily, or running miniwdl itself as root.
- Unlike with the Docker daemon, separate
miniwdl run
processes using the Podman runtime don’t coordinate their CPU/memory reservations for container scheduling. Running multiple resource-intensive workflows concurrently (with separateminiwdl run
invocations) is liable to overload the host.
Singularity (BETA)¶
Once a system administrator installs Singularity, it’s usually operated without root privileges. To configure miniwdl to use Singularity instead of Docker:
- Run the following command to sanity-check Singularity:
singularity run --fakeroot docker://ubuntu:20.04 cat /etc/issue
- Set the environment variable
MINIWDL__SCHEDULER__CONTAINER_BACKEND=singularity
or the equivalent configuration file option[scheduler] container_backend=singularity
- Test the configuration with
miniwdl run_self_test
Limitations:
- Inside a Singularity container, only the working directory and /tmp are writable, while the rest of the file system is read-only. Task commands that attempt to write elsewhere (e.g. installing software/libraries at runtime) will fail.
- Tasks’
runtime.docker
declarations are used as Docker (OCI) image tags with Singularity’s Docker image import features; SIF image files cannot yet be used directly. - Task containers aren’t actually restricted to use only their declared
runtime.cpu
andruntime.memory
resources once they start, although those reservations are considered for parallel scheduling.- If parallel container scheduling causes problems, then it can be disabled (serializing the workflow) by setting the environment variable
MINIWDL__SCHEDULER__TASK_CONCURRENCY=1
or the equivalent configuration file option[scheduler] task_concurrency=1
.
- If parallel container scheduling causes problems, then it can be disabled (serializing the workflow) by setting the environment variable
- Unlike with the Docker daemon, separate
miniwdl run
processes using the Singularity runtime don’t coordinate their CPU/memory reservations for container scheduling. Running multiple resource-intensive workflows concurrently (with separateminiwdl run
invocations) is liable to overload the host.
udocker (BETA)¶
udocker typically doesn’t require root privileges to operate nor even install, but it affects execution speed and provides less isolation between containers.
To configure miniwdl to use udocker:
- Run the following command to sanity-check udocker:
udocker run ubuntu:20.04 cat /etc/issue
- Set the environment variable
MINIWDL__SCHEDULER__CONTAINER_BACKEND=udocker
or the equivalent configuration file option[scheduler] container_backend=udocker
- Test the configuration with
miniwdl run_self_test
Limitations:
- Tasks running in udocker are able to overwrite their input files (unlike Docker which mounts them read-only). Doing so may have undefined effects on other tasks and the workflow as a whole.
- Task containers aren’t actually restricted to use only their declared
runtime.cpu
andruntime.memory
resources once they start, although those reservations are considered for parallel scheduling.- If parallel container scheduling causes problems, then it can be disabled (serializing the workflow) by setting the environment variable
MINIWDL__SCHEDULER__TASK_CONCURRENCY=1
or the equivalent configuration file option[scheduler] task_concurrency=1
.
- If parallel container scheduling causes problems, then it can be disabled (serializing the workflow) by setting the environment variable
- Unlike with the Docker daemon, separate
miniwdl run
processes using the udocker runtime don’t coordinate their CPU/memory reservations for container scheduling. Running multiple resource-intensive workflows concurrently (with separateminiwdl run
invocations) is liable to overload the host.
External plugins¶
- miniwdl-omics-run tool for the Amazon Omics workflow service
- AWS Batch
- SLURM
New container backends can be added through miniwdl’s Python plugin API. The miniwdl-backend-example provides a reference implementation as a starting point.