FAQ for miniwdl

miniwdl is a local runner and developer toolkit for the bioinformatics-focused Workflow Description Language (WDL).
TIP: If you are new to working with WDL workflow language, you may want to review the open source ‘learn-wdl’ course’ - link.
Also there is an embedded short course ‘learn-miniwdl’ which includes screencasts reviewing the concepts on this page in more detail - link
Common Questions
Install and verify miniwdl
Q: miniwdl won’t install (or work) on my machine
Verify installation of Python 3.6+
Verify installation of Docker.
Q: miniwdl isn’t working on my Mac
Verify Docker version (17+) and user permission for Docker.
You’ll first need to override the
TMPDIR
environment variable, e.g.export TMPDIR=/tmp
to allow Docker containers to mount shared working directories.Please file any other issues that arise!
Q: the
miniwdl run
command won’t run my WDL workflowVerify that each workflow task runs from a Docker container image.
Define the container image in the WDL script (task section(s)).
Q: what is the quickest way to verify that miniwdl is properly installed?
Run
miniwdl --help
and review the list of possible commands.
Run WDL workflow with the miniwdl run
command
Q: what is simplest way to test running a WDL workflow on miniwdl?
Run
miniwdl run_self_test
- it includes an example workflow.Verify this job returns the
ok
anddone
messages as expected.
Q: how can I run a quick
hello.wdl
WDL workflow?Copy this file to your miniwdl development environment.
Run
miniwdl run hello.wdl
, there are no input parameters used in this quick test.
Q: Where is the job output from miniwdl job?
miniwdl produces workflow, task and job logs as well as task output.
See the folder\file structure generated for each job run for these files.
Q: What does
miniwdl run myfile.wdl
do?If your WDL file defines inputs and outputs, then it prints a list of inputs and outputs and notes which inputs are required.
Q: How to do I assign values to input variables when I use miniwdl to run a WDL file?
you can assign directly on the command line, i.e.
miniwdl run myfile.wdl input1=input.bam input2= input.bai, etc...
or you can use the
--input
flag and assign input values via aninput.json
file
Q: How can I get more information about job execution?
Run the
miniwdl run myfile.wdl
commandUse the
--verbose
flag
Q: What is the
_LAST
directory?miniwdl generates a symbolic link
_LAST
pointing to the timestamped subdirectory for most recent run, and anout
directory tree containing symbolic links to the output files.
Scaling up miniwdl run
Q: Can miniwdl handle large-scale workloads?
A more-powerful host enables larger workloads, as it schedules WDL tasks in parallel up to the avaliable CPUs & memory
The optimization guide has host configuration suggestions
Q: Can miniwdl run workflows on my cloud/cluster?
If you can factor the workload into separate workflow runs, submit each one as a job to run miniwdl on a powerful worker node.
Alternate container runtimes widen local compatibility
External plugins enable cloud &cluster integrations, with an API to add new ones.
Check WDL syntax with the miniwdl check
command
Q: When I run
miniwdl check myfile.wdl
it doesn’t produce any outputRe-run the command, sometimes it takes a couple of seconds to complete the WDL file parsing
Q: Do I have to fix all of items listed (warnings) in the results of
miniwdl check myfile.wdl
This command generates both warnings (which you optionally fix) and errors (which you must fix).
Errors are shown in red.
Links
chanzuckerberg/miniwdl GitHub where issues & contributions are welcome
openwdl/wdl GitHub for WDL spec & proposals
OpenWDL Slack #miniwdl for discussions
CZI Science Tech sponsors this project