FAQ for miniwdl¶

miniwdl is a local runner and developer toolkit for the bioinformatics-focused Workflow Description Language (WDL).
TIP: If you are new to working with WDL workflow language, you may want to review the open source ‘learn-wdl’ course’ - link.
Also there is an embedded short course ‘learn-miniwdl’ which includes screencasts reviewing the concepts on this page in more detail - link
Common Questions¶
Install and verify miniwdl¶
- Q: miniwdl won’t install (or work) on my machine
- Verify installation of Python 3.6+
- Verify installation of Docker.
- Q: miniwdl isn’t working on my Mac
- Verify Docker version (17+) and user permission for Docker.
- You’ll first need to override the
TMPDIR
environment variable, e.g.export TMPDIR=/tmp
to allow Docker containers to mount shared working directories. - Please file any other issues that arise!
- Q: the
miniwdl run
command won’t run my WDL workflow- Verify that each workflow task runs from a Docker container image.
- Define the container image in the WDL script (task section(s)).
- Q: what is the quickest way to verify that miniwdl is properly installed?
- Run
miniwdl --help
and review the list of possible commands.
- Run
Run WDL workflow with the miniwdl run
command¶
- Q: what is simplest way to test running a WDL workflow on miniwdl?
- Run
miniwdl run_self_test
- it includes an example workflow. - Verify this job returns the
ok
anddone
messages as expected.
- Run
- Q: how can I run a quick
hello.wdl
WDL workflow?- Copy this file to your miniwdl development environment.
- Run
miniwdl run hello.wdl
, there are no input parameters used in this quick test.
- Q: Where is the job output from miniwdl job?
- miniwdl produces workflow, task and job logs as well as task output.
- See the folder\file structure generated for each job run for these files.
- Q: What does
miniwdl run myfile.wdl
do?- If your WDL file defines inputs and outputs, then it prints a list of inputs and outputs and notes which inputs are required.
- Q: How to do I assign values to input variables when I use miniwdl to run a WDL file?
- you can assign directly on the command line, i.e.
miniwdl run myfile.wdl input1=input.bam input2= input.bai, etc...
- or you can use the
--input
flag and assign input values via aninput.json
file
- you can assign directly on the command line, i.e.
- Q: How can I get more information about job execution?
- Run the
miniwdl run myfile.wdl
command - Use the
--verbose
flag
- Run the
- Q: What is the
_LAST
directory?- miniwdl generates a symbolic link
_LAST
pointing to the timestamped subdirectory for most recent run, and anout
directory tree containing symbolic links to the output files.
- miniwdl generates a symbolic link
Scaling up miniwdl run
¶
- Q: Can miniwdl handle large-scale workloads?
- A more-powerful host enables larger workloads, as it schedules WDL tasks in parallel up to the avaliable CPUs & memory
- The optimization guide has host configuration suggestions
- Q: Can miniwdl run workflows on my cloud/cluster?
- If you can factor the workload into separate workflow runs, submit each one as a job to run miniwdl on a powerful worker node.
- Alternate container runtimes widen local compatibility
- External plugins enable cloud &cluster integrations, with an API to add new ones.
Check WDL syntax with the miniwdl check
command¶
- Q: When I run
miniwdl check myfile.wdl
it doesn’t produce any output- Re-run the command, sometimes it takes a couple of seconds to complete the WDL file parsing
- Q: Do I have to fix all of items listed (warnings) in the results of
miniwdl check myfile.wdl
- This command generates both warnings (which you optionally fix) and errors (which you must fix).
- Errors are shown in red.
Links¶
- chanzuckerberg/miniwdl GitHub where issues & contributions are welcome
- openwdl/wdl GitHub for WDL spec & proposals
- OpenWDL Slack #miniwdl for discussions
- CZI Science Tech sponsors this project