FAQ for miniwdl

miniwdl is a local runner and developer toolkit for the bioinformatics-focused Workflow Description Language (WDL).

TIP: If you are new to working with WDL workflow language, you may want to review the open source ‘learn-wdl’ course’ - link.

Also there is an embedded short course ‘learn-miniwdl’ which includes screencasts reviewing the concepts on this page in more detail - link

Common Questions

Install and verify miniwdl

  • Q: miniwdl won’t install (or work) on my machine

    • Verify installation of Python 3.6+

    • Verify installation of Docker.

  • Q: miniwdl isn’t working on my Mac

    • Verify Docker version (17+) and user permission for Docker.

    • You’ll first need to override the TMPDIR environment variable, e.g. export TMPDIR=/tmp to allow Docker containers to mount shared working directories.

    • Please file any other issues that arise!

  • Q: the miniwdl run command won’t run my WDL workflow

    • Verify that each workflow task runs from a Docker container image.

    • Define the container image in the WDL script (task section(s)).

  • Q: what is the quickest way to verify that miniwdl is properly installed?

    • Run miniwdl --help and review the list of possible commands.

Run WDL workflow with the miniwdl run command

  • Q: what is simplest way to test running a WDL workflow on miniwdl?

    • Run miniwdl run_self_test - it includes an example workflow.

    • Verify this job returns the ok and done messages as expected.

  • Q: how can I run a quick hello.wdl WDL workflow?

    • Copy this file to your miniwdl development environment.

    • Run miniwdl run hello.wdl, there are no input parameters used in this quick test.

  • Q: Where is the job output from miniwdl job?

    • miniwdl produces workflow, task and job logs as well as task output.

    • See the folder\file structure generated for each job run for these files.

  • Q: What does miniwdl run myfile.wdl do?

    • If your WDL file defines inputs and outputs, then it prints a list of inputs and outputs and notes which inputs are required.

  • Q: How to do I assign values to input variables when I use miniwdl to run a WDL file?

    • you can assign directly on the command line, i.e. miniwdl run myfile.wdl input1=input.bam input2= input.bai, etc...

    • or you can use the --input flag and assign input values via an input.json file

  • Q: How can I get more information about job execution?

    • Run the miniwdl run myfile.wdl command

    • Use the --verbose flag

  • Q: What is the _LAST directory?

    • miniwdl generates a symbolic link _LAST pointing to the timestamped subdirectory for most recent run, and an out directory tree containing symbolic links to the output files.

Scaling up miniwdl run

  • Q: Can miniwdl handle large-scale workloads?

    • A more-powerful host enables larger workloads, as it schedules WDL tasks in parallel up to the avaliable CPUs & memory

    • The optimization guide has host configuration suggestions

  • Q: Can miniwdl run workflows on my cloud/cluster?

    • If you can factor the workload into separate workflow runs, submit each one as a job to run miniwdl on a powerful worker node.

    • Alternate container runtimes widen local compatibility

    • External plugins enable cloud &cluster integrations, with an API to add new ones.

Check WDL syntax with the miniwdl check command

  • Q: When I run miniwdl check myfile.wdl it doesn’t produce any output

    • Re-run the command, sometimes it takes a couple of seconds to complete the WDL file parsing

  • Q: Do I have to fix all of items listed (warnings) in the results of miniwdl check myfile.wdl

    • This command generates both warnings (which you optionally fix) and errors (which you must fix).

    • Errors are shown in red.