Python WDL package

miniwdl is a developer toolkit and local runner for the bioinformatics-focused Workflow Description Language (WDL). This documentation covers the Python3 WDL package facilitating parsing & static analysis of WDL documents. Simply import WDL once miniwdl has been installed.

WDL.load(uri: str, path: Optional[List[str]] = None, check_quant: bool = True, read_source: Optional[Callable[[str, List[str], Optional[WDL.Tree.Document]], Awaitable[ReadSourceResult]]] = None, import_max_depth: int = 10) → WDL.Tree.Document[source]

Parse a WDL document given filename/URI, recursively descend into imported documents, then typecheck the tasks and workflow.

Parameters:
  • path – local filesystem directories to search for imports, in addition to the current working directory
  • check_quant – set to False to relax static typechecking of the optional (?) and nonempty (+) type quantifiers. This is discouraged, but may be useful for older WDL workflows which assume less-rigorous static validation of these annotations.
  • read_source – async routine to read the WDL source code from filename/URI; see read_source_default() below for details
  • import_max_depth – to prevent recursive import infinite loops, fail when there are too many import nesting levels (default 10)
Raises:
class WDL.ReadSourceResult[source]

The NamedTuple to be returned by the read_source routine. Its source_text: str field provides the WDL source code, and the abspath: str field is the absolute filename/URI from which the source was read (e.g. after resolving a relative path).

WDL.values_from_json(values_json: Dict[str, Any], available: WDL.Env.Bindings[typing.Union[WDL.Tree.Decl, WDL.Type.Base]][Union[WDL.Tree.Decl, WDL.Type.Base]], required: Optional[WDL.Env.Bindings[typing.Union[WDL.Tree.Decl, WDL.Type.Base]][Union[WDL.Tree.Decl, WDL.Type.Base]]] = None, namespace: str = '') → WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base][source]

Given a dict parsed from Cromwell-style JSON and the available input (or output) declarations of a task or workflow, create a WDL.Env.Bindings[Value.Base].

Parameters:
  • required – raise an error if any of these required inputs aren’t present
  • namespace – expect each key to start with this namespace prefixed to the input/output names (e.g. the workflow name)
WDL.values_to_json(values_env: WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base], namespace: str = '') → Dict[str, Any][source]

Convert a WDL.Env.Bindings[WDL.Value.Base] to a dict which json.dumps to Cromwell-style JSON.

Parameters:namespace – prefix this namespace to each key (e.g. workflow name)
WDL.load_async(uri: str, path: Optional[List[str]] = None, check_quant: bool = True, read_source: Optional[Callable[[str, List[str], Optional[WDL.Tree.Document]], Awaitable[ReadSourceResult]]] = None, import_max_depth: int = 10) → WDL.Tree.Document[source]

Async version of load(), with all the same arguments

WDL.read_source_default(uri: str, path: List[str], importer: Optional[WDL.Tree.Document]) → WDL.ReadSourceResult[source]

Default async routine for the read_source parameter to load() and load_async(), which they use to read the desired WDL document and its imports. This default routine handles local files only, supplying the search path logic to resolve relative filenames; it fails with network URIs.

Parameters:
  • uri – Filename/URI to read, as provided to load() or the WDL import statement; may be relative
  • path – Local directories to search for relative imports
  • importer – The document importing the one here requested, if any; the importer.pos.uri and importer.pos.abspath fields may be relevant to resolve relative imports.
Returns:

ReadSourceResult(source_text="...", abspath="...")

Callers may wish to override read_source with logic to download source code from network URIs, and for local filenames fall back to return await WDL.read_source_default(...).

Note: the synchronous load() merely calls load_async() on the current asyncio.get_event_loop() and awaits the result.

WDL.resolve_file_import(uri: str, path: List[str], importer: Optional[WDL.Tree.Document]) → str[source]

Exposes the logic by which read_source_default() resolves uri to the absolute path of an extant file. If uri is already an absolute path, it’s normalized and returned. A relative uri is resolved by first joining it to either, the directory of the importer document (if any), or the process current working directory (otherwise). Failing that, it’s searched in the path directories (in reverse order).

Security-focused applications may wish to override read_source with logic to restrict allowable results of resolve_file_import, to prevent WDL source code from triggering access to arbitrary filesystem paths. No such restrictions are applied by default.

Tree

Abstract syntax tree (AST) for WDL documents, containing tasks and workflows, which contain declarations, calls, and scatter & if sections. The AST is typically constructed by load().

The WDL.Tree.* classes are also exported by the base WDL module, i.e. WDL.Tree.Document can be abbreviated WDL.Document.

Inheritance diagram of WDL.Tree
class WDL.Tree.StructTypeDef(pos: WDL.Error.SourcePosition, name: str, members: Dict[str, WDL.Type.Base], imported: Optional[Tuple[WDL.Tree.Document, WDL.Tree.StructTypeDef]] = None)[source]

Bases: WDL.Error.SourceNode

WDL struct type definition

name = None
Type:str

Name of the struct type (in the current document)

members = None
Type:Dict[str, WDL.Type.Base]

Member names and types

imported = None
Type:Optional[Tuple[Document,StructTypeDef]]

If this struct is imported from another document, references that document and its definition there. The referenced definition might itself be imported from yet another document.

type_id
Type:str

A string canonically describing the member names and their types, excluding the struct type name; useful to unify aliased struct types.

class WDL.Tree.WorkflowNode(workflow_node_id: str, pos: WDL.Error.SourcePosition)[source]

Bases: WDL.Error.SourceNode, abc.ABC

Base class for workflow “nodes” including declarations, calls, and scatter/if sections and their bodies.

Each node has a human-readable ID string which is unique within the workflow. It also exposes the set of workflow node IDs upon which it depends. Abstractly, workflow execution can proceed by “visiting” each node once all of its dependencies have been visited, performing some action(s) appropriate to the specific node type (such as evaluating a WDL expression and binding a name in the environment, or executing a task and binding its outputs).

workflow_node_id = None
Type:str

Human-readable node ID unique within the current workflow

scatter_depth = None
Type:int

How many nested scatter sections the node lies within. This information is useful for runtime dependency analysis in workflows with scatters. When scatter sections are nested within conditional sections or vice versa, this counts the scatters only.

workflow_node_dependencies
Type:Set[str]

Set of workflow node IDs on which this node depends. Available once workflow has been typechecked.

class WDL.Tree.Decl(pos: WDL.Error.SourcePosition, type: WDL.Type.Base, name: str, expr: Optional[WDL.Expr.Base] = None, id_prefix='decl')[source]

Bases: WDL.Tree.WorkflowNode

A value declaration within a task or workflow.

Within a task, the declarations can be viewed as “workflow nodes” insofar as they must be evaluated in an order consistent with their dependency structure, and ensured acyclic. The “workflow node IDs” of a task’s declarations are unique within the task only, and unrelated to the top-level workflow, if any, in the WDL document.

type = None
Type:WDL.Type.Base
name = None

Declared value name

Type:str
expr = None
Type:Optional[WDL.Expr.Base]

Bound expression, if any

decor = None
class WDL.Tree.Task(pos: WDL.Error.SourcePosition, name: str, inputs: Optional[List[WDL.Tree.Decl]], postinputs: List[WDL.Tree.Decl], command: WDL.Expr.String, outputs: List[WDL.Tree.Decl], parameter_meta: Dict[str, Any], runtime: Dict[str, WDL.Expr.Base], meta: Dict[str, Any])[source]

Bases: WDL.Error.SourceNode

WDL Task

name = None
Type:str
inputs = None
Type:Optional[List[WDL.Tree.Decl]]

Declarations in the input{} task section, if it’s present

postinputs = None
Type:List[WDL.Tree.Decl]

Declarations outside of the input{} task section

command = None
Type:WDL.Expr.String
outputs = None
Type:List[WDL.Tree.Decl]

Output declarations

parameter_meta = None
Type:Dict[str,Any]

parameter_meta{} section as a JSON-like dict

runtime = None
Type:Dict[str,WDL.Expr.Base]

runtime{} section, with keys and corresponding expressions to be evaluated

meta = None
Type:Dict[str,Any]

meta{} section as a JSON-like dict

effective_wdl_version = None
Type:str

Effective WDL version of the containing document

available_inputs
Type:WDL.Env.Bindings[WDL.Tree.Decl]

Yields the task’s input declarations. This is all declarations in the task’s input{} section, if it’s present. Otherwise, it’s all declarations in the task, excluding outputs. (This dichotomy bridges pre-1.0 and 1.0+ WDL versions.)

Each input is at the top level of the Env, with no namespace.

required_inputs
Type:WDL.Env.Bindings[WDL.Tree.Decl]

Yields the input declarations which are required to call the task (available inputs that are unbound and non-optional).

Each input is at the top level of the Env, with no namespace.

effective_outputs
Type:WDL.Env.Bindings[Type.Base]

Yields each task output with its type, at the top level of the Env with no namespace. (Present for isomorphism with Workflow.effective_outputs)

digest

Content digest of the task, for use e.g. as a cache key. The digest is an opaque string of a few dozen alphanumeric characters, sensitive to the task’s source code (with best effort to exclude comments and whitespace).

class WDL.Tree.Call(pos: WDL.Error.SourcePosition, callee_id: List[str], alias: Optional[str], inputs: Dict[str, WDL.Expr.Base], after: Optional[List[str]] = None)[source]

Bases: WDL.Tree.WorkflowNode

A call (within a workflow) to a task or sub-workflow

callee_id = None
Type:List[str]

The called task; either one string naming a task in the current document, or an import namespace and task name.

name = None
Type:string

Call name, defaults to task/workflow name

inputs = None
Type:Dict[str,WDL.Expr.Base]

Call inputs provided

callee = None
Type:Union[WDL.Tree.Task, WDL.Tree.Workflow]

Refers to the Task or imported Workflow object to be called (after AST typechecking)

after = None
Type:string

Call names on which this call depends (even if none of their outputs are used in this call’s inputs)

available_inputs
Type:WDL.Env.Bindings[WDL.Tree.Decl]

Yields the task/workflow inputs which are not supplied in the call inputs:, and thus may be supplied at workflow launch; in namespaces according to the call names.

required_inputs
Type:WDL.Env.Bindings[WDL.Tree.Decl]

Yields the required task/workflow inputs which are not supplied in the call inputs: (incomplete calls), and thus must be supplied at workflow launch; in namespaces according to the call name.

effective_outputs
Type:WDL.Env.Bindings[WDL.Tree.Decl]

Yields the effective outputs of the callee Task or Workflow, in a namespace according to the call name.

class WDL.Tree.Gather(section: WDL.Tree.WorkflowSection, referee: Union[WDL.Tree.Decl, WDL.Tree.Call, WDL.Tree.Gather])[source]

Bases: WDL.Tree.WorkflowNode

A Gather node symbolizes the operation to gather an array of declared values or call outputs in a scatter section, or optional values from a conditional section. These operations are implicit in the WDL syntax, but explicating them in the AST facilitates analysis of the workflow’s data types and dependency structure.

Each scatter/conditional section provides Gather nodes to expose the section body’s products to the rest of the workflow. When a WDL.Expr.Ident elsewhere identifies a node inside the section, its referee attribute is the corresponding Gather node, which in turn references the interior node. The interior node might itself be another Gather node, from a nested scatter/conditional section.

section = None
Type:WorkflowSection

The Scatter/Conditional section implying this Gather operation

referee = None
Type:Union[Decl, Call, Gather]

The Decl, Call, or sub-Gather node from which this operation “gathers”

final_referee

The Decl or Call node found at the end of the referee chain through any nested Gather nodes

class WDL.Tree.WorkflowSection(body: List[WDL.Tree.WorkflowNode], *args, **kwargs)[source]

Bases: WDL.Tree.WorkflowNode

Base class for workflow nodes representing scatter and conditional sections

body = None
Type:List[WorkflowNode]

Section body, potentially including nested sections.

gathers = None
Type:Dict[str, Gather]

Gather nodes exposing the section body’s products to the rest of the workflow. The dict is keyed by workflow_node_id of the interior node, to expedite looking up the corresponding gather node.

The section’s body and gather nodes do not explicitly include the section node among their dependencies. Such dependence is implicit because the body subgraph can be “instantiated” only upon visiting the section node at runtime.

class WDL.Tree.Scatter(pos: WDL.Error.SourcePosition, variable: str, expr: WDL.Expr.Base, body: List[WDL.Tree.WorkflowNode])[source]

Bases: WDL.Tree.WorkflowSection

Workflow scatter section

variable = None
Type:string

Scatter variable name

expr = None
Type:WDL.Expr.Base

Expression for the array over which to scatter

class WDL.Tree.Conditional(pos: WDL.Error.SourcePosition, expr: WDL.Expr.Base, body: List[WDL.Tree.WorkflowNode])[source]

Bases: WDL.Tree.WorkflowSection

Workflow conditional (if) section

expr = None
Tree:WDL.Expr.Base

Boolean expression

class WDL.Tree.Workflow(pos: WDL.Error.SourcePosition, name: str, inputs: Optional[List[WDL.Tree.Decl]], body: List[WDL.Tree.WorkflowNode], outputs: Optional[List[WDL.Tree.Decl]], parameter_meta: Dict[str, Any], meta: Dict[str, Any], output_idents: Optional[List[List[str]]] = None, output_idents_pos: Optional[WDL.Error.SourcePosition] = None)[source]

Bases: WDL.Error.SourceNode

name = None
Type:str
inputs = None
Type:List[WDL.Tree.Decl]

Declarations in the input{} workflow section, if it’s present

body = None
Type:List[Union[WDL.Tree.Decl,WDL.Tree.Call,WDL.Tree.Scatter,WDL.Tree.Conditional]]

Workflow body in between input{} and output{} sections, if any

outputs = None
Type:Optional[List[WDL.Tree.Decl]]

Workflow output declarations, if the output{} section is present

parameter_meta = None
Type:Dict[str,Any]

parameter_meta{} section as a JSON-like dict

meta = None
Type:Dict[str,Any]

meta{} section as a JSON-like dict

complete_calls = None

After typechecking, False if the workflow has a call which does not supply all required inputs (and thus cannot be called from another workflow).

effective_wdl_version = None
Type:str

Effective WDL version of the containing document

available_inputs
Type:WDL.Env.Bindings[WDL.Tree.Decl]

The workflow’s input declarations. This includes:

1. If the input{} workflow section is present, all declarations within that section. Otherwise, all declarations in the top-level workflow body, excluding outputs. (This dichotomy bridges pre-1.0 and 1.0+ WDL versions.) These appear at the top level of the Env, with no namespace.

  1. Available inputs of all calls in the workflow, namespaced by the call names.
required_inputs
Type:WDL.Env.Bindings[Decl]

The subset of available inputs which are required to start the workflow.

effective_outputs
Type:WDL.Env.Bindings[Decl]

If the output{} workflow section is present, yields the names and types therein, at the top level of the Env. Otherwise, yield all the call outputs, namespaced and typed appropriately.

get_node(workflow_node_id: str) → WDL.Tree.WorkflowNode[source]

Look up WorkflowNode by workflow_node_id

digest

Content digest of the workflow, for use e.g. as a cache key. The digest is an opaque string of a few dozen alphanumeric characters, sensitive to the workflow’s source code (with best effort to exclude comments and whitespace) and the tasks and subworkflows it calls.

class WDL.Tree.SourceComment(pos, text)

Bases: tuple

Position and text of a comment. The text includes the # and any preceding or trailing spaces/tabs.

pos

Alias for field number 0

text

Alias for field number 1

class WDL.Tree.DocImport(pos, uri, namespace, aliases, doc)

Bases: tuple

Represents one imported document, with position of the import statement, import URI, namespace, struct type aliases, and (after typechecking) the Document object.

aliases

Alias for field number 3

doc

Alias for field number 4

namespace

Alias for field number 2

pos

Alias for field number 0

uri

Alias for field number 1

class WDL.Tree.Document(source_text: str, pos: WDL.Error.SourcePosition, imports: List[WDL.Tree.DocImport], struct_typedefs: Dict[str, WDL.Tree.StructTypeDef], tasks: List[WDL.Tree.Task], workflow: Optional[WDL.Tree.Workflow], comments: List[WDL.Tree.SourceComment], wdl_version: Optional[str])[source]

Bases: WDL.Error.SourceNode

Top-level document, with imports, tasks, and up to one workflow. Typically returned by load().

imports = None
Type:List[DocImport]

Imported documents

struct_typedefs = None
Type:Env.Bindings[WDL.Tree.StructTypeDef]
tasks = None
Type:List[WDL.Tree.Task]
workflow = None
Type:Optional[WDL.Tree.Workflow]
source_text = None
Type:str

Original WDL source code text

source_lines = None
Type:List[str]

Original WDL source code text split by newlines. SourcePosition line numbers are one-based, so line number L corresponds to source_lines[L-1].

source_comments = None
Type:List[Optional[SourceComment]]

Lookup table for source code comments. source_comments has the same length as source_lines, and each entry is the WDL.Tree.SourceComment found on the corresponding source line, or None if the line has no comment.

wdl_version = None
Type:Optional[str]

Declared WDL language version, if any

effective_wdl_version = None

:type”

wdl_version if wdl_version is not None else "draft-2"

typecheck(check_quant: bool = True) → None[source]

Typecheck each task in the document, then the workflow, if any.

Documents returned by load() have already been typechecked.

class WDL.Tree.ReadSourceResult(source_text, abspath)

Bases: tuple

abspath

Alias for field number 1

source_text

Alias for field number 0

Expr

WDL expressions composing literal values, arithmetic, comparison, conditionals, string interpolation, arrays & maps, and function applications. These appear on the right-hand side of value declarations and in task command substitutions, task runtime sections, and workflow scatter and conditional sections.

The abstract syntax tree (AST) for any expression is represented by an instance of a Python class deriving from WDL.Expr.Base. Any such node may have other nodes attached “beneath” it. An expression can be evaluated to a Value given a suitable WDL.Env.Bindings[Value.Base].

Inheritance diagram of WDL.Expr
class WDL.Expr.Base(pos: WDL.Error.SourcePosition)[source]

Bases: WDL.Error.SourceNode, abc.ABC

Superclass of all expression AST nodes

type
Type:WDL.Type.Base

WDL type of this expression. Undefined on construction; populated by one invocation of infer_type.

infer_type(self, type_env : Env.Bindings[Type.Base], stdlib : StdLib.Base) → WDL.Expr.Base[source]

Infer the expression’s type within the given type environment. Must be invoked exactly once prior to use of other methods.

Parameters:
  • stdlib – a context-specific standard function library for typechecking
  • check_quant – when False, disables static validation of the optional (?) type quantifier when typecheck() is called on this expression, so for example type T? can satisfy an expected type T. Applies recursively to the type inference and checking of any sub-expressions.
Raises:

WDL.Error.StaticTypeMismatch – when the expression fails to type-check

Returns:

self

typecheck(self, expected : Type.Base) → WDL.Expr.Base[source]

Check that this expression’s type is, or can be coerced to, expected.

Raises:WDL.Error.StaticTypeMismatch
Returns:self
eval(env: WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base], stdlib: WDL.StdLib.Base) → WDL.Value.Base[source]

Evaluate the expression in the given environment

Parameters:stdlib – a context-specific standard function library implementation
literal

If the expression is a literal constant, return its value; otherwise return None. The result can be an instance of WDL.Value.Null which is distinct from None.

class WDL.Expr.Boolean(pos: WDL.Error.SourcePosition, literal: bool)[source]

Bases: WDL.Expr.Base

Boolean literal

value = None
Type:bool

Literal value

class WDL.Expr.Int(pos: WDL.Error.SourcePosition, literal: int)[source]

Bases: WDL.Expr.Base

Integer literal

value = None
Type:int

Literal value

class WDL.Expr.Float(pos: WDL.Error.SourcePosition, literal: float)[source]

Bases: WDL.Expr.Base

Numeric literal

value = None
Type:float

Literal value

class WDL.Expr.Null(pos: WDL.Error.SourcePosition)[source]

Bases: WDL.Expr.Base

WDL None literal

(called Null to avoid conflict with Python None)

value = None
Type:None
class WDL.Expr.Placeholder(pos: WDL.Error.SourcePosition, options: Dict[str, str], expr: WDL.Expr.Base)[source]

Bases: WDL.Expr.Base

Holds an expression interpolated within a string or command

options = None
Type:Dict[str,str]

Placeholder options (sep, true, false, default)

expr = None
Type:WDL.Expr.Base

Expression to be evaluated and substituted

children
Type:Iterable[SourceNode]

Yield all child nodes

class WDL.Expr.String(pos: WDL.Error.SourcePosition, parts: List[Union[str, WDL.Expr.Placeholder]], command: bool = False)[source]

Bases: WDL.Expr.Base

String literal, possibly interleaved with expression placeholders for interpolation

parts = None
Type:List[Union[str,WDL.Expr.Placeholder]]

The parts list begins and ends with the original delimiters (quote marks, braces, or triple angle brackets). Between these is a sequence of literal strings and/or interleaved placeholder expressions. Escape sequences in the literals will NOT have been decoded (although the parser will have checked they’re valid). Strings arising from task commands leave escape sequences to be interpreted by the shell in the task container. Other string literals have their escape sequences interpreted upon evaluation to string values.

command = None
Type:bool

True if this expression is a task command template, as opposed to a string expression anywhere else. Controls whether backslash escape sequences are evaluated or (for commands) passed through for shell interpretation.

children
Type:Iterable[SourceNode]

Yield all child nodes

literal

If the expression is a literal constant, return its value; otherwise return None. The result can be an instance of WDL.Value.Null which is distinct from None.

class WDL.Expr.Array(pos: WDL.Error.SourcePosition, items: List[WDL.Expr.Base])[source]

Bases: WDL.Expr.Base

Array literal

items = None
Type:List[WDL.Expr.Base]

Expression for each item in the array literal

children
Type:Iterable[SourceNode]

Yield all child nodes

literal

If the expression is a literal constant, return its value; otherwise return None. The result can be an instance of WDL.Value.Null which is distinct from None.

class WDL.Expr.Pair(pos: WDL.Error.SourcePosition, left: WDL.Expr.Base, right: WDL.Expr.Base)[source]

Bases: WDL.Expr.Base

Pair literal

left = None
Type:WDL.Expr.Base

Left-hand expression in the pair literal

right = None
Type:WDL.Expr.Base

Right-hand expression in the pair literal

children
Type:Iterable[SourceNode]

Yield all child nodes

literal

If the expression is a literal constant, return its value; otherwise return None. The result can be an instance of WDL.Value.Null which is distinct from None.

class WDL.Expr.Map(pos: WDL.Error.SourcePosition, items: List[Tuple[WDL.Expr.Base, WDL.Expr.Base]])[source]

Bases: WDL.Expr.Base

Map literal

items = None
Type:List[Tuple[WDL.Expr.Base,WDL.Expr.Base]]

Expressions for the map literal keys and values

children
Type:Iterable[SourceNode]

Yield all child nodes

literal

If the expression is a literal constant, return its value; otherwise return None. The result can be an instance of WDL.Value.Null which is distinct from None.

class WDL.Expr.Struct(pos: WDL.Error.SourcePosition, members: List[Tuple[str, WDL.Expr.Base]], struct_type_name: Optional[str] = None)[source]

Bases: WDL.Expr.Base

Struct literal

members = None
Type:Dict[str,WDL.Expr.Base]

The struct literal is modelled initially as a bag of keys and values, which can be coerced to a specific struct type during typechecking.

struct_type_name = None
Type:Optional[str]

In WDL 2.0+ each struct literal may specify the intended struct type name.

children
Type:Iterable[SourceNode]

Yield all child nodes

literal

If the expression is a literal constant, return its value; otherwise return None. The result can be an instance of WDL.Value.Null which is distinct from None.

class WDL.Expr.IfThenElse(pos: WDL.Error.SourcePosition, condition: WDL.Expr.Base, consequent: WDL.Expr.Base, alternative: WDL.Expr.Base)[source]

Bases: WDL.Expr.Base

Ternary conditional expression

condition = None
Type:WDL.Expr.Base

A Boolean expression for the condition

consequent = None
Type:WDL.Expr.Base

Expression evaluated when the condition is true

alternative = None
Type:WDL.Expr.Base

Expression evaluated when the condition is false

children
Type:Iterable[SourceNode]

Yield all child nodes

class WDL.Expr.Ident(pos: WDL.Error.SourcePosition, name: str)[source]

Bases: WDL.Expr.Base

An identifier referencing a named value or call output.

Ident nodes are wrapped in Get nodes, as discussed below.

name = None
Type:str

Name, possibly including a dot-separated namespace

referee = None

After typechecking within a task or workflow, stores the AST node to which the identifier refers: a WDL.Tree.Decl for value references; a WDL.Tree.Call for call outputs; a WDL.Tree.Scatter for scatter variables; or a WDL.Tree.Gather object representing a value or call output that resides within a scatter or conditional section.

children
Type:Iterable[SourceNode]

Yield all child nodes

class WDL.Expr.Get(pos: WDL.Error.SourcePosition, expr: WDL.Expr.Base, member: Optional[str])[source]

Bases: WDL.Expr.Base

AST node representing access to a value by identifier (including namespaced ones), or accessing a member of a pair or struct as .member.

The entaglement of these two cases is inherent in WDL. Consider the syntax leftname.midname.rightname. One interpretation is that leftname is an identifier for a struct value, and .midname.rightname represents a chain of struct member accesses. But another possibility is that leftname is a call, midname is a struct output of that call, and rightname is a member of that struct. These cases can’t be distinguished by the syntax parser alone, but must be resolved during typechecking with reference to the calls and identifiers available in the environment.

The typechecker does conveniently resolve such cases, and to minimize the extent to which it has to restructure the AST in doing so, all identifiers (with or without a namespace) are represented as a Get node wrapping an Ident node. The Get node may specify a member name to access, but may not if the identifier is to be accessed directly. On the other hand, the expression inside a Get node need not be a simple identifier, e.g. arr[1].memb.left is be represented as: Get(Get(Apply("_at", Get(Ident("arr")), 1),"memb"),"left")

expr = None
Type:WDL.Expr.Base

The expression whose value is accessed

member = None
Type:Optional[str]

If the expression is accessing a pair/struct member, then expr.type is WDL.Type.Pair or WDL.Type.StructInstance and this field gives the desired member name (left or right for pairs).

Otherwise the expression accesses expr directly, and member is None.

children
Type:Iterable[SourceNode]

Yield all child nodes

class WDL.Expr.Apply(pos: WDL.Error.SourcePosition, function: str, arguments: List[WDL.Expr.Base])[source]

Bases: WDL.Expr.Base

Application of a built-in or standard library function

function_name = None

Name of the function applied

Type:str
arguments = None
Type:List[WDL.Expr.Base]

Expressions for each function argument

children
Type:Iterable[SourceNode]

Yield all child nodes

Env

Environments, for identifier resolution during WDL typechecking and evaluation.

class WDL.Env.Binding(name: str, value: T, info: Any = None)[source]

Bases: typing.Generic

An individual, immutable binding of a possibly-namespaced name to a right-hand-side value of type T. T is typically Value.Base (value environments) or Type.Base (type environments). The binding may also reference an additional informational value of arbitrary type.

name
Type:str

Namedspaced names are flat, dot-separated strings.

value
Type:T
info
Type:Any
class WDL.Env.Bindings(binding: Optional[WDL.Env.Binding[T]] = None, next: Optional[WDL.Env.Bindings[T]] = None)[source]

Bases: typing.Generic

An environment consisting of an immutable linked-list of WDL.Env.Binding objects. WDL.Env.Bindings() is the empty environment. Bindings[T] is iterable for the individual Binding[T] objects:

env = WDL.Env.Bindings()
env = env.bind("x", 1).bind("y", 42)
print(env["x"])                             # 1
print(",".join(str(b.value) for b in env))  # 1,42
bind(name: str, value: T, info: Any = None) → WDL.Env.Bindings[~T][T][source]

Return an environment with a new binding prepended. Any existing binding for the same name is shadowed by the new one. (This should not usually arise in view of the immutability of WDL values.)

resolve_binding(name: str) → WDL.Env.Binding[~T][T][source]

Look up a WDL.Env.Binding object by name

Raises:KeyError – no such binding
resolve(name: str) → T[source]

Look up a bound value by name. Equivalently, env[name]

Raises:KeyError – no such binding
get(name: str, default: Optional[T] = None) → Optional[T][source]

Look up a bound value by name, returning the default value or None if there’s no such binding.

has_binding(name: str) → bool[source]

Determine existence of a binding for the name. Equivalently, name in env

map(f: Callable[[WDL.Env.Binding[~T][T]], Optional[WDL.Env.Binding[~S][S]]]) → WDL.Env.Bindings[~S][S][source]

Copy the environment with each binding transformed by the given function. If the function returns None then the binding is excluded.

filter(pred: Callable[[WDL.Env.Binding[~T][T]], bool]) → WDL.Env.Bindings[~T][T][source]

Copy the environment with only those bindings for which pred returns True

subtract(rhs: WDL.Env.Bindings[~S][S]) → WDL.Env.Bindings[~T][T][source]

Copy the environment excluding any binding for which rhs has a binding with the same name

namespaces
Type:Set[str]

Return the environment’s namespaces, all the distinct dot-separated prefixes of the binding names. Each element ends with a dot.

has_namespace(namespace: str) → bool[source]

Determine existence of a namespace in the environment

enter_namespace(namespace: str) → WDL.Env.Bindings[~T][T][source]

Generate an environment with only those bindings in the given namespace, with the namespace prefix removed from each binding’s name.

wrap_namespace(namespace: str) → WDL.Env.Bindings[~T][T][source]

Copy the environment with the given namespace prefixed to each binding name

WDL.Env.merge(*args) → WDL.Env.Bindings[~T][T][source]

Merge several Bindings[T] environments into one. Should the same name appear in multiple arguments, the first (leftmost) occurrence takes precedence. Otherwise, for efficiency, the largest environment should be supplied last.

Type

WDL data types

WDL has both atomic types such as Int, Boolean, and String; and parametric types like Array[String] and Map[String,Array[Array[Float]]]. Here, each type is represented by an immutable instance of a Python class inheriting from WDL.Type.Base. Such types are associated with expressions, statically prior to evaluation, as well as with values and identifier bindings after evaluation.

An atomic type like Int is represented by WDL.Type.Int(). Atomic types can be checked either with isinstance(t,WDL.Type.Int), which ignores the possible optional quantifier (thus satisfied by Int or Int?), or with t == WDL.Type.Int(optional=True) to include the quantifier in the comparison.

A parametric type like Array[String] is represented by WDL.Type.Array(WDL.Type.String()). Any kind of array satisfies isinstance(t,WDL.Type.Array), and WDL.Type.Array(WDL.Type.String()) == WDL.Type.Array(WDL.Type.String()), but for example WDL.Type.Array(WDL.Type.String()) != WDL.Type.Array(WDL.Type.Float()).

The type classes include a method indicating if a value of the type can be coerced to some other desired type, according to the following rules:

  1. Int coerces to Float
  2. Boolean, Int, Float, and File coerce to String
  3. String coerces to File, Int, and Float
  4. Array[T] coerces to String provided T does as well
  5. T coerces to T? but the reverse is not true in general*
  6. Array[T]+ coerces to Array[T] but the reverse is not true in general*

(*) The reverse coercions are statically permitted in expressions set up with Expr.infer_type(check_quant=False) although they may fail at runtime. This also enables coercion of T to Array[T]+ (an array of length 1).

Inheritance diagram of WDL.Type
class WDL.Type.Base[source]

Bases: abc.ABC

The abstract base class for WDL types

Each specific type inherits from this base, e.g.:

assert issubclass(WDL.Type.Int, WDL.Type.Base)
assert isinstance(WDL.Type.Array(WDL.Type.Int()), WDL.Type.Base)

All instances are immutable.

coerces(rhs: WDL.Type.Base, check_quant: bool = True) → bool[source]

True if this is the same type as, or can be coerced to, rhs.

Parameters:check_quant – when False, disables static enforcement of the optional (?) type quantifier
check(rhs: WDL.Type.Base, check_quant: bool = True) → None[source]

Verify this is the same type as, or can be coerced to rhs. The TypeError exception raised otherwise MAY include a specific error message (but not if the obvious “cannot coerce self to rhs” suffices).

Parameters:check_quant – when False, disables static enforcement of the optional (?) type quantifier
optional
Type:bool

True when the type has the optional quantifier, T?

parameters
Type:Iterable[WDL.Type.Base]

The type’s parameters, if any (e.g. item type of Array; left & right types of Pair; etc.)

copy(self, optional : Optional[bool] = None) → WDL.Type.Base[source]

Create a copy of the type, possibly with a different setting of the optional quantifier.

class WDL.Type.Any(optional: bool = False, null: bool = False)[source]

Bases: WDL.Type.Base

A symbolic type which coerces to any other type; used to represent e.g. the item type of an empty array literal, or the result of read_json().

The optional attribute shall be true only for WDL None literals, which coerce to optional types only.

class WDL.Type.Boolean(optional: bool = False)[source]

Bases: WDL.Type.Base

class WDL.Type.Float(optional: bool = False)[source]

Bases: WDL.Type.Base

class WDL.Type.Int(optional: bool = False)[source]

Bases: WDL.Type.Base

class WDL.Type.File(optional: bool = False)[source]

Bases: WDL.Type.Base

class WDL.Type.Directory(optional: bool = False)[source]

Bases: WDL.Type.Base

class WDL.Type.String(optional: bool = False)[source]

Bases: WDL.Type.Base

class WDL.Type.Array(item_type: WDL.Type.Base, optional: bool = False, nonempty: bool = False)[source]

Bases: WDL.Type.Base

Array type, parameterized by the type of the constituent items.

item_type = None
Type:WDL.Type.Base

item_type may be Any when not known statically, such as in a literal empty array [].

nonempty
Type:bool

True when the type has the nonempty quantifier, Array[T]+

parameters
Type:Iterable[WDL.Type.Base]

The type’s parameters, if any (e.g. item type of Array; left & right types of Pair; etc.)

copy(self, optional : Optional[bool] = None) → WDL.Type.Base[source]

Create a copy of the type, possibly with a different setting of the optional quantifier.

class WDL.Type.Map(item_type: Tuple[WDL.Type.Base, WDL.Type.Base], optional: bool = False, literal_keys: Optional[Set[str]] = None)[source]

Bases: WDL.Type.Base

Map type, parameterized by the (key,value) item type.

item_type = None
Type:Tuple[WDL.Type.Base,WDL.Type.Base]

The key and value types may be Any when not known statically, such as in a literal empty map {}.

literal_keys = None
parameters
Type:Iterable[WDL.Type.Base]

The type’s parameters, if any (e.g. item type of Array; left & right types of Pair; etc.)

class WDL.Type.Pair(left_type: WDL.Type.Base, right_type: WDL.Type.Base, optional: bool = False)[source]

Bases: WDL.Type.Base

Pair type, parameterized by the left and right item types.

left_type = None
Type:WDL.Type.Base
right_type = None
Type:WDL.Type.Base
parameters
Type:Iterable[WDL.Type.Base]

The type’s parameters, if any (e.g. item type of Array; left & right types of Pair; etc.)

class WDL.Type.StructInstance(type_name: str, optional: bool = False)[source]

Bases: WDL.Type.Base

Type of an instance of a struct

Not to be confused with struct type definition, WDL.Tree.StructTypeDef. To find the WDL.Tree.StructTypeDef in the current doc: WDL.Tree.Document corresponding to ty: WDL.Type.StructInstance, use doc.struct_typedefs[ty.type_name].

type_name = None
Type:str

The struct type name with which the instance is declared; note that the same struct type can go by different names.

members = None
Type:Dict[str,WDL.Type.Base]

Names and types of the struct members, from the struct type definition (available after typechecking)

type_id
Type:str

A string canonically describing the member names and their types, excluding the struct type name; useful to unify aliased struct types.

parameters
Type:Iterable[WDL.Type.Base]

The type’s parameters, if any (e.g. item type of Array; left & right types of Pair; etc.)

WDL.Type.unify(types: List[WDL.Type.Base], check_quant: bool = True, force_string: bool = False) → WDL.Type.Base[source]

Given a list of types, compute a type to which they’re all coercible, or WDL.Type.Any if no more-specific inference is possible.

Parameters:force_string – permit last-resort unification to String even if no item is currently a String, but all can be coerced

Value

WDL values instantiated at runtime

Each value is represented by an instance of a Python class inheriting from WDL.Value.Base.

Inheritance diagram of WDL.Value
class WDL.Value.Base(type: WDL.Type.Base, value: Any, expr: Optional[Expr.Base] = None)[source]

Bases: abc.ABC

The abstract base class for WDL values

type = None
Type:WDL.Type.Base
value = None

The “raw” Python value

expr

Reference to the WDL expression that generated this value, if it originated from WDL.Expr.eval

coerce(desired_type: Optional[WDL.Type.Base] = None) → WDL.Value.Base[source]

Coerce the value to the desired type and return it. Types should be checked statically on WDL.Expr.Base prior to evaluation.

Raises:ReferenceError for a null value and non-optional type
expect(desired_type: Optional[WDL.Type.Base] = None) → WDL.Value.Base[source]

Alias for coerce

json

Return a value representation which can be serialized to JSON using json.dumps

class WDL.Value.Boolean(value: bool, expr: Optional[Expr.Base] = None)[source]

Bases: WDL.Value.Base

value has Python type bool

class WDL.Value.Float(value: float, expr: Optional[Expr.Base] = None)[source]

Bases: WDL.Value.Base

value has Python type float

class WDL.Value.Int(value: int, expr: Optional[Expr.Base] = None)[source]

Bases: WDL.Value.Base

value has Python type int

class WDL.Value.String(value: str, expr: Optional[Expr.Base] = None, subtype: Optional[WDL.Type.Base] = None)[source]

Bases: WDL.Value.Base

value has Python type str

class WDL.Value.File(value: str, expr: Optional[Expr.Base] = None)[source]

Bases: WDL.Value.String

value has Python type str

class WDL.Value.Directory(value: str, expr: Optional[Expr.Base] = None)[source]

Bases: WDL.Value.String

value has Python type str

class WDL.Value.Array(item_type: WDL.Type.Base, value: List[WDL.Value.Base], expr: Optional[Expr.Base] = None)[source]

Bases: WDL.Value.Base

value is a Python list of other WDL.Value.Base instances

class WDL.Value.Map(item_type: Tuple[WDL.Type.Base, WDL.Type.Base], value: List[Tuple[WDL.Value.Base, WDL.Value.Base]], expr: Optional[Expr.Base] = None)[source]

Bases: WDL.Value.Base

class WDL.Value.Pair(left_type: WDL.Type.Base, right_type: WDL.Type.Base, value: Tuple[WDL.Value.Base, WDL.Value.Base], expr: Optional[Expr.Base] = None)[source]

Bases: WDL.Value.Base

class WDL.Value.Null(expr: Optional[Expr.Base] = None)[source]

Bases: WDL.Value.Base

Represents the missing value which optional inputs may take. type and value are both None.

class WDL.Value.Struct(type: Union[WDL.Type.Object, WDL.Type.StructInstance], value: Dict[str, WDL.Value.Base], expr: Optional[Expr.Base] = None, extra: Optional[Set[str]] = None)[source]

Bases: WDL.Value.Base

WDL.Value.from_json(type: WDL.Type.Base, value: Any) → WDL.Value.Base[source]

Instantiate a WDL value of the specified type from a parsed JSON value (str, int, float, list, dict, or null).

If type is WDL.Type.Any(), attempts to infer a WDL type & value from the JSON’s intrinsic types. This isn’t ideal; for example, Files can’t be distinguished from Strings, and JSON lists and dicts with heterogeneous item types may give undefined results.

Raises:WDL.Error.InputError – if the given value isn’t coercible to the specified type
WDL.Value.rewrite_paths(v: WDL.Value.Base, f: Callable[[Union[WDL.Value.File, WDL.Value.Directory]], Optional[str]]) → WDL.Value.Base[source]

Produce a deep copy of the given Value with all File & Directory paths (including those nested inside compound Values) rewritten by the given function. The function may return None to replace the File/Directory value with None/Null.

WDL.Value.rewrite_env_paths(env: WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base], f: Callable[[Union[WDL.Value.File, WDL.Value.Directory]], Optional[str]]) → WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base][source]

Produce a deep copy of the given Value Env with all File & Directory paths rewritten by the given function.

WDL.Value.rewrite_files(v: WDL.Value.Base, f: Callable[[str], Optional[str]]) → WDL.Value.Base[source]

Produce a deep copy of the given Value with all File names rewritten by the given function (including Files nested inside compound Values).

(deprecated: use rewrite_paths to handle Directory values as well)

WDL.Value.rewrite_env_files(env: WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base], f: Callable[[str], Optional[str]]) → WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base][source]

Produce a deep copy of the given Value Env with all File names rewritten by the given function.

(deprecated: use rewrite_env_paths to handle Directory values as well)

WDL.Value.digest_env(env: WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base]) → str[source]

Digest the Value Env, for use e.g. as a cache key. The digest is an opaque string of a few dozen alphanumeric characters.

Error

class WDL.Error.SourcePosition[source]

Bases: WDL.Error.SourcePosition

Source position attached to AST nodes and exceptions; NamedTuple of uri the filename/URI passed to WDL.load() or a WDL import statement, which may be relative; abspath the absolute filename/URI; and one-based int positions line end_line column end_column

exception WDL.Error.SyntaxError(pos: WDL.Error.SourcePosition, msg: str, wdl_version: str, declared_wdl_version: Optional[str])[source]

Bases: Exception

Failure to lex/parse a WDL document

exception WDL.Error.ImportError(pos: WDL.Error.SourcePosition, import_uri: str, message: Optional[str] = None)[source]

Bases: Exception

Failure to open/retrieve an imported WDL document

The __cause__ attribute may hold the inner error object.

class WDL.Error.SourceNode(pos: WDL.Error.SourcePosition)[source]

Bases: object

Base class for an AST node, recording the source position

pos = None
Type:SourcePosition

Source position for this AST node

children
Type:Iterable[SourceNode]

Yield all child nodes

exception WDL.Error.ValidationError(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]

Bases: Exception

Base class for a WDL validation error (when the document loads and parses, but fails typechecking or other static validity tests)

source_text = None
Type:Optional[str]

The complete source text of the WDL document (if available)

node = None
Type:Optional[SourceNode]
pos = None
Type:SourcePosition
exception WDL.Error.InvalidType(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.IndeterminateType(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.NoSuchTask(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], name: str)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.NoSuchCall(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], name: str)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.NoSuchFunction(node: WDL.Error.SourceNode, name: str)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.WrongArity(node: WDL.Error.SourceNode, expected: int)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.NotAnArray(node: WDL.Error.SourceNode)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.NoSuchMember(node: WDL.Error.SourceNode, member: str)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.StaticTypeMismatch(node: WDL.Error.SourceNode, expected: WDL.Type.Base, actual: WDL.Type.Base, message: str = '')[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.IncompatibleOperand(node: WDL.Error.SourceNode, message: str)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.UnknownIdentifier(node: WDL.Error.SourceNode, message: Optional[str] = None)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.NoSuchInput(node: WDL.Error.SourceNode, name: str)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.UncallableWorkflow(node: WDL.Error.SourceNode, name: str)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.MultipleDefinitions(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.StrayInputDeclaration(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.CircularDependencies(node: WDL.Error.SourceNode)[source]

Bases: WDL.Error.ValidationError

exception WDL.Error.MultipleValidationErrors(*exceptions)[source]

Bases: Exception

Propagates several validation/typechecking errors

exceptions = None
Type:List[ValidationError]
exception WDL.Error.RuntimeError(*args, more_info: Optional[Dict[str, Any]] = None, **kwargs)[source]

Bases: Exception

more_info = None

Backend-specific information about an error (for example, pointer to a centralized log system)

exception WDL.Error.EvalError(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]

Bases: WDL.Error.RuntimeError

Error evaluating a WDL expression or declaration

node = None
Type:Optional[SourceNode]
pos = None
Type:SourcePosition
exception WDL.Error.OutOfBounds(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]

Bases: WDL.Error.EvalError

exception WDL.Error.EmptyArray(node: WDL.Error.SourceNode)[source]

Bases: WDL.Error.EvalError

exception WDL.Error.NullValue(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition])[source]

Bases: WDL.Error.EvalError

exception WDL.Error.InputError(*args, more_info: Optional[Dict[str, Any]] = None, **kwargs)[source]

Bases: WDL.Error.RuntimeError

Error reading an input value/file

Lint

Annotate WDL document AST with hygiene warnings (underlies miniwdl check)

Given a doc: WDL.Document, the lint warnings can be retrieved like so:

import WDL
import WDL.Lint

lint = WDL.Lint.collect(WDL.Lint.lint(doc, descend_imports=False))
for (pos, lint_class, message, suppressed) in lint:
    assert isinstance(pos, WDL.SourcePosition)
    assert isinstance(lint_class, str) and isinstance(message, str)
    if not suppressed:
        print(json.dumps({
            "uri"        : pos.uri,
            "abspath"    : pos.abspath,
            "line"       : pos.line,
            "end_line"   : pos.end_line,
            "column"     : pos.column,
            "end_column" : pos.end_column,
            "lint"       : lint_class,
            "message"    : message,
        }))

The descend_imports flag controls whether lint warnings are generated for imported documents recursively (true, default), or otherwise only the given document (false).

Zip

Routines for packaging a WDL source file, with all imported source files, into a ZIP file.

New in v1.5.0

WDL.Zip.build(top_doc: WDL.Tree.Document, archive: str, logger: logging.Logger, inputs: Optional[Dict[str, Any]] = None, meta: Optional[Dict[str, Any]] = None, archive_format: str = 'zip', additional_files: Optional[List[str]] = None)[source]

Generate zip archive of the WDL document, all its imports, optional default inputs, and a generated manifest JSON.

If imports are drawn from outside the main WDL’s directory (or by URI), they’ll be stored in a special subdirectory and import statements will be rewritten to match.

class WDL.Zip.UnpackedZip(dir, main_wdl, input_file)

Contextual value of WDL.Zip.unpack(): absolute paths of source directory, main WDL, and default input JSON file (if any). The source directory prefixes the latter paths.

dir

Alias for field number 0

input_file

Alias for field number 2

main_wdl

Alias for field number 1

WDL.Zip.unpack(archive_fn: str) → Iterator[WDL.Zip.UnpackedZip][source]

Open a context with the WDL source archive unpacked into a temp directory, yielding UnpackedZip. The temp directory will be deleted on context exit.

A path to the MANIFEST.json of an already-unpacked source archive may also be used, or a directory containing one. In this case, it is NOT deleted on context exit.

with WDL.Zip.unpack("/path/to/source.zip") as unpacked:
    doc = WDL.load(unpacked.main_wdl)
    ...

runtime

The recommended way to run a WDL workflow programmatically is to invoke miniwdl run as a subprocess, capturing its JSON standard output. This leverages its logging, configuration, and flexible input loading features; and avoids conflicting with the runtime’s thread pools and signal handlers. Alternatively, it’s possible to call WDL.runtime.run() directly if needed.

WDL.runtime.run(cfg: WDL.runtime.config.Loader, exe: Union[WDL.Tree.Task, WDL.Tree.Workflow], inputs: WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base], **run_kwargs) → Tuple[str, WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base]][source]

Run the task or workflow given the inputs environment and configuration, returning the outputs environment. inputs may be parsed from a JSON dict using values_from_json(), which can also validate them; see example below.

Parameters:
  • run_id – a run identifier used in logs and filenames; defaults to executable name
  • run_dir – directory under which to create a timestamp-named subdirectory for this run (defaults to current working directory). If the final path component is . then operate in run_dir directly.

Typical usage:

import WDL
import WDL.runtime

# Convert JSON-like inputs dict to WDL environment, validating them against exe's available
# and required inputs. The dict keys should NOT be namespaced by the executable name;
# if namespaces are present, then add namespace=exe.name to effectively remove them.
inputs_env = WDL.values_from_json(inputs_dict, exe.available_inputs, exe.required_inputs)
# Load configuration (see below)
cfg = WDL.runtime.config.Loader(logging.getLogger(__name__))
# Run executable
run_subdir, outputs_env = WDL.runtime.run(cfg, exe, inputs_env, run_dir="/tmp")
# Generate JSON-like outputs dict, with keys namespaced by the executable name
outputs_dict = WDL.values_to_json(outputs_env, exe.name)
exception WDL.runtime.config.ConfigMissing[source]
class WDL.runtime.config.Loader(logger: logging.Logger, filenames: Optional[List[str]] = None, overrides: Optional[Dict[str, Dict[str, str]]] = None)[source]

Runtime configuration options, identified by section & key, are sourced in the following priority order:

  1. Supplied overrides dict, {"section": {"key": value}}
  2. Environment variables MINIWDL__SECTION__KEY (uppercased with double-underscores)
  3. Custom configuration file (mutually exclusive):
    1. filename given to __init__
    2. file named by environment variable MINIWDL_CFG
    3. miniwdl.cfg in XDG_CONFIG_HOME or XDG_CONFIG_DIRS, usually ~/.config/miniwdl.cfg
  4. WDL/runtime/config_templates/default.cfg from installed package

If filenames is an empty list, then no custom configuration file is used. Or filenames may be a prioritized list of candidate filenames instead of (a-c) above; the first extant file is used, if any.

plugin_defaults(options: Dict[str, Dict[str, Any]]) → None[source]

Last-priority default settings; typically added by plugins (whose options won’t be represented in default.cfg). (DEPRECATED)

log_all()[source]

Write a debug log message with all options

log_unused_options()[source]

After a run, log warnings about any options that were set, but were never accessed and don’t correspond to any default option.