Python WDL
package¶
miniwdl is a developer toolkit and local runner for
the bioinformatics-focused Workflow Description Language (WDL). This
documentation covers the Python3 WDL
package facilitating parsing & static analysis of WDL
documents. Simply import WDL
once miniwdl has been installed.
- GitHub repo for installation and further background
- Codelabs on using this package
-
WDL.
load
(uri: str, path: Optional[List[str]] = None, check_quant: bool = True, read_source: Optional[Callable[[str, List[str], Optional[WDL.Tree.Document]], Awaitable[ReadSourceResult]]] = None, import_max_depth: int = 10) → WDL.Tree.Document[source]¶ Parse a WDL document given filename/URI, recursively descend into imported documents, then typecheck the tasks and workflow.
Parameters: - path – local filesystem directories to search for imports, in addition to the current working directory
- check_quant – set to
False
to relax static typechecking of the optional (?) and nonempty (+) type quantifiers. This is discouraged, but may be useful for older WDL workflows which assume less-rigorous static validation of these annotations. - read_source – async routine to read the WDL source code from filename/URI; see
read_source_default()
below for details - import_max_depth – to prevent recursive import infinite loops, fail when there are too many import nesting levels (default 10)
Raises: - WDL.Error.SyntaxError – when the document is syntactically invalid under the WDL grammar
- WDL.Error.ValidationError – when the document is syntactically OK, but fails typechecking or other static validity checks
- WDL.Error.MultipleValidationErrors – when multiple validation errors are detected in one pass, listed in the
exceptions
attribute - WDL.Error.ImportError – when an imported sub-document can’t be loaded; the
__cause__
attribute has the specific error
-
class
WDL.
ReadSourceResult
[source]¶ The
NamedTuple
to be returned by theread_source
routine. Itssource_text: str
field provides the WDL source code, and theabspath: str
field is the absolute filename/URI from which the source was read (e.g. after resolving a relative path).
-
WDL.
values_from_json
(values_json: Dict[str, Any], available: WDL.Env.Bindings[typing.Union[WDL.Tree.Decl, WDL.Type.Base]][Union[WDL.Tree.Decl, WDL.Type.Base]], required: Optional[WDL.Env.Bindings[typing.Union[WDL.Tree.Decl, WDL.Type.Base]][Union[WDL.Tree.Decl, WDL.Type.Base]]] = None, namespace: str = '') → WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base][source]¶ Given a dict parsed from Cromwell-style JSON and the available input (or output) declarations of a task or workflow, create a
WDL.Env.Bindings[Value.Base]
.Parameters: - required – raise an error if any of these required inputs aren’t present
- namespace – expect each key to start with this namespace prefixed to the input/output names (e.g. the workflow name)
-
WDL.
values_to_json
(values_env: WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base], namespace: str = '') → Dict[str, Any][source]¶ Convert a
WDL.Env.Bindings[WDL.Value.Base]
to a dict whichjson.dumps
to Cromwell-style JSON.Parameters: namespace – prefix this namespace to each key (e.g. workflow name)
-
WDL.
load_async
(uri: str, path: Optional[List[str]] = None, check_quant: bool = True, read_source: Optional[Callable[[str, List[str], Optional[WDL.Tree.Document]], Awaitable[ReadSourceResult]]] = None, import_max_depth: int = 10) → WDL.Tree.Document[source]¶ Async version of
load()
, with all the same arguments
-
WDL.
read_source_default
(uri: str, path: List[str], importer: Optional[WDL.Tree.Document]) → WDL.ReadSourceResult[source]¶ Default async routine for the
read_source
parameter toload()
andload_async()
, which they use to read the desired WDL document and its imports. This default routine handles local files only, supplying the search path logic to resolve relative filenames; it fails with network URIs.Parameters: - uri – Filename/URI to read, as provided to
load()
or the WDL import statement; may be relative - path – Local directories to search for relative imports
- importer – The document importing the one here requested, if any; the
importer.pos.uri
andimporter.pos.abspath
fields may be relevant to resolve relative imports.
Returns: ReadSourceResult(source_text="...", abspath="...")
Callers may wish to override
read_source
with logic to download source code from network URIs, and for local filenames fall back toreturn await WDL.read_source_default(...)
.Note: the synchronous
load()
merely callsload_async()
on the currentasyncio.get_event_loop()
and awaits the result.- uri – Filename/URI to read, as provided to
-
WDL.
resolve_file_import
(uri: str, path: List[str], importer: Optional[WDL.Tree.Document]) → str[source]¶ Exposes the logic by which
read_source_default()
resolvesuri
to the absolute path of an extant file. Ifuri
is already an absolute path, it’s normalized and returned. A relativeuri
is resolved by first joining it to either, the directory of the importer document (if any), or the process current working directory (otherwise). Failing that, it’s searched in thepath
directories (in reverse order).Security-focused applications may wish to override
read_source
with logic to restrict allowable results ofresolve_file_import
, to prevent WDL source code from triggering access to arbitrary filesystem paths. No such restrictions are applied by default.
Tree¶
Abstract syntax tree (AST) for WDL documents, containing tasks and workflows, which contain
declarations, calls, and scatter & if sections. The AST is typically constructed by
load()
.
The WDL.Tree.*
classes are also exported by the base WDL
module, i.e. WDL.Tree.Document
can be abbreviated WDL.Document
.

-
class
WDL.Tree.
StructTypeDef
(pos: WDL.Error.SourcePosition, name: str, members: Dict[str, WDL.Type.Base], imported: Optional[Tuple[WDL.Tree.Document, WDL.Tree.StructTypeDef]] = None)[source]¶ Bases:
WDL.Error.SourceNode
WDL struct type definition
-
name
= None¶ Type: str Name of the struct type (in the current document)
-
members
= None¶ Type: Dict[str, WDL.Type.Base] Member names and types
-
imported
= None¶ Type: Optional[Tuple[Document,StructTypeDef]] If this struct is imported from another document, references that document and its definition there. The referenced definition might itself be imported from yet another document.
-
type_id
¶ Type: str A string canonically describing the member names and their types, excluding the struct type name; useful to unify aliased struct types.
-
-
class
WDL.Tree.
WorkflowNode
(workflow_node_id: str, pos: WDL.Error.SourcePosition)[source]¶ Bases:
WDL.Error.SourceNode
,abc.ABC
Base class for workflow “nodes” including declarations, calls, and scatter/if sections and their bodies.
Each node has a human-readable ID string which is unique within the workflow. It also exposes the set of workflow node IDs upon which it depends. Abstractly, workflow execution can proceed by “visiting” each node once all of its dependencies have been visited, performing some action(s) appropriate to the specific node type (such as evaluating a WDL expression and binding a name in the environment, or executing a task and binding its outputs).
-
workflow_node_id
= None¶ Type: str Human-readable node ID unique within the current workflow
-
scatter_depth
= None¶ Type: int How many nested scatter sections the node lies within. This information is useful for runtime dependency analysis in workflows with scatters. When scatter sections are nested within conditional sections or vice versa, this counts the scatters only.
-
workflow_node_dependencies
¶ Type: Set[str] Set of workflow node IDs on which this node depends. Available once workflow has been typechecked.
-
-
class
WDL.Tree.
Decl
(pos: WDL.Error.SourcePosition, type: WDL.Type.Base, name: str, expr: Optional[WDL.Expr.Base] = None, id_prefix='decl')[source]¶ Bases:
WDL.Tree.WorkflowNode
A value declaration within a task or workflow.
Within a task, the declarations can be viewed as “workflow nodes” insofar as they must be evaluated in an order consistent with their dependency structure, and ensured acyclic. The “workflow node IDs” of a task’s declarations are unique within the task only, and unrelated to the top-level workflow, if any, in the WDL document.
-
type
= None¶ Type: WDL.Type.Base
-
name
= None¶ Declared value name
Type: str
-
expr
= None¶ Type: Optional[WDL.Expr.Base] Bound expression, if any
-
decor
= None¶
-
-
class
WDL.Tree.
Task
(pos: WDL.Error.SourcePosition, name: str, inputs: Optional[List[WDL.Tree.Decl]], postinputs: List[WDL.Tree.Decl], command: WDL.Expr.String, outputs: List[WDL.Tree.Decl], parameter_meta: Dict[str, Any], runtime: Dict[str, WDL.Expr.Base], meta: Dict[str, Any])[source]¶ Bases:
WDL.Error.SourceNode
WDL Task
-
name
= None¶ Type: str
-
inputs
= None¶ Type: Optional[List[WDL.Tree.Decl]] Declarations in the
input{}
task section, if it’s present
-
postinputs
= None¶ Type: List[WDL.Tree.Decl] Declarations outside of the
input{}
task section
-
command
= None¶ Type: WDL.Expr.String
-
outputs
= None¶ Type: List[WDL.Tree.Decl] Output declarations
-
runtime
= None¶ Type: Dict[str,WDL.Expr.Base] runtime{}
section, with keys and corresponding expressions to be evaluated
-
effective_wdl_version
= None¶ Type: str Effective WDL version of the containing document
-
available_inputs
¶ Type: WDL.Env.Bindings[WDL.Tree.Decl] Yields the task’s input declarations. This is all declarations in the task’s
input{}
section, if it’s present. Otherwise, it’s all declarations in the task, excluding outputs. (This dichotomy bridges pre-1.0 and 1.0+ WDL versions.)Each input is at the top level of the Env, with no namespace.
-
required_inputs
¶ Type: WDL.Env.Bindings[WDL.Tree.Decl] Yields the input declarations which are required to call the task (available inputs that are unbound and non-optional).
Each input is at the top level of the Env, with no namespace.
-
effective_outputs
¶ Type: WDL.Env.Bindings[Type.Base] Yields each task output with its type, at the top level of the Env with no namespace. (Present for isomorphism with
Workflow.effective_outputs
)
-
digest
¶ Content digest of the task, for use e.g. as a cache key. The digest is an opaque string of a few dozen alphanumeric characters, sensitive to the task’s source code (with best effort to exclude comments and whitespace).
-
-
class
WDL.Tree.
Call
(pos: WDL.Error.SourcePosition, callee_id: List[str], alias: Optional[str], inputs: Dict[str, WDL.Expr.Base], after: Optional[List[str]] = None)[source]¶ Bases:
WDL.Tree.WorkflowNode
A call (within a workflow) to a task or sub-workflow
-
callee_id
= None¶ Type: List[str] The called task; either one string naming a task in the current document, or an import namespace and task name.
-
name
= None¶ Type: string Call name, defaults to task/workflow name
-
inputs
= None¶ Type: Dict[str,WDL.Expr.Base] Call inputs provided
-
callee
= None¶ Type: Union[WDL.Tree.Task, WDL.Tree.Workflow] Refers to the
Task
or importedWorkflow
object to be called (after AST typechecking)
-
after
= None¶ Type: string Call names on which this call depends (even if none of their outputs are used in this call’s inputs)
-
available_inputs
¶ Type: WDL.Env.Bindings[WDL.Tree.Decl] Yields the task/workflow inputs which are not supplied in the call
inputs:
, and thus may be supplied at workflow launch; in namespaces according to the call names.
-
required_inputs
¶ Type: WDL.Env.Bindings[WDL.Tree.Decl] Yields the required task/workflow inputs which are not supplied in the call
inputs:
(incomplete calls), and thus must be supplied at workflow launch; in namespaces according to the call name.
-
effective_outputs
¶ Type: WDL.Env.Bindings[WDL.Tree.Decl] Yields the effective outputs of the callee Task or Workflow, in a namespace according to the call name.
-
-
class
WDL.Tree.
Gather
(section: WDL.Tree.WorkflowSection, referee: Union[WDL.Tree.Decl, WDL.Tree.Call, WDL.Tree.Gather])[source]¶ Bases:
WDL.Tree.WorkflowNode
A
Gather
node symbolizes the operation to gather an array of declared values or call outputs in a scatter section, or optional values from a conditional section. These operations are implicit in the WDL syntax, but explicating them in the AST facilitates analysis of the workflow’s data types and dependency structure.Each scatter/conditional section provides
Gather
nodes to expose the section body’s products to the rest of the workflow. When aWDL.Expr.Ident
elsewhere identifies a node inside the section, itsreferee
attribute is the correspondingGather
node, which in turn references the interior node. The interior node might itself be anotherGather
node, from a nested scatter/conditional section.-
section
= None¶ Type: WorkflowSection The
Scatter
/Conditional
section implying this Gather operation
-
referee
= None¶ Type: Union[Decl, Call, Gather] The
Decl
,Call
, or sub-Gather
node from which this operation “gathers”
-
final_referee
¶ The
Decl
orCall
node found at the end of the referee chain through any nestedGather
nodes
-
-
class
WDL.Tree.
WorkflowSection
(body: List[WDL.Tree.WorkflowNode], *args, **kwargs)[source]¶ Bases:
WDL.Tree.WorkflowNode
Base class for workflow nodes representing scatter and conditional sections
-
body
= None¶ Type: List[WorkflowNode] Section body, potentially including nested sections.
-
gathers
= None¶ Type: Dict[str, Gather] Gather
nodes exposing the section body’s products to the rest of the workflow. The dict is keyed byworkflow_node_id
of the interior node, to expedite looking up the corresponding gather node.The section’s body and gather nodes do not explicitly include the section node among their dependencies. Such dependence is implicit because the body subgraph can be “instantiated” only upon visiting the section node at runtime.
-
-
class
WDL.Tree.
Scatter
(pos: WDL.Error.SourcePosition, variable: str, expr: WDL.Expr.Base, body: List[WDL.Tree.WorkflowNode])[source]¶ Bases:
WDL.Tree.WorkflowSection
Workflow scatter section
-
variable
= None¶ Type: string Scatter variable name
-
expr
= None¶ Type: WDL.Expr.Base Expression for the array over which to scatter
-
-
class
WDL.Tree.
Conditional
(pos: WDL.Error.SourcePosition, expr: WDL.Expr.Base, body: List[WDL.Tree.WorkflowNode])[source]¶ Bases:
WDL.Tree.WorkflowSection
Workflow conditional (if) section
-
expr
= None¶ Tree: WDL.Expr.Base Boolean expression
-
-
class
WDL.Tree.
Workflow
(pos: WDL.Error.SourcePosition, name: str, inputs: Optional[List[WDL.Tree.Decl]], body: List[WDL.Tree.WorkflowNode], outputs: Optional[List[WDL.Tree.Decl]], parameter_meta: Dict[str, Any], meta: Dict[str, Any], output_idents: Optional[List[List[str]]] = None, output_idents_pos: Optional[WDL.Error.SourcePosition] = None)[source]¶ Bases:
WDL.Error.SourceNode
-
name
= None¶ Type: str
-
inputs
= None¶ Type: List[WDL.Tree.Decl] Declarations in the
input{}
workflow section, if it’s present
-
body
= None¶ Type: List[Union[WDL.Tree.Decl,WDL.Tree.Call,WDL.Tree.Scatter,WDL.Tree.Conditional]] Workflow body in between
input{}
andoutput{}
sections, if any
-
outputs
= None¶ Type: Optional[List[WDL.Tree.Decl]] Workflow output declarations, if the
output{}
section is present
-
complete_calls
= None¶ After typechecking, False if the workflow has a call which does not supply all required inputs (and thus cannot be called from another workflow).
-
effective_wdl_version
= None¶ Type: str Effective WDL version of the containing document
-
available_inputs
¶ Type: WDL.Env.Bindings[WDL.Tree.Decl] The workflow’s input declarations. This includes:
1. If the
input{}
workflow section is present, all declarations within that section. Otherwise, all declarations in the top-level workflow body, excluding outputs. (This dichotomy bridges pre-1.0 and 1.0+ WDL versions.) These appear at the top level of the Env, with no namespace.- Available inputs of all calls in the workflow, namespaced by the call names.
-
required_inputs
¶ Type: WDL.Env.Bindings[Decl] The subset of available inputs which are required to start the workflow.
-
effective_outputs
¶ Type: WDL.Env.Bindings[Decl] If the
output{}
workflow section is present, yields the names and types therein, at the top level of the Env. Otherwise, yield all the call outputs, namespaced and typed appropriately.
-
get_node
(workflow_node_id: str) → WDL.Tree.WorkflowNode[source]¶ Look up
WorkflowNode
byworkflow_node_id
-
digest
¶ Content digest of the workflow, for use e.g. as a cache key. The digest is an opaque string of a few dozen alphanumeric characters, sensitive to the workflow’s source code (with best effort to exclude comments and whitespace) and the tasks and subworkflows it calls.
-
-
class
WDL.Tree.
SourceComment
(pos, text)¶ Bases:
tuple
Position and text of a comment. The text includes the
#
and any preceding or trailing spaces/tabs.-
pos
¶ Alias for field number 0
-
text
¶ Alias for field number 1
-
-
class
WDL.Tree.
DocImport
(pos, uri, namespace, aliases, doc)¶ Bases:
tuple
Represents one imported document, with position of the import statement, import URI, namespace, struct type aliases, and (after typechecking) the
Document
object.-
aliases
¶ Alias for field number 3
-
doc
¶ Alias for field number 4
-
namespace
¶ Alias for field number 2
-
pos
¶ Alias for field number 0
-
uri
¶ Alias for field number 1
-
-
class
WDL.Tree.
Document
(source_text: str, pos: WDL.Error.SourcePosition, imports: List[WDL.Tree.DocImport], struct_typedefs: Dict[str, WDL.Tree.StructTypeDef], tasks: List[WDL.Tree.Task], workflow: Optional[WDL.Tree.Workflow], comments: List[WDL.Tree.SourceComment], wdl_version: Optional[str])[source]¶ Bases:
WDL.Error.SourceNode
Top-level document, with imports, tasks, and up to one workflow. Typically returned by
load()
.-
struct_typedefs
= None¶ Type: Env.Bindings[WDL.Tree.StructTypeDef]
-
tasks
= None¶ Type: List[WDL.Tree.Task]
-
workflow
= None¶ Type: Optional[WDL.Tree.Workflow]
-
source_text
= None¶ Type: str Original WDL source code text
-
source_lines
= None¶ Type: List[str] Original WDL source code text split by newlines.
SourcePosition
line numbers are one-based, so line numberL
corresponds tosource_lines[L-1]
.
-
source_comments
= None¶ Type: List[Optional[SourceComment]] Lookup table for source code comments.
source_comments
has the same length assource_lines
, and each entry is theWDL.Tree.SourceComment
found on the corresponding source line, orNone
if the line has no comment.
-
wdl_version
= None¶ Type: Optional[str] Declared WDL language version, if any
-
effective_wdl_version
= None¶ :type”
wdl_version if wdl_version is not None else "draft-2"
-
Expr¶
WDL expressions composing literal values, arithmetic, comparison, conditionals, string interpolation, arrays & maps, and function applications. These appear on the right-hand side of value declarations and in task command substitutions, task runtime sections, and workflow scatter and conditional sections.
The abstract syntax tree (AST) for any expression is represented by an instance
of a Python class deriving from WDL.Expr.Base
. Any such node may have other
nodes attached “beneath” it. An expression can be evaluated to a Value
given a suitable WDL.Env.Bindings[Value.Base]
.

-
class
WDL.Expr.
Base
(pos: WDL.Error.SourcePosition)[source]¶ Bases:
WDL.Error.SourceNode
,abc.ABC
Superclass of all expression AST nodes
-
type
¶ Type: WDL.Type.Base WDL type of this expression. Undefined on construction; populated by one invocation of
infer_type
.
-
infer_type
(self, type_env : Env.Bindings[Type.Base], stdlib : StdLib.Base) → WDL.Expr.Base[source]¶ Infer the expression’s type within the given type environment. Must be invoked exactly once prior to use of other methods.
Parameters: - stdlib – a context-specific standard function library for typechecking
- check_quant – when
False
, disables static validation of the optional (?) type quantifier when typecheck() is called on this expression, so for example typeT?
can satisfy an expected typeT
. Applies recursively to the type inference and checking of any sub-expressions.
Raises: WDL.Error.StaticTypeMismatch – when the expression fails to type-check
Returns: self
-
typecheck
(self, expected : Type.Base) → WDL.Expr.Base[source]¶ Check that this expression’s type is, or can be coerced to,
expected
.Raises: WDL.Error.StaticTypeMismatch – Returns: self
-
eval
(env: WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base], stdlib: WDL.StdLib.Base) → WDL.Value.Base[source]¶ Evaluate the expression in the given environment
Parameters: stdlib – a context-specific standard function library implementation
-
literal
¶ If the expression is a literal constant, return its value; otherwise return None. The result can be an instance of
WDL.Value.Null
which is distinct from None.
-
-
class
WDL.Expr.
Boolean
(pos: WDL.Error.SourcePosition, literal: bool)[source]¶ Bases:
WDL.Expr.Base
Boolean literal
-
value
= None¶ Type: bool Literal value
-
-
class
WDL.Expr.
Int
(pos: WDL.Error.SourcePosition, literal: int)[source]¶ Bases:
WDL.Expr.Base
Integer literal
-
value
= None¶ Type: int Literal value
-
-
class
WDL.Expr.
Float
(pos: WDL.Error.SourcePosition, literal: float)[source]¶ Bases:
WDL.Expr.Base
Numeric literal
-
value
= None¶ Type: float Literal value
-
-
class
WDL.Expr.
Null
(pos: WDL.Error.SourcePosition)[source]¶ Bases:
WDL.Expr.Base
WDL
None
literal(called
Null
to avoid conflict with PythonNone
)-
value
= None¶ Type: None
-
-
class
WDL.Expr.
Placeholder
(pos: WDL.Error.SourcePosition, options: Dict[str, str], expr: WDL.Expr.Base)[source]¶ Bases:
WDL.Expr.Base
Holds an expression interpolated within a string or command
-
options
= None¶ Type: Dict[str,str] Placeholder options (sep, true, false, default)
-
expr
= None¶ Type: WDL.Expr.Base Expression to be evaluated and substituted
-
children
¶ Type: Iterable[SourceNode] Yield all child nodes
-
-
class
WDL.Expr.
String
(pos: WDL.Error.SourcePosition, parts: List[Union[str, WDL.Expr.Placeholder]], command: bool = False)[source]¶ Bases:
WDL.Expr.Base
String literal, possibly interleaved with expression placeholders for interpolation
-
parts
= None¶ Type: List[Union[str,WDL.Expr.Placeholder]] The parts list begins and ends with the original delimiters (quote marks, braces, or triple angle brackets). Between these is a sequence of literal strings and/or interleaved placeholder expressions. Escape sequences in the literals will NOT have been decoded (although the parser will have checked they’re valid). Strings arising from task commands leave escape sequences to be interpreted by the shell in the task container. Other string literals have their escape sequences interpreted upon evaluation to string values.
-
command
= None¶ Type: bool True if this expression is a task command template, as opposed to a string expression anywhere else. Controls whether backslash escape sequences are evaluated or (for commands) passed through for shell interpretation.
-
children
¶ Type: Iterable[SourceNode] Yield all child nodes
-
literal
¶ If the expression is a literal constant, return its value; otherwise return None. The result can be an instance of
WDL.Value.Null
which is distinct from None.
-
-
class
WDL.Expr.
Array
(pos: WDL.Error.SourcePosition, items: List[WDL.Expr.Base])[source]¶ Bases:
WDL.Expr.Base
Array literal
-
items
= None¶ Type: List[WDL.Expr.Base] Expression for each item in the array literal
-
children
¶ Type: Iterable[SourceNode] Yield all child nodes
-
literal
¶ If the expression is a literal constant, return its value; otherwise return None. The result can be an instance of
WDL.Value.Null
which is distinct from None.
-
-
class
WDL.Expr.
Pair
(pos: WDL.Error.SourcePosition, left: WDL.Expr.Base, right: WDL.Expr.Base)[source]¶ Bases:
WDL.Expr.Base
Pair literal
-
left
= None¶ Type: WDL.Expr.Base Left-hand expression in the pair literal
-
right
= None¶ Type: WDL.Expr.Base Right-hand expression in the pair literal
-
children
¶ Type: Iterable[SourceNode] Yield all child nodes
-
literal
¶ If the expression is a literal constant, return its value; otherwise return None. The result can be an instance of
WDL.Value.Null
which is distinct from None.
-
-
class
WDL.Expr.
Map
(pos: WDL.Error.SourcePosition, items: List[Tuple[WDL.Expr.Base, WDL.Expr.Base]])[source]¶ Bases:
WDL.Expr.Base
Map literal
-
items
= None¶ Type: List[Tuple[WDL.Expr.Base,WDL.Expr.Base]] Expressions for the map literal keys and values
-
children
¶ Type: Iterable[SourceNode] Yield all child nodes
-
literal
¶ If the expression is a literal constant, return its value; otherwise return None. The result can be an instance of
WDL.Value.Null
which is distinct from None.
-
-
class
WDL.Expr.
Struct
(pos: WDL.Error.SourcePosition, members: List[Tuple[str, WDL.Expr.Base]], struct_type_name: Optional[str] = None)[source]¶ Bases:
WDL.Expr.Base
Struct literal
-
members
= None¶ Type: Dict[str,WDL.Expr.Base] The struct literal is modelled initially as a bag of keys and values, which can be coerced to a specific struct type during typechecking.
-
struct_type_name
= None¶ Type: Optional[str] In WDL 2.0+ each struct literal may specify the intended struct type name.
-
children
¶ Type: Iterable[SourceNode] Yield all child nodes
-
literal
¶ If the expression is a literal constant, return its value; otherwise return None. The result can be an instance of
WDL.Value.Null
which is distinct from None.
-
-
class
WDL.Expr.
IfThenElse
(pos: WDL.Error.SourcePosition, condition: WDL.Expr.Base, consequent: WDL.Expr.Base, alternative: WDL.Expr.Base)[source]¶ Bases:
WDL.Expr.Base
Ternary conditional expression
-
condition
= None¶ Type: WDL.Expr.Base A Boolean expression for the condition
-
consequent
= None¶ Type: WDL.Expr.Base Expression evaluated when the condition is true
-
alternative
= None¶ Type: WDL.Expr.Base Expression evaluated when the condition is false
-
children
¶ Type: Iterable[SourceNode] Yield all child nodes
-
-
class
WDL.Expr.
Ident
(pos: WDL.Error.SourcePosition, name: str)[source]¶ Bases:
WDL.Expr.Base
An identifier referencing a named value or call output.
Ident
nodes are wrapped inGet
nodes, as discussed below.-
name
= None¶ Type: str Name, possibly including a dot-separated namespace
-
referee
= None¶ After typechecking within a task or workflow, stores the AST node to which the identifier refers: a
WDL.Tree.Decl
for value references; aWDL.Tree.Call
for call outputs; aWDL.Tree.Scatter
for scatter variables; or aWDL.Tree.Gather
object representing a value or call output that resides within a scatter or conditional section.
-
children
¶ Type: Iterable[SourceNode] Yield all child nodes
-
-
class
WDL.Expr.
Get
(pos: WDL.Error.SourcePosition, expr: WDL.Expr.Base, member: Optional[str])[source]¶ Bases:
WDL.Expr.Base
AST node representing access to a value by identifier (including namespaced ones), or accessing a member of a pair or struct as
.member
.The entaglement of these two cases is inherent in WDL. Consider the syntax
leftname.midname.rightname
. One interpretation is thatleftname
is an identifier for a struct value, and.midname.rightname
represents a chain of struct member accesses. But another possibility is thatleftname
is a call,midname
is a struct output of that call, andrightname
is a member of that struct. These cases can’t be distinguished by the syntax parser alone, but must be resolved during typechecking with reference to the calls and identifiers available in the environment.The typechecker does conveniently resolve such cases, and to minimize the extent to which it has to restructure the AST in doing so, all identifiers (with or without a namespace) are represented as a
Get
node wrapping anIdent
node. TheGet
node may specify a member name to access, but may not if the identifier is to be accessed directly. On the other hand, the expression inside aGet
node need not be a simple identifier, e.g.arr[1].memb.left
is be represented as:Get(Get(Apply("_at", Get(Ident("arr")), 1),"memb"),"left")
-
expr
= None¶ Type: WDL.Expr.Base The expression whose value is accessed
-
member
= None¶ Type: Optional[str] If the expression is accessing a pair/struct member, then
expr.type
isWDL.Type.Pair
orWDL.Type.StructInstance
and this field gives the desired member name (left
orright
for pairs).Otherwise the expression accesses
expr
directly, andmember
isNone
.
-
children
¶ Type: Iterable[SourceNode] Yield all child nodes
-
-
class
WDL.Expr.
Apply
(pos: WDL.Error.SourcePosition, function: str, arguments: List[WDL.Expr.Base])[source]¶ Bases:
WDL.Expr.Base
Application of a built-in or standard library function
-
function_name
= None¶ Name of the function applied
Type: str
-
arguments
= None¶ Type: List[WDL.Expr.Base] Expressions for each function argument
-
children
¶ Type: Iterable[SourceNode] Yield all child nodes
-
Env¶
Environments, for identifier resolution during WDL typechecking and evaluation.
-
class
WDL.Env.
Binding
(name: str, value: T, info: Any = None)[source]¶ Bases:
typing.Generic
An individual, immutable binding of a possibly-namespaced name to a right-hand-side value of type
T
.T
is typicallyValue.Base
(value environments) orType.Base
(type environments). The binding may also reference an additional informational value of arbitrary type.-
name
¶ Type: str Namedspaced names are flat, dot-separated strings.
-
value
¶ Type: T
-
-
class
WDL.Env.
Bindings
(binding: Optional[WDL.Env.Binding[T]] = None, next: Optional[WDL.Env.Bindings[T]] = None)[source]¶ Bases:
typing.Generic
An environment consisting of an immutable linked-list of
WDL.Env.Binding
objects.WDL.Env.Bindings()
is the empty environment.Bindings[T]
is iterable for the individualBinding[T]
objects:env = WDL.Env.Bindings() env = env.bind("x", 1).bind("y", 42) print(env["x"]) # 1 print(",".join(str(b.value) for b in env)) # 1,42
-
bind
(name: str, value: T, info: Any = None) → WDL.Env.Bindings[~T][T][source]¶ Return an environment with a new binding prepended. Any existing binding for the same name is shadowed by the new one. (This should not usually arise in view of the immutability of WDL values.)
-
resolve_binding
(name: str) → WDL.Env.Binding[~T][T][source]¶ Look up a
WDL.Env.Binding
object by nameRaises: KeyError – no such binding
-
resolve
(name: str) → T[source]¶ Look up a bound value by name. Equivalently,
env[name]
Raises: KeyError – no such binding
-
get
(name: str, default: Optional[T] = None) → Optional[T][source]¶ Look up a bound value by name, returning the default value or
None
if there’s no such binding.
-
has_binding
(name: str) → bool[source]¶ Determine existence of a binding for the name. Equivalently,
name in env
-
map
(f: Callable[[WDL.Env.Binding[~T][T]], Optional[WDL.Env.Binding[~S][S]]]) → WDL.Env.Bindings[~S][S][source]¶ Copy the environment with each binding transformed by the given function. If the function returns
None
then the binding is excluded.
-
filter
(pred: Callable[[WDL.Env.Binding[~T][T]], bool]) → WDL.Env.Bindings[~T][T][source]¶ Copy the environment with only those bindings for which
pred
returns True
-
subtract
(rhs: WDL.Env.Bindings[~S][S]) → WDL.Env.Bindings[~T][T][source]¶ Copy the environment excluding any binding for which
rhs
has a binding with the same name
-
namespaces
¶ Type: Set[str] Return the environment’s namespaces, all the distinct dot-separated prefixes of the binding names. Each element ends with a dot.
-
Type¶
WDL data types
WDL has both atomic types such as Int
, Boolean
, and String
; and
parametric types like Array[String]
and
Map[String,Array[Array[Float]]]
. Here, each type is represented by an
immutable instance of a Python class inheriting from WDL.Type.Base
. Such
types are associated with expressions, statically prior to evaluation, as well
as with values and identifier bindings after evaluation.
An atomic type like Int
is represented by WDL.Type.Int()
. Atomic types
can be checked either with isinstance(t,WDL.Type.Int)
, which ignores the
possible optional quantifier (thus satisfied by Int
or Int?
), or with
t == WDL.Type.Int(optional=True)
to include the quantifier in the
comparison.
A parametric type like Array[String]
is represented by
WDL.Type.Array(WDL.Type.String())
. Any kind of array satisfies
isinstance(t,WDL.Type.Array)
, and
WDL.Type.Array(WDL.Type.String()) == WDL.Type.Array(WDL.Type.String())
, but
for example
WDL.Type.Array(WDL.Type.String()) != WDL.Type.Array(WDL.Type.Float())
.
The type classes include a method indicating if a value of the type can be coerced to some other desired type, according to the following rules:
Int
coerces toFloat
Boolean
,Int
,Float
, andFile
coerce toString
String
coerces toFile
,Int
, andFloat
Array[T]
coerces toString
providedT
does as wellT
coerces toT?
but the reverse is not true in general*Array[T]+
coerces toArray[T]
but the reverse is not true in general*
(*) The reverse coercions are statically permitted in expressions set up with
Expr.infer_type(check_quant=False)
although they may fail at runtime. This
also enables coercion of T
to Array[T]+
(an array of length 1).

-
class
WDL.Type.
Base
[source]¶ Bases:
abc.ABC
The abstract base class for WDL types
Each specific type inherits from this base, e.g.:
assert issubclass(WDL.Type.Int, WDL.Type.Base) assert isinstance(WDL.Type.Array(WDL.Type.Int()), WDL.Type.Base)
All instances are immutable.
-
coerces
(rhs: WDL.Type.Base, check_quant: bool = True) → bool[source]¶ True if this is the same type as, or can be coerced to,
rhs
.Parameters: check_quant – when False
, disables static enforcement of the optional (?) type quantifier
-
check
(rhs: WDL.Type.Base, check_quant: bool = True) → None[source]¶ Verify this is the same type as, or can be coerced to
rhs
. TheTypeError
exception raised otherwise MAY include a specific error message (but not if the obvious “cannot coerce self to rhs” suffices).Parameters: check_quant – when False
, disables static enforcement of the optional (?) type quantifier
-
optional
¶ Type: bool True when the type has the optional quantifier,
T?
-
parameters
¶ Type: Iterable[WDL.Type.Base] The type’s parameters, if any (e.g. item type of Array; left & right types of Pair; etc.)
-
-
class
WDL.Type.
Any
(optional: bool = False, null: bool = False)[source]¶ Bases:
WDL.Type.Base
A symbolic type which coerces to any other type; used to represent e.g. the item type of an empty array literal, or the result of read_json().
The
optional
attribute shall be true only for WDLNone
literals, which coerce to optional types only.
-
class
WDL.Type.
Boolean
(optional: bool = False)[source]¶ Bases:
WDL.Type.Base
-
class
WDL.Type.
Float
(optional: bool = False)[source]¶ Bases:
WDL.Type.Base
-
class
WDL.Type.
Int
(optional: bool = False)[source]¶ Bases:
WDL.Type.Base
-
class
WDL.Type.
File
(optional: bool = False)[source]¶ Bases:
WDL.Type.Base
-
class
WDL.Type.
Directory
(optional: bool = False)[source]¶ Bases:
WDL.Type.Base
-
class
WDL.Type.
String
(optional: bool = False)[source]¶ Bases:
WDL.Type.Base
-
class
WDL.Type.
Array
(item_type: WDL.Type.Base, optional: bool = False, nonempty: bool = False)[source]¶ Bases:
WDL.Type.Base
Array type, parameterized by the type of the constituent items.
-
item_type
= None¶ Type: WDL.Type.Base item_type
may beAny
when not known statically, such as in a literal empty array[]
.
-
nonempty
¶ Type: bool True when the type has the nonempty quantifier,
Array[T]+
-
parameters
¶ Type: Iterable[WDL.Type.Base] The type’s parameters, if any (e.g. item type of Array; left & right types of Pair; etc.)
-
-
class
WDL.Type.
Map
(item_type: Tuple[WDL.Type.Base, WDL.Type.Base], optional: bool = False, literal_keys: Optional[Set[str]] = None)[source]¶ Bases:
WDL.Type.Base
Map type, parameterized by the (key,value) item type.
-
item_type
= None¶ Type: Tuple[WDL.Type.Base,WDL.Type.Base] The key and value types may be
Any
when not known statically, such as in a literal empty map{}
.
-
literal_keys
= None¶
-
parameters
¶ Type: Iterable[WDL.Type.Base] The type’s parameters, if any (e.g. item type of Array; left & right types of Pair; etc.)
-
-
class
WDL.Type.
Pair
(left_type: WDL.Type.Base, right_type: WDL.Type.Base, optional: bool = False)[source]¶ Bases:
WDL.Type.Base
Pair type, parameterized by the left and right item types.
-
left_type
= None¶ Type: WDL.Type.Base
-
right_type
= None¶ Type: WDL.Type.Base
-
parameters
¶ Type: Iterable[WDL.Type.Base] The type’s parameters, if any (e.g. item type of Array; left & right types of Pair; etc.)
-
-
class
WDL.Type.
StructInstance
(type_name: str, optional: bool = False)[source]¶ Bases:
WDL.Type.Base
Type of an instance of a struct
Not to be confused with struct type definition,
WDL.Tree.StructTypeDef
. To find theWDL.Tree.StructTypeDef
in the currentdoc: WDL.Tree.Document
corresponding toty: WDL.Type.StructInstance
, usedoc.struct_typedefs[ty.type_name]
.-
type_name
= None¶ Type: str The struct type name with which the instance is declared; note that the same struct type can go by different names.
-
members
= None¶ Type: Dict[str,WDL.Type.Base] Names and types of the struct members, from the struct type definition (available after typechecking)
-
type_id
¶ Type: str A string canonically describing the member names and their types, excluding the struct type name; useful to unify aliased struct types.
-
parameters
¶ Type: Iterable[WDL.Type.Base] The type’s parameters, if any (e.g. item type of Array; left & right types of Pair; etc.)
-
-
WDL.Type.
unify
(types: List[WDL.Type.Base], check_quant: bool = True, force_string: bool = False) → WDL.Type.Base[source]¶ Given a list of types, compute a type to which they’re all coercible, or
WDL.Type.Any
if no more-specific inference is possible.Parameters: force_string – permit last-resort unification to String
even if no item is currently aString
, but all can be coerced
Value¶
WDL values instantiated at runtime
Each value is represented by an instance of a Python class inheriting from
WDL.Value.Base
.

-
class
WDL.Value.
Base
(type: WDL.Type.Base, value: Any, expr: Optional[Expr.Base] = None)[source]¶ Bases:
abc.ABC
The abstract base class for WDL values
-
type
= None¶ Type: WDL.Type.Base
-
value
= None¶ The “raw” Python value
-
expr
¶ Reference to the WDL expression that generated this value, if it originated from
WDL.Expr.eval
-
coerce
(desired_type: Optional[WDL.Type.Base] = None) → WDL.Value.Base[source]¶ Coerce the value to the desired type and return it. Types should be checked statically on
WDL.Expr.Base
prior to evaluation.Raises: ReferenceError for a null value and non-optional type
-
json
¶ Return a value representation which can be serialized to JSON using
json.dumps
-
-
class
WDL.Value.
Boolean
(value: bool, expr: Optional[Expr.Base] = None)[source]¶ Bases:
WDL.Value.Base
value
has Python typebool
-
class
WDL.Value.
Float
(value: float, expr: Optional[Expr.Base] = None)[source]¶ Bases:
WDL.Value.Base
value
has Python typefloat
-
class
WDL.Value.
Int
(value: int, expr: Optional[Expr.Base] = None)[source]¶ Bases:
WDL.Value.Base
value
has Python typeint
-
class
WDL.Value.
String
(value: str, expr: Optional[Expr.Base] = None, subtype: Optional[WDL.Type.Base] = None)[source]¶ Bases:
WDL.Value.Base
value
has Python typestr
-
class
WDL.Value.
File
(value: str, expr: Optional[Expr.Base] = None)[source]¶ Bases:
WDL.Value.String
value
has Python typestr
-
class
WDL.Value.
Directory
(value: str, expr: Optional[Expr.Base] = None)[source]¶ Bases:
WDL.Value.String
value
has Python typestr
-
class
WDL.Value.
Array
(item_type: WDL.Type.Base, value: List[WDL.Value.Base], expr: Optional[Expr.Base] = None)[source]¶ Bases:
WDL.Value.Base
value
is a Pythonlist
of otherWDL.Value.Base
instances
-
class
WDL.Value.
Map
(item_type: Tuple[WDL.Type.Base, WDL.Type.Base], value: List[Tuple[WDL.Value.Base, WDL.Value.Base]], expr: Optional[Expr.Base] = None)[source]¶ Bases:
WDL.Value.Base
-
class
WDL.Value.
Pair
(left_type: WDL.Type.Base, right_type: WDL.Type.Base, value: Tuple[WDL.Value.Base, WDL.Value.Base], expr: Optional[Expr.Base] = None)[source]¶ Bases:
WDL.Value.Base
-
class
WDL.Value.
Null
(expr: Optional[Expr.Base] = None)[source]¶ Bases:
WDL.Value.Base
Represents the missing value which optional inputs may take.
type
andvalue
are both None.
-
class
WDL.Value.
Struct
(type: Union[WDL.Type.Object, WDL.Type.StructInstance], value: Dict[str, WDL.Value.Base], expr: Optional[Expr.Base] = None, extra: Optional[Set[str]] = None)[source]¶ Bases:
WDL.Value.Base
-
WDL.Value.
from_json
(type: WDL.Type.Base, value: Any) → WDL.Value.Base[source]¶ Instantiate a WDL value of the specified type from a parsed JSON value (str, int, float, list, dict, or null).
If type is
WDL.Type.Any()
, attempts to infer a WDL type & value from the JSON’s intrinsic types. This isn’t ideal; for example, Files can’t be distinguished from Strings, and JSON lists and dicts with heterogeneous item types may give undefined results.Raises: WDL.Error.InputError – if the given value isn’t coercible to the specified type
-
WDL.Value.
rewrite_paths
(v: WDL.Value.Base, f: Callable[[Union[WDL.Value.File, WDL.Value.Directory]], Optional[str]]) → WDL.Value.Base[source]¶ Produce a deep copy of the given Value with all File & Directory paths (including those nested inside compound Values) rewritten by the given function. The function may return None to replace the File/Directory value with None/Null.
-
WDL.Value.
rewrite_env_paths
(env: WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base], f: Callable[[Union[WDL.Value.File, WDL.Value.Directory]], Optional[str]]) → WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base][source]¶ Produce a deep copy of the given Value Env with all File & Directory paths rewritten by the given function.
-
WDL.Value.
rewrite_files
(v: WDL.Value.Base, f: Callable[[str], Optional[str]]) → WDL.Value.Base[source]¶ Produce a deep copy of the given Value with all File names rewritten by the given function (including Files nested inside compound Values).
(deprecated: use
rewrite_paths
to handle Directory values as well)
-
WDL.Value.
rewrite_env_files
(env: WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base], f: Callable[[str], Optional[str]]) → WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base][source]¶ Produce a deep copy of the given Value Env with all File names rewritten by the given function.
(deprecated: use
rewrite_env_paths
to handle Directory values as well)
Error¶
-
class
WDL.Error.
SourcePosition
[source]¶ Bases:
WDL.Error.SourcePosition
Source position attached to AST nodes and exceptions; NamedTuple of
uri
the filename/URI passed toWDL.load()
or a WDL import statement, which may be relative;abspath
the absolute filename/URI; and one-based int positionsline
end_line
column
end_column
-
exception
WDL.Error.
SyntaxError
(pos: WDL.Error.SourcePosition, msg: str, wdl_version: str, declared_wdl_version: Optional[str])[source]¶ Bases:
Exception
Failure to lex/parse a WDL document
-
exception
WDL.Error.
ImportError
(pos: WDL.Error.SourcePosition, import_uri: str, message: Optional[str] = None)[source]¶ Bases:
Exception
Failure to open/retrieve an imported WDL document
The
__cause__
attribute may hold the inner error object.
-
class
WDL.Error.
SourceNode
(pos: WDL.Error.SourcePosition)[source]¶ Bases:
object
Base class for an AST node, recording the source position
-
pos
= None¶ Type: SourcePosition Source position for this AST node
-
children
¶ Type: Iterable[SourceNode] Yield all child nodes
-
-
exception
WDL.Error.
ValidationError
(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]¶ Bases:
Exception
Base class for a WDL validation error (when the document loads and parses, but fails typechecking or other static validity tests)
-
source_text
= None¶ Type: Optional[str] The complete source text of the WDL document (if available)
-
node
= None¶ Type: Optional[SourceNode]
-
pos
= None¶ Type: SourcePosition
-
-
exception
WDL.Error.
InvalidType
(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
IndeterminateType
(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
NoSuchTask
(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], name: str)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
NoSuchCall
(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], name: str)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
NoSuchFunction
(node: WDL.Error.SourceNode, name: str)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
WrongArity
(node: WDL.Error.SourceNode, expected: int)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
NotAnArray
(node: WDL.Error.SourceNode)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
NoSuchMember
(node: WDL.Error.SourceNode, member: str)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
StaticTypeMismatch
(node: WDL.Error.SourceNode, expected: WDL.Type.Base, actual: WDL.Type.Base, message: str = '')[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
IncompatibleOperand
(node: WDL.Error.SourceNode, message: str)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
UnknownIdentifier
(node: WDL.Error.SourceNode, message: Optional[str] = None)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
NoSuchInput
(node: WDL.Error.SourceNode, name: str)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
UncallableWorkflow
(node: WDL.Error.SourceNode, name: str)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
MultipleDefinitions
(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
StrayInputDeclaration
(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
CircularDependencies
(node: WDL.Error.SourceNode)[source]¶ Bases:
WDL.Error.ValidationError
-
exception
WDL.Error.
MultipleValidationErrors
(*exceptions)[source]¶ Bases:
Exception
Propagates several validation/typechecking errors
-
exceptions
= None¶ Type: List[ValidationError]
-
-
exception
WDL.Error.
RuntimeError
(*args, more_info: Optional[Dict[str, Any]] = None, **kwargs)[source]¶ Bases:
Exception
-
more_info
= None¶ Backend-specific information about an error (for example, pointer to a centralized log system)
-
-
exception
WDL.Error.
EvalError
(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]¶ Bases:
WDL.Error.RuntimeError
Error evaluating a WDL expression or declaration
-
node
= None¶ Type: Optional[SourceNode]
-
pos
= None¶ Type: SourcePosition
-
-
exception
WDL.Error.
OutOfBounds
(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition], message: str)[source]¶ Bases:
WDL.Error.EvalError
-
exception
WDL.Error.
EmptyArray
(node: WDL.Error.SourceNode)[source]¶ Bases:
WDL.Error.EvalError
-
exception
WDL.Error.
NullValue
(node: Union[WDL.Error.SourceNode, WDL.Error.SourcePosition])[source]¶ Bases:
WDL.Error.EvalError
-
exception
WDL.Error.
InputError
(*args, more_info: Optional[Dict[str, Any]] = None, **kwargs)[source]¶ Bases:
WDL.Error.RuntimeError
Error reading an input value/file
Lint¶
Annotate WDL document AST with hygiene warnings (underlies miniwdl check
)
Given a doc: WDL.Document
, the lint warnings can be retrieved like so:
import WDL
import WDL.Lint
lint = WDL.Lint.collect(WDL.Lint.lint(doc, descend_imports=False))
for (pos, lint_class, message, suppressed) in lint:
assert isinstance(pos, WDL.SourcePosition)
assert isinstance(lint_class, str) and isinstance(message, str)
if not suppressed:
print(json.dumps({
"uri" : pos.uri,
"abspath" : pos.abspath,
"line" : pos.line,
"end_line" : pos.end_line,
"column" : pos.column,
"end_column" : pos.end_column,
"lint" : lint_class,
"message" : message,
}))
The descend_imports
flag controls whether lint warnings are generated for imported documents
recursively (true, default), or otherwise only the given document (false).
Zip¶
Routines for packaging a WDL source file, with all imported source files, into a ZIP file.
New in v1.5.0
-
WDL.Zip.
build
(top_doc: WDL.Tree.Document, archive: str, logger: logging.Logger, inputs: Optional[Dict[str, Any]] = None, meta: Optional[Dict[str, Any]] = None, archive_format: str = 'zip', additional_files: Optional[List[str]] = None)[source]¶ Generate zip archive of the WDL document, all its imports, optional default inputs, and a generated manifest JSON.
If imports are drawn from outside the main WDL’s directory (or by URI), they’ll be stored in a special subdirectory and import statements will be rewritten to match.
-
class
WDL.Zip.
UnpackedZip
(dir, main_wdl, input_file)¶ Contextual value of WDL.Zip.unpack(): absolute paths of source directory, main WDL, and default input JSON file (if any). The source directory prefixes the latter paths.
-
dir
¶ Alias for field number 0
-
input_file
¶ Alias for field number 2
-
main_wdl
¶ Alias for field number 1
-
-
WDL.Zip.
unpack
(archive_fn: str) → Iterator[WDL.Zip.UnpackedZip][source]¶ Open a context with the WDL source archive unpacked into a temp directory, yielding UnpackedZip. The temp directory will be deleted on context exit.
A path to the MANIFEST.json of an already-unpacked source archive may also be used, or a directory containing one. In this case, it is NOT deleted on context exit.
with WDL.Zip.unpack("/path/to/source.zip") as unpacked: doc = WDL.load(unpacked.main_wdl) ...
runtime¶
The recommended way to run a WDL workflow programmatically is to invoke miniwdl run
as a
subprocess, capturing its JSON standard output. This leverages its logging, configuration, and
flexible input loading features; and avoids conflicting with the runtime’s thread pools and signal
handlers. Alternatively, it’s possible to call WDL.runtime.run()
directly if needed.
-
WDL.runtime.
run
(cfg: WDL.runtime.config.Loader, exe: Union[WDL.Tree.Task, WDL.Tree.Workflow], inputs: WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base], **run_kwargs) → Tuple[str, WDL.Env.Bindings[WDL.Value.Base][WDL.Value.Base]][source]¶ Run the task or workflow given the inputs environment and configuration, returning the outputs environment.
inputs
may be parsed from a JSON dict usingvalues_from_json()
, which can also validate them; see example below.Parameters: - run_id – a run identifier used in logs and filenames; defaults to executable name
- run_dir – directory under which to create a timestamp-named subdirectory for this run
(defaults to current working directory).
If the final path component is
.
then operate in run_dir directly.
Typical usage:
import WDL import WDL.runtime # Convert JSON-like inputs dict to WDL environment, validating them against exe's available # and required inputs. The dict keys should NOT be namespaced by the executable name; # if namespaces are present, then add namespace=exe.name to effectively remove them. inputs_env = WDL.values_from_json(inputs_dict, exe.available_inputs, exe.required_inputs) # Load configuration (see below) cfg = WDL.runtime.config.Loader(logging.getLogger(__name__)) # Run executable run_subdir, outputs_env = WDL.runtime.run(cfg, exe, inputs_env, run_dir="/tmp") # Generate JSON-like outputs dict, with keys namespaced by the executable name outputs_dict = WDL.values_to_json(outputs_env, exe.name)
-
class
WDL.runtime.config.
Loader
(logger: logging.Logger, filenames: Optional[List[str]] = None, overrides: Optional[Dict[str, Dict[str, str]]] = None)[source]¶ Runtime configuration options, identified by section & key, are sourced in the following priority order:
- Supplied
overrides
dict,{"section": {"key": value}}
- Environment variables
MINIWDL__SECTION__KEY
(uppercased with double-underscores) - Custom configuration file (mutually exclusive):
- filename given to
__init__
- file named by environment variable
MINIWDL_CFG
miniwdl.cfg
in XDG_CONFIG_HOME or XDG_CONFIG_DIRS, usually~/.config/miniwdl.cfg
- filename given to
WDL/runtime/config_templates/default.cfg
from installed package
If
filenames
is an empty list, then no custom configuration file is used. Orfilenames
may be a prioritized list of candidate filenames instead of (a-c) above; the first extant file is used, if any.- Supplied