Contents
ExecutionEngine
Execution Engine
Internal execution engine that orchestrates rule-based dependency resolution and scheduling for requested resources.
The execution engine exposes both:
resolve.async()returning an execution object.resolve()as a blocking wrapper around async execution.
Runtime backend details are isolated in private submit(), ready(), and
collect() helpers so the
worker implementation can be swapped later without changing scheduling logic.
Rules may resolve a requested partial spec to a canonical spec by returning
list(spec = resolved.spec, data = value, ...). The scheduler stores the
canonical resource and records the mapping from requested id to resolved id in
execution state. Rules may also return outputs, a plain list of completed
output records (spec, data, optional meta, optional logs) discovered
during execution. Dynamic outputs are persisted immediately and attached to
the execution graph under their producing task.
Core allocation:
max.cores = NULLauto-detects available cores and uses at mostavailable - 1(minimum 1).- Detection prefers
availableCores()and falls back to common scheduler/job environment variables (SLURM_CPUS_PER_TASK,SLURM_CPUS_ON_NODE,PBS_NP,NSLOTS,LSB_DJOB_NUMPROC,NCPUS,OMP_NUM_THREADS) and thenparallel::detectCores().
Details
Execution Engine
Active bindings
requested.max.cores — Read-only user-requested max cores (NULL means auto).
max.cores — Read-only maximum cores for batched execution.
max.memory — Read-only maximum memory budget.
registry — Read-only rule registry used by this execution engine.
Methods
Public methods
ExecutionEngine$new()ExecutionEngine$resolve()ExecutionEngine$resolve.async()ExecutionEngine$await()ExecutionEngine$execute.status()ExecutionEngine$execute.summary()ExecutionEngine$execute.graph()ExecutionEngine$last.execution()ExecutionEngine$print()ExecutionEngine$clone()
Method new()
Create a reactive execution engine.
Usage
ExecutionEngine$new(
store,
registry = NULL,
max.cores = NULL,
max.memory = Inf,
execution.dir = NULL,
oversubscribe = FALSE
)
Arguments
store — ResourceStore instance.
registry — Optional RuleRegistry instance.
max.cores — Maximum cores for batched execution. NULL (default)
auto-selects up to available cores minus one.
max.memory — Maximum memory budget in GiB (gibibytes). Rejects a value strictly
between 0 and 1 as an almost-certain unit error (bytes or MB) -- see
polygenius.config.validate.max.memory().
execution.dir — Optional execution working directory.
oversubscribe — When TRUE, an explicit finite max.cores is honored
even above the detected-core cap. Intended for I/O-bound workloads (e.g.
many concurrent network fetches) where workers mostly block; off by default.
Method resolve()
Resolve output specs by using execution-engine scheduling and cache lookup.
Usage
ExecutionEngine$resolve(outputs, action = NULL, execution.status = NULL)
Arguments
outputs — Output spec(s): single ResourceSpec, list/vector of
ResourceSpec, or ResourceSpecSet.
action — Optional label shown in execution-engine status output.
execution.status — Optional execution-status mode override ("auto",
"yes", "no").
Returns
ResourceSpec when one output is requested, otherwise list of
ResourceSpec.
Method resolve.async()
Start async execution for requested outputs.
Usage
ExecutionEngine$resolve.async(outputs, action = NULL, execution.status = NULL)
Arguments
outputs — Output spec(s): single ResourceSpec, list/vector of
ResourceSpec, or ResourceSpecSet.
action — Optional label shown in execution-engine status output.
execution.status — Optional execution-status mode override ("auto",
"yes", "no").
Returns
An execution object accepted by await().
Method await()
Wait for an async execution to finish.
Usage
ExecutionEngine$await(
execution,
poll.interval = 0.25,
refill.budget = 0.5,
timeout = Inf
)
Arguments
execution — Execution object returned by resolve.async().
poll.interval — Poll interval in seconds while waiting for workers.
refill.budget — Wall-clock seconds the same-tick refill loop may spend
before the loop tail (drain, status render, progress emit, deadlock check)
is allowed to run. Bounds the refill loop; see the note at its call site.
timeout — Maximum wait time in seconds (Inf for no timeout).
Returns
Invisibly returns the updated execution.
Method execute.status()
Get execution status.
Usage
ExecutionEngine$execute.status(execution)
Arguments
execution — Execution object returned by resolve.async().
Returns
Character status.
Method execute.summary()
Get execution summary.
Usage
ExecutionEngine$execute.summary(execution)
Arguments
execution — Execution object returned by resolve.async().
Returns
Named list summarizing execution-engine progress.
Method execute.graph()
Return execution dependency graph with node states.
Usage
ExecutionEngine$execute.graph(execution)
Arguments
execution — Execution object returned by resolve.async().
Returns
Named list with nodes, edges, and blocked.by data frames.
Method last.execution()
Return the most recently started execution.
Usage
ExecutionEngine$last.execution()
Returns
Execution object or NULL when no execution has run yet.
Method print()
Print execution-engine summary.
Usage
ExecutionEngine$print(...)
Arguments
... — Unused.
Method clone()
The objects of this class are cloneable with this method.
Usage
ExecutionEngine$clone(deep = FALSE)
Arguments
deep — Whether to make a deep clone.