Run specific step — pipe_run

Run given pipeline step possibly together with upstream and/or downstream dependencies.

Usage

pipe_run_step(
  pip,
  step,
  upstream = TRUE,
  downstream = FALSE,
  cleanUnkept = FALSE
)

Arguments

pip: Pipeline object
step: string name of step
upstream: logical if TRUE, run all dependent upstream steps first.
downstream: logical if TRUE, run all depdendent downstream afterwards.
cleanUnkept: logical if TRUE all output that was not marked to be kept is removed after the pipeline run. This option can be useful if temporary results require a lot of memory.

Value

returns the Pipeline object invisibly

Examples

p <- pipe_new("pipe", data = 1)
pipe_add(p, "add1", \(x = ~data, y = 1) x + y)
pipe_add(p, "add2", \(x = ~add1, z = 2) x + z)
pipe_add(p, "mult", \(x = ~add1, y = ~add2) x * y)
pipe_run_step(p, "add2")
#> INFO  [2025-01-03 19:12:47.142] Start step run of 'pipe' pipeline:
#> INFO  [2025-01-03 19:12:47.143] Step 1/3 data (upstream)
#> INFO  [2025-01-03 19:12:47.145] Step 2/3 add1 (upstream)
#> INFO  [2025-01-03 19:12:47.147] Step 3/3 add2
#> INFO  [2025-01-03 19:12:47.149] Finished execution of steps.
#> INFO  [2025-01-03 19:12:47.149] Done.

pipe_run_step(p, "add2", downstream = TRUE)
#> INFO  [2025-01-03 19:12:47.151] Start step run of 'pipe' pipeline:
#> INFO  [2025-01-03 19:12:47.152] Step 1/4 data (upstream)
#> INFO  [2025-01-03 19:12:47.154] Step 2/4 add1 (upstream)
#> INFO  [2025-01-03 19:12:47.156] Step 3/4 add2
#> INFO  [2025-01-03 19:12:47.158] Step 4/4 mult (downstream)
#> INFO  [2025-01-03 19:12:47.161] Finished execution of steps.
#> INFO  [2025-01-03 19:12:47.162] Done.

pipe_run_step(p, "mult", upstream = TRUE)
#> INFO  [2025-01-03 19:12:47.163] Start step run of 'pipe' pipeline:
#> INFO  [2025-01-03 19:12:47.164] Step 1/4 data (upstream)
#> INFO  [2025-01-03 19:12:47.167] Step 2/4 add1 (upstream)
#> INFO  [2025-01-03 19:12:47.169] Step 3/4 add2 (upstream)
#> INFO  [2025-01-03 19:12:47.171] Step 4/4 mult
#> INFO  [2025-01-03 19:12:47.172] Finished execution of steps.
#> INFO  [2025-01-03 19:12:47.172] Done.