Skip to contents

All vignettes

Get started with pipeflow

Start here if this is your first time using pipeflow.

Modify existing pipelines

How to insert, replace, and remove steps in a pipeline.

Combine pipelines

How to combine different pipelines to a single pipeline.

Pipeline views

How to filter pipelines with pip_view() and the [ operator, compose views, and run only a subset of steps.

Collect and group output

How to collect and group pipeline output.

Split, map, and reduce

Shows how to split data, apply the pipeline to each subset, and then reduce the results back into a combined output.

Nested pipelines

Shows how to run a pipeline as a step of another pipeline with as.function(), so that changing its parameters re-executes the outer step and only the inner steps that are affected.

Handling failures

Shows what happens when a step fails, how to keep a pipeline running with on_error = “continue”, how to react to failed inputs with ~try() references, and how to recover once the cause is fixed.

Annotated parameter values

Shows how to store annotated values (e.g. values with units or labels) as step parameters and how to unwrap them right before a step function is called, using the pip_param_value() generic.

Annotated step outputs

Shows how to annotate the output of a step when the step returns, for example to stamp a result with the step it came from and the time it was computed, using the pip_output_value() generic.

Self-modifying pipelines

Shows how you can setup pipelines to modify themselves at runtime, which, for example, allows for changing pipeline parameters based on intermediate results or even dynamically modify the pipeline’s own structure during a pipeline run.

Comparison and benchmarks

Comparing general properties and performance of {pipeflow} vs {targets} in a scoped latency comparison.

Emulating targets features

Recipes that emulate features of {targets} with existing {pipeflow} functions and a few lines of R code, and where the emulation stops.