
Articles
All vignettes
- Get started with pipeflow
Start here if this is your first time using pipeflow.
- Modify existing pipelines
How to insert, replace, and remove steps in a pipeline.
- Combine pipelines
How to combine different pipelines to a single pipeline.
- Pipeline views
How to filter pipelines with
pip_view()and the[operator, compose views, and run only a subset of steps.- Collect and group output
How to collect and group pipeline output.
- Split, map, and reduce
Shows how to split data, apply the pipeline to each subset, and then reduce the results back into a combined output.
- Nested pipelines
Shows how to run a pipeline as a step of another pipeline with as.function(), so that changing its parameters re-executes the outer step and only the inner steps that are affected.
- Handling failures
Shows what happens when a step fails, how to keep a pipeline running with on_error = “continue”, how to react to failed inputs with ~try() references, and how to recover once the cause is fixed.
- Annotated parameter values
Shows how to store annotated values (e.g. values with units or labels) as step parameters and how to unwrap them right before a step function is called, using the pip_param_value() generic.
- Annotated step outputs
Shows how to annotate the output of a step when the step returns, for example to stamp a result with the step it came from and the time it was computed, using the pip_output_value() generic.
- Self-modifying pipelines
Shows how you can setup pipelines to modify themselves at runtime, which, for example, allows for changing pipeline parameters based on intermediate results or even dynamically modify the pipeline’s own structure during a pipeline run.
- Comparison and benchmarks
Comparing general properties and performance of {pipeflow} vs {targets} in a scoped latency comparison.
- Emulating targets features
Recipes that emulate features of {targets} with existing {pipeflow} functions and a few lines of R code, and where the emulation stops.