v0.1.4
New features
Config-based Pipeline constructions.
The newly introduced spdl.pipeline.defs module contains the definitions and helper functions you can use to build Pipeline object with declarative manner. For the detail, please refer to #902
You can now construct Pipeline in the following ways. Also please checkout the Hydra Integration Example.
| PipelineBuilder (existing) | PipelineDefinition (new) | Hydra |
builder = (
PipelineBuilder()
.add_source(Sampler())
.pipe(
funcA,
concurrency=...)
.pipe(
funcB,
concurrency=...,
executor=...)
.sink(...)
)
pipeline = builder.build(
num_threads=...) |
pdef = PipelineConfig(
src=Sampler(),
stages=[
PipeConfig(
funcA,
concurrency=...),
PipeConfig(
funcB,
concurrency=...,
executor=...),
],
sink=...
)
pipeline = build_pipeline(
pdef, num_threads=...) |
_target_: build_pipeline
num_threads: ...
definition:
_target_: PipelineConfig
src:
_target_: Sampler
stages:
- _target_: PipeConfig:
op: funcA
concurrency: ...
- _target_: PipeConfig:
op: funcB
concurrency: ...
executor: ...
sink:
... |
[Experimental] Windows Support
SPDL I/O now supports building on Windows. The binary distribution contains CUDA integration and NVDEC support.
Logging change
Previously, if a function passed to a pipe fails, the error was logged in one line. Now the full stack trace is printed.