BigStream is a library of tools for 3D registration including images too large to fit into memory and/or too large to register in a single (multi-threaded) process. BigStream can automate chunking of the alignment problem into overlapping blocks, distributes the blocks to independent workers running in parallel, and stitches the results into a single smooth transform. BigStream includes global affine, piecewise affine, and piecewise deformable alignments; it also includes tools for finding feature points of interest, applying transforms, and inverting transforms. The tools can be used individually to construct custom workflows, but pipelines are also provided for specific alignment problems.
pip install bigstream
BigStream has just undergone a major change to prepare for releasing v1.0.0. Improvements include:
- Better utilization of dask primitives for faster and more robust distribution
- More accurate linear blending of transform blocks for smoother transforms
- A single python function to run the easi-fish registration pipeline directly
- New alignment functions: feature point ransac affine, random affine search
- Full access to the SimpleITK ImageRegistrationMethod options for almost all alignments
- Better source code design providing easy modular access at many points in the funciton hierarchy
Development wise, bigstream has reached a milestone in stability and I can now focus on documenting, teaching, and maintaining the package instead of building core functionality.
The master branch is the most up to date version. With minimal modification it can be used in any distributed environment supported by dask-jobqueue.
The prototype branch is a record of the first implementation, built using a different software stack. Rather than DASK, it handles blocking, distribution, and stiching manually. The primary workflow can be seen in the stream.sh script. This version was built specifically for LSF clusters and minimal modification of the submit function in stream.sh would be required for using this version on other clusters.
For those interested in using the modular components of BigStream, ipython notebooks are provided walking you through the components that make up a pipeline. For example, here is the tutorial for the multifish_registration_pipeline. Included in the repository are several datasets useful for testing and demonstrating functionality in the tutorials.
Please use the github issue tracker on this page for issues of any kind.
The tutorials are a more in depth way to learn the package, but here are some usage examples.
Bigstream is flexible toolkit that can be used in many different ways. I'll discuss some of them in order of "largest" (pipelines that chain together many steps) to "smallest" (individual functions).
frombigstream.application_pipelinesimporteasifish_registration_pipeline# load all the input datafix_lowres=""" load a lowres version of your fixed image """fix_highres=""" (lazy, e.g. zarr) load a highres version of your fixed image """mov_lowres=""" load a lowres version of your moving image """mov_highres=""" (lazy, e.g. zarr) load a highres version of your moving image """fix_lowres_spacing=""" voxel spacing of lowres fixed image """fix_highres_spacing=""" voxel spacing of highres fixed image """mov_lowres_spacing=""" voxel spacing of lowres moving image """mov_highres_spacing=""" voxel spacing of highres moving images """blocksize= [128, 128, 128] # size of individual alignment blocks in voxelswrite_directory='./somewhere_to_save_transforms_and_images'affine, deform, aligned=easifish_registration_pipeline(
fix_lowres, fix_highres, mov_lowres, mov_highres,
fix_lowres_spacing, fix_highres_spacing,
mov_lowres_spacing, mov_highres_spacing,
blocksize=blocksize,
write_directory=write_directory,
)This pipeline runs 4 steps:
- global affine based on feature point ransac
- global affine refinement based on gradient descent on image intensities
- local affine based on feature point ransac
- local deform based on gradient descent on image intensities
These four steps can be customized using these optional parameters to the pipeline:
global_ransac_kwargsglobal_affine_kwargslocal_ransac_kwargslocal_deform_kwargsSee the docstring for easifish_registration_pipeline for more details.
frombigstream.alignimportalignment_pipelinefrombigstream.piecewise_alignimportdistributed_piecewise_alignment_pipelinefrombigstream.transformimportapply_transformfrombigstream.piecewise_transformimportdistributed_apply_transform# load all the input datafix_lowres=""" load a lowres version of your fixed image """fix_highres=""" (lazy, e.g. zarr) load a highres version of your fixed image """mov_lowres=""" load a lowres version of your moving image """mov_highres=""" (lazy, e.g. zarr) load a highres version of your moving image """fix_lowres_spacing=""" voxel spacing of lowres fixed image """fix_highres_spacing=""" voxel spacing of highres fixed image """mov_lowres_spacing=""" voxel spacing of lowres moving image """mov_highres_spacing=""" voxel spacing of highres moving images """blocksize= [128, 128, 128] # size of individual alignment blocks in voxelswrite_directory='./somewhere_to_save_transforms_and_images'# construct a global alignment pipelinerigid_kwargs= {} # see bigstream.align.affine_align docstring for optionsaffine_kwargs= {} # see bigstream.align.affine_align docstring for optionsdeform_kwargs= {} # see bigstream.align.deformable_align docstring for optionssteps= [('rigid', rigid_kwargs), ('affine', affine_kwargs), ('deform', deform_kwargs)]
# run the alignmentglobal_transform=alignment_pipeline(
fix_lowres, mov_lowres,
fix_lowres_spacing,
mov_lowres_spacing,
steps=steps,
)
# apply transformglobal_aligned=apply_transform(
fix_lowres, mov_lowres,
fix_lowres_spacing, mov_lowres_spacing,
transform_list=[global_transform,],
)
# construct a local alignment pipeline to refine global resultrigid_kwargs= {} # see bigstream.align.affine_align docstring for optionsaffine_kwargs= {} # see bigstream.align.affine_align docstring for optionsdeform_kwargs= {} # see bigstream.align.deformable_align docstring for optionssteps= [('rigid', rigid_kwargs), ('affine', affine_kwargs), ('deform', deform_kwargs)]
blocksize= [128, 128, 128]
# run the alignmentlocal_transform=distributed_piecewise_alignment_pipeline(
fix_highres, mov_highresfix_highres_spacing, mov_highres_spacing,
steps=steps,
blocksize=blocksize,
static_transform_list=[global_transform,]
write_path='./deform.zarr',
cluster_kwargs={## params to control your cluster},
)
# apply the transformslocal_aligned=distributed_apply_transform(
fix_highres, mov_highres,
fix_highres_spacing, mov_highres_spacing,
transform_list=[global_transform, local_transform],
blocksize=blocksize,
write_path='./deformed.zarr',
cluster_kwargs={## params to control your cluster},
)For details, see the docstrings for:
bigstream.align.alignment_pipelinebigstream.piecewise_align.distributed_piecewise_alignment_pipelinebigstream.transform.apply_transformbigstream.piecewise_transform.distributed_apply_transform
frombigstream.alignimportaffine_alignfrombigstream.transformimportapply_transform# load all the input datafix=""" load your fixed image """mov=""" load your moving image """fix_spacing=""" voxel spacing of fixed image """mov_spacing=""" voxel spacing of moving image """# run an affine alignmentaffine=affine_align(
fix, mov,
fix_spacing, mov_spacing,
# TONS of customization and efficiency options available here
)
# apply transformaligned=apply_transform(
fix, mov,
fix_spacing, mov_spacing,
transform_list=[affine,], # this list can be arbitrarily long
)The docstrings for the functions in bigstream.align show all the configurable options available. Any alignment function has full access to the SimpleITK ImageRegistrationMethod API.