Synthesis, Segmentation, and Tracking of Live Cell Microscopy Image Data

Motivation

Developmental biology focuses on studying the processes of how cells migrate, interact and divide. Specifically, cell location and morphology, analyzed by segmentation and tracking approaches, constitute one of the key tasks in understanding the underlying principles of development. Technological advances allow biologists to capture and store extensive amounts of highly detailed microscopy image data, which renders the application of automated approaches essential. However, the potential of recent deep learning-based approaches is severely limited by the scarcity of fully annotated image data sets. Therefore, this project focuses on synthesis of fully annotated microscopy video data sets, eliminating the need for time-consuming manual annotation efforts.

Scientific Questions

• How to generate realistic fully synthetic cell textures and shapes based on a limited amount of annotated real data?
• How to capture the spatiotemporal relations of cell bodies and reflect this on the generated data?
• How to make sure that there is enough variability in the synthetic data set and assess its quality?
• How much improvement can we get by training segmentation and tracking models on abundant synthetic data as opposed to training them with limited real data?

Partners

• G. U. Nienhaus and Dr. A. Kobitski, Institute of Applied Physics, Karlsruhe Institute of Technology (KIT)
• Prof. U. Strähle and Dr. M. Takamiya, Institute of Biological and Chemical Systems – Biological Information Processing, Karlsruhe Institute of Technology (KIT)
• apl. Prof. R. Mikut, Institute for Automation and Applied Informatics, Karlsruhe Institute of Technology (KIT)

External Funding

• DFG Research Grant, “On-the-fly Datensynthese für eine auf Deep Learning basierende Analyse von 3D+t Mikroskopie Experimenten”, Project Nr. 447699143

Contact

Publications

2024

Dennis Eschweiler, Rüveyda Yilmaz, Matisse Baumann, Ina Laube, Rijo Roy, Abin Jose, Daniel Brückner and Johannes Stegmaier
Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets
In: PLOS Computational Biology 20 (2)

2022

D. Eschweiler, I. Laube and J. Stegmaier
Spatiotemporal Image Generation for Embryomics Applications
In: Biomedical Image Synthesis and Simulation: Methods and Applications

2022

Dennis Eschweiler, Justus Schock and Johannes Stegmaier
Probabilistic Image Diversification to Improve Segmentation in 3D Microscopy Image Data
In: MICCAI International Workshop on Simulation and Synthesis in Medical Imaging (SASHIMI)

2021

Dennis Eschweiler, Malte Rethwisch, Mareike Jarchow, Simon Koppers and Johannes Stegmaier
3D fluorescence microscopy data synthesis for segmentation and benchmarking
In: PLOS ONE 16 (12)

2022

Dennis Eschweiler, Richard S. Smith and Johannes Stegmaier
Robust 3D Cell Segmentation: Extending the View of Cellpose
In: IEEE International Conference in Image Processing (ICIP)

2021

C. Yang, D. Eschweiler and J. Stegmaier
Semi-and Self-Supervised Multi-View Fusion of 3D Microscopy Images using Generative Adversarial Networks
In: MICCAI Workshop on Machine Learning for Medical Image Reconstruction (MLMIR)

2021

Dennis Eschweiler, Malte Rethwisch, Simon Koppers and Johannes Stegmaier
Spherical Harmonics for Shape-Constrained 3D Cell Segmentation
In: IEEE International Symposium on Biomedical Imaging (ISBI)

2021

D. Eschweiler D. Bähr and J. Stegmaier
CellCycleGAN: Spatiotemporal Microscopy Image Synthesis of Cell Populations using Statistical Shape Models and Conditional GANs
In: IEEE International Symposium on Biomedical Imaging (ISBI)

2019

Dennis Eschweiler, Tim Klose, Florian Nicolas Müller-Fouarge, Marcin Kopaczka and Johannes Stegmaier
Towards Annotation-Free Segmentation of Fluorescently Labeled Cell Membranes in Confocal Microscopy Images
In: MICCAI International Workshop on Simulation and Synthesis in Medical Imaging (SASHIMI)

2019

Thiago V. Spina Dennis Eschweiler
CNN-based Preprocessing to Optimize Watershed-based Cell Segmentation in 3D Confocal Microscopy Images
In: IEEE International Symposium on Biomedical Imaging (ISBI)