Synthesis, Segmentation, and Tracking of Live Cell Microscopy Image Data



Developmental biology focuses on studying the processes of how cells migrate, interact and divide. Specifically, cell location and morphology, analyzed by segmentation and tracking approaches, constitute one of the key tasks in understanding the underlying principles of development. Technological advances allow biologists to capture and store extensive amounts of highly detailed microscopy image data, which renders the application of automated approaches essential. However, the potential of recent deep learning-based approaches is severely limited by the scarcity of fully annotated image data sets. Therefore, this project focuses on synthesis of fully annotated microscopy video data sets, eliminating the need for time-consuming manual annotation efforts.

Scientific Questions

  • How to generate realistic fully synthetic cell textures and shapes based on a limited amount of annotated real data?
  • How to capture the spatiotemporal relations of cell bodies and reflect this on the generated data?
  • How to make sure that there is enough variability in the synthetic data set and assess its quality?
  • How much improvement can we get by training segmentation and tracking models on abundant synthetic data as opposed to training them with limited real data?



External Funding

  • DFG Research Grant, “On-the-fly Datensynthese für eine auf Deep Learning basierende Analyse von 3D+t Mikroskopie Experimenten”, Project Nr. 447699143




Dennis Eschweiler, Rüveyda Yilmaz, Matisse Baumann, Ina Laube, Rijo Roy, Abin Jose, Daniel Brückner and Johannes Stegmaier
Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets
In: PLOS Computational Biology 20 (2)


D. Eschweiler, I. Laube, J. Stegmaier
Spatiotemporal Image Generation for Embryomics Applications
In: Biomedical Image Synthesis and Simulation: Methods and Applications


Dennis Eschweiler, Justus Schock and Johannes Stegmaier
Probabilistic Image Diversification to Improve Segmentation in 3D Microscopy Image Data
In: MICCAI International Workshop on Simulation and Synthesis in Medical Imaging (SASHIMI)


Dennis Eschweiler, Malte Rethwisch, Mareike Jarchow, Simon Koppers and Johannes Stegmaier
3D fluorescence microscopy data synthesis for segmentation and benchmarking
In: PLOS ONE 16 (12)


Dennis Eschweiler, Richard S. Smith and Johannes Stegmaier
Robust 3D Cell Segmentation: Extending the View of Cellpose
In: IEEE International Conference in Image Processing (ICIP)


C. Yang, D. Eschweiler, J. Stegmaier
Semi-and Self-Supervised Multi-View Fusion of 3D Microscopy Images using Generative Adversarial Networks
In: MICCAI Workshop on Machine Learning for Medical Image Reconstruction (MLMIR)


Dennis Eschweiler, Malte Rethwisch, Simon Koppers and Johannes Stegmaier
Spherical Harmonics for Shape-Constrained 3D Cell Segmentation
In: IEEE International Symposium on Biomedical Imaging (ISBI)


D. Bähr, D. Eschweiler, A. Bhattacharyya, D. Moreno-Andrés, W. Antonin and J. Stegmaier
CellCycleGAN: Spatiotemporal Microscopy Image Synthesis of Cell Populations using Statistical Shape Models and Conditional GANs
In: IEEE International Symposium on Biomedical Imaging (ISBI)


Dennis Eschweiler, Tim Klose, Florian Nicolas Müller-Fouarge, Marcin Kopaczka, Johannes Stegmaier
Towards Annotation-Free Segmentation of Fluorescently Labeled Cell Membranes in Confocal Microscopy Images
In: MICCAI International Workshop on Simulation and Synthesis in Medical Imaging (SASHIMI)


Dennis Eschweiler, Thiago V. Spina, Rohan C. Choudhury, Elliot Meyerowitz, Alexandre Cunha, Johannes Stegmaier
CNN-based Preprocessing to Optimize Watershed-based Cell Segmentation in 3D Confocal Microscopy Images
In: IEEE International Symposium on Biomedical Imaging (ISBI)