Computer Vision in Retinal Implant with Deep Learning

Motivation

Neuroprosthetics is an emerging field in biomedical research and medical technology development. The loss of sensory or motor function has a significant negative impact on the quality of life and human well-being. Although significant progress has been made in ophthalmology in the past, conditions leading to untreatable blindness in a large number of people still exist. Restoration of vision can be achieved under these conditions by repairing cells with gene therapy, optogenetics, replacing cells with stem cells, or by bridging sensoric function with neuroprosthetic devices. All these concepts do have their advantages and disadvantages. In this RTG we are focussing on the neuroprosthetic approach.

RTG 2610 – Innovative Retinal Interfaces for Optimized Artificial Vision – InnoRetVision is a DFG funded Research Training Group dedicated to train graduate students with a background and interest in electrical engineering, neuroengineering, biophysics, sensory or neurophysiology, and or vision restoration. Our aim is to explore innovative methods for the stimulation of the visual system to improve current techniques for vision restoration in blind humans.

Scientific Questions

As a subproject of RTG InnoRetVision, we are focusing on the computer vision tasks in the retinal implants. The image signals taken from the camera(s) should be first downsampled to a drastically lower resolution, which are then sent to the electrode array to stimulate the neurons. The approaches to achieve this transformation effectively include image enhancement techniques, salient object detection and instance segmentation. In the case of stereo cameras or the combination of an RGB camera and e.g. an infrared camera or a depth camera (ToF), calibration techniques and information fusion algorithms are also of our interest.

Theses

New theses are regularly advertised in the area of Computer Vision in Retinal Interfaces via Deep Learning. In addition to the general overview, there are also numerous topics that have not yet been advertised, which will be gladly presented in a personal conversation.

External Funding

• DFG GRK 2610, “Innovative Retinal interfaces for optimized Artificial Vision – InnoRetVision”, Project nr. 424556709

Partner

• Prof. Dr. Peter Walter, Department of Opthalmology, RWTH Aachen University Hospital

Contact

M.Sc.

Yuli Wu
 +49 241 80 27974
 yuli.wu@lfb.rwth-aachen.de

Publications

2024

Yuli Wu, Julian Wittmann, Peter Walter and Johannes Stegmaier
Optimizing Retinal Prosthetic Stimuli with Conditional Invertible Neural Networks
In: arXiv preprint arXiv:2403.04884

2024

Yuli Wu, Weidong He, Dennis Eschweiler, Ningxin Dou, Zixin Fan, Shengli Mi, Peter Walter and Johannes Stegmaier
Retinal OCT Synthesis with Denoising Diffusion Probabilistic Models for Layer Segmentation
In: IEEE 21st International Symposium on Biomedical Imaging (ISBI)

2023

Yuli Wu, Laura Koch, Peter Walter and Dorit Merhof
Abstract: Convolutional Neural Network-based Inverse Encoder for Optimization of Retinal Prosthetic Stimulation
In: The Artificial Vision Symposium – The International Symposium on Visual Prosthetics

2023

Yuli Wu, Ivan Karetic, Johannes Stegmaier, Peter Walter and Dorit Merhof
A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation
In: International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)

2022

Yuli Wu, Peter Walter and Dorit Merhof
Multiscale Softmax Cross Entropy for Fovea Localization on Color Fundus Photography
In: Bildverarbeitung fuer die Medizin (BVM)