Geometric calibration and compensation of multispectral cameras

 

The project “Geometric calibration and compensation of multispectral cameras” that was funded by the German Research Fundation (DFG) fits into our research area multispectral imaging.
Project: AA5/2-1
Period: from 2009 to 2012

 

Aims of the research project

 

The methods for acquisition and straightforward processing of camera data from multispectral cameras featuring optical bandpass filters are well known. In the literature, the focus is often set on the setup of the camera, and the geometric calibration of the system is performed using usual tools, the cause of the aberrations present in the camera being poorly investigated. In this research project, physical models for the aberrations will be derived, and a geometric calibration of the system and a compensation of the distortions will be performed using exact measurements. In the second part of the project, a new type of cameras that enable the simultaneous acquisition of multispectral and depth information will be conceived and realized.

 

Results

 

Aims of this project were the geometric calibration and compensation of aberrations in multispectral cameras and the characterization and first measurements with a new type of camera: a 6-channel stereo multispectral camera.

In the first part of the project, the transversal aberrations were analyzed. These aberrations result in a translation of image points perpendicularly to the optical axis and are caused by both the color filters and the lens. The two types of aberrations were analyzed separately, which enabled the geometric modeling of the aberrations that could be utilized for the correction. These models depend on both the image position and the wavelength. An overall model for both aberrations has also been developed. Moreover, the transversal aberrations caused by filters that are positioned in front of the lens have been studied and compared to those caused by filters positioned between sensor and lens. A similar equation can be used for the geometric model, even though the order in which aberrations occur is different. Furthermore, a spectral and radiometric calibration has been performed for the developed multispectral system.

The second part of the project was focused on the longitudinal aberrations appearing in filter wheel multispectral cameras because of the different focal lengths of the color channels. The point spread function of the acquisition system was calculated using a special pattern imaged by the camera and can refer either to a reference color channel or to the original pattern. Important properties of the point spread function could be learned with simulations, that also corroborated results from real acquisitions. Different techniques were compared for the compensation of transversal aberrations, particularly in terms of sharpness and artifacts in the corrected image.

The transversal and longitudinal aberrations caused by optical filters and lens in filter wheel cameras have been thereby thoroughly analyzed. During the acquisition of multispectral images, no displacement of the sensor was necessary, and only the filter wheel was rotated. Thanks to image processing, distortions in multispectral images could be corrected without any complex mechanism or manual adjustment of the color channels during the acquisition. Multispectral imaging with a filter wheel camera was thus simplified. The compensation of longitudinal aberrations still requires an additional calibration, but first approaches to compensate aberrations without any calibration have been tested.

In the last part, a new type of multispectral cameras featuring two RGB sensor and two color filters was analyzed. Such cameras enable the simultaneous acquisition of multispectral and depth information from a 3D scene. The different elements of the acquisition system were optimally selected in order to reach the best color accuracy. Preliminary tests with different positions for the measuring device (either a multispectral camera or a spectrophotometer) showed that the angle dependence of the acquired objects was small enough to allow a stereo acquisition with precise color information. The color accuracy of this 6-channel camera was close to the color accuracy of the 7-channel camera utilized in the first parts of the project. Both types of cameras are suitable for acquisition of accurate color information. The differences of the disparity estimated with this 6-channel system and with conventional RGB stereo systems remained limited. After calibration and imaging, the stereo images are first rectified, the disparity is then estimated and finally the color differences of a known color pattern present in the 3D scene is calculated.

The possibility to combine multispectral and stereo imaging has thus been demonstrated. In particular, the results obtained with the stereo multispectral system made of two RGB cameras and filers showed that multispectral images could be acquired with a cost effective stereo camera.

 

ContaCt

 

Julie Klein, Tel.: +49 (241) 80-27866

 

Publications

 

	Julie Klein, Johannes Brauers and Til Aach Spatio-spectral modeling and compensation of transversal chromatic aberrations in multispectral imaging In: Journal of Imaging Science and Technology, vol. 55 (6), 060502, December 2011
	Julie Klein Unsupervised correction of relative longitudinal aberrations for multispectral imaging using a multiresolution approach In: IS&T/SPIE Electronic Imaging: Color Imaging XVIII San Francisco, California, USA, February 2013
	Julie Klein * invited talk * Farbgenauigkeit bei Stereo-Multispektralaufnahmen In: Jahrestagung der DfwG Aachen, Germany, October 2012
	Julie Klein and Bernhard Hill Multispectral stereo acquisition using two RGB cameras and color filters: color and disparity accuracy In: 18. Workshop Farbbildverarbeitung, p. 89-96 Darmstadt, Germany, September 2012
	Julie Klein and Til Aach Spectral and colorimetric constancy and accuracy of multispectral stereo systems In: IS&T Sixth European Conference on Color in Graphics, Imaging and Vision (CGIV 2012), p. 239-246 Amsterdam, Netherlands, May 2012
	Julie Klein and Til Aach Multispectral filter wheel cameras: modeling aberrations with filters in front of lens In: IS&T/SPIE Electronic Imaging: Digital Photography VIII, p. 82990R-1 – 82990R-9 San Francisco, California, USA, January 2012
	Julie Klein, Simon Gerhards and Til Aach Bildschärfe und Farbtreue bei Multispektralaufnahmen nach Korrektur von geometrischen Aberrationen In: 17. Workshop Farbbildverarbeitung, p. 85-96 Constance, Germany, September 2011
	Julie Klein Compensation of geometric distortions in multispectral imaging: Effects on sharpness and color accuracy In: Proceedings of the 15th International Student Conference on Electrical Engineering POSTER 2011 Prague, Czech Republic, May 2011
	Julie Klein, Johannes Brauers and Til Aach Methods for spectral characterization of multispectral cameras In: IS&T/SPIE Electronic Imaging: Digital Photography VII, p. 78760B-1 – 78760B-11 San Francisco, California, USA, January 2011
	Julie Klein, Johannes Brauers and Til Aach Spektrale Charakterisierung einer Multispektralkamera In: 16. Workshop Farbbildverarbeitung, p. 18-27 Ilmenau, Germany, October 2010 Presentation
	Julie Klein, Johannes Brauers and Til Aach Spatial and spectral analysis and modeling of transversal chromatic aberrations and their compensation In: IS&T Fifth European Conference on Color in Graphics, Imaging and Vision (CGIV 2010) 12th International Symposium on Multispectral Colour science, p. 516-522 Joensuu, Finland, June 2010 Presentation
	Johannes Brauers and Til Aach Direct PSF estimation using a random noise target In: IS&T/SPIE Electronic Imaging: Digital Photography VI, p. 75370B-1 – 75370B-10 San Jose, USA, January 2010
	Johannes Brauers, Claude Seiler and Til Aach Analyse und Kompensation von longitudinalen Aberrationen In: 15. Workshop Farbbildverarbeitung, p. 9-18 Berlin, Germany, October 2009
	Johannes Brauers and Til Aach Ghosting reflection compensation for multispectral high dynamic range imaging In: IS&T 17th Color Imaging Conference (CIC17), p. 170-174 Albuquerque, USA, November 2009

 

Last update: 2nd January 2014