This article was submitted to Optics and Photonics, a section of the journal Frontiers in Physics

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

Digital holographic microscopy enables the measurement of the quantitative light field information and the visualization of transparent specimens. It can be implemented for complex amplitude imaging and thus for the investigation of biological samples including tissues, dry mass, membrane fluctuation, etc. Currently, deep learning technologies are developing rapidly and have already been applied to various important tasks in the coherent imaging. In this paper, an optimized structural convolution neural network PhaseNet is proposed for the reconstruction of digital holograms, and a deep learning-based holographic microscope using above neural network is implemented for quantitative phase imaging. Living mouse osteoblastic cells are quantitatively measured to demonstrate the capability and applicability of the system.

Optical microscope is an effective diagnostic tool in modern healthcare which allows pathologists to clearly and qualitatively observe the details of cells and tissues, and make judgments based on experience. This technique is sufficient in most cases. However, a bright field optical microscope records the intensity information of the specimen and suffers from low contrast for transparent biological cells which presents minimal light absorption. Various labeling methods, including staining and fluorescent tagging, are designed to enhance the imaging effect of the microscope, but the dyes may cross-react with the biological processes and affect the objectivity of medical diagnosis [

In comparison, quantitative phase imaging techniques enable quantitative light field information and the visualization of transparent specimens [

In our previous work, a common-path digital holographic microscopy based on a beam displacer unit was proposed for quantitative and dynamic phase imaging of biological cells [

In recent years, deep learning technology has developed rapidly, and very significant achievements have been made in areas such as autonomous driving, natural language processing, computer vision and many more. Currently, deep learning has also made remarkable achievements in computational imaging, and it has already been applied to various important tasks in coherent imaging, such as phase unwrapping [

In this paper, an optimized structural convolution neural network PhaseNet is proposed for the reconstruction of digital holograms, and a deep learning-based holographic microscope (DLHM) using PhaseNet is implemented for quantitative phase imaging. Living mouse osteoblastic cells are quantitatively measured to demonstrate the capability and applicability of the system.

Suppose the intensity of the recorded digital hologram in DHM is

The hologram is usually reconstructed using convolution algorithm, corresponding reconstructed object light field

And then, the intensity and phase information of the specimen can be calculated subsequently by

After the object beam passes through the biological specimen, the optical path difference ΔOPD will be introduced due to the phase difference Δ_{
medium
}, the cell thickness _{
specimen
}. Therefore, the ΔOPD can be calculated as

The ΔOPD is an integral effect of the optical path along the optical axis. Different parts of the cell have different RI resulting in different ΔOPD. Thus, in a certain sense, the ΔOPD represents the thickness of the cell.

The proposed deep learning-based holographic microscope includes a set of digital holographic microscope for the hologram recording and a deep neural network PhaseNet for the numerical reconstruction of digital holograms.

The main body of the proposed DLHM is a common-path digital holographic microscope as shown in

Deep learning-based holographic microscope and PhaseNet architecture.

PhaseNet is one of the core components of data processing system of DLHM. It completes the intelligent reconstruction of the hologram and obtains the three-dimensional phase information of the specimens replacing the traditional convolution algorithm or the Fresnel transform algorithm in DHM.

The work procedure of the proposed DLHM are as follows:

Hologram recording and reconstruction. Using DLHM to record off-axis digital holograms of the biological cells.

Phase information acquisition. Reconstructing the holograms by use of convolution algorithm to calculate the phase information of the biological cells.

PhaseNet training and testing. The holograms and phase results of each cell are used as input and ground truth, respectively, to train the PhaseNet. 9,000 pairs images are used for training, 1,000 pairs images for testing. Gaussian noise with random standard deviations from 0 to 25 is added into the holograms of the training dataset for better robustness. The ADAM-based optimization with an initial learning rate of 0.001 (dropping to the previous 0.75 every five epochs) is adopted to update PhaseNet’s parameters. The network is trained for 200 epochs.

Network output obtaining. In the network training process, the PhaseNet output is calculated according to the input of the network.

Loss function calculation. The mean squared error (MSE) of the PhaseNet output with ground truth (the phase information of biological cells) is calculated and used as the loss function. And the loss function is back-propagated through the network.

Quantitatively phase imaging of the biological cells. After finishing the above operations, the network training can be finally completed, and a neural network PhaseNet matching this DLHM can be obtained. Then, the digital hologram recorded by DLHM can be randomly input PhaseNet and the quantitative phase images of the specimen can be rapidly output. The network reconstruction time for a phase image is ∼0.014 s.

For PhaseNet implementing, Pytorch framework based on Python 3.6.1 is used. The network training and testing are performed on a PC with Core i7-8700K CPU, using NVIDIA GeForce GTX 1080Ti GPU. The training process takes ∼4 h for 100 epochs (∼10,000 pairs images size of 128 × 128 pixels in a batch size of 48).

The living mouse osteoblastic cells are measured by the DLHM. These mouse osteoblastic cells IDG-SW3 are cultured in Alpha minimum essential medium (αMEM, gibco by life technologies). They stick to the bottom of the petri dish while maintaining activity and are placed on the DLHM for measurement in room temperature environment.

_{
medium
} = 1.3377 calibrated by an Abbe refractometer, and by assuming a constant and homogeneous cellular RI _{
specimen
} = 1.375, we can estimate that a phase difference of 1 rad corresponds to a cellular thickness of 2.27 μm according to

Numerical reconstruction results of mouse osteoblastic cells by use of convolution algorithm.

1990 holograms of mouse osteoblastic cells are taken with DLHM. Then the phase images are recovered by traditional convolution algorithm. In order to improve the generalization ability of the neural network, data augmentation is a standard method. The dataset including 1990 holograms and corresponding phase images is expanded to 10,000 by flipping, rotating, etc. After that, the holograms and phase images of each cell are used as input and ground truth, respectively.

Dataset examples. The upper part shows the holograms as input, and the lower part shows the phase results as ground truth.

As the training progresses, the MSE of the ground truth and the output are back-propagated to the network, and parameters such as weights are updated by gradient descent. After 100 epoch training, the network reaches the convergence state. In the beginning, the loss function drops the fastest, as the epoch progresses, it becomes slower and slower, and the speed is close to zero at 100 epochs.

After training, we feed the holograms in the test set to PhaseNet, and the corresponding phase results are quickly reconstructed. Part of the results is visualized in

Neural network reconstruction results.

After completing the network training, the deep learning-based holographic microscope is feasible for quantitative phase measurement of living biological cells. It can completely replace the traditional digital holographic microscope for label-free cell imaging. At the same time, due to the use of neural networks, the acquisition of three-dimensional information of specimens can be completed more quickly.

In conclusion, we proposed PhaseNet for the reconstruction of digital holograms, based on which the DLHM is implemented for quantitative phase imaging of biological specimens. In order to verify the capability and applicability of DLHM, we used the living mouse osteoblastic cells as samples to generate dataset and train PhaseNet. The testing results show that the average SSIM index of DLHM can reach 0.9404.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

JD and JZ conceived and supervised the project. YL and KW performed experiments and data analysis. KW, JW and JT contributed to data analysis. KW, JW and JD wrote the draft of the manuscript; All the authors edited the manuscript.

This work was supported by National Natural Science Foundation of China (NSFC) (62075183, 61927810) and NSAF (U1730137).

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.