Digital Inline Holographic Microscopy (DIHM) of Weakly-scattering Subjects

Summary

The three-dimensional locations of weakly-scattering objects can be uniquely identified using digital inline holographic microscopy (DIHM), which involves a minor modification to a standard microscope. Our software uses a simple imaging heuristic coupled with Rayleigh-Sommerfeld back-propagation to yield the three-dimensional position and geometry of a microscopic phase object.

Abstract

Weakly-scattering objects, such as small colloidal particles and most biological cells, are frequently encountered in microscopy. Indeed, a range of techniques have been developed to better visualize these phase objects; phase contrast and DIC are among the most popular methods for enhancing contrast. However, recording position and shape in the out-of-imaging-plane direction remains challenging. This report introduces a simple experimental method to accurately determine the location and geometry of objects in three dimensions, using digital inline holographic microscopy (DIHM). Broadly speaking, the accessible sample volume is defined by the camera sensor size in the lateral direction, and the illumination coherence in the axial direction. Typical sample volumes range from 200 µm x 200 µm x 200 µm using LED illumination, to 5 mm x 5 mm x 5 mm or larger using laser illumination. This illumination light is configured so that plane waves are incident on the sample. Objects in the sample volume then scatter light, which interferes with the unscattered light to form interference patterns perpendicular to the illumination direction. This image (the hologram) contains the depth information required for three-dimensional reconstruction, and can be captured on a standard imaging device such as a CMOS or CCD camera. The Rayleigh-Sommerfeld back propagation method is employed to numerically refocus microscope images, and a simple imaging heuristic based on the Gouy phase anomaly is used to identify scattering objects within the reconstructed volume. This simple but robust method results in an unambiguous, model-free measurement of the location and shape of objects in microscopic samples.

Introduction

Digital inline holographic microscopy (DIHM) allows fast three-dimensional imaging of microscopic samples, such as swimming microorganisms^1,2 and soft matter systems^3,4, with minimal modification to a standard microscope setup. In this paper a pedagogical demonstration of DIHM is provided, centered around software developed in our lab. This paper includes a description of how to set up the microscope, optimize data acquisition, and process the recorded images to reconstruct three-dimensional data. The software (based in part on software developed by D.G. Grier and others⁵) and example images are freely available on our website. A description is provided of the steps necessary to configure the microscope, reconstruct three-dimensional volumes from holograms and render the resulting volumes of interest using a free ray tracing software package. The paper concludes with a discussion of factors affecting the quality of reconstruction, and a comparison of DIHM with competing methods.

Although DIHM was described some time ago (for a general review of its principles and development, see Kim⁶), the required computing power and image processing expertise has hitherto largely restricted its use to specialist research groups with a focus on instrument development. This situation is changing in the light of recent advances in computing and camera technology. Modern desktop computers can easily cope with the processing and data storage requirements; CCD or CMOS cameras are present in most microscopy labs; and the requisite software is being made freely available on the internet by groups who have invested time in developing the technique.

Various schemes have been proposed for imaging the configuration of microscopic objects in a three-dimensional sample volume. Many of these are scanning techniques^7,8, in which a stack of images is recorded by mechanically translating the image plane through the sample. Scanning confocal fluorescence microscopy is perhaps the most familiar example. Typically, a fluorescent dye is added to a phase object in order to achieve an acceptable level of sample contrast, and the confocal arrangement used to spatially localize fluorescent emission. This method has led to significant advances, for example in colloid science where it has allowed access to the three-dimensional dynamics of crowded systems^9-11. The use of labeling is an important difference between fluorescence confocal microscopy and DIHM, but other features of the two techniques are worth comparing. DIHM has a significant speed advantage in that the apparatus has no moving parts. The mechanical scanning mirrors in confocal systems place an upper limit on the data acquisition rate - typically around 30 frames/sec for a 512 x 512 pixel image. A stack of such images from different focal planes can be obtained by physically translating the sample stage or objective lens between frames, leading to a final capture rate of around one volume per second for a 30 frame stack. In comparison, a holographic system based on a modern CMOS camera can capture 2,000 frames/sec at the same image size and resolution; each frame is processed `offline' to give an independent snapshot of the sample volume. To reiterate: fluorescent samples are not required for DIHM, although a system has been developed that performs holographic reconstruction of a fluorescent subject¹². As well as three-dimensional volume information, DIHM can also be used to provide quantitative phase contrast images¹³, but that is beyond the scope of the discussion here.

Raw DIHM data images are two-dimensional, and in some respects look like standard microscope images, albeit out of focus. The main difference between DIHM and standard bright field microscopy lies in the diffraction rings that surround objects in the field of view; these are due to the nature of the illumination. DIHM requires a more coherent source than bright field - typically an LED or laser. The diffraction rings in the hologram contain the information necessary to reconstruct a three-dimensional image. There are two main approaches to interpreting DIHM data; direct fitting and numerical refocusing. The first approach is applicable in cases where the mathematical form of the diffraction pattern is known in advance^3,4; this condition is met by a small handful of simple objects like spheres, cylinders, and half-plane obstacles. Direct fitting is also applicable in cases where the object's axial position is known, and the image can be fitted using a look-up table of image templates¹⁴.

The second approach (numerical refocusing) is rather more general and relies on using the diffraction rings in the two-dimensional hologram image to numerically reconstruct the optical field at a number of (arbitrarily spaced) focal planes throughout the sample volume. Several related methods exist for doing this⁶; this work uses the Rayleigh-Sommerfeld back propagation technique as described by Lee and Grier⁵. The outcome of this procedure is a stack of images that replicate the effect of manually changing the microscope focal plane (hence the name `numerical refocusing'). Once a stack of images has been generated, the position of the subject in the focal volume must be obtained. A number of image analysis heuristics, such as local intensity variance or spatial frequency content, have been presented to quantify the sharpness of focus at different points in the sample¹⁵. In each case, when a particular image metric is maximized (or minimized), the object is considered to be in focus.

Unlike other schemes that seek to identify a particular focal plane where an object is 'in focus', the method in this work picks out points that lie inside the object of interest, which may extend across a wide range of focal planes. This approach is applicable to a broad range of subjects and is particularly suitable for extended, weakly-scattering samples (phase objects), such as rod-shaped colloids, chains of bacteria or eukaryotic flagella. In such samples the image contrast changes when the object passes through the focal plane; a defocused image has a light center if the object is on one side of the focal plane and a dark center if it is on the other. Pure phase objects have little to no contrast when they lie exactly in the focal plane. This phenomenon of contrast inversion has been discussed by other authors^16,17 and is ultimately due to the Gouy phase anomaly¹⁸. This has been put on a more rigorous footing in the context of holography elsewhere, where the limits of the technique are evaluated; the typical uncertainty in position is of the order of 150 nm (approximately one pixel) in each direction¹⁹. The Gouy phase anomaly method is one of the few well-defined DIHM schemes for determining the structure of extended objects in three dimensions, but nevertheless some objects are problematic to reconstruct. Objects that lie directly along the optical axis (pointing at the camera) are difficult to reconstruct accurately; uncertainties in the length and position of the object become large. This limitation is in part due to the restricted bit depth of the pixels recording the hologram (the number of distinct gray levels that the camera can record). Another problematic configuration occurs when the object of interest is very close to the focal plane. In this case, the real and virtual images of the object are reconstructed in close proximity, giving rise to complicated optical fields that are difficult to interpret. A second, slightly less important concern here is that the resulting diffraction fringes occupy less of the image sensor, and this coarser-grained information leads to a poorer-quality reconstruction.

In practice, a simple gradient filter is applied to three-dimensional reconstructed volumes to detect strong intensity inversions along the illumination direction. Regions where intensity changes quickly from light to dark, or vice versa, are then associated with scattering regions. Weakly-scattering objects are well described as a noninteracting collection of such elements²⁰; these individual contributions sum to give the total scattered field which is easily inverted using the Rayleigh-Sommerfeld back propagation method. In this paper, the axial intensity gradient technique is applied to a chain of Streptococcus cells. The cell bodies are phase objects (the species E. coli has a refractive index measured²¹ to be 1.384 at a wavelength λ=589 nm; the Streptococcus strain is likely to be similar) and appear as a high-intensity chain of connected blobs in the sample volume after the gradient filter has been applied. Standard threshold and feature extraction methods applied to this filtered volume allow the extraction of volumetric pixels (voxels) corresponding to the region inside the cells. A particular advantage of this method is that it allows unambiguous reconstruction of an object's position in the axial direction. Similar methods (at least, those that record holograms close to the object, imaged through a microscope objective) suffer from being unable to determine the sign of this displacement. Although the Rayleigh-Sommerfeld reconstruction method is also sign-independent in this sense, the gradient operation allows us to discriminate between weak phase objects above and below the focal plane.

Protocol

1. Setup and Data Acquisition

1. Grow a culture of Streptococcus strain V4051-197 (smooth swimming) cells in KTY medium from frozen stock²². Incubate in a rotary shaker overnight at 35 °C and 150 rpm, to saturation.
2. Inoculate 10 ml of fresh KTY medium with 500 µl of the saturated culture. Incubate for a further 3.5 hr at 35°C and 150 rpm until the cells reach an optical density of approximately 1.0 at λ=600 nm (approximately 5 x 10⁸ cells/ml).
3. Dilute 1:400 in fresh media to obtain the final concentration of motile cells.
4. On a microscope slide, make a ring of grease (e.g. from a syringe filled with petroleum jelly) around 1 mm in height and place a drop of the sample solution in the center. Place a cover glass on top and press lightly at the edges to seal, ensuring that liquid is in contact with both slide and cover glass. The resulting sample chamber should be around 100-200 µm in depth.
5. Place the sample chamber in the microscope with the cover glass facing down, and carefully, to prevent the formation of air bubbles in the oil between glass and lens.
6. Focus the microscope on the bottom surface of the sample chamber, and bring the condenser to focus.
7. Switch off the standard illumination and place the LED head behind the condenser aperture of the microscope. Set the LED power supply to maximum output.
8. Close the condenser aperture to its fullest extent, and if necessary, nudge the LED in its mount until the illumination is centered on the objective lens aperture.
9. Switch on the computer and camera, and redirect all light to the microscope's camera port. Adjust the frame rate and the image size in the image acquisition software.
10. Make the frame exposure time as short as possible while still retaining good contrast. Check an image intensity histogram to ensure that the image is not saturated or under-exposed.
11. If necessary, refocus the microscope so that object of interest is slightly defocused (typically by 10-30 µm). The object and the focal plane should be in the same medium (i.e. the focal plane should lie inside the sample chamber).

2. Reconstruction

The first step of processing data is to numerically refocus a video frame at a series of different depths, producing a stack of images. User-friendly software for doing this may be found here: http://www.rowland.harvard.edu/rjf/wilson/Downloads.html along with example images (acquired using an inverted microscope, and a 60X oil immersion objective lens) and a scene file for ray traced rendering.

1. Input the frame of interest and its respective background as separate image files, in the relevant boxes on the interface. A background image is a frame that fairly represents the background of the video, in the absence of the hologram, and will be used to suppress any fixed-pattern noise that could interfere with the hologram localization and analysis.
2. Enter values into the global settings boxes on the left hand side of the screen before running the program. The first three are output stack parameters: Axial position of the first frame ('Start focus'); number of slices in the reconstructed stack ('Number of steps'); axial distance between each slice in the stack ('Step size').
3. In order to reconstruct objects with the correct proportions, the step size should be the same size as the lateral pixel spacing. Enter the camera's lateral sampling frequency (1/pixel spacing) in the 'Pixel/micron' box, the illumination wavelength and medium refractive index in the last two boxes. The program's default values are appropriate for reconstructing the example data.
4. Press the 'Flip Z-gradient' button if the center of the object is dark in the hologram (see example frame 2005 and the discussion section for more details).
5. Check the bandpass filter on/off state (default is on). This optional bandpass filter, located in a box below the gradient-flip switch, is responsible for suppressing the contribution from noisy pixels. The bandpass filter is applied to each image slice immediately after generation.
6. Check the intermediate output switch on/off states (default is on). Intermediate analysis steps are written in two output videos, as uncompressed .avi files: the first is the refocused stack (file name ending in '_stack.avi'), the second is the stack after the axial gradient operation (file name ending in '_gradient.avi'). Ideally, this second stack will contain the object of interest highlighted as a bright object against a dark background.
7. After setting all parameters, press 'Run' in the main window. The selected frame will appear in the main box of the software.
8. Use the magnifying glass tool found on the toolbar to the left of the image to zoom in (left click) and zoom out (shift+left click). Use the rectangle tool on the toolbar to select a region of interest (ROI). Capture as many of the hologram's fringes as possible inside the rectangle, as this will optimize the reconstruction.
9. Press the 'Process' button to generate the two image stacks. Inspect the resulting stacks using ImageJ (which can be obtained for free at http://rsb.info.nih.gov/ij/). If the object of interest can be clearly seen in the gradient image stack, proceed to the next step to extract the object coordinates.

3. Rendering

1. Besides frame refocusing, this program can locate the x, y, z coordinates for each voxel in the object of interest. Press the 'Feature Extraction' button to enable this function.
2. Enter a path, including extension (for example: c:\home\output.inc), for the output coordinates file. Select 'POV-Ray style' in the 'output coordinate style' box. This causes the program to write an object file that can be visualized using the free POV-Ray raytracing software package (which can be obtained from http://www.povray.org/).
3. Reprocess the images as in section 2 above. The program will provide a series of (x, y ,z) coordinates in the selected ROI, written to the file name in the 'Output coordinate' box. Extraction of object coordinates takes significantly longer than stack generation.
4. Make sure that the sample POV-Ray file (supplied with the reconstruction code) is in the same folder as the coordinates file just generated. Edit the sample .pov file and replace the file name in inverted commas on the line
   #include "MYFILE.inc"
   with the name of the data file generated by the reconstruction code.
5. Click the 'Run' button in POV-Ray to render a bitmap image. The camera position, lighting and texture options are just a few of the customizations possible in POV-Ray; see the online documentation for details.

Results

To demonstrate the capabilities of DIHM, experiments were performed on a chain of Streptococcus bacteria. The chain itself measured 10.5 mm long, and was composed of 6-7 sphero-cylindrical cells (two of the cells in the chain are close to dividing) with diameters in the range 0.6-1 μm. Figures 1a and 1b show the main interface of the reconstruction and rendering software. Examples of the numerical refocusing procedure are seen in Figure 2, where a...

Discussion

The most important step in this experimental protocol is the accurate capture of images from a stable experimental setup. With poor background data, high fidelity reconstruction is next to impossible. It is also important to avoid objective lenses with an internal phase contrast element (visible as a dark annulus when looking through the back of the objective), as this can degrade the reconstructed image. The object of interest should be far enough from the focal plane that a few pairs of diffraction fringes are visible ...

Materials

Name	Company	Catalog Number	Comments
Nikon Eclipse Ti-E inverted microscope	Nikon Corp.
LED	Thorlabs	M660L3	emission wavelength λ=660 nm, linewidth approximately 20 nm
LED power supply	Thorlabs	LEDD1B
Thread Adapter	Thorlabs	SM2T2
Thread Adapter	Thorlabs	SM1A2
Frame Grabber board	EPIX	PIXCI E4
High-Speed CMOS camera	Mikrotron	MC-1362