Matrix Fourier optics enables a compact full-Stokes polarization camera

Design strategy and optimization

If we insist on using a single-layer metasurface obeying Eq. 6 everywhere, then we may not have all light confined to an arbitrary set of diffraction orders acting as polarization state analyzers. But what if some light is allowed to leak into other diffraction orders? Then, the J˜k of some orders could behave as analyzers for arbitrary polarization states with a limited amount of leakage into other diffraction orders (implementing their own J˜k) so that the overall grating formed in Eq. 4 is still unitary everywhere. In other words, the grating can function as a coupled system in which neighboring diffraction orders compensate for desired polarization-dependent behavior on a selected few in a way that preserves overall unitarity for all (x,y).

Here, we seek to design a grating J˜(x,y) that locally obeys Eq. 6 with a set of diffraction orders {ℓ} each having J˜k that are analyzers for an arbitrarily specified set of polarization states {|qk〉} as defined in Eq. 5 with as little light as possible leaking into diffraction orders outside of {ℓ}. This is a question of optimization. We define the quantities Iqk=〈qk|J˜k†J˜k|qk〉, the power on diffraction order k when the preferred polarization |qk〉 is incident, and Iq⊥k=〈qk⊥|J˜k†J˜k|qk⊥〉, the power on diffraction order k when the orthogonal polarization |qk⊥〉 with 〈qk⊥|qk〉=0 is incident. The sum ∑k∈{ℓ}Iqk should be maximized to keep as much light as possible in the orders of interest. Simultaneously, the contrast of each order given by η=(Iqk−Iq⊥k)/(Iqk+Iq⊥k)—the polarization sensitivity of the order to the desired polarization |qk〉—should be as close to unity as possible to constrain the J˜k to act as the desired analyzers (in the sense of Eq. 5). This can be addressed with gradient-descent. We reserve detailed discussion of this optimization to the supplementary text [sections S1 and S2, where we also describe a path to an analytical form of the solution using variational methods (17, 27, 28)].

Many previous works have considered diffraction gratings capable of splitting, and thus analyzing, light on the basis of its polarization state. However, these works have generally taken a scalar approach, seeking to impart opposite blazed grating phase profiles on orthogonal polarization states. Consequently, several individually designed gratings must be interlaced (often called “spatial multiplexing” or “shared aperture”) (29–32) or, equivalently, cascaded in series (1, 33) to create a grating whose orders analyze for any more than two polarization states. This is inherently problematic: These configurations cannot implement polarimetry with any less than six measurements (whereas four is the minimum required), compromising sensor space. Moreover, the polarization states of the analyzers cannot be arbitrarily specified. Interlacing different gratings also introduces unwanted periodicity, mandating loss of light to out-of-plane diffraction. The matrix approach of this work shows that interlacing is not necessary—all functions can be integrated into a single grating—and moreover, affords possibilities not achievable by interlacing. The tetrahedron grating that we will present, as a simple example, is not possible by simple interlacing as none of its four analyzer states are orthogonal to any of the others.

Tetrahedron grating design

Gratings implementing parallel polarization analysis are of practical interest for Stokes polarimetry (24, 34) which requires a minimum of four projective polarization state measurements. For maximum fidelity of Stokes vector reconstruction, these four states should be as distinct from one another as possible, accomplished by choosing analyzer states corresponding to a tetrahedron inscribed in the Poincaré sphere (polarization state space) (35–37). We use the matrix formalism and optimization scheme described above to design a two-dimensional diffraction grating analyzing for these four polarization states on its innermost four orders. The leftmost panel of Fig. 2A gives a map of these orders and the polarization ellipses ({|qk〉}) they are designed to analyze for in k-space. These diffraction orders and desired polarization states can be fed into the aforementioned optimization, yielding a numerical J˜(x,y) that is locally of the form of Eq. 6 (unitary and linearly birefringent). It is then straightforward to map the locally required Jones matrix (ϕx, ϕy, θ) to geometries of actual structures by referring to a library of such structures in a material platform/wavelength regime of interest (8).

• Download high-res image
• Open in new tab
• Download Powerpoint

In this work, that material platform is TiO₂ pillars fabricated with an e-beam lithography and atomic layer deposition technique extensively documented elsewhere (38). This permits operation at technologically important visible wavelengths, aiding the camera application discussed below. We stress, however, that neither TiO₂ nor visible wavelengths are central to this work. The grating unit cells presented here have 11 such elements to a side with an interelement separation of 420 nm so that the diffraction angle at λ=532nm is θD∼6.6° (paraxial). The subwavelength spacing of the pillar elements themselves assures that radiative orders may only stem from the collective unit cell of 11 × 11 elements. The grating unit cell designed by optimization for the tetrahedron case is shown in Fig. 2A (top and bottom, respectively), both in design and as-fabricated (electron micrograph, right). This unit cell is tessellated hundreds of times to create a grating.

Mueller matrix polarimetry and results

In polarization optics, the Jones and Mueller matrices describe polarization-dependent behavior. Of the two, only the Mueller matrix is directly observable, being a description of optical intensities rather than electric fields. To that end, we perform Mueller matrix polarimetry, that is, the experimental determination of the 4 × 4 Mueller matrix M˜(m,n) of each of the four grating orders (m,n) of interest (as a 2D grating, the orders are labeled by two integer indices). The grating is illuminated with laser light at λ=532 nm in several (at least four) different input polarization states with known Stokes vectors {S→in}. As the polarization switches between the different known inputs, a full-Stokes polarimeter can be placed on the order of interest to record the corresponding output Stokes vectors, {S→out}. The matrix M˜(m,n) linking {S→in} to {S→out} can then be numerically determined from the data in the least-squares sense. This is sketched in Fig. 2B and discussed in more detail in supplementary text section S3 (17).

In analyzing the results, it is difficult to make a direct comparison of matrix quantities. Instead, because each order of interest is designed to act as a polarization analyzer, it makes sense to ask the following questions: Do the orders act as analyzers? For which polarizations? And, with what efficiency?

The first question is addressed by Fig. 2C, which plots the polarization contrast of each diffraction order. Each group of bars in Fig. 2C corresponds to the diffraction order designed to analyze for the polarization ellipse shown below it. Each group contains three color-coded bars: one for the numerically optimized J˜(x,y), one for a full-wave simulation, and one for measurement. On the left of Fig. 2D, the contrast (the normalized difference between the intensity on that order when its preferred polarization is incident versus the orthogonal one) is plotted. An ideal polarizer would have 100% contrast. The numerically optimized result predicts near perfection (100%). In a simulation of the grating design, the contrast decreases somewhat, and then again as measured in actuality. However, the four orders of the measured grating all show polarization contrasts in excess of 90%—the designed orders of the grating do act as polarizers (analyzers).

Second, it must be verified that the orders act as analyzers for the polarization states specified in the design. In Fig. 2D, the polarization states for which each grating order has maximum output intensity (as-optimized, in simulation, and as-measured) are plotted on the Poincaré sphere alongside the tetrahedron representing the goal of the design. It can be seen that these analyzer polarizations are close to their desired counterparts (there are no notable mixups in Fig. 2D, whereby one analyzer polarization lands very far away appearing to be closer to another, falsely exaggerating correspondence).

Figure 2, C and D, shows that the metasurface grating implements four arbitrarily specified analyzers in parallel with no polarizers or waveplates, lending validity to the matrix approach underlying its design. We address the question of efficiency in the supplementary text (section S3) (17). The grating’s diffraction efficiency cannot be quantified with a single number—it is polarization dependent. Averaged over all possible input polarizations, efficiency in terms of power diffracted into the four orders of interest over incident power exceeds 50%, high enough to enable practical use.

The data contained in Fig. 2, C and D, are derived from the first row of the Mueller matrix, which dictates the intensity S0 of the outgoing beam. The remainder of the Mueller matrix controls the output beam’s polarization state. This can be visualized by individually operating on a set of input Stokes vectors that uniformly sample the Poincaré sphere (1024 dots) with each order’s measured Mueller matrix M˜(m,n) and plotting the set of Stokes vectors that result as a new, distorted set of dots. This is shown for each of the four orders of interest in Fig. 2E on four spheres. If each behaved as a perfect analyzer (compare to Eq. 5), all polarizations would be mapped into a single point in the distorted diagram at the state corresponding to |pk〉. The grating is not perfect in experimental reality, so the spheres in Fig. 2E contain points distributed over the entire sphere; the extent to which these points are concentrated in one direction and extremely sparse elsewhere illustrates that each diffraction order does indeed act as an analyzer. For each grating order in Fig. 2E, a blue arrow corresponds to the polarization being analyzed (copied from Fig. 2D) (|q(m,n)〉in the notation of Eq. 5). A green arrow shows the polarization on the grating order averaged over the set of incident polarizations. Finally, a red arrow depicts |q(m,n)*〉, which has flipped handedness with respect to |q(m,n)〉 and is thus mirrored about the equator of the sphere. According to the picture presented above, the linear birefringence of the grating elements implies that the output must necessarily take on the form of |q(m,n)*〉. This aspect of the theory is supported by the close overlap of the red and green arrows in each plot of Fig. 2E.

In the supplementary text, results are given for a second grating, an octahedron grating, in which six diffraction orders act as analyzers (17).

Design of an imaging system

The task here is to integrate a metagrating into a photographic imaging system. The tetrahedron grating described above is chosen because it offers full-Stokes determination with only four measurements, the minimum necessary. We developed an imaging system composed of this grating (implemented with a TiO₂ metasurface as above) followed by an aspheric lens (f = 20 mm, whose choice is discussed in supplementary text section S4) and a standard monochrome complementary metal–oxide–semiconductor (CMOS) imaging sensor. This is depicted in Fig. 3A. Relative to Fig. 2A, the grating is rotated by 45° so that each of its orders forms an image of the scene on one quadrant of the imaging sensor. Each quadrant then contains a version of the photograph analyzed along its characteristic polarization—these images can be simultaneously acquired and the Stokes vector S→ reconstructed pixel-wise (Fig. 3B), forming a monochromatic polarization image.

• Download high-res image
• Open in new tab
• Download Powerpoint

Simple ray tracing is used to optimize all aspects of the system: the separation of the grating and the asphere, the separation of the asphere and the imaging sensor, and finally, the grating period (and thus, the diffraction angle θD). The goal of the optimization is to take parallel ray bundles over a ±5° field-of-view (FOV)—which are assumed to emerge from a very distant object—and focus them within the bounds of a quadrant of the sensor. An azimuthally symmetric, polarization-independent (scalar) phase profile is added on top of the matrix grating during the design and is experienced by all diffraction orders. This produces a weak lensing effect that aids in imaging. The design focuses on just one grating order. However, the other three will be imaged to their respective quadrants by default because the system is rotationally symmetric.

A real ray trace of the imaging system is shown in Fig. 3A with a clearer side view in Fig. 3C. The ray colors correspond to parallel bundles incident at different angles on the matrix grating. The optimized grating has N = 10 elements at a 420-nm pitch, yielding θD=7.3° at λ=532 nm. In Fig. 3D a microscope image of the 1.5-mm sample is shown. The sample is surrounded by metal to block stray light and displays Fresnel zones stemming from the weak lensing imparted on top of the grating (whose periodicity is too small to see at this magnification).

Finally, the entire system can be packaged into a prototype for practical use in polarization photography as shown in Fig. 3E with variable focus. The size of this prototype is much larger than is strictly necessary to permit easy optomechanical mounting—as is shown in Fig. 3C, the functional part of the system is only ~2 cm long. An aperture in front of the camera permits control of the FOV so that the four copies of the image do not overlap, and a 10-nm bandpass filter at 532 nm behind the sample prevents the dispersive nature of the grating from interfering with imaging. This is important to note—if a broad band of illuminating wavelengths were introduced to the imaging system, the grating would effectively smear the images together, with spatial and spectral information colocated on the sensor. The bandwidth of dye filters used in conventional color sensors (~100 nm for the Bayer filter) is too wide to effectively address this. For color or hyperspectral polarization imaging, a different approach could be employed, such as the use of a tunable filter at the input or incorporation of the grating into a pushbroom scanning system where only one space dimension is acquired at a time (as is common in remote sensing).

Polarization imaging

The metagrating camera can then be used for practical full-Stokes photography. The raw sensor acquisition approximates Fig. 3B. To form a polarization image, the four image copies must be aligned (registered) to one another, forming the vector I→=[I0  I1  I2  I3]T of measured intensity from the four quadrants at each pixel. If the polarizations analyzed for by each diffraction order are precisely known (calibration), the Stokes vector at each pixel can be computed as S→=A−1I→, where A is a matrix whose rows are these analyzer Stokes vectors. Image registration and polarimetric calibration are addressed simultaneously with an angle-dependent method (supplementary text section S4) (17).

In a polarization image, S→=[S0  S1  S2  S3]T is known at each pixel, which does not admit easy visualization. Instead, images can be formed from scalar quantities derived from the Stokes vector. In the literature of polarization imaging, two parameters of the polarization ellipse are commonly used. The first is the azimuth angle, arctan(S2/S1), which yields the physical orientation of the polarization ellipse. The second is the degree of polarization, given by p=S12+S22+S32/S0, which quantifies the degree to which light is (or is not) fully polarized. Finally, an image can be formed of just S0 which, as the intensity of the light, yields a traditional monochrome photograph.

In Fig. 4, exemplar photographs captured by the polarization imaging system are shown. For each, images of the raw acquisition on the sensor, S0, the azimuth angle, and the degree of polarization (DOP) are shown. In all of these images, the illuminating light is unpolarized and diffuse, that is, incident from all directions. The indoor images in Fig. 4 are taken under diffuse light-emitting diode illumination, whereas outdoor images are acquired in broad daylight. In the raw exposures, a bright disk in the center represents zero-order light that does not interact with the grating and thus forms a defocused version of the image (some stray light is also present).

• Download high-res image
• Open in new tab
• Download Powerpoint

Next, we describe each example. Figure 4A constitutes a simple test—a paper frame holding eight sheets of polarizing film whose axes are arranged radially outward with image-forming light allowed to transmit from behind. A traditional photograph sees no difference between the sheets (S0), but an image of the azimuth accurately reveals their angular orientations. Moreover, the DOP image shows that light passing through the sheets is highly polarized relative to the surrounding surface.

Figure 4, B to E, examine the polarization dependence of specular reflection (47). Unpolarized light becomes partially polarized upon specular reflection in a direction perpendicular to the plane of incidence. In Fig. 4B, a conventional plastic soda bottle is imaged head-on. From an intensity image (S0), the bottle’s curved shape could not be ascertained a posteriori. The azimuth image, however, displays smooth, continuous change around the top of the bottle evidencing its conical shape and could be used in three-dimensional (3D) reconstruction; indeed, the polarized nature of specular reflection has been used as a means of depth imaging (47–50). Figure 4C illustrates the same concept for a more complicated object — the face of one of the authors. Its 3D nature is not discernible from the S0 image (as opposed to, say, a mere printed photograph of a face), but the azimuth image traces its contour and could be used in 3D facial reconstruction.

Figure 4, D and E, depict outdoor scenes. In Fig. 4D, a bicycle is seen parked on a grassy field after rain on the Harvard campus. In front of it is a boundary between grass and asphalt, as well as a puddle. In the azimuth image, a strong delineation is seen between the wet pavement, where the polarization direction is well-defined parallel to the ground, and the grass and puddle, where the azimuth is somewhat random because the reflection is diffuse. Moreover, the azimuth reveals the presence of an asphalt walkway in the rear of the image, which is difficult to see in the intensity image. In Fig. 4E, a row of cars is photographed. Cars illuminated by sunlight are a favorite target in polarization imaging literature because the windshields tend to yield strong polarization signatures. This is seen for all three cars in the azimuth image, where the windshields and auto bodies have definite polarization azimuths and in the DOP image where the windshields are highly polarized relative to the rest of the car and background.

We note, however, that the examples in Fig. 4 make only indirect use of S3, the chiral Stokes component, through the DOP. Many existing polarization cameras, particularly using the division-of-focal-plane approach, are unable to measure S3 (42). Figure 5, however, directly demonstrates the full-Stokes capability of the camera. In Fig. 5, objects are illuminated from behind by light linearly polarized at 45° from a liquid crystal computer display. In each case, the raw sensor acquisition, S0, and S3 are shown. Figure 5A depicts a pair of 3D glasses whose frames contain opposite circular polarizers. This is not visible in the traditional intensity image S0, but is readily seen in the S3 image where each lens has a ±1 value. In Fig. 5B, a piece of acrylic is held in front of the camera, displaying little chiral component. Upon squeezing (Fig. 5C), no change is perceptible in the traditional photograph, but considerable information is now visible in S3 stemming from stress birefringence (the photoelastic effect). This is also evident in Fig. 5D where an injection molded plastic part—a tape dispenser—displays an in-built, complicated distribution of stress in S3 not visible with traditional photography. Visualization of stress fields is an important application of polarization imaging, one that is readily accomplished with the camera presented here.

• Download high-res image
• Open in new tab
• Download Powerpoint

These examples demonstrate powerful applications in machine vision, remote sensing, and other areas. The camera presented here requires no conventional polarization optics, offers simultaneous data acquisition with no moving parts, does not require an imaging sensor specially patterned with micropolarizers, and is compact and potentially mass-producible. Moreover, the camera’s simplicity—just a single grating with an imaging lens—suggests that these gratings could be designed around existing, conventional imaging systems to create polarization-sensitive ones.