A metasurface polarization camera
Imaging the polarization of light scattered from an object provides an additional degree of freedom for gaining information from a scene. Conventional polarimeters can be bulky and usually consist of mechanically moving parts (with a polarizer and analyzer setup rotating to reveal the degree of polarization). Rubin et al. designed a metasurface-based full-Stokes compact polarization camera without conventional polarization optics and without moving parts. The results provide a simplified route for polarization imaging.
Science, this issue p. eaax1839
Structured Abstract
INTRODUCTION
Polarization describes the path along which light’s electric field vector oscillates. An essential quality of electromagnetic radiation, polarization is often omitted in its mathematical treatment. Nevertheless, polarization and its measurement are of interest in almost every area of science, as well as in imaging technology. Traditional cameras are sensitive to intensity alone, but in a variety of contexts, knowledge of polarization can reveal features that are otherwise invisible. Determination of the full-Stokes vector—the most complete description of light’s polarization—necessitates at least four individual measurements. This results in optical systems that are often bulky, reliant on moving parts, and limited in time resolution.
RATIONALE
We introduce a formalism—matrix Fourier optics—for treating polarization in paraxial diffractive optics. This formalism is a powerful generalization of a large body of past work on optical elements in which polarization may vary spatially. Moreover, it suggests a path to realizing many polarization devices in parallel using a single optical element. We can then design diffraction gratings whose orders behave as polarizers for an arbitrarily selected set of polarization states, a new class of optical element. The intensity of light on a set of diffraction orders is then dictated by the polarization of the illuminating light, making these gratings immediately applicable to full-Stokes polarization imaging.
RESULTS
We theoretically investigate these gratings and develop an optimization scheme for their design. Our diffraction gratings were realized with dielectric metasurfaces in which subwavelength, anisotropic structures provide for tunable polarization control at visible frequencies. Characterization of the fabricated gratings shows that they perform as designed. Notably, an arbitrary set of polarizations may be analyzed by a single unit cell, in contrast to past approaches that relied on interlacing of several individually designed diffraction gratings, increasing the flexibility of these devices.
These gratings enable a snapshot, full-Stokes polarization camera—a camera acquiring images in which the full polarization state is known at each pixel—with no traditional polarization optics and no moving parts (see panel A of the figure). Polarized light from a photographic scene is incident on the grating inside of a camera. The polarization is “sorted” by the specially designed subwavelength metasurface grating. When combined with imaging optics (a lens) and a sensor, four copies of the image corresponding to four diffraction orders are formed on the imaging sensor. These copies have each, effectively, passed through a different polarizer whose functions are embedded in the metasurface. The four images can be analyzed pixel-wise to reconstruct the four-element Stokes vector across the scene. Several examples are shown at 532 nm, both indoors and outdoors. The figure depicts an example photograph of two injection-molded plastic pieces, a ruler and a spoon (illuminated by a linearly polarized backlight), that show in-built stresses (see panels C to E of the figure) that are not evident in a traditional photograph (panel B). The camera is compact, requiring only the grating (which is flat and monolithically integrated, handling all the polarization analysis in the system), a lens, and a conventional CMOS (complementary metal–oxide–semiconductor) sensor.
CONCLUSION
Metasurfaces can therefore simplify and compactify the footprint of optical systems relying on polarization optics. Our design formalism suggests future research directions in polarization optics. Moreover, it enables a snapshot, full-Stokes polarization imaging system with no moving parts, no bulk polarization optics, and no specially patterned camera pixels that is not altogether more complicated than a conventional imaging system. Our hardware may enable the adoption of polarization imaging in applications (remote sensing, atmospheric science, machine vision, and even onboard autonomous vehicles) where its complexity might otherwise prove prohibitive.
(A) Photographic scenes contain polarized light that is invisible to traditional, intensity-based imaging, which may reveal hidden features. Our camera uses a metasurface (inset) that directs incident light depending on its polarization, forming four copies of an image that permit polarization reconstruction. (B to E) A plastic ruler and spoon are photographed with the camera. (B) A monochrome intensity image (given by the S0 component of the Stokes vector) does not reveal the rich polarization information stemming from stress-birefringence readily evident in (C) to (E), which show a raw exposure, azimuth of the polarization ellipse, and the S3 component of the Stokes vector that describes circular polarization content, respectively.
Abstract
Recent developments have enabled the practical realization of optical elements in which the polarization of light may vary spatially. We present an extension of Fourier optics—matrix Fourier optics—for understanding these devices and apply it to the design and realization of metasurface gratings implementing arbitrary, parallel polarization analysis. We show how these gratings enable a compact, full-Stokes polarization camera without standard polarization optics. Our single-shot polarization camera requires no moving parts, specially patterned pixels, or conventional polarization optics and may enable the widespread adoption of polarization imaging in machine vision, remote sensing, and other areas.
Polarization refers to the path traversed by light’s electric field vector. As a fundamental characteristic of light, polarization and its measurement are of great interest in almost all areas of science and in imaging technology as well. Traditionally, polarization is spoken of as a property of a beam of light. However, advances in the last few decades in holographic media, micro- and nanofabrication, and other areas have enabled the practical realization of optical elements with tailored, spatially varying polarization properties, even on a subwavelength scale, at optical frequencies. In these devices, the polarization state of light can be varied controllably, point-to-point across an optical element.
Work of this nature now has an extensive literature across several disciplines of optics under various names, including diffractive optics (1, 2), polarization holography (3–6), and nanophotonics (metasurfaces) (7, 8), in addition to appreciable attention from the liquid crystal community (9, 10). These devices exhibit different behavior depending on the polarization of illuminating light, so a natural question that arises is how they can be designed to implement many polarization-dependent functions in parallel.
We consider a generalization of work in this area, which we call matrix Fourier optics, that suggests new design strategies for the design of polarization optics not previously achievable in a single element. In particular, we apply it to the design of metasurface diffraction gratings that can analyze several arbitrarily specified polarization states in parallel.
Matrix Fourier optics
We present a general way of viewing diffraction from a polarization-dependent obstacle. In the plane-wave expansion (or angular spectrum) picture of optics, an electromagnetic disturbance
Each plane wave with its weight
(A) A scalar optical field
This intuitive understanding of optical wave propagation is the basis of the field broadly known as Fourier optics (11) and underpins much of modern optical physics, including imaging and holography. Notably, however, this picture is a scalar one and does not include light’s polarization. Nonetheless, it is possible to conceive of optical elements in which polarization-dependent properties vary as a function of space. Then, instead of the scalar transmission function
We consider that such an optical element with a Jones matrix profile
This matrix Fourier transform is physically significant. Instead of an amplitude and phase, as in traditional Fourier optics, a given direction
In this work, we specialize to polarization-dependent diffraction gratings—that is, elements in which
This way of viewing polarization-dependent diffraction has not seen wide acknowledgment or use in the literature of polarization [with some limited exceptions in diffractive optics (13, 14) and polarization ray-tracing (15, 16)]. In a common strategy, optical elements—especially in the field of metasurfaces—are often designed so that when a given polarization is incident, the output polarization state is uniform across the element and a scalar phase profile is imparted. When the orthogonal polarization is incident, the output polarization state is again uniform and a second phase profile is experienced. In this way, optical elements with different functions for chosen linear, circular, and arbitrary elliptical polarization bases (7, 8, 17) can be realized. Notably, what is widely referred to as the “geometric” or “Pancharatnam-Berry” phase, at least as it relates to this problem, is a subcase of that approach, being used to create scalar phase profiles for circularly polarized light (2, 9, 18–21). In other past work, polarization is understood to vary with space, but a particular incident polarization state is assumed (22, 23). These past design strategies are subcases of the matrix approach presented here.
Parallel polarization analysis by unitary polarization gratings
The formalism presented in the last section is general. Equation 4 provides a prescription for the realization of a diffractive element implementing arbitrary polarization behavior, but does not specify the nature of the desired optical functions (contained in
We focus on cases in which
Equation 5 describes a Jones matrix analyzing for the chosen Jones vector
Once the desired analyzers
Stated differently, such a metasurface can realize a sampled matrix grating where
By inspection, Eq. 6 describes a Jones matrix that is (i) unitary everywhere—that is,
From the completely general matrix picture presented we have made two specific choices that the diffraction orders of the grating should behave as analyzers and that the grating should locally resemble Eq. 6, implying the above two constraints. But are these choices mathematically consistent with one another? That is, for a general set of diffraction orders
This question is a matrix analog of extensive work conducted on phase-only gratings (27, 28). We show in the supplementary text (section S1) that the linear birefringence required by Eq. 6 implies that the
Accepting this constraint guarantees linear birefringence (that is, matrix symmetry) for all
Design strategy and optimization
If we insist on using a single-layer metasurface obeying Eq. 6 everywhere, then we may not have all light confined to an arbitrary set of diffraction orders acting as polarization state analyzers. But what if some light is allowed to leak into other diffraction orders? Then, the
Here, we seek to design a grating
Many previous works have considered diffraction gratings capable of splitting, and thus analyzing, light on the basis of its polarization state. However, these works have generally taken a scalar approach, seeking to impart opposite blazed grating phase profiles on orthogonal polarization states. Consequently, several individually designed gratings must be interlaced (often called “spatial multiplexing” or “shared aperture”) (29–32) or, equivalently, cascaded in series (1, 33) to create a grating whose orders analyze for any more than two polarization states. This is inherently problematic: These configurations cannot implement polarimetry with any less than six measurements (whereas four is the minimum required), compromising sensor space. Moreover, the polarization states of the analyzers cannot be arbitrarily specified. Interlacing different gratings also introduces unwanted periodicity, mandating loss of light to out-of-plane diffraction. The matrix approach of this work shows that interlacing is not necessary—all functions can be integrated into a single grating—and moreover, affords possibilities not achievable by interlacing. The tetrahedron grating that we will present, as a simple example, is not possible by simple interlacing as none of its four analyzer states are orthogonal to any of the others.
Experiment
Tetrahedron grating design
Gratings implementing parallel polarization analysis are of practical interest for Stokes polarimetry (24, 34) which requires a minimum of four projective polarization state measurements. For maximum fidelity of Stokes vector reconstruction, these four states should be as distinct from one another as possible, accomplished by choosing analyzer states corresponding to a tetrahedron inscribed in the Poincaré sphere (polarization state space) (35–37). We use the matrix formalism and optimization scheme described above to design a two-dimensional diffraction grating analyzing for these four polarization states on its innermost four orders. The leftmost panel of Fig. 2A gives a map of these orders and the polarization ellipses
(A) A 2D grating unit cell is designed to analyze four polarization states corresponding to a tetrahedron inscribed in the Poincaré sphere. On the left is a map of the diffraction orders and the polarization ellipses they analyze in k-space. The designed 11 × 11 element grating unit cell containing TiO2 rectangular pillars implementing this at
In this work, that material platform is TiO2 pillars fabricated with an e-beam lithography and atomic layer deposition technique extensively documented elsewhere (38). This permits operation at technologically important visible wavelengths, aiding the camera application discussed below. We stress, however, that neither TiO2 nor visible wavelengths are central to this work. The grating unit cells presented here have 11 such elements to a side with an interelement separation of 420 nm so that the diffraction angle at
Mueller matrix polarimetry and results
In polarization optics, the Jones and Mueller matrices describe polarization-dependent behavior. Of the two, only the Mueller matrix is directly observable, being a description of optical intensities rather than electric fields. To that end, we perform Mueller matrix polarimetry, that is, the experimental determination of the 4 × 4 Mueller matrix
In analyzing the results, it is difficult to make a direct comparison of matrix quantities. Instead, because each order of interest is designed to act as a polarization analyzer, it makes sense to ask the following questions: Do the orders act as analyzers? For which polarizations? And, with what efficiency?
The first question is addressed by Fig. 2C, which plots the polarization contrast of each diffraction order. Each group of bars in Fig. 2C corresponds to the diffraction order designed to analyze for the polarization ellipse shown below it. Each group contains three color-coded bars: one for the numerically optimized
Second, it must be verified that the orders act as analyzers for the polarization states specified in the design. In Fig. 2D, the polarization states for which each grating order has maximum output intensity (as-optimized, in simulation, and as-measured) are plotted on the Poincaré sphere alongside the tetrahedron representing the goal of the design. It can be seen that these analyzer polarizations are close to their desired counterparts (there are no notable mixups in Fig. 2D, whereby one analyzer polarization lands very far away appearing to be closer to another, falsely exaggerating correspondence).
Figure 2, C and D, shows that the metasurface grating implements four arbitrarily specified analyzers in parallel with no polarizers or waveplates, lending validity to the matrix approach underlying its design. We address the question of efficiency in the supplementary text (section S3) (17). The grating’s diffraction efficiency cannot be quantified with a single number—it is polarization dependent. Averaged over all possible input polarizations, efficiency in terms of power diffracted into the four orders of interest over incident power exceeds 50%, high enough to enable practical use.
The data contained in Fig. 2, C and D, are derived from the first row of the Mueller matrix, which dictates the intensity
In the supplementary text, results are given for a second grating, an octahedron grating, in which six diffraction orders act as analyzers (17).
Full-Stokes polarization imaging
We now apply these matrix gratings in an area of great practical interest. If paired with four detectors, the tetrahedron grating presented above could function as a single-component, compact, and integrated full-Stokes polarimeter (a sensor to measure the polarization state of a beam), an area where metagratings (29–32) and integrated approaches (39, 40) have recently attracted considerable interest (34). However, for a variety of applications (41–44), a polarization camera, or imaging polarimeter, is of even more utility. A polarization camera captures the Stokes vector at each point in an image. In the case of the four-element Stokes vector, this necessitates four independent image acquisitions along independent polarization directions, which may be taken sequentially in time (“division-of-time,” limiting temporal resolution and often requiring moving parts), by patterning a focal plane array with micropolarizers (“division-of-focal-plane,” requiring expensive fabrication, usually without offering full-Stokes vector determination, and mandating loss of photons to absorptive micropolarizer elements), or by simultaneous capture of the image along four paths each with independent polarization optics (“division-of-amplitude,” substantially increasing system bulk and complexity) (41).
Grating-based approaches are not new to polarimetry (24, 45) (and imaging polarimetry), particularly in the liquid crystal (46) and, more recently, metasurface literatures (29–32). Owing to the limitations of previous approaches, however, multiple gratings—either cascaded in series or patterned adjacent to one another—are required to implement the measurements necessary for full-Stokes vector determination. The matrix gratings here are free from this complication and full-Stokes imaging can be implemented with a single polarization element, promising wide-ranging applications in machine vision and remote sensing.
Design of an imaging system
The task here is to integrate a metagrating into a photographic imaging system. The tetrahedron grating described above is chosen because it offers full-Stokes determination with only four measurements, the minimum necessary. We developed an imaging system composed of this grating (implemented with a TiO2 metasurface as above) followed by an aspheric lens (f = 20 mm, whose choice is discussed in supplementary text section S4) and a standard monochrome complementary metal–oxide–semiconductor (CMOS) imaging sensor. This is depicted in Fig. 3A. Relative to Fig. 2A, the grating is rotated by 45° so that each of its orders forms an image of the scene on one quadrant of the imaging sensor. Each quadrant then contains a version of the photograph analyzed along its characteristic polarization—these images can be simultaneously acquired and the Stokes vector
(A) A matrix metagrating (as in Fig. 2) is integrated with an aspheric lens to image four diffraction orders onto four quadrants of a CMOS imaging sensor. A ray trace of one diffraction order is shown. The camera is designed to image far-away objects, so each color corresponds to different parallel ray bundles incident on the grating from different angles over a
Simple ray tracing is used to optimize all aspects of the system: the separation of the grating and the asphere, the separation of the asphere and the imaging sensor, and finally, the grating period (and thus, the diffraction angle
A real ray trace of the imaging system is shown in Fig. 3A with a clearer side view in Fig. 3C. The ray colors correspond to parallel bundles incident at different angles on the matrix grating. The optimized grating has N = 10 elements at a 420-nm pitch, yielding
Finally, the entire system can be packaged into a prototype for practical use in polarization photography as shown in Fig. 3E with variable focus. The size of this prototype is much larger than is strictly necessary to permit easy optomechanical mounting—as is shown in Fig. 3C, the functional part of the system is only ~2 cm long. An aperture in front of the camera permits control of the FOV so that the four copies of the image do not overlap, and a 10-nm bandpass filter at 532 nm behind the sample prevents the dispersive nature of the grating from interfering with imaging. This is important to note—if a broad band of illuminating wavelengths were introduced to the imaging system, the grating would effectively smear the images together, with spatial and spectral information colocated on the sensor. The bandwidth of dye filters used in conventional color sensors (~100 nm for the Bayer filter) is too wide to effectively address this. For color or hyperspectral polarization imaging, a different approach could be employed, such as the use of a tunable filter at the input or incorporation of the grating into a pushbroom scanning system where only one space dimension is acquired at a time (as is common in remote sensing).
Polarization imaging
The metagrating camera can then be used for practical full-Stokes photography. The raw sensor acquisition approximates Fig. 3B. To form a polarization image, the four image copies must be aligned (registered) to one another, forming the vector
In a polarization image,
In Fig. 4, exemplar photographs captured by the polarization imaging system are shown. For each, images of the raw acquisition on the sensor,
Indoor (A to C) and outdoor (D and E) photography with the camera depicted in Fig. 3. In each case, the raw unprocessed exposure,
Next, we describe each example. Figure 4A constitutes a simple test—a paper frame holding eight sheets of polarizing film whose axes are arranged radially outward with image-forming light allowed to transmit from behind. A traditional photograph sees no difference between the sheets (
Figure 4, B to E, examine the polarization dependence of specular reflection (47). Unpolarized light becomes partially polarized upon specular reflection in a direction perpendicular to the plane of incidence. In Fig. 4B, a conventional plastic soda bottle is imaged head-on. From an intensity image (
Figure 4, D and E, depict outdoor scenes. In Fig. 4D, a bicycle is seen parked on a grassy field after rain on the Harvard campus. In front of it is a boundary between grass and asphalt, as well as a puddle. In the azimuth image, a strong delineation is seen between the wet pavement, where the polarization direction is well-defined parallel to the ground, and the grass and puddle, where the azimuth is somewhat random because the reflection is diffuse. Moreover, the azimuth reveals the presence of an asphalt walkway in the rear of the image, which is difficult to see in the intensity image. In Fig. 4E, a row of cars is photographed. Cars illuminated by sunlight are a favorite target in polarization imaging literature because the windshields tend to yield strong polarization signatures. This is seen for all three cars in the azimuth image, where the windshields and auto bodies have definite polarization azimuths and in the DOP image where the windshields are highly polarized relative to the rest of the car and background.
We note, however, that the examples in Fig. 4 make only indirect use of
Example imagery from the camera making explicit use of
These examples demonstrate powerful applications in machine vision, remote sensing, and other areas. The camera presented here requires no conventional polarization optics, offers simultaneous data acquisition with no moving parts, does not require an imaging sensor specially patterned with micropolarizers, and is compact and potentially mass-producible. Moreover, the camera’s simplicity—just a single grating with an imaging lens—suggests that these gratings could be designed around existing, conventional imaging systems to create polarization-sensitive ones.
Conclusion
We have introduced a general picture of polarization-dependent diffraction from polarization-sensitive obstacles. This matrix Fourier optics describes optical elements that can enact many polarization-dependent functions in their diffraction patterns. In this work, we applied this picture to the case of periodic gratings analyzing arbitrary polarizations in parallel, characterized these gratings, and showed how they enable a polarization camera with no additional polarization optics, moving parts, or specialized sensors. Practical polarization photography with this camera was demonstrated.
Our work generalizes a large body of research in polarization-sensitive diffractive optics and metasurfaces. Its emphasis is on the collective behavior of many elements at once as expressed in the Fourier transform, rather than the point-by-point consideration of each element alone, and thus advances design strategies in these areas beyond simple phase profiles. Moreover, this work suggests several interesting directions of research in multifunctional polarization optics. For instance, although polarization analyzers (polarizers) have been studied here, the matrix approach is quite general: Gratings implementing a wide variety of polarization operators on their orders [e.g., waveplates, optically active media, and possibly more exotic behavior (51)] could be envisioned. The constraint of linearly birefringent elements is by no means fundamental, and with the freedom afforded by lithographic fabrication, elements with more complex polarization responses (52) or multilayer structures could be used to realize these behaviors. This work illustrates the ability of metasurfaces to markedly simplify the architecture of systems using polarization optics. Gratings enabled by the approach here present a simpler means of full-Stokes polarization imaging that can be easily extended to other imaging systems and wavelengths. These compact, lightweight, and passive devices could enable the widespread adoption of polarization imaging.
Supplementary Materials
science.sciencemag.org/content/365/6448/eaax1839/suppl/DC1
Materials and Methods
Supplementary Text
Figs. S1 to S22
Tables S1 and S2
Movie S1
This is an article distributed under the terms of the Science Journals Default License.