Research Article

Matrix Fourier optics enables a compact full-Stokes polarization camera

See allHide authors and affiliations

Science  05 Jul 2019:
Vol. 365, Issue 6448, eaax1839
DOI: 10.1126/science.aax1839

A metasurface polarization camera

Imaging the polarization of light scattered from an object provides an additional degree of freedom for gaining information from a scene. Conventional polarimeters can be bulky and usually consist of mechanically moving parts (with a polarizer and analyzer setup rotating to reveal the degree of polarization). Rubin et al. designed a metasurface-based full-Stokes compact polarization camera without conventional polarization optics and without moving parts. The results provide a simplified route for polarization imaging.

Science, this issue p. eaax1839

Structured Abstract

INTRODUCTION

Polarization describes the path along which light’s electric field vector oscillates. An essential quality of electromagnetic radiation, polarization is often omitted in its mathematical treatment. Nevertheless, polarization and its measurement are of interest in almost every area of science, as well as in imaging technology. Traditional cameras are sensitive to intensity alone, but in a variety of contexts, knowledge of polarization can reveal features that are otherwise invisible. Determination of the full-Stokes vector—the most complete description of light’s polarization—necessitates at least four individual measurements. This results in optical systems that are often bulky, reliant on moving parts, and limited in time resolution.

RATIONALE

We introduce a formalism—matrix Fourier optics—for treating polarization in paraxial diffractive optics. This formalism is a powerful generalization of a large body of past work on optical elements in which polarization may vary spatially. Moreover, it suggests a path to realizing many polarization devices in parallel using a single optical element. We can then design diffraction gratings whose orders behave as polarizers for an arbitrarily selected set of polarization states, a new class of optical element. The intensity of light on a set of diffraction orders is then dictated by the polarization of the illuminating light, making these gratings immediately applicable to full-Stokes polarization imaging.

RESULTS

We theoretically investigate these gratings and develop an optimization scheme for their design. Our diffraction gratings were realized with dielectric metasurfaces in which subwavelength, anisotropic structures provide for tunable polarization control at visible frequencies. Characterization of the fabricated gratings shows that they perform as designed. Notably, an arbitrary set of polarizations may be analyzed by a single unit cell, in contrast to past approaches that relied on interlacing of several individually designed diffraction gratings, increasing the flexibility of these devices.

These gratings enable a snapshot, full-Stokes polarization camera—a camera acquiring images in which the full polarization state is known at each pixel—with no traditional polarization optics and no moving parts (see panel A of the figure). Polarized light from a photographic scene is incident on the grating inside of a camera. The polarization is “sorted” by the specially designed subwavelength metasurface grating. When combined with imaging optics (a lens) and a sensor, four copies of the image corresponding to four diffraction orders are formed on the imaging sensor. These copies have each, effectively, passed through a different polarizer whose functions are embedded in the metasurface. The four images can be analyzed pixel-wise to reconstruct the four-element Stokes vector across the scene. Several examples are shown at 532 nm, both indoors and outdoors. The figure depicts an example photograph of two injection-molded plastic pieces, a ruler and a spoon (illuminated by a linearly polarized backlight), that show in-built stresses (see panels C to E of the figure) that are not evident in a traditional photograph (panel B). The camera is compact, requiring only the grating (which is flat and monolithically integrated, handling all the polarization analysis in the system), a lens, and a conventional CMOS (complementary metal–oxide–semiconductor) sensor.

CONCLUSION

Metasurfaces can therefore simplify and compactify the footprint of optical systems relying on polarization optics. Our design formalism suggests future research directions in polarization optics. Moreover, it enables a snapshot, full-Stokes polarization imaging system with no moving parts, no bulk polarization optics, and no specially patterned camera pixels that is not altogether more complicated than a conventional imaging system. Our hardware may enable the adoption of polarization imaging in applications (remote sensing, atmospheric science, machine vision, and even onboard autonomous vehicles) where its complexity might otherwise prove prohibitive.

Metasurface-based polarization camera.

(A) Photographic scenes contain polarized light that is invisible to traditional, intensity-based imaging, which may reveal hidden features. Our camera uses a metasurface (inset) that directs incident light depending on its polarization, forming four copies of an image that permit polarization reconstruction. (B to E) A plastic ruler and spoon are photographed with the camera. (B) A monochrome intensity image (given by the S0 component of the Stokes vector) does not reveal the rich polarization information stemming from stress-birefringence readily evident in (C) to (E), which show a raw exposure, azimuth of the polarization ellipse, and the S3 component of the Stokes vector that describes circular polarization content, respectively.

Abstract

Recent developments have enabled the practical realization of optical elements in which the polarization of light may vary spatially. We present an extension of Fourier optics—matrix Fourier optics—for understanding these devices and apply it to the design and realization of metasurface gratings implementing arbitrary, parallel polarization analysis. We show how these gratings enable a compact, full-Stokes polarization camera without standard polarization optics. Our single-shot polarization camera requires no moving parts, specially patterned pixels, or conventional polarization optics and may enable the widespread adoption of polarization imaging in machine vision, remote sensing, and other areas.

Polarization refers to the path traversed by light’s electric field vector. As a fundamental characteristic of light, polarization and its measurement are of great interest in almost all areas of science and in imaging technology as well. Traditionally, polarization is spoken of as a property of a beam of light. However, advances in the last few decades in holographic media, micro- and nanofabrication, and other areas have enabled the practical realization of optical elements with tailored, spatially varying polarization properties, even on a subwavelength scale, at optical frequencies. In these devices, the polarization state of light can be varied controllably, point-to-point across an optical element.

Work of this nature now has an extensive literature across several disciplines of optics under various names, including diffractive optics (1, 2), polarization holography (36), and nanophotonics (metasurfaces) (7, 8), in addition to appreciable attention from the liquid crystal community (9, 10). These devices exhibit different behavior depending on the polarization of illuminating light, so a natural question that arises is how they can be designed to implement many polarization-dependent functions in parallel.

We consider a generalization of work in this area, which we call matrix Fourier optics, that suggests new design strategies for the design of polarization optics not previously achievable in a single element. In particular, we apply it to the design of metasurface diffraction gratings that can analyze several arbitrarily specified polarization states in parallel.

Matrix Fourier optics

We present a general way of viewing diffraction from a polarization-dependent obstacle. In the plane-wave expansion (or angular spectrum) picture of optics, an electromagnetic disturbance U(x,y) in a plane can be considered as being formed from the interference of many plane waves incident at different angles. An individual plane wave in this set is characterized by its in-plane wave-vector (kx,ky) and a weight given by the Fourier transform of the optical field as

A(kx,ky)=+U(x,y)ei(kxx+kyy) dx dy(1)

Each plane wave with its weight A(kx,ky) can be individually propagated forward in space to form the field at a different position (Fig. 1A). This formalism can treat light’s interaction with simple obstacles. If a planar obstacle having a transmission function t(x,y) (e.g., Young’s double slit, or a diffraction grating) is illuminated by an incident field of magnitude E0 (which may also be spatially-varying), the field immediately following the obstacle, t(x,y)E0, can be handled in this way.

Fig. 1 Matrix Fourier Optics.

(A) A scalar optical field U(x,y) has a plane-wave expansion A(kx,ky) obtained by Fourier transform (F) that permits its propagation forward in space. (B) An obstacle implementing a 2 × 2 Jones matrix J˜(x,y) that varies with space (and thus contains four scalar functions, which are schematically represented as different planes here) has a Fourier-domain representation A˜(kx,ky) that is also a Jones matrix. A˜(kx,ky) encodes the behavior of direction (kx,ky) as a function of incident polarization. (C) If J˜(x,y) represents a periodic grating, its orders are described by Jones matrix coefficients J˜(kx,ky), each describing an independent polarization device, for instance, a configuration of birefringent plates. The grating implements many such devices in parallel.

This intuitive understanding of optical wave propagation is the basis of the field broadly known as Fourier optics (11) and underpins much of modern optical physics, including imaging and holography. Notably, however, this picture is a scalar one and does not include light’s polarization. Nonetheless, it is possible to conceive of optical elements in which polarization-dependent properties vary as a function of space. Then, instead of the scalar transmission function t(x,y), we may consider that an optical element is associated with a matrix-valued function J˜(x,y). Here, J˜(x,y) denotes the local Jones matrix (12), the way in which the Jones polarization vector is modified, at each point. (In this work, matrix quantities are denoted by a tilde.)

We consider that such an optical element with a Jones matrix profile J˜(x,y) is illuminated by a field described by the Jones vector ket |E0 (in acknowledgment of its polarized nature). The field immediately after the obstacle is then given by J˜(x,y)|E0. This field, too, can be propagated by plane-wave expansion as given by the Fourier transform|A(kx,ky)=+J˜(x,y)|E0ei(kxx+kyy) dx dy(2)which is now vector-valued. If the incident wave is a normally incident, uniform plane-wave, it will carry no space-dependence and can be removed from the integral. The only integral to be evaluated, then, is given byA˜(kx,ky)=+J˜(x,y)ei(kxx+kyy) dx dy(3)which is a Fourier integral distributed across each of the four elements of the 2 × 2 Jones matrix yielding a Fourier coefficient A˜(kx,ky) which is itself a Jones matrix (Fig. 1B).

This matrix Fourier transform is physically significant. Instead of an amplitude and phase, as in traditional Fourier optics, a given direction (kx,ky) is associated with a polarization-dependent behavior given by the Jones matrix operator A˜(kx,ky). In a sense, this description decouples the optical element described by J˜(x,y) from the polarization of the illuminating light: The interaction of all possible incident polarizations with J˜(x,y) is handled at once by the matrix Fourier transform A˜(kx,ky).

In this work, we specialize to polarization-dependent diffraction gratings—that is, elements in which J˜(x,y) is a periodic function and the angular spectrum A˜(kx,ky) is discrete, yielding diffraction orders. Diffraction gratings simply reveal the consequences of this matrix Fourier optics, in particular, that it can be inverted and used as a design strategy for multifunctional polarization optics. Suppose there is a set of desired polarization devices to be realized (e.g., polarizers, waveplates, optically active elements, or any behavior that can be described by a Jones matrix) having the Jones matrices {J˜k} and a set of diffraction orders {}. The Fourier series expansionJ˜(x,y)=k{}J˜kei(kxx+kyy)(4)represents a single optical element realizing all of these (potentially complicated) polarization-dependent functions in parallel with each order implementing its own polarization device (Fig. 1C). In the case of a grating, the Fourier transform Eq. 3 becomes a Fourier series in Eq. 4 with the optical elements {J˜k} as coefficients. We refer to such a diffraction grating as a matrix grating.

This way of viewing polarization-dependent diffraction has not seen wide acknowledgment or use in the literature of polarization [with some limited exceptions in diffractive optics (13, 14) and polarization ray-tracing (15, 16)]. In a common strategy, optical elements—especially in the field of metasurfaces—are often designed so that when a given polarization is incident, the output polarization state is uniform across the element and a scalar phase profile is imparted. When the orthogonal polarization is incident, the output polarization state is again uniform and a second phase profile is experienced. In this way, optical elements with different functions for chosen linear, circular, and arbitrary elliptical polarization bases (7, 8, 17) can be realized. Notably, what is widely referred to as the “geometric” or “Pancharatnam-Berry” phase, at least as it relates to this problem, is a subcase of that approach, being used to create scalar phase profiles for circularly polarized light (2, 9, 1821). In other past work, polarization is understood to vary with space, but a particular incident polarization state is assumed (22, 23). These past design strategies are subcases of the matrix approach presented here.

Parallel polarization analysis by unitary polarization gratings

The formalism presented in the last section is general. Equation 4 provides a prescription for the realization of a diffractive element implementing arbitrary polarization behavior, but does not specify the nature of the desired optical functions (contained in {J˜k}) or how the resulting J˜(x,y) should be practically realized.

We focus on cases in which {J˜k}, the behaviors implemented by the grating orders, are polarization analyzers. This choice is not a fundamental one, but it is highly relevant for practical applications: Analyzers are perhaps the most fundamental polarization element and, moreover, devices that project light onto different states of polarization are of practical use in polarimetry, the measurement of light’s polarization state (24). A diffraction order k behaving as a polarizer can be described by a Jones matrix that is a dyadic (outer product) expressed as

J˜k=ak|pkqk|(5)

Equation 5 describes a Jones matrix analyzing for the chosen Jones vector |qk in accordance with Malus’ law: When |qk is incident, intensity transmission is maximum. If instead the orthogonal polarization |qk with qk|qk=0 arrives, the output light is quenched. The light emerging from J˜k will carry the polarization state |pk. (A traditional polarizer familiar from laboratory experience has |pk=|qk). Finally, ak is a complex (scalar-valued) weight.

Once the desired analyzers {J˜k} are specified (Eq. 5), the grating J˜(x,y) can be derived by Eq. 4. What physical optical element should implement the resulting J˜(x,y)? A highly convenient realization medium takes the form of metasurfaces (25, 26)—subwavelength-spaced arrays of nanophotonic phase shifters—composed of dielectric pillars possessing form birefringence (7, 8). Locally, the Jones matrix of these metasurfaces may be well-approximated by a linearly birefringent waveplate (8) given by

J˜(x,y)=R(θ(x,y))(eiϕx(x,y)00eiϕy(x,y))R(θ(x,y))(6)

Stated differently, such a metasurface can realize a sampled matrix grating where J˜(x,y) is locally of the form of Eq. 6. The use of dielectric metasurfaces obeying Eq. 6 is, again, not a fundamental choice, but an especially convenient one: ϕx, ϕy, and θ are all easily and continuously adjusted by varying the dimensions and angular orientation of a simple dielectric pillar, easily fabricated lithographically (R is a 2 × 2 rotation matrix). We refer to metagratings structured from these elements simply as “gratings,” but in contrast to more generic diffraction gratings, it should be understood that the gratings here possess special polarization properties owing to their subwavelength features.

By inspection, Eq. 6 describes a Jones matrix that is (i) unitary everywhere—that is, J˜J˜=1 for all (x,y), with 1 the 2 × 2 identity matrix—and more specifically, (ii) linearly birefringent, having linear polarizations as its eigenvectors for all (x,y).

From the completely general matrix picture presented we have made two specific choices that the diffraction orders of the grating should behave as analyzers and that the grating should locally resemble Eq. 6, implying the above two constraints. But are these choices mathematically consistent with one another? That is, for a general set of diffraction orders {} implementing the polarization analyzers J˜k=ak|pkqk| for k{}, will the J˜(x,y) mandated by Eq. 4 obey the form of Eq. 6 for all (x,y)?

This question is a matrix analog of extensive work conducted on phase-only gratings (27, 28). We show in the supplementary text (section S1) that the linear birefringence required by Eq. 6 implies that the J˜k must be symmetric mandating that each take the form J˜k=ak|qk*qk|. That is, the output polarization of each analyzer must be the complex conjugate of the Jones vector being analyzed for; this is equivalent to a mirroring about the equatorial plane of the Poincaré sphere, a change in sign of the third (chiral) Stokes component. Physically, this means that the analyzer leaves the polarization ellipse’s shape unchanged while reversing the handedness of its rotation, a generalization of a key conclusion of (7) and (8).

Accepting this constraint guarantees linear birefringence (that is, matrix symmetry) for all (x,y), but not necessarily unitarity everywhere. This unitarity restriction in particular implies that, if we insist on a matrix grating with exactly n orders that are polarization state analyzers, the form of Eq. 6 can only be matched everywhere if n = 2 and the polarizations analyzed for are strictly orthogonal (as proven in supplementary text section S1) (17).

Design strategy and optimization

If we insist on using a single-layer metasurface obeying Eq. 6 everywhere, then we may not have all light confined to an arbitrary set of diffraction orders acting as polarization state analyzers. But what if some light is allowed to leak into other diffraction orders? Then, the J˜k of some orders could behave as analyzers for arbitrary polarization states with a limited amount of leakage into other diffraction orders (implementing their own J˜k) so that the overall grating formed in Eq. 4 is still unitary everywhere. In other words, the grating can function as a coupled system in which neighboring diffraction orders compensate for desired polarization-dependent behavior on a selected few in a way that preserves overall unitarity for all (x,y).

Here, we seek to design a grating J˜(x,y) that locally obeys Eq. 6 with a set of diffraction orders {} each having J˜k that are analyzers for an arbitrarily specified set of polarization states {|qk} as defined in Eq. 5 with as little light as possible leaking into diffraction orders outside of {}. This is a question of optimization. We define the quantities Iqk=qk|J˜kJ˜k|qk, the power on diffraction order k when the preferred polarization |qk is incident, and Iqk=qk|J˜kJ˜k|qk, the power on diffraction order k when the orthogonal polarization |qk with qk|qk=0 is incident. The sum k{}Iqk should be maximized to keep as much light as possible in the orders of interest. Simultaneously, the contrast of each order given by η=(IqkIqk)/(Iqk+Iqk)—the polarization sensitivity of the order to the desired polarization |qk—should be as close to unity as possible to constrain the J˜k to act as the desired analyzers (in the sense of Eq. 5). This can be addressed with gradient-descent. We reserve detailed discussion of this optimization to the supplementary text [sections S1 and S2, where we also describe a path to an analytical form of the solution using variational methods (17, 27, 28)].

Many previous works have considered diffraction gratings capable of splitting, and thus analyzing, light on the basis of its polarization state. However, these works have generally taken a scalar approach, seeking to impart opposite blazed grating phase profiles on orthogonal polarization states. Consequently, several individually designed gratings must be interlaced (often called “spatial multiplexing” or “shared aperture”) (2932) or, equivalently, cascaded in series (1, 33) to create a grating whose orders analyze for any more than two polarization states. This is inherently problematic: These configurations cannot implement polarimetry with any less than six measurements (whereas four is the minimum required), compromising sensor space. Moreover, the polarization states of the analyzers cannot be arbitrarily specified. Interlacing different gratings also introduces unwanted periodicity, mandating loss of light to out-of-plane diffraction. The matrix approach of this work shows that interlacing is not necessary—all functions can be integrated into a single grating—and moreover, affords possibilities not achievable by interlacing. The tetrahedron grating that we will present, as a simple example, is not possible by simple interlacing as none of its four analyzer states are orthogonal to any of the others.

Experiment

Tetrahedron grating design

Gratings implementing parallel polarization analysis are of practical interest for Stokes polarimetry (24, 34) which requires a minimum of four projective polarization state measurements. For maximum fidelity of Stokes vector reconstruction, these four states should be as distinct from one another as possible, accomplished by choosing analyzer states corresponding to a tetrahedron inscribed in the Poincaré sphere (polarization state space) (3537). We use the matrix formalism and optimization scheme described above to design a two-dimensional diffraction grating analyzing for these four polarization states on its innermost four orders. The leftmost panel of Fig. 2A gives a map of these orders and the polarization ellipses ({|qk}) they are designed to analyze for in k-space. These diffraction orders and desired polarization states can be fed into the aforementioned optimization, yielding a numerical J˜(x,y) that is locally of the form of Eq. 6 (unitary and linearly birefringent). It is then straightforward to map the locally required Jones matrix (ϕx, ϕy, θ) to geometries of actual structures by referring to a library of such structures in a material platform/wavelength regime of interest (8).

Fig. 2 Matrix gratings for arbitrary parallel polarization analysis.

(A) A 2D grating unit cell is designed to analyze four polarization states corresponding to a tetrahedron inscribed in the Poincaré sphere. On the left is a map of the diffraction orders and the polarization ellipses they analyze in k-space. The designed 11 × 11 element grating unit cell containing TiO2 rectangular pillars implementing this at λ=532 nm is shown in design (middle) and as-fabricated [scanning electron micrograph (SEM), right]. Scale bar, 1 μm. (B) The grating is illuminated by light whose polarization is varied while recording the output polarization on a single diffraction order with a full-Stokes polarimeter permitting reconstruction of the order’s 4 × 4 experimental Mueller matrix M˜(m,n). (C) The polarization contrast of each order is shown. Each order is labeled by the polarization ellipse it analyzes, and results from the analytical grating design (as-optimized, red), a full-wave simulation of the grating (blue), and experiment (green) are given. (D) The polarizations analyzed by each order [for which they have the contrasts given in (C)] of the tetrahedron grating are shown on the Poincaré sphere alongside a tetrahedron indicating the desired analyzer polarizations as predicted by the optimization, a full-wave simulation, and as-measured. (E) A set of 1024 uniformly sampled input polarization states on the Poincaré sphere can be operated on by an experimentally determined Mueller matrix M˜(m,n) to form a distorted set of states depicting output polarization as a function of input. Each sphere on the right represents the behavior of one of the four engineered grating orders, whose designed analyzer polarizations are shown. On each, a blue arrow denotes the measured analyzer polarization |q(m,n) from (C), a red arrow the equator-mirrored polarization |q(m,n)*, and a green arrow the average output polarization over all points which, according to the formalism here, should overlap with the red one. (In some cases, the overlap of red and green arrows is too close to permit visual discrimination.)

In this work, that material platform is TiO2 pillars fabricated with an e-beam lithography and atomic layer deposition technique extensively documented elsewhere (38). This permits operation at technologically important visible wavelengths, aiding the camera application discussed below. We stress, however, that neither TiO2 nor visible wavelengths are central to this work. The grating unit cells presented here have 11 such elements to a side with an interelement separation of 420 nm so that the diffraction angle at λ=532nm is θD6.6° (paraxial). The subwavelength spacing of the pillar elements themselves assures that radiative orders may only stem from the collective unit cell of 11 × 11 elements. The grating unit cell designed by optimization for the tetrahedron case is shown in Fig. 2A (top and bottom, respectively), both in design and as-fabricated (electron micrograph, right). This unit cell is tessellated hundreds of times to create a grating.

Mueller matrix polarimetry and results

In polarization optics, the Jones and Mueller matrices describe polarization-dependent behavior. Of the two, only the Mueller matrix is directly observable, being a description of optical intensities rather than electric fields. To that end, we perform Mueller matrix polarimetry, that is, the experimental determination of the 4 × 4 Mueller matrix M˜(m,n) of each of the four grating orders (m,n) of interest (as a 2D grating, the orders are labeled by two integer indices). The grating is illuminated with laser light at λ=532 nm in several (at least four) different input polarization states with known Stokes vectors {Sin}. As the polarization switches between the different known inputs, a full-Stokes polarimeter can be placed on the order of interest to record the corresponding output Stokes vectors, {Sout}. The matrix M˜(m,n) linking {Sin} to {Sout} can then be numerically determined from the data in the least-squares sense. This is sketched in Fig. 2B and discussed in more detail in supplementary text section S3 (17).

In analyzing the results, it is difficult to make a direct comparison of matrix quantities. Instead, because each order of interest is designed to act as a polarization analyzer, it makes sense to ask the following questions: Do the orders act as analyzers? For which polarizations? And, with what efficiency?

The first question is addressed by Fig. 2C, which plots the polarization contrast of each diffraction order. Each group of bars in Fig. 2C corresponds to the diffraction order designed to analyze for the polarization ellipse shown below it. Each group contains three color-coded bars: one for the numerically optimized J˜(x,y), one for a full-wave simulation, and one for measurement. On the left of Fig. 2D, the contrast (the normalized difference between the intensity on that order when its preferred polarization is incident versus the orthogonal one) is plotted. An ideal polarizer would have 100% contrast. The numerically optimized result predicts near perfection (100%). In a simulation of the grating design, the contrast decreases somewhat, and then again as measured in actuality. However, the four orders of the measured grating all show polarization contrasts in excess of 90%—the designed orders of the grating do act as polarizers (analyzers).

Second, it must be verified that the orders act as analyzers for the polarization states specified in the design. In Fig. 2D, the polarization states for which each grating order has maximum output intensity (as-optimized, in simulation, and as-measured) are plotted on the Poincaré sphere alongside the tetrahedron representing the goal of the design. It can be seen that these analyzer polarizations are close to their desired counterparts (there are no notable mixups in Fig. 2D, whereby one analyzer polarization lands very far away appearing to be closer to another, falsely exaggerating correspondence).

Figure 2, C and D, shows that the metasurface grating implements four arbitrarily specified analyzers in parallel with no polarizers or waveplates, lending validity to the matrix approach underlying its design. We address the question of efficiency in the supplementary text (section S3) (17). The grating’s diffraction efficiency cannot be quantified with a single number—it is polarization dependent. Averaged over all possible input polarizations, efficiency in terms of power diffracted into the four orders of interest over incident power exceeds 50%, high enough to enable practical use.

The data contained in Fig. 2, C and D, are derived from the first row of the Mueller matrix, which dictates the intensity S0 of the outgoing beam. The remainder of the Mueller matrix controls the output beam’s polarization state. This can be visualized by individually operating on a set of input Stokes vectors that uniformly sample the Poincaré sphere (1024 dots) with each order’s measured Mueller matrix M˜(m,n) and plotting the set of Stokes vectors that result as a new, distorted set of dots. This is shown for each of the four orders of interest in Fig. 2E on four spheres. If each behaved as a perfect analyzer (compare to Eq. 5), all polarizations would be mapped into a single point in the distorted diagram at the state corresponding to |pk. The grating is not perfect in experimental reality, so the spheres in Fig. 2E contain points distributed over the entire sphere; the extent to which these points are concentrated in one direction and extremely sparse elsewhere illustrates that each diffraction order does indeed act as an analyzer. For each grating order in Fig. 2E, a blue arrow corresponds to the polarization being analyzed (copied from Fig. 2D) (|q(m,n)in the notation of Eq. 5). A green arrow shows the polarization on the grating order averaged over the set of incident polarizations. Finally, a red arrow depicts |q(m,n)*, which has flipped handedness with respect to |q(m,n) and is thus mirrored about the equator of the sphere. According to the picture presented above, the linear birefringence of the grating elements implies that the output must necessarily take on the form of |q(m,n)*. This aspect of the theory is supported by the close overlap of the red and green arrows in each plot of Fig. 2E.

In the supplementary text, results are given for a second grating, an octahedron grating, in which six diffraction orders act as analyzers (17).

Full-Stokes polarization imaging

We now apply these matrix gratings in an area of great practical interest. If paired with four detectors, the tetrahedron grating presented above could function as a single-component, compact, and integrated full-Stokes polarimeter (a sensor to measure the polarization state of a beam), an area where metagratings (2932) and integrated approaches (39, 40) have recently attracted considerable interest (34). However, for a variety of applications (4144), a polarization camera, or imaging polarimeter, is of even more utility. A polarization camera captures the Stokes vector at each point in an image. In the case of the four-element Stokes vector, this necessitates four independent image acquisitions along independent polarization directions, which may be taken sequentially in time (“division-of-time,” limiting temporal resolution and often requiring moving parts), by patterning a focal plane array with micropolarizers (“division-of-focal-plane,” requiring expensive fabrication, usually without offering full-Stokes vector determination, and mandating loss of photons to absorptive micropolarizer elements), or by simultaneous capture of the image along four paths each with independent polarization optics (“division-of-amplitude,” substantially increasing system bulk and complexity) (41).

Grating-based approaches are not new to polarimetry (24, 45) (and imaging polarimetry), particularly in the liquid crystal (46) and, more recently, metasurface literatures (2932). Owing to the limitations of previous approaches, however, multiple gratings—either cascaded in series or patterned adjacent to one another—are required to implement the measurements necessary for full-Stokes vector determination. The matrix gratings here are free from this complication and full-Stokes imaging can be implemented with a single polarization element, promising wide-ranging applications in machine vision and remote sensing.

Design of an imaging system

The task here is to integrate a metagrating into a photographic imaging system. The tetrahedron grating described above is chosen because it offers full-Stokes determination with only four measurements, the minimum necessary. We developed an imaging system composed of this grating (implemented with a TiO2 metasurface as above) followed by an aspheric lens (f = 20 mm, whose choice is discussed in supplementary text section S4) and a standard monochrome complementary metal–oxide–semiconductor (CMOS) imaging sensor. This is depicted in Fig. 3A. Relative to Fig. 2A, the grating is rotated by 45° so that each of its orders forms an image of the scene on one quadrant of the imaging sensor. Each quadrant then contains a version of the photograph analyzed along its characteristic polarization—these images can be simultaneously acquired and the Stokes vector S reconstructed pixel-wise (Fig. 3B), forming a monochromatic polarization image.

Fig. 3 Metagrating full-Stokes polarization camera.

(A) A matrix metagrating (as in Fig. 2) is integrated with an aspheric lens to image four diffraction orders onto four quadrants of a CMOS imaging sensor. A ray trace of one diffraction order is shown. The camera is designed to image far-away objects, so each color corresponds to different parallel ray bundles incident on the grating from different angles over a ±5° FOV. Not shown: A 10-nm bandpass filter at 532 nm and an aperture in front of the grating to limit the FOV to prevent overlap of the four subimages. (B) Each copy of the image on each quadrant has been analyzed along a different polarization. Pixel-wise differences in intensity can be used to synthesize a single polarization image of the scene in which the full Stokes vector is known at each point. (C) A clearer side view of the ray trace in (A). Blue, orange, and green correspond to ray bundles incident at +5°, 0°, and −5°, respectively. (D) Optical microscope image of the grating sample, which is 1.5 mm in diameter and shows Fresnel zones owing to the weak, polarization-independent lensing effect imposed on top of the metagrating. An SEM inset shows the subwavelength, form-birefringent TiO2 pillars comprising the metasurface. (E) The imaging system in (A) can be packaged into a practical, portable prototype with adjustable focus. The essential part of the camera, as shown in (C), is about 2 cm long. The prototype here is larger for ease of optomechanical mounting.

Simple ray tracing is used to optimize all aspects of the system: the separation of the grating and the asphere, the separation of the asphere and the imaging sensor, and finally, the grating period (and thus, the diffraction angle θD). The goal of the optimization is to take parallel ray bundles over a ±5° field-of-view (FOV)—which are assumed to emerge from a very distant object—and focus them within the bounds of a quadrant of the sensor. An azimuthally symmetric, polarization-independent (scalar) phase profile is added on top of the matrix grating during the design and is experienced by all diffraction orders. This produces a weak lensing effect that aids in imaging. The design focuses on just one grating order. However, the other three will be imaged to their respective quadrants by default because the system is rotationally symmetric.

A real ray trace of the imaging system is shown in Fig. 3A with a clearer side view in Fig. 3C. The ray colors correspond to parallel bundles incident at different angles on the matrix grating. The optimized grating has N = 10 elements at a 420-nm pitch, yielding θD=7.3° at λ=532 nm. In Fig. 3D a microscope image of the 1.5-mm sample is shown. The sample is surrounded by metal to block stray light and displays Fresnel zones stemming from the weak lensing imparted on top of the grating (whose periodicity is too small to see at this magnification).

Finally, the entire system can be packaged into a prototype for practical use in polarization photography as shown in Fig. 3E with variable focus. The size of this prototype is much larger than is strictly necessary to permit easy optomechanical mounting—as is shown in Fig. 3C, the functional part of the system is only ~2 cm long. An aperture in front of the camera permits control of the FOV so that the four copies of the image do not overlap, and a 10-nm bandpass filter at 532 nm behind the sample prevents the dispersive nature of the grating from interfering with imaging. This is important to note—if a broad band of illuminating wavelengths were introduced to the imaging system, the grating would effectively smear the images together, with spatial and spectral information colocated on the sensor. The bandwidth of dye filters used in conventional color sensors (~100 nm for the Bayer filter) is too wide to effectively address this. For color or hyperspectral polarization imaging, a different approach could be employed, such as the use of a tunable filter at the input or incorporation of the grating into a pushbroom scanning system where only one space dimension is acquired at a time (as is common in remote sensing).

Polarization imaging

The metagrating camera can then be used for practical full-Stokes photography. The raw sensor acquisition approximates Fig. 3B. To form a polarization image, the four image copies must be aligned (registered) to one another, forming the vector I=[I0  I1  I2  I3]T of measured intensity from the four quadrants at each pixel. If the polarizations analyzed for by each diffraction order are precisely known (calibration), the Stokes vector at each pixel can be computed as S=A1I, where A is a matrix whose rows are these analyzer Stokes vectors. Image registration and polarimetric calibration are addressed simultaneously with an angle-dependent method (supplementary text section S4) (17).

In a polarization image, S=[S0  S1  S2  S3]T is known at each pixel, which does not admit easy visualization. Instead, images can be formed from scalar quantities derived from the Stokes vector. In the literature of polarization imaging, two parameters of the polarization ellipse are commonly used. The first is the azimuth angle, arctan(S2/S1), which yields the physical orientation of the polarization ellipse. The second is the degree of polarization, given by p=S12+S22+S32/S0, which quantifies the degree to which light is (or is not) fully polarized. Finally, an image can be formed of just S0 which, as the intensity of the light, yields a traditional monochrome photograph.

In Fig. 4, exemplar photographs captured by the polarization imaging system are shown. For each, images of the raw acquisition on the sensor, S0, the azimuth angle, and the degree of polarization (DOP) are shown. In all of these images, the illuminating light is unpolarized and diffuse, that is, incident from all directions. The indoor images in Fig. 4 are taken under diffuse light-emitting diode illumination, whereas outdoor images are acquired in broad daylight. In the raw exposures, a bright disk in the center represents zero-order light that does not interact with the grating and thus forms a defocused version of the image (some stray light is also present).

Fig. 4 Full-Stokes polarization imagery.

Indoor (A to C) and outdoor (D and E) photography with the camera depicted in Fig. 3. In each case, the raw unprocessed exposure, S0 (the traditional monochrome intensity image), the azimuth of the polarization ellipse [in degrees, given by arctan(S2/S1)], and the degree of polarization (DOP, given by S12+S22+S32/S0) are shown. See text for detail on each case. Indoor images were acquired with exposures on the order of 100 ms, outdoor images with exposures on the order of 10 ms. In both cases, polarization imagery can be acquired at video framerate. The bright disk in the center of each raw exposure is zero-order light that does not interact with the grating and thus forms a defocused image of the scene. All images are at a single color (green, λ532 nm).

Next, we describe each example. Figure 4A constitutes a simple test—a paper frame holding eight sheets of polarizing film whose axes are arranged radially outward with image-forming light allowed to transmit from behind. A traditional photograph sees no difference between the sheets (S0), but an image of the azimuth accurately reveals their angular orientations. Moreover, the DOP image shows that light passing through the sheets is highly polarized relative to the surrounding surface.

Figure 4, B to E, examine the polarization dependence of specular reflection (47). Unpolarized light becomes partially polarized upon specular reflection in a direction perpendicular to the plane of incidence. In Fig. 4B, a conventional plastic soda bottle is imaged head-on. From an intensity image (S0), the bottle’s curved shape could not be ascertained a posteriori. The azimuth image, however, displays smooth, continuous change around the top of the bottle evidencing its conical shape and could be used in three-dimensional (3D) reconstruction; indeed, the polarized nature of specular reflection has been used as a means of depth imaging (4750). Figure 4C illustrates the same concept for a more complicated object — the face of one of the authors. Its 3D nature is not discernible from the S0 image (as opposed to, say, a mere printed photograph of a face), but the azimuth image traces its contour and could be used in 3D facial reconstruction.

Figure 4, D and E, depict outdoor scenes. In Fig. 4D, a bicycle is seen parked on a grassy field after rain on the Harvard campus. In front of it is a boundary between grass and asphalt, as well as a puddle. In the azimuth image, a strong delineation is seen between the wet pavement, where the polarization direction is well-defined parallel to the ground, and the grass and puddle, where the azimuth is somewhat random because the reflection is diffuse. Moreover, the azimuth reveals the presence of an asphalt walkway in the rear of the image, which is difficult to see in the intensity image. In Fig. 4E, a row of cars is photographed. Cars illuminated by sunlight are a favorite target in polarization imaging literature because the windshields tend to yield strong polarization signatures. This is seen for all three cars in the azimuth image, where the windshields and auto bodies have definite polarization azimuths and in the DOP image where the windshields are highly polarized relative to the rest of the car and background.

We note, however, that the examples in Fig. 4 make only indirect use of S3, the chiral Stokes component, through the DOP. Many existing polarization cameras, particularly using the division-of-focal-plane approach, are unable to measure S3 (42). Figure 5, however, directly demonstrates the full-Stokes capability of the camera. In Fig. 5, objects are illuminated from behind by light linearly polarized at 45° from a liquid crystal computer display. In each case, the raw sensor acquisition, S0, and S3 are shown. Figure 5A depicts a pair of 3D glasses whose frames contain opposite circular polarizers. This is not visible in the traditional intensity image S0, but is readily seen in the S3 image where each lens has a ±1 value. In Fig. 5B, a piece of acrylic is held in front of the camera, displaying little chiral component. Upon squeezing (Fig. 5C), no change is perceptible in the traditional photograph, but considerable information is now visible in S3 stemming from stress birefringence (the photoelastic effect). This is also evident in Fig. 5D where an injection molded plastic part—a tape dispenser—displays an in-built, complicated distribution of stress in S3 not visible with traditional photography. Visualization of stress fields is an important application of polarization imaging, one that is readily accomplished with the camera presented here.

Fig. 5 Polarization imaging of S3.

Example imagery from the camera making explicit use of S3, part of its full-Stokes capability. All images are acquired with a backlight linearly polarized at 45°. In each example, raw sensor acquisition, S0 (traditional intensity image), and S3 are shown. (A) 3D glasses are seen to contain opposite circular polarizers, invisible to the traditional intensity image. (B and C) A laser-cut acrylic piece is stressed by hand-squeezing and displays stress birefringence evident in the S3 image. (D) An injection-molded plastic part (a tape dispenser) has complex, in-built stresses that are visible in the S3 channel of the polarization image. Edge artifacts in these images and those in Fig. 4 are discussed in the supplementary text (section S4) (17).

These examples demonstrate powerful applications in machine vision, remote sensing, and other areas. The camera presented here requires no conventional polarization optics, offers simultaneous data acquisition with no moving parts, does not require an imaging sensor specially patterned with micropolarizers, and is compact and potentially mass-producible. Moreover, the camera’s simplicity—just a single grating with an imaging lens—suggests that these gratings could be designed around existing, conventional imaging systems to create polarization-sensitive ones.

Conclusion

We have introduced a general picture of polarization-dependent diffraction from polarization-sensitive obstacles. This matrix Fourier optics describes optical elements that can enact many polarization-dependent functions in their diffraction patterns. In this work, we applied this picture to the case of periodic gratings analyzing arbitrary polarizations in parallel, characterized these gratings, and showed how they enable a polarization camera with no additional polarization optics, moving parts, or specialized sensors. Practical polarization photography with this camera was demonstrated.

Our work generalizes a large body of research in polarization-sensitive diffractive optics and metasurfaces. Its emphasis is on the collective behavior of many elements at once as expressed in the Fourier transform, rather than the point-by-point consideration of each element alone, and thus advances design strategies in these areas beyond simple phase profiles. Moreover, this work suggests several interesting directions of research in multifunctional polarization optics. For instance, although polarization analyzers (polarizers) have been studied here, the matrix approach is quite general: Gratings implementing a wide variety of polarization operators on their orders [e.g., waveplates, optically active media, and possibly more exotic behavior (51)] could be envisioned. The constraint of linearly birefringent elements is by no means fundamental, and with the freedom afforded by lithographic fabrication, elements with more complex polarization responses (52) or multilayer structures could be used to realize these behaviors. This work illustrates the ability of metasurfaces to markedly simplify the architecture of systems using polarization optics. Gratings enabled by the approach here present a simpler means of full-Stokes polarization imaging that can be easily extended to other imaging systems and wavelengths. These compact, lightweight, and passive devices could enable the widespread adoption of polarization imaging.

Supplementary Materials

science.sciencemag.org/content/365/6448/eaax1839/suppl/DC1

Materials and Methods

Supplementary Text

Figs. S1 to S22

Tables S1 and S2

Movie S1

References (5357)

References and notes

  1. See supplementary materials.
Acknowledgments: We acknowledge M. Tamagnone, Y.-W. Huang, V. Ginis, S. Kheifets, Z. Li, A. Zaidi, and A. Dorrah (all of Harvard University) for helpful comments. Funding: N.A.R. acknowledges support from the National Science Foundation Graduate Research Fellowship Program (GRFP) under grant no. DGE1144152. This work was supported by the Air Force Office of Scientific Research under grant nos. FA9550-18-P-0024, FA9550-16-1-0156, and FA9550-14-1-0389 (MURI). We also acknowledge support from a Physical Sciences and Engineering Accelerator grant from Harvard University’s Office of Technology Development and a gift from the Google Accelerated Science team. F.C. acknowledges support from King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research (OSR) under grant no. OSR-2016-CRG5-2995. This work was performed in part at the Center for Nanoscale Systems (CNS), a member of the National Nanotechnology Coordinated Infrastructure (NNCI), which is supported by the National Science Foundation under NSF award no. 1541959. CNS is part of Harvard University. Author contributions: N.A.R. conceived the experiment and fabricated the gratings. G.D. wrote grating design codes and performed experimental characterization. N.A.R. and Z.S. clarified theoretical aspects of the work. W.T.C. contributed to camera design. N.A.R. and P.C. assembled the camera prototype, wrote camera software, and acquired the images in Figs. 4 and 5. N.A.R. and F.C. wrote the manuscript. F.C. supervised the research. Competing interests: A provisional patent application (US 62/834,343) has been filed on the subject of this work. Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper and/or the supplementary materials.
View Abstract

Stay Connected to Science

Navigate This Article