Report

All-optical machine learning using diffractive deep neural networks

See allHide authors and affiliations

Science  07 Sep 2018:
Vol. 361, Issue 6406, pp. 1004-1008
DOI: 10.1126/science.aat8084
  • Fig. 1 Diffractive deep neural networks (D2NNs).

    (A) A D2NN comprises multiple transmissive (or reflective) layers, where each point on a given layer acts as a neuron, with a complex-valued transmission (or reflection) coefficient. The transmission or reflection coefficients of each layer can be trained by using deep learning to perform a function between the input and output planes of the network. After this learning phase, the D2NN design is fixed; once fabricated or 3D-printed, it performs the learned function at the speed of light. L, layer. (B and C) We trained and experimentally implemented different types of D2NNs: (B) classifier (for handwritten digits and fashion products) and (C) imager. d, distance. (D) Comparison between a D2NN and a conventional neural network (14). Based on coherent waves, the D2NN operates on complex-valued inputs, with multiplicative bias terms. Weights in a D2NN are based on free-space diffraction and determine the interference of the secondary waves that are phase- and/or amplitude-modulated by the previous layers. “ο” denotes a Hadamard product operation. “Electronic neural network” refers to the conventional neural network virtually implemented in a computer. Y, optical field at a given layer; Ψ, phase of the optical field; X, amplitude of the optical field; F, nonlinear rectifier function [see (14) for a discussion of optical nonlinearity in D2NN].

  • Fig. 2 Experimental testing of 3D-printed D2NNs.

    (A and B) After the training phase, the final designs of five different layers (L1, L2, …, L5) of the handwritten digit classifier, fashion product classifier, and the imager D2NNs are shown. To the right of the network layers, an illustration of the corresponding 3D-printed D2NN is shown. (C and D) Schematic (C) and photo (D) of the experimental terahertz setup. An amplifier-multiplier chain was used to generate continuous-wave radiation at 0.4 THz, and a mixer-amplifier-multiplier chain was used for the detection at the output plane of the network. RF, radio frequency; f, frequency.

  • Fig. 3 Handwritten digit classifier D2NN.

    (A) A 3D-printed D2NN successfully classifies handwritten input digits (0, 1, …, 9) on the basis of 10 different detector regions at the output plane of the network, each corresponding to one digit. As an example, the output image of the 3D-printed D2NN for a handwritten input of “5” is demonstrated, where the red dashed squares represent the trained detector regions for each digit. Other examples of our experimental results are shown in fig. S9. (B) Confusion matrix and energy distribution percentage for our experimental results, using 50 different handwritten digits (five for each digit) that were 3D-printed, selected among the images for which numerical testing was successful. (C) Same as (B), except summarizing our numerical testing results for 10,000 different handwritten digits (~1000 for each digit), achieving a classification accuracy of 91.75% using a five-layer design. Our classification accuracy increased to 93.39% by increasing the number of diffractive layers to seven, using a patch of two additional diffractive layers added to an existing and fixed D2NN (fig. S2).

  • Fig. 4 Fashion product classifier D2NN.

    (A) As an example, the output image of the 3D-printed D2NN for a sandal input (Fashion-MNIST class 5) is demonstrated. The red dashed squares represent the trained detector regions for each fashion product. Other examples of our experimental results are shown in fig. S10. (B) Confusion matrix and energy distribution percentage for our experimental results, using 50 different fashion products (five per class) that were 3D-printed, selected among the images for which numerical testing was successful. (C) Same as (B), except summarizing our numerical testing results for 10,000 different fashion products (~1000 per class), achieving a classification accuracy of 81.13% using a five-layer design. By increasing the number of diffractive layers to 10, our classification accuracy increased to 86.60% (fig. S5).

Supplementary Materials

  • All-optical machine learning using diffractive deep neural networks

    Xing Lin, Yair Rivenson, Nezih T. Yardimci, Muhammed Veli, Yi Luo, Mona Jarrahi, Aydogan Ozcan

    Materials/Methods, Supplementary Text, Tables, Figures, and/or References

    Download Supplement
    • Materials and Methods
    • Figs. S1 to S16
    • References
     
    Correction (6 September 2018): In the top panel of fig. S2, the left y axis contained a labeling error: "0.4 Million Neurons" has been corrected to "0.2 Million Neurons."
    The original version is accessible here.

Stay Connected to Science

Navigate This Article