What is an OPU?

An Optical Processing Unit is an analog computation device that performs operations using light.


A monochromatic laser illuminates a digital micro-mirror device (DMD), which consists in a rectangular array of micro-mirrors. DMDs can display an image from the computer by turning micro-mirrors ON and OFF. The laser light is shaped and encodes digital information. The light beam goes through a random scattering medium and interferes randomly on the camera, which acquires the intensity of the incident light.

Using optics, we are performing a multiplication by a random matrix

\[\mathbf{Y} = \lvert \mathbf{x} \mathbf{R} \rvert^2 \mbox{ with } \mathbf{R} \mbox{ with complex gaussian entries.}\]

This operation is accelerated optically and can scale to very large dimensions.

The optical operation in detail

The operation performed by the OPU comprises two successive steps:

  • a matrix-vector multiplication

  • an element-wise non-linearity

In our notation:

  • \(\mathbf{x}\) is the input vector, written as a row vector of size \((1 \times n)\) of binary numbers and displayed on the DMD as a binary pattern

  • \(\mathbf{y}\) is the output vector, written as a row vector of size \((1 \times m)\) of uint8 data recorded on the camera sensor

  • \(\mathbf{R}\) is the random matrix of size \((n \times m)\) of complex values

Internally to the optical system, x and y are both displayed physically as 2-D arrays (images) but they should intrinsically be considered as 1-D vectors.

The matrix-vector multiplication outputs a \((1 \times m)\) vector, complex-valued. This is followed by the element-wise non-linearity \(\lvert . \rvert^2\) due to the fact that CMOS sensors are essentially measuring light intensity. There is also the quantization due to analog to digital conversion. Finally, the output of the OPU is \(\mathbf{y}\) a column vector of size \((1 \times m)\) of type uint8. The independence of the pixels of the speckle image (the entries of the output vector) means that the rows of the matrix R are independent.

For more details see Section III of Random Projections through multiple optical scattering: Approximating kernels at the speed of light.

The distribution of the random matrix

It is possible to write \(r=\mathbf{R}_{ij}\) in the following way:

\[r = u + iv, u \sim \mathcal{N}(0, \sigma^2), v \sim \mathcal{N}(0, \sigma^2)\]

The only parameter is a global gain: a multiplicative constant, that you may see as the variance of these distributions. It depends on many physical parameters.

When we make an OPU, we carry out a series of tests to make sure that the coefficients are i.i.d.: independent and identically distributed.

The binary input

In the DMD technology each micro-mirror only has 2 possible positions corresponding to 0 and 1.

For input data that is not binary to start with, you can use encoding schemes to transform any type of data into a binary array. Some of these encoding schemes are fixed, but we also have a data-adaptive scheme based on an autoencoder. Finally, there is also the possibility that the user designs its own encoding.

Understanding the output

The output recorded by the camera is a figure of diffraction, called a speckle.


The histogram of a theoretical speckle is a decreasing exponential. In our case, because we want to guarantee the independence of the pixels of the camera, we need the pixels to be of about the size of the speckle grains. For each pixel, the camera is doing a sampling of the speckle grains and low values cannot be found.