^{1}

^{2}

^{*}

^{1}

^{2}

^{1}

^{1}

^{2}

Edited by: Claudio Mirasso, Institute of Interdisciplinary Physics and Complex Systems (IFISC), Spain

Reviewed by: Apostolos Argyris, Institute of Interdisciplinary Physics and Complex Systems (IFISC), Spain; Vasileios Basios, Free University of Brussels, Belgium; Luis Pesquera, University of Cantabria, Spain

This article was submitted to Interdisciplinary Physics, a section of the journal Frontiers in Physics

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

We investigate, both numerically and experimentally, the usefulness of a distributed non-linearity in a passive coherent photonic reservoir computer. This computing system is based on a passive coherent optical fiber-ring cavity in which part of the non-linearities are realized by the Kerr non-linearity. Linear coherent reservoirs can solve difficult tasks but are aided by non-linear components in their input and/or output layer. Here, we compare the impact of non-linear transformations of information in the reservoirs input layer, its bulk—the fiber-ring cavity—and its readout layer. For the injection of data into the reservoir, we compare a linear input mapping to the non-linear transfer function of a Mach Zehnder modulator. For the reservoir bulk, we quantify the impact of the optical Kerr effect. For the readout layer we compare a linear output to a quadratic output implemented by a photodiode. We find that optical non-linearities in the reservoir itself, such as the optical Kerr non-linearity studied in the present work, enhance the task solving capability of the reservoir. This suggests that such non-linearities will play a key role in future coherent all-optical reservoir computers.

In this work, we discuss an efficient, i.e., high speed and low power, analog photonic computing system based on the concept of reservoir computing (RC) [

Within the field of reservoir computing two main approaches exist: in the network-based approach networks of neurons are implemented by connecting multiple discrete nodes [

Multiple opto-electronic reservoirs have been implemented, both delay-based [

State of the art photonic implementations target simple reservoir architectures [

In this paper, we study a delay-based reservoir computer, based on a passive coherent optical fiber ring cavity following reference [

Our reservoir computing simulations and experiments are based on the set of dynamical systems which are discussed in this section. The reservoir itself is implemented in the all-optical fiber-ring cavity shown in _{in} (originating from the green arrow) excites a polarization eigenmode of the fiber-ring cavity. A fiber coupler, characterized by its power transmission coefficient _{R}), the propagation loss α (taken here 0.18 dB km-1), the fiber non-linear coefficient γ (which is set to 0 to simulate a linear reservoir, and set to γ_{Kerr} = 2.6 mrad m^{-1} W^{-1} to simulate a non-linear reservoir), and the cavity detuning δ_{0}, i.e., the difference between the roundtrip phase and the nearest resonance (multiple of 2π). This low-finesse cavity is operated off-resonance, with a maximal input power of 50 mW (17 dBm). A network of time-multiplexed virtual neurons is encoded in the cavity field envelope. The output field _{out} is sent to the readout layer (through the orange arrow) where the neural responses are demultiplexed.

Schematic of the fiber-ring cavity of length ^{(n)}(_{R}) and

The input field _{in} can originate from one of two different optoelectronic input schemes. Firstly we consider a scenario where the input signal

Schematics of input and output layers connecting to the reservoir shown in _{LO} is used to implement coherent detection, allowing a quadrature of the complex optical field to be measured. Note that coherent detection requires two such readout arms with phase-shifted reference fields in order to measure the complex output field _{out}. In the non-linear output scheme

Similarly, the output field _{out} can be processed by two different optoelectronic readout schemes. Firstly we consider a coherent detection scheme as shown in _{LO} allows to record the complex neural responses, time-multiplexed in the output field _{out}. Secondly, we consider a readout scheme where a photodetector (PD) measures the optical power of the neural responses

With high optical power levels and small neuron spacing (meaning fast modulation of the input signal), dynamical and non-linear effects other than the Kerr non-linearity may appear, such as photon-phonon interactions causing Brillouin and Raman scattering, and bandwidth limitations caused by the driving and readout equipment. We want to focus in the present work on the effects of the Kerr non-linearity. Combined with the memory limitations of the oscilloscope, we therefore limit our reservoir to 20 neurons, with a maximal input power of 100 mW.

The current setup is not actively stabilized. We have found that the cavity detuning δ_{0} does not vary more than a few mrad over the course of any single reservoir computing experiment, where a few thousand input samples are processed. A short header, added to the injected signal, allows us to recover the detuning δ_{0} post-experiment. We effectively measure the interference between a pulse which reflects off the cavity and a pulse which completes one roundtrip through the cavity. However, we find that the precise value of δ_{0} has no significant influence on the experimental reservoir computing results.

Here we discuss the mean-field model used to describe the temporal evolution of the electric field envelope ^{(n)}(_{R} is time (bound by the cavity roundtrip time _{R} and 0 < ^{(n)}(^{(n)}(^{(n)}(^{(n+1)}(0, τ) from ^{(n)}(

Firstly, to model propagation in the fiber-ring cavity we take into account propagation loss and the non-linear Kerr-effect. Since the non-linear propagation model is independent from the roundtrip index

Here, α is the propagation loss and γ is the non-linear coefficient which is set to γ = 0 to simulate a linear reservoir, and set to γ = γ_{Kerr} to include the non-linear Kerr effect caused by the fiber waveguide. We do not include dispersion effects at the current operating point of the system, since the neuron separation is much larger than the diffusion length, hence also τ can be omitted in the non-linear propagation model. The evolution of the power |^{2} is readily obtained by solving the corresponding propagation equation

With ϕ_{z} the non-linear phase acquired during propagation over a distance

Since this non-linear phase depends on the power evolution given by Equation (2), an expression for ϕ_{z} is found to be

At this point, we can introduce the effective propagation distance _{eff} as

In general (since α ≥ 0) we have _{eff} ≤

Finally, we reinstitute the roundtrip index

In these equations, _{0} represents the cavity detuning (i.e., difference between the roundtrip phase and the closest cavity resonance). Further, the input field

The framework of reservoir computing allows to exploit the transient non-linear dynamics of a dynamical system to perform useful computation [_{M} of the input mask _{R}. Instead, we set _{M} = _{R}_{M}/^{(n)}(τ) injected into the RC is constructed by multiplying the input series ^{(n)}(τ) = ^{(n)}(τ) to the optical domain then

Note that in reference [_{S} is matched to the length of the input mask _{M}, allowing the reservoir to process 1 input sample approximately every roundtrip, as _{S} = _{M} ≲ _{R}. However, for reasons explained in the Results section, we will study different sample durations by holding input samples over multiple durations of the input mask, _{S} = _{M} with integer

Schematic of input and output timing, with _{S} the sample duration, _{M} the input mask duration and _{R} the roundtrip time. Input samples are injected during (integer) _{i}} (blue tick marks) during the last of those

Since the virtual neurons are time-multiplexed in this delay-based reservoir computer, they need to be de-multiplexed from _{i}} (with _{i} which are used to combine the neural readouts into a single scalar reservoir output

where the neural responses _{i}(_{i} are complex too, such that ^{2} + ν which is real-valued, and the readout weights will be real-valued too. Tasks are defined by the real-valued target output ŷ. Optimization of the readout weights occurs over a training set of _{train} input and target samples, and is achieved through least squares regression. This procedure minimizes the mean squared error between the reservoir output

These optimized readout weights are then validated on a test set of _{test} new input and target samples. A common figure of merit to quantify the reservoir's performance is the normalized mean square error (NMSE) defined as

Here we briefly investigate the relevant non-linearities which occur when mapping an electronic signal to an optical signal using an MZM. The operation of our balanced MZM can be described as

where _{0} represents the incident CW pump field, _{in} is the transmitted field which will be the input field to the optical reservoir, _{π} determines at which voltage the zero intensity point occurs (point of no transmission), and _{b} and a zero-mean signal _{s}, i.e., _{b} + _{s}. For our numerical investigation, we will set the amplitude of the signal voltage to |_{s}| = _{π}/2. First, we investigate the zero intensity bias point, _{b} = _{π}. In this case, we can approximate Equation (12) with the following Taylor expansion

With (_{in}/_{0})_{max} representing the maximal value of _{b} and signal amplitude |_{s}|, the relative error

is smaller than 1%. When the cubic term (_{s}) is omitted, this error increases to 11%. This means that at this operating point of the MZM, there is a significant non-linearity which scales with the input signal cubed.

Next, we investigate the linear intensity operating point, _{b} = _{π}/2. Although the MZM's transfer function at this operating point is the most linear in terms of the transmitted optical power, it is highly non-linear in terms of the transmitted optical field. In this case, we replace Equation (14) with

as we need all polynomial terms up to order 4 to keep the relative error defined by Equation (15) below 1%. In this case, omitting terms of orders above 1 in the approximation _{s}) increases the relative error of the Taylor expansion to 26%. This means that at this operating point of the MZM there are multiple polynomial non-linearities and that the total non-linear signal distortion is stronger compared with the zero intensity bias point.

Furthermore, during our experiments we have decided to operate the MZM in a linear regime. This allows for the non-linear effects inside the reservoir to be more readily measured. To this end, we tuned the MZM close to the zero intensity operating point, _{b} = _{π} − δ_{V} with δ_{V} ≪ _{π} and reduced the signal amplitude |_{s}|. The small deviation δ_{V} is used to generate a bias in the optical field injected into the reservoir.

To benchmark the performance of an RC, one can train it to perform one or several benchmark tasks. Alternatively, there exists a framework to quantify the system's total information processing capacity. This capacity is typically split into two main parts: the capacity of the system to retain past input samples is captured by the linear memory capacity [_{d}(

The ability of the RC to reconstruct each of these functions is evaluated by comparing the reservoir's trained output

where 〈.〉 denotes the average over all samples used for the evaluation of

Since the reservoirs are trained and their performance is evaluated on finite data sets, we run the risk of overestimating the memory capacities _{co} is used (_{co} ≈ 0.1 for 1,000 test samples) and capacities below this cutoff are neglected (i.e., they are assumed to be 0).

Note that the trade-off between linear and non-linear memory capacity is typically evaluated by comparing the total memory capacity of degree 1 (linear) with the total memory capacity of all higher degrees (non-linear). However, special attention is due when a PD is present in the readout layer of our RC. If a reservoir can (only) linearly retain past inputs _{i}) of those past inputs

and subsequently the optical power _{x} measured by the PD is given by

which consists of polynomial functions of past inputs of degrees 1 and 2. Thus, in this case the total linear memory capacity of the RC is represented by the total memory capacity of degrees 1 and 2 combined. In case the bias term

For the injection of input samples to the optical reservoir, we consider two strategies as discussed in section 2.1 and in

We have thus identified four different scenarios based on the absence or presence of non-linearities in the input and output layer of the reservoir computer. As we will show, we have for each of these cases numerically investigated the effect of the distributed non-linear Kerr effect, present in the fiber waveguide, on RC performance. For this evaluation, we have used 100 neurons to solve the Santa Fe time series prediction task [_{S} = _{M} with _{Kerr}).

Numerical results of fiber-ring reservoir computer on Santa Fe time series prediction tasks. In all panels the prediction error (NMSE) is plotted vs. the average neuron power 〈_{x}〉.

In _{Kerr}) induced by the fiber waveguide boosts the RC performance, with an optimal NMSE just below 1%. This can be readily understood as it is well-known that for this task, some non-linearity is required in order to obtain good RC performance. Note that the average neuron power 〈_{x}〉 can be used to estimate the average non-linear phase ϕ_{Kerr} the signals will acquire during the sample duration _{S}, as ϕ_{Kerr} = γ_{Kerr}〈_{x}〉_{S}/_{M}. We observe that without the presence of phase noise in the cavity, the boost to the RC performance due to the Kerr effect starts at very small values of the estimated non-linear phase, and breaks down when ϕ_{Kerr} ≳ 1. Switching to _{Kerr}). In _{bias} = _{π}). In terms of the optical field modulation, this is the most linear regime. It is thus no surprise that the performance of both linear and non-linear reservoirs mimics that _{bias} = _{π}) because of the small residual non-linearity at this operating point of the MZM. The round markers correspond with simulations where the MZM operates around the linear intensity operating point (_{bias} = _{π}/2). In terms of the optical field modulation, the non-linearity in the mapping of input samples to the optical field injected into the reservoir is more non-linear at this operating point. This is why even the linear reservoir manages to achieve errors below 4% (γ = 0, _{bias} = _{π}/2). Again we see that the introduction of the non-linear Kerr effect allows the NMSE to drop even further, to below 1% (γ = γ_{Kerr}). In fact, this scenario is similar to the scenario with linear input mapping and non-linear output mapping, _{bias} = _{π}/2) however, we observe a scenario in which the RC does not seem to benefit from the presence of the Kerr non-linear effect. It seems that with significant non-linearities present in both input and output layers of the RC the distributed non-linear effect inside the reservoir cannot further decrease the NMSE below values attained by the linear reservoir, which is below 1% (_{bias} = _{π}/2). In all other cases,

In this section we compare experimental results with detailed numerical simulations. For the experimental verification of our work, we are currently limited to operate with 20 neurons, as explained in section 2.1. Therefore, we have chosen not to perform the reservoir computing experiment on the Santa Fe task. With this few neurons, tasks like the Santa Fe task become hard for the reservoir. Instead we turn to a more academic task which allows us to quantify the reservoir's memory and non-linear computational capacity in a more complete and task-independent way. We experimentally measure the linear and non-linear memory capacities considered in section 2.5. Even with this few neurons the evaluation of the memory capacities can yield meaningful results while taking up comparatively little processing time.

For these experiments, the input layer to our fiber-ring reservoir contains a balanced MZM tuned to operate in a linear regime as outlined in section 2.4. The output layer employs a PD to measure the neural responses. That is, we use the setups of _{bias} = _{π}), and moving toward the _{bias} = _{π} − δ_{V}, with δ_{V} ≪ _{π}). This introduces a small bias component to the optical field injected into the reservoir, without compromising the linear operation of the MZM. The experiment was also repeated for different values of the sample duration _{S} with respect to the input mask periodicity _{M} (approximately equal to the cavity roundtrip _{R}). We expect the sample duration to play a very important role, since it determines how much time a piece of information spends inside the cavity, and thus how much non-linear phase can be acquired. The ratio _{S}/_{M} is gradually increased from _{S} = 2_{M} in (first row) _{S} = 6_{M} in (middle row) _{S} = 10_{M} in (bottom row) _{Kerr}) in (right column)

Comparison between experimental results _{Kerr}) reservoirs _{V}, is varied to include a small bias component to the injected optical field, where _{V} = 0 and _{V} ≪ _{π}. The sample duration _{S} is varied from 2 times _{M} (≈ cavity roundtrip time _{R}).

Firstly, in _{bias} = _{π}) the total memory capacity originates almost completely from the polynomial functions of degree 2 which means (given the presence of the PD in the readout layer) that the optical system is almost completely linear. Then, as an optical field bias is introduced we find that the total linear memory capacity of the system is now shared between degrees 1 and 2. As expected on account of quadratic non-linearity due to the PD, Equation (20), the contribution of (odd) degree 1 grows with the increasing bias. Beyond these capacities of degrees 1 and 2, we also observe a small contribution of capacities of degrees 3 and 4. We ascribe these contributions to the imperfect tuning of the MZM and thus a small residual non-linearity in the input mapping. Note that the simulations take into account the quasi-linear input mapping of the MZM, but seemingly underestimate the residual non-linearities to be insignificant. The imperfection of the MZM tuning also leads to a small residual bias component to the optical injected field, resulting in a small non-zero capacity of degree 1. Numerical simulations of linear (γ = 0) and non-linear (γ = γ_{Kerr}) reservoirs in _{S} = 2_{M}) neither simulations indicate any significant contributions of capacities with degrees beyond 2.

When increasing the sample duration (_{S} = 6_{M} and _{S} = 10_{M}), the experimental results in _{S} = _{M}≈_{R}) to control the cumulative non-linear effect inside the reservoir, we inevitably increase the mismatch between the inherent timescale of the input data (i.e., the sample duration _{S}) and the inherent timescale of the reservoir (i.e., the cavity roundtrip _{R}). and alter the reservoirs internal topology. When each sample is presented longer, past samples have spent more time inside the lossy cavity by the time they are accessed through the reservoirs noisy readout. Thus, on the longer timescales (_{S}) at which information is now processed, it is harder for the reservoir (operating at timescale _{R}) to retain past information. These aspects explain why the overall total memory capacity (summed over all degrees) decreases with increased sample duration _{S}. The numerical results on both the linear reservoir (γ = 0) in _{Kerr}) in _{S} the simulated non-linear reservoir shows the contribution of the total non-linear memory capacity (degrees 3 and 4) to the total memory capacity (all degrees) growing from 0 to 25.4%, and in the experiment this contribution starts at 6.4% and grows up to 23.6%. This sizable increase in non-linear computation capacity can be of considerable significance to the reservoir's performance on other tasks, as shown earlier. When comparing the experimental results with the non-linear reservoir model for all given sample durations _{S}, the main difference is that the capacities of degree 3 seem to appear sooner (i.e., for smaller sample duration) in the experiment. This can be explained by the residual bias component to the optical injected field. Such a bias makes it easier to produce polynomial functions of odd degrees, thus explaining their earlier onset. This can be explained by the quadratic nature of the Kerr non-linearity, as the reasoning previously applied to the quadratic non-linearity of the PD in Equation (20) can be generalized to memory capacities of higher degree.

We have identified and investigated the role of non-linear transformation of information inside a photonic computing system based on a passive coherent fiber-ring reservoir. Non-linearities can occur at different places inside a reservoir computer: the input layer, the bulk and the readout layer. State-of-the-art opto-electronic RC systems often include one or several components which inevitably introduce non-linearities to the computing system. On the reservoir's input side, we have compared a linear input regime with the usage of a MZM, which has a non-linear transfer function, to convert electronic data to an optical signal. On the reservoir's output side, we have compared a linear output regime with the usage of a PD which measures optical power levels, that scale quadratically with the optical field strength of the neural responses. We numerically evaluated such systems using a benchmark test and found that non-linear input and/or output components are needed to obtain good RC performance when the optical reservoir itself (i.e., the core of the RC system) is a strictly linear system.

Internal to the reservoir, we investigated the effect of the optical Kerr non-linear effect on RC performance. Our numerical benchmark test showed a large band of optical powers where the presence of this distributed non-linear effect, caused by the waveguiding material of the reservoir, significantly decreased the RC's error figure. Our numerical and experimental measurements of the linear and non-linear memory capacity of this RC system showed that the accumulation of non-linear phase due to the distributed non-linear Kerr effect strongly improves the system's non-linear computational capacity. We can thus conclude that for photonic reservoir computers with non-linear input and/or output components, the presence of a distributed non-linear effect inside the optical reservoir improves the RC performance. Furthermore, the distributed non-linearity is essential for good performance in the regime where non-linearities are absent from both the input and output layer. This may be the case in an all-optical reservoir computer (i.e., with optical input and output layers). We have shown that the effect of the distributed non-linearity is strong enough to compensate for the lack of non-linear transformation of information elsewhere in the system, and that it allows to build a computationally strong photonic computing system.

Finally, we expect a design approach including distributed non-linear effects to improve the scalability of these types of computational devices. In general, when harder tasks are considered, larger reservoirs are required. One way to increase the size of a delay-based reservoir is to implement a longer delay-line. This increase in length of the signal propagation path naturally increases the effect of distributed non-linearities as considered in this work. Similarly, increasing the size of a network-based reservoir will also lead to more and/or longer signal paths, resulting in the increased accumulation of non-linear effects, although waveguides with stronger non-linear effects may have to be considered to compensate for the shorter connection lengths in on-chip implementations. We believe that the natural increase in the strength of non-linear effects, following the increase in size of the reservoir, may diminish the need to place discrete non-linear components inside large networks used for strongly non-linear tasks. As such, both the complexity and cost of such systems would be reduced. Since the waveguiding material itself is used to induce non-linear effects, the waveguide properties (such as material and geometry) determines the optical field confinement and thus regulate the strength of non-linear interactions. Consequently it may be possible to create reservoirs where deliberate variations in the waveguide properties are used to tune the strength of the distributed non-linear effect in different regions of the system. This would allow for a trade off between the system's linear memory capacity and its non-linear computational capacity, such that a large number of past input samples can be retained (in some parts of the system) and then non-linearly processed to solve difficult tasks (in other parts of the system). These considerations indicate why distributed non-linear effects may play a major role in future implementations of powerful photonic reservoir computers.

The data used in this study for the Sante Fe prediction task [

The idea was first conceived by GVa and finalized together with GVe and SM. JP was responsible for the physical modeling, the numerical calculations, the experimental verification, and wrote most of the manuscript. All coauthors contributed to the discussion of the results and writing of the manuscript.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The reviewer VB declared a shared affiliation, with no collaboration, with the authors JP and SM to the handling editor at time of review.