Please use this identifier to cite or link to this item:
PIRA download icon_1.1View/Download Full Text
Title: Advancing theoretical understanding and practical performance of signal processing for nonlinear optical communications through machine learning
Authors: Fan, QR 
Zhou, G 
Gui, T 
Lu, C 
Lau, APT 
Issue Date: 2020
Source: Nature communications, 2020, v. 11, 3694, p. 1-11
Abstract: In long-haul optical communication systems, compensating nonlinear effects through digital signal processing (DSP) is difficult due to intractable interactions between Kerr nonlinearity, chromatic dispersion (CD) and amplified spontaneous emission (ASE) noise from inline amplifiers. Optimizing the standard digital back propagation (DBP) as a deep neural network (DNN) with interleaving linear and nonlinear operations for fiber nonlinearity compensation was shown to improve transmission performance in idealized simulation environments. Here, we extend such concepts to practical single-channel and polarization division multiplexed wavelength division multiplexed experiments. We show improved performance compared to state-of-the-art DSP algorithms and additionally, the optimized DNN-based DBP parameters exhibit a mathematical structure which guides us to further analyze the noise statistics of fiber nonlinearity compensation. This machine learning-inspired analysis reveals that ASE noise and incomplete CD compensation of the Kerr nonlinear term produce extra distortions that accumulates along the DBP stages. Therefore, the best DSP should balance between suppressing these distortions and inverting the fiber propagation effects, and such trade-off shifts across different DBP stages in a quantifiable manner. Instead of the common 'black-box' approach to intractable problems, our work shows how machine learning can be a complementary tool to human analytical thinking and help advance theoretical understandings in disciplines such as optics.
Publisher: Nature Publishing Group
Journal: Nature communications 
EISSN: 2041-1723
DOI: 10.1038/s41467-020-17516-7
Rights: Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit
© The Author(s) 2020
The following publication Fan, Q. R., Zhou, G., Gui, T., Lu, C., & Lau, A. P. T. (2020). Advancing theoretical understanding and practical performance of signal processing for nonlinear optical communications through machine learning. Nature Communications, 11, 1-11 is available at
Appears in Collections:Journal/Magazine Article

Files in This Item:
File Description SizeFormat 
Fan_Signal_Machine_Learning.pdf1.1 MBAdobe PDFView/Open
Open Access Information
Status open access
File Version Version of Record
View full-text via PolyU eLinks SFX Query
Show full item record

Page views

Last Week
Last month
Citations as of May 28, 2023


Citations as of May 28, 2023


Citations as of Jun 1, 2023


Last Week
Last month
Citations as of Jun 1, 2023

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.