Please use this identifier to cite or link to this item:
PIRA download icon_1.1View/Download Full Text
Title: An improved algorithm for learning long-term dependency problems in adaptive processing of data structures
Authors: Cho, SY
Chi, ZG 
Siu, WC 
Tsoi, AC
Issue Date: Jul-2003
Source: IEEE transactions on neural networks, July 2003, v. 14, no. 4, p. 781-793
Abstract: For the past decade, many researchers have explored the use of neural-network representations for the adaptive processing of data structures. One of the most popular learning formulations of data structure processing is backpropagation through structure (BPTS). The BPTS algorithm has been successful applied to a number of learning tasks that involve structural patterns such as logo and natural scene classification. The main limitations of the BPTS algorithm are attributed to slow convergence speed and the long-term dependency problem for the adaptive processing of data structures. In this paper, an improved algorithm is proposed to solve these problems. The idea of this algorithm is to optimize the free learning parameters of the neural network in the node representation by using least-squares-based optimization methods in a layer-by-layer fashion. Not only can fast convergence speed be achieved, but the long-term dependency problem can also be overcome since the vanishing of gradient information is avoided when our approach is applied to very deep tree structures.
Keywords: Adaptive processing of data structures
Back-propagation through structure (BPTS)
Least-squares method
Long-term dependency
Publisher: Institute of Electrical and Electronics Engineers
Journal: IEEE transactions on neural networks 
ISSN: 1045-9227
DOI: 10.1109/TNN.2003.813831
Rights: © 2003 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.
Appears in Collections:Journal/Magazine Article

Files in This Item:
File Description SizeFormat 
adaptive-processing_03.pdf891.03 kBAdobe PDFView/Open
Open Access Information
Status open access
File Version Version of Record
View full-text via PolyU eLinks SFX Query
Show full item record

Page views

Last Week
Last month
Citations as of Jun 4, 2023


Citations as of Jun 4, 2023


Last Week
Last month
Citations as of Jun 1, 2023


Last Week
Last month
Citations as of Jun 1, 2023

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.