Please use this identifier to cite or link to this item:
http://hdl.handle.net/10397/189
| Title: | An improved algorithm for learning long-term dependency problems in adaptive processing of data structures | Authors: | Cho, SY Chi, ZG Siu, WC Tsoi, AC |
Issue Date: | Jul-2003 | Source: | IEEE transactions on neural networks, July 2003, v. 14, no. 4, p. 781-793 | Abstract: | For the past decade, many researchers have explored the use of neural-network representations for the adaptive processing of data structures. One of the most popular learning formulations of data structure processing is backpropagation through structure (BPTS). The BPTS algorithm has been successful applied to a number of learning tasks that involve structural patterns such as logo and natural scene classification. The main limitations of the BPTS algorithm are attributed to slow convergence speed and the long-term dependency problem for the adaptive processing of data structures. In this paper, an improved algorithm is proposed to solve these problems. The idea of this algorithm is to optimize the free learning parameters of the neural network in the node representation by using least-squares-based optimization methods in a layer-by-layer fashion. Not only can fast convergence speed be achieved, but the long-term dependency problem can also be overcome since the vanishing of gradient information is avoided when our approach is applied to very deep tree structures. | Keywords: | Adaptive processing of data structures Back-propagation through structure (BPTS) Least-squares method Long-term dependency |
Publisher: | Institute of Electrical and Electronics Engineers | Journal: | IEEE transactions on neural networks | ISSN: | 1045-9227 | DOI: | 10.1109/TNN.2003.813831 | Rights: | © 2003 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder. |
| Appears in Collections: | Journal/Magazine Article |
Files in This Item:
| File | Description | Size | Format | |
|---|---|---|---|---|
| adaptive-processing_03.pdf | 891.03 kB | Adobe PDF | View/Open |
Access
View full-text via PolyU eLinks 
Page views
60
Last Week
0
0
Last month
Citations as of Jun 4, 2023
Downloads
96
Citations as of Jun 4, 2023
SCOPUSTM
Citations
37
Last Week
0
0
Last month
1
1
Citations as of Jun 1, 2023
WEB OF SCIENCETM
Citations
29
Last Week
0
0
Last month
0
0
Citations as of Jun 1, 2023
Google ScholarTM
Check
Altmetric
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.