Home // International Journal On Advances in Intelligent Systems, volume 14, numbers 1 and 2, 2021 // View article
Authors:
Ryotaro Kamimura
Keywords: collective interpretation; selective information; cost; partial compression; generalization
Abstract:
The present paper aims to interpret multi-layered neural networks by considering as many possible internal representations as possible, which is called ``collective interpretation.'' The interpretation is performed in a syntagmatic and paradigmatic way. In the syntagmatic processing, all representations created in each step of the learning processes from the beginning to the final stage are considered. Then, in the paradigmatic approach, we try to deal with all possible representations by the syntagmatic processing. In addition, to make this collective interpretation easier, we control collective interpretation by the selective information, which is simplified to control the cost in terms of the strength of connection weights. The collective interpretation with the simplified selective information augmentation by the cost control was applied to three actual data sets: the traffic, facility for the elderly, and wine data sets. With the first two data sets, we could observe that the networks tried to extract simple and clear relations between inputs and outputs. For the wine data set, because the simple cost reduction could not be effective, the cost was first augmented to reduce the selective information, and then it was increased. The final compressed weights were also simplified for clearer interpretation. The results showed that the collective interpretation with the simple selective information control by the cost control could flexibly deal with input and output information for producing simple and interpretable representations.
Pages: 175 to 192
Copyright: Copyright (c) to authors, 2021. Used with permission.
Publication date: December 31, 2021
Published in: journal
ISSN: 1942-2679