Home // COGNITIVE 2021, The Thirteenth International Conference on Advanced Cognitive Technologies and Applications // View article
Authors:
Ryotaro Kamimura
Keywords: selective information, mutual information, partial compression, collective interpretatin, correlation coefficient
Abstract:
This paper aims to propose a new type of information-theoretic method to control information content stored and transmitted in neural networks. To make the meaning of information concretely interpretable, we introduce the selective information and a method to control it, called ``selective information-driven learning''. The new method is more suited for modeling neural learning than conventional information-theoretic measures, such as mutual information, because we can easily maximize and minimize the information, and we can interpret the meaning of information more concretely. The method was applied to the well-known wine data set. The experimental results show that the selective information could be maximized and minimized, and we could easily interpret the meaning of information in terms of the number of strong weights. In addition, the partial compression of multi-layered neural networks revealed that maximum information tended to be focused on output information, while minimum information tried to consider input information in addition to output information. Finally, collective weights, averaged over all compressed weights obtained in learning, were similar to the original correlation coefficients between inputs and targets, meaning that the selective information can disentangle complicated connection weights into simple, linear, and independent ones to be easily interpreted.
Pages: 20 to 27
Copyright: Copyright (c) IARIA, 2021
Publication date: April 18, 2021
Published in: conference
ISSN: 2308-4197
ISBN: 978-1-61208-847-1
Location: Porto, Portugal
Dates: from April 18, 2021 to April 22, 2021