Home // IMMM 2012, The Second International Conference on Advances in Information Mining and Management // View article

Parallel Processing of Very Many Textual Customers' Reviews Freely Written Down in Natural Languages

Authors:
Jan Zizka
Frantisek Darena

Keywords: text mining; natural language; parallel processing; decision tree; data subset size; computational complexity.

Abstract:
Text mining of hundreds of thousand or millions of documents written in a natural language is limited by the computational complexity (time and memory) and computer performance. Many applications can use only standard personal computers. In this case, the whole data set has to be divided into smaller subsets that can be processed in parallel. This article deals with the problem how to divide the original data set, which represents a typical collection containing two millions of customers’ reviews written in English. The main goal is to mine information the quality of which is comparable with information obtained from the whole set despite the fact that the mining is carried out using subsets of the original large data set. The article suggests a method of dividing the set into subsets including a possibility of evaluating the mining results by comparing the unified outputs of individual subsets with the original set. The suggested method is illustrated with a task that searches for significant words expressing the customers’ opinions on hotel services. It is shown that there is always a certain boundary under which the subset sizes cannot fall as well as how to experimentally find this border.

Pages: 147 to 153

Copyright: Copyright (c) IARIA, 2012

Publication date: October 21, 2012

Published in: conference

ISSN: 2326-9332

ISBN: 978-1-61208-227-1

Location: Venice, Italy

Dates: from October 21, 2012 to October 26, 2012