ANALYSIS OF DOCUMENTS RANKING ALGORITHMS BASED ON THE ARTIFICIAL NEURAL NETWORKS

Abstract

Effective search of documents in documentation databases of companies every year turns into an increasingly complex task. The major reason of such a situation is the rapid growth of information and the emergence of new features of the data collections. Modern search engines constantly optimize their work, paying attention to the ranking of documents found according to their relevance to the request The object of this study is to analyse the effectiveness of document ranking algorithms in search engines that use artificial neural networks to match the texts. The purpose of the study was to inspect a neural network model of text document ranking that uses clustering, factor analysis, and multi-layered network architecture. The work of neural network algorithms was compared with the standard statistical search algorithm OkapiBM25. The result of the study is to evaluate the effectiveness of the use of particular models and to recommend model selection for specific datasets. System identification algorithm is proposed text search in documents databases of companies, including factor and regression data analysis. Factor analysis includes usage-based data clustering using Kohonen network. For regression analysis we suggest using one of two neural network models: based on the hybrid neural net-work or based on the multilayer perceptron. Algorithm test results show successful model training and low values for training and testing errors. Moreover, a model based on a hybrid neural network has learning difficulties if a large number of disjoint characteristics. The bottleneck of the algorithm is factor analysis method, allowing to identify significant factors. In addition to the proposed statistical method that identifies the most important factors, it is possible to use the Bayesian or fuzzy logic methods [18, 19]. Model modification using the weights of the characteristics that will influence the learning and results interpretation is also possible. Development may involve an analysis of the influence of complex non-linear factors and their combinations.