Artificial neural networks for pattern recognition in multilingual text. (Poster)

Abstract

Neural networks are able to discover patterns in data; it need not be textual data. When a neural network is used to analyze textual data, however, it does not need grammar, heuristics, parsing or other linguistic artifacts to recognize, store and retrieve clusters. This is because what the system considers is not words per se, but rather, patterns in the stream of bits that represents the words. The neural network searches for recurrent patterns in this bit stream. Previous neural software has been based on ASCII code, but the present study utilizes UNICODE rather than ASCII code and extends the possibility of text analysis to a wide range of languages, including those such as Hindi, Korean and Chinese which are pictorial rather than alphabetic. If language is a vehicle for culture, then different cultures may use language differently. This would be suggested by inconsistent language usage and patterns. Using a system able to look at multilingual texts, different languages can be compared with each other quantitatively to explore the idea of discrepancy between cultures. This paper utilizes a neural network for analysis of text across cultures and languages. Different language versions of the United Nations' Human Rights Declaration were compared with each other to find out whether language difference affects the translation of official texts. Observed differences are presented quantitatively and graphically.