A genetic-based HAC technique for parallel clustering of bilingual Malay-English corpora

Ng Zhen Wei and Chan Chen Jie and Rayner Alfred and Joe Henry Obit (2012) A genetic-based HAC technique for parallel clustering of bilingual Malay-English corpora. International Journal of Computer Science and its Applications, 2. pp. 161-168. ISSN 2250-3765

	Text A genetic-based HAC technique for parallel clustering of bilingual Malay-English corpora_ABSTRACT.pdf Download (61kB)
	Text A Genetic-Based HAC Technique for Parallel Clustering of Bilingual Malay-English Corpora FULL TEXT.pdf Restricted to Registered users only Download (350kB)

Abstract

Multi Multilingual corpora, containing the same documents in a variety of languages, are becoming an essential resource for natural language processing. Clustering multilingual corpora provides us with an insight into the differences between languages when term frequency-based Information Retrieval (IR) tools are used. It also allows one to use the Natural Language Processing (NLP) and IR tools in one language to implement IR for another language. For instance, in this way, the most relevant articles to be translated from language Malay to language English can be selected after studying the clusters of abstracts in language English. In this paper, we report on our work on applying Hierarchical Agglomerative Clustering (HAC) to a large corpus of documents where each appears both in Malay and English. We cluster these documents for each language and compare the results both with respect to the content of clusters produced. On the data available, the results of clustering one language resemble the other, provided the number of clusters required is relatively small. Further, we study the effects of changing the method used to compute the inter-clusters distance that includes single link, complete link and average link distance between clusters. Finally, we describe an experiment employing a genetic algorithm to fine-tune the individual term weights in order to reproduce more closely a predefined set of clusters. In this way, clustering becomes a supervised learning technique that is trained to better reproduce known clusters in language Malay when applied to the corresponding documents in language English. Other possible applications include training the algorithm on a hand clustered set of documents, and subsequently applying it to a superset, including unseen documents, incorporating in this way expert knowledge about the domain in the clustering algorithm.

Item Type:	Article
Keyword:	Bilingual corpora , Hierarchical agglomerative clustering , Parallel clustering , Genetic algorithm , Malay-English corpora
Subjects:	Q Science > QA Mathematics > QA1-939 Mathematics > QA71-90 Instruments and machines > QA75.5-76.95 Electronic computers. Computer science > QA76.75-76.765 Computer software Q Science > QH Natural history > QH301-705.5 Biology (General) > QH426-470 Genetics
Department:	FACULTY > Faculty of Computing and Informatics
Depositing User:	SAFRUDIN BIN DARUN -
Date Deposited:	10 Sep 2021 09:59
Last Modified:	10 Sep 2021 09:59
URI:	https://eprints.ums.edu.my/id/eprint/29029

Actions (login required)

View Item