No Detail Found!

Automatic taxonomy generation and incremental evolution on apache spark parallelization framework

Author(s)
Kanwal Aalijah
Rabia Irfan
Umara Umar
Sanam Nayab
Abstract
The term “Big Data” refers to a large volume of information usually in terabytes and petabytes. It includes both structured and unstructured data. Unstructured data is conventionally text-heavy, but may also contain data such as facts, dates, and numbers. To use this unstructured information effectively, it needs to be processed and organized. Taxonomy is considered a powerful way of organizing information. For automatic taxonomy generation, various techniques have been proposed in the past. However, the substantial nature of big data presently crosses the processing abilities of traditional techniques. Thus, to meet this challenge an extensible and scalable technique is required to potentially accelerate the process of taxonomy generation and its evolution upon arrival of new data, hence catering to a large amount of unstructured big data. This paper proposes a technique for both the taxonomy generation and evolution of Apache Spark infrastructure. The proposed technique is evaluated on a text dataset from a computing domain. The evaluation results show that the technique presented in this paper outperformed the existing techniques in terms of time and quality metrics. The time and quality-based evaluation showed that the use of the MapReduce environment has resolved the scalability issues of the current taxonomy generation and evolution process.
Publication Details
Page(s) 15-36
Published Journal: KIET Journal of Computing & Information Sciences, Volume: 5, Issue: 1, Year: 2022

Request the full text from PASTIC

Keywords
Taxonomy, Taxonomy, Big Data, unstructured data, Apache Spark, Map-Reduce, Hadoop, Scalable
0

Citations

0

Downloads

76

Views