No Detail Found!

Towards a trustworthy and efficient ETL pipeline for ATM transaction data

Author(s)
Muhammad Ahmad Ashfaq Faculty of Computing and Information Technology, Shahrahe Quaid-e-Azam Allama Iqbal Campus (Old Campus), Lahore, Pakistan
Nimra Haq Faculty of Computing and Information Technology, Shahrahe Quaid-e-Azam Allama Iqbal Campus (Old Campus), Lahore, Pakistan
Usman Arshad Faculty of Computing and Information Technology, Shahrahe Quaid-e-Azam Allama Iqbal Campus (Old Campus), Lahore, Pakistan
Muhammad Farooq Faculty of Computing and Information Technology, Shahrahe Quaid-e-Azam Allama Iqbal Campus (Old Campus), Lahore, Pakistan
Shuja ur Rehman Baig Faculty of Computing and Information Technology, Shahrahe Quaid-e-Azam Allama Iqbal Campus (Old Campus), Lahore, Pakistan
Abstract
ATMs generate vast amounts of data daily, which needs to be analyzed and stored. Dealing with this data also termed big data, is a complex task, and here comes the role of ETL pipelines. ETL pipelines need extensive resources for operations, and their performance optimization is necessary as data must be dealt with in near or even real-time. If the pipeline deals with financial data such as ATM transactions, steps should be taken to ensure the data's security, privacy, confidentiality, and integrity. This can be achieved using Blockchain technology. It is a distributed ledger technology having an immutable nature. It has significant advantages in terms of providing security, but it has disadvantages as well, such as low throughput and transactional latency. If blockchain is used in an ETL pipeline, it will affect the overall performance. So, to prevent the decline in performance, steps should be taken to optimize it. In this paper, we are using parallelization and partitioning as techniques to optimize performance. The primary goal here is to achieve maximum security while maintaining performance. 
Publication Details
Page(s) 9-24
DOI DOI not available
Published Journal: Sukkur IBA Journal of Computing and Mathematical Sciences, Volume: 7, Issue: 2, Year: 2023

Request the full text from PASTIC

Keywords
Big Data, Blockchain, Performance optimization, Spark, ETL Pipeline, Kafka
0

Citations

0

Downloads

44

Views