Dedupeer: An Algorithm for File Deduplication Through Partitioned Processing

1 Paulo Fernando Almeida Soares, 2Frederico Araújo Durão, 3Vinícius Cardoso Garcia, 4Rodrigo Elia Assad, 5Diogo Vinícius de Sousa Silva, 6Anderson Fonseca e Silva, 7Silvio Romero de Lemos Meira
1Federal University of Pernambuco,
2Federal University of Bahia,
3Federal University of Pernambuco,
4Federal Rural University of Pernambuco,
5Federal University of Bahia,
6Federal University of Pernambuco,
7Federal University of Pernambuco,

Published in:

JCIT (Journal of Convergence Information Technology)
Volume 14 Issue 1, January, 2019
Pages 100-111
ISSN 1975-9320 (Print) 2233-9299 (Online)
GlobalCIS (Convergence Information Society, Republic of Korea)

Studies of deduplication techniques started in mid-2004. There are research papers focused on the implementation of algorithms, but they are rare and the main goals are not focused on the integration with existing systems. This study aims at creating an algorithm, called Dedupper, for deduplication of files at the source with remote data, through partitioned processing and fingerprint comparisons. The Dedupeer algorithm was integrated in a software component with an interoperable interface to facilitate its use in any storage system to benefit the system with storage economy and help in the data transfer when the system was distributed. In addition to the software component, a storage system was also developed with data management, which makes it able to be distributed with the goal of validating the deduplication algorithm. The integration of the software component with the storage system was implemented and evaluated in this research.

Deduplication, Storage, Data, Dompaction, Compression

2019 Article
View Contents
View Coverpage
View Paper
- Clicks: 349
- Downloads: 116
Journal Home Page
Editorial Board
Scope, Paper type, Format
Paper Submission