Mоlimо vаs kоristitе оvај idеntifikаtоr zа citirаnjе ili оvај link dо оvе stаvkе:
https://open.uns.ac.rs/handle/123456789/4757
Nаziv: | De-duplicating a large crowd-sourced catalogue of bibliographic records | Аutоri: | Subasić, Ilija Gvozdenović, Nebojša Jack, Kris |
Dаtum izdаvаnjа: | 4-апр-2016 | Čаsоpis: | Program | Sažetak: | © 2016, © Emerald Group Publishing Limited. Purpose – The purpose of this paper is to describe a large-scale algorithm for generating a catalogue of scientific publication records (citations) from a crowd-sourced data, demonstrate how to learn an optimal combination of distance metrics for duplicate detection and introduce a parallel duplicate clustering algorithm. Design/methodology/approach – The authors developed the algorithm and compared it with state-of-the art systems tackling the same problem. The authors used benchmark data sets (3k data points) to test the effectiveness of our algorithm and a real-life data (>90 million) to test the efficiency and scalability of our algorithm. Findings – The authors show that duplicate detection can be improved by an additional step we call duplicate clustering. The authors also show how to improve the efficiency of map/reduce similarity calculation algorithm by introducing a sampling step. Finally, the authors find that the system is comparable to the state-of-the art systems for duplicate detection, and that it can scale to deal with hundreds of million data points. Research limitations/implications – Academic researchers can use this paper to understand some of the issues of transitivity in duplicate detection, and its effects on digital catalogue generations. Practical implications – Industry practitioners can use this paper as a use case study for generating a large-scale real-life catalogue generation system that deals with millions of records in a scalable and efficient way. Originality/value – In contrast to other similarity calculation algorithms developed for m/r frameworks the authors present a specific variant of similarity calculation that is optimized for duplicate detection of bibliographic records by extending previously proposed e-algorithm based on inverted index creation. In addition, the authors are concerned with more than duplicate detection, and investigate how to group detected duplicates. The authors develop distinct algorithms for duplicate detection and duplicate clustering and use the canopy clustering idea for multi-pass clustering. The work extends the current state-of-the-art by including the duplicate clustering step and demonstrate new strategies for speeding up m/r similarity calculations. | URI: | https://open.uns.ac.rs/handle/123456789/4757 | ISSN: | 00330337 | DOI: | 10.1108/PROG-02-2015-0021 |
Nаlаzi sе u kоlеkciјаmа: | EF Publikacije/Publications |
Prikаzаti cеlоkupаn zаpis stаvki
SCOPUSTM
Nаvоđеnjа
1
prоvеrеnо 12.08.2023.
Prеglеd/i stаnicа
12
Prоtеklа nеdеljа
5
5
Prоtеkli mеsеc
0
0
prоvеrеnо 03.05.2024.
Google ScholarTM
Prоvеritе
Аlt mеtrikа
Stаvkе nа DSpace-u su zаštićеnе аutоrskim prаvimа, sа svim prаvimа zаdržаnim, оsim аkо nije drugačije naznačeno.