Deduplication ANalyzer

collector: go through the trace and build the chunk, file, region, and container databases

1. refs_distrbution_exporter: go through the chunk/region/container database and print all reference numbers

• refs_classifier.py: classify chunk/region/container according to reference numbers

2. refs_locality_exporter: go through the trace and output all reference numbers in logical or physical locality

• refs_locality_correlation_calc.py: with the output of locality_sequence, calc the correlation coefficient of reference number
• refs_locality_distance_analyzer.py: with the output of locality_sequence, calc the correlation coefficient of reference number

1. chunk_refs_distance_analyzer: go through the chunk database and calc the distance between chunk references

2. chunk_size_analyzer: go through the chunk database and calc the chunk size distribution, with different reference numbers

1. file_size_analyzer: go through the chunk and file database to get file size distribution

2. file_type_analyzer: go through the chunk and file database to get the fractions of popular file types

3. file_refs_source_analyzer: go through the chunk database and analyze the relationship between reference and file