I’m in the process of starting a proper backup solution however over the years I’ve had a few copy-paste home directory from different systems as a quick and dirty solution. Now I have to pay my technical debt and remove the duplicates. I’m looking for a duplication removal tool.

  • accept a destination directory
  • source locations should be deleted after the operation
  • if files content is the same then delete the redundant copy
  • if files content is different, move and change the name to avoid name collision I tried doing it in nautilus but it does not look at the files content, only the file name. Eg if two photos have the same content but different name then it will also create a redundant copy.
  • FryAndBender@lemmy.world
    link
    fedilink
    arrow-up
    0
    ·
    1 year ago

    Here are the stats from a backup of 1 server with approx 600gig


                       Original size      Compressed size    Deduplicated size
    

    This archive: 592.44 GB 553.58 GB 13.79 MB All archives: 14.81 TB 13.94 TB 599.58 GB

                       Unique chunks         Total chunks
    

    Chunk index: 2760965 19590945

    13meg… nice