My father told me he wanted to make USB flash drives of all the scanned and digitized family photos and other assorted letters and mementos. He planned to distribute them to all family members hoping that at least one set would survive. When I explained that they ought to be recipes to new media every N number of years or risk deteriorating or becoming unreadable (like a floppy disk when you have no floppy drive), he was genuinely shocked. He lost interest in the project that he’d thought was so bullet proof.

  • TommySoda@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    edit-2
    2 months ago

    In my life I’ve had several HDD’s and SSD’s fail on me. None of that shit lasts forever and you would honestly be surprised how often they fail. For huge servers like data centers the average failure rate is between 1% and 2% per year, and sometimes even higher. They literally have to replace hard drives on a daily basis because of failure.

    Disk failure is one of the reasons I have copies of all of my important stuff saved to every hard drive on my computer, along with an external just in case. One time my system drive failed and I couldn’t save anything because my operating system stopped working. Had to reinstall the OS and do a data recovery. Ended up with a bunch of photos that were corrupted, distorted, or missing half the photo.

    • st3ph3n@midwest.social
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      I just replaced a drive today that has been running 24/7 for the last 5.18 years. It hadn’t failed yet but was in a predictive failure state due to the amount of bad sectors it has accumulated. 3 other disks in the same giant raid array are showing some bad sectors but not enough to be over the ‘replace me now’ threshold.

  • mox@lemmy.sdf.org
    link
    fedilink
    English
    arrow-up
    0
    ·
    edit-2
    2 months ago

    I explained that they ought to be recipes to new media every N number of years or risk deteriorating or becoming unreadable

    This is important, and for some media, it should be more often than that.

    People forget that flash memory uses electrical charge to store data. It’s not durable. If left unpowered for too long, that data will get corrupted. A failure might not even be visible without examining every bit of every file.

    Keep backups. Include recovery data (e.g. PAR2) with them. Store them on multiple media. Keep them well-maintained. Mind their environment. Copy them to new storage devices before the old ones become obsolete.

    It’s funny that with all our technology, paper is still the most durable storage medium (under normal conditions) that doesn’t cost an arm and a leg.

    • SkyNTP@lemmy.ml
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      It’s funny that with all our technology, paper is still the most durable storage medium (under normal conditions) that doesn’t cost an arm and a leg.

      Sophistication often creates fragility. The human mind marvels at sophistication naturally; appreciation for resilience usually only comes after that fragile thing has broken. Of course it’s too late by then.

      All them young whipper snappers will continue to learn these life lessons the hard way, it seems.

    • MonkderVierte@lemmy.ml
      link
      fedilink
      English
      arrow-up
      0
      ·
      edit-2
      2 months ago

      People forget that flash memory uses electrical charge to store data. It’s not durable. If left unpowered for too long, that data will get corrupted.

      Yeah, but the link in the article, strict checks and no data loss over 52 weeks. Not neccessarily in USB sticks though. And sure, backups.

      • mox@lemmy.sdf.org
        link
        fedilink
        English
        arrow-up
        0
        ·
        edit-2
        2 months ago

        That number comes from a single manufacturer’s performance targets. It is not a guarantee of real-world results. You might be able to get Intel to replace an SSD if one of them corrupts data in under 52 weeks (assuming you notice it) but your data will still be gone.

        Hardware performance can and does vary by brand, model, and manufacturing run. Even the nominally identical cores within a single CPU have slightly different performance limits. YMMV.

        Note also: that 52 week target is halved at just 5° higher power-off temperature.

    • BehindTheBarrier@programming.dev
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      If I had a cent every time an artist on patron had their computer die on them and lost works in progress or all their old stuff… I’d afford a few coffees.

  • hendrik@palaver.p3x.de
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Well, nothing lasts forever. I’d say distributing them on something that lasts 10+ years is better than doing noting. Otherwise they just get lost, buried in the attic or the next harddrive crash takes them.

  • merc@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    To me, this is just another story of the music industry’s technical incompetence.

    Even in the 1990s, everyone would have known that hard drives were not a long-term archival storage solution. This is like crumpling up a piece of paper, tossing it in the corner, then being upset decades later when your “archival solution” had issues.

    • iopq@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      The piece of paper is basically much more likely to survive in a corner barring a fire. I have crumpled pieces of paper from 20 years ago. My PATA hard drive… I don’t even have a computer with that connector

      • merc@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        A bunch of paper tossed into a corner could get wet, mouldy, get munched on by rats, etc. But, I know what you mean. Spinning plates full of magnetized bits with a connector technology that only lasts a decade at most is hardly going to be reliable, even if stored under ideal conditions.

    • some_guy@lemmy.sdf.orgOP
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      I’m sure they considered anyone telling them they needed to spend money to be a pain in the ass the same way companies don’t follow the recommendations of their IT departments.

    • sushibowl@feddit.nl
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      The problem isn’t even the hard drives, it’s how they are managing them. There’s not many digital data storage solutions around that you can dump into a closet for a few decades and then still read.

      You have to regularly test your hard drives, so that when one fails you can take your other copy of the data and put it on a new drive.

  • Phoenicianpirate@lemm.ee
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    I got four HDDs, some are almost 10 years old. They work great but I know that won’t be for forever.

    • Zement@feddit.nl
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      I have probably lost lots of pictures die to head crash. WD especially

        • Onsotumenh@discuss.tchncs.de
          link
          fedilink
          English
          arrow-up
          0
          ·
          2 months ago

          The only one that didn’t die because of my own fault (two externals and a laptop one sigh), was one of the infamous IBM/Hitachi Deathstars.

          • PolarisFx@lemmy.dbzer0.com
            link
            fedilink
            English
            arrow-up
            0
            ·
            edit-2
            2 months ago

            I’ve had many deathstars fail.

            Old sysadmin trick I was taught was to freeze the drives overnight, have used this trick on multiple occasions, but once the drive heats back up it’s really dead. But you’ve generally got ample time to backup the drive before it dies.

    • tibi@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      Zfs is just software raid, not an archival /backup solution. Sure, you can hold data on a zfs array for long term, but not without active maintenance (powering the drives periodically, replacing old drives, doing some kind of data refresh / scrubbing) and backups.

  • Imgonnatrythis@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    I still have a dvdr and good quality discs stored properly can last decades. Not perfect and harder and harder to find readers, but for long term backups its an affordable solution.

  • mctoasterson@reddthat.com
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    This is where “piracy” is actually the industry’s saving grace. Decades or centuries later, will record labels exist and be well-managed (and flush with cash) enough to preserve archival copies of their artists catalogs? Probably not.

    Will obscure weirdos exist all around the world on Usenet, IRC, or seeding torrents? Possibly.

    • frostwhitewolf@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      What is really being discussed here is archiving of master recordings and session files. The publically avaliable releases themselves aren’t really in jeopardy. Orthough piracy probably does provide an extra layer of security to more obscure releases.

      • tunetardis@lemmy.ca
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        I thought I read somewhere that when they were making one of the Toy Story movies, there was some catastrophic data loss that nearly tanked the whole production. But then one of the animators came back from maternity and said wait, I think I have most of it synced to my home server? And the next thing you know, John Lasseter himself is barrelling down the highway to her place and it turned out yeah, she did have it.

    • curry@programming.dev
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      Many films and tv programs survive only thanks to a total stranger keeping their own copy. For a long term survival of any media it has to be copied and distributed far and wide.

  • TechSquidTV@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Im really hoping, waiting, for a good dense long-term storage medium. It doesn’t have to be fast, but large, cheap, and durable. I want a way to backup my plex library, or even, daily backups of documents and project files, and I don’t want to think about them ever again.

    • SkunkWorkz@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      Tape is cheap and durable if you store it properly. Except the tape drive is expensive af.

      Microsoft is working on glass storage. A glass plate can last 10,000 years according to Microsoft. Hopefully that tech will get miniaturized and available to consumers within our lifetimes.

          • Flying Squid@lemmy.world
            link
            fedilink
            English
            arrow-up
            0
            ·
            2 months ago

            It seemed like the person I was talking to didn’t. The implication was that tape was viable as long-term storage. It isn’t. I’ve seen tapes rot after a year. DATs were especially prone to that, but even things like 2" multitrack audio tape can go bad that quickly.

    • kerrigan778@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      Do optical disks degrade if protected from the elements? A stack of Blu-ray disks could store quite a bit.

      • futatorius@lemm.ee
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        The commonly used optical disk technologies degrade over time. CD-RW more rapidly than CDR. It’s even worse for higher-density media.

    • BluesF@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      Punch cards? Stored correctly there’s no reason they couldn’t last many human lifetimes. But… Yeah it’ll take a while to encode everything.

      I would have thought that with modern technology we could come up with something like punch cards but more space/time efficient. Physical storage of data - only one write cycle of course, but extremely durable. Even just the same system as punch cards but using tiny, tiny holes very close together on a roll of paper. Could be punched or read by a machine at high speed (compared to a regular punch card, presumably still Ber slow compared to flash media).

      • futatorius@lemm.ee
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        Paper doesn’t last centuries. Anyway, punched cards don’t have a storage density that’s adequate for modern data volumes. You need something that’ll durably store nanometre-sized marks.

    • SidewaysHighways@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      It certainly would make planning the dang home lab easier. Im in a small place! I don’t have room for all the stuff i wanna play with!

    • LarmyOfLone@lemm.ee
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      M-Disk is rated to last like 100 years. They are also working on a 125 Terabyte CD. Optical storage is the way to go.

        • LarmyOfLone@lemm.ee
          link
          fedilink
          English
          arrow-up
          0
          ·
          2 months ago

          Well yeah but BluRay is still much more expensive and smaller capacity. Lets hope this new 125TB disk works out

        • kalleboo@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          edit-2
          2 months ago

          The main cause of bitrot in older disks is the organic dyes fading (aside from REALLY cheap disks where delamination was a problem), whereas M-Disc uses an inorganic carbon material

      • feedum_sneedson@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        Was it sapphire or something? But one and done. I wonder if you could just keep writing and just “cross-out” the old stuff with that kind of capacity.

        • LarmyOfLone@lemm.ee
          link
          fedilink
          English
          arrow-up
          0
          ·
          2 months ago

          Not sure what sapphire means, but here is the article. Just appending records and differential backups would seem to be the way to go.

          the new optical disks are claimed to be “highly stable so there are no special storage requirements.” The researchers tout an expected shelf life of 50 to 100 years

  • vaxhax@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    It’s not a bad start though, and how hard would it be for the people who have surviving copies to copy it to “the next best new thing” in 10 years? The problem is of course that sequence would need to be continued on, like a tradition, which is doable as well.

    • mister_flibble@lemm.ee
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      Honestly in a use case like family photos, redoing it every x amount of time is probably a good idea anyway so new ones can be added.

  • arthurpizza@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    My system is to duplicate to fresh media once in a while. It’s more hands on, but it’s the only option I have. My NAS will be cloned to new drives in the next few years.

    • Onsotumenh@discuss.tchncs.de
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      You might want to look at snapraid. I’ve recently overhauled my own NAS and love it. It is snapshot based (so not perfect safety) but it is highly configurable and provides parity and scrubbing for corruption even with a JBOD array.

  • Toes♀@ani.social
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Yeah if you’re looking for long term it needs to be archival media. Many people think the flash drive will hold it forever but they are potentially the most fickle.

    • MrSoup@lemmy.zip
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      USB-A is best bet today, will live longer than other formats and USB adaptors will still exist when USB-A will disappear entirely.

        • MrSoup@lemmy.zip
          link
          fedilink
          English
          arrow-up
          0
          ·
          edit-2
          2 months ago

          Werent we talking about usb flash drives?
          Since usb flash drives use usb, I think we can keep using them to store data i long term, rather than using floppy, cd or other analogic archive.

          • floofloof@lemmy.ca
            link
            fedilink
            English
            arrow-up
            0
            ·
            edit-2
            2 months ago

            The problem is that USB flash drives don’t keep their data intact for very long when they’re powered down. It lasts long enough for everyday use, but not even as long as a hard disk for archival.

            • MrSoup@lemmy.zip
              link
              fedilink
              English
              arrow-up
              0
              ·
              2 months ago

              Oh, gotcha. Was thinking about the peripheral type support rather than its actual lifetime.

            • Dave.@aussie.zone
              link
              fedilink
              English
              arrow-up
              0
              ·
              2 months ago

              when they’re powered down.

              There’s no periodic cell refresh in flash memory like there is in DRAM. When USB sticks are plugged in, all you are doing is powering up the flash chip and interface ICs.

              You’d have to read a block then write it back to actually refresh the stored charges in the cells.

              • floofloof@lemmy.ca
                link
                fedilink
                English
                arrow-up
                0
                ·
                2 months ago

                Do SSDs do that automatically in the background, or is all the data I’m not actively refreshing gradually rotting away?

    • leisesprecher@feddit.org
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      But what actually is “archival”?

      Like, what technology normal person has access to counts at least as enthusiast level archival?

      Magnetic tape, optical media, flash, HDD all rot away, potentially within frighteningly short timeframes and often with subtle bitrot.

        • Angry_Autist (he/him)@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          2 months ago

          Holographic storage is a fluff project, the resolution we’d need to match modern density is simply to narrow to be done optically. I mean it sounds fun but will never be practical

          • whocares314@lemmy.world
            link
            fedilink
            English
            arrow-up
            0
            ·
            2 months ago

            That’s just straight up factually incorrect. From the link:

            As a storage technology, Silica offers volumetric data densities higher than current magnetic tapes (raw capacity upwards of 7TB in a square glass platter the size of a DVD), and using beam steering of the laser beam, we’re able to achieve system-level aggregate write throughputs comparable to current archival systems.

      • The Snark Urge@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        M-DISC, at a guess. The media would last long enough at least for grandkids, who will have bigger things to worry about.

        • thejml@lemm.ee
          link
          fedilink
          English
          arrow-up
          0
          ·
          2 months ago

          Don’t forget, you also need drives that work that long and connect to computers or some other device to utilize the bits, and the bus they use must be available and working, and the disk format they’re written in must be readable, and the images themselves encoded with an algorithm that we still have access to, etc. it’s not just the media.

          I think it’s possible, thanks to the retro enthusiasts, we still have access to some things from the 70s and 80s, but they’re getting fewer and fewer, especially in a working state. That’s only 50yrs ago. What happens when you want to go 100? Or 500? A few thousand? We are familiar with journals from the Civil War, and have found items and notes from Egypt, Roman, and Ancient Greek civilizations, how can we preserve what happened in the currently information rich time we live in, for future generations? Especially as much of it migrates online to blog posts and social networks and news sites that eventually shut down due to corporate issues or shifting internet traffic?

          • chiisana@lemmy.chiisana.net
            link
            fedilink
            English
            arrow-up
            0
            ·
            2 months ago

            Upload it to the cloud and make it someone else’s problem to deal with keeping up with the physical medium changes. Then your descendants only have to worry about figuring out how to deal with an outdated file format they can no longer open… and even when they can finally open it, it’d be super low quality… just like how we have to squint really hard at videos from VCDs now days.

            • thejml@lemm.ee
              link
              fedilink
              English
              arrow-up
              0
              ·
              2 months ago

              There have been plenty of cloud services that have shut down and taken their data offline. And plenty of current ones deleted data after users have gone inactive. Or require constant payments to keep accounts active. Cloud, as it exits now, is not the answer to the archival question.

              • chiisana@lemmy.chiisana.net
                link
                fedilink
                English
                arrow-up
                0
                ·
                2 months ago

                You’ll be very hard pressed to find anything else that’d out last the day when all three of AWS, Azure and GCP shutdown and take their data offline.

                I get it though, Lemmy doesn’t want to admit these services exist other than to dunk on them in the most anti-corporate fashion… so continue to pretend such is the case!

                • thejml@lemm.ee
                  link
                  fedilink
                  English
                  arrow-up
                  0
                  ·
                  2 months ago

                  They take your data down pretty quick when you die and stop paying for it. And as much as we all want to think AWS and GCP and Azure are sticking around forever there’s no reason at this time to believe they will be around in 100+ years.

      • MangoPenguin@lemmy.blahaj.zone
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        Like, what technology normal person has access to counts at least as enthusiast level archival?

        Cloud storage? Store it on 2 different providers like B2 and iDrive or something, pretty low complexity.

        • leisesprecher@feddit.org
          link
          fedilink
          English
          arrow-up
          0
          ·
          2 months ago

          Is it? It’s rather expensive and would you really know, if the data is gone or corrupted?

          You’d have to download every single file in certain intervals and check it. That’s not really low complexity.

            • leisesprecher@feddit.org
              link
              fedilink
              English
              arrow-up
              0
              ·
              2 months ago

              And who does that?

              I think you don’t really get my point. I’m not arguing that there are no ways to archive data. I’m arguing that there are no technologies available for average Joe.

              It is hardly a good strategy to basically set up half a datacenter at home.

      • tibi@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        edit-2
        2 months ago

        Hard drives offer the best price/capacity ratio, but they need to be powered periodically (at least once or twice per year). As with any other storage medium, include parity data and have multiple backups to avoid data loss.

        Tape is too expensive.

        Optical media can also be pretty good as long as you get discs made from inorganic materials and store them properly. M-disc is supposed to last like 100 years. The biggest problem is that they are on the path to obsolescence and optical drives may stop being manufactured. Also, it’s a good idea to check on the condition of the discs periodically and redo any that shows signs of degradation (probably a good idea to replace non-M discs every 10 years regardless).

        But regardless of the media, there is no archival method that doesn’t require active maintenance, like periodically checking the data, ensuring you have multiple backups, refreshing any aged media.

  • francisfordpoopola@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Buy Spinrite. It’s not perfect but it’s the best thing available for drive maintenance and recovery. I have used it for over 10 years. If the drive is dying it’ll take forever, but I’ve recovered data that was nearly gone due to sector loss. It goes down to the bit level BTW. Someday Steve will release v7 … someday.

    • n3m37h@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      This is the first time I’ve seen anyone else talk about SpinRite, known about it a long time too

    • LostXOR@fedia.io
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      If your data is really important, you should send it to a reputable data recovery service. Using the drive any more (even with a tool like SpinRite) risks further damage.

      • francisfordpoopola@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        I think you are mistaken and don’t understand how Spinrite works. It reads at the bit level and only reads once at level 1. If the data is going to be lost at the first read then it’ll crash when read by a professional company.

        • LostXOR@fedia.io
          link
          fedilink
          arrow-up
          0
          ·
          2 months ago

          Do you have a source for that? I am unaware of any modern hard drives that support reading individual bits; the minimum unit of data that can be read is generally one sector, or 512 bytes. If the sector fails to be read, the drive will usually attempt to read it several times before giving up and reporting a read error to the PC.

          Data recovery companies can remove the platters from a damaged drive and put them in a working drive, as long as the platters are in good condition, preventing further damage. (If the platters themselves are damaged, you’re screwed either way).

          • francisfordpoopola@lemmy.world
            link
            fedilink
            English
            arrow-up
            0
            ·
            2 months ago

            This is the best I can find in a pinch. It’s possible it reads at the sector level and repairs at the bit level. It’s been awhile since I’ve been knee deep in Steve’s ass and testimonials. Old Security Now episodes have a lot of info on how it works.

            https://www.grc.com/sr/faq.htm

            Data recovery companies work the hardware which can be a point of failure. Spinrite tries to recover the data before the hardware fails. The greater density a disk has, the more failures are expected and error correction just assigns bad sectors as they fail. Between that and the OS, my understanding is that the slow degradation of a drive is managed until it can’t be. Even running Spinrite on a new drive is beneficial because new drives come with bad sectors. By assigning them as bad up front you get ahead of bad sectors and even can catch a lemon before it crashes. I’ve recovered unreadable drives with Spinrite. It’s impressive. It doesn’t solve all issues, but it’s really good.

            • francisfordpoopola@lemmy.world
              link
              fedilink
              English
              arrow-up
              0
              ·
              2 months ago

              I talk out of my ass at times, and the bit level statement could be one of them. Nevertheless Spinrite is a little known but amazing tool for HDD and SDD maintenance and recovery. Just go hop over to the forums. I used to be a member in them back in the late 2000s. You’ll see. There are deep drive nerds and they know their stuff.

  • Random123@fedia.io
    link
    fedilink
    arrow-up
    0
    ·
    2 months ago

    That shouldnt be all doom like.

    Easiest thing is to make 2-3 copies of usbs and label the date the usb began its use, put a rough estimate on when the next generation should repeat the whole process until the pictures become irrelevent.

    • floofloof@lemmy.ca
      link
      fedilink
      English
      arrow-up
      0
      ·
      edit-2
      2 months ago

      USB sticks and SSDs are no good for long-term storage. The data on them degrades rapidly if they’re not powered up. Spinning disks last longer. So your process would be better done with those.