Story 2014-05-16 3KZ What Is Your Offsite Storage Solution?

What Is Your Offsite Storage Solution?

by
in ask on (#3KZ)
We're talking data here, not your funky old couch and cassette collection. Steven J. Vaughan-Nichols is reviewing six solutions for stuffing all your data in the cloud [1]. He reviews Amazon CloudDrive, Box, Dropbox, GoogleDrive, OneDrive, and SpiderOak. He then concludes, lamely , "I can't tell you what the perfect cloud storage is because there's no such thing. It all depends on your needs."

OK, so the article was clickbait, and I'll stick with my current back-up solution: burning lots of DVDs, labelling, and then mailing them offsite in case my house burns down. I'm guessing the Pipedot community can do better: what offsite services do you use and recommend? Any providers you'd avoid? What's the best option for a small business hoping to maintain access to docs from different locations and systems? What's the best option for a homebody nerd making sure his carefully curated collection of .. um .. downloaded images stays backed up in case of catastrophic hardware failures at home?

[1]Footnote: Interesting article, but also a test of whether you have successfully installed this browser plug-in .
Reply 18 comments

Was tempted... (Score: 1)

by songofthepogo@pipedot.org on 2014-05-16 17:01 (#1NZ)

...to answer "my butt" or "right here in my pants", but bravely managed to resist.

Back on topic, I've been avoiding the offsite backup thing because I never thought I had anything so terribly important that it required that sort of protection. Backups, absolutely yes, but offsite? Nope. Then again, I'm also someone who can't imagine the house burning down or flooding (or what if both??). That said, I have briefly looked into Amazon Glacier. Do I really, really need to insure all those photos of my trips and videos of my cats are protected against acts of god/fsm or theft? Those of you who do do offsite backups, what sort of data are you protecting/preserving?

Re: Was tempted... (Score: 2, Insightful)

by Anonymous Coward on 2014-05-16 18:06 (#1P2)

Only the things that would have a huge amount of effort to replace, and have a need. Tax and financial records, copies of legal documents, and some original work I have done. Somewhere around 150 MB total.

Rolled my Own (Score: 3, Insightful)

by commonjoe@pipedot.org on 2014-05-16 19:53 (#1P4)

I'm also someone who can't imagine the house burning down or flooding (or what if both??).
Most of my friends and family got hit by Katrina. I happened to live out of state at the time so almost everything I had was ok. (Almost.) Nearly everything they owned was gone: out of print sci-fi games and comic books, priceless heirlooms and wedding albums and family pieces, vehicles and houses.

Back on topic: I back up on hard drive and swap out hard drives between my apartment and a lock box somewhere off site. There is always at least one hard drive off site. I rotate at least once or twice every two weeks.

I've dealt with my personal files and backups of those of a two small businesses. I was so sick of backup programs that failed (and I've broken every single one I tried), I finally designed my own and got as far as a prototype. (I'm a programmer by profession.) Unfortunately, the prototype never made it out of the very early stages. It has enough to keep me happy but not enough to be viable on the Internet. Basically, it syncs some folders and makes a full copy of other folders. Then it double checks them. It makes a note of any failures.
Those of you who do do offsite backups, what sort of data are you protecting/preserving?
I have about 100,000 files / 200 GB that I backup on a regular basis. A lot of it is music and videos, but there are photos of my family, programs I've half written, stories and books I've written, and information I've downloaded from the deep recesses of the Internet (some of which I've had so long that they cannot be found again).

I'd love to write my own backup program. A lot of professional backups claim to do what I want to do now, but the big question is: will it work when disaster strikes? I've seen failures on the most insane things (and sometimes, the restore / backup program wouldn't even tell you about the error; you just had to accidentally encounter it). 256 character limit for file/folder path names. Crapping out on restore when you have over 50,000 files. Taking over an hour to scan your (large) restore drive so you can recover just one file. Telling you a compare failed, but it won't tell you which file has the problem (the bit rot problem). Windows machines that cannot backup files on a Linux shared drive. (Windows doesn't know what to do with two files named "HelloWorld" and "helloworld".

Then there are the other things that I think should be standard: Deduplicating files on the backup and saving space. (Deduplication seems to be more the norm today, but it wasn't when I started this list.) Multiple versions of a file on backup. (I hear Mac is pretty good at this.) Settings that pop up alarms if the number of old copies on your backup drop below a certain number. Copy of the backup / restore program on the backup disk itself. Backup program that isn't locked into a particular operating system. The ability to skip locked files and come back to them later without user intervention or program termination. The ability to perform a compare on every file on a client and test it against the copies on the server without having both online at the same time except for about 60 seconds. (Yes, that is possible.) The ability to detect if the hard drive or CPU is too bogged down so the backup program lightens the load so the user has more control over the computer. Easy to read reports. Easy to use program. Easy to read documentation about the program. Backups that span multiple computers or multiple hard drives. Stopping a backup in the middle and picking up where it left off several hours or days later.

Here's one I have to see: Plug the external hard drive in and only the backup program takes care of everything... Basically, the backup program leaves the user alone unless something like "A folder did not back up without failure at least once every 24 hours" or every 7 days or whatever the setting is. Then it gets in the user's face.

Sorry for the rant. Maybe such a program exist for the small time user, but I haven't seen it yet. The programs in Linux that I've seen aren't very sophiscated and the Windows versions cost a fair bit of money for stuff that I thought should be standard. If I had the time and energy to program it, I would. It's been a dream of mine to get something out there like that, but life gets in the way.

I have 12 pages of ideas. Despite all of the ideas, I think it's still possible to make it stupid simple for the noob while also having the crazy advanced settings for the expert. I just don't see that option very often in most programs!

Timemachine (Score: 1)

by bryan@pipedot.org on 2014-05-16 23:10 (#1P6)

Apple made a pretty simple backup device called Timemachine. It is a small ARM computer with a large hard drive and it's own WiFi that you place in a closet, plug in, and pretty much forget about it. Unfortunatly, it's only as "remote" as the WiFi or ethernet signal can travel - so huricanes and such are still a problem.

Re: Timemachine (Score: 1)

by songofthepogo@pipedot.org on 2014-05-16 23:56 (#1P9)

Yep, for local backups I'm using a combo of Time Machine (daily) and SuperDuper (weekly). I did not care at all for the frequency with which Time Machine backed up so, after playing around with some other solutions for tweaking its frequency and finding them not suited to my needs, I gave up and used Automator to handle it. I ended up using a similar solution for SuperDuper when it stopped behaving well wrt its inbuilt scheduling feature.

Re: Timemachine (Score: 2, Informative)

by Anonymous Coward on 2014-05-18 04:39 (#1PW)

Bryan's talking about the Apple TimeCapsule, a router/NAS combo.

TimeMachine, of course, is the OSX software.

Re: Rolled my Own (Score: 1)

by songofthepogo@pipedot.org on 2014-05-17 00:02 (#1PA)

Most of my friends and family got hit by Katrina. I happened to live out of state at the time so almost everything I had was ok. (Almost.) Nearly everything they owned was gone: out of print sci-fi games and comic books, priceless heirlooms and wedding albums and family pieces, vehicles and houses.
Man, that's awful. That had to have been simply devastating for them.
I have 12 pages of ideas. Despite all of the ideas, I think it's still possible to make it stupid simple for the noob while also having the crazy advanced settings for the expert. I just don't see that option very often in most programs!
You just might be right. I can think of programs that do some of what you outline, but not everything. If you write such a program, you'd likely have a lot of fans.

Re: Rolled my Own (Score: 2, Interesting)

by commonjoe@pipedot.org on 2014-05-17 03:59 (#1PE)

Man, that's awful. That had to have been simply devastating for them.
Yes. It even affected me 800+ miles away because I suddenly had my folks, my brother, and their dogs living with me for several months. I mention that not for sympathy, but because you said you said:
I've been avoiding the offsite backup thing because I never thought I had anything so terribly important that it required that sort of protection. [Snip] Then again, I'm also someone who can't imagine the house burning down or flooding
Disaster will happen. It's not a question of if, but a question of when, how, and how bad. ("The how" can be really bizarre as it was in the case of friends and family. Interestingly enough, nearly everyone who had a laptop brought their laptop with them before Katrina struck, and that helped them reconnect with friends and resources very quickly.) I had another friend who had a house fire and had no insurance. (I met him several years after the incident.) He had to start over

To me, as a programmer who writes not only programs but short stories and novels and can't remember what he ate for breakfast (even though it's the same thing every morning), the very thought of permanently losing everything digital scares the hell out of me. My life are those programs, stories, and notes / articles I have stored. I'd rather lose all the rest of my earthly possessions than lose everything on my hard drive. I couldn't get in touch with all of my friends again (scattered across the world) if I lost the names, addresses, phone numbers, and email addresses of everyone I know stored only in hard drives and backups.

Most of today's business couldn't live if they lost what was put on storage. Frankly, it amazes me so many companies trust "the cloud" especially when companies or technologies come and go at a whim. I think cloud storage can be great, but I see it as a way to supplement, not replace my hard drive technologies. I wouldn't use the cloud as my only backup because I don't trust their security. It's another reason I'd like to roll my own backup because I'd be in control of the encryption program -- not them. (It's another thing in my design notes.) Also, the cloud cost a lot of money for the sizes I'm talking about. Well, it's a lot of money for me. Also, I don't think my ISP would like me downloading 200+ GB every week for my usual check in addition to my usual surfing.

I know not everyone places the same emphasis as I do on their bytes and bits. I just mention all of it merely as food for thought.

No need for a service. (Score: 3, Interesting)

by Anonymous Coward on 2014-05-16 17:46 (#1P0)

I have a storage array in my basement, and a friend with one as well. We use each other as an offsite location. Sure, it won't cover against a huge hundred mile wide disaster, but then I have far bigger problems.

Re: No need for a service. (Score: 1)

by songofthepogo@pipedot.org on 2014-05-16 18:14 (#1P3)

Ah, that's a clever idea, and seems like it would serve some individuals (i.e., not businesses, but normal people with normal things to back up) well. I think I remember someone else suggesting/using such a method a while back, but had forgotten. This method wouldn't have the kind of reliability that Backblaze or other services would have, but perhaps it's good enough for mortals and it's something I'd definitely consider trying myself, were I to have data I wanted to ensure was backed up offsite ... and friends.

Re: No need for a service. (Score: 1)

by bryan@pipedot.org on 2014-05-16 23:19 (#1P7)

I have two NAS devices - both Synology 8 drive units. I keep one at my house and donated the other to my parents. Because 21 terabytes is too much to transfer over the Internet (at least until Google fiber comes here,) I manually sync them every so often.

No contest (Score: 2, Interesting)

by tempest@pipedot.org on 2014-05-16 17:51 (#1P1)

tarsnap

Re: No contest (Score: 2, Informative)

by songofthepogo@pipedot.org on 2014-05-17 00:08 (#1PB)

Interesting. I've definitely never heard of Tarsnap before. From the description, I'm not sure it'd pass the "so simple my grandma uses it" test, but perhaps they're aiming for the more technically savvy user.

It's a mix (Score: 2, Interesting)

by zafiro17@pipedot.org on 2014-05-17 18:30 (#1PJ)

I've been mixing it up, but I'm still not happy with my backup system. On my Mac I've got a combination of TImeMachine doing basically daily backups, and Chronsync which does backups of the whole machine to a separate disk that I keep on the other side of the house. That second USB disk is basically my "go disk" if Al Qaeda busts in with a nuclear warhead and I need to skedaddle with whatever I can in a rucksack. Videos of the kids and so forth though now takes up too much space for a simple USB disk, and I'm a bit tired of having drawers full of external drives that all require managing and that can - any of them - crash hard and die at any given moment, taking their contents with them. So I've got that stuff on a NAS.

I played with SpiderOak a bit because I dig their philosophy and they offered a native client for openSUSE Linux, which is a big sell for me. I've heard people say its UI is awful but I never thought so - works perfectly well for me, and it's perfectly intuitive. Tarsnap looks damned interesting though, and their pricing is great. I was looking at other rsync providers to back up my NAS and it gets expensive super fast!

Finally, for music not only is all my stuff uploaded to Google Music (which I'm not sure is considered a backup, and I'm not sure I can get it back out, either). And I actually do burn the monthly disk of family photos, manuscripts to my books, and a few other things that are truly irreplaceable. Got to plan for an EMP too, right?

Short Answer: Seafile (Score: 3, Informative)

by Anonymous Coward on 2014-05-18 04:37 (#1PV)

It's the best self hosted open source off site Dropbox-like file synchronization software. Period.

It makes OwnCloud look like the slow, unreliable, and perennially flaky scripted kludge that it is. And it is much more straightforward than git-annex.

Trust me, your friendly anonymous cowardly pal. Seafile is really well made, and it's very easy to use.

It even has group and public file sharing and versioning. (It borrows heavily from git.)

Re: Short Answer: Seafile (Score: 2, Informative)

by zafiro17@pipedot.org on 2014-05-18 10:47 (#1Q4)

I'm definitely in the market for something like that, so I'm checking it out. Funny how cloud backup systems are set to be commoditized within another year or so - tough business to be in! Here's the URL for anyone else shopping: http://seafile.com/en/home/

Multiple approaches (Score: 1, Informative)

by Anonymous Coward on 2014-05-18 06:15 (#1PY)

Several times I lost everything, or close too. Lots of stuff is gone. So, now I store what I can everywhere.Personal files including scanned documents are copied into a truecrypt volume (with all paranoid options used) and uploaded to a free cloud storage. Dropbox is a good start for those looking for a direction. Only upload well protected encrypted files. If they want your data then they can damn well work for it.Other files are uploaded or copied to lots of different places around the web. Useful bits and pieces are put onto wikis and qa sites so I can find them again. Code and useful objects are attached in various online places. Most in duplicate locations so if I need anything I can get it all back.I buy an external hard drive every year, create encrypted files, copy everything across, then send it to a relative with a note asking them nicely to stick it in a secure dry safe place in their house. The drive is inside a plastic box with its power supply and cables. The box is sealed with duct tape then wrapped with three layers or 200um plastic plastic folded multiple times around the box. The outside is coated with a plastic sealer I picked up from the local hardware store for a final seal.Any files I have of 'community value' I make sure several people I know have copies of. Storage is cheap. For a couple of those I have purchased hard drives for them on the condition that quarter of the drive content is never deleted.I always have at least two computers in the house which are either portable or use an external drive with a working encrypted copy of my key files.Why? I will tell you. I have been robbed multiple times. A friend walked away with a laptop of mine with my files still on it. Several drives went missing from a closet. Many hard drive failures. Another friend borrowed a drive formatted it then returned it formatted. Devices have gone missing from places I have worked (stupid me for leaving anything electronic on my desk). House was once directly hit by lightening with most devices surviving but dying within a few months. Paranoid? Me? No. Paranoia is where you THINK they are out to get you.