Article 536G7 Seeking Recommendations: Linux only file server setup that's as fast as local drives, without permission hell?

Seeking Recommendations: Linux only file server setup that's as fast as local drives, without permission hell?

by
lightnb
from LinuxQuestions.org on (#536G7)
I'm looking for recommendations for a network and server setup, for the following scenario:

A Linux only digital art studio (3d animation, 2d graphics, video production and editing, music production, etc) with a few users.

Primary needs are:
  1. Ridiculous amounts of space. I'm currently using 98% of a 2TB local disk.
  2. Speed of I/O over LAN should be same as SSD in local computer.
  3. Better Organization of files allowing centralized backups.
  4. Work from a different workstation and share files as a team.
  5. Secure remote access over public internet, for work that doesn't require high speed.
Security:
  1. NFS Security and backup needs should be based around accidents, hardware failure, and internet threats, not internal malice.
  2. If an employee "goes crazy", they could take a sledgehammer to the computer and all the file permissions in the world won't stop them. (Remote access is a different story)
  3. We'll probably use some kind of script rotated rsync setup to backup everything into daily/weekly/monthly sets.
Computers:
  1. 2-3 user workstations used for music production, animation and video editing. Audio samples, video, files, etc, live on a file server accessible from all workstations.
  2. An unknown (at this time/scaleable) number of headless render servers that read 3D files, render them, then write the rendered image sequence to a location on the file server.
Usage:
  1. You can pretty much assume that there will be some shared music library with people playing music off of it onto their workstation while working.
  2. Music production using a shared library of samples, LAN only.
  3. Rendering - LAN machines, but we may want some way to crowdsource rendering, where anybody can help render frames if it comes to it. This might use a custom PHP script on a webserver though, so the webserver would be the SSHFS user, and PHP would handle the public facing stuff.
  4. Remote work - write a script at Starbucks / remote voice actor uploads recording of session / remote artist opens and saves work to central server for collaboration/backups.
  5. Work on something on a different LAN workstation while my main one is doing a long test render.
  6. Collaborate with other artists on the LAN, Pull in sound effects, music compositions, renders, from other workstations, put together in video editor/compositor. (LAN only for this).
Technology:

NFS, SSHFS, Samba, or has something better come along?
  1. Samba, has always been Windows oriented. Is there any reason to use this on a pure Linux environment?
  2. NFS - You need all your user IDs to have the same number, or an LDAP server, which is just one more thing to maintain or break.
  3. SSHFS - I use this for website development. I have no idea how it performs with large files. Will probably add this anyway (in addition to something else) so we can access files remotely.
This is a pure Linux setup, no need to cater to Windows clients, etc. Unless someone can make a compelling case for Samba in a pure Linux environment, I'm going to assume that we're either going with:

1. SSHFS by itself

or, if NFS is faster on a LAN or with larger files,

2. NFS for LAN, and SSHFS for guests and remote work (with the understanding that "heavy" work (video editing, audio production) will have to be local due to public network bandwidth).

Considerations:

I need a solution that "just works". I don't want to have to stop working to mess with file permissions, and especially not several times per day. Ideally, I want to set it and forget it, except adding new users when necessary. Subfolders to infinite levels should just work, and no client should be able to mess up the system, even root access clients.

I've done an office before with LDAP and NFS remote homes, and it makes everything sluggish. I'm not sure how important "roaming profiles" is and can do without it. Or maybe there is a smarter way to set it up so programs that "stream" data to the home folder (ie browsers, GIMP, etc) are separate from config files that just get read once at login.

Permission Hell:

I've never been able to get "sticky bits" or umasks to work right, and I always end up with permission hell where new files and folders are created by different users, not group writable, etc. If someone knows how to do the equivalent of a chgrp mygroup /share -R and chmod g+r /share -R on a cron that runs every minute, which is what sticky bits is supposed to do(?), but for whatever reason, always breaks / never works, that would be great.

I really need an enforceable "group lock" option that simply doesn't allow any files or folders under a top-level share to be any group other than the specified group for the share and always be group writable (if I want the whole share to be), regardless of what the user/computer creating the file says. Some kind of an enforceable, transparent, automatic group permission override on every file write. It needs to be at the server level, so a misconfigured client can't screw up the permissions for eveyone else no matter how hard it tries.

What I really need is a way to set ACLs/permission on a folder and at the folder, say "everything beneath this folder now or in the future at any number of levels ALWAYS inherits permissions from the top folder permissions, unless server root says otherwise explicitly, NO EXCEPTIONS!". I've never found an option that does this and works right.

Worth getting into ACLs or just a bigger headache?

Hardware:

Is there any advantage to have "faster" hardware on a file server? ie, more RAM (past a point) or a faster processor (past a point)?

If I have an AMD Phenom II X6 1090T (3.2Ghz) Processor with 8GB of RAM, is there any performance difference whatsoever going to a Ryzen 3950 with 128GB (for example)? Other than the disk drives, disk controller and network card, is it correct to say that for a file server, CPU doesn't matter all that much? Sure, RAM can cache, but it's not going to cache terabytes of data.

What about an AMD Athlon II X4 630 Processor (2.8 GHz) and 4GB? (I happen to have these old computers already). I don't want to gimp performance with old hardware, but if the old stuff doesn't matter for serving files, there's no point spending money. (I've also seen similar spec old computers for sale for $30-$50 on marketplace).

Disks:

How failure prone is a multi-volume logical volume without RAID? ie. If I have three 2TB disks, and make them into a 6TB logical volume, and one drive fails, what happens? It's probably better not to span across drives and simply create a new share if needed? They make 15TB drives now...

OS:

I'm most familiar with Debian, but I'd probably go with CentOS for a file server since it's going to have a "proven"/stable set of NFS, OpenLDAP, etc. (Ubuntu Server has shipped with buggy NFS and LDAP implementations before).

MISC Questions:

What type of network architecture would be required to get the same speed as local SSDs? How fast of a network do we need? Gigabit? Or beyond? (When does disk IO become the issue?) (This question pertains to LAN only, not over the public internet.)

At some point, the Motherboard controller and/or the physical drive become the bottleneck.

What are the performance considerations of SSHFS vs NFS on a LAN for streaming large files (ie video editing)?latest?d=yIl2AUoC8zA latest?i=BBH2Qnv34F4:54ZQeB05DmA:F7zBnMy latest?i=BBH2Qnv34F4:54ZQeB05DmA:V_sGLiP latest?d=qj6IDK7rITs latest?i=BBH2Qnv34F4:54ZQeB05DmA:gIN9vFwBBH2Qnv34F4
External Content
Source RSS or Atom Feed
Feed Location https://feeds.feedburner.com/linuxquestions/latest
Feed Title LinuxQuestions.org
Feed Link https://www.linuxquestions.org/questions/
Reply 0 comments