Maybe I'll write a PR at some point, but for now just generic thoughts:

• HDD mounts in cases. Modern helium HDDs don't have bottom middle holes and biggest ones (>12Tb) also don't have side middle holes.
• Powering multiple HDDs. With modular PSUs you can usually ask manufacturer about proper additional cables with 3x or 4xSATA with T- or Y-type SATA connectors with various lengths. Avoid using "one Molex/SATA to 5 SATA" adapters as this may cause failures due to high current on startup and bad adapter quality.
• Hot-swap server cages and other "server" things are usually very noisy and designed to be used with high-RPM small fans. You probably want to avoid them at home because of that. Example: 3x5.25">5x3.5" server cage with 92/80mm fan (like SuperMicro CSE-M35T) would be louder than non-hotswappable bracket from China with 120mm fan.

This is pathetic

Just a suggestion to add 'category.create=mfs' to the mergerfs mount command in /etc/fstab. Otherwise it defaults to epmfs, which is not optimal if you are starting from empty drives - which this tutorial assumes.

https://github.com/trapexit/mergerfs#what-policies-should-i-use:

Why are all my files ending up on 1 drive?!
Did you start with empty drives? Did you explicitly configure a category.create policy? Are you using an existing path / path preserving policy?

The default create policy is epmfs. That is a path preserving algorithm. With such a policy for mkdir and create with a set of empty drives it will select only 1 drive when the first directory is created. Anything, files or directories, created in that first directory will be placed on the same branch because it is preserving paths.

This catches a lot of new users off guard but changing the default would break the setup for many existing users. If you do not care about path preservation and wish your files to be spread across all your drives change to mfs or similar policy as described above. If you do want path preservation you'll need to perform the manual act of creating paths on the drives you want the data to land on before transferring your data. Setting func.mkdir=epall can simplify managing path preservation for create. Or use func.mkdir=rand if you're interested in just grouping together directory content by drive.

Hello!

First off, thanks for a great site and resources for setting up a perfect media server!

I'm about to setup my own PMS following your guides here. I went and installed Proxmox and spun up a LXC container with Ubuntu and now trying to figure out how I pass in the drives and setup mergerfs in the best way but noticed that I can't pass in full disk as is into a LXC or maybe I'm missing something, but seems I can only bind directories? How have you set it up?

1. Ubuntu VM pass through drives as is and do everything in the VM with mergerfs etc?
2. Ubuntu LXC with directory mounts and then mergerfs in Ubuntu container?
3. Setting up drives in Proxmox host with mergerfs and mounting the resulting pool in Ubuntu VM/LXC?
4. Or something else? :)

Are there something to be extra careful with here to not bit yourself in the foot later, something that doesn't play nice with mergerfs or so?

Thanks again!

I was wondering if you had any info on using/setting Backblaze or IDrive to backup your pms?

When setting up i had issues getting write permission working on a windows PC that was connected. I could read the directory but not write to it despite having read only = no and/or writeable =yes. As i understand this is quite a common issue but the guide as is would indicate it should work at that point.

I ended up fixing it by giving myself ownership of the samba directory and adding in a create mask and directory mask to the samba config.

I can provide more details and/or submit a PR but i'm not sure if what i did was best practice, i was an edge case and this isn't common, or if there's a better way to fix this issue.

How are you setting up your storage / disks when you moved to proxmox?

Great stuff, thanks!

Having the https://github.com/n1trux/awesome-sysadmin referenced as well would be a great addition to the top10apps page

The SnapRAID section of the Wiki states that SnapRAID isn't suitable for real-time use cases, and users should consider an alternative. However the page makes no mention of any alternatives. I've did some initial Googling and found no good solutions as well. For a use-case like running daily backups to a NAS or running a database, what real-time alternatives to SnapRAID are there?

Hi! I love PMS and frequently come back to reread it. I love how it continues to change and grow. Something that I'm learning about even more with this whole fun adventure is server security. As you showed, using a Bastion and other security features is recommended for any public facing server. Can you point users like me to resources for how to set up security for this kind of machine more clearly? The https://github.com/geerlingguy/ansible-role-security repo seems like it is a good start, but not complete. I read everywhere that you should have security, but rarely do I find detailed instructions on how to actually set it up and what to run.

I've learned a little about Snort (and Pulled Pork to accompany it). Are they necessary? Recommended? How to implement them correctly/easily? What about Firewalls? As a layman (and I consider myself pretty smart and learned on stuff like this), I still struggle with how to set up a firewall correctly, especially at home without spending hundreds of dollars on expensive equipment? Does the bastion and firewall have to be physically different servers? Or could it be a Proxmox virtual machine on the main server?

Recommend setting cache.files from off to cache.files=partial as =off will not play nicely with some services, such as torrents. from the trapexit/merferfs wiki:

BASIC SETUP

If you don't already know that you have a special use case then just start with one of the following option sets.
You need mmap (used by rtorrent and many sqlite3 base software)

allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mf

Example where cache.files=off causing issues:

linuxserver/docker-deluge#140 (comment)

=off actually worked fine for me with podman until I switched back to docker where I ran into this issue, though it's possible it was a recent update that caused this as I just pulled new images due to the switch

Love this project, Been using it for a couple years now after my xigmaNAS setup was no longer covering my needs.
Now that I have been running for a while I am looking to increase my pms capacity.
I am considering the following.

1. Adding additional volumes, this looks pretty straightforward from the documentation.
2. Removing / Upgrading some smaller volumes, I am not sure the process to do this to ensure there is no loss of data.
   Can you add some documentation on the process to remove / replace a failed volume as well as replacing a smaller disk with a larger one.
   Thanks for this great project and all your selfhosted tips!

Hi, it seems that the picture describing the setup of SnapRAID with mergerfs is missing. Can you add it back? The pictures really help to visualise a lot better. Thanks :)

Link to file: https://github.com/ironicbadger/pms-wiki/blob/main/docs/tech-stack/snapraid.md

You don't support RSS/atom for feed crawling?
Why not? #living-the-rss-life

No issues, just wanted to say thanks for your time and effort put into this , on the other hand my wallet and wife wont be very happy 😂

The go to repo header item when clicked -> presently opens in the current window/tab.

Recommend making this link open in a NEW window/tab, if possible \/

There is a badly formatted link on the home page

On https://perfectmediaserver.com/tech-stack/snapraid/ the image for MergerFS SnapRAID diagram is broken.

It points to https://perfectmediaserver.com/tech-stack/images/diagram-mergerfs-snapraid.png where it should point to https://perfectmediaserver.com/images/diagram-mergerfs-snapraid.png.

It appears this change broke it: 6d7f3fe

On git hub (https://github.com/ironicbadger/pms-wiki/blob/main/docs/tech-stack/snapraid.md) ../images works
but on the pms site (https://perfectmediaserver.com/tech-stack/snapraid/) it needs to be ../../images

Hi Alex,

Great guide, thanks for your work.

The link on docker-compose page to the bash alias section on the quality of life tweaks page is broken.

It appears that the list of bash aliases on the first page is not duplicated on the second, and there is not a corresponding #bash-alias heading or tag in the second page.

I just finished reading the section ZFS with MergerFS and from my understanding it is not clear where the snapraid parity data for the mergerfs disks is actually stored. In the diagram for the snapraid configuration is a extra parity disk. This disk is missing in the diagram for the zfs mergerfs combined configuration. This is confusing.

Is it stored on the mirrored zfs pool or is stored on 1 of the 5 disks or is stored on a unmentioned disk 6?

MergerFS also does not provide mitigation for drive failure, this is where SnapRAID comes in. It is used to calculate a snapshot of parity across all data drives that are not using ZFS. The ZFS drives are configured in a mirror and take care of their own parity and data integrity checks.

Something like this:

The ZFS link on the home page is broken. My assumption link should point to https://perfectmediaserver.com/tech-stack/zfs/

I think it is useful to document how the storage pool would be accessed inside the VM’s. Is this done through Samba or NFS in your current setup?

Tried to fix it, but it's just down to the site not being included on the main page. When setting it it changes the tab title for the page.

Will have a think on how to fix this, but it may land up being a nothing burger of an issue

I've been looking at replacing unraid and came across your wiki (very helpful btw). It has given me several ideas to roll-my-own ubuntu box to replace unraid with snapraid. There is one piece missing in this puzzle though: unraid's "mover" and the ability to use a NVME disk as "cache" for all my data.

Could you make some recommendations/documentation discussing "tiered storage" model for PMS? I think many people would want it.

Unraid's ability to have FIFO writes to a NVME then later offloaded onto spinning hard drives (with spun-down settings to lower power consumption) is a major feature for home media server users. Snapraid (as you suggested) fixes many flaws and gaps that unraid model has in term of data assurance/checksumming.

I think a blog post and configuration examples with a "mover like script" to take care of transitioning data off NVME back onto the archive/spinning-rust disks would be great. While the author of mergerfs @trapexit has some information on this (https://github.com/trapexit/mergerfs#tiered-caching); a working example on your blog I think would greatly help others trying to learn many new things at once to replace unraid.

at this page

I've forked the PMS wiki to add my own review of a couple chassis I've worked with over the years, and saw the CS380 already posted there - I've had some extensive experience with this case, modding it due to some design issues with air flow, along with some backplane/sled concerns, but wasn't sure how one might go about adding such details to an existing review.

How (if at all that is) would you prefer such things be handled?

Is Docker installed on the host itself or inside a Proxmox VM/unprivileged LXC? I found this article on ServeTheHome that deemed the former method not recommend for production. See also this Reddit post.

Running on the host itself makes access to the MergerFS pool a lot simpler, but I’m worried about the security issues some Reddit users brought up. Running inside KVM requires an NFS share which results in extra overhead (related to #6). Running inside LXC does allow for host directory mounts but the storage driver needs to be changed to overlay(2). I’m curious what your current setup looks like.

As a side note, I’m currently in the process of setting up Proxmox + Docker + MergerFS + SnapRaid using your guides as a starting point for my perfect home media server. If you want, I can contribute with PR’s containing my findings along the way.

Hello!

I hate being "that guy" and I know Soon is just whenever you get to it, and you probably have a heap of other stuff on the go - but any guideline on when (or if) you'll have a chance to pull together the Proxmox manual install guide? I'm happy to be a beta tester if that would be helpful...

If it's not going to be a thing or it's going to be a while off, I'll just bite the bullet and wing it, but I do like having a good example to refer to!

Presumably it's not vastly different to the Ubuntu build (other than some stuff being prebuilt due to the nature of Proxmox etc) so I could use that as a exemplar?

The current main home page logo is a png (i.e. raster) image, and thus ~500 kB.

This could be vectorized to a svg to reduce load times and size.

(appeared to load last for me on Starlink tonight, hence prompted the change suggestion..

The old nextcloud.com/try link doesnt work anymore. The new correct one seems to be try.nextcloud.com,

The site footer reads:

Licensed under GPLv3

As seen here:

The question becomes:
Is this referring to the content? The software?

If the license is intended for the content, then... isn't GPL a software-specific license, and wouldn't something like Creative Commons be more appropriate?

The link under About points to soon.perfectmediaserver.com

I suppose you meant begin.

https://github.com/IronicBadger/pms-wiki/blame/main/docs/installation/manual-install.md#L331

Images on docs/01-overview/nas-software-comparison.md does not show on the live site, they do however show in the markdown viewer in github, suggesting funny paths

First of all, thank you for your Traefiki 101 guide—it got me up and running after a lot of searching. However, I ran into one hiccup, and that was not specifying the resolvers here like you do in your infra repo. Once I included them I was up and running, so maybe they should be included in your bare minimum setup. Thanks again, love all you do!

The TiddlyWiki container linked in your top 10 is sadly no longer maintained. The advertised fork also appears unmaintained (last updated in April 2023).

https://github.com/djmaze/tiddlywiki-docker
https://github.com/m0wer/tiddlywiki-docker

When sharing PMS links via social means, the preview is rather mundane but mostly problematic: It's pixelated. Or doesn't exist at all..

I'd like to argue this type of preview can be much improved.

Signal messenger link preview:

Here I think it's just a stretched favicon?

Telegram:

no preview at all..

Element:

Preview could look as fancy as:

link to click/share for testing: https://perfectmediaserver.com/06-hardware/new-drive-burnin/

I just checked Ubuntu Server 20.04.2 and it contains the previously missing devices of /dev/disk/by-id/ata-*, but also kept the scsi-* ones.

nlz@chaos:/mnt$ ls -l /dev/disk/by-id/ | grep sdc$
lrwxrwxrwx 1 root root  9 Jun 30 13:36 ata-ST4000VN008-2DR166_ZDH4RTVD -> ../../sdc
lrwxrwxrwx 1 root root  9 Jun 30 13:36 scsi-0ATA_ST4000VN008-2DR1_ZDH4RTVD -> ../../sdc
lrwxrwxrwx 1 root root  9 Jun 30 13:36 scsi-1ATA_ST4000VN008-2DR166_ZDH4RTVD -> ../../sdc
lrwxrwxrwx 1 root root  9 Jun 30 13:36 scsi-35000c500b27d88be -> ../../sdc
lrwxrwxrwx 1 root root  9 Jun 30 13:36 scsi-SATA_ST4000VN008-2DR1_ZDH4RTVD -> ../../sdc
lrwxrwxrwx 1 root root  9 Jun 30 13:36 wwn-0x5000c500b27d88be -> ../../sdc

So maybe the advice against Ubuntu Server could be downgraded into a warning that Canonical likes to mess stuff up.