initial feedback from test of chrisbecke/glusterfs-volume from dockerhub about glusterfs-volume HOT 8 CLOSED

hmm well i just realized i can do this for the subdir issue

volumes:
  influxdb:
    driver: gluster-vol1
  grafana:
    driver: gluster-vol1
  poller:
    driver: gluster-vol1

but i don't think i should have, this looks bad :-)

oddly the containers work just fine, i think the one with the coredump might have failed initial provisioning - not sure if thats related or not

from glusterfs-volume.

One last issue, once installed it seems the driver mis reports creation time on this view.

Also not sure how (and it may have been me because i deleted two things in that volume UI) but somehow i blew away my traefik folder that was mounted this way:

    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "/mnt/gluster-vol1/traefik/acme.json:/acme.json"
      - "/mnt/gluster-vol1/traefik/sites-enabled:/sites-enabled"

I think it was because for some reason after installing the plugin made the - "/mnt/gluster-vol1/traefik/ vol show up in the volumes UI as 'unused' with your driver name.... and i was stupid enough to click delete lol, may be an aberration of just how drivers work... or maybe you are enumerating all folders even if they exists - not sure this is a bug, just a thing to be aware of? (like that test folder... that is one i manually created in the mount point)

from glusterfs-volume.

hmm well i just realized i can do this for the subdir issue

volumes:
  influxdb:
    driver: gluster-vol1
  grafana:
    driver: gluster-vol1
  poller:
    driver: gluster-vol1

but i don't think i should have, this looks bad :-)

This is certainly the intention. The folders / volumes get named the normal way - with the stack name as a prefix. You will see this with normal "local" volumes if you create a data volume for the portainer stack, the resulting volume will be called "portainer_data". Unlike kubernetes docker does not have namespaces, so this prefixing ensures that a "data" volume deployed for stack "portainer" does not conflct with the "data" volume deployed for stack "grafana".

As the GlusterFS plugin stores volumes as folders, well, I dont do much but just use the docker generated volume names that include the prefix.

oddly the containers work just fine, i think the one with the coredump might have failed initial provisioning - not sure if thats related or not

Ah, you refer to the wierd gid:uid combos? This is an artifact of containerization. When containers create things as root:root they come through nicely as thats always 1:1, but each container has its own users and groups that are aliased in various ways to the outer operating system. Normally this is entirely transparent and invisible, except when things are persisted on disk, as a container might have decided to use 1000:1000 for some purpose, and that needs to be preserved on disk, but that gid:uid might have no meaning to the host os / some other meaning entirely.

influx and grafana presumably use non root users for their containers, and the created volumes and files are going to reflect that.

the driver didnt show up here, not sure why, this would give folks ability to create volumes as they see fit, if needed

Ah - not sure how Portainer enumerates drivers. Docker swarm does not install plugins on all nodes for you, its up to you to install a plugin on whichever nodes you want to. And Portainer doesn't seem to gather plugins via the agents, it only lists the plugins on the principal connected agent. I'm not sure.

Also not sure how (and it may have been me because i deleted two things in that volume UI) but somehow i blew away my traefik folder that was mounted this way

Oops. Perhaps this needs to be documented a bit more clearly. The plugin is quite simple and simply manages, and expects, each folder in the root of the gluster volume, to be a docker volume. It doesn't expect or require any other metadata - the presence of a folder will enumerate it as a potential volume that can be mounted. This may, or may not be a desirable feature.

One last issue, once installed it seems the driver mis reports creation time on this view.

An actual bug. Cosmetic issue but would be nice to fix.

from glusterfs-volume.

Thanks for the detailed feedback. In no particular order.

1. ah that makes sense about the guid:uid, i should have figured that out out, i think it was the coredump one that confused me lol, and the portainer UI had a brain-fart at the same time so i made bad assumption about an possible crash. Yes Grafana is a massive PITA with their guid:uid use - makes container a pain with mapped volumes. In this case your plugin removes that difficulty.

2. you were right, I missed installing the plugin on one node, once installed on all the driver was enumerated in the add volume ui.

3. I love the stack name as a prefix of the directory. I think what i was trying to show two perspectives:

• if the admin uses one volume name for the stack each service munges their data into the root of the volume
• if one specifies a unique volume per service one ends up in volume sprawl in the root of the gluster-vol if a stack has many services
• i don't think this is an issue (beyond my own OCD) if you were to make this more publicly / broadly available it would just need to be explained. My preference would still be that each service was ina subdir of the task. But thats entirely my own problem not yours :-)

4. yes I agree the plugin enumerating all dirs in the volume makes sense and documentation of that is all that is needed.

tl;dr this is pretty darn robust and now i need to bug portainer folks about collapsig volumes that span the cluster....

from glusterfs-volume.

you were right, I missed installing the plugin on one node, once installed on all the driver was enumerated in the add volume ui.

This is an interesting point however. Docker uses the presence of plugins as an implicit placement constraint.

So, you could isolate a set of swarm nodes in your intranets 'trusted' zone, provision just those nodes with an iops-gdpr glusterfs alias, and then ensure that your sql databases allocated their data stores using that driver alias, and they'd only ever land on the special servers.

from glusterfs-volume.

So, you could isolate a set of swarm nodes in your intranets 'trusted' zone, provision just those nodes with an iops-gdpr glusterfs alias, and then ensure that your sql databases allocated their data stores using that driver alias, and they'd only ever land on the special servers.

nice idea,
2 months later (and this is asking for trouble me saying this) everything has been running great!!

from glusterfs-volume.

and i was right, i spoke too soon, on a couple of nodes the plugin didn't start at boot on a reboot and I am not sure why. I had to disable and delete another plugin and then enable this one - i will do more testing to see if it was just me having another plugin or something else

from glusterfs-volume.

close

from glusterfs-volume.