Giter Club home page Giter Club logo

Comments (5)

thockin avatar thockin commented on August 19, 2024 1

You can inotify on the symlink - you're right that signalling across containers is not yet supported.

from git-sync.

thockin avatar thockin commented on August 19, 2024 1

You can not use subpath for this because when Linux bind-mounts a symlink, it binds the target of the link, not the link itself. IOW, you get whatever revision is currently checked out (or nothing!) and no updates ever again.

You have to mount the whole directory. But you want it mixed in with other files, which makes it much harder. You can put the symlink in etc and then mount the whole volume (so /etc/data -> /mnt/data/data -> {sha}), but that requires you to change your container image. initContainers can't work because you need access to the final container's filesystem, which doesn't exist. You can't cross-mount container filesystems because they are re-created every time the app restarts.

If mounting the whole directory is acceptable, you can use either the webhook or exec notifications, or you can use inotify to watch the symlink.

Unfortunately, I just don't know of a way to bind-mount a symlink and retain symlink properties.

from git-sync.

jeff-knurek avatar jeff-knurek commented on August 19, 2024

sorry for the naive questions here, but how is this suppose to work?
The demo example with hugo is pretty close to my use cases, and it seems like the hugo comes with --watch, (but my tool of choice doesn't have that feature). And in this issue you're talking about using inotify.

Basically, I want to be able to trigger a bash command once git-sync updated sources, but I don't see a clear way of doing that...

from git-sync.

thockin avatar thockin commented on August 19, 2024

I just realized the widely used fsnotify lib doesn't handle symlinks well.  Bah.

Blatantly ripped off from older golang.org/x/exp/inotify

package main

import (
	"errors"
	"log"
	"os"
	"path"
	"strings"
	"syscall"
	"unsafe"
)

func main() {
	Notify("/tmp/git/link", func(ev *Event) { log.Println("event:", ev.Name) })
}

// Notify calls fn whenever link is moved to (as per inotify).  This will not
// return unless there is an error.
func Notify(link string, fn func(*Event)) error {
	n, err := newNotifier()
	if err != nil {
		return err
	}
	dir, file := path.Split(link)
	err = n.watch(dir)
	if err != nil {
		return err
	}
	for {
		select {
		case ev := <-n.Event:
			if ev.Name == file {
				fn(ev)
			}
		case err := <-n.Error:
			return err
		}
	}
	// Should never get here.
	return nil
}

type Event struct {
	Mask   uint32 // Mask of events
	Cookie uint32 // Unique cookie associating related events (for rename(2))
	Name   string // File name (optional)
}

type notifier struct {
	fd    int         // File descriptor (as returned by the inotify_init() syscall)
	wd    uint32      // Watch descriptor (as returned by the inotify_add_watch() syscall)
	Error chan error  // Errors are sent on this channel
	Event chan *Event // Events are returned on this channel
}

// newNotifier creates and returns a new inotify instance using inotify_init(2)
func newNotifier() (*notifier, error) {
	fd, errno := syscall.InotifyInit()
	if fd == -1 {
		return nil, os.NewSyscallError("inotify_init", errno)
	}
	notifier := &notifier{
		fd:    fd,
		Event: make(chan *Event),
		Error: make(chan error),
	}

	go notifier.readEvents()
	return notifier, nil
}

// watch watches dir for IN_MOVED_TO events.
func (notifier *notifier) watch(dir string) error {
	wd, err := syscall.InotifyAddWatch(notifier.fd, dir, IN_MOVED_TO|IN_DONT_FOLLOW|IN_ONLYDIR)
	if err != nil {
		return &os.PathError{
			Op:   "inotify_add_watch",
			Path: dir,
			Err:  err,
		}
	}

	notifier.wd = uint32(wd)
	return nil
}

// readEvents reads from the inotify file descriptor, converts the
// received events into Event objects and sends them via the Event channel
func (notifier *notifier) readEvents() {
	var buf [syscall.SizeofInotifyEvent * 4096]byte

	for {
		n, err := syscall.Read(notifier.fd, buf[:])

		// If EOF...
		if n == 0 {
			// The syscall.Close can be slow.  Close
			// notifier.Event first.
			close(notifier.Event)
			err := syscall.Close(notifier.fd)
			if err != nil {
				notifier.Error <- os.NewSyscallError("close", err)
			}
			close(notifier.Error)
			return
		}
		if n < 0 {
			notifier.Error <- os.NewSyscallError("read", err)
			continue
		}
		if n < syscall.SizeofInotifyEvent {
			notifier.Error <- errors.New("inotify: short read in readEvents()")
			continue
		}

		var offset uint32 = 0
		// We don't know how many events we just read into the buffer
		// While the offset points to at least one whole event...
		for offset <= uint32(n-syscall.SizeofInotifyEvent) {
			// Point "raw" to the event in the buffer
			raw := (*syscall.InotifyEvent)(unsafe.Pointer(&buf[offset]))
			event := new(Event)
			event.Mask = uint32(raw.Mask)
			event.Cookie = uint32(raw.Cookie)
			nameLen := uint32(raw.Len)
			if nameLen > 0 {
				// Point "bytes" at the first byte of the filename
				bytes := (*[syscall.PathMax]byte)(unsafe.Pointer(&buf[offset+syscall.SizeofInotifyEvent]))
				// The filename is padded with NUL bytes. TrimRight() gets rid of those.
				event.Name = strings.TrimRight(string(bytes[0:nameLen]), "\000")
			}
			// Send the event on the events channel
			notifier.Event <- event
			// Move to the next event in the buffer
			offset += syscall.SizeofInotifyEvent + nameLen
		}
	}
}

const (
	// Options for AddWatch
	IN_DONT_FOLLOW uint32 = syscall.IN_DONT_FOLLOW
	IN_ONLYDIR     uint32 = syscall.IN_ONLYDIR

	// Events
	IN_MOVED_TO uint32 = syscall.IN_MOVED_TO
)

from git-sync.

shubhamc183 avatar shubhamc183 commented on August 19, 2024

@thockin can you elaborate on how to inotify the symlink?

I am mounting the data(symlink) via subPath but the /etc/data is giving sh: getcwd: No such file or directory error.

........................................
      containers:
        - name: alpine
          image: alpine:3.13.5
          volumeMounts:
          - name: gitvolume
            mountPath: /etc/data
            subPath: data
          stdin: true
        - name: git-sync
          image: k8s.gcr.io/git-sync/git-sync:v3.3.0
          volumeMounts:
          - name: gitvolume
            mountPath: "/workspace"
          - name: ssh-key
            mountPath: /etc/git-secret/
            readOnly: true
          env:
          - name: GIT_SYNC_ROOT
            value: "/workspace"
          - name: GIT_SYNC_DEST
            value: "data"
          - name: GIT_SYNC_WAIT
            value: "10"
........................................
      - name: gitvolume
        emptyDir: {}

If I make the above git-sync as initContainer then it works fine.

Content is not synced as docker volume symlink is refreshed. (I'm assuming).
image

Without subPath the contents can be seen at /etc/data/*
image

So, is it possible to mount under /etc/data/ instead of /etc/data/data with git-sync as a sidecar where /etc/ have other directories as well?

from git-sync.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.