Comments (5)
You can inotify on the symlink - you're right that signalling across containers is not yet supported.
from git-sync.
You can not use subpath for this because when Linux bind-mounts a symlink, it binds the target of the link, not the link itself. IOW, you get whatever revision is currently checked out (or nothing!) and no updates ever again.
You have to mount the whole directory. But you want it mixed in with other files, which makes it much harder. You can put the symlink in etc and then mount the whole volume (so /etc/data -> /mnt/data/data -> {sha}), but that requires you to change your container image. initContainers can't work because you need access to the final container's filesystem, which doesn't exist. You can't cross-mount container filesystems because they are re-created every time the app restarts.
If mounting the whole directory is acceptable, you can use either the webhook or exec notifications, or you can use inotify to watch the symlink.
Unfortunately, I just don't know of a way to bind-mount a symlink and retain symlink properties.
from git-sync.
sorry for the naive questions here, but how is this suppose to work?
The demo example with hugo
is pretty close to my use cases, and it seems like the hugo comes with --watch
, (but my tool of choice doesn't have that feature). And in this issue you're talking about using inotify
.
Basically, I want to be able to trigger a bash command once git-sync updated sources, but I don't see a clear way of doing that...
from git-sync.
I just realized the widely used fsnotify lib doesn't handle symlinks well. Bah.
Blatantly ripped off from older golang.org/x/exp/inotify
package main
import (
"errors"
"log"
"os"
"path"
"strings"
"syscall"
"unsafe"
)
func main() {
Notify("/tmp/git/link", func(ev *Event) { log.Println("event:", ev.Name) })
}
// Notify calls fn whenever link is moved to (as per inotify). This will not
// return unless there is an error.
func Notify(link string, fn func(*Event)) error {
n, err := newNotifier()
if err != nil {
return err
}
dir, file := path.Split(link)
err = n.watch(dir)
if err != nil {
return err
}
for {
select {
case ev := <-n.Event:
if ev.Name == file {
fn(ev)
}
case err := <-n.Error:
return err
}
}
// Should never get here.
return nil
}
type Event struct {
Mask uint32 // Mask of events
Cookie uint32 // Unique cookie associating related events (for rename(2))
Name string // File name (optional)
}
type notifier struct {
fd int // File descriptor (as returned by the inotify_init() syscall)
wd uint32 // Watch descriptor (as returned by the inotify_add_watch() syscall)
Error chan error // Errors are sent on this channel
Event chan *Event // Events are returned on this channel
}
// newNotifier creates and returns a new inotify instance using inotify_init(2)
func newNotifier() (*notifier, error) {
fd, errno := syscall.InotifyInit()
if fd == -1 {
return nil, os.NewSyscallError("inotify_init", errno)
}
notifier := ¬ifier{
fd: fd,
Event: make(chan *Event),
Error: make(chan error),
}
go notifier.readEvents()
return notifier, nil
}
// watch watches dir for IN_MOVED_TO events.
func (notifier *notifier) watch(dir string) error {
wd, err := syscall.InotifyAddWatch(notifier.fd, dir, IN_MOVED_TO|IN_DONT_FOLLOW|IN_ONLYDIR)
if err != nil {
return &os.PathError{
Op: "inotify_add_watch",
Path: dir,
Err: err,
}
}
notifier.wd = uint32(wd)
return nil
}
// readEvents reads from the inotify file descriptor, converts the
// received events into Event objects and sends them via the Event channel
func (notifier *notifier) readEvents() {
var buf [syscall.SizeofInotifyEvent * 4096]byte
for {
n, err := syscall.Read(notifier.fd, buf[:])
// If EOF...
if n == 0 {
// The syscall.Close can be slow. Close
// notifier.Event first.
close(notifier.Event)
err := syscall.Close(notifier.fd)
if err != nil {
notifier.Error <- os.NewSyscallError("close", err)
}
close(notifier.Error)
return
}
if n < 0 {
notifier.Error <- os.NewSyscallError("read", err)
continue
}
if n < syscall.SizeofInotifyEvent {
notifier.Error <- errors.New("inotify: short read in readEvents()")
continue
}
var offset uint32 = 0
// We don't know how many events we just read into the buffer
// While the offset points to at least one whole event...
for offset <= uint32(n-syscall.SizeofInotifyEvent) {
// Point "raw" to the event in the buffer
raw := (*syscall.InotifyEvent)(unsafe.Pointer(&buf[offset]))
event := new(Event)
event.Mask = uint32(raw.Mask)
event.Cookie = uint32(raw.Cookie)
nameLen := uint32(raw.Len)
if nameLen > 0 {
// Point "bytes" at the first byte of the filename
bytes := (*[syscall.PathMax]byte)(unsafe.Pointer(&buf[offset+syscall.SizeofInotifyEvent]))
// The filename is padded with NUL bytes. TrimRight() gets rid of those.
event.Name = strings.TrimRight(string(bytes[0:nameLen]), "\000")
}
// Send the event on the events channel
notifier.Event <- event
// Move to the next event in the buffer
offset += syscall.SizeofInotifyEvent + nameLen
}
}
}
const (
// Options for AddWatch
IN_DONT_FOLLOW uint32 = syscall.IN_DONT_FOLLOW
IN_ONLYDIR uint32 = syscall.IN_ONLYDIR
// Events
IN_MOVED_TO uint32 = syscall.IN_MOVED_TO
)
from git-sync.
@thockin can you elaborate on how to inotify
the symlink?
I am mounting the data
(symlink) via subPath
but the /etc/data is giving sh: getcwd: No such file or directory
error.
........................................
containers:
- name: alpine
image: alpine:3.13.5
volumeMounts:
- name: gitvolume
mountPath: /etc/data
subPath: data
stdin: true
- name: git-sync
image: k8s.gcr.io/git-sync/git-sync:v3.3.0
volumeMounts:
- name: gitvolume
mountPath: "/workspace"
- name: ssh-key
mountPath: /etc/git-secret/
readOnly: true
env:
- name: GIT_SYNC_ROOT
value: "/workspace"
- name: GIT_SYNC_DEST
value: "data"
- name: GIT_SYNC_WAIT
value: "10"
........................................
- name: gitvolume
emptyDir: {}
If I make the above git-sync as initContainer
then it works fine.
Content is not synced as docker volume symlink is refreshed. (I'm assuming).
Without subPath
the contents can be seen at /etc/data/*
So, is it possible to mount under /etc/data/
instead of /etc/data/data
with git-sync as a sidecar where /etc/
have other directories as well?
from git-sync.
Related Issues (20)
- High vulnerability CVE-2023-6246 and CVE-2023-6779 HOT 1
- Issue with PYTHONPATH Resolution in git-sync 4.0.0 HOT 2
- v4 slack webhook HOT 6
- exechook behavior HOT 4
- Remediate 4.2.2 security vulnerabilities HOT 1
- Support GitHub App authentication HOT 7
- TODO: infra to make #878 work HOT 1
- Curious about real example raising race condition when using v3 HOT 1
- exechook working HOT 8
- TODO: shellcheck test_e2e.sh and test_git.sh HOT 5
- /bin/sh: ./build/test.sh: not found
- test_e2e.sh: use `local` for all variables defined inside functions HOT 2
- git-sync Multiple Vulnerabilities HOT 3
- how can put specific html folder from my git repo to nginx pod? HOT 2
- git-sync produced an intermediate file,the path of command is wrong HOT 1
- How to use GITSYNC_SYNC_ON_SIGNAL parameter through helm in k8s? HOT 2
- Issue with building Git Sync Container HOT 3
- Issue with Git-sync repo inside a repo, Nested repo for airflow files HOT 7
- On initial sync, all files are in deleted status HOT 5
- V4: GITSYNC_REF or V3: GIT_SYNC_BRANCH HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from git-sync.