Is there any maintainer interest in additional features that would increase parity with watchman?

In particular, it'd be nice to be able to

• query/watch only specific kinds of files (i.e. *.css)
• query files modified since an arbitrary time (or just all files that currently exist)

https://facebook.github.io/watchman/docs/file-query.html has a lot of different types of file queries, but for my use case basic glob or file extension would probably suffice.

Obviously this can be done in userland, but it would be neat if this was on the C++ end.

Are there any hopes for TypeScript support?

Thumbs up for this awesome file watcher, by the way!

Watcher has been crashing on macOS (10.14/10.15) with a segmentation fault. Should be easily reproduced by watching a large javascript repo and running rm -rf node_modules && yarn a number of times. I built a debug version of node 12.18.1 and a debug version of the watcher and was able to catch these stack traces:

[1]

Process 48323 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x000000010600e154 watcher.node`bool std::__1::__tree_is_left_child<std::__1::__tree_node_base<void*>*>(__x=0x00000001039a6f60) at __tree:75:16
   72  	bool
   73  	__tree_is_left_child(_NodePtr __x) _NOEXCEPT
   74  	{
-> 75  	    return __x == __x->__parent_->__left_;
   76  	}
   77
   78  	// Determines if the subtree rooted at __x is a proper red black subtree.  If
Target 0: (node) stopped.
Process 48323 launched: '/Users/mteegarden/dev/repos/node/node' (x86_64)
(llnode) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x000000010600e154 watcher.node`bool std::__1::__tree_is_left_child<std::__1::__tree_node_base<void*>*>(__x=0x00000001039a6f60) at __tree:75:16
    frame #1: 0x000000010600e0bf watcher.node`std::__1::__tree_end_node<std::__1::__tree_node_base<void*>*>* std::__1::__tree_next_iter<std::__1::__tree_end_node<std::__1::__tree_node_base<void*>*>*, std::__1::__tree_node_base<void*>*>(__x=0x00000001039a6f60) at __tree:178:13
    frame #2: 0x000000010600e06f watcher.node`std::__1::__tree_iterator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, void*>*, long>::operator++(this=0x00007ffeefbf6bd8) at __tree:839:11
    frame #3: 0x000000010600e039 watcher.node`std::__1::__map_iterator<std::__1::__tree_iterator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, void*>*, long> >::operator++(this=0x00007ffeefbf6bd8) at map:803:35
    frame #4: 0x000000010600d87f watcher.node`std::__1::__map_iterator<std::__1::__tree_iterator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, void*>*, long> >::operator++(this=0x00007ffeefbf6bd8, (null)=0) at map:808:9
    frame #5: 0x000000010600d18a watcher.node`EventList::toJS(this=0x00000001039354c0, env=0x00007ffeefbf6d80) at Event.hh:66:58
    frame #6: 0x0000000106010df7 watcher.node`Watcher::fireCallbacks(handle=0x0000000102e262c0) at Watcher.cc:104:44
    frame #7: 0x000000010093fb13 node`uv__async_io(loop=0x0000000101ce3080, w=<unavailable>, events=<unavailable>) at async.c:163:5 [opt]
    frame #8: 0x0000000100953fef node`uv__io_poll(loop=<unavailable>, timeout=-1) at kqueue.c:0 [opt]
    frame #9: 0x0000000100940141 node`uv_run(loop=0x0000000101ce3080, mode=UV_RUN_DEFAULT) at core.c:381:5 [opt]
    frame #10: 0x00000001000ddba5 node`node::NodeMainInstance::Run(this=0x00007ffeefbff490) at node_main_instance.cc:130:9 [opt]
    frame #11: 0x0000000100073295 node`node::Start(argc=<unavailable>, argv=<unavailable>) at node.cc:992:38 [opt]
    frame #12: 0x00007fff670a53d5 libdyld.dylib`start + 1
    frame #13: 0x00007fff670a53d5 libdyld.dylib`start + 1

[2]

Process 18582 stopped
* thread #8, stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
    frame #0: 0x0000000103a1b155 watcher.node`std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long(this="") const at string:1421:22
   1418
   1419	    _LIBCPP_INLINE_VISIBILITY
   1420	    bool __is_long() const _NOEXCEPT
-> 1421	        {return bool(__r_.first().__s.__size_ & __short_mask);}
   1422
   1423	#if _LIBCPP_DEBUG_LEVEL >= 2
   1424
Target 0: (node) stopped.
Process 18582 launched: '/Users/mteegarden/dev/repos/node/node' (x86_64)
(llnode) bt
* thread #8, stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x0000000103a1b155 watcher.node`std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long(this="") const at string:1421:22
    frame #1: 0x0000000103a10ab9 watcher.node`std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::size(this="") const at string:948:17
    frame #2: 0x0000000103a1af43 watcher.node`std::__1::enable_if<__can_be_converted_to_string_view<char, std::__1::char_traits<char>, std::__1::basic_string_view<char, std::__1::char_traits<char> > >::value, int>::type std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::compare<std::__1::basic_string_view<char, std::__1::char_traits<char> > >(this="", __t=0x0000700011267780) const at string:3681:23
    frame #3: 0x0000000103a36269 watcher.node`std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::compare(this="", __str="/Users/mteegarden/dev/repos/frontbucket/dist/parcel/present/panel-error.885c0998.js.948365684") const at string:3699:12
    frame #4: 0x0000000103a3621d watcher.node`bool std::__1::operator<<char, std::__1::char_traits<char>, std::__1::allocator<char> >(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)(__lhs="", __rhs="/Users/mteegarden/dev/repos/frontbucket/dist/parcel/present/panel-error.885c0998.js.948365684") at string:3924:18
    frame #5: 0x0000000103a361e1 watcher.node`std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >::operator(this=0x0000000102f06238, __x="", __y="/Users/mteegarden/dev/repos/frontbucket/dist/parcel/present/panel-error.885c0998.js.948365684")(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const at __functional_base:55:21
    frame #6: 0x0000000103a36185 watcher.node`std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>::operator(this=0x0000000102f06238, __x=0x4000000010b5ab0d, __y="/Users/mteegarden/dev/repos/frontbucket/dist/parcel/present/panel-error.885c0998.js.948365684")(std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const at map:517:17
    frame #7: 0x0000000103a35fc0 watcher.node`std::__1::__tree_iterator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, void*>*, long> std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event> > >::__lower_bound<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(this=0x0000000102f06228, __v="/Users/mteegarden/dev/repos/frontbucket/dist/parcel/present/panel-error.885c0998.js.948365684", __root=0x4000000010b5aaed, __result=0x000000010b8bba10) at __tree:2670:14
    frame #8: 0x0000000103a35e99 watcher.node`std::__1::__tree_iterator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, void*>*, long> std::__1::__tree<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, std::__1::__map_value_compare<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, true>, std::__1::allocator<std::__1::__value_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event> > >::find<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(this=0x0000000102f06228, __v="/Users/mteegarden/dev/repos/frontbucket/dist/parcel/present/panel-error.885c0998.js.948365684") at __tree:2599:20
    frame #9: 0x0000000103a35d0d watcher.node`std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, Event, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, Event> > >::find(this=0x0000000102f06228 size=0, __k="/Users/mteegarden/dev/repos/frontbucket/dist/parcel/present/panel-error.885c0998.js.948365684") at map:1375:68
    frame #10: 0x0000000103a34b73 watcher.node`EventList::update(this=0x0000000102f06228, path="/Users/mteegarden/dev/repos/frontbucket/dist/parcel/present/panel-error.885c0998.js.948365684") at Event.hh:35:26
    frame #11: 0x0000000103a34b01 watcher.node`EventList::create(this=0x0000000102f06228, path=<unavailable>) at Event.hh:30:20
    frame #12: 0x0000000103a6e579 watcher.node`FSEventsCallback(streamRef=0x0000000102e10fb0, clientCallBackInfo=0x0000000102f061e8, numEvents=17, eventPaths=0x000000010c0c59a0, eventFlags=0x0000000102dfe000, eventIds=0x0000000102dfd000) at FSEventsBackend.cc:65:13
    frame #13: 0x00007fff3c6d612e FSEvents`implementation_callback_rpc + 2991
    frame #14: 0x00007fff3c6d550a FSEvents`_Xcallback_rpc + 231
    frame #15: 0x00007fff3c6d5406 FSEvents`FSEventsD2F_server + 55
    frame #16: 0x00007fff3c6d7cbf FSEvents`FSEventsClientProcessMessageCallback + 43
    frame #17: 0x00007fff3b13032d CoreFoundation`__CFMachPortPerform + 246
    frame #18: 0x00007fff3b13022b CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 41
    frame #19: 0x00007fff3b130189 CoreFoundation`__CFRunLoopDoSource1 + 527
    frame #20: 0x00007fff3b11817c CoreFoundation`__CFRunLoopRun + 2524
    frame #21: 0x00007fff3b11754e CoreFoundation`CFRunLoopRunSpecific + 455
    frame #22: 0x00007fff3b117362 CoreFoundation`CFRunLoopRun + 40
    frame #23: 0x0000000103a6f453 watcher.node`FSEventsBackend::start(this=0x0000000102f06608) at FSEventsBackend.cc:189:3
    frame #24: 0x0000000103a312a0 watcher.node`Backend::run(this=0x0000000102f064d8)::$_0::operator()() const at Backend.cc:79:7
    frame #25: 0x0000000103a3123d watcher.node`decltype(__f=0x0000000102f064d8)::$_0>(fp)()) std::__1::__invoke<Backend::run()::$_0>(Backend::run()::$_0&&) at type_traits:4361:1
    frame #26: 0x0000000103a311a5 watcher.node`void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, Backend::run()::$_0>(__t=size=2, (null)=__tuple_indices<> @ 0x000070001126deb8)::$_0>&, std::__1::__tuple_indices<>) at thread:342:5
    frame #27: 0x0000000103a30a76 watcher.node`void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, Backend::run()::$_0> >(__vp=0x0000000102f064d0) at thread:352:5
    frame #28: 0x00007fff672992eb libsystem_pthread.dylib`_pthread_body + 126
    frame #29: 0x00007fff6729c249 libsystem_pthread.dylib`_pthread_start + 66
    frame #30: 0x00007fff6729840d libsystem_pthread.dylib`thread_start + 13

Hey there!
Something that I am missing are sync functions.
watcher.writeSnapshotSync()
watcher.unsubscribeSync()

The reason is that when you use a separated thread to use this module you end up with a top level await functions that you have to make some gimmick to be able to use. When you have to exit your program you are not able to use some async functions.
How I am going to write a snapshot without waiting to exit when you receive a process.exit() signal? At this point you need something synchronous because main thread needs to wait the command to finish. It can´t be async. The program is exiting, the main process sends a kill signal, the sub process will do things synchronously while main thread is waiting on process.kill command until everything is done.
So to be able to use this module using async I will have to override the whole exit way of node, then send an ipc, and this ipc will use this async command that will emit a response to main thread, main thread then will be able to exit.
This way of doing things does not work seamlessly with a dedicated thread for this module
Thanks!

🔦 Context

Hi,
I am using @parcel/watcher as part of one of my personal project, and I noticed that my program crash sometime without any error message from node.

After some investigation it appears that @parcel/watcher is segfaulting on this line:

😯 Description

I can't reproduce the issue, and sometime I don't have it for a few days. So what I did is I added an extensive logging to my app so I can get a hint.

One hour ago , I got the crash again so this time I looked at my logs and found this:

[paul@latitude-5410 ~]$ cat /var/log/onemon/onemon.07e9d5ae84.socket.log
...
{
  message: 'CHILD EVENT',
  event: 'close',
  code: null,
  signal: 'SIGSEGV',
  level: 'info',
  timestamp: '2021-03-04 15:10:51'
}

So now that I know I have a SEGFAULT at 15:10:51, I looked at my core dumps (I run on Linux):

[paul@latitude-5410 ~]$ coredumpctl list
...
Thu 2021-03-04 15:10:51 CET   36842  1000  1000  11 present   /usr/bin/node

Then I looked at the stack trace:

[paul@latitude-5410 ~]$ coredumpctl gdb 36842

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fd50a5e9e3b in std::_Hashtable<std::string, std::string, std::allocator<std::string>, std::__detail::_Identity, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, true, true> >::count(std::string const&) const ()
   from /home/paul/dashboard.server/node_modules/@parcel/watcher/prebuilds/linux-x64/node.napi.glibc.node
[Current thread is 1 (Thread 0x7fd4f2ffd640 (LWP 36860))]
(gdb) bt
#0  0x00007fd50a5e9e3b in std::_Hashtable<std::string, std::string, std::allocator<std::string>, std::__detail::_Identity, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, true, true> >::count(std::string const&) const ()
   from /home/paul/dashboard.server/node_modules/@parcel/watcher/prebuilds/linux-x64/node.napi.glibc.node
#1  0x00007fd50a5e777f in InotifyBackend::handleSubscription(inotify_event*, std::shared_ptr<InotifySubscription>) ()
   from /home/paul/dashboard.server/node_modules/@parcel/watcher/prebuilds/linux-x64/node.napi.glibc.node
#2  0x00007fd50a5e91e3 in InotifyBackend::handleEvent(inotify_event*, std::unordered_set<Watcher*, std::hash<Watcher*>, std::equal_to<Watcher*>, std::allocator<Watcher*> >&) ()
   from /home/paul/dashboard.server/node_modules/@parcel/watcher/prebuilds/linux-x64/node.napi.glibc.node
#3  0x00007fd50a5e959c in InotifyBackend::handleEvents() () from /home/paul/dashboard.server/node_modules/@parcel/watcher/prebuilds/linux-x64/node.napi.glibc.node
#4  0x00007fd50a5e98f8 in InotifyBackend::start() () from /home/paul/dashboard.server/node_modules/@parcel/watcher/prebuilds/linux-x64/node.napi.glibc.node
#5  0x00007fd50a5cf9e6 in ?? () from /home/paul/dashboard.server/node_modules/@parcel/watcher/prebuilds/linux-x64/node.napi.glibc.node
#6  0x00007fd513060bc4 in std::execute_native_thread_routine (__p=0x560f06d17210) at /build/gcc/src/gcc/libstdc++-v3/src/c++11/thread.cc:80
#7  0x00007fd512e18299 in start_thread () from /usr/lib/libpthread.so.0
#8  0x00007fd512d41053 in clone () from /usr/lib/libc.so.6

Unfortunately, I am not able to reproduce the issue and most of the time my program don't crash, I am not even able to tell you if there is a pattern or whatever.

One thing I can tell you is that I am using multiple calls to subscribe to watch different directories in the same time, so maybe this segfault is triggered by one of the watcher when some file is updated.:

On the other hand, I can provide you a complete core dump here so you can use gdb or whatever to have more info on this segfault.

🌍 Your Environment

Currently, when installing a package that depends on @parcel/watcher in an Alpine Docker image on my M1 Mac, none of the existing prebuilds match and a rebuild from source is required each time my node_modules changes. It also bloats the image size, since g++/make/python is required.

Is it possible to provide a musl prebuild for darwin-arm64?

My current workaround is to build @parcel/watcher locally for musl and manually copy the whole dependency + extra prebuild to the Dockerfile. It works fine but it would be nice to avoid the hack.

Steps to repro

• Create a directory foo (with foo/App.js inside)
• Write a snapshot
• Rename directory foo to bar
• Call getEventsSince and notice these events in the resolved array:
  • { type: "delete", path: "foo" }
  • { type: "create", path: "foo/App.js" } <-- should be bar/App.js
  • { type: "create", path: "bar" }

Environment

• @parcel/watcher v2.0.0-alpha.10
• default backend
• macOS 10.15.7

Steps:

• setup the watcher to watch on a folder
• from the finder duplicate a file via copy + paste
• 🐛 you get a created event for the file that is being copied

This does not seem to happen when duplicating a folder (that is empty).

I tested on Windows, macOS and Linux how the watcher behaves when the path to be watched is being deleted and found that macOS and Windows can deal with this well: when the folder is being restored, watching continues to work (at least from my basic testing that is what I figured).

But on Linux there is a delete event when the path is being deleted but the watching does not pick up again when restoring the folder.

(I tested this on macOS and Windows)

Use the following code and notice how no event is being emitted from the watcher:

const watcher = require('@parcel/watcher');
const fs = require('fs');

// Subscribe to events
(async function () {
    let now = Date.now();
    let subscription = await watcher.subscribe('<path to folder>', (err, events) => {
        console.log(err, events);
        now = Date.now();
    }).catch(err => {
        console.error(err);
    });
})();

setInterval(() => {
    fs.writeFileSync('<path to file in folder>', 'Hello World');
}, 10);

It looks like the debouncing in the watcher is never allowing to release events but is always catching up with the changes. I would have expected that 1 event is released every debounce interval at least.

Steps:

• clone the project to a folder with spaces in the name
• try to yarn

It fails:

CC(target) Release/obj.target/nothing/node_modules/node-addon-api/nothing.o
  LIBTOOL-STATIC Release/nothing.a
warning: /Library/Developer/CommandLineTools/usr/bin/libtool: archive library: Release/nothing.a the table of contents is empty (no object file members in the library define global symbols)
  CXX(target) Release/obj.target/watcher/src/binding.o
clang: error: no such file or directory: 'folder/node_modules/node-addon-api'
make: *** [Release/obj.target/watcher/src/binding.o] Error 1

As discussed here.

I agree 100% running that on install isn't a good solution. Feel free to open an issue here: https://github.com/parcel-bundler/watcher I might look into it when I have some time, already did this for the source-maps cpp package so it should be fairly straightforward to also do it for the watcher

🐛 bug report

I'm unable to install parcel@next on a computer without internet access.
At work, I do most of my work in an air gapped network (i.e. no internet connection). I usually install NPM packages by copying them over to a local NPM registry within our air gapped network, and then installing them as normal (npm/yarn install whatever). This worked for parcel v1, but does no longer work for parcel v2.

🎛 Configuration (.babelrc, package.json, cli command)

npm i -g parcel@next

🤔 Expected Behavior

Parcel gets installed on the computer.

😯 Current Behavior

It stops at the @parcel/watcher trying to run prebuild-install -r napi || node-gyp rebuild. Since the computer doesn't have internet access, prebuild-install fails with an error, and the installation gets cancelled.

Hi! I'm working on a N-API implementation on top of deno_core and Tokio (https://github.com/littledivy/napi-rusty-v8/) and we're trying to get all of parcel's native modules to work. The implementation is eventually going to be merged in Deno.

Since this is built on top of Tokio, we're avoiding to expose libuv polyfills. This makes @parcel/watcher incompatible. Would it be possible to use napi_async_work instead of libuv for the Watcher?

const path = require("path");
const watcher = require("@parcel/watcher");

(async () => {
  let cwd = process.cwd();
  let opts = {
    ignore: [path.join(cwd, "node_modules")],
  };

  let subscription = await watcher.subscribe(
    cwd,
    (err, events) => {
      console.log("events", events);
    },
    opts
  );

  let events = await watcher.getEventsSince(cwd, "snapshot.txt", opts);
  console.log("eventsSince", events);
})();

Editing node_modules is correctly ignored by the subscription, but getEventsSince is still returning events for node_modules.

• @parcel/[email protected]
• macOS 10.15.3

This allows the removal of the flow-libs entry in Parcel core. Also necessary for #5297.

In VS Code we have moved to parcel-bundler/watcher for most file watching tasks except for non-recursive watching where we still use node.js fs.watch. It would be great if we could have an option to disable the recursive watching and only watch:

• a folder and its children, or
• a file

Depending on the OS this would mean:

• Linux: not to recurse the file tree for adding watchers in children of children
• Windows: not setting bWatchSubtree for ReadDirectoryChangesW
• macOS: I could not find an option to drive this so it would probably require to do some filtering

I don't know how to do this

• macOS 10.15.7
• Watchman installed
• Default backend

I noticed this while writing tests that awaited a writeSnapshot call before reading the snapshot with getEventsSince.

Currently, I wait 100ms after the promise resolves before calling getEventsSince.

A file modification is reported as creation when saving via temporary file (rename), like gedit does. This case is successfully reported in chokidar as one atomic change.

Example with chokidar raw log and parcel watcher:

chokidar raw: rename .goutputstream-K1W8H1 { watchedPath: '/tmp/testDir' }
chokidar raw: change .goutputstream-K1W8H1 { watchedPath: '/tmp/testDir' }
chokidar raw: change .goutputstream-K1W8H1 { watchedPath: '/tmp/testDir' }
@parcel/watcher: [
  {
    path: '/tmp/testDir/.goutputstream-K1W8H1',
    type: 'create'
  }
]

chokidar raw: rename .goutputstream-K1W8H1 { watchedPath: '/tmp/testDir' }
chokidar raw: rename test.txt { watchedPath: '/tmp/testDir' }
chokidar raw: change test.txt {
  watchedPath: '/tmp/testDir/test.txt'
}
chokidar raw: rename test.txt {
  watchedPath: '/tmp/testDir/test.txt'
}
chokidar raw: rename test.txt {
  watchedPath: '/tmp/testDir/test.txt'
}
@parcel/watcher: [
  {
    path: '/tmp/testDir/.goutputstream-K1W8H1',
    type: 'delete'
  },
  {
    path: '/tmp/testDir/test.txt',
    type: 'create'
  }
]

This is related to #62 but not the same. It was found in microsoft/vscode#135902 during our test week where one assignment was to test the new file watcher in VS Code.

There is a check after inotify_add_watch that will throw an error when the call failed:

My understanding is that when the call-site is e.g. in a for loop (e.g. from here), this would essentially prevent any other folders from being watched as soon as the first call to inotify_add_watch fails. I would argue that this needs to be handled more gracefully, i.e. not throw an error but log an error and treat this subscription specially.

To proof the point, here is a scenario where watching fails and it seems to be related to the python3 command uses symbolic links when installing environments. Steps:

• watch a fresh empty folder and log both errors and events
• run python3 -m venv .venv
• you should see an error printed inotify_add_watch failed: Not a directory and if not please rerun until you hit this (could be a race condition)
• run source ./.venv/bin/activate (this might fail and require you to run sudo apt install python3.8-venv first)
• run python -m pip install pip-tools

=> 🐛 many of the file events are not visible now. If you setup file watching via inotifywait -mr <test folder> and run the same steps you can see how many more events are printed.

When running this script the watcher segfaults (basically any time you throw an error in the callback):

let dir = path.join(
  fs.realpathSync(require("os").tmpdir()),
  Math.random().toString(31).slice(2)
);
fs.mkdirpSync(dir);
await new Promise((resolve) => setTimeout(resolve, 100));

let sub = await watcher.subscribe(dir, (err, events) => {
  throw new Error("test");
});

fs.writeFile(path.join(dir, "test1.txt"), "test1");
fs.writeFile(path.join(dir, "test2.txt"), "test1");
fs.writeFile(path.join(dir, "test3.txt"), "test1");
fs.writeFile(path.join(dir, "test4.txt"), "test1");
fs.writeFile(path.join(dir, "test5.txt"), "test1");

This is a feature request to enhance the ignore option to support simple glob patterns.

As an example, I recently came across this PR for nsfw that discusses such support for their library: Axosoft/nsfw#124

Bottom line: ignore patterns are very important on Linux at least, because of the operating system global limit of opened file handles, so you really need to exclude folders from the file watcher to not run into issues with opening too many file handles. Having support for glob patterns in the ignore option would mean, clients could define a pattern such as **/node_modules and have that work regardless of the folder they are in.

Wondering if I need to use slash on the file paths provided by this package.

Thanks!

Somewhere through the current dependency tree, [email protected] gets used.

my recent project package-lock.json shows
[email protected] --> @parcel/[email protected] --> [email protected] --> [email protected] --> [email protected]

parcel-bundler is at latest. @parcel/watcher is in 2.0 alpha now? How can I force the update to my dependencies?

Minimist README says this:

Previous versions had a prototype pollution bug that could cause privilege escalation in some circumstances when handling untrusted user input.

Please use version 1.2.3 or later: https://snyk.io/vuln/SNYK-JS-MINIMIST-559764

I think upgrade to chokidar@3 will fix this. I THINK it's already fixed in 2.0 alpha. Should I report on parcel-bundler instead?

It should be

  "targets": {
    "default": {
      "outputFormat": "esmodule"
    }
  }

Originally posted by @mischnic in parcel-bundler/parcel#6755 (comment)

The subscribe/unsubscribe APIs are currently synchronous, and may do some significant filesystem operations depending on the platform. We should make them async and return promises like the other APIs.

Using [email protected] on Windows 11.

I am a big fan of the ignore option that seems to work nicely across all platforms. On Windows that pattern seems to be sensitive to casing though, meaning if you:

• watch on a path such as c:\....
• configure a ignore pattern starting with C:\...

The ignore pattern will not apply. The same is probably true when different casing is used in folder/file names, but I am less worried about that. The drive letter casing though is often ending up being different.

Steps:

• run a simple watcher as in [1]
• rename a file to same name but different case
• 🐛 2 create events are emitted when I would expect a delete and a create event

const watcher = require('@parcel/watcher');

// Subscribe to events
(async function () {
    let subscription = await watcher.subscribe('<path to folder>', (err, events) => {
        console.log(events);
    });
})();

VS Code itself does some event coalescing (see watcher.ts). I think these rules can be pushed down to the watcher itself so that everyone benefits from this.

I would have expected the event coalescing to emit a change event for this case where a file is quickly deleted and then recreated again:

(async function () {
    let subscription = await watcher.subscribe('<path to folder>', (err, events) => {
        console.log(err, events);
    }).catch(err => {
        console.error(err);
    }).then(() => {
        setTimeout(() => {
            fs.unlinkSync('<path to existing file>');
            setTimeout(() => {
                fs.writeFileSync('<path to existing file>', 'Hello World');
            }, 1);
        }, 1);
    });
})();

From my testing on macOS and Windows, I get a single create event.

The scenario where this is interesting is applications that do "atomic" writes to files, meaning they create a tmp file, write the contents there and then move it over to the target file to have a single file operation for the write.

Originally reported in microsoft/vscode#138863

Steps:

• have a multi user Linux setup (i.e. /home having more than one entry)
• make sure the one user cannot access the folder of the other user
• watch in /home

Error: Bad file descriptor

The problem seems to be the iterateDir function not handling permission errors gracefully:

The return type of fstatat is not checked for -1 and then calling fdopendir results in this error.

@devongovett would you think a PR makes sense to handle this case and do an early return? In my local testing this seems to unblock the issue.

int err = fstatat(new_fd, ".", &rootAttributes, AT_SYMLINK_NOFOLLOW);
if (err == -1) {
    return;
}

It slightly changes the behaviour of the watcher though:

• before: any downstream user would immediately get an error that one of the folders in the workspace cannot be watched
• after: such an error would be silently ignored and a downstream user might be surprised about some of the folders not being watched

I found this while testing support for watching UNC (SO thread) and subst paths.

Steps:

• create a subst for a folder on disk (e.g. subst x: <path>)
• watch x: or x:\\ ([1])
• make changes to files in the watched path

=> 🐛 paths are reported with \\, for example x:\\src\extension instead of x:\src\extension

This breaks mapping the path back to VSCode.

[1]

const watcher = require('@parcel/watcher');

// Subscribe to events
(async function () {
    let subscription = await watcher.subscribe('X:\\', (err, events) => {
        console.log(err, events);
    });
})();

Problem

@parcel/watcher includes all prebuilds (#34) regardless of the architecture. This breaks brew audit for any packages that publish to homebrew and rely on @parcel/watcher in their dependency tree.

Example

Example failure:

==> brew audit code-server --online --git --skip-style
==> FAILED
Full audit code-server --online --git --skip-style output
  code-server:
    * Binaries built for a non-native architecture were installed into code-server's prefix.
      The offending files are:
        /usr/local/Cellar/code-server/4.0.1/libexec/vendor/modules/code-oss-dev/node_modules/@parcel/watcher/prebuilds/darwin-arm64/node.napi.glibc.node	(arm64)

Additional Context

We (coder/code-server) publish code-server (VS Code in the browser) to homebrew. In October 2021, VS Code switched to using @parcel/watcher as their main file watcher. Now, we're running into issues publishing code-server to homebrew.

Workaround

Our current solution is to check the architecture and delete non-native binaries in @parce/watcher/prebuilds. See example:

  def install
    node = Formula["node@14"]
    system "yarn", "--production", "--frozen-lockfile"
    # @parcel/watcher bundles all binaries for other platforms & architectures
    # This deletes the non-matching architecture otherwise brew audit will complain.
    prebuilds = buildpath/"vendor/modules/code-oss-dev/node_modules/@parcel/watcher/prebuilds"
    (prebuilds/"darwin-x64").rmtree if Hardware::CPU.arm?
    libexec.install Dir["*"]
    env = { PATH: "#{node.opt_bin}:$PATH" }
    (bin/"code-server").write_env_script "#{libexec}/out/node/entry.js", env
  end

The installation fails on our build machine with the following error.

> @parcel/[email protected] install /path/to/project/node_modules/@parcel/watcher
> node-gyp-build
make: Entering directory '/path/to/project/node_modules/@parcel/watcher/build'
  CC(target) Release/obj.target/nothing/../../node-addon-api/nothing.o
  AR(target) Release/obj.target/../../node-addon-api/nothing.a
  COPY Release/nothing.a
  CXX(target) Release/obj.target/watcher/src/binding.o
  CXX(target) Release/obj.target/watcher/src/Watcher.o
  CXX(target) Release/obj.target/watcher/src/Backend.o
  CXX(target) Release/obj.target/watcher/src/DirTree.o
  CXX(target) Release/obj.target/watcher/src/watchman/BSER.o
  CXX(target) Release/obj.target/watcher/src/watchman/WatchmanBackend.o
  CXX(target) Release/obj.target/watcher/src/shared/BruteForceBackend.o
  CXX(target) Release/obj.target/watcher/src/linux/InotifyBackend.o
  CXX(target) Release/obj.target/watcher/src/unix/legacy.o
../src/linux/InotifyBackend.cc: In member function 'void InotifyBackend::watchDir(Watcher&, DirEntry*, std::shared_ptr<DirTree>)':
../src/linux/InotifyBackend.cc:9:47: error: 'IN_EXCL_UNLINK' was not declared in this scope
   IN_MOVED_TO | IN_DONT_FOLLOW | IN_ONLYDIR | IN_EXCL_UNLINK
                                               ^
../src/linux/InotifyBackend.cc:76:61: note: in expansion of macro 'INOTIFY_MASK'
   int wd = inotify_add_watch(mInotify, entry->path.c_str(), INOTIFY_MASK);
                                                             ^
make: *** [watcher.target.mk:124: Release/obj.target/watcher/src/linux/InotifyBackend.o] Error 1

I am not super familiar with the C++ build stack, but I can try to extract more information about the system and build environment if you tell me how to do that.

This is a variant of #76 with a different repro. There seems to be a possible race condition between detection a new folder that get's added to the watched root, the resulting call to inotify_add_watch and the folder getting deleted at the same time. If this happens, the watcher terminates with Error: inotify_add_watch on '<path>' failed: No such file or directory

Repro:

const watcher = require('./index');
const fs = require('fs');

let firstDeleteThenCreate = false;

(async function () {
    let subscription = await watcher.subscribe('<path>', (err, events) => {
        console.log(1, err, events);
    }).catch(err => {
        console.error(2, err);
    }).then(async () => {

        for (let i = 0; i < 1000; i++) {
            await fs.promises.mkdir('<path>/folder');
            await fs.promises.rmdir('<path>/folder');
        }

        console.log('Done');
    });
})();

Instead of throwing an error here:

I think it would be better to ignore this event and remove the path from the tree.

Using [email protected] on Ubuntu 20. Steps:

• create a symbolic link pointing to a folder: ln -s <linked-folder-path> <folder-path>
• install watcher on that symbolic link

Error: inotify_add_watch failed: Not a directory

Other watcher liks nsfw are not running into that problem it seems.

Hi. Thanks for your nice work on this lib!
I was wondering if someone could help with using this lib in top level module without await.

  let watcherObj;
  (async () => {
    let subscription = await watcher.subscribe(syncDir, (err, events) => {
    });
    watcherObj = subscription;
  })().catch((e) => {
    childLogger.error(`Error in watcher: ${e}`);
  });

I tried something like this, but i am not being able to get back the subscribe result to the main code.
Any hints?

  watcherObj.unsubscribe();
             ^
TypeError: watcherObj.unsubscribe is not a function

Can´t unsubscribe while exiting the program.

Thanks
Pucci

VS Code itself does some event coalescing (see watcher.ts). I think these rules can be pushed down to the watcher itself so that everyone benefits from this.

I would have expected the event coalescing to ignore this case where a file is quickly created and then deleted:

(async function () {
    let subscription = await watcher.subscribe('<path to folder>', (err, events) => {
        console.log(err, events);
    }).catch(err => {
        console.error(err);
    }).then(() => {
        setTimeout(() => {
            fs.writeFileSync('<path to new file in folder>', 'Hello World');
            setTimeout(() => {
                fs.unlinkSync('<path to new file in folder>');
            }, 1);
        }, 1);
    });
})();

From my testing on macOS, I get a delete event. Interestingly on Windows I do not get an event as expected.

Originally reported as microsoft/vscode#142694

Steps:

• setup parcel watcher to watch on an empty folder
• execute touch something in that folder and verify you see a file event
• execute mkdir -p 1/2/3/4/5 and verify you get an event for the folder 1
• execute touch 1/2/3/4/5/something
• => 🐛 no event is fired

I have debugged this down to here:

inotify seems to fire an event only for the first folder created (1) and parcel watcher rightfully starts to watch 1. However, since watchDir itself is not recursive, the child folders are not getting watched.

It might be nice to have a JS wrapper around the APIs to make them slightly easier to use. The C++ module just exposes functions, but maybe it would be nice to have a class-based API so you don't need to keep passing the same options and callback function back to all of the methods. We could also do some pre-normalization of arguments before sending them to C++, e.g. making sure all paths are absolute etc.

I'm getting this error when trying to run npx parcel -p 1234 --hmr-port 1235 index.html with 2.0.0-alpaha.1.1

/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.23' not found (required by /app/node_modules/@parcel/core/node_modules/@parcel/watcher/build/Release/watcher.node) 

Hi there,

What advantages does this library bring over NodeJS' native fs.watch?

Thank you

I noticed that @parcel/watcher has a memory leak that affects Linux (I've tested Linux and MacOS).

This is most noticeable once you watch a larger number of pages. My reproduction has 10k.

If you loop over writing a snapshot, creating/modifying a file in the watched directory, and then getting events, rss will increase a consistent amount through each loop.

For ~10k files, it's roughly 4.2mb / loop. The amount increases/decrease linearly with the number of files. The memory increase is all in rss but not on the heap which makes me suspect it's held in the native extension.

On MacOS there's a one time increase in the amount of memory but then doesn't increase no matter how many times you run the loop.

Happy to dig in further or help in other ways.

Great package! We like it a lot at Gatsby.

🐛 bug report

🤔 Expected Behavior

Parcel installs correctly.

😯 Current Behavior

Parcel fails to install because python is not available in Alpine containers (e.g. node:10-alpine)

💻 Code Sample

Repro:

$ docker run -it --rm node:10-alpine sh
# cd ~ && yarn init -y
# yarn add [email protected]

Error:

error /root/node_modules/@parcel/watcher: Command failed.
Exit code: 1
Command: prebuild-install -r napi || node-gyp rebuild
Arguments:
Directory: /root/node_modules/@parcel/watcher
Output:
prebuild-install WARN install No prebuilt binaries found (target=3 runtime=napi arch=x64 libc=musl platform=linux)
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | x64
gyp ERR! configure error
gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
gyp ERR! stack     at PythonFinder.failNoPython (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:484:19)
gyp ERR! stack     at PythonFinder.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:406:16)
gyp ERR! stack     at F (/usr/local/lib/node_modules/npm/node_modules/which/which.js:68:16)
gyp ERR! stack     at E (/usr/local/lib/node_modules/npm/node_modules/which/which.js:80:29)
gyp ERR! stack     at /usr/local/lib/node_modules/npm/node_modules/which/which.js:89:16
gyp ERR! stack     at /usr/local/lib/node_modules/npm/node_modules/isexe/index.js:42:5
gyp ERR! stack     at /usr/local/lib/node_modules/npm/node_modules/isexe/mode.js:8:5
gyp ERR! stack     at FSReqWrap.oncomplete (fs.js:154:21)
gyp ERR! System Linux 4.9.184-linuxkit
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /root/node_modules/@parcel/watcher

🌍 Your Environment

Curious if y'all have thought about using https://man7.org/linux/man-pages/man7/fanotify.7.html to watch on Linux? It's much more performant than inotify. You can monitor all changes on a fs mount — which I assume that + filtering out any changes not related to the directory of interest would be pretty performant.

For our use case, we don't care about changes to directories so have to filter the list of path changes to exclude directories. I imagine @parcel/watcher already has this info so it'd be cheaper to ignore these changes at the source.

Hi! Could you please add a license, so we can safely use this library at my company's open-source projects?

There are a bunch of C++ throw statements floating around in the code which aren't currently caught and passed back to JS. We should do that somehow. May require an API change for subscribe to add an error argument to the callback. For other APIs, errors should apply to the promise.