Giter Club home page Giter Club logo

firewatch's Introduction

firewatch

Firewatch watches Firebase for change events. It does not keep data synchronized like the official Firebase client, so memory usage is very low.

Why?

Say you have a key with millions of children, and you want to know when aan arbitrary change is made to a child. With the official client, you'd have to keep the entire key in-memory -- which may not even be possible due to its large size.

root = new Firebase(...);
root.child('hugeList').on('value', ...);

With Firewatch, you can watch for changes without all the overhead.

watcher = new FirebaseWatcher({
    db: 'example',
    auth: '<secret token>'
});

watcher.watch('hugeList', function(newValue) {
    // newValue = { 'child-key': ... }
});

watcher.connect();

New in v2

Firewatch v1 was lazy and simply requested a watcher on the entire Firebase (the root -- /) and then filtered events to watch callbacks. This worked, but was wasteful since you may end up transferring large amounts of data that you're not interested in. It also incurred a massive penalty on the server (see below), and recent Firebase server changes actually broke this approach.

v2 fixes this by only registering Firebase listeners for the paths you pass in to watch, which is a win for everyone.

The only API change between 1.x and 2.x was the removal of two events: initProgress and serverReady.

Full details in #2.

Startup penalty

Unfortunately, the Firebase service insists on first sending the entire current value of your data when you ask to listen for changes. This means that when Firewatch first connects to Firebase, it must download a copy of all data under keys passed to watch (and throw this data out).

During this process, your Firebase instance will become unresponsive to all clients while the Firebase server copies your data into a server-side buffer. All other clients' operations will queue. Expect 1 minute per GB of data. Once the copy is completed, normal operations resume while the data is transferred over the wire (so network speed does not affect downtime length).

See Also

firesentry is a utility that uses this library. It provides a HTTP API that git pulls a repo and then does hot code reloading of firebase watchers. This allows you to update your code without incurring the startup penalty.

firewatch's People

Contributors

daguej avatar yipingliao avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

nishantdesigns

firewatch's Issues

"Cannot convert undefined or null to object"

This was introduced in #3:

TypeError: Cannot convert undefined or null to object
    at Function.keys (native)
    at convertHelper (D:\casetext\firewatch\watch.js:491:11)
    at D:\casetext\firewatch\watch.js:493:5
    at Array.forEach (native)
    at convertHelper (D:\casetext\firewatch\watch.js:491:31)
    at _convertObjToLevels (D:\casetext\firewatch\watch.js:463:2)
    at handleUpdate (D:\casetext\firewatch\watch.js:507:23)
    at handleMessage (D:\casetext\firewatch\watch.js:209:7)
    at WebSocket.<anonymous> (D:\casetext\firewatch\watch.js:102:5)
    at emitTwo (events.js:100:13)

Created a value (/foo/bar = 1), then deleted bar.

Does not handle merge updates

firewatch does not handle protocol data updates (t: 'd') that are merge (d: { a: 'm' }) rather than set (a: 'd').

This results in some changes getting silently ignored.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.