Giter Club home page Giter Club logo

Comments (13)

katowulf avatar katowulf commented on August 27, 2024

Correct. It assumes we are monitoring records of data that will be added, deleted, and changed over time.

Can you explain the use case for monitoring a single field? Can't you just field.match(//) against it if your needs are this simple?

from flashlight.

stephencarr avatar stephencarr commented on August 27, 2024

My table has a number of fields, I only want to search against specific fields. I have limited experience with elastic search so pardon my ignorance if I have this back-to-front.

from flashlight.

katowulf avatar katowulf commented on August 27, 2024

Specify the fields you want to search in config.js using the fields array

from flashlight.

stephencarr avatar stephencarr commented on August 27, 2024

Perhaps my original explanation was a little rushed, but this is what I had done and it doesn't seem to work. With config.js specifying fields: ['x', 'y', 'z'] I still get results from matches on fields 'a', 'b', and 'c'.

from flashlight.

katowulf avatar katowulf commented on August 27, 2024

That could be a bug. Would it be possible to share your config.js, data structure, and client code?

from flashlight.

stephencarr avatar stephencarr commented on August 27, 2024

I think Config is the only file I have changed from the cloned code. I am really only just testing this out.

Config.js

/**
 * This config file is provided as a convenience for development. You can either
 * set the environment variables on your server or modify the values here.
 *
 * At a minimum, you must set FB_URL and Paths to Monitor. Everything else is optional, assuming your
 * ElasticSearch server is at localhost:9200.
 */

/** Firebase Settings
 ***************************************************/

// Your Firebase instance where we will listen and write search results
exports.FB_URL = 'https://economatic.firebaseio.com/';

// Either your Firebase secret or a token you create with no expiry, used to authenticate
// To Firebase and access search data.
exports.FB_TOKEN = process.env.FB_TOKEN || null;

// The path in your Firebase where clients will write search requests
exports.FB_REQ = process.env.FB_REQ || 'search/request';

// The path in your Firebase where this app will write the results
exports.FB_RES = process.env.FB_RES || 'search/response';


/** ElasticSearch Settings
 *********************************************/

if (process.env.BONSAI_URL) {
    processBonsaiUrl(exports, process.env.BONSAI_URL);
} else {
    // ElasticSearch server's host URL
    exports.ES_HOST = process.env.ES_HOST || 'localhost';

    // ElasticSearch server's host port
    exports.ES_PORT = process.env.ES_PORT || '9200';

    // ElasticSearch username for http auth
    exports.ES_USER = process.env.ES_USER || null;

    // ElasticSearch password for http auth
    exports.ES_PASS = process.env.ES_PASS || null;
}


/** Paths to Monitor
 *
 * Each path can have these keys:
 * {string}   path:    [required] the Firebase path to be monitored, for example, `users/profiles`
 *                     would monitor https://<instance>.firebaseio.com/users/profiles
 * {string}   index:   [required] the name of the ES index to write data into
 * {string}   type:    [required] name of the ES object type this document will be stored as
 * {Array}    fields:  list of fields to be monitored (defaults to all fields)
 * {Function} filter:  if provided, only records that return true are indexed
 * {Function} parser:  if provided, the results of this function are passed to ES, rather than the raw data (fields is ignored if this is used)
 ****************************************************/

exports.paths = [{
    path: "cars",
    index: "firebase",
    type: "car",
    fields: ['MANUFACTURER', 'MODEL', 'YEAR']
}];


/** Config Options
 ***************************************************/

// How often should the script remove unclaimed search results? probably just leave this alone
exports.CLEANUP_INTERVAL =
    process.env.NODE_ENV === 'production' ?
    3600 * 1000 /* once an hour */ :
    60 * 1000 /* once a minute */ ;

function processBonsaiUrl(exports, url) {
    var matches = url.match(/^https?:\/\/([^:]+):([^@]+)@([^/]+)\/?$/);
    exports.ES_HOST = matches[3];
    exports.ES_PORT = 80;
    exports.ES_USER = matches[1];
    exports.ES_PASS = matches[2];
    console.log('Configured using BONSAI_URL environment variable', url, exports);
}

from flashlight.

katowulf avatar katowulf commented on August 27, 2024

So all we would need is a short sample of your data and the client-side code you're using to read the results and determine that it searches on all fields. Then I should be able to pinpoint why you're not getting the desired results. Feel free to email data to wulf at firebase or just sanitize sensitive fields; a few records should suffice.

from flashlight.

stephencarr avatar stephencarr commented on August 27, 2024

Data emailed to you, thanks.

from flashlight.

jwall avatar jwall commented on August 27, 2024

I have the same issue. No matter what I put in the fields array be it

exports.paths = [
                 {
                     path:  "profiles",
                     index: "firebase",
                     type:  "profile",
                     fields: ['foobar']
                 }
                 ];

I always get back a match on all fields when I would think if I don't have any key of 'foobar', I would get no matches.

from flashlight.

katowulf avatar katowulf commented on August 27, 2024

Thanks for the update; will have a look at this soon.

from flashlight.

dustinboss avatar dustinboss commented on August 27, 2024

So, I dug into this, and it looks like the "fields" param in export.paths isn't referenced anywhere else in Flashlight. The logical place would be in PathMonitor at line 6 (where the Firebase ref is being constructed), but I think that even that would be unnecessary.

Perhaps just eliminate "fields" as a parameter altogether, and just create more complete paths. Example: path: "profiles" could be path: "profiles/foobar"

In either case, having "fields" in the config.js comments is confusing, since it doesn't do anything as it currently stands.

from flashlight.

katowulf avatar katowulf commented on August 27, 2024

Fixed in latest head branch. Thanks gents!

from flashlight.

stephencarr avatar stephencarr commented on August 27, 2024

w00t! 👍

from flashlight.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.