Hi reyesr,
Thank you for creating such a powerful search engine!
I'm writing a script for a program that use javascript as its script language. But I'm new to javascript, and I have some difficulties to figure out a couple of things.
First, I'm using the "score engine", I know that we have a score property for each search result. Can we add a second score to each item?
For example, if I search for "tom", and I select "Tom Cruise" from the results, then "Tom Cruise" gains one point. Each time I select one item from the results, that item will gain one point. The score will be stored in the database. So the next time I search for a string, if one item's score (the score generated by "score engine") are equal to another one's score, the item with a higher score(read from the database) will be displayed in front of the item with a lower score.
Can I do this using fullproof?
Second, I want to add some categories to my data, so that I can search results by categories. For example, if I search for "people:tom", I can use regex to split it to "people" and "tom". "people" is the category, "tom" is the string I want to search. Then the search engine will only search for results in "people" category. How can I do this?

Sorry for my poor English. ;-)

Thanks
Zhiqiang li

The search field and button do not display. This was a test using the animal example.

Any plan for compiling with closure compiler?

I am interested in using togather with my indexeddb library https://bitbucket.org/ytkyaw/ydn-db

If i try to split text which ends with the split object - index file will not be created.
For example: I have text, and i try to split it with ".", so if text ends with "." last element of the array it will be empty (even if there are spaces after, they will be trimmed with Fullproof engine). As a result callback in engine.open() doesn't execute

Is apostrophe considered not a proper letter? If yes how do I add it to the set of proper letters so that when searching for "it's" I stop getting results with "it".

Hello.

There are invalid chars in the source code.
Check the pich char at line 17
https://github.com/reyesr/fullproof/blob/master/src/misc/dataloader.js#L17
and line 30
https://github.com/reyesr/fullproof/blob/master/src/misc/dataloader.js#L30

Thanks.

I have been receiving a runtime error in VS when trying to open an indexeddb store.

Seems like this line in indexeddb_store.js was causing it:
this.dbVersion = version || "1.0"; --> Corrected --> this.dbVersion = version || 1.0;

& now everything works fine.

P.S>
Important: The version number is an unsigned long long number, which means that it can be a very big integer. It also means that you can't use a float, otherwise it will be converted to the closest lower integer and the transaction may not start, nor the upgradeneeded event trigger. So for example, don't use 2.4 as a version number:
var request = indexedDB.open("MyTestDatabase", 2.4); // don't do this, as the version will be rounded to 2

The function returned by fullproof.make_synchro_point does not check to see if it has already called the callback it is passed. This can result in the callback being called more than once (for example from fullproof.AbstractEngine.prototype.injectDocument). I don't know all the details about exactly when this happens, but I just spent several hours tracking this down (mainly scratching my head over my own code) and when I added a simple guard to make_synchro_point my problem immediately went away.

Pull request coming up.

There are several minor issues that stop the build-all.sh script from running cleanly.

Web workers do not have access to the window object, so these lines do not work for finding the IndexedDB API

        fullproof.store.indexedDB =  window.indexedDB || window.webkitIndexedDB || window.mozIndexedDB || window.msIndexedDB;
        fullproof.store.IDBTransaction = window.IDBTransaction || window.webkitIDBTransaction || window.mozIDBTransaction || window.msIDBTransaction || {};

fortunately, on Chrome at least, indexedDB and IDBTransaction are both present in the global scope, so there is a trivial fix (coming up in a pull request).

If inserted string contains word "constructor" - engine fails during injecting.

Line 94 of scoring-engine.js has this:

callback(new fullproof.ResultSet(store.caps.getComparatorObject()));

But store does not seem to be defined.

I'd like to search for "cat OR dog". What is the best way to do that?
If it's not possible how to search for more than one search string by sending only one request?

The first step of the tutorial causes users to fall at the first herdle
<script type="text/javascript" src="fullproof-all.js"></script> does not exist.

i am a little confused, so my apologies if this is covered in the docs somewhere. i am inserting text into a fullproof search engine and the callback for injectDocument() is getting called. however when i try to do a lookup() on a word in the document it can take several minutes before it actually comes back with a match. (until then, the result set is false.) my questions are:

1. is this long wait while it is actually building the index?
2. if so, is there a way to speed it up (the text is only a few hundred words long)
3. is there a callback i can set somewhere for when the entire index is ready to be searched?

here is what my code looks like:

searchEngine.injectDocument(text, node.id, function() { _doneIndexing(node); });

and the results, from console (each attempt was done about 15 sec apart):

booker.search('bluff');
undefined
nope! popup.js:312
booker.search('bluff');
undefined
nope! popup.js:312
booker.search('bluff');
undefined
nope! popup.js:312
booker.search('bluff');
undefined
fullproof.ResultSet {comparatorObject: Object, data: Array[1], last_insert: "7", insert: function, merge: function…}
 popup.js:309

(the function _doneIndexing() is called almost immediately after the inject happens, for what it is worth.) i wonder if it has to do with me doing multiple iterations of injecting documents? i step through dozens of nodes in a loop and inject the text content from each via the above method. i am trying to discern why i am getting these long delays and some strange lookup results.

thanks!
-jon

I'm writing an offline HTML5 app. People go to my website, and it loads an interface to manage their own personal data. No other server requests are made other than the initial loading of the page.

This data can be quite large, as the data might grow without my control. However I'd like my users to be able to safely backup their data, say to one or several JSON files to their local computer. So they can decide to maybe safely store their data somewhere else. Is this possible through fullproof? Also I'd want to re-import this data, but this is possible thanks to the examples shown on the homepage.

Any insight if this might work for my scenario? Also, last question, can I search for specific field-names? For example in Lucene you can do name:luca and it will only search for stuff in the field name.

Thanks and great job!

how does one remove an item from the an index? basically the opposite of injecting, i guess. when i delete a document that the index references, i wish to remove it from the index as well.

thanks,
-jon

It would be nice if the readme had a list of supoorted browsers or even required technolgies so that users don't have to do a whole load of testing / research to know if they can use this very nice library. 👍

Is it possible to update the indexes (ie. inject more items) after the initialization? How would one do that?

It looks like in this code (common-engine.js line 227), index can be undefined in the loop:

fullproof.AbstractEngine.prototype.injectDocument = function(text, value, callback) {
    var synchro = fullproof.make_synchro_point(function(data) {
        callback();
    });

    this.forEach(function(name, index, parser) {
        if (name) {
            parser.parse(text, function(word) {
                if (word) {
                    index.inject(word, value, synchro); // the line number is the value stored
                } else {
                    synchro(false);
                }
            })
        }
    }, false);
    return this;
};

I'm calling injectDocument on the result of engine.open and seeing a stack trace that looks like this:

Error in event handler for 'undefined': Cannot call method 'inject' of undefined TypeError: Cannot call method 'inject' of undefined
at chrome-extension://gbplakaibfhhognfeabdecdmbnildeil/vendor.js:2424:444
    at chrome-extension://gbplakaibfhhognfeabdecdmbnildeil/vendor.js:2414:306
    at f (chrome-extension://gbplakaibfhhognfeabdecdmbnildeil/vendor.js:2413:158)
    at parse (chrome-extension://gbplakaibfhhognfeabdecdmbnildeil/vendor.js:2413:308)
    at parse (chrome-extension://gbplakaibfhhognfeabdecdmbnildeil/vendor.js:2414:79)
    at chrome-extension://gbplakaibfhhognfeabdecdmbnildeil/vendor.js:2424:420
    at fullproof.AbstractEngine.forEach (chrome-extension://gbplakaibfhhognfeabdecdmbnildeil/vendor.js:2426:189)
    at fullproof.AbstractEngine.injectDocument (chrome-extension://gbplakaibfhhognfeabdecdmbnildeil/vendor.js:2424:391)
    at Object.Liber.Fullproof.add (chrome-extension://gbplakaibfhhognfeabdecdmbnildeil/background.js:822:18)
    at Object.Liber.Index.add (chrome-extension://gbplakaibfhhognfeabdecdmbnildeil/background.js:874:34)

?

First let me say Fullproof looks impressive. It is terrific to see the code is heavily commented and you've written a tutorial etc. If only all dev's did this.

I'm wondering whether you've given any thought to using Fullproof on the server with Node.js and say MongoDB. MongoDB doesn't have full text search.

I couldn't agree more with your comment on Browser based databases, it is truly a mess. I'm using WebSQL in one app and am looking at IndexedDB/WebSQL shims and a newish library that lets you write database access code that works on IndexedDB, WebSQL and LocalStorage. Let me know if you want links.

Finally any plans to provide build's of the minified and full code in a single JS file?

-Neville

This code in boolean-engine.js:

for (var i = 0; i < array_of_words.length; ++i) {
    unit.index.lookup(array_of_words[i], lookup_synchro);
}

causes a "Cannot call lookup method of undefined" error when browsing examples/tutorial.html.

IE results differ to chrome. Why is this?

Most of the examples fall over in old because the .trim() function doesn't exist.

It may be worth including the following fix in within the library.

if(typeof String.prototype.trim !== 'function') {
   String.prototype.trim = function() {
     return this.replace(/^\s+|\s+$/g, ''); 
   }
 }

IE10 throws InvalidAccessError in indexeddb_store.js line 368

Hi,

fullproof looks like something that i can use but one disadvantage is initial building of indexes. So couple questions/ideas about that:

Would it be possible to build indexes on server and send it (as json maybe?) to client together with actual data?

Can indexes be built by web workers in paralleled manner?

From line 389 in indexeddb_store.js:

        openRequest.onupgradeneeded = function(ev) {
            createStores(ev.target.result, reqIndexArray, self.metaStoreName);
            updated = true;
        };

If the above case happens, neither the callback nor the errback passed to open will be called.

I've not experienced this, but from just looking at the code it looks like it might be a bug. (If it's not, maybe a comment could explain why the callback/errback is not called.) What does update affect?