kurokikaze / limestone Goto Github PK

If I use a persistent connection in Limestone, sometimes it just "breaks". Is there a way to automatically connect back to the Sphinx server when this happens?

limestone.connect(9312, true, function(err){
});

app.get('/index/search/:query', function(req,res){
    limestone.query... //perform queries whenever someone hits a page
})

When I use a multi attribut type in my index I have a bug :

Sending search request of 155 bytes

buffer.js:305

  return this.parent.utf8Slice(start, end);
















      ^

TypeError: Bad argument.
at Buffer.toString (buffer.js:305:26)
at Object.shiftLString as lstring
at /home/project/lib/limestone.js:651:43
at /home/project/lib/limestone.js:567:13
at Object.runCallbackIfDone (/home/project/lib/limestone.js:557:34)
at Socket.readResponseData (/home/project/lib/limestone.js:495:18)
at Socket.emit (events.js:64:17)
at Socket._onReadable (net.js:672:14)
at IOWatcher.onReadable as callback

in ls-test.js :

      response.sendHeader(500, {"Content-Type": "text/plain"});
      response.write("Request timed out\n\n");

However, no response.sendHeader() method is present in the node 0.5.6 api or node 0.4.1 api

is it an alias? do I need to change them by writeHead() ?

I've checked latest releases 2.0.5 and 2.0.4 and both doesn't work. Would be great to update README to know which version of sphinx searchd supported.

I just installed limestone and am trying out the same program in the readme file. I get the following error:

Connected, sending query
Extended search for 'test' yielded 0 results: {"status":0,"num_fields":5,"fields":[{"name":"name"},{"name":"author"},{"name":"homepage"},{"name":"summary"},{"name":"description"}],"attributes":[{"name":"redis_key","type":7}],"matches":[],"num_attrs":1,"match_count":0,"id64":0,"total":0,"total_found":0,"msecs":0,"words_count":1,"words":{"test":{"docs":0,"hits":0}}}

/Users/jimmyjohn/test/web/node_modules/limestone/limestone.js:589
var new_buffer = new Buffer(this.data.length + data.length);
^
TypeError: Cannot read property 'length' of null
at Object.response_output.append (/Users/jimmyjohn/test/web/node_modules/limestone/limestone.js:589:68)
at Socket.readResponseData (/Users/jimmyjohn/test/web/node_modules/limestone/limestone.js:561:25)
at Socket.EventEmitter.emit (events.js:92:17)
at emitReadable_ (_stream_readable.js:408:10)
at emitReadable (_stream_readable.js:404:5)
at Socket.Readable.read (_stream_readable.js:266:5)
at Socket.onSocketEnd (net.js:258:10)
at Socket.EventEmitter.emit (events.js:92:17)
at TCP.onread (net.js:552:10)
Jimmys-MacBook-Pro-2:web jimmyjohn$

I get valid results from sphinx but see this error as well. I am just learning node.js so this may be a rookie mistake.

Sphinx version: Sphinx 2.0.8-release (r3831)
Node: v0.10.13
Limestone: 0.1.4

First I want to say thank you very much for this library. It is immensely helpful and it's obvious the amount of work that went into it. Couldn't be more enthusiastic about being able to use it with our Node assets.

I have a small problem though. I see when searching for results with limestone I get a list sorted differently than with a standard bin/search operation. The results from limestone tend to be like this:

matches:
   [ { doc: 28, weight: 2, attrs: {} },
     { doc: 18, weight: 1, attrs: {} },
     { doc: 41, weight: 1, attrs: {} },
     { doc: 71, weight: 1, attrs: {} },

But when I search with bin/search I get

displaying matches:                                                                                                                      
1. document=28, weight=2777                                                                                                              
2. document=80, weight=1770                                                                                                              
3. document=100, weight=1770                                                                                                             
4. document=18, weight=1725

So not only are the weights reduced to a single digit, they are sorted differently. Is there a way I can get the standard 4-digit response weight? I tried different weighting algorithms but I don't believe that's the core issue. I believe it may be a matter of parsing the digits on their return from the server.

Can anyone shed some light? Thanks!

We are building autocompletion using nodejs, sphinx and trying out limestone.

Autocompletion containts only 6 items
Everything works perfect except one performance issue.

When searching we use limit: 6, so we need quite small amount of data to return.
Sphinx returns results in average time from 1 - 9ms

However with header Transfer-Encoding': 'chunked. it takes browser 180-200 ms to receive data. And it is huge time loss for us.
If i turn off chunked header, then browser recieved data in 10-16ms, which id perfect and desired result, but the response is not complete if headers ar not specified as chunked.

If i take full response data as string from chunked result and send it directly as body, then it also takes 10-16ms. So the problem is definately in that chunked data output or buffering.

Is there any way to output straight and not chunked data for small amounts of items? Otherwise these 180-200ms are huge performance loss for us.

Thanks

Just random sample text

I got this error:

npm ERR! couldn't unpack /tmp/npm-1306773374076/1306773374076-0.4486181079410016
5/tmp.tgz to /tmp/npm-1306773374076/1306773374076-0.44861810794100165/contents
npm ERR! Failed to parse json
npm ERR! Unexpected token }
npm ERR! File: /tmp/npm-1306773374076/1306773374076-0.44861810794100165/contents
/package/package.json
npm ERR! JSON.parse Failed to parse package.json data.
npm ERR! JSON.parse package.json must be actual JSON, not just JavaScript.
npm ERR! JSON.parse
npm ERR! JSON.parse This is not a bug in npm.
npm ERR! JSON.parse Tell the package author to fix their package.json file.
npm ERR!
npm ERR! System Linux 2.6.38.2-grsec-xxxx-grs-ipv6-64
npm ERR! command "node" "/usr/local/bin/npm" "install" "limestone"
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /var/nodejs/npm-debug.log
npm not ok

tcp stream is created and connects, but the data event is never fired. :/

Node 0.10 has updated the streams objects, and should fall back to old behaviour (without loss of data) when a data event is added. But it still doesn't work. :/

1. First there is an error in limestone.js line 296

request.push_lstring(filter.attr);

i think it sould be

request.push.lstring(filter.attr);

2. Even i fix the problem there is no way i can specify the simplest filters, like filter by type_id=3

If i try:

var filter = { 'type': Sphinx.filterTypes.VALUES, 'attr': 'type_id', values: [ 3 ] };
query_config['filters'] = [ filter ];

i receive "Search error"

Any ideas on correct syntax to specify filters?
Maybe it is related to #22, however even i try to fix iterating problems, i still got "Search error"

Thanks

Am i missing something or there are some iterating errors.
I see in some places in limestone.js following JS syntax:

for (var filter in query.filters) {
   request.push.int32(filter.attr.length);
...

I think it should be at least like this:

for (var filter in query.filters) {
   filter = query.filters[filter];
   request.push.int32(filter.attr.length);
...

Since when iterating using for(var X in arr) returns X directly as object and not key?

Probably same is in this line 301:

for (var value in filter.values) { 
   request.push.int64(0, value)

I noticed you have Sphinx object with all the constants defined in SphinxClient class.
Why not make it accessible from outside?
So we can use it while configuring, becaouse who can remember what these 0 and 1 means :)

This is how i did it in limestone.js:

self.Sphinx = Sphinx;

And while configuring my query now i can use them:

var Sphinx = limestone.Sphinx;
var query_config = 
{
    'mode':  Sphinx.searchMode.ALL,
    'ranker': Sphinx.rankingMode.MATCHANY,
...

I have just updated to 0.1.4 using npm, using Node 0.10.13 and having the issue with both Sphinx 2.0.4 and now 2.0.8. I had been happily using 0.1.3 with Node 0.8.15 and Sphinx 2.0.4.

On performing a query I get the correct result back, however I then get a null response back shortly after.

This is my code snippet using async

async.waterfall([
    function(callback) { // connect to sphinx
        limestone.connect(config.sphinx.host + ":" + config.sphinx.port, callback);
    },
    function(callback) { // perform query
        limestone.query({
            query: params.keywords
          , maxmatches: params.limit
          , indexes: 'test1'
          , mode: 4 // Extended
        }, callback);
    },
    function(answer, callback) { // disconnect and convert results for output
        limestone.disconnect();
        util.log("Extended search for '" + params.keywords + "' yielded " +
        (answer ? answer.match_count: 0) + " results: ");
        if (!answer || !answer.match_count) return callback(null, []);

        var personIds = [];
        util.log('answer length:' + answer.matches.length);
        for (i = 0; i < answer.matches.length; i++) {
            personIds.push(answer.matches[i].doc);
        }
        util.log('personIds:' + personIds);
        orm.model('Person').getMappedPeople({
            personIds: personIds
          , canInvite: params.canInvite
          , order: 'lastName, firstName '
        }, callback);
    }
], next);

I also added a logging line on limestone.js at line 560
console.log('data:' + data);
This is log output

data:[0,0,1,25,0,0,0,221,0,0,0,0,0,0,0,3,0,0,0,4,110,97,109,101,0,0,0,9,102,105,114,115,116,110,97,109,101,0,0,0,8,108,97,115,116,110,97,109,101,0,0,0,4,0,0,0,5,105,115,112,114,111,0,0,0,1,0,0,0,16,103,108,111,98,97,108,116,114,97,99,107,99,111,117,110,116,0,0,0,1,0,0,0,13,102,111,108,108,111,119,101,114,99,111,117,110,116,0,0,0,1,0,0,0,14,102,111,108,108,111,119,105,110,103,99,111,117,110,116,0,0,0,1,0,0,0,2,0,0,0,1,0,0,0,0,0,0,0,38,0,0,6,155,0,0,0,0,0,0,0,1,0,0,0,36,0,0,0,66,0,0,0,0,0,0,0,65,0,0,6,155,0,0,0,0,0,0,0,2,0,0,0,21,0,0,0,25,0,0,0,2,0,0,0,2,0,0,0,0,0,0,0,1,0,0,0,4,97,108,101,120,0,0,0,2,0,0,0,2]
11 Jul 05:27:41 - Extended search for 'alex' yielded 2 results:
11 Jul 05:27:41 - answer length:2
11 Jul 05:27:41 - personIds:38,65
data:null

At the end you can see the data:null line which then crashes Node with the following error

home/ubuntu/releases/20130711011458/node_modules/limestone/limestone.js:591
var new_buffer = new Buffer(this.data.length + data.length);
^
TypeError: Cannot read property 'length' of null
at Object.response_output.append (/home/ubuntu/releases/20130711011458/node_modules/limestone/limestone.js:591$
at Socket.readResponseData (/home/ubuntu/releases/20130711011458/node_modules/limestone/limestone.js:563:26)
at Socket.EventEmitter.emit (events.js:92:17)
at emitReadable_ (_stream_readable.js:408:10)
at emitReadable (_stream_readable.js:404:5)
at Socket.Readable.read (_stream_readable.js:266:5)
at Socket.onSocketEnd (net.js:258:10)
at Socket.EventEmitter.emit (events.js:92:17)
at TCP.onread (net.js:552:10)

I have patched this locally by returning from the readResponseData() function if data is found to be null. Not completely sure what the impact of this is though.

Hi,
Awesome module. I'm currently converting a project from PHP to node.js. In PHP, I can do this $results = $client->Query("",'index'); but this throws an error in limestone, i.e.

limestone.query{'query': "" }j, function(err, response){ -------

any ideas?

This looks handy (I use Sphinx with PHP all the time). You should submit it to npm!

Test issue for closing and reopening

Just random sample text

Does buildExcerpts function working??

On line 479 is a console.log('Sending build excerpt request of ' + request_buf.length + 'bytes');

Please remove...

Thanks for an otherwise excelent lib...

J*

Example ( use persist connection ) from README.markdown not work for last sphinx with default configure. After start example script freeze and it is all ...