100% cpu usage about goworker HOT 17 OPEN

@etopian I was having the same problem and switched to the following instead of Interval after digging through the code which seems to solve the issue as least for me.

    settings := goworker.WorkerSettings{
        URI:            "redis://localhost:6379/",
        Connections:    2,
        Queues:         []string{"hello"},
        UseNumber:      true,
        ExitOnComplete: false,
        Concurrency:    25,
        Namespace:      "resque:",
        IntervalFloat:  5.0,
    }

from goworker.

When I monitor redis I see this:

1471764750.337179 [0 172.17.0.1:43928] "LPOP" "resque:queue:queues"
1471764750.337349 [0 172.17.0.1:43930] "LPOP" "resque:queue:myqueue"
1471764750.337481 [0 172.17.0.1:43930] "LPOP" "resque:queue:delimited"

printed continuously and rapidly. should it not be doing this? should not setting Interval to something higher slow this process down. I have tried setting it to 5000.0 with no result.

from goworker.

Setting the commandline arg to -interval works but it does not seem to be reading the settings.

i.e.

settings := goworker.WorkerSettings{
    URI:            "redis://172.17.0.1:6379/",
    Connections:    1,
    Queues:         []string{"hello"},
    UseNumber:      true,
    ExitOnComplete: false,
    Concurrency:    1,
    Namespace:      "resque:",
    Interval:       1.0,
}    

goworker.SetSettings(settings)

from goworker.

Ahh, I think there's an issue with your Interval value - it's a Time.Duration, so 1.0 is actually one nanosecond. Try Interval: time.Second and see if that fixes it. I'll see if there's a way to make this less confusing.

from goworker.

Well I would expect that the command line -interval flag should work the same work as the way that WokerSettings do. so setting -interval 5.0 should be the same as setting WorkerSettings Interval to 5.0. However you decide to make this less confusing I would definitely say document it on the readme because the settings defined as an example, 5.0 nano seconds, don't really make much sense.

from goworker.

Anyhow thank you very much for writing this. It's very useful.

from goworker.

Actually even setting that to 10000000000, which is 10 seconds in nano seconds does not seem to work. So likely the problem is not just the time representation.

from goworker.

Hey @etopian , I've tried to recreate this, but not able too, Could you provide a repo where this happens, and I can try to re-build your steps?

from goworker.

https://github.com/etopian/goworker-example

built using go version go1.6.2 linux/amd64

from goworker.

@rjrobinson posted above.

from goworker.

Awesome. And the exact command line argument too please. Just so I'm covering all my bases.

from goworker.

./worker -interval 5 works fine, as does ./worker -interval 1... currently the settings interval is set to 1 which is suppose to be 1 nano second, but turning it up in the settings does not help either from my experience.

from goworker.

@jasonwells much thx

from goworker.

@jasonwells Thank you!

from goworker.

I believe that I have found the root cause to this problem. It appears that the IntervalFloat value will always override the Interval value (and by default it is 5.0). It is caused by the flags() function:

func flags() error {
    ...
    if err := workerSettings.Interval.SetFloat(workerSettings.IntervalFloat); err != nil {
        return err
    }
    ...
}

This is a good example for why issue #19 is important. Using flags to configure a library that can be used as a dependency is a bad idea.

from goworker.

@wuman Go team! Awesome!

from goworker.

@wuman: @FrankChung contributed #46 - do you think that will fix this issue? If so, would you mind merging it in?

from goworker.