cocaine / cocaine-framework-python Goto Github PK

Where is the cocaine.decorators.wsgi and cocaine.decorator.http ?

Hello,
Could you please advise a proper way to use cocaine-framework-python in Windows environment.
I would like to connect to mastermind from windows but results to this traceback in cocaine:

import mastermind
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\mastermind__init__.py", line 1, in
from mastermind.client import MastermindClient
File "C:\Python27\lib\mastermind\client.py", line 4, in
from mastermind.service import ReconnectableService
File "C:\Python27\lib\mastermind\service.py", line 6, in
from cocaine.logging import Logger
File "C:\Python27\lib\cocaine\logging__init__.py", line 1, in
from cocaine.logging.logger import Logger
File "C:\Python27\lib\cocaine\logging\logger.py", line 25, in
from cocaine.asio.service import Service
File "C:\Python27\lib\cocaine\asio\service.py", line 11, in
from cocaine.asio.pipe import Pipe
File "C:\Python27\lib\cocaine\asio\pipe.py", line 6, in
import fcntl
ImportError: No module named fcntl

I am using cocaine-framework-python-0.11 branch.
Then I've found a dummy module
https://communities.cisco.com/servlet/JiveServlet/download/150596-65641/fcntl.py.zip
which returns always zero for that function:

def fcntl(fd, op, arg=0):
return 0

and inserted it to python's Lib folder.
But it isn't solved the problem, because apparently the fcntl is module that can only be used in linux os. What should I do?

Hello!

I have such an app:

#!/usr/bin/env python
from cocaine.worker import Worker
from cocaine.logging import Logger

log = Logger()

def raiser(request, response):
    msg = yield request.read()
    log.debug("Recieved:\n{0}".format(msg))
    a = 1 / 0
    response.close()

W = Worker()
W.run({
    'raise': raiser
})

It should always raise ZeroDivisionError.

I have deployed it and ran:

cocaine-tool call raiser enqueue "'raise', '123123'"
error in service "raiser" - Error in event 'raise' handler integer division or modulo by zero [1]

The cocaine log says:

[Thu Nov 21 08:20:21 2013] [DEBUG] service/locator: accepted a new client from '[::ffff:127.0.0.1]:38112' on fd 25
[Thu Nov 21 08:20:21 2013] [DEBUG] service/locator: processing type 0 message using slot 'resolve'
[Thu Nov 21 08:20:21 2013] [DEBUG] service/locator: providing 'raiser' using local node
[Thu Nov 21 08:20:21 2013] [DEBUG] service/raiser: accepted a new client from '[::ffff:127.0.0.1]:47297' on fd 26
[Thu Nov 21 08:20:21 2013] [DEBUG] service/raiser: processing type 0 message using slot 'enqueue'
[Thu Nov 21 08:20:21 2013] [DEBUG] app/raiser: slave 3e293866-755c-45e5-975a-080118fcbda5 has started processing session 2
[Thu Nov 21 08:20:21 2013] [DEBUG] service/logging: processing type 0 message using slot 'emit'
[Thu Nov 21 08:20:21 2013] [DEBUG] app/raiser: Recieved:
123123
[Thu Nov 21 08:20:21 2013] [DEBUG] service/logging: processing type 0 message using slot 'emit'
[Thu Nov 21 08:20:21 2013] [ERROR] app/raiser: ZeroDivisionError('integer division or modulo by zero',)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/cocaine/decorators/_callablewrappers.py", line 64, in on_error
    res.get()
  File "/usr/local/lib/python2.7/dist-packages/cocaine/futures/chain.py", line 50, in get
    return self._returnOrRaise(self.result)
  File "/usr/local/lib/python2.7/dist-packages/cocaine/futures/chain.py", line 54, in _returnOrRaise
    raise result
ZeroDivisionError: integer division or modulo by zero
[Thu Nov 21 08:20:21 2013] [DEBUG] app/raiser: slave 3e293866-755c-45e5-975a-080118fcbda5 received type 5 message in session 2
[Thu Nov 21 08:20:21 2013] [DEBUG] app/raiser: slave 3e293866-755c-45e5-975a-080118fcbda5 received session 2 error, code: 1, reason: Error in event 'raise' handler integer division or modulo by zero
[Thu Nov 21 08:20:21 2013] [DEBUG] app/raiser: slave 3e293866-755c-45e5-975a-080118fcbda5 received type 6 message in session 2
[Thu Nov 21 08:20:21 2013] [DEBUG] service/raiser: client on fd 26 has disconnected
[Thu Nov 21 08:20:21 2013] [DEBUG] app/raiser: slave 3e293866-755c-45e5-975a-080118fcbda5 has completed session 2
[Thu Nov 21 08:20:21 2013] [DEBUG] service/locator: client on fd 25 has disconnected

Ok, how can I see the traceback from my real app?

Hi!

Cocaine logger can't log Russian utf-8 letters.

from cocaine.logging import Logger
log = Logger()

...
    log.debug(u"Ололо")
...

Here is log entry:

app/bio_auth: UnicodeEncodeError('ascii', u'Traceback (most recent call last):\n   File "/var/spool/cocaine/bio_auth/bio_auth_worker.py", line 40, in authenticate\n    log.debug(u"\u041e\u043b\u043e\u043b\u043e")\n   File "/usr/local/lib/python2.7/dist-packages/cocaine/logging/logger.py", line 66, in func\n    cls._logger._writableStream.write(Message("Message", cls._counter, _lvl,  cls.target, str(data)).pack())\n UnicodeEncodeError: \'ascii\' codec can\'t encode characters in position 0-4: ordinal not in range(128)\n', 134, 139, 'ordinal not in range(128)')
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/cocaine/decorators/_callablewrappers.py", line 64, in on_error
    res.get()
  File "/usr/local/lib/python2.7/dist-packages/cocaine/futures/chain.py", line 50, in get
    return self._returnOrRaise(self.result)
  File "/usr/local/lib/python2.7/dist-packages/cocaine/futures/chain.py", line 54, in _returnOrRaise
    raise result
UnicodeEncodeError: 'ascii' codec can't encode characters in position 134-138: ordinal not in range(128)

i successfully deployed app from example https://github.com/cocaine/cocaine-framework-python/wiki/Echo-application-example , when checking status by cocaine-tool app check -n Echo - i get running state. But i am unable to invoke it from python code from example ( it just does nothing when i execute
chunk = service.enqueue('doIt', 'SomeMessage').get().
I also tried executing this by
cocaine-tool call Echo doIt 'Test message'
but i get
"error in service Echo - method "doIt" is not found [1]"
What am i doing wrong?

Le code:

import asyncio
import logging
import sys

from cocaine.services import Service

log = logging.getLogger('cocaine.service')
formatter = logging.Formatter('[%(levelname)-6s] [%(service)-8s] [%(id)d]: %(message)s')
handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(formatter)
log.addHandler(handler)
log.setLevel(logging.DEBUG)
log.propagate = False


@asyncio.coroutine
def node_list():
    l = yield node.list()
    l = yield l.get()
    print(l)


node = Service("node", version=0)
loop = asyncio.get_event_loop()
loop.run_until_complete(node_list())
loop.close()

Log output:

[DEBUG ] [node    ] [4547481744]: invoking generic method: 'list'
[DEBUG ] [node    ] [4547481744]: invoking 'list' method with args: ()
[DEBUG ] [node    ] [4547481744]: None None None
[DEBUG ] [node    ] [4547481744]: checking if service connected
[INFO  ] [node    ] [4547481744]: connecting ...
[DEBUG ] [locator ] [4547482128]: invoking generic method: 'resolve'
[DEBUG ] [locator ] [4547482128]: invoking 'resolve' method with args: ('node',)
[DEBUG ] [locator ] [4547482128]: checking if service connected
[DEBUG ] [locator ] [4547482128]: connecting ...
[DEBUG ] [locator ] [4547482128]: successfully connected: ['localhost', 10053]
[DEBUG ] [locator ] [4547482128]: sending message: [0, 0, ('node',)]
[DEBUG ] [locator ] [4547482128]: received message: Chunk(4, 0, ('\x93\x92\xbf95.108.174.57-red.dhcp.yndx.net\xcd\xd4\xe8\x01\x83\x00\xa9start_app\x01\xa9pause_app\x02\xa4list',))
[DEBUG ] [locator ] [4547482128]: received message: Choke(6, 0, ())
[INFO  ] [node    ] [4547481744]: successfully connected: ['95.108.174.57-red.dhcp.yndx.net', 54504]
[DEBUG ] [node    ] [4547481744]: checking if service connected
[DEBUG ] [node    ] [4547481744]: connecting ...
[DEBUG ] [node    ] [4547481744]: successfully connected: ['95.108.174.57-red.dhcp.yndx.net', 54504]
[DEBUG ] [node    ] [4547481744]: sending message: [2, 0, ()]
[DEBUG ] [node    ] [4547481744]: received message: Chunk(4, 0, ('\x91\xa4echo',))
[DEBUG ] [node    ] [4547481744]: received message: Choke(6, 0, ())
['echo']

Seems like service is reconnecting on every method call. :(

This made unit testing for modules, which uses dealer binding, impossible. please do a workaround

Hi! I've got several questions about using msgpack with cocaine...

1. First of all - why do you use msgpack dumping while writing Worker's response in all the examples?
   https://github.com/cocaine/cocaine-framework-python/blob/master/examples/chunker/chunker.py
   Data is automatically msgpack'ed in the write() method (https://github.com/cocaine/cocaine-framework-python/blob/master/cocaine/asio/stream.py#L169) and extracted in _on_message() method (https://github.com/cocaine/cocaine-framework-python/blob/master/cocaine/asio/service.py#L204)
   So you make double-msgpacked responses o__O
   This leads to manual double extracting data in Native framework, as I understand...
2. Why not to use msgpack-2.0 support? (https://github.com/msgpack/msgpack-python#note-for-msgpack-20-support)
   If I use just your internal msgpacking and send response response.write(u"Привет, Мир!")
   I'll get str-type response on the client side! Of course, decoding-encoding solves the problem, but if I send large dicts as response - it will be crazy to decode all the strings it contains.
   The sollution is simple - if don't wanna use use_bin_type=True, you can just add response_encoding param to Service constructor. If you use msgpack.unpackb(data, encoding="utf8") without using bin type - you'll get unicode-type for all decoded strings.
3. Why Service.enqueue method doesn't have auto-msgpack'ing like response.write()??

It leads to 404 page

@http 
def f(request, response):
    with response:
        pass

raises exit

Hi!

It would be much better if you rename warn() method to warning() - for compatibility with standard Python loggers.

Thanks!

There are strange dependance in package for Debian (wheezy) - "python-msgpack-python". In version for Ubuntu everything ok.

I am trying to reproduce https://github.com/cocaine/cocaine-framework-python/wiki/Echo-application-example. I have tried this on 10.6 and 11 versions of cocaine-core, but still have the same problem. Seems to me like my version of python does not support bytearray for this operations.

cocaine-tool app start --name Echo --profile=EchoProfile
WARNING:cocaine.asio.service:"_on_message" method has caught an error - unhashable type: 'bytearray'
ERROR:tornado.application:Exception in I/O handler for fd 7
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado-3.1.1-py2.7.egg/tornado/ioloop.py", line 672, in start
self._handlers[fd](fd, events)
File "/usr/local/lib/python2.7/dist-packages/tornado-3.1.1-py2.7.egg/tornado/stack_context.py", line 331, in wrapped
raise_exc_info(exc)
File "/usr/local/lib/python2.7/dist-packages/tornado-3.1.1-py2.7.egg/tornado/stack_context.py", line 302, in wrapped
ret = fn(_args, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/cocaine-0.11.1.0-py2.7.egg/cocaine/asio/ev.py", line 117, in proxy
self._callbacks(fd, self.READ)
File "/usr/local/lib/python2.7/dist-packages/cocaine-0.11.1.0-py2.7.egg/cocaine/asio/stream.py", line 120, in _on_event
self._callback(self._buffer)
File "/usr/local/lib/python2.7/dist-packages/cocaine-0.11.1.0-py2.7.egg/cocaine/asio/service.py", line 185, in dispatch
on_event(chunk)
File "/usr/local/lib/python2.7/dist-packages/cocaine-0.11.1.0-py2.7.egg/cocaine/asio/service.py", line 214, in _on_message
raise err
TypeError: unhashable type: 'bytearray'
Timeout

$ export PYTHONPATH=/opt/elliptics/build/lib/python2.7/site-packages/
$ python setup.py install --prefix=/opt/elliptics/build/
... skip buid step
Find version 0.10.4-6 in debian/changelog
running install
running bdist_egg
running egg_info
writing requirements to cocaine.egg-info/requires.txt
writing cocaine.egg-info/PKG-INFO
writing top-level names to cocaine.egg-info/top_level.txt
writing dependency_links to cocaine.egg-info/dependency_links.txt
reading manifest file 'cocaine.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'cocaine.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
installing package data to build/bdist.linux-x86_64/egg
running install_data
copying scripts/cocaine-tool -> /usr/bin/
error: /usr/bin/cocaine-tool: Permission denied

This can be worked around by using --without-tools command line option, but this is not a real fix - cocaine-tool is not copied into installation directory and cocaine can not be configured

When initializing Logger or any other Service object inside service, e.g.

log = Logger()
storage = Service("storage")

Service crashes with following error:

Logger init error: 10053:10053 - service resolving failed. Reason: [0.0.39.69:10053 - [Errno 22] Invalid argument]. Use stderr logger
Traceback (most recent call last):
File "/var/cocaine/spool/Flood/service.py", line 5, in
from flood import *
File "/var/cocaine/spool/Flood/flood/init.py", line 2, in
from test import TestService
File "/var/cocaine/spool/Flood/flood/test.py", line 11, in
storage = Service("storage")
File "/usr/local/lib/python2.7/site-packages/cocaine-0.10.6.11-py2.7.egg/cocaine/asio/service.py", line 378, in init
self.connect(host, port, blocking=True)
File "/usr/local/lib/python2.7/site-packages/cocaine-0.10.6.11-py2.7.egg/cocaine/asio/service.py", line 43, in wrapper
return strategy.init(func, blocking)(_args, *_kwargs)
File "/usr/local/lib/python2.7/site-packages/cocaine-0.10.6.11-py2.7.egg/cocaine/asio/service.py", line 53, in wrapper
chunk = g.send(chunk)
File "/usr/local/lib/python2.7/site-packages/cocaine-0.10.6.11-py2.7.egg/cocaine/asio/service.py", line 399, in connect
yield locator.connect(host, port, timeout, blocking=blocking)
File "/usr/local/lib/python2.7/site-packages/cocaine-0.10.6.11-py2.7.egg/cocaine/asio/service.py", line 300, in connect
return self._connectToEndpoint(host, port, timeout, blocking=blocking)
File "/usr/local/lib/python2.7/site-packages/cocaine-0.10.6.11-py2.7.egg/cocaine/asio/service.py", line 43, in wrapper
return strategy.init(func, blocking)(_args, *_kwargs)
File "/usr/local/lib/python2.7/site-packages/cocaine-0.10.6.11-py2.7.egg/cocaine/asio/service.py", line 53, in wrapper
chunk = g.send(chunk)
File "/usr/local/lib/python2.7/site-packages/cocaine-0.10.6.11-py2.7.egg/cocaine/asio/service.py", line 187, in _connectToEndpoint
raise ConnectionError((host, port), reason)
cocaine.asio.exceptions.
cocaine.asio.exceptions.ConnectionError: 10053:10053 - service resolving failed. Reason: [0.0.39.69:10053 - [Errno 22] Invalid argument]

It looks like problem in a 'host' parameter. If you explicitly provide locator host and port then Service object initializes properly.

storage = Service("storage", host="localhost", port=10053)

Logger still crashes because it does not have any initialization parameters. It would be nice to have them.

Hello.

Can you show me actual example of usage cocaine apps on different languages, at least python.
Example in wiki is too old and doesn't work.

AttributeError: 'Future' object has no attribute 'get'

As app, I use example — cocaine-framework-python/examples/echo/init.py

Hello,

could you please advise a proper way to implement this scenario in cocaine:

There are many clients, each should process a huge, data which are stored on some shared storage, which is available from both worker and client machines. For the processing an iterative method, that may require an user interaction, is used. Under the interaction I mean the case, when a user should proof or correct processed part of data after several iterations. Moreover it would be nice to have an ability to monitor how the processing going at some time point.

I have checked provided examples but their complexity unfortunately does not cover all related functionality.

Thanks!