google / protorpc Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
This results in errors like the following when using the App Engine "endpoints" library:
ERROR 2018-05-21 14:00:19,811 wsgi.py:263]
Traceback (most recent call last):
File "/root/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/root/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/root/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/opt/root/src/appengine-cloudsql-benchmark/main.py", line 27, in <module>
import endpoints
File "/opt/root/src/appengine-cloudsql-benchmark/lib/endpoints/__init__.py", line 27, in <module>
from .apiserving import *
File "/opt/root/src/appengine-cloudsql-benchmark/lib/endpoints/apiserving.py", line 81, in <module>
from protorpc.wsgi import service as wsgi_service
ImportError: No module named wsgi
The solution is to do make the following change to setup.py
:
diff --git a/setup.py b/setup.py
index 2c83901..0ff14c0 100644
--- a/setup.py
+++ b/setup.py
@@ -18,7 +18,7 @@
import platform
-from setuptools import setup
+from setuptools import find_packages, setup
# Configure the required packages and scripts to install, depending on
# Python version and OS.
@@ -34,9 +34,6 @@ if py_version < '2.6':
REQUIRED_PACKAGES.append('simplejson')
_PROTORPC_VERSION = '0.12.0'
-packages = [
- 'protorpc',
-]
setup(
name='protorpc',
@@ -46,7 +43,7 @@ setup(
author='Google Inc.',
author_email='[email protected]',
# Contained modules and scripts.
- packages=packages,
+ packages=find_packages(),
entry_points={
'console_scripts': CONSOLE_SCRIPTS,
},
I'm setting up a project using Google's Bazel as my build system. Many Google projects already support Bazel. It would be helpful if this repository (protorpc) exposed its tools in a way that can easily be consumed by other Bazel targets.
More context: I'd like a build rule that takes the .proto files as srcs and produces the resulting generated Python Message classes, so that I can use those in Google Cloud Datastore (the generated proto code produced by py_proto_library does not seem usable in Datastore models, since it does not subclass the Message class.)
Source of googleapis/google-cloud-python#1133
File "../gcloud-python/gcloud/storage/blob.py", line 28, in <module>
from apitools.base.py import http_wrapper
File "../gcloud-python/.tox/py27/lib/python2.7/site-packages/apitools/base/py/__init__.py", line 10, in <module>
from apitools.base.py.extra_types import *
File "../gcloud-python/.tox/py27/lib/python2.7/site-packages/apitools/base/py/extra_types.py", line 38, in <module>
class DateField(messages.Field):
File "../gcloud-python/.tox/py27/lib/python2.7/site-packages/apitools/base/py/extra_types.py", line 50, in DateField
class __metaclass__(messages.Field.__metaclass__):
AttributeError: type object 'Field' has no attribute '__metaclass__'
Hi, i'm new in this. I tried to run the project but give me an error in
google.net.proto import ProtocolBuffer
note: win 8 with python 2.7.7
Starting from Python 3.7 async
is a reserved keyword and cannot be used, as it causes SyntaxError
:
lib/python3.7/site-packages/protorpc/remote.py", line 581
return async_method(self.async, *args, **kwargs).response
^
SyntaxError: invalid syntax
Does protorpc convert .proto files into Message instances that can be stored in Cloud Datastore? I don't understand why the Datastore client doesn't accept the standard Python codegen produced by the protoc compiler.
According to this StackOverflow answer, there is no current facility for storing arbitrary protocol buffers in Datastore, but it looks to me that this library may be capable of doing that.
There's much confusion as to ProtoRPC and webapp2 support. Can you clarify if this does indeed support webapp2?
webapp2 framework used to provide support for ProtoRPC but that code was removed - supposedly it only worked with Python 2.5 and the origional webapp framework. What I need to do is host my remote.Service methods from an existing webapp2 application.
Context: gsutil uses apitools to create messages and send them in HTTP requests to our JSON API. In the case of PATCH requests used to change object metadata (see [1]), one must set a field's value to null to clear it (as mentioned in the patch semantics at [2]). However, this isn't possible to do for predefined message fields. For example, an Object
message, which describes a GCS object's metadata, has the field cacheControl
:
class Object(_messages.Message):
...
cacheControl = _messages.StringField(3)
...
but if we attempt to set that field to None
, it's treated as if the field was unset:
(Pdb) obj_metadata = apitools_messages.Object()
(Pdb) obj_metadata
<Object
acl: []>
(Pdb) obj_metadata.cacheControl = 'public'
(Pdb) obj_metadata
<Object
acl: []
cacheControl: 'public'>
(Pdb) obj_metadata.cacheControl = None
(Pdb) obj_metadata
<Object
acl: []>
... and it won't be included in the JSONified representation of the message that's sent as part of the HTTP request to our API. In order to clear these fields' values via PATCH requests, we need a way to set them to some value such that they'll be represented as null when translated to JSON.
Note: This isn't the first time we've run into this limitation, and others have run into it as well, e.g. GoogleCloudPlatform/endpoints-proto-datastore#65 (comment).
[1] https://cloud.google.com/storage/docs/json_api/v1/objects/patch
[2] https://cloud.google.com/storage/docs/json_api/v1/how-tos/performance#patch
from protorpc import messages
class LogMessage(messages.Message):
is_error = messages.BooleanField(1)
# message value is silently ignored if it's a bytestring
# make sure it is a unicode string!
message = messages.StringField(2, required=True)
>>> LogMessage(is_error=False, message='fnord')
<LogMessage
is_error: False>
>>> LogMessage(is_error=False, message=u'fnord')
<LogMessage
is_error: False
message: u'fnord'>
>>> LogMessage(is_error=False, message='fnord').check_initialized()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File ".tox/py27/lib/python2.7/site-packages/protorpc/messages.py", line 802, in check_initialized
(type(self).__name__, name))
protorpc.messages.ValidationError: Message LogMessage is missing required field message
>>> LogMessage(is_error=False, message=u'fnord').check_initialized()
>>> LogMessage(is_error=False, message='fnord').is_initialized()
False
>>> LogMessage(is_error=False, message=u'fnord').is_initialized()
True
I have no problem with StringFields being required to be unicode strings. I do have a problem with non-unicode strings being silently ignored.
This seems to be a regression since 0.11.1; I do not experience this problem on 0.11.1 with #25 applied manually; I do experience it on master with #25 applied. (Needless to say, one cannot use this library without #25 applied anymore.)
The Chromium project (www.chromium.org) pulls in protorpc indirectly through the catapult (https://github.com/catapult-project/catapult/) and gsutil (https://github.com/GoogleCloudPlatform/gsutil) source repositories. In order for Chromium to be pulled into various Linux source distributions there's a requirement that all of the third party files pass the Linux licensecheck utility. Currently there are many files in the protorpc repository missing per-file licenses. From a current run of licensecheck, they are:
$ licensecheck -r . | grep "No copyright"
./protorpc/google_imports.py: *No copyright* UNKNOWN
./protorpc/protorpc_test_pb2.py: *No copyright* GENERATED FILE
./protorpc/non_sdk_imports.py: *No copyright* UNKNOWN
./protorpc/webapp/google_imports.py: *No copyright* UNKNOWN
./ez_setup.py: *No copyright* UNKNOWN
./experimental/javascript/build.sh: *No copyright* UNKNOWN
./demos/quotas/backend/quotas/__init__.py: *No copyright* UNKNOWN
./demos/experimental/wsgi/services.py: *No copyright* UNKNOWN
./demos/tunes_db/server/appengine_config.py: *No copyright* UNKNOWN
We'd like to ask that per-file licenses be added to these files to make it easier to integrate protorpc not only into Chromium, but also Linux distributions in general. Thanks.
I'm using Google NDB, to do a post put hook to generate a search Index. However, I am receiving the error below when it attempts to generate the document. I'm not sure why I am getting this as all my types. All the types are already in Unicode when checking the type. Unsure if this should be here or elsewhere...
GAE Standard, Py27
My Model:
class DirectionModel(ndb.Model):
created_at = ndb.DateTimeProperty(auto_now_add=True)
deviceId = ndb.StringProperty()
deviceName = ndb.StringProperty()
updated_at = ndb.DateTimeProperty(auto_now=True)
settings = ndb.StringProperty(indexed=False)
keywords = ndb.StringProperty(repeated=True)
testId = ndb.StringProperty()
def _post_put_hook(self, future):
doc = search.Document(doc_id=self.key.id(), fields=[
search.TextField(name='deviceId', value=self.deviceId),
search.TextField(name='deviceName', value=self.deviceName),
search.TextField(name='settings', value=self.settings),
search.TextField(name='testId', value=self.testId)
])
search.Index('cookingDirections').put(doc)
Encountered unexpected error from ProtoRPC method implementation: TypeError (coercing to Unicode: need string or buffer, long found) (/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/third_party/protorpc-1.0/protorpc/wsgi/service.py:191)
Traceback (most recent call last):
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/third_party/protorpc-1.0/protorpc/wsgi/service.py", line 181, in protorpc_service_app
response = method(instance, request)
File "/base/data/home/apps/s~hyperwave-168417/cookingdirections:20180309t143359.408189414954364822/lib/endpoints/api_config.py", line 1272, in invoke_remote
return remote_method(service_instance, request)
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/third_party/protorpc-1.0/protorpc/remote.py", line 414, in invoke_remote_method
response = method(service_instance, request)
File "/base/data/home/apps/s~hyperwave-168417/cookingdirections:20180309t143359.408189414954364822/main.py", line 135, in create
keywords=keywords.split(',')).put()
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3458, in _put
return self._put_async(**ctx_options).get_result()
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
self.check_success()
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 378, in check_success
self.wait()
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 362, in wait
if not ev.run1():
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/eventloop.py", line 268, in run1
delay = self.run0()
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/eventloop.py", line 230, in run0
callback(*args, **kwds)
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 533, in _on_future_completion
self._help_tasklet_along(ns, ds_conn, gen, val)
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 444, in _help_tasklet_along
self.set_result(result)
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 327, in set_result
callback(*args, **kwds)
File "/base/data/home/apps/s~hyperwave-168417/cookingdirections:20180309t143359.408189414954364822/main.py", line 59, in _post_put_hook
search.TextField(name='testId', value=unicode(self.testId))
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/api/search/search.py", line 1926, in __init__
doc_id = _ConvertToUnicode(doc_id)
File "/base/alloc/tmpfs/dynamic_runtimes/python27/a7637d5531ec9deb_unzipped/python27_lib/versions/1/google/appengine/api/search/search.py", line 247, in _ConvertToUnicode
return unicode(some_string, 'utf-8')
TypeError: coercing to Unicode: need string or buffer, long found
This is the same error seen in apitools here: google/apitools#175
Hello,
I am using appengine and I noticed that some of my requests crashed without outputing any logs.
After investigating I found some logs that were not forwarded to the output :
File "/home/vmagent/python_vm_runtime/lib/protorpc-1.0/protorpc/wsgi/service.py", line 191, in protorpc_service_app
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 3604468: ordinal not in range(128)
It's happening here : https://github.com/google/protorpc/blob/master/protorpc/wsgi/service.py#L190
When the error raised is a unicode
instead of an str
it's crashing. You can reproduce the issue easily with this piece of code :
In [1]: try:
....: raise ValueError(u'sample unicode : \xa0')
....: except Exception as err:
....: logging.exception('%s %s' % ('text', err))
....:
---------------------------------------------------------------------------
UnicodeEncodeError Traceback (most recent call last)
<ipython-input-22-6518b912bd3d> in <module>()
2 raise ValueError(u'sample unicode : \xa0')
3 except Exception as err:
----> 4 logging.exception('%s %s' % ('text', err))
5
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 17: ordinal not in range(128)
In order to fix this you just have to add a u
before the string in the logging statement :
In [2]: try:
raise ValueError(u'sample unicode : \xa0')
except Exception as err:
logging.exception(u'%s %s' % ('text', err))
....:
ERROR:root:text sample unicode :
Traceback (most recent call last):
File "<ipython-input-23-fded89cbdcf2>", line 2, in <module>
raise ValueError(u'sample unicode : \xa0')
ValueError: sample unicode : \xa0
Is that something that could be fixed ? Would you accept a PR with this fix ?
In protojson.decode_message
, if encoded_message
is an empty string, the validation will be skipped (and no ValidationError will be raised even if the message has some fields marked as required).
In https://github.com/google/protorpc/blob/master/protorpc/protojson.py#L201 , the following
if not encoded_message.strip():
return message_type()
should be changed to
if not encoded_message.strip():
message = message_type()
message.check_initialized()
return message
I pip install gsutil
on a Mac OS 10.10 laptop and receive this error. I was under the impression that enum was a python 3.4 feature. The pypi package may need updating to list this as a dependency?
Even a short description of what this is would be handy, I'm guessing RPC over protocol buffers ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.