Python import issue about docker-protoc HOT 8 OPEN

You also have another option other than the one you stated:
Change the subfolder init.py file to include:

import os
import sys
sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))

from docker-protoc.

Hi @ysyyork sorry for the late reply.
Is this still an issue for you?

from docker-protoc.

I am also observing this issue. A workaround is to call the container with an explicit --grpc-out argument:

docker run --rm \
  -v `pwd`:/project \
  -w /project \
  namely/protoc-all:1.50 \
    -f project.proto \
    -l python \
    -o . \
    --grpc-out ./proto

Now all *.py files are inside proto/ and the imports work.

What I don't like is that __init__.py files are created recursively for each directory starting at ., so this workaround definitely has drawbacks.

@ido-namely Do you think this is something we can fix inside the container?

from docker-protoc.

Hi @maxjakob ,
Thank you for your input. I'll try to take some time to better understand the issue.
I personally haven't used this project to generate protos for python.
if you can clarify in the meantime though - in your example, do project_pb2.py project_pb2_grpc.py still get generated in the same dir?
What is the import command that they are generated with?

from docker-protoc.

@ido-namely After running

$ docker run --rm \
  -v `pwd`:/project \
  -w /project \
  namely/protoc-all:1.50 \
    -f project.proto \
    -l python \
    -o ./ \
    --grpc-out ./proto

I get the following files:

$ ls proto/
project_pb2_grpc.py  project_pb2_grpc.pyi  project_pb2.py  project_pb2.pyi  __init__.py

The import paths in these files are such that the issue reported by @ysyyork goes away, the imports work with this combination of parameters.

However, you now also get __init__.py files in the root and all subdirs recursively:

$ ls .
proto/  otherdir/  __init__.py
$ ls otherdir/
__init__.py

These excess __init__.py files are a problem in our git and linting workflows. Is there some way to turn off the generation of these? Or is there a combination of arguments that both fixes the issue described by @ysyyork and prevents these files from being generated everywhere?

from docker-protoc.

For now, we are going to call protoc directly which also solves this issue:

EDIT: I spoke too soon. This does not solve the import issue.

from docker-protoc.

I think it's actually this issue: grpc/grpc#29459, though I think this is even an issue without proto files importing other proto files. It's already issue with the generated gGRPC file importing the other generated Python file.

See also grpc/grpc#9575 (comment)

from docker-protoc.

I figured out a solution now. If you have the directory structure

project/
  proto/
    project.proto

inside the project/ directory, you can run this command:

$ docker run --rm \
  -v `pwd`:/work \
  -w /work \
  --entrypoint="" \
  namely/protoc-all:1.50_0 \
    bash -c \
      "protoc --python_out=. --mypy_out=. --grpc_out=. --mypy_grpc_out=. --plugin=protoc-gen-grpc=/usr/local/bin/grpc_python_plugin ./proto/project.proto"

and add an __init__.py file yourself:

touch proto/__init__.py

to end up with these files:

project/
  proto/
    __init__.py
    project_pb2_grpc.py
    project_pb2.py
    project_pb2.pyi
    project.proto

The imports in project_pb2_grpc.py will have the form from proto import project_pb2 as ... and will therefore work as expected.

I know that the idea of entrypoint.sh is to abstract away the exact configuration for all the parameters. Maybe this gives somebody enough pointers to fix it there.

from docker-protoc.