adamrehn / ue4-grpc-demo Goto Github PK

Dockerised Unreal Engine 4 microservice demo using gRPC

License: MIT License

Python 20.34% Dockerfile 9.27% C# 21.79% C++ 44.10% C 0.26% Shell 4.23%

ue4-grpc-demo's Introduction

UE4 gRPC Microservice Demo

The code in this repository demonstrates running an Unreal Engine-powered microservice in the Docker containers built by the ue4-docker Python package. The client and server use Google's popular gRPC framework for communication. Note that this code is provided as an example only and is NOT designed for production use.

Prerequisites
Running the demo
Explanation
Legal

Prerequisites

Currently only Linux containers are supported (Windows container support will be added soon, although just for demonstration purposes since Linux containers are strongly recommended for running UE4 microservices.)
Docker and Docker Compose 1.19.0 or newer
ue4-docker 0.0.39 or newer

Running the demo

Ensure you have built the Docker image adamrehn/ue4-full:4.23.1 using ue4-docker version 0.0.39 or newer. Assuming you have the package installed and the host system is configured correctly, you can build the image using the following command:
```
ue4-docker build 4.23.1 --no-engine
```
Start the server in a terminal by running the ./start-server.sh shell script. Docker will build and run the images for the demo. This may take some time. You will know everything is built and running when you see a line in the log output that looks like this:
```
LogLoad: Took 0.535869 seconds to LoadMap(/Game/Default)
```
Run the interactive client in a second terminal by running the ./start-client.sh shell script. The script communicates with the server to monitor the current transform of a cube in a blank scene with a floor beneath it. You can use the controls to reset the cube to its starting point above the floor or apply a random physics force to the cube, and watch the cube's transform update in real-time.
Once you are done, hit the Q key in the second terminal to close the interactive client, and Ctrl-C in the first terminal to stop the server.

Explanation

This demo shows off an extremely basic example of a UE4-powered microservice that makes use of the Engine's physics simulation capabilities. The demo project contains a scene with a floor and a single physics-enabled cube, along with a gRPC service that allows clients to query the state of the cube and perturb it so that the results of physics acting on the cube can be observed.

The implementation is quite straightforward:

The infrastructure from conan-ue4cli is used to build a version of gRPC and its dependencies that are compatible with the Unreal Engine. These libraries are statically linked and incur no additional runtime dependencies for the demo project once it is built.
The GrpcServerWrapper Actor is placed in the scene and has its cube property set to reference the cube Actor.
The wrapper is responsible for starting the gRPC server with an instance of the GrpcServiceImp class, which contains the actual service implementation.
The methods of the service implementation are run on a background thread spawned by gRPC, so each method must communicate asynchronously with the game thread in order to safely manipulate the state of the cube.

For the sake of simplicity (and support for configurations other than just Linux + NVIDIA hardware), the demo does not perform any rendering. However, most real-world use cases for UE4 microservices involve performing rendering in addition physics simulations. Cloud rendering support can be added quite easily so long as the server is run via NVIDIA Docker under Linux. See the ue4-cloud-rendering-demo repository for an example of performing cloud rendering in Docker containers and streaming the results to a web browser via WebRTC.

Legal

ue4-grpc-demo's People

Contributors

Stargazers

Watchers

ue4-grpc-demo's Issues

grpc build fails

Hi,
I have just tried to get this example running on an ubuntu 20.04 system. I have successfully installed the ue4docker tools etc. and have the full docker image. However when I run the start-server.sh script the dockerfile fails on the build grpc-ue4 step, please see the error below.
Thanks for any help understanding this issue.

ERROR OUTPUT:

-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
CMake Warning at cmake/abseil-cpp.cmake:33 (message):
  gRPC_INSTALL will be forced to FALSE because gRPC_ABSL_PROVIDER is "module"
  and CMake version (3.10.2) is less than 3.13.
Call Stack (most recent call first):
  CMakeLists.txt:219 (include)


-- Found c-ares: /home/ue4/.conan/data/cares-ue4/1.16.1/adamrehn/4.23/package/2d5d64d24a14ce3202bc0a7ab502fb3deb6344eb/lib/cmake/c-ares/c-ares-config.cmake (found suitable version "1.16.1", minimum required is "1.13.0") 
CMake Error at cmake/protobuf.cmake:58 (find_package):
  Could not find a package configuration file provided by "Protobuf" with any
  of the following names:

    ProtobufConfig.cmake
    protobuf-config.cmake

  Add the installation prefix of "Protobuf" to CMAKE_PREFIX_PATH or set
  "Protobuf_DIR" to a directory containing one of the above files.  If
  "Protobuf" provides a separate development package or SDK, be sure it has
  been installed.
Call Stack (most recent call first):
  CMakeLists.txt:224 (include)


-- Configuring incomplete, errors occurred!
See also "/home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed/CMakeFiles/CMakeOutput.log".
See also "/home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed/CMakeFiles/CMakeError.log".
Warning: the libcxx package has been deprecated and will be removed in a future version of conan-ue4cli.
grpc-ue4/1.30.2@adamrehn/4.23: 
cares-ue4/1.16.1@adamrehn/4.23: WARN: This conanfile has no package step
protobuf-ue4/3.12.3@adamrehn/4.23: WARN: This conanfile has no package step
grpc-ue4/1.30.2@adamrehn/4.23: ERROR: Package 'bbaadf9c14f1821b000882e0282e20b2519fdbed' build failed
grpc-ue4/1.30.2@adamrehn/4.23: WARN: Build folder /home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed
ERROR: grpc-ue4/1.30.2@adamrehn/4.23: Error in build() method, line 98
        cmake.configure(source_folder="grpc", args=self.cmake_flags())
        ConanException: Error 1 while executing cd '/home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed' && cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release" -DCONAN_IN_LOCAL_CACHE="ON" -DCONAN_COMPILER="clang" -DCONAN_COMPILER_VERSION="8" -DCONAN_CXX_FLAGS="-m64" -DCONAN_SHARED_LINKER_FLAGS="-m64" -DCONAN_C_FLAGS="-m64" -DCONAN_LIBCXX="libc++" -DCMAKE_INSTALL_PREFIX="/home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/package/bbaadf9c14f1821b000882e0282e20b2519fdbed" -DCMAKE_INSTALL_BINDIR="bin" -DCMAKE_INSTALL_SBINDIR="bin" -DCMAKE_INSTALL_LIBEXECDIR="bin" -DCMAKE_INSTALL_LIBDIR="lib" -DCMAKE_INSTALL_INCLUDEDIR="include" -DCMAKE_INSTALL_OLDINCLUDEDIR="include" -DCMAKE_INSTALL_DATAROOTDIR="share" -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY="ON" -DCONAN_EXPORTED="1" -Wno-dev '-DgRPC_BUILD_CSHARP_EXT=OFF' '-DgRPC_BUILD_TESTS=OFF' '-DgRPC_MSVC_STATIC_RUNTIME=OFF' '-DgRPC_CARES_PROVIDER=package' '-DgRPC_PROTOBUF_PACKAGE_TYPE=CONFIG' '-DgRPC_PROTOBUF_PROVIDER=package' '-DgRPC_SSL_PROVIDER=package' '-DgRPC_ZLIB_PROVIDER=package' '-Dc-ares_DIR=/home/ue4/.conan/data/cares-ue4/1.16.1/adamrehn/4.23/package/2d5d64d24a14ce3202bc0a7ab502fb3deb6344eb/lib/cmake/c-ares' '-DOPENSSL_SYSTEM_LIBRARIES=' '-DOPENSSL_USE_STATIC_LIBS=ON' '-DOPENSSL_ROOT_DIR=/home/ue4/.conan/data/OpenSSL/ue4/adamrehn/4.23/package/62aff1e13bfa6a447a1ecd8198b9e13b18a63e91' '-DProtobuf_DIR=/home/ue4/.conan/data/protobuf-ue4/3.12.3/adamrehn/4.23/package/85671f6de7947f1cae40e263f8639f526bdf7cfe/cmake' '-DZLIB_INCLUDE_DIR=/home/ue4/.conan/data/zlib/ue4/adamrehn/4.23/package/62aff1e13bfa6a447a1ecd8198b9e13b18a63e91/include' '-DZLIB_LIBRARY=/home/ue4/.conan/data/zlib/ue4/adamrehn/4.23/package/62aff1e13bfa6a447a1ecd8198b9e13b18a63e91/lib/libz_fPIC.a' '/home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed/grpc'
Traceback (most recent call last):
  File "/usr/local/bin/ue4", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/dist-packages/ue4cli/cli.py", line 222, in main
    SUPPORTED_COMMANDS[command]['action'](manager, args)
  File "/usr/local/lib/python3.6/dist-packages/conan_ue4cli/main.py", line 48, in main
    SUBCOMMANDS[subcommand]['function'](manager, args[1:])
  File "/usr/local/lib/python3.6/dist-packages/conan_ue4cli/commands/build.py", line 130, in build
    builder.build(name, version, args.options if args.options is not None else [])
  File "/usr/local/lib/python3.6/dist-packages/conan_ue4cli/common/PackageBuilder.py", line 45, in build
    self._executor.execute(command, cwd=tempDir, check=True)
  File "/usr/local/lib/python3.6/dist-packages/conan_ue4cli/common/CommandExecutor.py", line 19, in execute
    return subprocess.run(command, **kwargs).returncode == 0
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['conan', 'install', 'grpc-ue4/1.30.2@adamrehn/4.23', '--profile=ue4.23-Linux-x86_64-unknown-linux-gnu', '--build=missing']' returned non-zero exit status 1.
The command '/bin/sh -c ue4 conan build grpc-ue4' returned a non-zero code: 1
ERROR: Service 'server' failed to build

Another grpc (1.30.2) build failure: error: use of undeclared identifier 'TCP_USER_TIMEOUT'

Running ./start-server.sh produces the following gRPC build failure:

[ 71%] Building CXX object CMakeFiles/grpc.dir/src/core/lib/json/json_writer.cc.o
[ 71%] Building CXX object CMakeFiles/grpc.dir/src/core/lib/security/context/security_context.cc.o
[ 71%] Building CXX object CMakeFiles/grpc.dir/src/core/lib/security/credentials/alts/alts_credentials.cc.o
/home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed/grpc/src/core/lib/iomgr/socket_utils_common_posix.cc:325:42: error: use of undeclared identifier 'TCP_USER_TIMEOUT'
if (0 != setsockopt(fd, IPPROTO_TCP, TCP_USER_TIMEOUT, &timeout,
^
/home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed/grpc/src/core/lib/iomgr/socket_utils_common_posix.cc:330:42: error: use of undeclared identifier 'TCP_USER_TIMEOUT'
if (0 != getsockopt(fd, IPPROTO_TCP, TCP_USER_TIMEOUT, &newval, &len)) {

2 errors generated
CMakeFiles/grpc.dir/build.make:6470: recipe for target 'CMakeFiles/grpc.dir/src/core/lib/iomgr/socket_utils_common_posix.cc.o' failed
make[3]: *** [CMakeFiles/grpc.dir/src/core/lib/iomgr/socket_utils_common_posix.cc.o] Error 1
make[3]: *** Waiting for unfinished jobs....
^[[0m[ 73%] Building CXX object CMakeFiles/grpc.dir/src/core/lib/security/credentials/alts/check_gcp_environment.cc.o
Building CXX object CMakeFiles/grpc.dir/src/core/lib/security/credentials/alts/check_gcp_environment_linux.cc.o
make[2]: *** [CMakeFiles/grpc.dir/all] Error 2
CMakeFiles/Makefile2:861: recipe for target 'CMakeFiles/grpc.dir/all' failed
CMakeFiles/Makefile2:1180: recipe for target 'CMakeFiles/grpc++.dir/rule' failed
make[1]: *** [CMakeFiles/grpc++.dir/rule] Error 2
Makefile:489: recipe for target 'grpc++' failed
make: *** [grpc++] Error 2
Warning: the libcxx package has been deprecated and will be removed in a future version of conan-ue4cli.
grpc-ue4/1.30.2@adamrehn/4.23:
cares-ue4/1.16.1@adamrehn/4.23: WARN: This conanfile has no package step
protobuf-ue4/3.12.3@adamrehn/4.23: WARN: This conanfile has no package step
grpc-ue4/1.30.2@adamrehn/4.23: ERROR: Package 'bbaadf9c14f1821b000882e0282e20b2519fdbed' build failed
grpc-ue4/1.30.2@adamrehn/4.23: WARN: Build folder /home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed
ERROR: grpc-ue4/1.30.2@adamrehn/4.23: Error in build() method, line 99
cmake.build(target="grpc++")
ConanException: Error 2 while executing cmake --build '/home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed' '--target' 'grpc++' '--' '-j40'
Traceback (most recent call last):
File "/usr/local/bin/ue4", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.6/dist-packages/ue4cli/cli.py", line 222, in main
SUPPORTED_COMMANDS[command]['action'](manager, args)
File "/usr/local/lib/python3.6/dist-packages/conan_ue4cli/main.py", line 48, in main
SUBCOMMANDS[subcommand]['function'](manager, args[1:])
File "/usr/local/lib/python3.6/dist-packages/conan_ue4cli/commands/build.py", line 130, in build
builder.build(name, version, args.options if args.options is not None else [])
File "/usr/local/lib/python3.6/dist-packages/conan_ue4cli/common/PackageBuilder.py", line 45, in build
self._executor.execute(command, cwd=tempDir, check=True)
File "/usr/local/lib/python3.6/dist-packages/conan_ue4cli/common/CommandExecutor.py", line 19, in execute
return subprocess.run(command, **kwargs).returncode == 0
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['conan', 'install', 'grpc-ue4/1.30.2@adamrehn/4.23', '--profile=ue4.23-Linux-x86_64-unknown-linux-gnu', '--build=missing']' returned non-zero exit status 1.
Service 'server' failed to build: The command '/bin/sh -c ue4 conan build grpc-ue4' returned a non-zero code: 1

server->Wait()

Hi~When I read GRPC tutorials, I found they call "server->Wait()" when starting a server, which is different from this repo. So, is there any reason that "Wait()" shouldn't be called here?

[request/enhancement/discussion] Shift gRPC service implementation to a plugin

Hi @adamrehn,

Thanks a lot for this demo!

I was trying something similar but with the service implementation in Plugin code, for portability. The problem I'm facing is not an issue of this repo, but I hope you will be able to help me and it might be useful for others as well.

I created a project without the docker part of your project, and was able to launch gRPC server on it. Now, I'm trying to port the GrpcServiceImp.h & it's .cpp part into a Plugin along with the threadingHelper code. The server is still created during BeginPlay() of an Actor/Pawn whom I'd like to control. However, engine is crashing when I try to send commands from a python client. The trace was something like this

<CALLSTACK START>
grpc_impl::internal::RpcMethodHandler<PawnMovementDemo::Service, Direction, Empty>::Deserialize(grpc_call*, grpc_byte_buffer*, grpc::Status*, void**) Address = 0x174ee7926 [~/.local/include/grpcpp/impl/codegen/method_handler_impl.h, line 99] [in UE4Editor-MyPlugin-0021.dylib]
grpc_impl::Server::SyncRequest::CallData::Run(std::__1::shared_ptr<grpc_impl::Server::GlobalCallbacks> const&, bool) Address = 0x1753e9966 (filename not found) [in UE4Editor-ConanUE4-7613.dylib]
grpc_impl::Server::SyncRequestThreadManager::DoWork(void*, bool, bool) Address = 0x1753e880b (filename not found) [in UE4Editor-ConanUE4-7613.dylib]
grpc::ThreadManager::MainWorkLoop() Address = 0x175424101 (filename not found) [in UE4Editor-ConanUE4-7613.dylib]
grpc::ThreadManager::WorkerThread::Run() Address = 0x175423dec (filename not found) [in UE4Editor-ConanUE4-7613.dylib]
grpc::ThreadManager::WorkerThread::WorkerThread(grpc::ThreadManager*)::$_0::operator()(void*) const Address = 0x17542507c (filename not found) [in UE4Editor-ConanUE4-7613.dylib]
grpc::ThreadManager::WorkerThread::WorkerThread(grpc::ThreadManager*)::$_0::__invoke(void*) Address = 0x175425055 (filename not found) [in UE4Editor-ConanUE4-7613.dylib]
grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&)::'lambda'(void*)::operator()(void*) const Address = 0x1759537ef (filename not found) [in UE4Editor-ConanUE4-7613.dylib]
grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&)::'lambda'(void*)::__invoke(void*) Address = 0x1759536e5 (filename not found) [in UE4Editor-ConanUE4-7613.dylib]
_pthread_start Address = 0x7fff73926109 (filename not found) [in libsystem_pthread.dylib]
thread_start Address = 0x7fff73921b8b (filename not found) [in libsystem_pthread.dylib]
<CALLSTACK END>

Initially I had the gRPC dependencies in my project's .build.cs. Now, I've included the same dependencies in the Plugin's .build.cs and removed it from the project's .build.cs. This is static linkage of gRPC libraries.

The .uplugin part contains

"FileVersion": 3,
"Version": 1,
"VersionName": "1.0",
"FriendlyName": "",
"Description": "",
"Category": "Other",
"CreatedBy": "",
"CreatedByURL": "",
"DocsURL": "",
"MarketplaceURL": "",
"SupportURL": "",
"CanContainContent": true,
"IsBetaVersion": false,
"IsExperimentalVersion": false,
"Installed": false,
"Modules": [
{
"Name": "MyPlugin",
"Type": "Runtime",
"LoadingPhase": "Default"
}
]

and I've added the dependency of the Plugin module in the project's .build.cs.

I tried different LoadingPhase for the Plugin module - PreDefault, Default, PostDefault and none of them worked.

The functionality is not failing when I start the game, but when I send my first message from the client. For me, it looks like some thread issue - one thread is not able to communicate to another? I'm launching the service implementations on GameThreads as done in your code. Is this a problem when a plugin is involved? I'm unable to understand what is causing this issue. Any help that could direct me towards potential problems would be really helpful!

Thanks in advance!

Cheers!

./start-server.sh failed.

Hello~:)
I typed the following command.(in Ubuntu)

./start-server.sh

As a result, the following error occurred.

conan-cli, conan-docker are lastest version and I have "adamrehn/ue4-full:4.23.1-opengl" image.
Is this a bug related to [conan-recipi] that was updated a few days ago? I need help.

-- Configuring incomplete, errors occurred!
See also "/home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed/CMakeFiles/CMakeOutput.log".
See also "/home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed/CMakeFiles/CMakeError.log".
grpc-ue4/1.30.2@adamrehn/4.23:
cares-ue4/1.16.1@adamrehn/4.23: WARN: This conanfile has no package step
protobuf-ue4/3.12.3@adamrehn/4.23: WARN: This conanfile has no package step
grpc-ue4/1.30.2@adamrehn/4.23: ERROR: Package 'bbaadf9c14f1821b000882e0282e20b2519fdbed' build failed
grpc-ue4/1.30.2@adamrehn/4.23: WARN: Build folder /home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed
ERROR: grpc-ue4/1.30.2@adamrehn/4.23: Error in build() method, line 98
cmake.configure(source_folder="grpc", args=self.cmake_flags())
ConanException: Error 1 while executing cd '/home/ue4/.conan/data/grpc-ue4/1.30.2/adamrehn/4.23/build/bbaadf9c14f1821b000882e0282e20b2519fdbed' && cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release" -DCONAN_IN_LOCAL_CACHE="ON" -DCONAN_COMPI

Build failed ./start-server.sh

Hi, how to solve this problem ?

Docker images

adamrehn/ue4-full 4.23.1 431c572bbc63 2 hours ago 20.4GB
adamrehn/ue4-full 4.23.1-opengl-ubuntu18.04 431c572bbc63 2 hours ago 20.4GB
adamrehn/ue4-minimal 4.23.1 50059ae76c42 2 hours ago 19.2GB
adamrehn/ue4-minimal 4.23.1-opengl-ubuntu18.04 50059ae76c42 2 hours ago 19.2GB
adamrehn/ue4-source 4.23.1 e64166660925 18 hours ago 38.5GB
adamrehn/ue4-source 4.23.1-opengl-ubuntu18.04 e64166660925 18 hours ago 38.5GB
adamrehn/ue4-build-prerequisites opengl-ubuntu18.04 75c2a6d4a9cd 19 hours ago 783MB

I change just this lines in dockerfiles

FROM adamrehn/ue4-build-prerequisites:opengl
to
FROM adamrehn/ue4-build-prerequisites:opengl-ubuntu18.04

FROM adamrehn/ue4-full:4.23.1-opengl as builder
to
FROM adamrehn/ue4-full:4.23.1-opengl-ubuntu18.04 as builder

if i don't change this lines i get this error

Building server
Sending build context to Docker daemon 678.9kB
Step 1/12 : FROM adamrehn/ue4-full:4.23.1-opengl as builder
pull access denied for adamrehn/ue4-full, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
ERROR: Service 'server' failed to build : Build failed
Building server
Sending build context to Docker daemon 678.9kB
Step 1/12 : FROM adamrehn/ue4-full:4.23.1-opengl as builder
pull access denied for adamrehn/ue4-full, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
ERROR: Service 'server' failed to build : Build failed

./start-server.sh error

cares-ue4/1.16.1@adamrehn/4.23: WARN: This conanfile has no package step
protobuf-ue4/3.19.1@adamrehn/4.23: WARN: This conanfile has no package step
ERROR: grpc-ue4/1.42.0@adamrehn/4.23: Error in package_info() method, line 123
self.user_info.build_data = json.dumps([self.deps_cpp_info["protobuf-ue4"].bin_paths[0], self.cpp_info.bin_paths[0]])
IndexError: list index out of range
Traceback (most recent call last):
File "/usr/local/bin/ue4", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.6/dist-packages/ue4cli/cli.py", line 222, in main
SUPPORTED_COMMANDS[command]['action'](manager, args)
File "/usr/local/lib/python3.6/dist-packages/conan_ue4cli/main.py", line 48, in main
SUBCOMMANDS[subcommand]['function'](manager, args[1:])
File "/usr/local/lib/python3.6/dist-packages/conan_ue4cli/commands/build.py", line 130, in build
builder.build(name, version, args.options if args.options is not None else [])
File "/usr/local/lib/python3.6/dist-packages/conan_ue4cli/common/PackageBuilder.py", line 45, in build
self._executor.execute(command, cwd=tempDir, check=True)
File "/usr/local/lib/python3.6/dist-packages/conan_ue4cli/common/CommandExecutor.py", line 19, in execute
return subprocess.run(command, **kwargs).returncode == 0
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['conan', 'install', 'grpc-ue4/1.42.0@adamrehn/4.23', '--profile=ue4.23-Linux-x86_64-unknown-linux-gnu', '--build=missing']' returned non-zero exit status 1.

Cannot rebuild unreal projects with newer version of Unreal

I currently have Unreal Engine 4.23.1 installed on my Linux (Ubuntu 18.04) system. I am trying to run your microservices grpc demo but am facing the following error:

------- Build details --------
Using toolchain located at '/home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v15_clang-8.0.1-centos7/x86_64-unknown-linux-gnu'.
Using clang (/home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v15_clang-8.0.1-centos7/x86_64-unknown-linux-gnu/bin/clang++) version '8.0.1' (string), 8 (major), 0 (minor), 1 (patch)
Using bundled libc++ standard C++ library.
Using lld linker
Using llvm-ar : /home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v15_clang-8.0.1-centos7/x86_64-unknown-linux-gnu/bin/llvm-ar
Using fast way to relink circularly dependent libraries (no FixDeps).

Performing 8 actions (12 in parallel)
[1/8] Compile SharedPCH.Engine.h
[2/8] Compile CubePhysicsDemo.cpp
[3/8] Compile GrpcServerWrapper.cpp
[4/8] Compile GrpcServiceImp.cpp
[5/8] Compile CubePhysicsDemo.init.gen.cpp
[6/8] Compile GrpcServerWrapper.gen.cpp
[7/8] Link (ld) libUE4Editor-CubePhysicsDemo.so
/tmp/CubePhysicsDemo/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/CubePhysicsDemo/GrpcServiceImp.cpp.o: In function GrpcServiceImp': /tmp/CubePhysicsDemo/Source/CubePhysicsDemo/Private/GrpcServiceImp.cpp:4: undefined reference to CubePhysicsDemo::Service::Service()'
/tmp/CubePhysicsDemo/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/CubePhysicsDemo/GrpcServiceImp.cpp.o: In function GrpcServiceImp::GetCubeTransform(grpc::ServerContext*, Empty const*, Transform*)': /home/ue4/.conan/data/protobuf-ue4/3.6.1/adamrehn/4.23/package/85671f6de7947f1cae40e263f8639f526bdf7cfe/include/google/protobuf/message_lite.h:391: undefined reference to Vector3D* google::protobuf::Arena::CreateMaybeMessage(google::protobuf::Arena*)'
/home/ue4/.conan/data/protobuf-ue4/3.6.1/adamrehn/4.23/package/85671f6de7947f1cae40e263f8639f526bdf7cfe/include/google/protobuf/message_lite.h:391: undefined reference to Vector3D* google::protobuf::Arena::CreateMaybeMessage<Vector3D>(google::protobuf::Arena*)' /home/ue4/.conan/data/protobuf-ue4/3.6.1/adamrehn/4.23/package/85671f6de7947f1cae40e263f8639f526bdf7cfe/include/google/protobuf/message_lite.h:391: undefined reference to Vector3D* google::protobuf::Arena::CreateMaybeMessage(google::protobuf::Arena*)'
/home/ue4/.conan/data/protobuf-ue4/3.6.1/adamrehn/4.23/package/85671f6de7947f1cae40e263f8639f526bdf7cfe/include/google/protobuf/message_lite.h:391: undefined reference to Vector3D* google::protobuf::Arena::CreateMaybeMessage<Vector3D>(google::protobuf::Arena*)' /home/ue4/.conan/data/protobuf-ue4/3.6.1/adamrehn/4.23/package/85671f6de7947f1cae40e263f8639f526bdf7cfe/include/google/protobuf/message_lite.h:391: undefined reference to Vector3D* google::protobuf::Arena::CreateMaybeMessage(google::protobuf::Arena*)'
/tmp/CubePhysicsDemo/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/CubePhysicsDemo/GrpcServiceImp.cpp.o:/home/ue4/.conan/data/protobuf-ue4/3.6.1/adamrehn/4.23/package/85671f6de7947f1cae40e263f8639f526bdf7cfe/include/google/protobuf/message_lite.h:391: more undefined references to Vector3D* google::protobuf::Arena::CreateMaybeMessage<Vector3D>(google::protobuf::Arena*)' follow /tmp/CubePhysicsDemo/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/CubePhysicsDemo/GrpcServiceImp.cpp.o: In function ~GrpcServiceImp':
/tmp/CubePhysicsDemo/Source/CubePhysicsDemo/Public/GrpcServiceImp.h:7: undefined reference to CubePhysicsDemo::Service::~Service()' /tmp/CubePhysicsDemo/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/CubePhysicsDemo/GrpcServiceImp.cpp.o:(.data.rel.ro+0x10): undefined reference to CubePhysicsDemo::Service::~Service()'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
Using user-specified engine root: /home/ue4/UnrealEngine
Traceback (most recent call last):
File "/usr/local/bin/ue4", line 11, in
sys.exit(main())
File "/usr/local/lib/python3.6/dist-packages/ue4cli/cli.py", line 216, in main
SUPPORTED_COMMANDS[command]['action'](manager, args)
File "/usr/local/lib/python3.6/dist-packages/ue4cli/cli.py", line 70, in
'action': lambda m, args: m.buildDescriptor(os.getcwd(), args.pop(0) if len(args) > 0 else 'Development', args.pop(0) if len(args) > 0 else 'Editor', args),
File "/usr/local/lib/python3.6/dist-packages/ue4cli/UnrealManagerBase.py", line 339, in buildDescriptor
self._runUnrealBuildTool(target, self.getPlatformIdentifier(), configuration, baseArgs + args, capture=suppressOutput)
File "/usr/local/lib/python3.6/dist-packages/ue4cli/UnrealManagerBase.py", line 635, in _runUnrealBuildTool
Utility.run(arguments, cwd=self.getEngineRoot(), raiseOnError=True)
File "/usr/local/lib/python3.6/dist-packages/ue4cli/Utility.py", line 138, in run
raise Exception('child process ' + str(command) + ' failed with exit code ' + str(returncode))
Exception: child process ['/home/ue4/UnrealEngine/Engine/Build/BatchFiles/Linux/Build.sh', 'CubePhysicsDemoEditor', 'Linux', 'Development', '-project=/tmp/CubePhysicsDemo/CubePhysicsDemo.uproject'] failed with exit code 5
ERROR: Service 'server' failed to build: The command '/bin/sh -c ue4 build' returned a non-zero code: 1

Additionally, I tried to rebuild the 4.21 version of the CubePhysicsDemo project using Unreal Engine ver 4.23.1 that I have currently installed. It is throwing the following errors when I try rebuilding your project:

Please let me know how I can "rebuild the project from source manually". I also suspect there is some linker error with clang++. Here is what it says:

/tmp/CubePhysicsDemo/Source/CubePhysicsDemo/Public/GrpcServiceImp.h:7: undefined reference to CubePhysicsDemo::Service::~Service()' /tmp/CubePhysicsDemo/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/CubePhysicsDemo/GrpcServiceImp.cpp.o:(.data.rel.ro+0x10): undefined reference to CubePhysicsDemo::Service::~Service()'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

I tried following the steps in your tutorial for making a build using connan-ue4cli, but I am confused after the generation of wrappers instruction. When does one make a build. How do I monitor whether a third party tool such as gRPC is successfully built in the ue4cli cache?

Thank you for all your help! I really hope that the implementations become more detailed in the future for the use cases.