envoyproxy / envoy-mobile Goto Github PK

We should bundle the Envoy OSS docs at the Envoy SHA we are using in our docs, which will allow direct Sphinx ref-linking for more stable documentation.

This broke with the Abseil update in upstream Envoy.

We can hoist Envoy's check_format to do this.

We need to design a threading model in the client (both iOS and Android) that is compatible with Envoy's threading model. More information on Envoy's threading model can be found in Matt's blog post.

cc @rebello95 @goaway

Using native sockets on iOS/Android is the recommended approach for performing low-level networking. We should investigate:

• What benefits we can get from using native sockets/APIs
• The work required to use these APIs, both within Envoy Mobile and upstream Envoy

In order for us to be able to ship Envoy on the client we need to optimize for binary size.

We will need to reduce Envoy Mobile’s final application bloat (compressed) down to < 5MB to be able to ship in a production mobile application

ios and android need to provide legal notices for OSS software that gets shipped to clients. The notices will need to be updated for envoy-mobile, and all of envoy-mobile's diamond dependencies.

All end-users of this software will have the same requirements, so an approach that easily handles this for Lyft and for downstream users would be ideal.

An approach that would likely work here, is to ensure the NOTICE file includes information for all dependencies (ideally diamond dependencies).

We're currently using https://github.com/keith/rules_kotlin which is a fork of https://github.com/loreto/rules_kotlin. We'll want to have a more permanent solution which doesn't rely on @keith's repository.

Building the Android demo can only be built with a single architecture.

The current android_binary Bazel rule has a bug (bazelbuild/bazel#8283) where it doesn't pack multiple architectures from aar_imports. Underneath the android_binary, it uses aar_imports for any aar dependencies.

Any Android applications using Bazel depending on Envoy Mobile will have this issue.

After we have documentation cleaned up we should handoff the project to mobile devs at Lyft and make sure the instructions are sufficient to build all demos from scratch.

Design an API for envoy to be aware of OS level signals (e.g connection to WiFi vs mobile network, app backgrounded) in order for envoy to react appropriately.

We should add docs for building the Android demo using the structure implemented in https://github.com/lyft/envoy-mobile/pull/22/files.

During master CI we should publish a compiled library for envoy that people can use to avoid compiling on their own.

https://envoy-mobile.github.io/docs/envoy-mobile/latest/index.html#

Top left reads "envoy" instead of "Envoy Mobile". I will fix once we merge from the dev branch.

We have left todos as we have gone. I want to take a pass the day before release and make sure that we have resolved all resolvable TODOs, and left behind only ones that we mean to leave behind.

Validate that BSD sockets (as they're used in upstream Envoy) will not be a problem on Android like they are on iOS.

Note: We should compare this to Cronet's implementation.

We should make sure that CI runs all 4 demo targets successfully.

Currently Envoy has a file logging sink, and a stderr logging sink. Both of these do not work with android. We need to implement an Android logging sink in order to get Envoy logs in Android. We can lift up spdlog's android_sink.

This issue is a subtask of #55

(It might be right that this should be reflected as an Envoy OSS issue, but going to discuss here because of the context)

Envoy OSS has a filesystem abstraction in order to manipulate the host's filesystem. The interfaces live here. A few months ago several folks from the OSS refactored the implementation of the filesystem abstraction in order to allow for platform specific implementations of the filesystem via bazel select statements like this.

The current implementations depend on APIs that don't exist or currently are not supported. Moreover, in Mobile architectures the Envoy thread would not have full access to the filesystem like it does when running on a machine in "server mode". Thus in order to be able to run tests that depend on the filesystem libraries, we need implementations that work against mobile architectures.

Note: I haven't looked into the watcher area of this problem space yet.

Where are these platform dependent libraries used in tests?

directory_lib
|_> //test/common/filesystem:directory_test
|_> //test/test_common:utility_lib

filesystem_lib
|_> //test/common/filesystem:filesystem_impl_test
|_> //test/test_common:utility_lib
|_> //test/test_common:file_system_for_test_lib
    |_> |_> //test/test_common:utility_lib

Note: it is funky that //test/test_common:utility_lib depends on both //test/test_common:file_system_for_test_lib and //test/test_common:file_system_lib. I need to look into that.

Options for a filesystem implementation that works against mobile architectures

1. Create usable filesystem implementations against each mobile architecture. This would allow us to depend on working implementations for tests, and potentially in production code. This might be easily done in a future version of Xcode that provides a sandboxed filesystem, and with the standardized std::filesystem library in C++17. I have not done research on this for Android.
2. Alternatively, we could create a fake filesystem implementation that builds an in memory filesystem against the API that Envoy has. This could very simply be implemented with a map of paths to simple FakeFile structs (that have a file type, a buffer, etc.). This option would be viable to run tests against both mobile architectures. However, it would obviously not be a usable model in production code.

This pretty easy to do, standard in networking libraries, and simplifies usage.

Creating this issue because I think it would be good to document what pieces of core envoy are compiled in/out of envoy mobile. This information allows users to reason about the pieces of envoy we use here. It would also allow developers to know what pieces of envoy they can rely on being compiled in/out when making contributions to envoy mobile.

We have decided to have all documentation in sphinx.

Once #45 is done, we should add docs like those being added in #64 for the Kotlin demo.

Context: #64 (comment)

Implement c++ layer of the defined v0.2 APIs in order to accept requests to encode and dispatch.

Instead of talking to Envoy (upstream) over a socket, expose interfaces from Envoy upstream which are called into directly from the Envoy Mobile (common) code.

Note: This is something we can de-prioritize and push to v0.3 if necessary, but it’s possible that this is a blocker for shipping to production

Create v0.1 release on github once the repo is golden for public. The release should have both the iOS framework and the Android aar as distributables.

• Security review
• Pete approval
• Ryan + Derek approval

Right now the Azure organization used for CI doesn't allow contributors to this repo to see details on CI jobs. We'll need to update these policies to allow others to see this information.

@junr03 feel free to add more details here.

A lot of things in the codebase are not intuitive. We should take a pass on the codebase and add comments were things are not transparent. This will help with all the eyes we are getting on the repo soon.

Create the project website. The current proposal is to use a static site on github pages.

We should add docs for building the iOS demo using the structure implemented in https://github.com/lyft/envoy-mobile/pull/22/files.

Proposal for refactoring in core envoy to allow for seamless use of transport in iOS and Android as well as current server mode.

Currently Envoy OSS's test suite does not run against mobile architectures. We would like to enable the full (or as close as reasonably possible) test suite to run against mobile architectures. This will enable envoy-mobile's CI to run tests against upstream.

Changes need to happen on two broad categories for this to be possible:

1. Build changes: we need to create bazel infrastructure to selectively depend, compile, and run code against mobile architectures. Inspiration can be take from envoy_cc_platform_dep which allows to select dependencies against different build targets.
2. Code changes: there are a few refactors, and code additions that need to happen in order to be able to run tests against mobile architecture targets:
   • filesystem: we need to use iOS and Android filesystem constructs were appropriate. Thankfully, folks from the OSS community made the barrier to entry of this work more manageable when introducing win32 filesystem implementations.
   • via @keith the google test stuff has some "expect death" stuff that needs to be solved somehow since on iOS you only ever have 1 process. totally unclear to me, even from the feature side, how we would want that to work his opinion is that this could be deferred.

cc @keith who was previously working on this.

Integrate with clang format on CI the same way upstream Envoy does.

We all agree that our style guide should be encoded in linters, not up to reviewers eyes. CI should catch errors in a "check_format" run.

In order for us to be able to ship Envoy on the client we need to optimize for startup time.

Our target is TBD. @tonya11en @goaway what should the target be?

What is the configuration that we are going to use for Lyft's MVP?

Enable use of a bazel cache

Create a sample project that people can play with in Swift.

Related: #21

We should run this formatter on Objective-C files just like we do for C++.

cc @goaway @mattklein123 @junr03

We've comprised a proposal for Envoy Mobile's public APIs, broken down by release/milestone. This document will serve as a living roadmap for these APIs as they grow and evolve over time.

As tasks are prioritized, they'll be filed as GitHub issues and associated with relevant milestones/labels.

Google Doc: Link

clang-format can be used for auto-formatting Java files the same way it is for Objective-C and C++.

The iOS build job has been moved over to Azure, and we should do the same with the rest of our jobs since this is what we'll be using going forward.

Write up an initial announcement blog post for the project. The current idea is to clean up, and synthesize the Client Networking vision document: https://drive.google.com/open?id=1StRxZzY8efOOs0maLJjcSxGyBHZ66zy9LlQoySYAVNE

Integrate SwiftLint on CI for Swift styling.

We all agree that our style guide should be encoded in linters, not up to reviewers eyes. CI should catch errors in a "check_format" run.

What should envoy do when the App is backgrounded/killed? related to #10

Lets use this issue to track all the bots we want for the project:

• stalebot
• DCO bot / OSS CLA DCO
• RepoKitteh (needs discussion w/ legal)

Validate that we actually need all of the link options we're currently passing through Bazel, as some extra ones may have been added during initial development.

Because we are building a Library, we should have API style docs in addition to the Sphinx docs. Doxygen is a candidate tool.

Integrate with a linter on CI for Kotlin styling.

We all agree that our style guide should be encoded in linters, not up to reviewers eyes. CI should catch errors in a "check_format" run.