From #15:

I think it makes sense for the app to only ever have one refresh token. It might use that refresh token to grant multiple different access tokens with different combinations of scopes, but it'll be simpler both here and on the server if we can ensure it only has one refresh token that lists all the scopes granted to the application. But that can be for a follow-up issue.

Case 1:
If a token with the scope "profile https://identity.mozilla.com/apps/oldsync" is cached in FirefoxAccount and if I call get_oauth_token with only profile, I should get the cached token instead of requesting a new one.

Case 2:
If a token with the scope profile is cached in FirefoxAccount and I call get_oauth_token with profile:write, I should get the profile token since it supersedes profile:write.

See related PR: https://github.com/mozilla/fxa-shared/pull/21/files

I'm still debating if it should live in the Rust crate or if it is the responsibility of the host app to do it. Thoughts?

┆Issue is synchronized with this Jira Story
┆Story Points: 8

It would be nice to be certain that the FxA structure is Send, given that it's used from multiple threads in mozilla-mobile/android-components#362 (well, that has issues in that it would require Sync, but mozilla-mobile/android-components#388 will still require Send)

This is still a large open question! It's not clear if we should implement part/all of the sync ping, or leave it to callers, possibly providing them with some data that they may want to include in telemetry.

We have a fork of lockbox here that almost works with scoped keys, it is just missing the profile request:

https://github.com/vladikoff/lockbox-ios/tree/fxa-client-sandvich

Right now, the initial landing branch doesn't have a Rustdoc "landing page" for consumers. There are enough assumptions in the existing expression, especially about Sync 1.5 and the path to Sync.next, that we should capture them for the benefit of potential consumers.

cc @shane-tomlinson

I think we are missing that post

I believe that to simplify and shrink our Rust libraries builds, we should move away from OpenSSL.
Instead, we should use the ring rust library which doesn't need to be linked against a static library.
However, ring is still incomplete and we are missing some important crypto primitives, so we should consider this issue more as a laundry list.

┆Issue is synchronized with this Jira Story

At a minimum it needs to handle backoff, but likely a lot more of the sync state machine. iOS's code might be a good model.

Some discussion: #12 (comment)

At a minimum, regardless of how it's implemented, we absolutely need to

• Handle backoff (both from the tokenserver and the storage server).
• Handle outdated meta/global, possibly uploading it if we're the first there.
• Handle outdated crypto/keys, possibly uploading new keys.
• Handle refreshing the sync token.
• ...

The initial logins landing goes to a great deal of effort to splice local metadata into Sync 1.5 password records uploaded to the service. However, because Bug 555755 is still open, uploading local metadata changes to the service might actually be bad for a client that really wants to use the metadata changes! With logins based on Mentat, however, we're explicitly modeling the local metadata as local usage events. We already expose metadata about individual records: see

We need a way to match proxy, networking, tor, gecko-view settings in fxa-rust-client based on what they are in Fenix

From #6 (comment):

• We should have a script for building libs/ for x86_64 only (travis ci/local testing). Maybe there's a way to code re-use what we have for iOS, or not.
• We should have a warm-up cache phase so we don't end up building/caching the same libs/ twice.
• Unlike what we're doing right now, we should build our dependencies in a build.rs file.

This is a POC as discussed in IRC last night. https://github.com/mozilla-lockbox/lockbox-ios-fxa-sync/tree/master/Storage/SQL is a good starting point.

Note: #58 was landed, possibly a bit early, as to not block this.

My vague plan, at least recently has been something like:

pub struct RecordChangeset<T> {
    pub changed: Vec<BsoRecord<T>>,
    pub deleted_ids: Vec<String>,
    /// For GETs, the last sync timestamp that should be persisted after
    /// applying the records.
    /// For POSTs, this is the XIUS timestamp.
    pub timestamp: ServerTimestamp,
}

impl Sync15Service {
    pub fn get_changes<T: Sync15Record>(
        &mut self,
        since: ServerTimestamp
    ) -> Result<RecordChangeset<T>>;

    pub fn post_changes<T: Sync15Record>(
        &mut self,
        changes: RecordChangeset<T>,
        // Whether or not oversized and/or server-failed records
        // blow everything up.
        fully_atomic: bool
    ) -> Result<ServerTimestamp>; // Might want to return the Vec of failed ids too?
}

Note that we will likely want to trim down the Result to the set of errors that can actually happen for this.

Also note that the methods might be on something that isn't Sync15Service, but just occupies the role it currently holds.

Implementation note: both methods would encrypt the records immediately to avoid having more code than is necessary monomorphized for each record type.

Pros

1. API encourages clients to store sync change metadata with the record.
2. Minimizes callbacks between Swift and Rust. Entirely avoids having passing around FFI function pointers (both in Rust and in swift/java). Not having an event driven system also avoids problems like sync tracker's ignoreAll and such.
3. The API makes deletions explicit and not dependent on the caller knowing about the tombstone format (unlike using JSON directly).

Cons

1. Clients might decide that storing any sort of sync change metadata is a PITA and do a sync for each change they want to upload (e.g. Pro number 1 may backfire). I don't think this is that likely for shipping code, though.
2. Leaves managing sync change timestamps fully up to the app (I think this is going to be more or less necessary for most designs, unfortunately).
3. Shaped nothing like what is proposed in the lockbox sync google doc (Not sure how much this matters, there's a good chance we can either write some sort of wrapper that does what they want, or we'd have problems with it anyway).
4. Likely requires that T get exposed over the FFI?

@mhammond I expect you'll have some opinions about this?

We have namespaces, no need to repeat ourselves.

Instead of a list of servers from fxa-client-configuration

we need to use https://accounts.firefox.com/.well-known/openid-configuration to discovery oauth endpoints.

Especially important for mozilla/fxa-content-server#6113

cc @rfk

Over in https://github.com/mozilla/mentat, we have a Rust compiler version check and we try to run Travis CI against that version (https://github.com/mozilla/mentat/blob/88c6a4b05ca41d3900c7dded53b928e4aa9c7ab8/.travis.yml#L8).

It would be nice if application-services culted that approach so that I don't have to translate error messages into compiler requirements.

@eoger could we pull in the fxa-rust-client into here?

@eoger tracking issue, not sure if already have enough detailed issues for this

We should be able to do incremental builds. All three libs use autotools, so I think this would probably work out of the box if we just used normal vpath builds (e.g. when you invoke configure from a different directory than the directory where it lives).

I think this would look like:

mkdir -p build-$ARCH-$HOST
cd build-$ARCH-$HOST
../configure --host="$HOST" --with-openssl="$OPENSSL_DIR" --with-jansson="$JANSSON_DIR" && 
cd -

around https://github.com/mozilla/application-services/blob/master/libs/build-cjose-ios.sh#L40 or so, (and in the other build scripts ofc) as long as we also update the parts that grab the build artifacts out of the folders so that they look in the right place.

There shouldn't be any need to have a make clean also, because we wouldn't have the build artifacts for the wrong build lying around. In fact, keeping make clean would defeat the whole point. There might be a use case for it so it's possible that we either want to keep a script that does it, maybe.

Sorry in advance if there's a reason we aren't doing this already.

┆Issue is synchronized with this Jira Story

Right now, the initial landing branch produces a lot of "empty" Mentat transactions (i.e., ones which only have a :db/txInstant) as part of the Sync 1.5 flow. This ticket tracks the narrow outcome of ensuring that a single "sync" results in a single Mentat transaction.

However, those empty transactions are a sign of a larger problem: it's not natural to build large transactions in Rust, because the transaction datatype (Entities) is awkward to manipulate. In Clojure, the transaction datatype is built of Clojure-native things (vectors, lists, maps) so it's very natural to grow a single structure for transaction. This ticket might point to larger changes in the input structures that Mentat can process.

Instead of having the app doing

fxa.operation_that_mutates_interal_state()
let json = fxa.to_json()
persistInSecureStorage(json)

We could do something like:

fxa.addPersistListener((json) => { persistInSecureStorage(json) });
fxa.operation_that_mutates_interal_state()
// Nothing to do, the callback has been called automatically in operation_that_mutates_interal_state

At the moment it is entirely in-memory. We probably want to leave actually persisting the state to the library user, but give them a JSON string to save (More granular updates may be necessary in the future but ATM it doesn't seem worth it).

At a minimum it should be saving: syncID (and version?) for each collection (this comes from meta/global), lastSync timestamp for each collection (which is a remote timestamp), and probably more?

Is there anything from the token we need to save? (I think the answer is no?)

We need to figure out how do we want to expose errors to callers (using an out err parameter), either a struct or simply an error code.

Discussion is in #94, specifically #94.

@vladikoff can we remove it?

We only use it for HMACs, which are not be too hard to implement correctly. We should use https://github.com/RustCrypto/MACs/tree/master/hmac

These leak on every call that returns an error. This should be pretty easy to do now that we've basically hammered out the patterns.

I think this is blocked on #83. Or at least, unless there's a reason not to block it on that. (I don't mind taking this, but I'd rather not go through it twice)

In our meeting last week, @eoger @vladikoff and I kicked around some ideas for what API to expose the FxA client apps such as lockbox. It will hopefully be different from (and simpler than) the FxA Client API used by Firefox Desktop, because it's based on OAuth rather than our bespoke login protocol.

Here's some more detailed thoughts on what it might look like.

At a high level, we need the following functionality:

• The ability to serialize state for storage by the application, and to restore from serialized state.
• The ability for the application to request an OAuth access_token (and any corresponding keys) for a given set of scopes. This could fail for the following reasons:
  • The user hasn't signed in at all yet.
  • The user has signed in, but didn't grant that particular scope.
  • The user has signed in, but our tokens have expired or been revoked
    • This could happen due to explicit revokation, or password reset.
• The ability for the application, in response to one of the above errors, to launch an OAuth flow and provide back updated credentials.

We could also provide some additional helpers on top of this basic functionality, e.g. ability to fetch and cache profile data. But the above is the core OAuth functionality on which the rest would be based.

The FxA client's internal state would need to contain:

• The app's OAuth client_id, and OAuth issuer URL ("https://accounts.firefox.com" in production).
• Basic info about the user who has logged in, e.g. their uid.
• The list of scopes on that refresh_token
• The JWKs for any corresponding keys for the above scopes

A client attempting to access, say, sync data, would begin by loading the FxAClient from serialized state:

fxaClient = FxAClient::from_seralized_state(APP_CLIENT_ID, "....")

It could then ask for an access_token and keys with which to talk to the sync service:

token, keys = fxaClient.get_access_token_and_keys(scope=["https://identity.mozilla.com/apps/oldsync"])

Unfortunately the user hasn't signed in yet, so it will return an error to the app, and the app will need to prompt the user for authorization. It calls into the client to initiate an OAuth flow:

url = fxaClient.begin_oauth_flow(
    scope=["https://identity.mozilla.com/apps/oldsync"]
    redirect_uri=APP_REGISTERED_REDIRECT_URI
)

The client would set up some internal state for an in-progress OAuth flow, such as a keys_jwk and an OAuth state token, and would return a URL for the app to load for the user. When the OAuth dance completes, the app would capture the result from the redirect and pass it back to the client:

fxaClient.complete_oauth_flow(code="ABCD", state="XYZ")

The client would do all the tricky oauth bits, like validating state, exchanging the code for some OAuth tokens, decrypting the keys_jwe, etc. It would store the results into its internal state, so that when the client asks again for tokens to talk to sync:

token, keys = fxaClient.get_access_token_and_keys(scope=["https://identity.mozilla.com/apps/oldsync"])

It can return them successfully, possibly by using its refresh_token to mint a new access_token.

I think that's almost everything that would be required for a minimal client - the token and keys from the above can be passed downstream to a sync client library for actual fetching of sync data.

@eoger @vladikoff does that make sense? How well does the above map onto your "sandwich" model prototype?

Hardcoding it sounds like a bad idea.

Instead, the app should be expected to provide it, either as a FirefoxAccount constructor argument or in FxAConfig.

In custom implementations of the OAuth flow, implementers are able to specify an action (one of email, signup, signin, or force_auth) in the request parameters for the version of the OAuth login they would like to use. Ideally, the beginOAuthFlow() method signature could be extended to support the action desired by consumers of the library.

At the moment it's all or nothing. Ideally the all_records method in service.rs would be gone, and replaced with something that takes a timestamp (where you'd get all records by downloading the records since SERVER_EPOCH).

Currently we call the rust crate code synchronously, which is very problematic since we make network requests.
I don't really know the Swift conventions, but I assume a callback function has to be passed as the last argument of an asynchronous function.

A laundry list of concerns I have with the initial landing of the logins crate:

• verify that ServerPassword instances with missing usernames are accepted
• verify that multiple ServerPassword instances with overlapping credentials are not content resolved to the same credential
• verify that metadata changes are uploaded and correctly mirrored to the store

Minor enhancements:

• make sync15-adapter manage the sync transaction ID, so that it's not a member Option
• store the last-synced timestamp as an opaque blob rather than a floating point value
• implement wiping (excising?) the underlying Mentat store
• rather than :form/syncPassword, model the relationship correctly, with :sync.password/form

Major enhancements:

• capture the current device in each login usage
• support adding forms to the Mentat store, and correctly generate ServerPassword records for them

Based on our roadmap we need to create a simple prototype Android version of an app that imports the fxa-rust-client

We have an ios example already: https://github.com/mozilla/application-services/tree/master/sandvich-ios

@csadilek what was your plan for this? Did you still want to use JNI?

Right now, the initial landing branch doesn't have comprehensive test for the "passwords" module. This ticket tracks improving that, so that we have more confidence in the Sync 1.5-specific functionality.

We need to work out what delivery vehicle we use for lockbox. It seems ideal if we could do the same thing other android components do (ie, have the api in the android-components repo and built using the same approach as the fxa sandvich), but whatever we come up with it, needs to be documented (including documentation for updating it), agreed to by the lockbox team and a first version released.

We want to add capabilities for:

• Commands / Send Tab
• Display Name
• Push registration and notifications handling

This requires to implement the /device API, however we should learn from our Desktop implementation and try to come up with something that doesn't look like it's been "bolted on".

This issue is to discuss how do we shape that client API to look natural for users, and how do we track all that state inside the crate.

Requires #390

This had been superseded by #15

At the moment it's largely taken straight from desktop, but this seems error prone and less than ideal.

See #12 (comment) for some more context.

┆Issue is synchronized with this Jira Story

You can't really mix these libs (rust-lang-deprecated/error-chain#240), and we need to expose these to consumers (e.g. in logins-api or w/e)

There are some options in the openssl build that can minimize the size of the large lib and solve all of our problems with the lib size!

We need to investigate that 🔍

We haven't really documented how we build ssl, cjose, jansson etc which are all required to build fxa-rust-client.