orleanscontrib / orleans.cosmosdb Goto Github PK

With the case of wanting to store both the Application Data and the Orleans data within the same CosmosDB instance, where that CosmosDB instance is using the new "shared" throughput facilities, it is a requirement to set partition keys for each collection, according to the output [below].

Lifecycle start canceled due to errors at stage 20000
warn: Orleans.Runtime.Scheduler *stg/22/00000016.WorkItemGroup[101215]
      Task [Id=11, Status=RanToCompletion] in WorkGroup [SystemTarget: S10.1.1.18:11111:282256074*stg/22/00000016@S00000016] took elapsed time 0:00:01.1214201 for execution, which is longer than 00:00:00.2000000. Running on thread System.Threading.Thread
dbug: Orleans.Runtime.DeploymentLoadPublisher[0]
      UpdateRuntimeStatistics from S10.1.1.18:11111:282256074
   at Orleans.LifecycleSubject.OnStart(CancellationToken ct)
   at Orleans.Runtime.Scheduler.AsyncClosureWorkItem.Execute()
   at Orleans.Runtime.Silo.StartAsync(CancellationToken cancellationToken)
   at .....Silo.Program.StartSilo() in .....Silo\Program.cs:line 154
   at .....Silo.Program.Main(String[] args) in .....Silo\Program.cs:line 130
--------------------------------------------------------------------------------
Provisioned throughput collection should have a partition key
ActivityId: 44fa21ea-8e0a-43ad-a6e2-a5bf0d909d80, Microsoft.Azure.Documents.Common/2.2.0.0, Linux/9 documentdb-netcore-sdk/1.9.1
   at Microsoft.Azure.Documents.Client.ClientExtensions.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings)
   at Microsoft.Azure.Documents.GatewayStoreModel.<>c__DisplayClass20_0.<<InvokeAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.<>c__DisplayClass1_0.<<ExecuteAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetry(Func`1 callbackMethod, Func`3 callShouldRetry, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetry(Func`1 callbackMethod, Func`3 callShouldRetry, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteAsync(Func`1 callbackMethod, IRetryPolicy retryPolicy, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.GatewayStoreModel.InvokeAsync(DocumentServiceRequest request, ResourceType resourceType)
   at Microsoft.Azure.Documents.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request)
   at Microsoft.Azure.Documents.Client.DocumentClient.CreateAsync(DocumentServiceRequest request)
   at Microsoft.Azure.Documents.Client.DocumentClient.CreateDocumentCollectionPrivateAsync(String databaseLink, DocumentCollection documentCollection, RequestOptions options)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.<>c__DisplayClass1_0.<<ExecuteAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetry(Func`1 callbackMethod, Func`3 callShouldRetry, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetry(Func`1 callbackMethod, Func`3 callShouldRetry, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteAsync(Func`1 callbackMethod, IRetryPolicy retryPolicy, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.Client.DocumentClient.CreateDocumentCollectionIfNotExistsPrivateAsync(Uri databaseUri, DocumentCollection documentCollection, RequestOptions options)
   at Orleans.Reminders.CosmosDB.CosmosDBReminderTable.TryCreateCosmosDBResources()
   at Orleans.Reminders.CosmosDB.CosmosDBReminderTable.Init()
   at Orleans.OrleansTaskExtentions.WithTimeout(Task taskToComplete, TimeSpan timeout, String exceptionMessage)
   at Orleans.Runtime.ReminderService.LocalReminderService.Start()
   at Orleans.Runtime.Scheduler.AsyncClosureWorkItem.Execute()
   at Orleans.OrleansTaskExtentions.WithTimeout(Task taskToComplete, TimeSpan timeout, String exceptionMessage)
   at Orleans.Runtime.Silo.<OnActiveStart>g__StartReminderService|74_0()
   at Orleans.Runtime.Silo.StartAsyncTaskWithPerfAnalysis(String taskName, Func`1 task, Stopwatch stopWatch)
   at Orleans.Runtime.Silo.OnActiveStart(CancellationToken ct)
   at Orleans.Runtime.SiloLifecycleSubject.MonitoredObserver.OnStart(CancellationToken ct)
   at Orleans.LifecycleSubject.WrapExecution(CancellationToken ct, Func`2 action)
   at Orleans.LifecycleSubject.OnStart(CancellationToken ct)
warn: Orleans.Runtime.Silo[100220]

Trying to use v3.0.3 with Orleans 3.5.0 and seeing this exception:

{"Type":"System.InvalidOperationException","HResult":-2146233079,"Message":"A suitable constructor for type 'Orleans.Persistence.CosmosDB.CosmosDBGrainStorage' could not be located. Ensure the type is concrete and services are registered for all parameters of a public constructor.","Source":"Microsoft.Extensions.DependencyInjection.Abstractions"}

Partial call stack:

System.InvalidOperationException:
at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.CreateInstance (Microsoft.Extensions.DependencyInjection.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.CreateInstance (Microsoft.Extensions.DependencyInjection.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
at Orleans.Persistence.CosmosDB.CosmosDBGrainStorageFactory.Create (Orleans.Persistence.CosmosDB, Version=3.0.3.0, Culture=neutral, PublicKeyToken=null)
at Orleans.Runtime.KeyedSingletonService2+<>c__DisplayClass4_0.<.ctor>b__0 (Orleans.Core, Version=2.0.0.0, Culture=neutral, PublicKeyToken=null) at System.Lazy1.ViaFactory (System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Lazy1.ExecutionAndPublication (System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Lazy1.CreateValue (System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Lazy1.get_Value (System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Orleans.Runtime.KeyedSingletonService2.GetService (Orleans.Core, Version=2.0.0.0, Culture=neutral, PublicKeyToken=null)

I'm not sure if this means we're missing something in our configuration or if there's a mismatch between the CosmosDBGrainStorageFactory and the constructor of CosmosDBGrainStorage.

I get this error when starting Silos:
ArgumentException: ConsistencyLevel Strong specified in the request is invalid when service is configured with consistency level Session. Ensure the request consistency level is not stronger than the service consistency level.

Hi, I try to use Cosmos DB in Orleans 2.0, but it doesn't work.
Maybe you can provide samples how to I can config it?

I need to use JsonSerializerSettings to set up the serializer for Nodatime Instant and Duration. My issue is probably related to #6. Is custom serializer settings not possible at all?

I am referring to:

From my understanding of the code, the Orleans.Persistence.CosmosDB Storage Provider always does point reads (only using ID and Partition Key); so it uses CosmosDB as a KV Store and never queries using the indexed values.

We are trying to optimize our CosmosDB usage and it looks like for indexing CosmosDB charges you extra RU/s. I was wondering if we could just go ahead and turn off indexing and this library would still work correctly.

As far as I can see it would, but I'm wondering if there is any reason that I'm missing

Update CosmosDBMembershipTable to allow for using a managed identity instead of account key

It seems as if there are methods being used by Orleans.CosmosDB that are no longer existing within the 2.x version of Microsoft.Azure.DocumentDB.Core. We should probably patch the nuget package to prevent folks from using v2.0, v2.1, or v2.2 of Microsoft.Azure.DocumentDB.Core. As a subsequent patch, this library should go to v2.0.x and be upgraded to the v2.x Microsoft.Azure.DocumentDB.Core.

Is it possible to store grains in a separate collection rather than a single one called OrleansStorage?

Current Orleans provider's configuration APIs are not DI-friendly. I made a work around to have this provider to support DI at some extent but it turns out that doing that, I'm only allowed to have 1 persistence provider registered.

Check with Orleans team when/if we will have:

1. DI-friendly provider APIs
2. Named options support

Implement proper logging and exception handling for the DocumentException exceptions.

As part of our scaling efforts we are looking at our CosmosDB utilization and costs. We noticed the procedure calling JSON.parse and wanted to see if we could send the extracted values (graintype, id and etag) as parameters instead of parsing it, and how this would affect the request charge (RC). Short version it doesn't.

We did however find the RU cost somewhat high compared to calling Cosmos operations directly. Based on the document below[1], here's our findings

Average RC read using stored procedure: 5.63
Average RC create using stored procedure: 11.68
Average RC update using stored procedure: 14.89

RC read using GET: 1
RC create using PUT: 6.29
RC update using POST: 10.29

This seems rather excessive, and we're wondering if one can improve the stored procedures or skip it and call Cosmos db directly.

{
  "id": "b780de47-a7d2-44c5-8a8f-7eea29e9561f__GrainReference=4f3d549833293c5e5d419486ec1f04a30600000031c1478e+7054fbd0-9dce-48ed-ab5c-4ed85ba0c66c",
  "GrainType": "Ps.Personlig.Grains.FavorittGrainRadio",
  "State": {
    "State": {
      "Case": "V1",
      "Fields": [
        {
          "FavoriserbarId": "K|saann_er_du",
          "Medium": "radio",
          "KildeType": "manuell",
          "Registrert": 15481904238110000,
          "Slettet": {
            "Case": "Some",
            "Fields": [
              15518746264096320
            ]
          }
        }
      ]
    }
  },
  "PartitionKey": "testh"
}

As this provider is based on JavaScript SProcs, it would be nice to have them deployed and (optionally) updated whenever a new version is published.

One way to do it, is to have a separated entity within the same collection of EntityType == Sproc where we would keep a reference to the current sprocs installed, and which version to use. That table would be read at startup and (optionally) if the version deployed doesn't match the one with the nuget package, it should update them.

This feature is optional as it would require the credentials to have access at the collection to create sprocs and sometimes it isn't desirable.

After a bit of load something goes wrong in the CosmosDB driver, we're getting a lot of TransportExpcetion. The TransportException type was introduced in Microsoft.Azure.DocumentDB.Core 2.2.0, but we did not see this exception before upgrading to 2.2.2. Version 2.2.1 works fine.

We have created our own package built with Microsoft.Azure.DocumentDB.Core 2.2.1 to get around this problem. We've had an initial chat with our MSFT contact and are trying to get the CosmosDB team involved.

We're interested if anyone else experience this problem.

Currently only JSON serialization is supported but Orleans allows configuring any other serializer - can this be updated to use other configured serializers?

To reduce request costs, explicitly set the idexing to only properties being queried by the providers.

Implement retry policy according to the recommendations here: https://blogs.msdn.microsoft.com/bigdatasupport/2015/09/02/dealing-with-requestratetoolarge-errors-in-azure-documentdb-and-testing-performance/

In a fresh .NETCore 3.1 project I'm trying to build an Orleans silo using the new generic host builder.
Unfortunately I keep running into the following error:

System.InvalidOperationException: 'Cannot resolve scoped service 'Microsoft.Extensions.Options.IOptionsSnapshot`1[Orleans.Persistence.CosmosDB.CosmosDBStorageOptions]' from root provider.'

UseCosmosDBMembership and UseCosmosDBReminderService work perfectly. As soon as I add GrainStorage(AsDefault) the error occurs.

As the DI cannot resolve CosmosDBStorageOptions I suspect I'm missing a package or reference.

Besides a few Serilog and Swashbuckle nuget-packages the following Orleans packages are installed:

• Microsoft.Orleans.Server v3.0.2
• Orleans.Clustering.CosmosDB v3.0.0
• Orleans.Persistence.CosmosDB v3.0.0
• Orleams.Reminders.CosmosDB v3.0.0

My HostBuilder looks as follows:

var Host = new HostBuilder()
    .UseContentRoot(Directory.GetCurrentDirectory())

    .UseDefaultServiceProvider((Context, Options) =>
    {
        var IsDevelopment = Context.HostingEnvironment.IsDevelopment();
        Options.ValidateScopes = IsDevelopment;
        Options.ValidateOnBuild = IsDevelopment;
    })
    .UseSerilog()
    .UseOrleans(SiloBuilder =>
    {
        _ = SiloBuilder.Configure<ClusterOptions>(ClusterOptions =>
        {
            ClusterOptions.ClusterId = "SomeId";
            ClusterOptions.ServiceId = "SomeId"; 
        });
        
        _ = SiloBuilder.ConfigureEndpoints(siloPort: x, gatewayPort: x);

        // Silo clustering persistance provider.
        _ = SiloBuilder.UseCosmosDBMembership(Options =>
        {
            // Configuration
        });

        // Default grain persistance storage provider.
        _ = SiloBuilder.AddCosmosDBGrainStorageAsDefault(Options =>
        {
            // Configuration
        });

        // Reminder persistance storage provider.
        _ = SiloBuilder.UseCosmosDBReminderService(Options =>
        {
            // Configuration
        });
    })
    .ConfigureWebHostDefaults(WebHostBuilder =>  
        _ = WebHostBuilder.UseStartup<HostStartup>()                        
        .UseUrls("SomeUrl"))
    .Build();

Any clue as to what I'm doing wrong? Or am I facing a compatibility bug with the newer Orleans versions?

Kind regards,
Jan van Linge

Current version is no longer compatible with latest Microsoft Orleans 7.0

Looks like this PR might be related, although not sure. Using package version 3.0.3

#56

I am trying to use a combo of Orleans, Orleans transactions, CosmosDB and polymorphism in my application.

So far I came up to the conclusion that the only way to make polymorphism work is to pass CosmosClient with a custom CosmosSerializer (just copied native implementation with proper TypeHandling) into the storage provider during the initialization. All the other attempts failed as JsonSerializerSettings and TypeHandling values of CosmosDBStorageOptions are ignored during the WriteStateAsync.

However, this custom CosmosClient has one drawback (or maybe a feature). During ReadStateAsync, the value of the State field has already the correct type (in case of Orleans transactions, the state is of TransactionalStateRecord type) when returned from CosmosClient whereas, without the custom implementation, the type of the State is JsonObject. This fact is causing an error on this line as .ToString() returns just a type name on TransactionalStateRecord, not a valid JSON.

Fixing is possible by checking if the type is JsonObject (do the current logic) or something else (set the state directly without deserialization) which I can do. However, I don't think this is the most beautiful solution in the world, haven't you already investigated a similar issue? Could be there some other pitfalls?

Just downloaded the project and tried to run tests and none of the persistence test succeeded.
Same error for each test:
Message:
System.ArgumentException : GenericArguments[0], 'Orleans.Persistence.CosmosDB.CosmosDBGrainStorage', on 'Microsoft.Extensions.Options.OptionsMonitor1[TOptions]' violates the constraint of type 'TOptions'. ---- System.TypeLoadException : GenericArguments[0], 'Orleans.Persistence.CosmosDB.CosmosDBGrainStorage', on 'Microsoft.Extensions.Options.OptionsMonitor1[TOptions]' violates the constraint of type parameter 'TOptions'.
Stack Trace:
RuntimeType.ValidateGenericArguments(MemberInfo definition, RuntimeType[] genericArguments, Exception e)
RuntimeType.MakeGenericType(Type[] instantiation)
CallSiteFactory.TryCreateOpenGeneric(ServiceDescriptor descriptor, Type serviceType, CallSiteChain callSiteChain, Int32 slot)
CallSiteFactory.TryCreateOpenGeneric(Type serviceType, CallSiteChain callSiteChain)
CallSiteFactory.CreateCallSite(Type serviceType, CallSiteChain callSiteChain)
<>c__DisplayClass7_0.b__0(Type type)
ConcurrentDictionary2.GetOrAdd(TKey key, Func2 valueFactory)
CallSiteFactory.GetCallSite(Type serviceType, CallSiteChain callSiteChain)
ServiceProviderEngine.CreateServiceAccessor(Type serviceType)
ConcurrentDictionary2.GetOrAdd(TKey key, Func2 valueFactory)
ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
ServiceProviderEngineScope.GetService(Type serviceType)
ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
CosmosDBGrainStorageFactory.Create(IServiceProvider services, String name) line 467
<.ctor>b__0()
Lazy1.ViaFactory(LazyThreadSafetyMode mode) Lazy1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
Lazy1.CreateValue() Lazy1.get_Value()
KeyedSingletonService2.GetService(IServiceProvider services) KeyedServiceCollection2.GetService(IServiceProvider services, TKey key)
KeyedServiceCollectionExtensions.GetServiceByKey[TKey,TService](IServiceProvider services, TKey key)
KeyedServiceCollectionExtensions.GetRequiredServiceByKey[TKey,TService](IServiceProvider services, TKey key)
KeyedServiceCollectionExtensions.GetRequiredServiceByName[TService](IServiceProvider services, String name)
<>c.b__27_2(IServiceProvider s, String n) line 289
<.ctor>b__0()
Lazy1.ViaFactory(LazyThreadSafetyMode mode) Lazy1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
Lazy1.CreateValue() Lazy1.get_Value()
KeyedSingletonService2.GetService(IServiceProvider services) <.ctor>b__64_1(IKeyedService2 s)
SelectArrayIterator2.MoveNext() Silo.ctor(ILocalSiloDetails siloDetails, IServiceProvider services) RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions) RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context) CallSiteVisitor2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
CallSiteVisitor2.VisitCallSite(ServiceCallSite callSite, TArgument argument) CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context) CallSiteVisitor2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
CallSiteVisitor2.VisitCallSite(ServiceCallSite callSite, TArgument argument) CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context) CallSiteVisitor2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
CallSiteVisitor2.VisitCallSite(ServiceCallSite callSite, TArgument argument) CallSiteRuntimeResolver.VisitIEnumerable(IEnumerableCallSite enumerableCallSite, RuntimeResolverContext context) CallSiteVisitor2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
<>c__DisplayClass1_0.b__0(ServiceProviderEngineScope scope)
ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
ServiceProviderEngineScope.GetService(Type serviceType)
ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
Host.StartAsync(CancellationToken cancellationToken)
----- Inner Stack Trace -----
RuntimeTypeHandle.Instantiate(Type[] inst)
RuntimeType.MakeGenericType(Type[] instantiation)

As Orleans don't have a black box unit test kit for providers, I tried to copy the tests from main Orleans repo to this one in order to make some basic testing.

It turns out that it will be way much complicated than I expected to make it work.

Need to get some tests working as the current ones are getting complicated to.

Currently when creating resources, serverless cosmos will return a 400 error

Membership failed to start: Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: 400 Substatus: 0 Reason: (Microsoft.Azure.Documents.DocumentClientException: Setting offer throughput or autopilot on container is not supported for serverless accounts.ActivityId: 4360c6e4-0533-441c-936a-97591ad4ed95, Microsoft.Azure.Documents.Common/2.11.0, {"RequestStartTimeUtc":"2020-11-03T16:21:20.2286515Z","RequestEndTimeUtc":"2020-11-03T16:21:20.2731706Z","RequestLatency":"00:00:00.0445191","IsCpuOverloaded":false,"NumberRegionsAttempted":1,"ResponseStatisticsList":[],"AddressResolutionStatistics":[{"StartTime":"2020-11-03T16:21:20.2287812Z","EndTime":"2020-11-03T16:21:20.2731706Z","TargetEndpoint":"https://sharedc2c-southcentralus.documents.azure.com/dbs/orleans-local/colls"}],"SupplementalResponseStatistics":[],"FailedReplicas":[],"RegionsContacted":[],"ContactedReplicas":[]}, Windows/10.0.19042 cosmos-netstandard-sdk/3.4.2 at Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings, DocumentServiceRequest request) at Microsoft.Azure.Cosmos.GatewayStoreClient.InvokeAsync(DocumentServiceRequest request, ResourceType resourceType, Uri physicalAddress, CancellationToken cancellationToken) at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken) at Microsoft.Azure.Cosmos.Handlers.TransportHandler.SendAsync(RequestMessage request,

Whenever the GrainStateEntity.State size is bigger than the allowed document/field size, the data should be chunked in more documents.

Make CircleCI build to play the ball with Azure CosmosDB emulator so it can run tests in CI.

Since the State payload is serialized as object, it goes just fine inside the StateEntity in CosmosDB as you may see in the following sample:

{
    "id": "00000000-0000-0000-0000-000000000000_GrainReference=00000000000000000000000000000000030000004e9c6618",
    "GrainType": "Orleans.CosmosDB.Tests.Grains.TestGrain",
    "State": {
        "Data": [
            "Test 0",
            "Test 0",
            "Test 0"
        ]
    },
    "_etag": "\"0000db02-0000-0000-0000-5a66a9550000\"",
    "_rid": "2UIfAKJEnwACAAAAAAAAAA==",
    "_self": "dbs/2UIfAA==/colls/2UIfAKJEnwA=/docs/2UIfAKJEnwACAAAAAAAAAA==/",
    "_attachments": "attachments/",
    "_ts": 1516677461
}

However, when we deserialize the State field back, it comes as a JObject because Json.Net doesn't know the proper type to deserialize to. That can be fixed if we use a custom JsonSerializerSettings object when initializing the DocumentClient and tell it to include the Type to the object serialized. I tried that, and when sending back the document to the database with that settings applied, the sproc parameters are all undefined.

To workaround the deserialization issue, I introduced the workaround here.

Just logging this issue so we can investigate how to properly use the JsonSerializerSettings and avoid the call to JObject.ToObject<T>() when deserializing the state.

If I understand correctly, the current implementation is partitioning by Grain Type. I do not think this is optimal for most scenarios.

1. A partition key must be relatively uniformly distributed, to better balance load when storage does not fit in a single partition. Some GrainTypes might have 10 grains while others millions.
   
2. A partition key defines the scope of transactions. When your entities share the same transaction key you can do a server-side javascript SProcs. While I understand that a transaction across different grains of the same type could be occasionally useful (build a relation between two People for example), it would be good to allow the liberty to decide on different scopes.
   
   As such, I think we should be able to mark a property of a GrainState class as the partition Key. This can also enable some advanced scenarios. Example:
   
   I have a collection of People. There I store 2 types of entities: person and personChanges (an audit trail of JSON patch diffs, modifications on the person that have or have not yet been approved). Both share the same "personId" as partitionKey.
   
   Let's imagine a user is working on some potential change on a person (it could span days before it's validated, it could be for example a person's divorce). This goes to a PersonChangeGrain (which saves its state nice and easily), until a specific operation is called "ValidateChange". In this particular case the code instead of calling WriteState, it would call the PersonGrain, which would apply the change on its state, and then call a javascript Sproc which (thanks to the fact that both share the same partition key) can in a transactional way 1) save the new PersonState and 2) mark the PersonChange (another entity) as applied. When the operation completes, the Person can request the original PersonChangeGrain to Refresh its State. This last operation might fail in a non transactional way, but it doesn't matter, as the next time the PersonChangeGrain will be reactivated, it will read the correct updated state from storage.
   
   If you're interested, I could attempt to propose code for declaring which property to use as key (by grain state class), even though I'm a newbie in Orleans (but less so in CosmosDB).

Update CosmosDBMembershipTable to allow for using a managed identity instead of account key