For example, setting partition key in RequestOptions on create container, instead of defintion. This occured in a user test.

Problems

Setting the throughput via the options setting is kinda weird on create.

When wanting to update to scale out, you have to look up the offer, which isn't very obvious.

Suggested improvement

1. Include RUs in the container & db definition

   • Have to detect that property and update request options appropriately on create
   • Have to detect that property and make a subsequent request to /offers/ api to update the offer
   • Have to change read def/etc. APIs to grab the offer info

2. Add readOffer & updateOffer method on container & db reference objects; get rid of top level properties

   • Very explicit on how to update, no need to accidentally touch other def properties
   • Leaks the offer concept
   • Simpler than option 1
   • Doesn't solve the request options issue

3. Possible twist on 2 - have an overload of create that takes an object that is just the RUs to include

   • makes it more obvious that giant request options object (via intellisense)

We're filtering the docs out in tsconfig, which strips all the docs content. That isn't intentional. :)

Right now, we abuse the headers property bag to track metrics for things like RUs across many operations.

We should reconsider how we're exposing headers/etc.

The current client has the ability to set a custom partition resolver functions at client.partitionResolver usually with an instance of HashPartitionResolver. We should investigate:

• What use cases need this functionality
• Are any existing users are relying on it
• How would we expose this in the new Object Model

Goals

For each of the objects we operate on (database, collection, document, etc.), we want to move their operations off of DocumentClient and onto objects which represent those operations. We also want to rename those objects to better capture their functionality in today's Cosmos terms (as opposed to legacy Document DB terms). We want the SDK to be easy to use & grok (understand).

Nomenclature updates

Surface Area updates

A few notable changes:

• We'll be moving the operations for a given object to the parent of that object. (aka container will have operations for readItem on it).
• We'll have a "fluent" style API which will replace the current URI based model of referencing objects. (aka await client.getDatabase("foo").getContainer("bar").readItem("baz"))
• "builder" operations (basically get operations) will be sync and not talk to server
• QCRRUD operations will be async (and will talk to the server)

We take some inspiration from REST models, but have a goal of being easy to understand/code versus being technically pure.

Examples

Query Items

const {result: items} = await client
    .getDatabase(databaseName)
    .getContainer(containerName)
    .queryItems("select * from C")
    .toArray();

Note: Might want to support queryItems (for interfaces) to allow users to get help on the interface of items.

Pass a container object around

Register collection with DI framework

ioc.register("todoCollection", client.getDatabase(databaseName).getContainer(containerName));

Consume container in route logic

public async getTodos(@Injectable("todoContainer") Container todoContainer) {
    const {result: todos} = await todoContainer.readItems();
    return todos;
}

Objects

Heirarchy

• A given object can have many children

  • "A database has collections" (1:N)

• A given object only has 1 parent

  • "A collection has a database" (1:1)

• client

  • database account
  • offer
  • databases
    • database
    • users
      • user
        • permissions
          • permission
    • containers
      -container
      - items
      - item
      - attachments
      - attachment
      - triggers
      - trigger
      - user defined functions
      - user defined function
      - stored procedures
      - stored procedure

Overall pattern will be:

const parent // any given parent object
parent.children.query // query many
parent.children.read //read many
parent.child("foo").read //read one
parent.child("foo").replace // replace one
parent.child.parent

CosmosClient

• #.getDatabaseAccount(options?: RequestOptions) => Promise<DatabaseAccount>
• #.databases: Databases
• #.offers: Offers

DatabaseAccount

???

Databases

• #.getDatabase(id: string) => Database
• #.query(query: string | SqlQuerySpec, options?: FeedOptions) => QueryIterator<DatabaseDefinition>
• #.create(body: object, options?: RequestOptions) => Promise<Response<DatabaseDefinition>>
• #.read(options?: FeedOptions) => QueryIteratory<DatabaseDefinition>
  • TBD: Worried about confusion with getDatabase

Database

• #.id
• #.containers: Containers
• #.read(options?: RequestOptions) => Promise<Response<DatabaseDefinition>>
• #.replace(options?: RequestOptions) => Promise<Response<DatabaseDefinition>>
• #.delete(options?: RequestOptions) => Promise<Response<DatabaseDefinition>>

Containers

• #.getContainer(id: string) => Container
• #.query(query: string | SqlQuerySpec, options?: FeedOptions) => QueryIterator<ContainerDefinition>
• #.create(body: object, options?: RequestOptions) => Promise<Response<ContainerDefinition>>
• #.read(options?: FeedOptions) => QueryIterator<ContainerDefinition>

Container

• #.id
• #.database: Database
• #.items: Items
• #.read(name: string, options?: RequestOptions) => Promise<Response<ContainerDefinition>>
• #.replace(name: string, body: ContainerDefinition, options?: RequestOptions) => Promise<Response<ContainerDefinition>>
• #.delete(name: string, options?: RequestOptions) => Promise<Response<ContainerDefinition>>

Items

• #.getItem(id: string, pk?: string) => Item
• #.query(query: string | SqlQuerySpec, options?: FeedOptions) => QueryIterator<?>
• #.read(options?: FeedOptions) => QueryIterator<?>
• #.create<T>(body: T, options?: RequestOptions) => Promise<Response<T>>
• #.upsert<T>(body: T, options?: RequestOptions) => Promise<Response<T>>

Item

• #.id
• #.primaryKey
• #.container: Container
• #.readItem<T>(id: string, options?: RequestOptions) => Promise<Response<T>>
• #.replaceItem<T>(id: string, body: T, options?: RequestOptions) => Promise<Response<T>>
  - Do we need this id? How do we sniff out if we're using a partition resolver?
• #.deleteItem<T>(id: string, options?: ResquestOptions) => Promise<Response<T>>

StoredProcedures

• #.getStoredProcedure(id: string) => StoredProcedure
• #.query(query: string | SqlQuerySpec, options?: FeedOptions) => QueryIterator<StoredProcedureDefinition>
• #.read(options?: FeedOptions) => QueryIterator<StoredProcedureDefinition>
• #.create(body: StoredProcedureDefinition, options?: RequestOptions) => Promise<Response<StoredProcedureDefinition>>
• #.upsert(body: StoredProcedureDefinition, options?: RequestOptions) => Promise<Response<StoredProcedureDefinition>>

StoredProcedure

• #.id
• #.container: Container
• #.read(options?: RequestOptions) => Promise<Response<StoredProcedureDefinition>>
• #.replace(body: StoredProcedureDefinition, options?: RequestOptions) => Promise<Response<StoredProcedureDefinition>>
• #.delete(options?: RequestOptions) => Promise<Response<StoredProcedureDefinition>>
• #.execute<T>(params?: any[], options?: RequestOptions) => Promise<Response<T>>

Triggers

• #.getTrigger(id: string) => Trigger
• #.query(query: string | SqlQuerySpec, options?: FeedOptions) => QueryIterator<TriggerDefinition>
• #.read(options?: FeedOptions) => QueryIterator<TriggerDefinition>
• #.create(body: TriggerDefinition, options?: RequestOptions) => Promise<Response<TriggerDefinition>>
• #.upsert(body: TriggerDefinition, options?: RequestOptions) => Promise<Response<TriggerDefinition>>

Trigger

• #.id
• #.container: Container
• #.read(options?: RequestOptions) => Promise<Response<TriggerDefinition>>
• #.replace(body: TriggerDefinition, options?: RequestOptions) => Promise<Response<TriggerDefinition>>
• #.delete(options?: RequestOptions) => Promise<Response<TriggerDefinition>>

UserDefinedFunctions

• #.getUserDefinedFunction(id: string) => UserDefinedFunction
• #.query(query: string | SqlQuerySpec, options?: FeedOptions) => QueryIterator<UserDefinedFunctionDefinition>
• #.read(options?: FeedOptions) => QueryIterator<UserDefinedFunctionDefinition>
• #.create(body: UserDefinedFunctionDefinition, options?: RequestOptions) => Promise<Response<UserDefinedFunctionDefinition>>
• #.upsert(body: UserDefinedFunctionDefinition, options?: RequestOptions) => Promise<Response<UserDefinedFunctionDefinition>>

UserDefinedFunction

• #.id
• #.container: Container
• #.read(id: string, options?: RequestOptions) => Promise<Response<UserDefinedFunctionDefinition>>
• #.replace(id: string, body: UserDefinedFunctionDefinition, options?: RequestOptions) => Promise<Response<UserDefinedFunctionDefinition>>
• #.delete(id: string, options?: RequestOptions) => Promise<Response<UserDefinedFunctionDefinition>>

Offers

• #.getOffer(id: string) => Offer
• #.queryOffers(query: string | SqlQuerySpec, options?: FeedOptions) => QueryIterator<OfferDefinition>
• #.readOffers(options?: FeedOptions) => QueryIterator<OfferDefinition>

Offer

• #.id
• #.readOffer(id: string, options?: RequestOptions) => Promise<Response<OfferDefinition>>
• #.replaceOffer(body: OfferDefinition, options?: RequestOptions) => Promise<Response<OfferDefinition>>

User

• #.id
• ???

Open Questions

• Most of the API is top down, but can we put bottom up calls where it makes sense?

  For instance, client.readDatabase("foo") instead being client.database("foo").read()

• What are we doing with Users?

• Are we going to continue to support partition resolver?

Change Log

• 2018-06-13: Modified the API structure to isolate operations to their representative objects

We should instrument the client for better supportability.

Types of instrumentation:

• Tracing (i.e. operations & http requests - things that take time)

• Events (i.e. retries - actions and events the SDK encounters)

• Meterics (i.e. query metrics, operation duration, retry counts, etc, - numerical representations of activities and events in the SDK)

• How to expose instrumetnation?

• What instrumentation to expose?

Scenario:

When a Node.js application has several backend services its talking to, or is running in a resource constrained environment, it is necessary to tweak the behavior of any outgoing HTTP calls. Being able to provide a custom https.Agent is a common practice for Node.js SDK and allows users to use a common set of constraints for their http based clients.

Implementation details:

Need to provide for an https.Agent being provided in a options property bag of the constructor.

Known workarounds:

This is currently a public object on the client, so it can be modified after creation.

Problem

The difference in experience between the two types of containers is fairly large. When you're using partitions, you need to almost always think about partition key or setting some Request/Feed Option.

Users shouldn't need to know the ins and outs of our SDK's options just to use partitions, especially since partitions are the only way to scale.

Proposed solution

Split the experiences, so that we have Containers always expect a partition key (so we can enforce this at creation).

// Need to do more work to define what the APIs would look like/the difference in behaviors.

Do we want to maintain v1 backwards compat, or should we make a clean break?

There are some query metrics missing from the Node.js SDK (as compared to .NET)

TBD:

• What types of metrics are missing?

We have samples in our readme/etc. We should make sure that code gets tested automatically somehow.

In some cases, there is an error based on a malformed header or URL. We should include the requestOptions in the error body to make it easier to debug these cases.

We can be a bit more aggressive on retries for 503s for idempotent operations.

Notes:

1. We should simplify and standardize error handling irrespective of connection mode and SDK language. With that we should not be exposing E_CONN* to customers.
2. PUT/DELETE are not idempotent operation.
3. Upsert without ETAG is idempotent operation.
4. All read operations are idempotent.
5. 500 – We should not recommend retry. Most likely it is not retriable error. We should also ensure we don’t return 500 for transient issue, instead should use 408/503 across the stack.

Only some of the tests use the helper methods in src/test/common/TestHelpers.ts. If we refactor all the tests to use them it will be easier to parallelize test runs.

Running const { container } = await db.containers.createIfNotExists(collectionDefinition); fails and doesnt drop into the catch block, when the container already exists.

See example below.

const cosmos = require('@azure/cosmos');
const CosmosClient = cosmos.CosmosClient;
const endpoint = 'your url';
const masterKey =  'your key';
const client = new CosmosClient({ endpoint, auth: { masterKey } });

const databaseDefinition = { id: 'sample database' };
const collectionDefinition = { id: 'sample collection' };
const documentDefinition = { id: 'hello world doc', content: 'Hello World!' };

async function helloCosmos() {
  const { database: db } = await client.databases.createIfNotExists(databaseDefinition);

  // This line fails, but doesn;t drop into the catch block either.
  const { container } = await db.containers.createIfNotExists(collectionDefinition);
  console.log('created collection ${container}');

  const { body } = await container.items.create(documentDefinition);
  console.log(`Created item with content: `, body.content);

  await db.delete();
  console.log(`Deleted database ${db}`);
}

helloCosmos().catch(err => {
  console.error(err);
});

cc @christopheranderson

We've gotten some feedback from user studies that .get is confusing.

Let's track in this issue our ideas

Good morning,

As requested by @southpolesteve , I'm opening the issue here rather than the other repository.

I'm trying to access cosmosdb from a react/typescript application. I've imported the library and trying to just read a database. I'm using the v2 @azure/cosmos npm rather than documentdb npm. I was getting CORS issues with documentdb so trying to see if v2 fixes that.

These are the exceptions showing in the client.

ERROR in [at-loader] ./node_modules/@azure/cosmos/lib/queryIterator.d.ts:68:25
TS2304: Cannot find name 'AsyncIterable'.
ERROR in [at-loader] ./node_modules/@azure/cosmos/lib/request/request.d.ts:17:55
TS2694: Namespace '"url"' has no exported member 'UrlWithStringQuery'.
ERROR in [at-loader] ./node_modules/@azure/cosmos/lib/retry/retryUtility.d.ts:44:17
TS2694: Namespace '"url"' has no exported member 'UrlWithStringQuery'.
ERROR in [at-loader] ./node_modules/@azure/cosmos/lib/retry/sessionReadRetryPolicy.d.ts:32:70
TS2694: Namespace '"url"' has no exported member 'UrlWithStringQuery'.

Any ideas how I fix these.

Lots of folks don't think to look under auth, when we're doing user studies.

We can support the property at both the top level and in the auth object, then default to the auth object.

We need to have 100% (type)docs coverage for us to ship our first preview.

This is a high level tracking item for tracking breaking changes in v2

Proposed:

• Extensibility
• Roll packages forward
• options property bag for greater control and configuration

Planned:

Completed:

• "Fluent" API - each object (aka Database/Collection/Document) gets its object, instead of a massive client object
• Remove all "documentClient" mentions in favor of cosmosClient

Seems to me like a good thing https://palantir.github.io/tslint/rules/no-unnecessary-class/

This is a meta issue to track other work and questions around browser compatibility. One of our goals is to make the SDK useable in the browser and we still have some work to do. The lib currently webpack'd as part of the build step but not much has been done to verify it is usable in browsers.

Current open questions:

• Does it actually work in all major browsers?
• Can we get tests running in the browser and part of CI?
• What is the current best practice around shipping npm packages for both node and browsers?
• Can our lib be easily consumed by downstream webpack or rollup bundles?
• Can we make our bundled size reasonable? Current webpack -p: 3.17 MiB 😱

Coming out of #39 we decided to move go with approach 5. get methods will become singular methods one level up the object hierarchy:

// Current
client.databases.get(id)
client.databases.get(id).containers.get(id)
client.databases.get(id).containers.get(id).items.get(id)

// New
client.database(id)
client.database(id).container(id)
client.database(id).container(id).item(id)

Consider a simple case of reading and updating an item:

const item = await cosmos.item(id).read()

item.foo = "bar"

await cosmos.items.upsert(item)

While this API works for basic operations, it does not expose HTTP headers or any other raw request data. Many advanced uses of cosmos need this data. We currently solve this by returning a wrapper object for all methods with underlying network requests:

const { body: item, headers }  = await cosmos.item(id).read()

item.foo = "bar"

await cosmos.items.upsert(item)

Unfortunately, this complicates the returned object and API for simple usage. Ideally we could support both use cases. Return objects could be simple but also provide metadata that advanced users need. With newer JavaScript features this may be possible. These are some approaches we are considering.

1: Use Symbol

Cosmos SDK can use a symbol to store data directly on the returned database object.

// Cosmos SDK Code
cosmos.headers = Symbol('headers')

// User code
const item = await cosmos.item(id).read()
const headers = item[cosmos.headers]

#34 contains some previous discussion of this approach.

Pros

• Getting headers looks just like normal object property access
• Symbols as keys allows us to add metadata without conflicting with user data. These keys are omitted when iterated or stringified so users can safely pass them back to cosmos without polluting data.

Cons

• Requires advanced users to be familiar with Symbol which is a relatively new JS language feature
• Requires polyfill in IE

2: getHeaders method

This is a twist on proposal 1. Symbols are still used as keys, but we expose a utility method so symbols are not exposed to user code.

// Cosmos SDK Code
const headersKey = Symbol('headers')
comsos.getHeaders = obj => obj[headersKey]

// User code
const item = await cosmos.item(id).read()
const headers = cosmos.getHeaders(item)

Pros

• Similar to apporach 1 but without exposing users to Symbol

Cons

• Requires polyfill in IE
• getHeaders may look like magic to some users

3: Extend Promise

This approach returns a special class of promise that would allow users to control the return type.

// Cosmos SDK code
class CosmosRequestPromise extends Promise {
	includeHeaders() { /* implementation ommited */ }
}

// User code
const item = await cosmos.item(id).read()

const { body: item, headers } = await cosmos.item(id).read().includeHeaders()

Pros

• Doesn't use Symbol at all

Cons

• Requires creating and maintaining our own promise type
• includeHeaders may look like magic to some users
• Implementation details are still unclear. This approach seems possible, but we haven't built a POC.

According to the API reference "hasMoreResults" is abolished.
However, there is no sample of alternative, so it is difficult to understand how to change it.
In addition, "hasMoreResults" is still used for many tests.

I think it would be nice to have samples on how to change the following code.

while (queryIterator.hasMoreResults()) {
    let { result: items } = await queryIterator.executeNext();
    console.log(items);
}

We are trying to deprecate the Sql mode for Query Compatibility (we only want QueryJson). Should we remove this in v2?

/cc @kirankumarkolli @kirillg

The current linting config does not check for this

Right now you have to know how to set the options on items.readAll() in order to get change feed type support. We should support the ChangeFeedQuery type of format that .NET has (https://docs.microsoft.com/en-us/azure/cosmos-db/change-feed#using-the-sdk)

We should also evaluate creating the equivalent of the ChangeFeedProcessorHost that .NET has to make the experience even easier.

Related issues: #265

We don't need PartitionKind enum (it only has 1 property). C# doesn't expose this, so we're safe.

Problem:
.get is confusing, but we need a way to get a reference to a given resource. We decided on a combo approach of renaming .get and adding the reference to the response of async calls.
From #39:

4. Return reference object in response from async operations.
   - Example: const { resource: db} = client.databases.create({id: foo});
   - Pros: Helps reduce number of lines of code for many scenarios (like db and container create)
   - Cons: Doesn't remove need for .get because we don't want to force an async operation just to get a reference. Won't work with Steve's idea for simplifying the response types.

Additionally, we'll rename result to body to

Implementation details:

interface CosmosResponse<T, U> {
    body: T;
    headers: Headers;
    ref: U;
}

interface DatabaseResponse extends CosmosResponse<DatabaseDefinition, Database> {
    body: DatabaseDefinition;
    headers: Headers;
    ref: Database;
    database: Database;
}

/* Similar pattern for other types */

A couple user feedback sessions have shown that returning response wrapper objects from function calls is a bit confusing. Most users (especially new ones) will not need access to raw response headers or other metadata. However, advanced users do which is why we return these objects. I think we can support both users needs by storing response metadata using Symbols. These will allow us to safely store metadata on an object without conflicting with user keys.

// ====  Current ====
function createItem(body: any, options?: RequestOptions): Promise<Response<T>>;
function updateItem(body: any, options?: RequestOptions): Promise<Response<T>>;

const { result: item, headers } = createItem({})
item.name = 'foo'
updateItem(item)

// ====  Using Symbols ====
// Probably setup once and exported from the client
const headersSymbol = Symbol('CosmosDBHeaders');

interface Response {
    [headersSymbol]: RequestHeaders;
};

function createItem(body: any, options?: RequestOptions): Promise<T & Response>;
function updateItem(body: any, options?: RequestOptions): Promise<T & Response>;

const item = createItem({})
const headers = item[headersSymbol]
// OR
const { ...item, [sym]: headers } = createItem({})
// OR Can also be encapsulated on a client method if we don't want to expose Symbol magic
const headers  = client.getHeaders(item)

item.name = 'foo'
updateItem(item)

We could support an overload where callbacks are supported for forEach, this would help with issues where certain versions of Node doesn't have the AsyncIterator symbol yet.

See #71

Some user feedback during testing that item() or items would have create.

Given that we have a great experience for development in TypeScript, I propose we move our existing samples to TypeScript.

We probably need 1 JS sample around. I propose we write a new one which does a little bit of everything.

Right now, id is readonly, so it can't be updated. If you call replace on an object that allows for mutable ids, and you replace the id, it will effectively break that object (all methods will return 404). You have to get a new object from the factory with the new id.

This is probably the least worst option for this.

Alternatives could be:

• Sniffing the id on replace and updating the id (and change id to private set)
• Make id public and let users change it themselves so avoid going back to the factory (yolo public properties)

Gotten some feedback that object destructuring isn't super common, so we should show some samples with the response object, not just object destructuring.

David needs a simple test he can run which creates a document (gateway and direct mode, but we only support gateway)

We can give him one after we've done a stable release.

A couple of issues on offer experience:

• Offer is RID based, which means it doesn't fit into our name based OM very cleanly
• To update an offer, you need to pass it the RID of a resource (you cannot use named based links)
  • This means even if we put it under container/database, we would still have to shim the call with a read container/database call to grab the RID before we could update the offer for it.

It's currently disabled in the new OM, but it's required for updating an existing container/database.

The string enums are really sensitive. If you don't reference the enum directly, not writing out an equivalent string, you get type errors. This is probably overkill and we should think about changing this up.

An upstream bug is causing one of our session tests assertions to fail. Will open a PR to comment out this assertion for now. We should re-enable once the bug is fixed.

Scenario:

Currently, the SDK creates an https.Agent with unlimited sockets. This is not a good idea in many environments, especially on Azure (Azure Functions only allows 300 open sockets at a time). We should change the default to something more reasonable like 200.

Some of the samples aren't updated to use the new OM or is using a slightly older OM. We should update all of these to use the latest and greatest before we ship.

Most folks who want to use forEach's AsyncIterator need to use a polyfill to make it work.

See microsoft/TypeScript#14151

Polyfill:

(<any>Symbol).asyncIterator = Symbol.asyncIterator || Symbol.for("Symbol.asyncIterator");

I'm going to propose that we add this polyfill to the typedocs to make it easier to figure out.

I'm going to open a second issue to track adding the callback based model back (simulatenous support)

There is a few customers who have requested or looked for CreateIfNotExists methods on our objects. In general, we've tried to stick to the simplest version of the SDK and want customer to write that logic themselves.

We should discuss whether it makes sense to support higher level methods that wrap underlying operations.

The way you make calls to paritioned containers vs non-partitioned containers is fairly different, and we should have more samples/docs/typedoc examples which show how this works. The way the SDK works today will lead most folks to using non-partitioned containers.

• npm package published
• Docs updated on docs.microsoft.com (this is automated, but should confirm manually they are updated and formatting looks good
• Final team review
• Blog published
• Celebrate