Comments (7)
Notes from Design Meeting 4/14
- Generally agreed on the decorator-based syntax
- Want to make sure we cover
- homogenous streaming scenarios (covered)
- heterogeneous message streaming
- There are likely desirable additional metadata for protocol-specific bindings for streaming, but these are orthogonal to the proposal
- Discussed alternate modeling for systems like webPubSub, where client messages / subscription management are represented as individual operations, and there is a separate operation for receiving streamed messages, e.g..
op joinGroup(JoinGroupRequest): void; op leaveGroup(LeaveGroupRequest): void; op listen() : @stream GroupMessage[];
- Need to explore events / webHooks a bit more before the group is completely comfortable with adopting the proposal
- Would be nice to explore how batching would be similar / different
from cadl.
Notes from design meeting
- Update to reflect http metadata proposal
- Need to add examples for request streaming
- messaging
- web pub-sub
- large file streaming (already covered, I believe)
- Think about what parts of the proposal for streaming could also apply to batching (for example, representing send-once and send-multiple metadata fields)
from cadl.
Options
1. @stream
decorator
// Helper model
model Stream<T> {
@stream stream: T[]; // or not an array?
}
// Stream response string
op foo(): {@stream foo: string[]}
op foo(): {...Stream<string>}
// Stream input string
op foo(@stream foo: string[]): void
op foo(...Stream<string>): void
// Stream custom message
model MyMessage {id: string, value: string}
op foo(): {@stream foo: MyMessage[]}
op foo(): {...Stream<MyMessage>}
// Stream custom message
op foo(@stream foo: MyMessage[]): void
op foo(...Stream<MyMessage>): void
2. Stream
intrinsic type
Similar to how you would want to return a primitive type as the body(string
, number
, etc.)
// Helper model
@intrinsic
model Stream<T> {
stream: T;
}
// Stream response string
op foo(): Stream<string>
// Stream custom message
model MyMessage {id: string, value: string}
op foo(): Stream<MyMessage>
op foo(@body stream Stream<MyMessage>): void;
Doesn't make that much sense with websockets?
op foo(@body stream Stream<MyMessage>): Stream<MyMessage>;
Examples
Messaging
model Message {
id: string;
from: string;
group: string;
content: string;
}
// Option 1
op messages(@stream messages: Message[]): {@stream messages: Message[]}
// Option 2
op messages(@body messages: Stream<Message>): {@body messages: Stream<Message>}
Webpub sub
https://docs.microsoft.com/en-us/azure/azure-web-pubsub/reference-json-webpubsub-subprotocol
alias WebPubSubRequest = JoinGroupRequest | LeaveGroupRequest | SentToGroupRequest
model JoinGroupRequest {
type: "joinGroup";
group: string;
ackId: int32;
}
model LeaveGroupRequest {
type: "leaveGroup";
group: string;
ackId: int32;
}
model SentToGroupRequest {
type: "sendToGroup";
group: string;
ackId : number;
noEcho: boolean;
dataType : "json" |"text"|"binary";
data: string | {};
}
model WebPubSubResponse = AckResponse | MessageResponse;
model AckResponse {
type: "ack";
ackId: number;
success: boolean;
error?: {
name: "Forbidden"|"InternalServerError"|"Duplicate";
message: string;
}
}
model MessageResponse {
type: "message";
from: "group";
group: string;
dataType : "json" |"text"|"binary";
data: string | {};
fromUserId: string;
}
model ConnectedSystemResponse {
type: "system";
event: "connected";
userId: string;
connectionId: string;
}
model DisconnectedSystemResponse {
type: "system";
event: "disconnected";
message: string;
}
// Option 1
op webpubsub(@stream messages: WebPubSubRequest[]): {@stream messages: WebPubSubResponse[]}
// Option 2
op webpubsub(@body messages: Stream<WebPubSubRequest>): Stream<WebPubSubResponse>
Server side events
model ServerSideEvent {
id: string;
event: string;
data: {};
}
// Option 1
op messages(): {
@header("Cache-Control") cacheControl: "no-store";
@header("Content-Type") contentType: "text/event-stream";
@stream events: ServerSideEvent[];
}
// Option 2
op messages(): {
@header("Cache-Control") cacheControl: "no-store";
@header("Content-Type") contentType: "text/event-stream";
@body events: Stream<ServerSideEvent>;
}
// Option 2 alt with named model
model MyType is Stream<ServerSideEvent>{
@header("Cache-Control") cacheControl: "no-store";
@header("Content-Type") contentType: "text/event-stream";
}
op messages(): MyType
from cadl.
- Convert from non stream to stream api
- How align with pageable?
from cadl.
Case 1 simple return string
op readLogs(@query fromBytes: int64): string;
// ------------ convert to stream -------------
op readLogs(@query fromBytes: int64): Stream<string>;
Case 2: Include header
op readLogs(@query fromBytes: int64): {@body body: string, @header contentType: "text/plain"};
// ------------ convert to stream -------------
op readLogs(@query fromBytes: int64): {@body stream: Stream<String>, @header contentType: "text/event-stream"};
Case 3: Metadata extract
op readLogs(@query fromBytes: int64): {data: string, lineCount: number, @header contentType: "application/json"};
// ------------ convert to stream -------------
// Option 1 wrap everything in stream type and have everything that should belong in the body be there.
op readLogs(@query fromBytes: int64): Stream<{data: string, lineCount: number, @header contentType: "application/json"}>;
// Option 2: Extract body manually
op readLogs(@query fromBytes: int64): {@body stream: Stream<{data: string, lineCount: number}>, @header contentType: "application/stream+json"};
// Option 3: @stream
op readLogs(@query fromBytes: int64): {@stream stream: {data: string, lineCount: number}, @header contentType: "application/stream+json"};
Case 4: Convert array to stream
op readItems(@query skipCount: number): Foo[];
// ------------ convert to stream -------------
// Now stream the foo items.
op readItems(@query skipCount: number): Stream<Foo>;
Case 5: convert input array to stream
op postItems(@body items: Foo[]): void;
// ------------ convert to stream -------------
// Now stream the foo items into the server.
op readItems(@body items: Stream<Foo>): void;
from cadl.
This came up in the API Stewardship Board today in a review, and doing some research on this I also found application/x-ndjson
- newline-delimited JSON e.g.,
{"foo":"bar","baz":1}
{"foo":"qux","baz":2}
Seems it's gaining traction and certainly seems a lot more compact than SSE.
from cadl.
Want to bump this thread too. I'm working with the batch team, and they have some operations that we generate as a streamed response (swagger ex here)
from cadl.
Related Issues (20)
- Json Schema emitter doesn't carry base scalar constraint
- Json schema emitter throws error
- Json schema doesn't support default
- Json Schema doesn't handle template scalars
- Json Schema emitter add type of enum is string when there is 0 as an option
- Don't change the response from 200 to 204 for empty implicit response body HOT 1
- Add `unixTimestamp32` to std lib HOT 1
- Json Schema should a union of literals be converted to an enum
- Model spreading itself, should it be allowed HOT 3
- `oneOf` and `useRef` in an awkward location HOT 2
- Add an equivalent to `valueOf` for `ModelProperty` references HOT 2
- Add model to HTTP library for "problem details" HOT 2
- Migrate openapi3 emitter to use emitter framework HOT 4
- Migrate website to astro
- "tsp format" has different indentation on Linux vs Windows HOT 4
- PR artifact build should we relax dep requirements
- Unreachable types are resolved with `read` visibility HOT 3
- how to specify the element type encoding of an array of scalar or a record of scalar ? HOT 3
- Provide a suppression mechanism for @typespec/http/no-routes HOT 3
- [OpenAPI3] `duplicate-header` error issued for shared routes when unioned ErrorModel contains a header HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cadl.