mostjs / core Goto Github PK

See #152

See https://github.com/invertase/denque

See: #152

@nikoskalogridis has been working on a @most/test package. Thank you!

I'd like to use this as a place to discuss the design. Although the core team has discussed it a lot, we've not yet found the time to implement anything. So, I'd like to use this as a place to discuss your design. I think getting on the same page will help us review code more effectively later.

An initial question we could answer:

• What is the relationship between @most/core and most?

Let's brainstorm a few others here.

See #152

redux-most refers to this repo for checking out a preview of Most 2, although I'm not what I'm suppose to see here. Does mostjs/core:master constitute Most 2... is there v2 discussions somewhere?

redux-most readme:

Most 2.0 will be even better, as it will feature an auto-curried API like lodash/fp and ramda, but for working with streams instead of arrays. For a preview of what's to come, check out what's going on here. https://github.com/mostjs/core

In our quest to make custom streams available to all, let's go ahead and pound out a question 2:

What is a Sink?

I mean, I really feel like this word should be meaningful, but I can't seem to wrap my head around it. What even is this thing we call a Sink? And what is it's roll in a Stream?

See #152

Use relative time that starts at 0 instead of using system clock time. Also, since the system clock isn't guaranteed to be monotonic, use performance.now or process.hrtime where available.

See: cujojs/most#452

Based on discussions in cujos/most Issue #449 it seems like there is some interest in improving documentation for the custom stream api.

This is the first of hopefully many issues I will open, where I will try to leverage my naivety to create good questions about how things work, thus hopefully generating nice content for better docs. I will try my best to focus on core issues first (although since I don't really know what I'm doing I might make some mistakes about what is core), hopefully leading to enough lego-blocks for someone to jump right into creating a fully custom stream type.

So, as shown in the title, here is question 1:

What are the minimum required pieces to satisfy the definition of a Source?

(Bonus question: What are the suggested minimum requirements to create a safe production-ready Source?)

Should multicast go in core? I can modify the implementation and provide a pull request if so.

Just wondering if core can be used interchangeably with @cycle/most-run? or if it's best to stick with the main most library for now?

FYI... Definitely liking the direction you're taking most ...towards a more classical frp approach (i.e. continuous time, behaviors, etc). Excited for what's to come!

If the input stream is empty(), return the canonical empty().

Reference #152

PropagateTask currently calls its run function with itself as this. This was an easy (but not simple) way to provide access to the active state, which is needed to stop early in the case where the task is canceled due to any of the various ways the stream could be disposed, such as take().

That leads to WTFs like where does this.active come from?!?. It would be nice to find a better solution.

If the input stream is empty(), return the canonical empty().

References #150, #152

Currently all of our types specify Array<A> which disallows users to fully take advantage of types like ReadonlyArray<A> or ArrayLike<A> which, in TypeScript, both ensure that you can not mutate them. However, because we are using @most/prelude, mutating arrays is avoided all together.

I think it would be very useful to update the types across each package switch 100% to ArrayLike<A>. This would be a non-breaking change as ArrayLike<A> is a subset of ReadonlyArray<A> which is a subset of Array<A>.

There is no first class Source concept in most/core, so the docs shouldn't talk about "source" as if it is first class.

I don't think its harmful to use the word "source" occasionally to refer the the origin of a Stream's events. For example, the source of the event produced by a fromPromise Stream is the promise that was passed in.

This section puts too much emphases on "source". As per slack discussion, we should rename that section to "Streams and Sinks" and talk about Streams instead of "source chains".

Since @most/sample exists as a "better" sampling implementation, what should we do with the existing sample and sampleWith functions in @most/core?

Some possible options:

1. Keep them as is. I'm not sure why we would do this. Are there any compelling reasons?
2. Keep them as exports, but reimplement them using @most/sample. I can't think of a compelling reason to do this, since @most/core has no backward compatibility concerns.
3. Remove them and direct people to @most/sample
4. Deprecate @most/sample, and integrate its API and implementation directly into @most/core. If we did this, we'd still need to decide whether to keep the existing sample and sampleWith. If we kept them, I can't see a reason not to reimplement them.

I don't have a sense for how often they are used, which may be a factor: if they're common, then people would always find themselves needing both @most/core and @most/sample that's an interesting data point that could push us in the direction of 4 rather than 3.

Thoughts?

The current implementation of sample() is:

function sample<A, B, C>(f: (a: A, b: B) => C, sampler: Stream<A>, s: Stream<B>): Stream<C>;

I'd like to suggest that b should be the first argument of f, which would allow simpler composition in common scenarios, seeing as the point of sampling is generally to use the sampler as a signal to read the values on the main stream (b). By moving b to the first argument position, it'll allow the following, as an example:

const clock = sample(identity, periodic(1000), Date.now);
// or
const firstName = sample(get('firstName'), sampler, person);

I can submit a PR if desired.

I'd really love to try something new here, but I don't know if it's feasible. Since we'll have a flow type definition file after #21 is merged, I wonder if we could find a way to write markdown comments in it and generate an API doc from that. I'm thinking something very simple, along the lines of docco, to generate markdown or HTML.

Thoughts? Other ideas?

There quite a few operations where if at least one input stream is empty(), the operation will always return a stream containing zero events. In such cases, some operations return the canonical empty(), and some don't.

Return empty() when possible

• Tracked in #152

It might be interesting to always return the canonical empty() in those cases. That would allow empty() to annihilate other operations more effectively. It trades some up-front cost for reduction in cpu and memory later when a stream is observed.

const s0 = empty()

// s1 contains no events.  All of these layers could be annihilated by empty()
const s1 = combine(combiner, anotherStream, map(f, filter(p, s0)))

// This isn't true today, but I think it's technically possible
s1 === s0

Make slice fusion smarter about empty()

• Tracked in #151

Somewhat related is that slice, take, and skip could produce empty() even without having received it as an input. Currently, slice does minimal checking for a zero-length slice. However, slice fusion doesn't check to see if it has ended up with a zero-length slice after fusing. We should refactor it to do that.

Downsides

The downside, at least right now, is that it requires putting checks in lots of operations (we already check in some operations, but not all). This seems like something that we'd ultimately like a compiler to be able to do for us.

Notes

Some notable exceptions:

• merge() and mergeArray() - it's interesting that empty() streams could effectively be dropped. If mergeArray ends up with a zero length array after dropping empty() streams, then it can also return empty()
• continueWith() - continueWith(f, empty) is effectively the same as calling f. However, it would probably be surprising to users who happen to write a side-effecting f that it's called immediately. Note that recoverWith() can be annihilated by empty()
• scan() - empty() can't annihilate scan entirely, but rather scan(f, x, empty()) is now(x)
• startWith - similar to scan, startWith(x, empty()) is now(x)

ie

export function copy (a) {
  return a.slice(0);
}

At the current state the library, although highly modular (👏), cannot be effectively tree-shaken (well, dead code eliminated to be more exact), because of the heavy curry usage. It is like that because bundlers/minifiers cannot know for sure that the call to curry is pure and wont cause any side-effects - they cannot be sure that it is safe to remove that call, even if its result stays unused.

UglifyJS has introduced possibility to mark calls as pure with comment annotation - #__PURE__. This could be leveraged in the library.

While #__PURE__ comments are useful today, it's quite ugly to put them into the source code and also it's error-prone, it's easy to forget to put them everywhere during development. I've created a babel plugin which helps with that and automates this task, but at the moment you are using buble and not babel for es2015+ transpilation and Im not sure if you are open to a change and if you want your code to get annotated (in any way, doesnt have to be with a plugin I've created).

So in general - what do you think about this? Would you like me to send a PR with buble -> babel + the plugin?

More reading about the problem - here

See #152

Currently the types allow only the event values via Stream<A>. What if we also included the error type Stream<A, B>?

Reference TylorS/typed-stream#1
cc @goodmind

I mentioned it in a comment in a closed PR so I'm opening an issue here.

The performance.now method is being called as a function without binding this to performance and it's causing an error. I guess this wasn't detected in testing because the tests run in a node.js environment?

Continuing the conversation from cujojs/most#347

One way to solve the "problem" of variadic functions not being curry-friendly is to provide variants with specific arities. Haskell does this, i.e. zip2, zip3, et al. We could do the same by keeping the variadic versions and simply providing fixed-arity wrappers. For example:

import { combine } from './combinators/combine'

export { combine } // variadic

export combine2 = curry3((f, s1, s2) => combine(f, s1, s2))
export combine3 = // ...
// etc

Of course, there's always the option of waiting until folks ask for this before actually implementing it. But, I wanted to get some current thoughts on it.

Crosspost from cujojs/most#516

Observables are one of the most general abstractions we have available in Computer Science. They provide a common interface to operate on singular or plural and synchronous or asynchronous items.

But programming with Observables is a path riddled with traps, partly due to its novelty, but also that our tooling mostly is built around synchronous imperative code, and not asynchronous functional code.

Typing can get us really far in removing common errors when piping Observables through our code, but our types are not going to help us with the problem of time.

I have spent some days philosophizing whether we need a temporal-type to decorate our Observables, to get a better immediate idea of the consequences when merging, combining, delaying, debouncing, skipping, etc...

At first I set out to categories Observables in my own made-up categories, but then I realized we already have a quite extensive and rigid categorization of temporal events, this system is known as Grammar more properly conjugation, and it provides us with an opportunity to type our Observables not just with their expected value, but when that value might arrive.

In Latin conjugation we can operate with four dimensions, mood, voice, tense, and person. These four dimensions map well to what we need to properly time-categorize Streams.

Grammatical Mood

Grammatical mood helps us know the certainty of an event. Moods can be divided into Realis and Irrealis, things that are certain, and things that are uncurtain. Indicative is a Realis mood, the rest are Irrealis.

They may not all be applicable for Observables

enum Mood {
  Indicative, // realis certain
  Subjunctive, // it might
  Conditional, // if
  Optative, // hope
  Imperative, // command
  Jussive, // command to non-present
  Potential, // it could
  Hypothetical, // it is unlikely
  Inferential, // tree falling in forest without observer
  Interrogative // asking (probably better for a pull based thing)
}

Grammatical Voice

Voice indicates whether something is doing something, or something is happening to something. In observables it could indicate whether the Observable producer has all it's futures defined, or whether it awaits input from the outside (such as passing of time, mouse clicks).
Most Observables would be Passive, with the exception of Observables created from arrays or constants.

enum Voice {
  Acitve,
  Passive
}

Grammatical Tense

Grammatical tense indicates when something may happen, or when something did happen, we might want to add a Never, or Never may simply be an undefined Tense.

enum Tense {
  Present, // now
  Imperfect, // past and ongoing
  Perfect, //past and complete
  Future, // soon
  Pluperfect, // happened before something else in the past
  FuturePerfect // will happen at this time
};

Gramatical Person

Finally we have grammatical persons, that indicates who is doing what. We could use the person to indicate singular and plural events of a stream, and the degree of stream (higher-order-streams)

enum Person {
  FirstPersonSingular, // a stream of one thing
  FirstPersonPlural, // a stream of many things
  SecondPersonSinglar, // a stream of one stream of one thing
  SecondPersonPlural, // a stream of streams of things.
  ThirdPersonSingular, // a stream of one stream of one stream of one thing
  ThirdPersonPlural // a stream of streams of streams of things
}

Once our Observable producers are properly decorated with grammatical metadata. We can proceed to ensure that our operators determine what the conjugation of the resulting stream would be.

Practical Applications

Imagine you are combining stream a$ and b$, they are both Stream<number> but a$ is defined far away from where you are currently working, a$ happens to be never() while b$ is just(3) but you are unaware and spend some time trying to find the culprit. With gramatical typings you could get a warning, indicating you are trying to combine something b$ (Present, FirstPersonSingular, Active, Indicative) with a$ (Never), you would get a hint as to where something is amiss.

I would welcome some input on the ideas above, is it useful?

See #152

This will also cover, mergeConcurrently, join, and chain.

i will give this one a whirl (tonight/tomorrow am) if you guys dont mind so i can be more familiar with both nb and the assert lib

Inside seedValue.d.ts there are numerous base types: Sink, Scheduler &c.

Except for the SeedValue itself, no other exported type seems to be used anywhere: the correct types are imported from @most/types. Still, they can confuse an IDE into suggesting them if you don’t have @most/type installed.

Heres a reproducible case, there are additional comments in the code to define the problem.
https://www.webpackbin.com/bins/-KvrqudMj6C-0H494Itg

See #150

There quite a few operations where if at least one input stream is empty(), the operation will always return a stream containing zero events. In such cases, some operations return the canonical empty(), and some don't.

It might be interesting to always return the canonical empty() in those cases. That would allow empty() to annihilate other operations more effectively. It trades some up-front cost for reduction in cpu and memory later when a stream is observed.

Operations that could return empty()

• map
• filter
• ap
• loop
• withItems & zipItems (previously implemented)
• zip & zipArray
• combine & combineArray
• mergeMapConcurrently, and thus join, chain, and concatMap
• snapshot & sample (#186)
• skipRepeats & skipRepeatsWith
• slice, take, skip
• takeWhile
• skipWhile
• skipAfter
• withLocalTime - end time still needs to be localized, so withLocalTime can't be annihilated by empty()
• awaitPromises
• switchLatest
• throttle
• debounce
• merge(empty(), empty()) and mergeArray when all streams are empty()
• recoverWith(f, empty())
• multicast

Operations that need more discussion

• delay - What is the meaning of delay(n, empty()), especially wrt end?

Related optimizations

Optimizations that don't produce canonical empty, but noting here so we don't lose them.

• scan - scan(f, x, empty()) is now(x) rather than empty(). It's not clear whether this is a worthwhile optimization.

Anyone use it regularly? The only place it's used internally is in its own unit test. Is there a compelling reason to keep it?

If the input stream is empty(), return the canonical empty().

Reference #152

See #152

tl;dr The defaultScheduler is convenient. However, it introduces a few things that are less than ideal in my mind. I propose we remove defaultScheduler, observe, drain, and reduce in favor of a single runStream primitive.

Problems

Module-load-time side effect

The creation of defaultScheduler is a module-load-time side effect. This is a minor issue, but not great from a purity standpoint. If Scheduler (or any of its constituents) is modified to have a non-harmless side-effect, it would automatically impact any program that ends up loading the defaultScheduler module.

Encourages problematic design patterns

This one is probably much more controversial 😄

In the Promise world, it is well-established that a good default model is to write functions that accept non-promise parameters, and return a Promise. Viewing a program as a tree, this encourages propagating promises "up" from program leaf nodes toward the root. In a lazy Future/Task world, this model is even more desirable because it centralizes the activation of side effects at or near the root. Scattering the activation of side effects out to leaf nodes makes programs much harder to reason about, and any coordination among those scattered side effects will be implicit or ad hoc and may only "work" by happy accident.

Event streams are a similar construct, and imho, it's desirable to program in the same way: to return event streams and centralize observation, which is the event stream version of "activate side effects", at or near the root.

Through time and examples, the "return promises/futures/tasks" model has become an accepted best practice. Another way to help that process along is to make it unergonomic, or just plain annoying, to observe at leaf nodes. One way to do that is to require that any function that activates an event stream accept a Scheduler instance as a parameter. So, a program that observes at leaf nodes would need to either: 1) construct lots of Schedulers, which is, imho, fairly obviously The Wrong Thing, and will make it impossible to coordinate side effects among observations that use different Schedulers, or 2) pass a Scheduler down to leaf nodes, which (hopefully) will be quite annoying and ugly.

Proposal

To encourage the practice of returning event streams and centralizing observation, I propose we:

1. remove defaultScheduler
2. remove observe, drain, and reduce
3. export a function which can be used to build observe, drain, and reduce in the cujojs/most layer
4. export Scheduler and its constituents (Timeline, etc.)
5. export a createDefaultScheduler() function (see below)

type Observer a = { event :: a -> (), error :: Error e => e -> (), end :: () -> () }

runStream :: Observer a -> Scheduler -> Stream a

createDefaultScheduler :: () -> Scheduler

See #152

See #150 #152

Improve documentation for time types, and link to it from functions that accept time types.

Wondering if you have discussed/decided on whether the code base should be in plain Javascript as it is now or amended with Flow types or migrate to Typescript.

My personal opinion would be to switch to using Flow types.

Advantages:

• Keeping the current build stack (babel, rollup) intact
• Completely static type check all the files in the code base
• Gradually migrate files (without renaming all of them in .ts)
• If we decide to change this decision and revert to plain Javascript we can simply strip the Flow types from the files and be done with it
• If we decide to switch to Typescript at some time in the future that will be easier as we would already have made most of the work required. We would only need to change some incompatibilities between Flow and Typescript.

I would be interested to hear what the members think on this

Based on Slack discussion with @briancavalier we would want to research the potentially differences (specifically performance) with current
fusion implementation using closure tree structures compared to fusion using array structures.

Currently we have composition for something like Map fusion or map.map which simply looks like compose(f, source.f)

The array fusion would simply hold the functions inside an array [f, source.f, ...] and then apply the array to the source values.