Equinox provides a unified programming model for Command handling against event-sourced stream stores.
Current supported backends are:
- EventStore - this codebase itself has been in production since 2017 (commit history reflects usage), with elements dating back to 2016.
- Azure Cosmos DB (See
cosmos
branch - will converge withmaster
when the time is right). - (For integration test purposes only) Volatile in-memory store.
The underlying patterns have their roots in the DDD-CQRS-ES community, and the hard work and generosity of countless folks there presenting, explaining, writing and hacking over the years. It would be unfair to single out even a small number of people despite the immense credit that is due.
While the implementations are distilled from code from Jet.com
systems dating all the way back to 2013, the abstractions in the API design are informed significantly by work, discussions and documentation and countless hours invested with no expectation of any reward from many previous systems, frameworks, samples, forks of samples and the outstanding continuous work of the ๐ EventStore founders, team and community over the years.
- Designed to be non-invasive to application code; Domain tests can be written directly against the models without any need to use Equinox assemblies or constructs as part of writing those tests.
- Encoding of events via
Equinox.UnionCodec
provides for pluggable encoding events based on either:- Providing an explicitly coded pair of
encode
andtryDecode
functions - Using a versionable convention-based approach (using
Typeshape
'sUnionContractEncoder
under the covers), providing for serializer-agnostic schema evolution with minimal boilerplate
- Providing an explicitly coded pair of
- Independent of the store used, Equinox provides for caching using the .NET
MemoryCache
to minimize roundtrips, latency and bandwidth / request charges costs by maintaining the folded state without any explicit code within the Domain Model - Logging is both high performance and pluggable (using Serilog to your hosting context (we feed log info to Splunk atm and feed metrics embedded in the LogEvent Properties to Prometheus; see relevant tests for examples)
- Extracted from working software; currently used for all data storage within Jet's API gateway and Cart processing.
- Significant test coverage for core facilities, and per Storage system.
Equinox.EventStore
Transactionally-consistent Rolling Snapshots: Command processing can be optimized by employing in-stream 'compaction' events in service of the following ends:- no additional roundtrips to the store needed at either the Load or Sync points in the flow
- support, (via
UnionContractEncoder
) for the maintenance of multiple co-existing compaction schemas in a given stream (A snapshot isa Event) - compaction events typically do not get deleted (consistent with how EventStore works), although it is safe to do so in concept
- NB while this works well, and can deliver excellent performance (especially when allied with the Cache), it's not a panacea, as noted in this excellent EventStore article on the topic
The Equinox components within this repository are delivered as a series of multi-targeted Nuget packages targeting net461
(F# 3.1+) and netstandard2.0
(F# 4.5+) profiles; each of the constituent elements is designed to be easily swappable as dictated by the task at hand. Each of the components can be inlined or customized easily:-
Equinox.Handler
(Nuget:Equinox
, depends onSerilog
(but no specific Serilog sinks, i.e. you can forward toNLog
etc)): Store-agnostic Decision flow runner that manages the optimistic concurrency protocolEquinox.Codec
(Nuget:Equinox.Codec
, depends onTypeShape
, (optionally)Newtonsoft.Json >= 11.0.2
but can support any serializer): a scheme for the serializing Events modelled as an F# Discriminated Union with the following capabilities:- independent of any specific serializer
- allows tagging of Discriminated Union cases in a versionable manner with low-dependency
DataMember(Name=
tags using TypeShape'sUnionContractEncoder
Equinox.Cosmos
(Nuget:Equinox.Cosmos
, depends onSystem.Runtime.Caching
,FSharp.Control.AsyncSeq
,TypeShape
, ): Production-strength Azure CosmosDb Adapter with integrated transactional snapshotting facilitating optimal read performance in terms of latency and RU costs, instrumented to the degree necessitated by Jet's production monitoring requirements.Equinox.EventStore
(Nuget:Equinox.EventStore
, depends onEventStore.Client[Api.NetCore] >= 4
,System.Runtime.Caching
,FSharp.Control.AsyncSeq
,TypeShape
): Production-strength EventStore Adapter instrumented to the degree necessitated by Jet's production monitoring requirementsEquinox.MemoryStore
(Nuget:Equinox.MemoryStore
): In-memory store for integration testing/performance baseliningsamples/Store
(in this repo): Example domain types reflecting examples of how one applies Equinox to a diverse set of stream-based modelsEquinox.Cli
(in this repo): General purpose tool incorporating a scenario runner that facilitates running representative load tests composed of transactions insamples/Store
against each backend store; this allows perf tuning and measurement in terms of both latency and transaction charge aspects.
Please raise GitHub issues for any questions so others can benefit from the discussion.
The aim in the medium term (and the hope from the inception of this work) is to run Equinox as a proper Open Source project at the point where there is enough time for maintainers to do that properly.
We are getting very close to that point and are extremely excited by that. But we're not there yet; this is intentionally a soft launch.
For now, the core focus of work here will be on converging the cosmos
branch, which will bring changes, clarifications, simplifications and features, which all need to be integrated into the production systems built on it, before we can consider broader based additive changes and/or significantly increasing the API surface area.
For these reasons, the barrier for contributions will unfortunately be inordinately high in the short term:
- bugfixes with good test coverage are always welcome - PRs yield MyGet-hosted NuGets and in general we'll seek to move them to NuGet prerelease and then NuGet release packages with relatively short timelines.
- minor improvements / tweaks, subject to discussing in a GitHub issue first to see if it fits, but no promises at this time, even if the ideas are fantastic and necessary ๐ญ
- tests, examples and scenarios are always welcome; Equinox is intended to address a very broad base of usage patterns; Please note that the emphasis will always be (in order) 1) providing advice on how to achieve your aims without changing Equinox 2) how to open up an appropriate extension point in Equinox 3) (when all else fails), add to the complexity of the system by adding API surface area or logic.
- we will likely punt on non-IO perf improvements until such point as Cosmos support is converged into
master
- Naming is hard; there is definitely room for improvement. There likely will be a set of controlled deprecations, switching to names, and then removing the old ones. However, PRs other than for discussion purposes probably don't make sense right now.
Run, including running the tests that assume you've got a local EventStore and pointers to a CosmosDb database and collection prepared (see PROVISIONING):
./build.ps1
./build -s
dotnet pack build.proj
./build -se -scp
& .\cli\Equinox.Cli\bin\Release\net461\Equinox.Cli.exe es run
& dotnet run -f netcoreapp2.1 -p cli/equinox.cli -- es run
For EventStore, the tests assume a running local instance configured as follows to replicate as much as possible the external appearance of a Production EventStore Commercial cluster :-
# requires admin privilege
cinst eventstore-oss -y # where cinst is an invocation of the Chocolatey Package Installer on Windows
# run as a single-node cluster to allow connection logic to use cluster mode as for a commercial cluster
& $env:ProgramData\chocolatey\bin\EventStore.ClusterNode.exe --gossip-on-single-node --discover-via-dns 0 --ext-http-port=30778
While EventStore rarely shows any negative effects from repeated load test runs, it can be useful for various reasons to drop all the data generated by the load tests by casting it to the winds:-
# requires admin privilege
rm $env:ProgramData\chocolatey\lib\eventstore-oss\tools\data