deveel / deveel.repository Goto Github PK

A common filtering function provided by data storage systems (databases and search engines) uses the distance of geographic points to filter and sort results.

The repository model should provide a new IQueryFilter that provides the possibility to select the origin field, the point and a minimum/maximum distance, in a fashion that can be ported across multiple implementations of the repository.

• MongoFramework provides an extension function to translate into a native Mongo function - we might use this function or rather use the native FilterBuilder
• EntityFramework offers provider-specific functions for the case, but seems to depend from the NetTopologySuite in all cases, to have a GIS object model (1)
  • The SQL Server translator uses EF.Functions.Distance static method in a lambda expression
  • The Postgres translator uses the .Distance method of the parameter of a lambda expression to write the native SQL function(3)

(1) https://learn.microsoft.com/en-us/ef/core/modeling/spatial
(3) https://www.npgsql.org/efcore/mapping/nts.html?tabs=with-datasource

A previous iteration of the Repository model wished to implement a revision support, by holding states of an entity, that is a related value project carrying the status and timestamp of the state change of the main entity.
Although this approach provides some value, it doesn't achieve the goal it was intended to, because it doesn't provide the previous state of the entity, but only one single portion of it (the status).

We should remove this support from the Repository model, which only creates unneeded complexity and reduces the code coverage of the libraries

The concepts of Filter and Sort are currently separated, and the only sorting is happening when querying pages.

In some scenarios, even when returning one single item, it is required that the collection is sorted before returning that item.

We should provide a mechanism to allow filtering and sorting in the same request to a IFilterableRepository<TEntity>

The current implementation threats the identifier / key of an entity only as a string, but many implementations of databases might (and typically do) use other forms, such as ObjectID for Mongo, integer or GUID for SQL databases.

We should change the type returned by FindById and GetEntityId to object, or to a generic argument of the IRepository, letting implementations to deal with it.

The Repository pattern provides several functions to access data sources, but it doesn't provide added value for the validation and management of those data.
For instance, a Repository doesn't check if an entity exists before adding it to the database, and if this happens an exception denoting conflict will be raised.

The scope of an Entity Manager is to provide a service that would operate some validations and checks before writing operations, ensuring the consistency of data, without throwing an exception, but instead returning an object describing the state of the operation (eg. Success, Failure, Unchanged, etc.).

The implementation of the tenant resolution process is quite complex right now

• Implementations of the IRepositoryProvider<TEntity, TKey> and IRepositoryProvider<TEntity> require the specification of a generic argument TTenantInfo derived from ITenantInfo, to align with the Finbuckle's mechanism to resolve the tenant
• The signature of those implementations is difficult and limiting the extensibility

We should find an alternative way to scan for tenants, not forcing implementations of the Repository Provider contract to request for the specification of a TenantInfo generic argument.

A possible solution to this cosmetic and functional problem would be the definition of a service that can scan for instance of IMultiTenantRepository, using the original contract ITenantInfo.

For example:

public interface ITenantResolver {
    Task<ITenantInfo?> FindTenantAsync(string tenantId, CancellationToken cancellationToken);
}

public class EntityRepositoryProvider<TContext, TEntity, TKey> 
    where TContext : DbContext
    where TEntity : class
    where TKey : notnull {

    public EntityRepositoryProvider(ITenantResolver tenantResolver) {
    }
}

Provide a cache layer on top of the data access layer increases performances of the applications, reducing the I/O throughput of the disks, reducing the penalties in the connections between applications and data storages, and providing faster responses to queries from clients.

Anyway, caching is a difficult matter to deal with, especially in scenarios of distributed applications, which might cause race conditions on the cache, interlocking, stale data, concurrent evictions, etc.

To enhance the business capabilities provided out-of-the-box by the EntityManager<TEntity> class, we should include an abstraction layer between the application and the Repository.

• Find requests should Get-or-Set from/in cache the latest version of an entity, eventually using the repository function to produce it, on cache miss (cause of expiration)
• Add commands should introduce the entities, with a configured expiration set
• Remove commands should evict entities from the cache, causing all future calls to miss the cache
• Update commands should cause the replacement of existing entities in cache, or the set in cache (on miss)

The panorama of cache libraries in the .NET ecosystem is rich and includes low-level providers to cache-specific systems, but also intermediate abstractions like

• Cache Tower
• EasyCaching
• Cache Manager

We should aim for an abstraction that provides the specific functions required for scopes mentioned above, delegating the configuration of the infrastructure to the specific library, without trying to override it

The current implementation of the IRepository<TEntity> uses an object type for identity keys of the entities, eventually trying to normalize the key value to the one defined by the type metadata (when available).

A potential improvement of the query constraint by key might be to introduce a new super-contract IRepository<TEntity, TKey>, providing a constraint for the type of the primary key of the entity directly in the specification of the repository.

public interface IRepository<TEntity, TKey> where TEntity : class {
    Task<TEntity?> FindByKeyAsync(TKey key, CancellationToken cancellationToken = default);
}

Negative Impact

A negative outcome of this approach might be the increased complexity of the signatures of repository types.

Considering a possible subcontract like

public interface IRepository<TEntity> : IRepository<TEntity, object> {
}

The automatic registration implemented by .AddRepository() and .AddRepositoryProvider() methods might also be negatively affected, by not being able to determine the entity type and the key type.

The EntityManager<TEntity> will also be affected by the signature change, requiring a similar constraint set.

public class EntityManager<TEntity, TKey> where TEntity : class {
    public IRepository<TEntity, TKey> Repository { get; }
}

The Repository pattern is not naturally designed to be used in an event-sourcing context, given the nature of the event stream and aggregate models.

Anyway, a repository is intended to be a construct between a database and an application, to abstract access to the data through a common pattern: even-driven storage systems work with a different paradigm than the relational database system or the NoSQL systems, but they have an element in common, that is the object-centricity of the information (Entity and Agrgegate).

In an event-driven logic, the core element is the Event: the aggregation of a stream of events (identified by an arbitrary logic) makes an Aggregate, that presents the latest state of an Entity

Proposed Changes

We should try to pursue the implementation of a Repository that supports event-sourcing, working on the principles:

• Implementation of a base Aggregate contract that allows the aggregation of events to be persisted by the Repository
• The EventRepository retrieves the uncommitted events available from the Aggregate
• Definition of an AggregateKey construct that identifies the aggregate and its revision

Possible Limitations

Repositories that implement the support for event-sourcing might also be limited:

• A repository might not be dynamically queryable or filterable - Projectsions are constructs that require a pre-defined aggregation logic
• Listing the aggregates might be not possible - event streams are specific to a given entity, and data storage systems might not support retrieving events for more than one entity

The current implementation of the Repository framework is based on .NET 6.0, which is a version of the .NET framework.

Microsoft has since long marked the version of the framework as in Long Time Support (LTS), with the plan to remove it from support in a few years, following the strategy of releasing newer versions every year.

Several benchmarks on various domains of the framework have proven that performances have been greatly improved by the latest versions of the framework: this seems to indicate that adopting .NET 7.0 would improve the overall performances of the repository framework.

It is reasonable to think that this update would not cause breaking changes to the code.

Microsoft is releasing a preview of the support for MongoDB through the Entity Framework.

Currently, the project uses the MongoFramework library to interface MongoDB, but we might want to investigate the possibility of migrating the MongoDB access to that.

Ideally, the only needed library would end up being the Deveel.Repository.EntityFramework, which can potentially be extended with MongoDB-specific functions (if not already supported by EF)