[RMP] Support for simple ML/CF Models (like Implicit) in Merlin Models and Systems about merlin HOT 6 OPEN

I've updated the description to make it clearer which tasks are in scope for 22.07. I think we can achieve a self-contained operator for Implicit for the upcoming release.

from merlin.

@benfred , please add the tickets related to this epic here

from merlin.

@EvenOldridge @benfred , is Batch prediction in NVTabular section required for 22.04 ?

from merlin.

For the serving part. My understanding is that, for both Implicit and LightFM, the way to serve these at scale is similar to the two tower / bi-encoder design of the DLRM example we have.

Where the item embeddings are stored in a approximate-nearest neighbour search index. And we search through this space using a user embedding. (for user->item recommendations).

The difference from the DLRM example, is in how the user embedding is computed at serving time.

Computing the User Embedding:

• LightFM represents users and items as an aggregation (sum) of their feature embeddings. In this case we would lookup all the feature embeddings corresponding to the user and then aggregate (sum) these.
• Implicit has support for a few different models. Each has the concept of an item and user (factor/embedding)
  The alternating least-squares implementation (ALS) appears to be the only one that supports calculating the user factor (embedding) for unseen/new users by running the solver provided items the user has interacted and some model parameters (I'm currently unsure if it would be feasible to store this in practice in a serving scenario / how fast this inference would be).
  However, for the case where we have the user factors/embeddings computed already, we can look these up at query-time.

It seems that one of the main questions is where do we store the user/item/feature embeddings for fast lookup at serving time. Ideally with support for versioning (for migrating to new model version) / updating (for new users and items).

For vector/embedding based item->user, user->user, item->item recommendations. two main components are:

• Storing user/item/feature embeddings (Key-value lookup) + optional aggregation (e.g. in the case of LightFM)
• Approximate Nearest Neighbour search through item or user space.
  We currently have the QueryFaiss operator. Which is currently limited to a pre-computed index. (so would need to re-compute the index and re-deploy Triton for new items to be considered.) -> Issue for extending functionality of the nearest neighbour candidate retrieval here NVIDIA-Merlin/systems#10

We could start with a similar approach to the nearest neighbour implementation we have now. A first pass at this could be to store the user/feature embeddings as static data inside Triton. Albeit with similar challenges to having the item embeddings index stored as file(s) inside Triton (updating, and scaling to large numbers of items/users). Or they could be managed in a database outside Triton.

from merlin.

@benfred / @oliverholworthy , please fill up the problem, goal and constraints section in the description. You may have provided this in the comments. Please summarise it there. Let me know if you are facing any difficulties.

from merlin.

@oliverholworthy , I have added 22.07 in the description to specify that the task is in scope for 22.07. If it needs to be moved out to future release, please specify

from merlin.