vowpalwabbit / estimators Goto Github PK

• support for python 3.9, linter, test coverage etc

• will fix issue #12 with this issue

There is a room for tests improvements:

• more corner cases
• more e2e scenarios

• Add interface for ccb estimator (base.py) - similar to slates, but reward is list of the same length as probabilities

• Make FirstSlotEstimator implementation:

facade over bandits estimator that is feeding only data from the first slot. (+ tests)

n -> ips_numerator (this can also be called snips_numerator)
d -> snips_denominator
N -> total_events
Ne -> offline_policy_events
c -> max_ips_numerator

p_log -> p_online_policy (probability of chosen action by online policy)
p_pred -> p_offline_policy
r -> reward

There is open 2020 RLOS PR which has several new bandits estimators implementation, that can be adapted to new interface

Send out a package structure proposal for estimator library. Two use cases the design needs to address:

1. for users to consume this library, a clear structure to find the interfaces to use for their problem type (cb, ccb, slates, ca etc.)
2. for people to add new estimators, a clear structure to know which interfaces need to be implemented.

Since the package is not structured with a setup.py it is hard to install and consume. We would like to be able to install it like so:

python -m pip install git+https://github.com/VowpalWabbit/estimators.git

• what about cressieread?

Estimators

• ips
• weighted ips (aka snips)
• mle
• DR with importance weights transformation (truncate, clipped, mle-like, etc.) - dm, dr, switch estimator, ips, are all special cases of this. SN-versions are also possible

Components

• Python
• Testing framework
• C++
• C#

Confidence interval

• Design the interfaces for each problem type (cb, ccb, ca, slates)
• Extend the interfaces for parallel estimation (start with easier ones e.g. ips)

Two type of test cases can be add:

• Tests for correctness
• Maybe some general tests for all problem types, e.g. verify output is in correct format, confidence interval always converges and not negative and some other sanity checks.

• For intervals store min_bound and max_bound in tuple instead of List
• For Interval get() method set alpha default value
• Change inconsistent variable names { (w and p_over_p), (num and den in get method) }
• Imports in the beginning of the file: change here