Optimized w[3] too low about fsrs-optimizer HOT 12 CLOSED

Because we use sqrt(count) in 8ac5d5b#diff-66ddf16d3b863ea428c3d0d49c515150cb2c4cd81dc1abe88123b4c08824d550R847, so the impact of reviews with interval = 24 decreases.

from fsrs-optimizer.

But the orange line doesn't match any of the data points.

from fsrs-optimizer.

I guess that 8ac5d5b#diff-66ddf16d3b863ea428c3d0d49c515150cb2c4cd81dc1abe88123b4c08824d550L851 is the cause. You should probably add sum(weight) in place of the removed total_count. This is not the cause because total_count has been removed from the denominator of both logloss and L1.

Unrelated, but in 8ac5d5b#diff-66ddf16d3b863ea428c3d0d49c515150cb2c4cd81dc1abe88123b4c08824d550R831, shouldn't the range be (1, 4) rather than (1, 5)?

from fsrs-optimizer.

shouldn't the range be (1, 4) rather than (1, 5)?

In python, range(1, 4) doesn't include 4.

But the orange line doesn't match any of the data points.

I think 129 data points are not enough to allow the parameter to deviate the default value too much.

from fsrs-optimizer.

I think 129 data points are not enough to allow the parameter to deviate the default value too much.

But, I think that w[3] for my collection should still be somewhat higher. Even if it doesn't reach 60, it should still be at least 40.

from fsrs-optimizer.

I prefer to be conservative here. And a short interval could allow FSRS to collect the data more quickly.

from fsrs-optimizer.

In that case, I will have to manually increase the value of w[3] because such a low value is not acceptable to me (especially when it is equal to w[2]).

By the way, after manually increasing the value of w[3] to 60 and clicking Evaluate, log loss decreased from 0.2139 to 0.2137.

from fsrs-optimizer.

@Expertium, what do you think?

from fsrs-optimizer.

Well, according to my testing, the new implementation is more accurate. That's all I can say.

from fsrs-optimizer.

And the default w[3] has decreased in the recent update. It would also induce this issue. I will update the default weights in the this week.

from fsrs-optimizer.

I think that the real issue is 8ac5d5b#diff-66ddf16d3b863ea428c3d0d49c515150cb2c4cd81dc1abe88123b4c08824d550R656.

I used python optimizer version 4.12.0 to obtain a more complete S0 dataset from my deck file. A screenshot of the relevant portion is here:

As you can see, I don't have only 129 datapoints with initial_rating = 4, but 294 datapoints. The above change wiped off more than half of the available data.

Well, at one point of time, I myself suggested filtering out such datapoints (open-spaced-repetition/fsrs4anki#282 (comment)). But now, I am very sure that filtering this data is not a good idea.

from fsrs-optimizer.

I decided to remove that line myself and see what happens. (user1823@803d4b2)

from fsrs-optimizer.