Move selection in deep endgames is broken about leela-zero HOT 7 CLOSED

  T1 ->   39036 (V: 83.18%) (N:  0.5%) PV: T1 P5 N4 R1 O6 R5 N3 
 K17 ->     726 (V: 64.74%) (N: 12.9%) PV: K17 A3 H3 D1 Q10 G3 C18 
 C18 ->     619 (V: 66.16%) (N:  9.2%) PV: C18 G3 P11 H3 B17 A3 pass R16 
 B17 ->     572 (V: 66.17%) (N:  8.4%) PV: B17 D1 G3 H3 C18 G6 P11 
  H3 ->     556 (V: 63.31%) (N: 10.6%) PV: H3 G6 Q10 R16 B17 G7 K17 
  G3 ->     509 (V: 63.71%) (N:  9.6%) PV: G3 A3 B17 H3 K17 N1 Q10 
pass ->     372 (V: 71.83%) (N:  4.0%) PV: pass D1 K17 G3 C18 R16 
 Q10 ->     265 (V: 47.67%) (N:  9.0%) PV: Q10 P11 T6 R9 T10 
 P11 ->     248 (V: 51.19%) (N:  7.6%) PV: P11 Q10 P10 T10 R11 
42903 visits, 1632236 nodes, 44177 playouts, 883 n/s

This looks a lot better.

from leela-zero.

Should move selection be based on win % and not # of playouts? I realize the winning moves will get more playouts, so over time the higher win % will have the highest # of playouts if there's no constraints. But when we cap playouts at a specific number, once half the playouts are on move #1 then no other move can be chosen, since a better move #2 won't have more playouts by the time the playout cap is hit.

(If confidence intervals are available on the win %s, perhaps choose the move with the highest lower CI bound not just the predicted win % itself, so a move with high win % because we've barely explored it (found right before playout cap kicked in) won't be chosen because confidence is too low.)

from leela-zero.

It must be playouts, selecting on win% makes the program far, far weaker because it will play blunders than win in 1 ply but lose in 2 and have only a few nodes expanded.

The whole point of UCT is that it picks based on the Upper Confidence Bound, so what you're asking for is exactly what is already implemented.

(It's true that in some cases you can take an early exit because nothing can overtake the top move, but the system learns on the probability distribution of the search outcome, so it must still be computed)

from leela-zero.

This is a recently played game with the latest network:

http://eidogo.com/#2YKh7bmNO

Does this still represent a problem with passing? And if so, is it from the engine, or from the network that was trained with the broken engine?

EDIT: Also is scoring correct? W+318.5 seems wrong.

from leela-zero.

I'm not sure what you're asking. The engines don't know anything about passing whatsoever. They simply haven't learned about this yet.

In the original example above, the engine that has a vague idea about counting was winning easily, but suicided its group. The vague count should be enough to see that killing a large amount of your own stones is bad, and capturing a large opponent group is good. And indeed, with the bug fixed, this is what it does, and you can see that the value network likes capturing the opponent (83% winrate) and dislikes killing itself (47% and 51%).

from leela-zero.

Sorry, I should have been more clear. White continued to play even though black passed several times starting around 623, and white was winning decisively.

I was asking if white's lack of passing was a problem.

from leela-zero.

No, as long as it is not killing itself it's fine. As said, they simply don't know about early passing yet, and I'm not sure that in computer vs computer play under Chinese (really Tromp Taylor) rules they ever will. The only time it's right to pass is when you are otherwise forced to suicide yourself. Because they can't count very well yet, you now sometimes have games that they pass out way too early.

I disabled some leftover heuristics for passing just before fixing this bug in commit 7f4d8d2.

So the slightly older version would have reliably passed out that game because it knew it could do an exact count. But now, it has to see that passing is at least as good a move as any.

from leela-zero.