can you supply a valiation tool like leelaz about sai HOT 22 OPEN

I don't know if validation works with SAI, but one can use gogui-twogtp

from sai.

i guess irename sai.exe to leelaz.exe will let validation work:)

from sai.

i copied the sai.exe to leelaz directory, use following command works

validation -k sai40_lz24 -n 2ae.gz -o "-g -v 100 -r 5 -w" -n 338.gz -o "-g -v 100 -r 5 -w" -- sai -- sai

from sai.

did it work?

from sai.

yes

2ae.gz   v 338.gz   ( 33 games)
              wins        black       white
2ae.gz     24 72.73%   12 75.00%   12 70.59%
338.gz      9 27.27%    4 25.00%    5 29.41%
                       16 48.48%   17 51.52%

from sai.

sai41 vs lz030

91 wins, 110 losses
The first net is worse than the second
c1d.gz   v 273.gz   ( 201 games)
              wins        black       white
c1d.gz     91 45.27%   48 45.71%   43 44.79%
273.gz    110 54.73%   57 54.29%   53 55.21%
                      105 52.24%   96 47.76%

sai43 vs lz035

4 wins, 30 losses
The first net is worse than the second
d8a.gz   v 300.gz   ( 34 games)
              wins        black       white
d8a.gz      4 11.76%    2 11.76%    2 11.76%
300.gz     30 88.24%   15 88.24%   15 88.24%
                       17 50.00%   17 50.00%

sai43 vs lz033

d8a.gz   v aaa.gz   ( 57 games)
              wins        black       white
d8a.gz     31 54.39%   14 56.00%   17 53.13%
aaa.gz     26 45.61%   11 44.00%   15 46.88%
                       25 43.86%   32 56.14%

sai44 vs lz033

35d.gz   v aaa.gz   ( 35 games)
              wins        black       white
35d.gz     27 77.14%   12 85.71%   15 71.43%
aaa.gz      8 22.86%    2 14.29%    6 28.57%
                       14 40.00%   21 60.00%

from sai.

Thank you. Please note, that we are already comparing SAI nets and LZ nets: the 10 games matches you will see from time to time are of that kind. We are currently on par with LZ036 apparently

from sai.

validation -k sai58_lz40 -n 39c.gz -o "-g -v 200 -r 5 -w" -n 300.gz -o "-g -v 200 -r 5 -w" -- sai -- sai
sai58 vs lz40

Stopping engine.
11 wins, 37 losses
The first net is worse than the second
39c.gz   v 300.gz   ( 48 games)
              wins        black       white
39c.gz     11 22.92%    5 21.74%    6 24.00%
300.gz     37 77.08%   18 78.26%   19 76.00%
                       23 47.92%   25 52.08%

sai59 vs lz40

Stopping engine.
14 wins, 40 losses
The first net is worse than the second
c16.gz   v 300.gz   ( 54 games)
              wins        black       white
c16.gz     14 25.93%    4 19.05%   10 30.30%
300.gz     40 74.07%   17 80.95%   23 69.70%
                       21 38.89%   33 61.11%

sai61 vs lz47

9df.gz   v 830.gz   ( 18 games)
              wins        black       white
9df.gz      5 27.78%    2 25.00%    3 30.00%
830.gz     13 72.22%    6 75.00%    7 70.00%
                        8 44.44%   10 55.56%

from sai.

sai65 vs lz147

c99.gz   v 830.gz   ( 24 games)
              wins        black       white
c99.gz      8 33.33%    5 35.71%    3 30.00%
830.gz     16 66.67%    9 64.29%    7 70.00%
                       14 58.33%   10 41.67%

from sai.

sai68 vs lz147

2aa.gz   v 830.gz   ( 28 games)
              wins        black       white
2aa.gz      8 28.57%    4 28.57%    4 28.57%
830.gz     20 71.43%   10 71.43%   10 71.43%
                       14 50.00%   14 50.00%
28 games played.

from sai.

I run validation with the following command
validation -k dir --network saibest.gz...
but it says GTP: Unrecognized argument: saibest.gz

from sai.

please post your full command line

from sai.

./validation -k 10k177selfplay -u 1 -g 1 --network saibest.gz -o "-g -t 16 --noponder --lambda 0 --mu 0 --nrsymm -v 10000" --network saibest.gz -o "-g -t 16 --noponder --lambda 0 --mu 0 --nrsymm -v 10000"-- /home/kenny/Desktop/sai/build/sai -- /home/kenny/Desktop/sai/build/sai

from sai.

my command line works on windows

validation -k sai77_lz250 -n af4.gz -o "-g -v 400 -r 5 -w" -n networks/3d415846183a7f51a40aaa80a007d886668e759c37be8712febf09ec2823d257.gz -o "-g -v 1 -r 5 -w" -- sai -- sai

from sai.

i think you need add the -w in -o " ...."

from sai.

oh, it automatically put the net in the end of the option?

from sai.

It still can't work.
I tried to use the absolute path.
But the engine can run.
The error is still "GTP: Unrecognized argument: saibest.gz"

from sai.

I have never used validation and cannot give you advice, but I reopen the issue so that people can better see it and answer.

from sai.

@kennychenfs, as @l1t1 pointed out, you have to add '-w' right to the end of the options:

./validation -k 10k177selfplay -u 1 -g 1 --network saibest.gz -o "-g -t 16 --noponder --lambda 0 --mu 0 --nrsymm -v 10000 -w" --network saibest.gz -o "-g -t 16 --noponder --lambda 0 --mu 0 --nrsymm -v 10000 -w"-- /home/kenny/Desktop/sai/build/sai -- /home/kenny/Desktop/sai/build/sai

from sai.

@akdtg I tried it, as above.

from sai.

Is the weight file in the same directory as the validation file? If not, you have to give the path to it. Also, there should be a space before -- /home... for the first binary. I'm using windows, so I can't verify if that works.

from sai.

from sai.