Comments (22)
I don't know if validation works with SAI, but one can use gogui-twogtp
from sai.
i guess irename sai.exe to leelaz.exe will let validation work:)
from sai.
i copied the sai.exe to leelaz directory, use following command works
validation -k sai40_lz24 -n 2ae.gz -o "-g -v 100 -r 5 -w" -n 338.gz -o "-g -v 100 -r 5 -w" -- sai -- sai
from sai.
did it work?
from sai.
yes
2ae.gz v 338.gz ( 33 games)
wins black white
2ae.gz 24 72.73% 12 75.00% 12 70.59%
338.gz 9 27.27% 4 25.00% 5 29.41%
16 48.48% 17 51.52%
from sai.
sai41 vs lz030
91 wins, 110 losses
The first net is worse than the second
c1d.gz v 273.gz ( 201 games)
wins black white
c1d.gz 91 45.27% 48 45.71% 43 44.79%
273.gz 110 54.73% 57 54.29% 53 55.21%
105 52.24% 96 47.76%
sai43 vs lz035
4 wins, 30 losses
The first net is worse than the second
d8a.gz v 300.gz ( 34 games)
wins black white
d8a.gz 4 11.76% 2 11.76% 2 11.76%
300.gz 30 88.24% 15 88.24% 15 88.24%
17 50.00% 17 50.00%
sai43 vs lz033
d8a.gz v aaa.gz ( 57 games)
wins black white
d8a.gz 31 54.39% 14 56.00% 17 53.13%
aaa.gz 26 45.61% 11 44.00% 15 46.88%
25 43.86% 32 56.14%
sai44 vs lz033
35d.gz v aaa.gz ( 35 games)
wins black white
35d.gz 27 77.14% 12 85.71% 15 71.43%
aaa.gz 8 22.86% 2 14.29% 6 28.57%
14 40.00% 21 60.00%
from sai.
Thank you. Please note, that we are already comparing SAI nets and LZ nets: the 10 games matches you will see from time to time are of that kind. We are currently on par with LZ036 apparently
from sai.
validation -k sai58_lz40 -n 39c.gz -o "-g -v 200 -r 5 -w" -n 300.gz -o "-g -v 200 -r 5 -w" -- sai -- sai
sai58 vs lz40
Stopping engine.
11 wins, 37 losses
The first net is worse than the second
39c.gz v 300.gz ( 48 games)
wins black white
39c.gz 11 22.92% 5 21.74% 6 24.00%
300.gz 37 77.08% 18 78.26% 19 76.00%
23 47.92% 25 52.08%
sai59 vs lz40
Stopping engine.
14 wins, 40 losses
The first net is worse than the second
c16.gz v 300.gz ( 54 games)
wins black white
c16.gz 14 25.93% 4 19.05% 10 30.30%
300.gz 40 74.07% 17 80.95% 23 69.70%
21 38.89% 33 61.11%
sai61 vs lz47
9df.gz v 830.gz ( 18 games)
wins black white
9df.gz 5 27.78% 2 25.00% 3 30.00%
830.gz 13 72.22% 6 75.00% 7 70.00%
8 44.44% 10 55.56%
from sai.
sai65 vs lz147
c99.gz v 830.gz ( 24 games)
wins black white
c99.gz 8 33.33% 5 35.71% 3 30.00%
830.gz 16 66.67% 9 64.29% 7 70.00%
14 58.33% 10 41.67%
from sai.
sai68 vs lz147
2aa.gz v 830.gz ( 28 games)
wins black white
2aa.gz 8 28.57% 4 28.57% 4 28.57%
830.gz 20 71.43% 10 71.43% 10 71.43%
14 50.00% 14 50.00%
28 games played.
from sai.
I run validation with the following command
validation -k dir --network saibest.gz...
but it says GTP: Unrecognized argument: saibest.gz
from sai.
please post your full command line
from sai.
./validation -k 10k177selfplay -u 1 -g 1 --network saibest.gz -o "-g -t 16 --noponder --lambda 0 --mu 0 --nrsymm -v 10000" --network saibest.gz -o "-g -t 16 --noponder --lambda 0 --mu 0 --nrsymm -v 10000"-- /home/kenny/Desktop/sai/build/sai -- /home/kenny/Desktop/sai/build/sai
from sai.
my command line works on windows
validation -k sai77_lz250 -n af4.gz -o "-g -v 400 -r 5 -w" -n networks/3d415846183a7f51a40aaa80a007d886668e759c37be8712febf09ec2823d257.gz -o "-g -v 1 -r 5 -w" -- sai -- sai
from sai.
i think you need add the -w in -o " ...."
from sai.
oh, it automatically put the net in the end of the option?
from sai.
It still can't work.
I tried to use the absolute path.
But the engine can run.
The error is still "GTP: Unrecognized argument: saibest.gz"
from sai.
I have never used validation and cannot give you advice, but I reopen the issue so that people can better see it and answer.
from sai.
@kennychenfs, as @l1t1 pointed out, you have to add '-w' right to the end of the options:
./validation -k 10k177selfplay -u 1 -g 1 --network saibest.gz -o "-g -t 16 --noponder --lambda 0 --mu 0 --nrsymm -v 10000 -w" --network saibest.gz -o "-g -t 16 --noponder --lambda 0 --mu 0 --nrsymm -v 10000 -w"-- /home/kenny/Desktop/sai/build/sai -- /home/kenny/Desktop/sai/build/sai
from sai.
@akdtg I tried it, as above.
from sai.
Is the weight file in the same directory as the validation file? If not, you have to give the path to it. Also, there should be a space before -- /home...
for the first binary. I'm using windows, so I can't verify if that works.
from sai.
from sai.
Related Issues (20)
- self-play games lost comments when I interrupt autogtp with the q command HOT 1
- consider to switch 20x256 net at number 512? HOT 6
- sai 510 p1600 has 1/10 chance to beat lz elf v2 p800 HOT 1
- what is the meaning of the experiment type of matchs HOT 5
- removing the old networks HOT 6
- to get the similar elo, the training games is about 1:2 of leelazero now
- Latest news from telegram channel HOT 16
- katago training so fast HOT 4
- all the elo of nets drop to 500 HOT 2
- network: 61be4874d51e397e8e58f raise *ERROR*: Could not talk to engine after launching. HOT 3
- What effect are the Lambda and Mu? HOT 2
- sai 0.18.1 run with problem on window7 32bits HOT 20
- sai 0.18.1 vs 0.17.6 (my impression) HOT 16
- Doesn't compile CPU only version HOT 1
- Is it possible to do 64-bit Linux binary CPU only release? HOT 4
- suggest use katago games to increase self play games HOT 2
- autogtp build error HOT 2
- Just suggestion: go to 40b or close... HOT 6
- Hope to support ONNX Runtime (Training version & Inferencing version) and DirectML HOT 1
- The 9x9 weighs was missed. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sai.