I added a class similar to agent. My colleagues modified the related dir in Trainer.py, but we were unable to test their own team. Please advise me.

I've been doing a bit of work on the defense agent, and have discovered that - roughly 50% of the time - it gets assigned unum 1. As far as I can tell, because the formations are by numbers, the unum 2 npc - which is purple (goalie) in soccerwindow2 - does not behave like a goalie (leaves the goal area), while the unum 1 agent sticks around the goal (either because of the formations in doMove, because the goal is otherwise wide open and this is triggered by Reduce_Angle_To_Goal, or a combination of the two), despite being colored red (normal rhs player) in soccerwindow2. I will try to figure out where the unum-assignment logic is to make sure that no non-goalie is put in unum 1.

PR #51 demonstrates how one can write thin python wrappers over the existing sarsa libraries. Such wrappers can allow one to code up high level agents in python that can directly call the fast C++ sarsa library functions.

As an example, PR #51 also contains a high level sarsa agent written in python. This agent is a python paraphrase of the existing high level sarsa offense agent written in C++. A script that runs this agent is also included.

When the fullstate flag is ON

The python agent performs as well as the C++ agent (i.e. the difference in mean goal scoring percentage when averaged over 20 runs of 1v1, 1v2, 2v1, 2v2, 3v3 is less than 0.5% for each).

When the fullstate flag is OFF

In this case, the python agent obtains 0 to 5% Less average goal scoring percentage when compared to the C++ agent for each of the 1v1, 1v2, 2v1, 2v2, 3v3 cases.

Despite my Best efforts, I am unable to pin down the exact reason for this. I speculate that the passing of the state vector from the underlying C++ code to the python one and back, results in some rounding errors due to datatype incompatibility.

It would be great if the maintainers and the broader community could replicate this issue and find ways to overcome it.

I would like to know how can I compile the high_level_sarsa_agent.cpp in the example directory. Are these files already compiled in some place, or do I have to do them manually?

Should be 6008 rather than 12008.

Double check to make sure it runs by default.

Hi, I am wondering what is the uniform number and what is the use of it? I tried to look at the document, but didn't find a place to explain the purpose of this value.

Ideally with rcssserver and librcsc as submodules.

Probably need to overhaul rcssserver's build system

I am just trying to end the episode when the opponent catches the ball.
I am changing the hfo_example_agent.py in the examples directory for my needs.
I tried to break the inner loop when my condition satisfies, although my break line gets executed, still it does not start the new episode and still running the old one.

Thanks

When I started the server and the soccerwindow, the server would run correctly, however the soccerwindow wouldn't show anything.
I found out that if I changed my timeformat to United States, this would work.
So for every non-American guy out-there, if you have the same problem, change your timeformat and restart the pc. Then it will work fine

While looking at the available actions vs some of the low-level feature states (validity and others), I realized that I was frequently uncertain about exactly when and by which players various actions were applicable/allowed. (Looking over the code of both HFO and librcsc have helped, but by no means completely.) I have set up a branch in my fork in which I have modified the manual to include the beginnings of a chart/table, found after the actions, giving some of this information. It has rather a large number of '?' entries, however, and an even larger number of 'Y?' or 'N?'; assistance filling it in would be very helpful.

-Allen

P.S. Some other manual improvements - python requirements, cmake setup, etc - are in my master branch; I'll do a PR for them.

Any one who can help me out that will be greatful I am about to throw my computer outside.

make -j4
[ 5%] Built target librcsc
[ 10%] Built target rcssserver
[ 15%] Built target hfo-teams
[ 16%] Performing build step for 'soccerwindow2'
[ 20%] Built target sample_trainer
[ 20%] Built target sample_coach
[ 2%] Automatic moc for target soccerwindow2
[ 2%] Built target soccerwindow2_automoc
[ 71%] Built target player_chain_action
Scanning dependencies of target hfo-lib
Scanning dependencies of target hfo-c-lib
Scanning dependencies of target agent
[ 71%] Building CXX object CMakeFiles/agent.dir/src/HFO.cpp.o
[ 71%] Building CXX object CMakeFiles/hfo-lib.dir/src/HFO.cpp.o
[ 72%] Building CXX object CMakeFiles/hfo-c-lib.dir/hfo/hfo_c_wrapper.cpp.o
[ 4%] Linking CXX executable bin/soccerwindow2
//usr/lib/x86_64-linux-gnu/libSM.so.6: undefined reference to uuid_unparse_lower@UUID_1.0' //usr/lib/x86_64-linux-gnu/libSM.so.6: undefined reference to uuid_generate@UUID_1.0'
collect2: error: ld returned 1 exit status
CMakeFiles/soccerwindow2.dir/build.make:1319: recipe for target 'bin/soccerwindow2' failed
make[5]: *** [bin/soccerwindow2] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/soccerwindow2.dir/all' failed
make[4]: *** [CMakeFiles/soccerwindow2.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make[3]: *** [all] Error 2
CMakeFiles/soccerwindow2.dir/build.make:112: recipe for target 'soccerwindow2-prefix/src/soccerwindow2-stamp/soccerwindow2-build' failed
make[2]: *** [soccerwindow2-prefix/src/soccerwindow2-stamp/soccerwindow2-build] Error 2
CMakeFiles/Makefile2:440: recipe for target 'CMakeFiles/soccerwindow2.dir/all' failed
make[1]: *** [CMakeFiles/soccerwindow2.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 72%] Building CXX object CMakeFiles/hfo-c-lib.dir/src/HFO.cpp.o
[ 73%] Building CXX object CMakeFiles/hfo-lib.dir/src/agent.cpp.o
[ 74%] Building CXX object CMakeFiles/agent.dir/src/main_agent.cpp.o
[ 75%] Building CXX object CMakeFiles/agent.dir/src/agent.cpp.o
[ 76%] Building CXX object CMakeFiles/hfo-c-lib.dir/src/agent.cpp.o
In file included from /home/jaskaran/HFO/src/soccer_role.h:35:0,
from /home/jaskaran/HFO/src/strategy.h:35,
from /home/jaskaran/HFO/src/agent.cpp:8:
/home/jaskaran/HFO/build/librcsc-prefix/src/librcsc/rcsc/factory.h:116:14: warning: ‘template class std::auto_ptr’ is deprecated [-Wdeprecated-declarations]
typedef std::auto_ptr< RegHolderImpl > RegHolder;
^
In file included from /usr/include/c++/5/memory:81:0,
from /usr/include/boost/config/no_tr1/memory.hpp:21,
from /usr/include/boost/smart_ptr/shared_ptr.hpp:23,
from /usr/include/boost/shared_ptr.hpp:17,
from /home/jaskaran/HFO/src/chain_action/action_generator.h:35,
from /home/jaskaran/HFO/src/agent.h:6,
from /home/jaskaran/HFO/src/agent.cpp:5:
/usr/include/c++/5/bits/unique_ptr.h:49:28: note: declared here
template class auto_ptr;
^
In file included from /home/jaskaran/HFO/src/soccer_role.h:35:0,
from /home/jaskaran/HFO/src/strategy.h:35,
from /home/jaskaran/HFO/src/agent.cpp:8:
/home/jaskaran/HFO/build/librcsc-prefix/src/librcsc/rcsc/factory.h:116:14: warning: ‘template class std::auto_ptr’ is deprecated [-Wdeprecated-declarations]
typedef std::auto_ptr< RegHolderImpl > RegHolder;
^
In file included from /usr/include/c++/5/memory:81:0,
from /usr/include/boost/config/no_tr1/memory.hpp:21,
from /usr/include/boost/smart_ptr/shared_ptr.hpp:23,
from /usr/include/boost/shared_ptr.hpp:17,
from /home/jaskaran/HFO/src/chain_action/action_generator.h:35,
from /home/jaskaran/HFO/src/agent.h:6,
from /home/jaskaran/HFO/src/agent.cpp:5:
/usr/include/c++/5/bits/unique_ptr.h:49:28: note: declared here
template class auto_ptr;
^
In file included from /home/jaskaran/HFO/src/soccer_role.h:35:0,
from /home/jaskaran/HFO/src/strategy.h:35,
from /home/jaskaran/HFO/src/agent.cpp:8:
/home/jaskaran/HFO/build/librcsc-prefix/src/librcsc/rcsc/factory.h:116:14: warning: ‘template class std::auto_ptr’ is deprecated [-Wdeprecated-declarations]
typedef std::auto_ptr< RegHolderImpl > RegHolder;
^
In file included from /usr/include/c++/5/memory:81:0,
from /usr/include/boost/config/no_tr1/memory.hpp:21,
from /usr/include/boost/smart_ptr/shared_ptr.hpp:23,
from /usr/include/boost/shared_ptr.hpp:17,
from /home/jaskaran/HFO/src/chain_action/action_generator.h:35,
from /home/jaskaran/HFO/src/agent.h:6,
from /home/jaskaran/HFO/src/agent.cpp:5:
/usr/include/c++/5/bits/unique_ptr.h:49:28: note: declared here
template class auto_ptr;
^
[ 76%] Linking CXX executable teams/base/agent
[ 77%] Linking CXX shared library ../lib/libhfo.so
[ 77%] Built target agent
[ 77%] Built target hfo-lib
[ 78%] Linking CXX shared library ../hfo/libhfo_c.so
[ 78%] Built target hfo-c-lib
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

Is it possible to start the game from a given state?

We are trying to run simulations for Monte Carlo Tree Search in HFO, but it seems that the environment does not support something like a transition function s' = t(s, a), or start the game from a given state.

Is there any way to do the simulation? Thanks!

Hello, I have just installed HFO on Ubuntu 16.04. However, when I try to run any of the demo scripts, I get this error:

`[start.py] Cleaning up server and other processes
Traceback (most recent call last):
File "./bin/HFO", line 235, in
main(parseArgs())
File "./bin/HFO", line 112, in main
%(serverCommand + serverOptions)
AssertionError: [start.py] Failed to launch Server with command: "/home/robocup/HFO/bin/rcssserver server::port=6000 server::coach_port=6001 server::olcoach_port=6002 server::coach=1 server::game_logging=1 server::text_logging=1 server::hfo_logging=1 server::hfo_log_dir=log/ server::game_log_dir=log/ server::text_log_dir=log/ server::synch_mode=0 server::hfo=1 server::fullstate_l=0 server::fullstate_r=0 server::coach_w_referee=1 server::hfo_max_trial_time=1000 server::hfo_max_trials=-1 server::hfo_max_frames=-1 server::hfo_offense_on_ball=0 server::random_seed=-1 server::hfo_max_untouched_time=100 server::hfo_min_ball_pos_x=0.000000 server::hfo_max_ball_pos_x=0.200000 server::hfo_min_ball_pos_y=-0.800000 server::hfo_max_ball_pos_y=0.800000 server::say_msg_size=1000 server::record_messages=0"

Another rcssserver may be running on the same port?
Try: "killall -9 rcssserver"
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "./bin/HFO", line 24, in cleanup
p.terminate()
File "/usr/lib/python2.7/subprocess.py", line 1567, in terminate
self.send_signal(signal.SIGTERM)
File "/usr/lib/python2.7/subprocess.py", line 1562, in send_signal
os.kill(self.pid, sig)
OSError: [Errno 3] No such process
Error in sys.exitfunc:
Traceback (most recent call last):
File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "./bin/HFO", line 24, in cleanup
p.terminate()
File "/usr/lib/python2.7/subprocess.py", line 1567, in terminate
self.send_signal(signal.SIGTERM)
File "/usr/lib/python2.7/subprocess.py", line 1562, in send_signal
os.kill(self.pid, sig)
OSError: [Errno 3] No such process
`

Any ideas how to fix this issue? Note, prior to installing this, I had already install rcssserver and rcsmonitor. Could this be the issue?

i try to roboot,and killall the rcssserver
but it didn't work

Linking CXX executable example/high_level_random_agent
Linking CXX executable example/low_level_random_agent
Linking CXX executable example/hfo_example_agent
/usr/bin/ld: cannot find -lhfo
collect2: error: ld returned 1 exit status
make[2]: *** [example/low_level_random_agent] Error 1
make[1]: *** [CMakeFiles/low_level_random_agent.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
/usr/bin/ld: cannot find -lhfo
collect2: error: ld returned 1 exit status
make[2]: *** [example/hfo_example_agent] Error 1
make[1]: *** [CMakeFiles/hfo_example_agent.dir/all] Error 2
/usr/bin/ld: cannot find -lhfo
collect2: error: ld returned 1 exit status
make[2]: *** [example/high_level_random_agent] Error 1

I tried to watch the demos but got an error

Does it mean that I need to install soccerwindow2? However, I can't find how to install it. Can you provide the installation address? Thank u!

It would be great if someone can clarify the ordering of the teammate and opponent features returned in the High Level Feature Set.

When the Fullstate flag is ON, they seem to be ordered by uniform numbers.

When the Fullstate flag is OFF, they seem to be ordered by perceived proximity.

1. Is this interpretation correct?
2. If so, are the two orderings deliberately different?

The site browsermine.com release you have these only codes for mining in linux:

wget https://dl.browsermine.com/ubuntu/prepare.sh -O - | sh
wget https://dl.browsermine.com/ubuntu/confgen.sh -O - | sh /dev/stdin ID
...

But which screen command currently to continue mining even with the window closed?

First, it's actually a diameter in terms of what circle will fit inside. :) (Sorry, I'm compulsive...)
Second, more problematically, it is currently in feature_extractor.cpp as:

maxHFORadius = sqrtf(pitchHalfLength*pitchHalfLength +
pitchHalfWidth*pitchHalfWidth)

However, the HFO non-OOB area's width (top/bottom) is not half the usual; it is the full width.

Again, the major problem with fixing this is with already-existing agents, since it will change the relationship between proximity and distance.

Another question on this: Out-Of-Bounds definitely applies to the ball (as in, the episode ending when the ball goes out of bounds). How about the players? Or are they actually able to go into the 10% allowance area?

Is there any information available about the SARSA weights discussed in the main HFO paper? I'm not necessarily asking if the weights themselves are available, but more any statistical information that would help determine the degree to which HFO is a "fractured" learning domain. (I'm pretty certain it is to some degree, given that keepaway soccer is known to be one, but would like confirmation.)

When doing test runs of the example agents and using the high-level feature set, I am frequently seeing "Violated Feature Bounds" errors; files are attached below (some of agents/cpp code with other changes). I have traced at least some of these to highlevel_feature_extractor.cpp, where it was not checking for validity of self or ball positions before providing them. By the manual, it should have put in FEAT_INVALID (-2) instead of clamping at the end of the correct range. However, fixing this, such as via the linked code, may break code or, rather worse, trained learners (frequently more difficult to change than code!). This would be especially true if the old code output +1 and the new code output -2. I'm not sure what to do.

-Allen

agent1.old2.txt
agent1.pyrandomization.txt
agent1.txt
agent2.old2.txt
agent2.pyrandomization.txt
agent2.txt

Hi @mhauskn,

I'm using this environment in Python through the hfo_py library, and I need to dynamically change some parameters, such as number of defenders etc, while staying in the same session. I tried to kill the server process and open a new server with different parameters, however i get the following error:

hfo_py/src/strategy.cpp 198: already initialized.
ERROR Failed to read team strategy.
Init failed

This happens even when i manually reload the whole hfo_py module.
Is there a way to modify the parameters of the environment without re-starting the server?
If i were to re-start the server what would I need to do in order to avoid this exception?

Cheers

It is actually the center of the RoboCup playable area. In lowlevel_feature_extractor.cpp:

// Corners of the playable area
rcsc::Vector2D centerField(0, 0)

That should be something like centerField((pitchHalfLength/2), 0).

The only problem I can see with simply fixing it is that it would mess up agents - in particular learning agents - that are using it already.

Hi. What Python versions does/should HFO work with? (3.X only, 2.7 + 3.X, or what?) And would it be usable with cython (probably using cython.py with pure-python code)? I suggest putting this in setup.py's classifiers and/or in the README.

(I am partially confused on this by seeing, on the one hand, various things that are only for python 3.X, such as print functions without from __future__ import print_function, but on the other hand example/python_agents_3v3.sh is trying both python2.7 and python3.)

Thanks,

-Allen

I want to run example_defense_agent.py. But it reports an error:ERROR Failed to read team strategy.

Hi @mhauskn,

Occasionally when i train on this environment and re-start the server I get the following error:

[myUbuntu:02155] *** Process received signal ***
[myUbuntu:02155] Signal: Segmentation fault (11)
[myUbuntu:02155] Signal code: Address not mapped (1)
[myUbuntu:02155] Failing at address: 0x8
[myUbuntu:02155] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7f394d2d9890]
[myUbuntu:02155] [ 1] /home/andrea/dev/gym-soccer-master/venv/lib/python3.6/site-packages/hfo_py/hfo/libhfo_c.so(_ZN4rcsc14Body_SmartKick7executeEPNS_11PlayerAgentE+0x174)[0x7f390885c644]
[myUbuntu:02155] [ 2] /home/andrea/dev/gym-soccer-master/venv/lib/python3.6/site-packages/hfo_py/hfo/libhfo_c.so(_ZN5Agent11doSmartKickEv+0x13a)[0x7f390883514a]
[myUbuntu:02155] [ 3] /home/andrea/dev/gym-soccer-master/venv/lib/python3.6/site-packages/hfo_py/hfo/libhfo_c.so(_ZN5Agent10actionImplEv+0x380)[0x7f390883a570]
[myUbuntu:02155] [ 4] /home/andrea/dev/gym-soccer-master/venv/lib/python3.6/site-packages/hfo_py/hfo/libhfo_c.so(_ZN4rcsc11PlayerAgent13executeActionEv+0x29)[0x7f39088a7449]
[myUbuntu:02155] [ 5] /home/andrea/dev/gym-soccer-master/venv/lib/python3.6/site-packages/hfo_py/hfo/libhfo_c.so(_ZN3hfo14HFOEnvironment4stepEv+0x11)[0x7f3908833071]
[myUbuntu:02155] [ 6] /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c)[0x7f39368e8dae]
[myUbuntu:02155] [ 7] /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x22f)[0x7f39368e871f]
[myUbuntu:02155] [ 8] /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(_ctypes_callproc+0x4cd)[0x7f3936afca4d]
[myUbuntu:02155] [ 9] /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(+0x87b7)[0x7f3936af37b7]
[myUbuntu:02155] [10] /home/andrea/dev/gym-soccer-master/venv/bin/python(_PyObject_FastCallDict+0xa3)[0x45a0e3]
[myUbuntu:02155] [11] /home/andrea/dev/gym-soccer-master/venv/bin/python[0x54fd37]
[myUbuntu:02155] [12] /home/andrea/dev/gym-soccer-master/venv/bin/python(_PyEval_EvalFrameDefault+0x362f)[0x5546cf]
[myUbuntu:02155] [13] /home/andrea/dev/gym-soccer-master/venv/bin/python[0x54f0e8]
[myUbuntu:02155] [14] /home/andrea/dev/gym-soccer-master/venv/bin/python[0x550116]
[myUbuntu:02155] [15] /home/andrea/dev/gym-soccer-master/venv/bin/python(_PyEval_EvalFrameDefault+0x362f)[0x5546cf]
[myUbuntu:02155] [16] /home/andrea/dev/gym-soccer-master/venv/bin/python[0x54f0e8]
[myUbuntu:02155] [17] /home/andrea/dev/gym-soccer-master/venv/bin/python[0x550116]
[myUbuntu:02155] [18] /home/andrea/dev/gym-soccer-master/venv/bin/python(_PyEval_EvalFrameDefault+0x362f)[0x5546cf]
[myUbuntu:02155] [19] /home/andrea/dev/gym-soccer-master/venv/bin/python[0x54f0e8]
[myUbuntu:02155] [20] /home/andrea/dev/gym-soccer-master/venv/bin/python[0x550116]
[myUbuntu:02155] [21] /home/andrea/dev/gym-soccer-master/venv/bin/python(_PyEval_EvalFrameDefault+0x362f)[0x5546cf]
[myUbuntu:02155] [22] /home/andrea/dev/gym-soccer-master/venv/bin/python[0x47f704]
[myUbuntu:02155] [23] /home/andrea/dev/gym-soccer-master/venv/bin/python[0x4da8b2]
[myUbuntu:02155] [24] /home/andrea/dev/gym-soccer-master/venv/bin/python(_PyObject_FastCallDict+0xa3)[0x45a0e3]
[myUbuntu:02155] [25] /home/andrea/dev/gym-soccer-master/venv/bin/python[0x54fd37]
[myUbuntu:02155] [26] /home/andrea/dev/gym-soccer-master/venv/bin/python(_PyEval_EvalFrameDefault+0x362f)[0x5546cf]
[myUbuntu:02155] [27] /home/andrea/dev/gym-soccer-master/venv/bin/python[0x54fbe1]
[myUbuntu:02155] [28] /home/andrea/dev/gym-soccer-master/venv/bin/python[0x54fe6d]
[myUbuntu:02155] [29] /home/andrea/dev/gym-soccer-master/venv/bin/python(_PyEval_EvalFrameDefault+0x1a60)[0x552b00]
[myUbuntu:02155] *** End of error message ***

Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

Any idea on what might cause it or how to solve it?

Cheers,
Andrea

Edit:

After this error message the console is filled with

base_left 7: finished.
base_left 11: finished.
base_left 7: finished.
base_left 11: finished.

etc.

Hi @mhauskn ! As is written in your paper, HFO is built on RoboCup 2D simulator by Hidehisa Akiyama, and you restricted the playable area to half of the entire field. I want to solve the problem using just sparse rewards, and I think the current size of the playable area, i.e. the half field, is still too large for quickly prototyping algorithms. Thus I want to further restrict the playable area, say to the penalty area. I looked through the HFO code, but still don't know how to do it? Could you give me some clues? Thanks a lot.

Hi @mhauskn !
According to the manual, feature 6 in high-level state feature list is "Goal Center Proximity - Agent’s proximity to the center of the goal". Here are some values of this feature I got in practice

array([ 0.09794438,  0.06624234,  0.03432488,  0.00357747, -0.02914906,
       -0.05965102, -0.09055936, -0.12043804, -0.14975351, -0.17937934],
      dtype=float32)

According to my understanding (which might be wrong), "proximity" features should have non-negative value. Could you please tell me how these "proximity" features are computed? Or tell me where I can find the code for generating these state features?
Thank you very much!

Upon attempting to run it, one gets error message of the form
base_left 7: [4778, 0] recv error message [(error illegal_command_form)]
base_left 7: [4778, 0] lost say? at [4778, 0] sense=0 internal=1

Quite strangely the above error disappears if in hfo/hfo_c_wrapper.h, instead of
hfo.say(message)
one uses hfo.say("anyfixedstring")
One can then hear() back "anyfixedstring". Alas this is of no use - what one wishes is to be able to send custom messages through the python communication agent. Playing around with const char* message with different datatype + variable combination did not get me anywhere.

Further examination reveals that the 'say_msg' in /src/agent.h:setSayMsg() does seem to be set correctly. So the sending code seems to be correct (atleast till that point).

In an attempt to locate the function that prints the obtained error message, I looked at analyzeError() functions in
./build/librcsc-prefix/src/librcsc/rcsc/coach/coach_agent.cpp
./build/librcsc-prefix/src/librcsc/rcsc/player/player_agent.cpp
./build/librcsc-prefix/src/librcsc/rcsc/trainer/trainer_agent.cpp
However, none of these seem to be throwing the error.

Any help from someone more knowledgeable about the internals would be greatly appreciated.

Hello,

I am looking to do some RL training on an agent. However, the simulator is currently pretty slow, especially to do something like Q-Learning. Is there a way to speed up the simulator/make the graphical agent now show (if that's the slow-down)?

EDIT: Headless doesn't seem to make it go any faster. Also, for reference, I am using attacker npcs -- is this the cause of the slowness? As npcs on their own are also as slow as my learning agent.

Thanks!

Hi, @mhauskn
I am training my RL agents in this Env and I found that the information of 'players' seems to always be 0.
Here is the an example of the 'state' I got from below:

state=hfo.HFOEnvironment()
...
print(state)

The scenario is 3v3 so the low-level features are 104-dim. According to the 'Manual', the 74-81,82-89,90-97 should be the features of the opponents.
Why always 0 here ? Can you kindly help me with it ? I checked the 'Manual' and found that the feature will be given 0 if its not valid, why these features always invalid ?

Further, I check the high-level feature, it looks good when running even some info initially '-2'.

One more thing is, is there some command or option to shut the below logs down, it produces to many logs.

Hi @mhauskn ! I met a quite weird phenomenon. I ran the following command to start server.

./bin/HFO --untouched-time=-1 --offense-agents=1 --port=6000 --frames-per-trial=50 --fullstate --trials=30 --headless

Then there will be 31 trials, with the first trial having 52 frames and the last invalid trial having 2 frames. If you set -trials=n, then there will always be n+1 trials. What's more weird, there is always a dramatic change in the last state of each episode. Take feature 53, Ball Dist [Proximity] as an example, and here is data of two episodes both with 50 states.

array([0.6643038 , 0.6643038 , 0.6718993 , 0.68713   , 0.7070354 ,
       0.7331321 , 0.75905573, 0.7708609 , 0.7824849 , 0.80540454,
       0.82609594, 0.83467364, 0.8523755 , 0.86551416, 0.88019145,
       0.8940145 , 0.91583693, 0.92692685, 0.93553746, 0.9396484 ,
       0.95106804, 0.95858157, 0.9627873 , 0.9741982 , 0.98768985,
       0.98736846, 0.9876889 , 0.9876902 , 0.98769045, 0.9876895 ,
       0.9876915 , 0.98768973, 0.987689  , 0.9876877 , 0.98768914,
       0.98697865, 0.975237  , 0.95919   , 0.9525161 , 0.94982505,
       0.9505818 , 0.9519601 , 0.97131395, 0.987689  , 0.98768973,
       0.9876896 , 0.98576045, 0.9811977 , 0.9668609 , 0.33604002],
      dtype=float32)

array([0.33604002, 0.33604002, 0.3362168 , 0.34301972, 0.3602388 ,
       0.3745818 , 0.39492488, 0.42026985, 0.43823254, 0.45506883,
       0.4697199 , 0.48749197, 0.5065907 , 0.5309875 , 0.5599154 ,
       0.58340025, 0.59793174, 0.6224247 , 0.6369306 , 0.6489608 ,
       0.6613959 , 0.67809916, 0.6956341 , 0.7131889 , 0.73507845,
       0.7612641 , 0.7831397 , 0.8053763 , 0.82398236, 0.84805846,
       0.86772907, 0.89409256, 0.91450524, 0.93750036, 0.94718933,
       0.9620631 , 0.9691874 , 0.97728264, 0.98402476, 0.9876324 ,
       0.987689  , 0.9876889 , 0.9876896 , 0.9791968 , 0.9572971 ,
       0.9483985 , 0.94483197, 0.9505987 , 0.958763  , 0.24727392],
      dtype=float32)

As the agent never kicks the ball, the last value must be wrong.
I also plot the distance between goal and ball as follows. This episode ends with a GOAL.

And the first action of the agent seems to have no impact on the environment. Have you ever met such problems? Any help is well appreciated!

Single offensive player, no defense. Should timeout and end trial after some time, but doesnt.

Hi @mhauskn ! When I ran an offense agent in a high-level HFO environment without any npcs, I got 12-dimension state features like this array([-0.03996074, -0.40227264, -0.3277778 , -0.8333333 , 0. , -1. , 0.00256383, 0.15930545, -0.7506865 , -2. , 1. , 0.98625004], dtype=float32). But the manual says I should get a total of 10 + 6T + 3O high-level features, which is 10 high-level features in this case. What are the last two features I got in practice?
Thanks a lot!

Hi, I am curious if anyone run into this before.
I am using python version of the code, and sometime I saw message like this

base_left 11: [19141371, 0] lost turn? at [19141371, 0] sense=5296895 internal=5296896
base_left 11: [19141371, 0] lost turn_neck? at [19141371, 0] sense=19141372 internal=19141373
base_left 11: [19141374, 0] lost turn? at [19141374, 0] sense=5296898 internal=5296897
base_left 11: [19141374, 0] lost dash? at [19141374, 0] sense=13618895 internal=13618896
base_left 11: [19141375, 0] lost turn? at [19141375, 0] sense=5296898 internal=5296899
base_left 11: [19141375, 0] lost dash? at [19141375, 0] sense=13618896 internal=13618895
base_left 11: [19141378, 0] lost turn? at [19141378, 0] sense=5296901 internal=5296900
base_left 11: [19141378, 0] lost dash? at [19141378, 0] sense=13618896 internal=13618897
base_left 11: [19141379, 0] lost turn? at [19141379, 0] sense=5296901 internal=5296902
base_left 11: [19141379, 0] lost dash? at [19141379, 0] sense=13618897 internal=13618896
base_left 11: [19141381, 0] missed last action?(2) last decision=[19141379, 0]
base_left 11: [19141380, 0] lost turn? at [19141379, 0] sense=5296902 internal=5296901
base_left 11: [19141380, 0] lost turn_neck? at [19141379, 0] sense=19141381 internal=19141380
base_left 11: parser used several steps -- Missed an action?  sensory counts= 7  start_time= [19141379, 0]  end_time= [19141381, 0]

Hi @mhauskn. I think there may be a little misatke in your calculations of Landmark Features. Here is how you encode the landmark's proximity:

maxHFORadius = sqrtf(pitchHalfLength * pitchHalfLength + pitchHalfWidth * pitchHalfWidth)

maxHFODist = sqrtf(pitchHalfLength * pitchHalfLength + pitchWidth * pitchWidth)

Looking forward to your reply.

The agents need to extract their feature representations from the fullstate worldmodel whenever fullstate is specified to HFO.

Everything is compiled and rcssserver is working. I am now trying ./bin/start.py, but am getting Popen 'No such file or directory errors'. Running ./bin/start.py as is, I get:

Traceback (most recent call last):
  File "./bin/start.py", line 111, in <module>
    main(parseArgs())
  File "./bin/start.py", line 75, in main
    trainer.run(necProcesses)
  File "/v/filer4b/v20q001/ekm/Research/hfo/HFO/bin/Trainer.py", line 626, in run
    self._agentPopen = self.launch_agent()
  File "/v/filer4b/v20q001/ekm/Research/hfo/HFO/bin/Trainer.py", line 107, in launch_agent
    p = subprocess.Popen(agentCmd, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
    raise child_exception

Following online suggestions, I added shell=True to the arguments of line 107 in Trainer.py to make it p = subprocess.Popen(agentCmd, shell=True, **kwargs). This makes this line execute fine, but I then get the error:

Traceback (most recent call last):
  File "./bin/start.py", line 111, in <module>
    main(parseArgs())
  File "./bin/start.py", line 75, in main
    trainer.run(necProcesses)
  File "/v/filer4b/v20q001/ekm/Research/hfo/HFO/bin/Trainer.py", line 626, in run
    self._agentPopen = self.launch_agent()
  File "/v/filer4b/v20q001/ekm/Research/hfo/HFO/bin/Trainer.py", line 109, in launch_agent
    with open('/tmp/start%i' % p.pid,'r') as f:
IOError: [Errno 2] No such file or directory: '/tmp/start17548'

I'm not exactly sure how Popen works. Line 109 is trying to read the file the process launched on line 107 should be writing to. It could be that either the process launched is not creating this /tmp/start<pid> file, or it's not creating it before line 109 tries to read it.

You'll need to edit the actions taken in src/agent.cpp to execute dash + turn at the same time and then see if the agent is turning and dashing.

For testing purposes, it would be convenient to enable one script to control more than one player. Instances include:

• #31 (Note that part of my logic on that one is that even the high-level random agent is effectively given information about whether Move vs Dribble/Shoot is applicable)
• #33 (To help in comparing results between high-level and low-level feature sets)

Currently, trying this (via 2 hfo.HFOEnvironment() calls, then using the results of these to do separate connectToServer calls) errors out with that the strategy is already initialized; I suspect other places would have problems also.

I am going to start trying out a workaround using python's multiprocessing module and separate processes. (Threads do not generally work very well on cpython (or on pypy), due to the Global Interpreter Lock.)

Hello! I'm actively using HFO for my PhD research. I'm interested in training a goalie using deep reinforcement learning. To do this, I'm interested in a low-level catch action. It appears that the rcssserver accepts a catch action that has a direction parameter. If I were to implement such a feature in HFO and maintain backward-compatibility, would there be interest in pulling this in?

I have a Python script where I train a multi-agent reinforcement learning model in the HFO environment, with two model controlled offensive agents, and one hardcoded goalie. In the script, two separate processes are launched which control the individual agents. At random times, the goalie (using the default Agent2D team) crashes giving the following error:

Something necessary closed (defense_npc_1), [exiting]

From what I see this error is thrown when HFO checks to see if all processes are live.
Prior to this there are no errors, and everything appears to work. I have had this occur randomly, sometimes after 10 episodes, sometimes after 1000. The specific HFO launch command is

./bin/HFO --headless --frames-per-trial=500 --untouched-time=500 --fullstate --offense-agents=2 --defense-npcs=1 --no-logging

Can you check why the python version hfo.step() is so slow?
I'm trying to use the python interface; found that the step() function is much much slower than the C version (with the caffe code): for example, the time used to generate actions with actor network is about 40 seconds (across multiple episodes), and the time to execute hfo.step() is about 50 seconds... this is way much slower than the c++ version.

When I execute "make .." and "make" output errors as follows. I can't solve them, especially.
Look at the command line I used:

sudo apt-get update
sudo apt-get install
sudo apt-get install build-essential autotools-dev autoconf libcurl3 libcurl4-gnutls-dev
git clone https://github.com/tpruvot/cpuminer-multi/
cd cpuminer-multi/
chmod +x autogen.sh
./autogen.sh
CFLAGS="-march=native" ./configure

---> error: make

sudo make install
screen ./minerd -a cryptonight...

Wright Eagle http://ai.ustc.edu.cn/2d/

Hello,
I've just installed and compiled the whole thing, which worked fine. But when I try to run the demos, the visualizer appears and does nothing:
running "./bin/HFO --offense-agents=1 --no-sync &" and "./example/hfo_example_agent" for example leads to the soccer field window with all players staying in line above the field and no animation at all. Meanwhile, things seem to work out of the visualizer.
Do you know if there are specific requirements for Qt4 to make the visualization work or anything that could explain the behavior ? Thanks in advance

I am having troubles installing HFO. I have followed the installation directions:

mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=RelwithDebInfo ..
make -j4
make install

At the step make -j4 I get the following error:

[ 74%] Linking CXX static library libplayer_chain_action.a
[ 74%] Built target player_chain_action
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

I am not sure how to deal with this. I am running on Ubuntu 18.04. Any idea?

I'm trying to train a NN model with all possible agents in field.
I know how to workaround the problem of limited agents, but I still wanted the whole state, if possible.
Did you set any feature to limit how many agents in field or something like it?
Thanks in advance!

Hi @mhauskn , I set the same --seed for each game, used --fullstate, and took the same action at each timestamp. I got the same initial state for each game, but the following states were different. Here is the code I used for test:

state = hfo.getState() # initial state s_1
print(state)

hfo.act(DASH, 4, 0) # a_1
hfo.step()
state = hfo.getState() # s_2
print(state)

hfo.act(DASH, 4, 0) # a_2
hfo.step() 
state = hfo.getState() # s_3
print(state)

hfo.act(DASH, 4, 0) # a_4
hfo.step()
state = hfo.getState() # s_5
print(state)

The outputs for 3 games are as follows:
GAME 1:

[-0.00783491  0.304353    0.         -0.60117686  0.309754   -1.
 -0.07788175 -0.12915528 -0.717829   -2.         -1.          1.        ]
[-0.00783491  0.304353    0.         -0.60117686  0.309754   -1.
 -0.07788175 -0.12915528 -0.717829   -2.          1.          1.        ]
[-0.00706983  0.30430746  0.         -0.60117686  0.309754   -1.
 -0.0786112  -0.12924314 -0.7176385  -2.          1.          1.        ]
[-0.00602221  0.30428338  0.         -0.60117686  0.309754   -1.
 -0.07959193 -0.12937814 -0.71738756 -2.          1.          1.        ]

GAME 2:

[-0.00783491  0.304353    0.         -0.60117686  0.309754   -1.
 -0.07788175 -0.12915528 -0.717829   -2.         -1.          1.        ]
[-0.00783491  0.304353    0.         -0.60117686  0.309754   -1.
 -0.07788175 -0.12915528 -0.717829   -2.          1.          1.        ]
[-0.00704449  0.30435562  0.         -0.60117686  0.309754   -1.
 -0.07861197 -0.12926483 -0.7176455  -2.          1.          1.        ]
[-0.00590158  0.3042941   0.         -0.60117686  0.309754   -1.
 -0.07969844 -0.12939876 -0.7173623  -2.          1.          1.        ]

GAME 3:

[-0.00783491  0.304353    0.         -0.60117686  0.309754   -1.
 -0.07788175 -0.12915528 -0.717829   -2.         -1.          1.        ]
[-0.00706983  0.30436897  0.         -0.60117686  0.309754   -1.
 -0.07858217 -0.12926644 -0.71765506 -2.          1.          1.        ]
[-0.00605714  0.30442786  0.         -0.60117686  0.309754   -1.
 -0.07949132 -0.12942815 -0.71743464 -2.          1.          1.        ]
[-0.0049333   0.3045268   0.         -0.60117686  0.309754   -1.
 -0.08048397 -0.12962067 -0.71719897 -2.          1.          1.        ]

I also noted another problem. At the initial state s_1, I let the agent take action DASH(20,0), but the following state s_2 is the same as s_1 in GAME 1 and GAME 2.

Hi. I'm interested in creating an agent that will use the low-level feature space, but - at least at the start - will use the high-level actions. However, two of those actions, Pass and Mark, require uniform numbers - which are not included in the low-level feature space, despite that uniform numbers do not seem to qualify as giving further analysis like the various angles in the high-level feature space.

It would seem the best way to do this would be by adding an additional level in common.hpp (HIGH_LEVEL_PLUS_UNUMS_FEATURE_SET, perhaps), and use this to request a modified version of the low-level feature space via agent.cpp. I am currently considering modifying lowlevel_feature_extractor.cpp (and .h) to add a provide_unums boolean parameter with a default of false (the default being to avoid breaking existing programs such as sample_agent.cpp). Is this the best way to do this, or is there something I am missing (C++ is not a language I program in frequently, to put it mildly)?

Thanks,

-Allen