in skadi/decoder/recv_prop/float.py you can find the following:

    def _decode_normal(self, stream):
        """
        Decode 'normal' float, which appears to be 11 low bits in a float
        multiplied by specific float-encoding-related values.

        Arguments:
        stream -- a Stream (skadi.io.stream)

        Returns a float.

        """
        s = stream.read_numeric_bits(1) # sign
        l = stream.read_numeric_bits(11) # low
        b = bytearray(0, 0, l & 0x0000ff00, l & 0x000000ff)
        v = struct.unpack('f', b)[0]

        # not sure this is ever called. what does bitshifting a float mean?
        if v >> 31:
            v += 4.2949673e9

        v *= 4.885197850512946e-4

        return v * -1 if s else v

which puts the eleven bits read in l into the bytearray b.
However, the third value put in, l & 0x0000ff00 will be zero all the time.

Proposed fix:
put (l & 0x0000ff00) >> 8 into the third byte.

When iterating through a specific replay, I get

 Traceback (most recent call last):
  File "hooks.py", line 83, in <module>
    print(successful_hooks(game))
  File "hooks.py", line 15, in successful_hooks
    for tick, user_messages, game_events, world, modifiers in game.stream(tick=0):
  File "build/bdist.linux-x86_64/egg/skadi/demo.py", line 136, in __iter__
  File "build/bdist.linux-x86_64/egg/skadi/demo.py", line 186, in advance
  File "build/bdist.linux-x86_64/egg/skadi/engine/user_message.py", line 67, in parse
  File "/scratch/02213/sh37476/vis/evaluate/dota-analysis/lib/python2.7/site-packages/google/protobuf/message.py", line 182, in ParseFromString
    self.MergeFromString(serialized)
  File "/scratch/02213/sh37476/vis/evaluate/dota-analysis/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 795, in MergeFromString
    if self._InternalParse(serialized, 0, length) != length:
  File "/scratch/02213/sh37476/vis/evaluate/dota-analysis/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 827, in InternalParse
    pos = field_decoder(buffer, new_pos, end, self, field_dict)
  File "/scratch/02213/sh37476/vis/evaluate/dota-analysis/lib/python2.7/site-packages/google/protobuf/internal/decoder.py", line 410, in DecodeField
    field_dict[key] = local_unicode(buffer[pos:new_pos], 'utf-8')
UnicodeDecodeError: 'utf8' codec can't decode byte 0xd1 in position 30: unexpected end of data

http://s000.tinyupload.com/?file_id=91725447477505323662

Does the "In a Hurry" section run? Using a demo of mine and the test demo, I do not see ehandle or state, but could be missing something. I am poking around the objects and not seeing those attributes.

with io.open('tests/data/test.dem', 'r+b') as infile:
  demo = rd.construct(infile)
  stream=demo.stream()
  for tick, string_tables, world in stream:
    print dir(world), world.items()

['class', 'delattr', 'dict', 'doc', 'format', 'getattribute', 'hash', 'init', 'iter', 'module', 'new', 'reduce', 'reduce_ex', 'repr', 'setattr', 'sizeof', 'slotnames', 'str', 'subclasshook', 'weakref', 'by_cls', 'by_dt', 'by_ehandle', 'by_index', 'classes', 'create', 'delete', 'fetch_cls', 'fetch_recv_table', 'find', 'find_all_by_cls', 'find_all_by_dt', 'find_by_cls', 'find_by_dt', 'find_index', 'recv_tables', 'update']
Traceback (most recent call last):
File "", line 5, in
AttributeError: 'World' object has no attribute 'items'

Demo: http://replay111.valve.net/570/271123757_1205152291.dem.bz2

Loading up the demo, the following throws a key error:

s = demo.stream(tick=23174)
i = iter(s)
i.next()

However, doing something like this:

s = demo.stream(tick=20000)
for state in s:
    if state[0] > 23200:
        break

doesn't throw.

The index that is being thrown on in 1132. In tick 23174, if you go directly there, that index does not exist. However, if you create the stream at tick 20000 and then work your way forwards, it is created at 23174. Looking up the recv_table of the entity shows that it is a lane creep.

On Ubuntu 13.04, Python 2.7

Using python setup.py build_ext --inplace produced the following, noting that the .c file 'cythonize' was meant to produce did not exist:

missing cimport in module 'skadi.io': skadi/io/unpacker/cProp.pyx
missing cimport in module 'skadi.io': skadi/io/unpacker/cEntity.pyx
running build_ext
building 'skadi.io.cBitstream' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c skadi/io/cBitstream.c -o build/temp.linux-x86_64-2.7/skadi/io/cBitstream.o
x86_64-linux-gnu-gcc: error: skadi/io/cBitstream.c: No such file or directory
x86_64-linux-gnu-gcc: fatal error: no input files
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 4

After reading into Cython more, I learned I could generate the .c files from the .pyx files myself with cython -a whatever.pyx, so I used the following sequence of commands and everything worked:

cython -a skadi/io/cBitstream.pyx
cython -a skadi/io/unpacker/cProp.pyx
cython -a skadi/io/unpacker/cEntity.pyx
python setup.py build_ext --inplace

This is just a workaround however, setup.py still fails on a fresh repo for me.

Parsing fails when a user message of key 112 is found.

Hi,

While using your parser with newer replays I noticed my reliable and unreliable gold was 0 all the time.

After some digging through the code and the netprop explorer I noticed that the following property does work:

prop = RemoteProperty("DT_DOTA_Data{}".format(team))\
           .used_by(IndexedProperty("DT_DOTA_DataNonSpectator", "m_iReliableGold")) 

I hope it can help!

(Should have been on Tarrasque sorry about that)

<skadi.protoc.dota_usermessages_pb2.CDOTAUserMsg_ParticleManager object at 0x14f80210> is not too readable. Would be nice if print() would help inspecting the data a bit.

I saw this comment in the Reddit thread, so I'm adding this note for posterity.

I looked into it briefly, and the first hurdle is that the google protobuf library does not support Python 3. There are a couple community forks that tried to port it, as seen in this thread:

https://groups.google.com/forum/#!searchin/protobuf/python3/protobuf/Qalty-PenDA/NM_l7zzjaTwJ

but I was unable to compile either of them successfully (only spent a few minutes trying). I think they may be using an older version of protobuf (those two repos are a few months old with no updates).

There might only need to be minimal changes to skadi itself to get it working in Python 3, but the protobuf hurdle might be large enough to just close this as wontfix until google releases a Python 3 version of protobuf (if ever)