[Discussion] Better unit test framework about serpent-tools HOT 6 CLOSED

Did some digging today and I found that using __file__ is moderately common in some popular python packages

• pandas
• numpy
• matplotlib

So I think using __file__ might not be as big of a deal as I suspected. When it comes to storing the files, I think we can use something like pgkutil.get_data to load in the files. This would require we include these files in out setup.py.

from serpent-tools.

As we move towards files with larger arrays, i.e. xsplot, results, sensitivity, we may have to rethink how we unit test the readers with these large arrays. Currently, we manually type in the whole array, or sections of it test_history.py.

It has been proposed to use a pickled version of the reader that can be unpickled (loaded into memory) and test against that. However, the unpickling process has the potential to execute arbitrary code and can be a security risk. From the docs

Warning: The pickle module is not secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.

So this is not something we want.

One alternative would be to create compressed versions of the arrays using numpy.savez and then load the arrays themselves with numpy.load(<file>, pickle=False). Passing the pickle=False argument will not load numpy object arrays, meaning there is less risk of malicious code execution.

We can have a script that create these compressed files, and then list the hash/MD5 for the files in release notes, so people can verify the files we present and load.

from serpent-tools.

OK, how about this:

In general, almost all readers generate a dictionary consisting of named objects. At least, that's where the important data is.

You could create a dictionary of the contents of each object by using something along this lines of:

vardict = dict([(xs.name, vars(xs)) for xs in readerResults])

These can then be printed to a file, gzipped, and compared to the trusted results using diff. This in particular isn't incredibly pythonic but would be compact and secure.

from serpent-tools.

That's one method I'm leaning towards. The real issue I want to mitigate is having to write exceptionally large/multidimensional arrays, (a la #13 with upwards of 5 dimensions).

The numpy savez lets us write a collection of arrays to a file (even compressed) which can be loaded in a similar manner to what you describe.

 In [1]   import numpy
 In [2]   a1 = numpy.arange(9).reshape((3,3))
 In [3]   numpy.savez('out', a1, a1=a1)
 In [4]   out = numpy.load('out.npz')
 In [5]   out.keys()
 Out[5]  ['a1', 'arr_0']

from serpent-tools.

Hm, yeah, that would be nice. Have you looked at how efficient compressing these results actually gets? There's not any underlying structure so I'd guess they'd be only about 80% their original size.

from serpent-tools.

Closing this as we no longer require the use of __file__ since we bundle all matlab test files in the package now - #198 and #220 allows us to suppress and test messages from our logging module. Still open for suggestions on improving our testing, but this issue has served its purpose

from serpent-tools.