bangerth / sampleflow Goto Github PK

View Code? Open in Web Editor NEW

3.0 3.0 5.0 1.33 MB

SampleFlow -- a library for composing complex statistical sampling algorithms

License: GNU Lesser General Public License v2.1

C++ 98.52% CMake 1.48%

sampleflow's People

Contributors

Stargazers

Watchers

Forkers

mantautasrimkus davidaristoff dawsoneliasen tjhei dklong-csu

sampleflow's Issues

When test fails, don't remove results file

@dawsoneliasen mentions that that happens when a test fails, and so one can't create the correct output this way.

metropolis_hasting_producer_10 fails

@dawsoneliasen -- the metropolis_hasting_producer_10 fails on my system, because it generates 10 samples but the stored output file has 20 lines (or the other way around). Can you come up with a pull request that fixes this and for which you can confirm that when you run ctest that all tests succeed? Thanks!

All tests failing

When I run make test, I am seeing 100% tests failed.

If I run ctest -V -R mean_01, this is the output (other tests look similar):

UpdateCTestConfiguration  from :/home/dawson/projects/SampleFlow/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/dawson/projects/SampleFlow/DartConfiguration.tcl
Test project /home/dawson/projects/SampleFlow
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 24
    Start 24: mean_01

24: Test command: /usr/bin/cmake "--build" "/home/dawson/projects/SampleFlow" "--target" "run_mean_01"
24: Test timeout computed to be: 10000000
24: Scanning dependencies of target mean_01
24: [  0%] Building CXX object tests/CMakeFiles/mean_01.dir/mean_01.cc.o
24: [100%] Linking CXX executable mean_01
24: [100%] Built target mean_01
24: [100%] Running test <mean_01>...
24: /bin/sh: NUMDIFF_EXECUTABLE-NOTFOUND: command not found
24: make[3]: *** [tests/CMakeFiles/run_mean_01.dir/build.make:83: tests/mean_01.result] Error 127
24: make[3]: *** Deleting file 'tests/mean_01.result'
24: make[2]: *** [CMakeFiles/Makefile2:1913: tests/CMakeFiles/run_mean_01.dir/all] Error 2
24: make[1]: *** [CMakeFiles/Makefile2:1920: tests/CMakeFiles/run_mean_01.dir/rule] Error 2
24: make: *** [Makefile:942: run_mean_01] Error 2
1/1 Test #24: mean_01 ..........................***Failed    3.03 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   3.04 sec

The following tests FAILED:
	 24 - mean_01 (Failed)
Errors while running CTest

However, I can run the tests/mean_01 executable just fine.

I'm not sure when things stopped working, but I must have messed something up. Based on the output from ctest, it has something to do with the numdiff executable. I tried installing numdiff but there was no change.

MH sampler has a bug

The MH sampler class takes as one argument a function object that, given the current sample, returns a new trial sample. (What this function does is, in essence, draw a sample from a proposal distribution.)

This is all that is necessary for symmetric proposal distributions where the probability of going from x -> y is the same as from y -> x. But in other cases, one wants biased proposal distributions, and in that case, the MH algorithm doesn't just need to get the trial sample y, but also the ratio pi(y,x)/pi(x,y) of proposal ratios -- see the Wikipedia page.

I had forgotten about this when I wrote the MH class, but remembered in another context that this needs to be fixed: dealii/code-gallery#64 We need to apply a similar patch to SampleFlow as was done in the referenced PR.

CI follow-up steps

#113, #114, #115, #116 all set up the CI infrastructure. We should add two more steps to the script:

Check generating the documentation by calling make doc
Check that the indentation is correct. This steps requires installing the astyle tool in the right version, then calling make indent, and finally calling git diff-files --quiet to check whether there are any changes.

@dawsoneliasen -- feel free to tackle one or both of these issues in separate PRs :-)

Differential evaluation: Indexing error?

Follow-up to #138.

@dawsoneliasen : After #143, we have the following code block to pick two chains:

                  std::uniform_int_distribution<typename std::vector<OutputType>::size_type>
                  a_dist(0, n_chains - 2);

                  typename std::vector<OutputType>::size_type a = a_dist(rng);
                  if (a >= generation)                                           **********************************************************
                    a += 1;
                  const OutputType trial_a = current_samples[a];

                  std::uniform_int_distribution<typename std::vector<OutputType>::size_type>
                  b_dist(0, n_chains - 3);

                  typename std::vector<OutputType>::size_type b = b_dist(rng);
                  if (b >= std::max<typename std::vector<OutputType>::size_type>(a, chain))
                    b += 2;
                  else if (b >= std::min<typename std::vector<OutputType>::size_type>(a, generation))   ***************************
                    b += 1;
                  const OutputType trial_b = current_samples[b];

I think that in the two lines marked by asterisks, the occurrence of generation is wrong and it should have been chain. Do you agree? We want to exclude the current chain, after all.

Accept -max over -infinity

-max() might be zero probability, but it's still better than -infinity(). Accept samples if that happens. See

          const bool trial_sample_has_zero_probability
            = ((trial_log_likelihood == -std::numeric_limits<double>::max())
               ||
               (trial_log_likelihood == -std::numeric_limits<double>::infinity()));
          const bool current_sample_has_zero_probability
            = ((current_log_likelihood == -std::numeric_limits<double>::max())
               ||
               (current_log_likelihood == -std::numeric_limits<double>::infinity()));

          bool repeated_sample;
          if (!(trial_sample_has_zero_probability && !current_sample_has_zero_probability)

@dklong-csu -- FYI

When a consumer/producer goes away, sever its connections both up and downstream.

This avoids running into the problem that samples are sent into nirvana.

Give state-less objects (e.g. filters) move constructors.

This will make it possible to use them as temp objects in things such as

  producer << make_filter(13) << consumer;

This should apply to most filters. For the other classes, explicitly default or delete copy and move constructors.

Differential evaluation crossover_gap = 0

If crossover_gap in DifferentialEvaluationMetropolisHastings is equal to 0 -- i.e. if the DE-MC algorithm is always used instead of also having normal metropolis steps -- then the sampler attempts the operation generation % crossover_gap which ends up being generation % 0 and you get a divide by zero error.

Fix differential evolution acceptance probability

The MH sampler goes to length with corner cases where the likelihood is -inf or some such. The DE sampler does not. Copy that code over.

Implement full delayed rejection algorithm

The current delayed rejection in #136 is not complete because it assumes that the proposal distribution does not depend on the already rejected samples. The publication at https://arxiv.org/pdf/0904.2207.pdf gives a nice overview of how the acceptance probability for a second, third, ... sample has to be adjusted to take into account the delayed samples. We should fix this in our implementation.

Questions about the spurious auto-covariance consumer

@MantautasRimkus -- Looking through the code that implements the spurious autocovariance consumer, here are a couple of questions:

In the formulas in the class documentation, the gamma terms are defined with a factor 1/n times the sum that runs from 1 to n-l. But in the code, you use 1/(n-l) in all places I could find. I think the latter is actually correct. Do you agree? If so, can you update the formulas in the class documentation?
In the documentation, the last term for gamma_n(l) is \frac{n-l}{n} \bar{x}_n^T \bar{x}_n. I think this is also correct. But in the code, the update at the end of the get() function does not have the factor (n-l)/n: https://github.com/bangerth/SampleFlow/blob/master/include/sampleflow/consumers/spurious_autocovariance.h#L297 Here, we simply add \bar{x}_n^T \bar{x}_n. Do you agree that there is a bug here?

Error on Conversion Filter instantiation

Instantiating SampleFlow::Filters::Conversion results in an error.

6: In file included from /home/dawson/projects/SampleFlow/tests/conversion_01.cc:24:
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:58:49: error: ‘OutputType’ was not declared in this scope; did you mean ‘InputType’?
6:    58 |     class Conversion : public Filter<InputType, OutputType>
6:       |                                                 ^~~~~~~~~~
6:       |                                                 InputType
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:58:59: error: template argument 2 is invalid
6:    58 |     class Conversion : public Filter<InputType, OutputType>
6:       |                                                           ^
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:41: error: there are no arguments to ‘OutputType’ that depend on a template parameter, so a declaration of ‘OutputType’ must be available [-fpermissive]
6:    70 |         Conversion (const std::function<OutputType (const InputType &)> &conversion_function
6:       |                                         ^~~~~~~~~~
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:41: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:71: error: template argument 1 is invalid
6:    70 |         Conversion (const std::function<OutputType (const InputType &)> &conversion_function
6:       |                                                                       ^
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:41: error: there are no arguments to ‘OutputType’ that depend on a template parameter, so a declaration of ‘OutputType’ must be available [-fpermissive]
6:    70 |         Conversion (const std::function<OutputType (const InputType &)> &conversion_function
6:       |                                         ^~~~~~~~~~
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:71: error: template argument 1 is invalid
6:    70 |         Conversion (const std::function<OutputType (const InputType &)> &conversion_function
6:       |                                                                       ^
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:41: error: there are no arguments to ‘OutputType’ that depend on a template parameter, so a declaration of ‘OutputType’ must be available [-fpermissive]
6:    70 |         Conversion (const std::function<OutputType (const InputType &)> &conversion_function
6:       |                                         ^~~~~~~~~~
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:71: error: template argument 1 is invalid
6:    70 |         Conversion (const std::function<OutputType (const InputType &)> &conversion_function
6:       |                                                                       ^
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:41: error: there are no arguments to ‘OutputType’ that depend on a template parameter, so a declaration of ‘OutputType’ must be available [-fpermissive]
6:    70 |         Conversion (const std::function<OutputType (const InputType &)> &conversion_function
6:       |                                         ^~~~~~~~~~
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:71: error: template argument 1 is invalid
6:    70 |         Conversion (const std::function<OutputType (const InputType &)> &conversion_function
6:       |                                                                       ^
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:41: error: there are no arguments to ‘OutputType’ that depend on a template parameter, so a declaration of ‘OutputType’ must be available [-fpermissive]
6:    70 |         Conversion (const std::function<OutputType (const InputType &)> &conversion_function
6:       |                                         ^~~~~~~~~~
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:71: error: template argument 1 is invalid
6:    70 |         Conversion (const std::function<OutputType (const InputType &)> &conversion_function
6:       |                                                                       ^
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:27: error: invalid use of template-name ‘std::function’ without an argument list
6:    70 |         Conversion (const std::function<OutputType (const InputType &)> &conversion_function
6:       |                           ^~~
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:27: note: class template argument deduction is only available with ‘-std=c++17’ or ‘-std=gnu++17’
6: In file included from /usr/include/c++/9.3.0/functional:59,
6:                  from /usr/include/boost/smart_ptr/detail/shared_count.hpp:39,
6:                  from /usr/include/boost/smart_ptr/shared_ptr.hpp:28,
6:                  from /usr/include/boost/shared_ptr.hpp:17,
6:                  from /usr/include/boost/signals2/deconstruct.hpp:28,
6:                  from /usr/include/boost/signals2.hpp:15,
6:                  from /home/dawson/projects/SampleFlow/include/sampleflow/producer.h:20,
6:                  from /home/dawson/projects/SampleFlow/include/sampleflow/consumer.h:20,
6:                  from /home/dawson/projects/SampleFlow/include/sampleflow/consumers/stream_output.h:19,
6:                  from /home/dawson/projects/SampleFlow/tests/conversion_01.cc:23:
6: /usr/include/c++/9.3.0/bits/std_function.h:128:11: note: ‘template<class _Signature> class std::function’ declared here
6:   128 |     class function;
6:       |           ^~~~~~~~
6: In file included from /home/dawson/projects/SampleFlow/tests/conversion_01.cc:24:
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:70:40: error: expected ‘,’ or ‘...’ before ‘<’ token
6:    70 |         Conversion (const std::function<OutputType (const InputType &)> &conversion_function
6:       |                                        ^
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:89:35: error: ‘OutputType’ was not declared in this scope; did you mean ‘InputType’?
6:    89 |         boost::optional<std::pair<OutputType, AuxiliaryData> >
6:       |                                   ^~~~~~~~~~
6:       |                                   InputType
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:89:60: error: template argument 1 is invalid
6:    89 |         boost::optional<std::pair<OutputType, AuxiliaryData> >
6:       |                                                            ^
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:89:62: error: template argument 1 is invalid
6:    89 |         boost::optional<std::pair<OutputType, AuxiliaryData> >
6:       |                                                              ^
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:97:29: error: there are no arguments to ‘OutputType’ that depend on a template parameter, so a declaration of ‘OutputType’ must be available [-fpermissive]
6:    97 |         const std::function<OutputType (const InputType &)> conversion_function;
6:       |                             ^~~~~~~~~~
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:97:59: error: template argument 1 is invalid
6:    97 |         const std::function<OutputType (const InputType &)> conversion_function;
6:       |                                                           ^
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:103:36: error: wrong number of template arguments (2, should be 1)
6:   103 |     Conversion<InputType,OutputType>::
6:       |                                    ^
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:58:11: note: provided for ‘template<class InputType> class SampleFlow::Filters::Conversion’
6:    58 |     class Conversion : public Filter<InputType, OutputType>
6:       |           ^~~~~~~~~~
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:104:5: error: conflicting declaration of template ‘template<class InputType, class OutputType> int SampleFlow::Filters::Conversion(const std::function<OutputType(const InputType&)>&)’
6:   104 |     Conversion (const std::function<OutputType (const InputType &)> &conversion_function)
6:       |     ^~~~~~~~~~
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:58:11: note: previous declaration ‘template<class InputType> class SampleFlow::Filters::Conversion’
6:    58 |     class Conversion : public Filter<InputType, OutputType>
6:       |           ^~~~~~~~~~
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:112:22: error: expected initializer before ‘<’ token
6:   112 |     ComponentSplitter<InputType>::
6:       |                      ^
6: /home/dawson/projects/SampleFlow/tests/conversion_01.cc: In function ‘int main()’:
6: /home/dawson/projects/SampleFlow/tests/conversion_01.cc:37:59: error: wrong number of template arguments (2, should be 1)
6:    37 |     SampleFlow::Filters::Conversion<SampleType, ResultType> filter;
6:       |                                                           ^
6: In file included from /home/dawson/projects/SampleFlow/tests/conversion_01.cc:24:
6: /home/dawson/projects/SampleFlow/include/sampleflow/filters/conversion.h:58:11: note: provided for ‘template<class InputType> class SampleFlow::Filters::Conversion’
6:    58 |     class Conversion : public Filter<InputType, OutputType>
6:       |           ^~~~~~~~~~
6: /home/dawson/projects/SampleFlow/tests/conversion_01.cc:38:12: error: request for member ‘connect_to_producer’ in ‘filter’, which is of non-class type ‘int’
6:    38 |     filter.connect_to_producer(producer);
6:       |            ^~~~~~~~~~~~~~~~~~~
6: /home/dawson/projects/SampleFlow/tests/conversion_01.cc:43:34: error: cannot convert ‘int’ to ‘SampleFlow::Producer<double>&’
6:    43 |     consumer.connect_to_producer(filter);
6:       |                                  ^~~~~~
6:       |                                  |
6:       |                                  int
6: In file included from /home/dawson/projects/SampleFlow/include/sampleflow/consumers/stream_output.h:19,
6:                  from /home/dawson/projects/SampleFlow/tests/conversion_01.cc:23:
6: /home/dawson/projects/SampleFlow/include/sampleflow/consumer.h:150:45: note:   initializing argument 1 of ‘void SampleFlow::Consumer<InputType>::connect_to_producer(SampleFlow::Producer<InputType>&) [with InputType = double]’
6:   150 |   connect_to_producer (Producer<InputType> &producer)
6:       |                        ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~
6: make[3]: *** [tests/CMakeFiles/conversion_01.dir/build.make:83: tests/CMakeFiles/conversion_01.dir/conversion_01.cc.o] Error 1

In particular, note this error message:

`OutputType` was not declared in this scope; did you mean `InputType`?

I think this error comes from line 57 in include/sampleflow/filters/conversion.h which declares the Conversion class as a template requiring an InputType, but not an OutputType, even though OutputType is used by the class. Furthermore, at the bottom of the file, ComponentSplitter<InputType> is used instead of Conversion<InputType, OutputType>.

Differential evaluation: Skip crossover step in generation 0

Follow-up to #138: In the code as it is after #140, the differential evaluation sampler does a cross-over step in generation zero. That seems a bit silly: Nothing has evolved yet, though it is also not wrong.

@dawsoneliasen -- what do you think? Would it be worth skipping the cross-over step in generation zero? If so, want to give it a try?

Make sure connections are broken.

#225 adds a test in which we have a filter that is immediately destroyed again. This should break the connection between producer and consumer, but apparently does not. Investigate and fix this.

CMake fails: Could not find a package configuration file provided by "boost_signals"

@bangerth
On my Manjaro Linux box, I am trying to install SampleFlow for development. Following the instructions in the README, I executed cmake . and got this output:

CMake Error at /usr/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:164 (message):
  Could NOT find Boost (missing: Boost_INCLUDE_DIR signals)
Call Stack (most recent call first):
  /usr/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:445 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.17/Modules/FindBoost.cmake:2145 (find_package_handle_standard_args)
  CMakeLists.txt:34 (FIND_PACKAGE)


-- Configuring incomplete, errors occurred!
See also "/home/dawson/projects/SampleFlow/CMakeFiles/CMakeOutput.log".

I figured I just needed to install boost, so I did that and then executed cmake . again. Then I got this:

CMake Error at /usr/lib64/cmake/Boost-1.72.0/BoostConfig.cmake:120 (find_package):
  Could not find a package configuration file provided by "boost_signals"
  (requested version 1.72.0) with any of the following names:

    boost_signalsConfig.cmake
    boost_signals-config.cmake

  Add the installation prefix of "boost_signals" to CMAKE_PREFIX_PATH or set
  "boost_signals_DIR" to a directory containing one of the above files.  If
  "boost_signals" provides a separate development package or SDK, be sure it
  has been installed.
Call Stack (most recent call first):
  /usr/lib64/cmake/Boost-1.72.0/BoostConfig.cmake:185 (boost_find_component)
  /usr/share/cmake-3.17/Modules/FindBoost.cmake:444 (find_package)
  CMakeLists.txt:34 (FIND_PACKAGE)


-- Configuring incomplete, errors occurred!
See also "/home/dawson/projects/SampleFlow/CMakeFiles/CMakeOutput.log".

Am I doing something wrong with the installation? Are there other dependencies that need to be installed?

Consistenly use operator>> in the test suite.

Generalize StreamOutput for vector-like objects

We need to generalize the write() function in stream_output.h for everything that has a operator[]. We can use the traits class for this.

The MH class should have a way to seed the random number generator

Fix documentation of producers.

Now that some of the arguments to the sample() functions of producers have been moved into their classes' Parameters structures, make sure that the documentation is up to date.

Make sure producers can only be move-constructed if no connections exist.

Consumer(Consumer &&) already ensures that move operations can only succeed if no connections exist. Make sure that's also true for producers because it is unclear how a downstream consumer's connection would react when the signal it connects to (in the producer) is moved.