In particular, check whethether any redundant copy is performed
and why LinearListDictionary is not compatible with AlphabetEncoding.

In order to stop computation when a user pressed stop.

Depending on a specific input alphabet structure, a different dictionary may turn out to be more efficient.

For example, taking into account tidysq case, a simple array would be better because the alphabet is encoded using small integers.

Secondly, for alphabet with relatively small number of elements, using a simple linear list dictionary would turn out to be more efficient than a more complex hash map.

An error occurs when the garbage collector is invoked at the same time as the computation of the algorithm - it releases the memory that is going to be used soon.

It turns out that if we change methods which allocate Rcpp objects and do not return them back to R, everything works as expected.

The goal: C_unpack_raws should return a pure C++ data structure instead of Rcpp structure.

If the parallel mode is not enabled, there is no need to create c++ objects

Add a suffix that determines a k-mer type

Use a data structure to store common function params,
for example, named Rcpp::List.
Prepare this structure on the R side.
In order to add a new param easier.

In order to determine whether the computations should be done in parallel or sequential mode.

std::unordered_map is thought not to be perfectly optimized data structure`.
It is possible to make it better in some cases.
a very valuable cpp conference talk: https://www.youtube.com/watch?v=M2fKMP47slQ

For example: https://github.com/sparsehash/sparsehash

An integer representing a position of a k-mer in a sequence tends to be larger than relatively small P constants used in hash function formula. Therefore, it is not recommended to use it during the hashing of a sequence.

How to fix it?
There are two solutions:

1. Use a large P
2. Different positional k-mer hashing approach - use (d + 1)-dimensional representations of k-mer (one extra integer indicates not transformed k-mer position)
3. Change the dictionary structure and use a specialized version for positional variant; for example, 2-level approach: the first level is the position, the second level -- the hash

Add window slide size param in order to count k-mers using a fixed, configurable window slide step. Currently, after each step, the window moves by one position.

In order to prevent exposing methods that are not necessary for a user and currently they are exposed just because of testing.

Check out: https://github.com/mi2-warsaw/FSelectorRcpp

Therefore, unpacking all tidysq sequences at once would be unnecessary => memory optimization

In particular, let a user know about the current progress of batch computations.

Fix the following issue:

installed size is 15.4Mb
    sub-directories of 1Mb or more:
      libs        13.2Mb
      thirdparty   1.3Mb

Add a new param max_threads_num_param to the algorithm method.

Perhaps, an additional helper buffer for k-mer formatting would improve performance and prevent from the heap lock in multithreaded k-mer string generation.

https://www.drdobbs.com/tools/memory-constraints-on-thread-performance/231300494

parallel workers should take a lambda function that access i-th row:

std::function<input_vector_t(int rowNum)>

This function will be used in parallel k-mer counting procedure (kmer_counting_common.h).

For example for Rcpp StringMatrix the lambda is:

[](int rowNum) -> Rcpp::StringMatrix::Row { return seqMatrix(rowNum, Rcpp::_); }

Bloom filter would be an extra layer between KMerManager and a custom dictionary.

1. AlphabetEncoding as a template instead of a class
2. Move the responsibility of checking if a sequence element is allowed to AlphabetEncoding from the algorithm
   (for encode method: return a notAllowedElementCode if the input element is invalid)
3. Each sequence type has custom specialization of AlphabetEncoding

1. Add more examples
2. Add benchmarks

Add fill or complete parameter to count_kmers to add columns with k-mers that weren't present in the data. It will streamline the binding of the k-mer matrices.

A param with_kmer_names would determine whether the output matrix should contain named columns or not. A user would be able to make computations faster if the exact column names are not used.

1. Typed find_kmers functions should have the form .count... (a param in Rpp::export attribute can be added to rename it)
2. Remove @export and move corresponding R tests into C++ form

Currently, an Rcpp::IntegerMatrix is returned, although the matrix would be sparse in the most of cases.

simple_triplet_matrix is an alternative solution, however, it also requires the normal matrix generation step, which wastes both CPU time and RAM resources.

Another solution is connected with Rcpp::Modules and creating a custom sparse data structure.

Currently, it is computed using a simple polynomial formula (hash/complex_hash.h)
The recommended way is to use a dedicated function from the boost lib.

As the total algorithm efficiency may depend on the perfomance of accessing k-mer data, it is important to choose an appropriate dictionary. The performance might depend on the size of data that is to be stored. Surprisingly, a linear list performs better that a hash map when the total number of elements is relatively small. Thus, it may be useful to make it possible for users to specify a dictionary type on their own because they know better the input data that is provided to the algorithm.

Handle the case when params: alphabet, sequences are nulls.
Unfortunately, in R: c() produces null in Rcpp

Add a proxy method count_kmers in R code that dispatches further processing to a specialized Rcpp method

1. Make the package work on macOS
2. Add ubuntu 20.04
3. Remove Travis CI
4. Fix build on windows R 3.6

Try to get rid of modulo that is pretty costly.

Try to use: https://github.com/lemire/fastmod ; https://arxiv.org/pdf/1902.01961.pdf

If the last input sequences do not contain any k-mer, then the corresponding output matrix raws are not generated.
The problem does not occur when such input sequences are not in the end.

A problematic example:

count_kmers(list("aaaaacbb", "aa"), 5, letters)

The output is:

A 1x4 simple triplet matrix.

However, it should be:

A 2x4 simple triplet matrix.

The aforementioned problem does not occur for the following example:

count_kmers(list("aaaaacbb", "aa", "aaaaaaa"), 5, letters)

Currently, the input memory has to be copied due to the fact that Rcpp structures aren't thread-safe and RcppParallel does not provide any thread-safe wrapper for Rcpp::StringMatrix.
This is very time consuming and inefficient. Particularly, creating std::string structures seems to be very inefficient.

Actually, the memory is copied sequentially (in other words, the action is not parallel)
and after that, we take advantage of parallel algorithm.

In fact, it seems more efficient just not to use parallel version.
This hypothesis should be tested.