kaleidicassociates / lubeck Goto Github PK

View Code? Open in Web Editor NEW

66.0 10.0 14.0 161 KB

High level linear algebra library for Dlang

Home Page: http://lubeck.libmir.org/

License: Boost Software License 1.0

D 98.29% Meson 1.16% Makefile 0.55%

dlang linear-algebra ndslice numerical-methods matlab octave quantitative-finance big-data numpy native-code

lubeck's People

Contributors

Stargazers

Watchers

Forkers

shigekikarita yazd 080993 wilzbach k3kaimu jmh530 joedlcolvin framecca inkrementator nordlow numbersxxx symmetryinvestments dkorpel

lubeck's Issues

Add new API

svd
Pseudo inv, pinv - Moore-Penrose pseudoinverse of matrix.
Gaussian elimination (backward subs) (inv)
FFT - std.numeric for now
Matrix Creation
- Easy to stack matrices (Hstack and Vstack). See mir.ndslice.concatenation
- FlipUD, FlipLR (oops, got in the wrong order) reversed!0/reversed!1
- special kind of B=P*A for P=permute, A[p].fuse, where p is 1D ndslice that is permutation, allocates new matrix with permuted raws.

Qr decomposition: qrDecompand solve method
Cholesky: choleskyDecompand solve method
LU Decomp: luDecompand solve method
LDL Decomp: ldlDecompand solve method
Eigen values/vectors (less frequent)

Matrix exponentiation (depends on SVD for A>=0, or on Eig for general matrices)
Gaussian Elimination for triangular matrices (should be trivial, but as long as it's there)
OLS (i.e., just QR then Pinv for triangular bit)

Pure diagonal matrixes
Special routines for special matrices - Triangular (U or L), Diagonal (i.e., trivial stuff), Block Diagonal (much trickier), Banded and Tri-diagonal, Permutation matrices, Toeplitz
- MMult,
- Pinv (e.g., gaussian elim)
- Schur Complement for block diagonal

Trying to build minimal example at https://tour.dlang.org/tour/en/dub/lubeck

I installed some packages at last:
sudo apt install libopenblas-dev

But now I am getting this error:
when running dub -v
....

Search for versions of lapack77 (1 package suppliers)

No versions for lapack77 for registry at https://code.dlang.org/ (fallback ["registry at http://code.dlang.org/", "registry at https://code-mirror.dlang.io/", "registry at https://code-mirror2.dlang.io/", "registry at https://dub-registry.herokuapp.com/"])
Nothing found for lapack77
Search for versions of mir-blas (1 package suppliers)
Return for mir-blas: [0.1.0, 0.0.4, 0.0.2, 0.0.1, ~master]
Search for versions of cblas (1 package suppliers)
Return for cblas: [2.0.1, 2.0.0, 1.0.0, 0.2.0, 0.1.0, 0.0.4, 0.0.3, 0.0.2, 0.0.1, 0.0.0, ~master, ~remove-unittest, ~link-1, ~link, ~John-Colvin-patch-1]
Package mir-lapack contains invalid dependency lapack77 (no version candidates) (ci=10)

last line repeated infinitly

What ca I do? (Ubuntu 17.10. 64 Bit)

Newest release doesn't compile anymore on Buildkite

https://buildkite.com/dlang/dmd/builds/3946#3311f4b4-ae52-4df5-8388-6315ccc1db18

Unresolvable dependencies to package mir-algorithm:
  | mir-blas 1.1.4 depends on mir-algorithm >=2.0.0-beta2 <4.0.0
  | mir-random 2.2.0 depends on mir-algorithm ~>3.2.1
  | mir-random 2.2.0 depends on mir-algorithm ~>3.2.1 (optional)
  | Unresolvable dependencies to package mir-algorithm:
  | mir-blas 1.1.4 depends on mir-algorithm >=2.0.0-beta2 <4.0.0
  | mir-random 2.2.0 depends on mir-algorithm ~>3.2.1
  | mir-random 2.2.0 depends on mir-algorithm ~>3.2.1 (optional)

Just a dependency problem, but as this seems to happen quite frequently it would be great if we could either change the Buildkite code to use the same testing as here or add the same testing here.

eigSymmetric -

I am trying to use lubeck to calculate the eigenvalues of a symmetric matrix.
My code is as follows:

import std.stdio;
import mir.ndslice;
import lubeck;

void main()
{
    // Test eigSymmetric
    auto n = 4;
    auto A = slice!double(n, n);
    A[] = [[  0.239960151286054,                  0,  0.178712872351482,    0],
           [ -0.372217567033019,  1.000000000000000,  0.270264106474929,    0],
           [ -0.990087548834596,                  0,  0.138859726355787,    0],
           [-48.935406546734953, 64.100000000000009,  2.399234111713982,  1.0]];

    auto ATA = slice!double(n, n);
    ATA = A.transposed.mtimes(A);
    auto eigs = ATA.eigSymmetric;
}

In my dub.json file I have the following:

    "dependencies": {
        "mir"   : ">=3.1.0",
        "lubeck": ">=1.1.0"
    },

When I try to build to code using ldc-1.9 I get the following error:
Error: template lubeck.eigSymmetric cannot deduce function from argument types !()(Slice!(double*, 2LU, cast(mir_slice_kind)2)),

Could this be a bug?

Thanks,

Max

Seems there is a dub conflict with this package.

See for instance:

https://buildkite.com/dlang/druntime/builds/3254#175779b7-d453-438c-98f0-350a1ad3889e/115

eigSymmetric fails to compile with mir.ndslice

Using eigSymmetric fails to compile.

/+dub.sdl:
dependency "mir-algorithm" version="~>3.9.2"
dependency "mir-random" version="~>2.2.14"
dependency "lubeck" version="~>1.3.0"
+/

import mir.ndslice;
import mir.random.variable : uniformVar;
import mir.random.algorithm : randomSlice;
import kaleidic.lubeck : eigSymmetric;

void main() {
    auto s = uniformVar!int(-2, 2).randomSlice(10, 10);
    s.eachUploPair!((upper, ref lower) { lower = upper; }); 
    auto r = eigSymmetric('L', s);
}

attempting to run dub temp.d

blasint = int
.dub/packages/lubeck-1.3.0/lubeck/source/kaleidic/lubeck.d(1084,12): Error: module mir.math.numeric import Prod not found, did you mean template mir.math.numeric.prod(F, Range)(Range r) if (isFloatingPoint!F && isIterable!Range)?
temp.d(15,23): Error: template instance kaleidic.lubeck.eigSymmetric!(cast(Flag)true, int*, cast(mir_slice_kind)2) error instantiating
/usr/bin/dmd failed with exit code 1.

Add Windows Install Instructions (in Wiki?)

There was a question on the forum about how to install this on Windows. I tried to provide some help from my experiences, but it could probably be fleshed out in more detail and put in the wiki or something.

Fails to build due to a dub dependency problem

This breaks the BuildKite builds for all dlang repos. An example failure:
https://buildkite.com/dlang/phobos/builds/878#321a0649-a44d-463b-b358-ee3917549508

Unresolvable dependencies to package mir-core:
--
  | mir-algorithm 3.1.21 depends on mir-core >=0.0.10 <1.0.0
  | mir-random 2.1.5 depends on mir-core ~>0.0.6
  | Unresolvable dependencies to package mir-core:
  | mir-algorithm 3.1.21 depends on mir-core >=0.0.10 <1.0.0
  | mir-random 2.1.5 depends on mir-core ~>0.0.6

mir-lapack does not have a lapack configuration

The dub-sdl for lubeck has:

configuration "lapack" {
subConfiguration "mir-lapack" "lapack"
}

but the dub.sdl for mir-lapack does not have a lapack configuration
https://github.com/libmir/mir-lapack/blob/master/dub.sdl

The errors in this case are not so informative.

Helper Function for QRResult to Calculate Q/R

I think QRResult needs a helper function(s) that produces Q and R from matrix and tau.

I am comparing the result of one of the unittests to how I would do it in R and it doesn't match up. Compare https://run.dlang.io/is/fcMfi9 with below (in R). The lubeck version has a zero in the top left corner for matrix, but I thought it was upper trapezoidal...I know that given R, we can calculate Q as a*inv(R), but I'm not sure I would get the correct R given's lubeck's output.

> A <- matrix(c(1, 1, 0, 1, 0, 1, 0, 1, 1), 3, 3)
> X <- qr(A)
> X
$qr
           [,1]       [,2]       [,3]
[1,] -1.4142136 -0.7071068 -0.7071068
[2,]  0.7071068  1.2247449  0.4082483
[3,]  0.0000000 -0.8164966  1.1547005

$rank
[1] 3

$qraux
[1] 1.707107 1.577350 1.154701

$pivot
[1] 1 2 3

attr(,"class")
[1] "qr"
> qr.Q(X)
           [,1]       [,2]       [,3]
[1,] -0.7071068  0.4082483 -0.5773503
[2,] -0.7071068 -0.4082483  0.5773503
[3,]  0.0000000  0.8164966  0.5773503
> qr.R(X)
          [,1]       [,2]       [,3]
[1,] -1.414214 -0.7071068 -0.7071068
[2,]  0.000000  1.2247449  0.4082483
[3,]  0.000000  0.0000000  1.1547005
>

Make BlasType public

Can I make BlasType public? That operation can be useful when writing a module outside of lubeck that depends on calling lubeck functions.

error in the s/dger case (mx1 vs 1xn) in mtimes

I met the following errors in this special case. I think using ger inside mtimes is the best.

unittest
{
    import std.stdio;
    auto a = [1.0, 2.0].sliced(2, 1);
    auto b = [1.0, 2.0].sliced(2, 1);
    auto bt = b.transposed;

    // without slice
    // (linked with cblas) ** On entry to cblas_dgemm parameter number 11 had an illegal value
    // (linked with mkl_rt) Intel MKL ERROR: Parameter 11 was incorrect on entry to cblas_dgemm.
    writeln(mtimes(a, bt)); // [[6.911e-310, 9.85365e-317], [0, 0]]

    // but OK with slice
    assert(mtimes(a, bt.slice) == [[1, 2], [2, 4]]);

    // also I think mir.blas.ger is nice when a.length!1 == 1 && b.length!0 == 1
    import mir.blas : ger;
    auto c = uninitSlice!double(a.length!0, bt.length!1);
    c[] = 0;
    ger(1.0, a[0..$,0], b[0..$,0], c);
    assert(c == [[1, 2], [2, 4]]);
}

I think this is related to the LDB (the 11th arg) specification http://www.netlib.org/lapack/explore-html/d1/d54/group__double__blas__level3_gaeda3cbd99c8fb834a60a6412878226e1.html#gaeda3cbd99c8fb834a60a6412878226e1

       On entry, LDB specifies the first dimension of B as declared
       in the calling (sub) program. When  TRANSB = 'N' or 'n' then
       LDB must be at least  max( 1, k ), otherwise  LDB must be at
       least  max( 1, n ).

where B.shape == [k, n]

P.S.

The following transposed B (1 < k < n) seems to be OK. I think only k=1 has the problem.

unittest
{
    auto a = [1.0, 2.0,
              3.0, 4.0,
              5.0, 6.0].sliced(3, 2);
    auto b = [1.0, 2.0,
              3.0, 4.0,
              5.0, 6.0].sliced(3, 2);
    auto bt = b.transposed;
    auto result = [[5, 11, 17], [11, 25, 39], [17, 39, 61]];
    // OK
    assert(mtimes(a, bt) == result);
}

undefined reference to `dgetri_'

Tried building a test package with lubeck and mir-algorithm specifically trying to call inv function. Get the following error:

.dub/packages/mir-lapack-0.0.3/mir-lapack/source/mir/lapack.d:53: undefined reference to `dgetri_'

In the dub.json I include:

"dependencies": {"lubeck": "~~>0.0.3", "mir-algorithm": "~~>0.6.6"}

allocations of complex number arrays fails at GC since 2.085

@k3kaimu, I disabled the unittest c76f4f8

DMD 2.084 and LDC 1.14 works

Can't use tolerance argument of pinv

I think this could easily be a dmd problem, but I can't work it out right now.

pinv([1.0, 2, 3, 4].sliced(2, 2), 0.1)

Error: template lubeck.pinv cannot deduce function from argument types !()(Slice!(double*, 2LU, cast(mir_slice_kind)2), double), candidates are:
/home/john/.dub/packages/lubeck-1.1.8/lubeck/source/lubeck.d(754,5):        pinv(Flag allowDestroy = No.allowDestroy, Iterator, mir_slice_kind kind)(Slice!(Iterator, 2, kind) matrix, BlasType!Iterator tolerance = BlasType!Iterator.nan)

workaround is to manually instantiate the template e.g.

pinv!(No.allowDestroy, double*, SliceKind.contiguous)([1.0, 2, 3, 4].sliced(2, 2), 0.1)

On entry to SGEMM parameter number 8 had an illegal value

The following code outputs the runtime warning

 ** On entry to SGEMM  parameter number  8 had an illegal value

and also produced a wrong/invalid result:

import std.stdio;

import mir.ndslice.slice;
import mir.ndslice.allocation;
import lubeck;

void main()
{
    Slice!(cast(SliceKind)2, [2LU], float*) a = slice!float(1, 1);
    Slice!(cast(SliceKind)0, [2LU], float*) b1 = slice!float(16, 1).transposed;
    Slice!(cast(SliceKind)2, [2LU], float*) b2 = slice!float(1, 16);

    a[] = 3;
    b1[] = 4;
    b2[] = 4;

    writefln("a: %s\nb1: %s\nb2: %s", a, b1, b2);

    // Outputs: ** On entry to SGEMM  parameter number  8 had an illegal value
    // And gives invalid result
    writefln("a x b1 = %s", a.mtimes(b1));
    // Works
    writefln("a x b2 = %s", a.mtimes(b2));
}

dub file:

{
	"name": "mtimes_bug",
	"authors": [
		"Marenz"
	],
	"dependencies": {
		"lubeck": "~>0.0.4"
	},
	"license": "GPLv3",
    "libs": [ "lapack", "blas", "cblas"]
}

Docs?

Is there a link to the docs anywhere?
Thanks

lubeck documentation tests fail

The following tests are failing currently:

detSymmetric
eigSymmetric
luSolve
ldlSolve
choleskyDecomp
qrSolve

Additionally, I think it would be better if examples included import kaleidic.lubeck; since the newcomers will be confused by having import mir.ndslice; but things not working.

Documentation contains redundant backward slashes here and there: \min ||A * X - B|| Using the QR factorization: \A = Q * R

eigSymmetric fails to instantiate

/home/john/.dub/packages/lubeck-1.3.1/lubeck/source/kaleidic/lubeck.d(1097,31): Error: template mir.ndslice.slice.sliced cannot deduce function from argument types !()(double[1], int, int), candidates are:
/home/john/.dub/packages/mir-algorithm-3.9.13/mir-algorithm/source/mir/ndslice/slice.d(245,6):        sliced(ulong N, Iterator)(Iterator iterator, size_t[N] lengths...)
  with N = 2LU,
       Iterator = double[1]
  must satisfy the following constraint:
       !isStaticArray!Iterator
/home/john/.dub/packages/mir-algorithm-3.9.13/mir-algorithm/source/mir/ndslice/slice.d(312,12):        sliced(T)(T[] array)
/home/john/.dub/packages/mir-algorithm-3.9.13/mir-algorithm/source/mir/ndslice/slice.d(335,5):        sliced(Iterator, ulong N, mir_slice_kind kind)(Slice!(Iterator, 1, kind) slice, size_t[N] lengths...)
Error: template instance kaleidic.lubeck.eigSymmetric!(cast(Flag)false, double*, cast(mir_slice_kind)2) error instantiating