libmir / mir-glas Goto Github PK

I updated mir-glas and mir-cpuid as explained in README. Then compiling gemm_report with
dub build --compiler=ldmd2 -b release --single gemm_report.d gives an error:

Compiling ../../../../../.dub/packages/mir-algorithm-0.6.5/mir-algorithm/source/mir/ndslice/topology.d...
Compiling ../../../../../.dub/packages/mir-algorithm-0.6.5/mir-algorithm/source/mir/primitives.d...
Compiling ../../../../../.dub/packages/mir-algorithm-0.6.5/mir-algorithm/source/mir/timeseries.d...
Compiling ../../../../../.dub/packages/mir-algorithm-0.6.5/mir-algorithm/source/mir/utility.d...
Linking...
Linking...
.dub/build/application-release-linux.posix-x86_64-ldc_2072-6A7D442D6DF721FDCC6B5AF6994A4118/gemm_report.o: In function `_Dmain':
../../../../../.dub/packages/mir-algorithm-0.6.5/mir-algorithm/source/mir/utility.d:(.text._Dmain[_Dmain]+0x8d4): undefined reference to `cblas_sgemm'
/home/miguel/Dokumente/DLang/mir-glas-0.2.3/mir-glas//libmir-glas.a(home.miguel.Dokumente.DLang.mir-glas-0.2.3.mir-glas.source.glas.precompiled.context.d.o): In function `glas_init':
../source/glas/precompiled/context.d:(.text.glas_init[glas_init]+0x17): undefined reference to `cpuid_init'
../source/glas/precompiled/context.d:(.text.glas_init[glas_init]+0x1c): undefined reference to `cpuid_dCache'
../source/glas/precompiled/context.d:(.text.glas_init[glas_init]+0x27): undefined reference to `cpuid_uCache'
collect2: Fehler: ld gab 1 als Ende-Status zurück
Error: /usr/bin/gcc failed with status: 1
ldmd2 failed with exit code 1.`

I suceesfully compiled the benchmark gemm_report.d provided by mir-glas. I ran it twice.
One comparing with OpenBLAS and another comparing against ACML-5.3.1.
As you can see from the benchmarks mir-glas does not yield full performance for large matrices.
Peak performance for my machine is about 23 GFLOPs for double precision.
But also ACML does noch achieve full performance.
So I decided to compare with dgemm.goto and dgemm.acml benchmark programs provided in
OpenBLAS/benchmark. Here ACML reaches peak performance too. Is there any overhead calling
ACML from D?

See libmir/mir#394

• gemm micro kernel, simple
• gemm micro kernel, generic
• LLVM 3.9, broadwell, assembler output
• LLVM 4.0, broadwell, assembler output
• LLVM 4.0, avx512f, assembler output

LLVM 4.0, avx512f has the same issue as LLVM 4.0, broadwell.

LDC - the LLVM D compiler (1.1.1):
  based on DMD v2.071.2 and LLVM 4.0.0
  built with LDC - the LLVM D compiler (0.17.3)
  Default target: x86_64-apple-darwin16.4.0
  Host CPU: haswell
  http://dlang.org - http://wiki.dlang.org/LDC

vs

LDC - the LLVM D compiler (1.1.1):
  based on DMD v2.071.2 and LLVM 3.9.1
  built with LDC - the LLVM D compiler (1.1.1)
  Default target: x86_64-apple-darwin16.4.0
  Host CPU: haswell
  http://dlang.org - http://wiki.dlang.org/LDC

I am trying to compile the gemm_report.d in the bench folder. But I got an error:
[mig@antergos-mig bench]$ dub build --compiler=ldmd2 -b release --single gemm_report.d The determined compiler type "ldc" doesn't match the expected type "dmd". This will probably result in build errors. dub.json(12): Error: Expected '}' or ',' - got '"'.

I guess I have to remove the commented out part for the OpenBLAS libs:
`!/usr/bin/env dub
/+ dub.json:
{
"name": "gemm_report",
"dependencies": {"mir-glas": {"path": "../"}, },
"libs": ["blas"],
"lflags": ["-L$MIR_GLAS_PACKAGE_DIR", "-L$MIR_CPUID_PACKAGE_DIR", "-L.."],
"dependencies": {
"cblas": ">1.0.0",
"mir-glas":{
"path": "../"
}
"mir-cpuid": ">0.4.2",
},
"dflags-ldc": ["-mcpu=native"],
}
+/
"lflags": ["-L/opt/OpenBLAS/lib"],
"libs": ["openblas"],

// Set up your libblas to approporiate version, or just copy it to the benchmarks/glas folder.
// Note: GLAS is single thread for now.
...`

Hi,

Your glas library is very interesting, and the benchmark for your gemm is impressive. I think users and potential contributors would benefit greatly from a series of blogs or documentation explaining the design approach you have taken for the whole library and for the gemm algorithm.

One of the issues about D is that the advanced methods, idioms, and techniques are not particularly well popularised as they are in C++. So I think a good set of blogs describing the techniques used in this library would greatly help the community. The same thing extends to other mir libraries but I am more interested in glas.

Thank you

I'm on Windows 7 64bit with dmd 2.74.0, ldc 1.3.0 beta2, dub 1.3.0, and Visual Studio 2017.

The dub header to the mir-glas readme.md suggests that one can include a mir-glas project in a dub project essentially automatically. However, I have not had much success with this.

I created a new dub project with the following dub.json

{
	"name": "testing_mir_glas",
	"authors": [
		"jmh530"
	],
	"dependencies": {
		"mir-algorithm": ">=0.6.6",
		"mir-glas": ">=0.2.3",
		"mir-cpuid": ">=0.5.2"
	},
	"dflags-ldc2": ["-mcpu=native"],
	"lflags": ["-L$MIR_GLAS_PACKAGE_DIR", "-L$MIR_CPUID_PACKAGE_DIR"],
	"description": "A minimal D application.",
	"license": "proprietary"
}

and source\app.d (was going to add mir-glas stuff later)

import std.stdio;
import mir.ndslice.algorithm : sliced;

void main()
{
	auto x = [1, 2, 3, 4, 5].sliced;
	writeln("done.");
}

I tried compiling with dub build, dub build --compiler=ldmd2, and dub build --compiler=ldc2. The result is basically the same

C:\ProgrammingFiles\DFiles\testing\testing_mir_glas>dub build --compiler=ldmd2
The determined compiler type "ldc" doesn't match the expected type "dmd". This w
ill probably result in build errors.
Performing "debug" build using ldmd2 for x86_64.
mir-algorithm 0.6.6: target for configuration "library" is up to date.
mir-cpuid 0.5.2: building configuration "library"...
testing_mir_glas ~master: building configuration "application"...
Running pre-build commands...
The determined compiler type "ldc" doesn't match the expected type "dmd". This w
ill probably result in build errors.

Neither a package description file, nor source/app.d was found in
'C:\ProgrammingFiles\DFiles\dubFolder\mir-glas-0.2.3\'
Please run DUB from the root directory of an existing package, or run
"dub init --help" to get information on creating a new package.

No valid root package found - aborting.
Command failed with exit code 2

Hello, I am planning to user mir-glas for a project that needs to quickly process a huge amount of data. Binary compatibility across multiple systems is not an issue, so we are free to use SIMD instructions. How is SIMD implemented in mir-glas? Do I need to enable anything special? And while I'm on it, what LLVM is recommended for the most performance? Any other thing I should have in mind?

Thank you,
Filipe

See https://travis-ci.org/libmir/mir-glas for details

mir-glas has depedency on mir-algorithm:
"mir-algorithm" : ">=0.6.9 <0.9.0"
that prevents using newer version of mir-algorithm