bhamon / gpuplotgenerator Goto Github PK

gpuPlotGenerator.exe generate direct w:\9515785206432086504_3747841_5734400_4096

GPU plot generator v4.1.0

Author: Cryo
Bitcoin: 138gMBhCrNkbaiTCmUhP9HLU9xwn5QKZgD
Burst: BURST-YA29-QCEW-QXC3-BKXDL

Loading platforms...
Loading devices...
Loading devices configurations...
Initializing generation devices...
[0] Device: Baffin (OpenCL 2.0 AMD-APP (2348.3))
[0] Device memory: 2GB 0MB
[0] CPU memory: 2GB 0MB
Initializing generation contexts...

[ERROR] Unable to prevent zero filling

gpuPlotGenerator-bin-win-x64-4.0.3>gpuPlotGenerator
exe generate direct w:\9515785206432086504_3747841_5734400_4096

GPU plot generator v4.0.3

Author: Cryo
Bitcoin: 138gMBhCrNkbaiTCmUhP9HLU9xwn5QKZgD
Burst: BURST-YA29-QCEW-QXC3-BKXDL

Loading platforms...
Loading devices...
Loading devices configurations...
Initializing generation devices...
[0] Device: Baffin (OpenCL 2.0 AMD-APP (2348.3))
[0] Device memory: 2GB 0MB
[0] CPU memory: 2GB 0MB
Initializing generation contexts...
[0] Path: w:/9515785206432086504_3747841_5734400_5734400
[0] Nonces: 3747841 to 9482240 (1TB 376GB 0MB)
[0] CPU memory: 1GB 0MB

Devices number: 1
Plots files number: 1
Total nonces number: 5734400
CPU memory: 3GB 0MB

Generating nonces...
0.00% (0/5734400 remaining nonces), 0.00 nonces/minutes, ETA: 10y 48w 6d 5h 20m

0.00% (0/5734400 remaining nonces), 0.00 nonces/minutes, ETA: 10y 48w 6d 5h 20m

0.00% (0/5734400 remaining nonces), 0.00 nonces/minutes, ETA: 10y 48w 6d 5h 20m

0.00% (0/5734400 remaining nonces), 0.00 nonces/minutes, ETA: 10y 48w 6d 5h 20m
0s...

In the [listDevices] command, the device [Type] displays [CPU] rather than [GPU] for graphic cards.

I've had this issue on Ubuntu (HiveOS) and all I had to do is :

sudo apt-get remove build-essential
sudo add-apt-repository ppa:ubuntu-toolchain-r/test  
sudo apt-get update
sudo apt-get install build-essential libstdc++```

If you could place that somewhere in your Readme section it'd help 👍 

Hi, this might only be a beauty thing.
From day1 I used this plotter I have an awful output. (see the gif, its made from a realtime screencap, this is as it look always). Could you please make an option where I can say, print it line by line? This would also give me more control, as I can parse the output better with my little fish wrapper.
I am connected to the server by ssh from a Mac Terminal with xterm256 emulation and the plotter is running in a screen session. So there are a couple of layers.

this gif is realtime.

Thank you very much for your awesome work.

Is this normal?

I run

sudo ./gpuPlotGenerator generate direct /media/burst-4t/377350976255672439_0_1250000_2000

but the file generated is

/media/burst-4t/377350976255672439_0_1250000_1250000

More and more GPUs come with a lot of RAM, but the max allocation size tends to be quite limited.
The idea would be to implement a multi-buffer generation in the OpenCL kernel to use the full GPUs RAM capacity.

The hard part is syncing between multiple buffers with possibly different sizes.
Or would it be better to instanciate multiple kernels at the same time ?

Hello

( sorry bad english)

Here the program has this execution error

[ERROR][-1][CL_DEVICE_NOT_FOUND] Unable to retrieve the OpenCL devices number

I will attach the compilation and execution logs.

I use branch master, of today: d1a3e81

gpuPlotGenerator-cmake.log
gpuPlotGenerator-listDevices-0.log
gpuPlotGenerator-listDevices-1.log
gpuPlotGenerator-listPlatforms.log
gpuPlotGenerator-make.log
gpuPlotGenerator-setup.log

At the moment he is ignoring the gpu of nvidia.

I took the test with cuda-8 and the same thing happens.

Even creating the devices.txt manually the same error occurs.

I used these examples as a base.

Devices reference list

Do you have any extra dependency?

I use it here Slackware64-14.2.

Thanks for listening.

What do you think about writing fully optimised plots directly like XPlotter does?

Recently I've added this ability to mjminer: r-majere/mjminer#4

We just need to preallocate file and write stagger with 4096 seeks and writes.

Any plans to add the new 2.0 POC stuff?

Add a feature to resume plots generation on a file.

Create a [status] command to tell the generation status of a file.

Looks like recursive call in first line…

OpenclError& OpenclError::operator=(const OpenclError& p_other) {
    *this = p_other;
    m_code = p_other.m_code;

    return *this;
}

Hello.

( sorry bad english)

In this master branch 130aaa9

Compilation ends well without fail

The kernel directory is paired with the binary.

$ ls
devices.txt gpuPlotGenerator kernel

Here only the poc1 option is working.

The error option poc2.

Poc2 with error in "buffer and direct".

I tested the poc2 option with 20480 (5gb) and the same error occurs

Note.

Could put command line poc1 and poc2

./gpuPlotGenerator generate direct /path/to/files/123456_0_50000_5000_poc1

./gpuPlotGenerator generate direct /path/to/files/123456_0_50000_5000_poc2

Thanks for listening.

The [direct] writing strategy can be improved by reordering the scoops in a stagger-like pre-buffer.

Add a [benchmark] command to visualize the raw devices performances to enhance parameters tunning.

Hello

Can you please provide the linux x64 binaries for the latest version?

Also, how can we set the OpenCL location in the latest cmake version?
I've tried:
export OpenCL_INCLUDE_DIR=/usr/local/cuda-8.0/include/
export OpenCL_LIBRARY=/usr/local/cuda-8.0/lib64/
without success.
cuda installed in /usr/local/cuda by default on centos7, is it possible to make the cmake script check this location as well?

Thank you for this great software!

From the readme:

"This command verify overlapping nonces between two plots files. It is usefull to verify a plots file integrity."

This kind of feature really interests me, however the wording seems little scattered, it could be interpreted as two features of the verification.

1. Scans for overlapped nonces - this one's intent easy to interpret however isn't a very difficult thing to avoid on your own.

2. Verify plots intergrity - I interpret that as it will make sure a plot is full of nonces between X and XX, is that the intent of the wording?

My interpretation of the second sentence is what really interests me and I'm hoping for a little clarification. I use GPU plotter more and more now and I'm not stranger to how unstable versions have been in the past and would really like to check to make sure my plots are actually full of nonces.

Thanks!

When I try to plot (which works on my computer also only, if I plot a file which has less space than like 1/2 of available ram. Otherwise it plots with like 5 nonces/min or so after 1-2mins) this program hangs on a regular basis. I sometime have to restart 5-10 times in a row until finally some files are generated (sometimes also just one) until i have to restart again.
Settings:
devices.txt: 0 0 8192 256 8192
parameter: gpuPlotGenerator.exe generate direct L:\mining\ACCOUNTID_51265710_12000_12000

Line 17 of PlotsFile-mac.cpp should read:
if(fcntl(fd, F_SETSIZE, &p_size) == -1) {

and not:
if(fnctl(fd, F_SETSIZE, &p_size) == -1) {

Create a new [setup] command to help in the [devices.txt] creation process.

Add an [optimize] command to generate files with higher staggerSize.

Write the plots to file directly with = .

This version will require much more IO operations, but less disk-stress afterward without having to optimize it after generation. Consider making it an option.

Hello

Is it possible to add an automated deletion of the plot if it fails?
0.52% (39200/7610000 remaining nonces), 6461.54 nonces/minutes, ETA: 19h 31m 40s...
[ERROR] Error while writting to plots file

It would be very good, especially in direct mode, as sometimes it's hard to catch broken plots, when plotting multiple hdds.

Thank you

Can get it to cmake

getting:

CMake Error at /usr/share/cmake-3.5/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
  Could NOT find OpenCL (missing: OpenCL_LIBRARY) (found version "2.0")
Call Stack (most recent call first):
  /usr/share/cmake-3.5/Modules/FindPackageHandleStandardArgs.cmake:388 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindOpenCL.cmake:150 (find_package_handle_standard_args)
  CMakeLists.txt:7 (find_package)

Compiling on U16.04Lts (4.13.0-36). with latest AMDSDK (3.0) installed in /opt/AMDAPPSDK-3.0/

./gpuPlotGenerator generate direct ./tmp/9633051684328757284_0_40_4096

Initializing generation contexts...
[0] Path: ./tmp/9633051684328757284_0_4096_4096

Hi There!

I really like gpuplotter, thank you for this!

Acutally i'm struggling with direct plots to networkdrives. I would like to plot a file direct from my pc to my servers networkdrive.

My pc runs windows 10, my server (miner) is running debian with samba to share my drives. This is working just fine.
I'm able to map the drive on my windows machine in command prompt using "net use" command. Every time gpuplotgenerator throw the error [ERROR] Unable to prevent zero filling: 87.
On my debian server i tried to format my network drive form EXT4 to NFTS. This is working, but gpuplotgenerator throws the same message.

Mysterious is, the plotfile is allocated on my networkdrive (10GB in my test case). The poltfile is there after try to start the plotter, i see it from my debain and windows machine.

Maybe you see the problem to help me with my problem.
An option to try it without allocating the diskspace before plot could be a workaround. only my thought.

Sorry for my bad english and thank you

Hello! When I try to run the setup command, it gives just an infinite output, like this:

And so on, and so on...

Hi,

are NTFS partitions supported under linux? I always get [ERROR] Unable to extend output file error when I try to plot.

This is a command that I use.

./gpuPlotGenerator generate direct ..\8836757514483927694_20000000000_30181735_30181735
-------------------------
GPU plot generator v4.1.4
-------------------------
Author:   Cryo
Bitcoin:  138gMBhCrNkbaiTCmUhP9HLU9xwn5QKZgD
Burst:    BURST-YA29-QCEW-QXC3-BKXDL
----
Loading platforms...
Loading devices...
Loading devices configurations...
Initializing generation devices...
    [0] Device: GeForce GTX 1050 Ti (OpenCL 1.2 CUDA)
    [0] Device memory: 1009MB
    [0] CPU memory: 1009MB
Initializing generation contexts...

[ERROR] Unable to extend output file

I also tried CPU plotting using https://github.com/alter3d/omdcct which works fine.

./plot -k 8836757514483927694 -x 1 -d /media/emphasis/data-002/burst/plots 
Using SSE2 core.
Device has 8001115611136 bytes of free space, leaving 5000000000 bytes after plotting.
Creating plots for nonces 959111738435597828 to 959111738466100512 (8001 GB) using 12794 MB memory and 4 threads

I also tried checking if I can preallocate the space using fallocate, though I am not sure if it is related:

fallocate -l 1G test 
fallocate: fallocate failed: Operation not supported

I used the latest source code to compile gpuPlotGenerator.

Make a list of devices along with their optimal parameters.

When compiling in Debian Jessie with both the 4.1.3 and 4.1.4 I get the following error:

Scanning dependencies of target gpuPlotGenerator
[  4%] Building CXX object CMakeFiles/gpuPlotGenerator.dir/Command.cpp.o
[  9%] Building CXX object CMakeFiles/gpuPlotGenerator.dir/CommandGenerate.cpp.o
In file included from /root/gpuPlotGenerator-4.1.3/CommandGenerate.cpp:25:0:
/root/gpuPlotGenerator-4.1.3/build/GenerationDevice.h:18:23: fatal error: constants.h: No such file or directory
 #include "constants.h"
                       ^
compilation terminated.
CMakeFiles/gpuPlotGenerator.dir/build.make:86: recipe for target 'CMakeFiles/gpuPlotGenerator.dir/CommandGenerate.cpp.o' failed
make[2]: *** [CMakeFiles/gpuPlotGenerator.dir/CommandGenerate.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/gpuPlotGenerator.dir/all' failed
make[1]: *** [CMakeFiles/gpuPlotGenerator.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Any help is appreciated!

Add multi-disks support to enhance GPU usage.

I'm trying to build for Ubuntu 16.04. I have CUDA and OpenCL both installed and working for other applications.

I'm getting the following error message:

gpuPlotGenerator/build/GenerationDevice.h:18:23: fatal error: constants.h: No such file or directory

All I've done so far is clone the repo, mkdir build within repo. Place myself in build directory and use the following commands

cmake .. -DOpenCL_LIBRARY=/opt/AMDAPPSDK-3.0/lib/x86_64/ -DOpenCL_INCLUDE_DIR=/opt/AMDAPPSDK-3.0/include make

Q1: Can someone place a screenshot of the expected output during operation?

My final line disappears and sometimes parts of the line reappear.
I can't tell if my plot is actually still successfully being created.

I started with 4TB size, moved down several times, now down to 256GB because I don't want to leave it there for a week and then nothing happens. (That's a week wasted when I could be mining XMR 😛 )

For reference here is what I am using:

I have 5 Nvidia GPUs and an Intel Celeron. 4GB of RAM (more coming in the mail tomorrow)

Q2: Is there something such as 'too many' gpus? 😆

My devices.txt looks like this

0 0 1024 64 8192
0 1 1024 64 8192
0 2 1024 64 8192
0 3 1024 64 8192
0 4 1024 64 8192

Anything larger and the plot would fail very quickly.

Q3: Is there a correlation between how much RAM I need and how much of the GPU I can utilize?

I have a 1080ti in there and it has 11GB, but I can't even use more than 1024 on all the GPUs, when the "setup" command had me input recommended numbers that were several decimal places larger.

Not sure what this error means? I've configured my devices.txt using gpuPlotGenerator setup and am getting this error when I run gpuPlotGenerator generate buffer G:\burst\123456789_7628721_19075408_19075408

Hey, not sure if this a bug or something i'm doing wrong.

When plotting (In direct mode) i have noticed it will freeze at 0.??% and it will stay at that percentage until it has filled all the space on the drives you have plotting. Once it has filled them it then starts again from the beginning and actually starts plotting. This i'm guessing roughly doubles the time. Only on the second pass does the percent start going up. The free space on the drives does not decrease anymore at this point.

Example mine was stuck on 0.7% for roughly 12 hours on 2 3TB and 1 2TB drives. After 12 hours it had wrote 3 plot files (one to each the correct size) then it started again and the percent started increasing.

During the "First" pass the GPU remains at 0% load and on the second round it is actually used.

During the second round no Reads are done from the drives (from what i can tell from watching it) Meaning its not trying to optimise what it wrote the first time.

Any input / help would be appreciated!
Thanks

Intel i7 4790K
16GB Ram
GTX 970

my setup: 16G ram ，8Tb wd ，R9 380 4gb
devices.txt ( 0 0 4096 256 4096 )
useing buffer mode
I can get a constant 40k nonces
But after a few hours, the program hung up，I can only close the window and start from the beginning.

I've tried the latest version and 4.0.3
Same result. my to50Ti has 4Gb memory

C:\Users\Rob\Downloads\gpuPlotGenerator-bin-win-x64-4.0.3>gpuPlotGeneratate buffer d:\plots\13103407935388750190_0_18432000_4096

GPU plot generator v4.0.3

Author: Cryo
Bitcoin: 138gMBhCrNkbaiTCmUhP9HLU9xwn5QKZgD
Burst: BURST-YA29-QCEW-QXC3-BKXDL

Loading platforms...
Loading devices...
Loading devices configurations...
Initializing generation devices...
[0] Device: GeForce GTX 1050 Ti (OpenCL 1.2 CUDA)
[0] Device memory: 512MB
[0] CPU memory: 512MB
Initializing generation contexts...
[0] Path: d:/plots/13103407935388750190_0_18432000_4096
[0] Nonces: 0 to 18431999 (4TB 404GB 0MB)
[0] CPU memory: 1GB 0MB

Devices number: 1
Plots files number: 1
Total nonces number: 18432000
CPU memory: 1GB 512MB

Generating nonces...
0.00% (0/18432000 remaining nonces), 0.00 nonces/minutes, ETA: 35y 8w 4d

Here's my setup. rx 480 4gb I'm plotting in direct.
devices.txt ( 0 0 8192 256 4 )
On SMR Drive goes from, 40k nonces - 2k nonces after 1 hour
On SSD Drive goes from 40k nonces - 6k nonces after 1 hour
Is there something wrong here?

edit: I only use ~5gb of ram since I only have 8gb of ram while creating the plot.

Good evening!
In a current form it is not easy to plot a huge drive since one has to calculate a number of nonces and RAM manually and then form file name.
omdcct allows simplified process:

• if amount of nonces is not given, use most of a drive space (minus 5 GB IIRC)
• if buffer size is not given, use some percentage

These changes greatly simplify using a plotter.
I believe implementing an alternate syntax (e.g. compatible with omdcct since community is more or less familiar with it) would be great.

Any suggestions?

Best regards,
Kristian Popov

Remove the actual restriction in OpenCL devices listing to allow full APU support.

Do you release this one anywhere ? i would like to give a try

Add a benchmark feature to see pure device computing power (without writing to disk).

So I feel like my issue may be related to hardware, but it seems really odd to me and may be software instead.
So at the beginning of direct gpu plotting multiple seagate smr 8tb drives, the gpu is continually at 100% usage and the hdd are writing at about 20-30MB/s per drive. After about 0.5% to 1% of plotting, it suddenly gets really slow. The gpu goes to 0% usage. The drives start to write at about 1-2MB/s per drive. Everything is really slow. After a while, the gpu will spike to 100% for a few seconds and the drives will again be writing at 20-30MB/s per drive. Then, the gpu will fall back to 0% and the drives will be writing really slow. It seems to have a cycle, but the amount of time at 0% or 100% seems random. Sometimes it will cycle every minute or so. Other times it will take it 15-20 min before it spikes to 100% again.
Maybe it is just the SMR drives, but it just seemed really odd to be so I thought I would post the issue and see if you have any ideas about what could be causing the problem.
Thanks for any help and support.

I thought I should add one extra note, the ram fills completely significantly before the plot reaches 0.5% to 1% and the gpu drops to 0% and things slow down.

Using the latest master I'm having some issues generating POC2 files. POC1 work fine. Using the examples in the readme:

gpuPlotGenerator.exe generate buffer 123456_0_50000_5000_1 works as expected, generating a POC1 file.
gpuPlotGenerator.exe generate buffer 123456_0_50000_5000_2 crashes the application as soon as nonce generation starts.
gpuPlotGenerator.exe generate buffer 123456_0_50000_5000_1 123456_50000_10000_2000_2 crashes at 0.85%, I assume when POC2 generation starts.
Generating direct using any of the examples gives the same result.

Unfortunately my C knowledge is.... lacking, but opening it up for debugging gives me this, I have no idea if it's useful to anyone: https://i.imgur.com/gH5fCO4.png

My devices.txt looks like 0 0 512 128 20.

I'm aware that master is probably unsupported as it isn't a release, but since there isn't a POC2 release and there's a lack of commandline GPU miners for burstcoin I thought I'd give it a go anyway. Cheers guys!

Whenever I try to create plot with direct method on NTFS partition I get error "Unable to extend output file".
Plotting on ext4 works fine, as does plotting with buffer method on NTFS partitions.

Add more details in the [listDevices] command to give better clues on which value to use for the parameter.

Test

At .09% the statistics just stopped updating. Looking at the perfmon, there's ton's of drive activity so it seems like it's still working.

GTX 970

Devices.txt:
0 0 1024 128 2048

Plot Command:
gpuPlotGenerator generate direct P:\PlotFiles\15190589747919327836_4199849_7577600_8192

Hi

I really can not understand how to run the GPU ploter on ubuntu.
Is there a step by step guide I can read?

Thank you