detect free gpu automatically about gpur HOT 6 CLOSED

Hi @beibeijiang, can you provide a short, reproducible script of how you are doing this locking and GPU availability detection? Are you just using gpuInfo() and checking the available field? As far as I know, there isn't a way in OpenCL to detect how much RAM is currently available on a GPU (I will need to look around some more to be sure).

from gpur.

GPUsetWorkContext <- function(lock.p = "/var/lock/GPU/lock",...){
  ngpu <- detectGPUs()
  gpubusy.count <- 0
  for (i in 1:ngpu) {
    gpulock <- paste(lock.p, i, sep = "")
    if(!file.exists(gpulock)){
      setContext(i)
      file.create(gpulock)
      return(i)
      break
    }
    else
      gpubusy.count <- gpubusy.count + 1
  }
  if(gpubusy.count == ngpu){
    cat("Error: no avaliable resource!\n")
    q(save = "no")
  } 
}

lock.path = "/var/lock/GPU/lock"
#search free gpu and set working gpu
gpuid <- GPUsetWorkContext(lock.p = lock.path)
...# core functions
#finish work, remove the lock file
file.remove(paste(lock.path, gpuid, sep = "")

from gpur.

@beibeijiang I would recommend using a tryCatch statement around whatever process you are using so that if it fails it has an exception handler to remove the 'lock' file. For example, some pseudo-code would look like this:

result = tryCatch({
    # the gpu call
    expr
}, error = function(e) {
    # error handler code
    print(paste0("the following error happend: ", e))
    file.remove(paste(lock.path, gpuid, sep = ""))
}, finally = {
    # cleanup code
    rm(object)
    gc()
    file.remove(paste(lock.path, gpuid, sep = ""))
}

from gpur.

@cdeterman Many thanks for your work and help. It works well.
Look forward to your parallel version of gpuR().

from gpur.

@beibeijiang glad it solved your current problem. The parallel version of gpuR is somewhat complicated do to the fact that R itself is not internally thread safe. I think users may be able to use parallel backends like foreach so long as they don't intend to return any gpuR objects such as gpuMatrix/vclMatrix but this requires some further testing. Given this unforeseen complication the parallel version is likely going to be pushed back to either version 1.3.0 or potentially a separate package. I'm still not sure at the moment.

from gpur.

@cdeterman I understand. It takes time to figure out the gpuR parallel Currently I could split the input into chunks then submit simple R script into different gpu. It works well. This is also good for running jobs on gpu cluster. I am satisfied with the high performance. You have already made great work. It helps me a lot. Thank you very much.

from gpur.