Package org.apache.sysds.runtime.instructions.gpu.context

# Class GPUContext

java.lang.Object
org.apache.sysds.runtime.instructions.gpu.context.GPUContext

public class GPUContext extends Object
Represents a context per GPU accessible through the same JVM. Each context holds cublas, cusparse, cudnn... handles which are separate for each GPU.

## Method Summary

Modifier and Type
Method
Description
`jcuda.Pointer`
`allocate(String instructionName, long size)`
Default behavior for gpu memory allocation (init to zero)
`jcuda.Pointer`
`allocate(String instructionName, long size, boolean initialize)`
Invokes memory manager's malloc method
`void`
`clearMemory()`
Clears all memory used by this `GPUContext`.
`void`
`clearTemporaryMemory()`
`GPUObject`
`createGPUObject(MatrixObject mo)`
Instantiates a new `GPUObject` initialized with the given `MatrixObject`.
`void`
`cudaFreeHelper(String instructionName, jcuda.Pointer toFree, boolean eager)`
Does cudaFree calls, lazily.
`static int`
`cudaGetDevice()`
Returns which device is currently being used.
`void`
`destroy()`
Destroys this GPUContext object.
`void`
`ensureComputeCapability()`
Makes sure that GPU that SystemDS is trying to use has the minimum compute capability needed.
`long`
`getAvailableMemory()`
Gets the available memory on GPU that SystemDS can use.
`jcuda.jcublas.cublasHandle`
`getCublasHandle()`
Returns cublasHandle for BLAS operations on the GPU.
`jcuda.jcudnn.cudnnHandle`
`getCudnnHandle()`
Returns the cudnnHandle for Deep Neural Network operations on the GPU.
`jcuda.jcusolver.cusolverDnHandle`
`getCusolverDnHandle()`
Returns cusolverDnHandle for invoking solve() function on dense matrices on the GPU.
`jcuda.jcusparse.cusparseHandle`
`getCusparseHandle()`
Returns cusparseHandle for certain sparse BLAS operations on the GPU.
`int`
`getDeviceNum()`
Returns which device is assigned to this GPUContext instance.
`jcuda.runtime.cudaDeviceProp`
`getGPUProperties()`
Gets the device properties for the active GPU (set with cudaSetDevice()).
`JCudaKernels`
`getKernels()`
Returns utility class used to launch custom CUDA kernel, specific to the active GPU for this GPUContext.
`int`
`getMaxBlocks()`
Gets the maximum number of blocks supported by the active cuda device.
`long`
`getMaxSharedMemory()`
Gets the shared memory per block supported by the active cuda device.
`int`
`getMaxThreadsPerBlock()`
Gets the maximum number of threads per block for "active" GPU.
`GPUMemoryManager`
`getMemoryManager()`
`int`
`getWarpSize()`
Gets the warp size supported by the active cuda device.
`void`
`initializeThread()`
Sets the device for the calling thread.
`void`
`printMemoryInfo(String opcode)`
Print information of memory usage.
`GPUObject`
`shallowCopyGPUObject(GPUObject source, MatrixObject mo)`
Shallow copy the given source `GPUObject` to a new `GPUObject` and assign that to the given `MatrixObject`.
`String`
`toString()`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

## Method Details

### getMemoryManager

public GPUMemoryManager getMemoryManager()

### cudaGetDevice

public static int cudaGetDevice()
Returns which device is currently being used.

Returns:
  the current device for the calling host thread

### printMemoryInfo

public void printMemoryInfo(String opcode)
Print information of memory usage.

Parameters:
  `opcode` - opcode of caller

### getDeviceNum

public int getDeviceNum()
Returns which device is assigned to this GPUContext instance.

Returns:
  active device assigned to this GPUContext instance

### initializeThread

public void initializeThread()
Sets the device for the calling thread. This method must be called after `ExecutionContext.getGPUContext(int)` If in a multithreaded environment like parfor, this method must be called when in the appropriate thread.

### allocate

public jcuda.Pointer allocate(String instructionName, long size, boolean initialize)
Invokes memory manager's malloc method

Parameters:
  `instructionName` - name of instruction for which to record per instruction performance statistics, null if you don't want to record
  `size` - size of data (in bytes) to allocate
  `initialize` - if cudaMemset() should be called
Returns:
  jcuda pointer

### allocate

public jcuda.Pointer allocate(String instructionName, long size)
Default behavior for gpu memory allocation (init to zero)

Parameters:
  `instructionName` - Name of the instruction calling allocate
  `size` - size in bytes
Returns:
  jcuda pointer

### cudaFreeHelper

public void cudaFreeHelper(String instructionName, jcuda.Pointer toFree, boolean eager)
Does cudaFree calls, lazily.

Parameters:
  `instructionName` - name of the instruction for which to record per instruction free time, null if you do not want to record
  `toFree` - `Pointer` instance to be freed
  `eager` - true if to be done eagerly

### getAvailableMemory

public long getAvailableMemory()
Gets the available memory on GPU that SystemDS can use.

Returns:
  the available memory in bytes

### ensureComputeCapability

public void ensureComputeCapability()
Makes sure that GPU that SystemDS is trying to use has the minimum compute capability needed.

### createGPUObject

public GPUObject createGPUObject(MatrixObject mo)
Instantiates a new `GPUObject` initialized with the given `MatrixObject`.

Parameters:
  `mo` - a `MatrixObject` that represents a matrix
Returns:
  a new `GPUObject` instance

### shallowCopyGPUObject

public GPUObject shallowCopyGPUObject(GPUObject source, MatrixObject mo)
Shallow copy the given source `GPUObject` to a new `GPUObject` and assign that to the given `MatrixObject`. This copy doesn't memcopy the device memory.

Parameters:
  `source` - a `GPUObject` which is the source of the copy
  `mo` - a `MatrixObject` to associate with the new `GPUObject`
Returns:
  a new `GPUObject` instance

### getGPUProperties

public jcuda.runtime.cudaDeviceProp getGPUProperties()
Gets the device properties for the active GPU (set with cudaSetDevice()).

Returns:
  the device properties

### getMaxThreadsPerBlock

public int getMaxThreadsPerBlock()
Gets the maximum number of threads per block for "active" GPU.

Returns:
  the maximum number of threads per block

### getMaxBlocks

public int getMaxBlocks()
Gets the maximum number of blocks supported by the active cuda device.

Returns:
  the maximum number of blocks supported

### getMaxSharedMemory

public long getMaxSharedMemory()
Gets the shared memory per block supported by the active cuda device.

Returns:
  the shared memory per block

### getWarpSize

public int getWarpSize()
Gets the warp size supported by the active cuda device.

Returns:
  the warp size

### getCudnnHandle

public jcuda.jcudnn.cudnnHandle getCudnnHandle()
Returns the cudnnHandle for Deep Neural Network operations on the GPU.

Returns:
  cudnnHandle for current thread

### getCublasHandle

public jcuda.jcublas.cublasHandle getCublasHandle()
Returns cublasHandle for BLAS operations on the GPU.

Returns:
  cublasHandle for current thread

### getCusparseHandle

public jcuda.jcusparse.cusparseHandle getCusparseHandle()
Returns cusparseHandle for certain sparse BLAS operations on the GPU.

Returns:
  cusparseHandle for current thread

### getCusolverDnHandle

public jcuda.jcusolver.cusolverDnHandle getCusolverDnHandle()
Returns cusolverDnHandle for invoking solve() function on dense matrices on the GPU.

Returns:
  cusolverDnHandle for current thread

### getKernels

public JCudaKernels getKernels()
Returns utility class used to launch custom CUDA kernel, specific to the active GPU for this GPUContext.

Returns:
  `JCudaKernels` for current thread

### destroy

public void destroy()
Destroys this GPUContext object.

### clearMemory

public void clearMemory()
Clears all memory used by this `GPUContext`. Be careful to ensure that no memory is currently being used in the temporary memory before invoking this. If memory is being used between MLContext invocations, they are pointed to by a `GPUObject` instance which would be part of the `MatrixObject`. The cleanup of that `MatrixObject` instance will cause the memory associated with that block on the GPU to be freed up.

### clearTemporaryMemory

public void clearTemporaryMemory()

### toString

public String toString()

Overrides:
  `toString` in class `Object`