Package org.apache.sysds.runtime.instructions.gpu.context

# Class GPUMemoryManager

java.lang.Object
org.apache.sysds.runtime.instructions.gpu.context.GPUMemoryManager

public class GPUMemoryManager extends Object
- All cudaFree and cudaMalloc in SystemDS should go through this class to avoid OOM or incorrect results. - This class can be refactored in future to accept a chunk of memory ahead of time rather than while execution. This will only thow memory-related errors during startup.

## Nested Class Summary

Nested Classes
Modifier and Type
Class
Description
`static class`
`GPUMemoryManager.EvictionPolicyBasedComparator`
Class that governs the eviction policy

## Constructor Summary

Constructors
Constructor
Description
`GPUMemoryManager(GPUContext gpuCtx)`

## Method Summary

Modifier and Type
Method
Description
`void`
`clearMemory()`
Clear the allocated GPU objects
`void`
`clearTemporaryMemory()`
Clears up the memory used by non-dirty pointers that are not inside lineage cache
`void`
`free(String opcode, jcuda.Pointer toFree, boolean eager)`
Deallocate the pointer
`GPULazyCudaFreeMemoryManager`
`getGPULazyCudaFreeMemoryManager()`
`GPUMatrixMemoryManager`
`getGPUMatrixMemoryManager()`
`long`
`getSizeAllocatedGPUPointer(jcuda.Pointer ptr)`
Get size of allocated GPU Pointer
`void`
`guardedCudaFree(jcuda.Pointer toFree)`
`void`
`guardedCudaFree(jcuda.Pointer toFree, boolean noStats)`
Note: This method should not be called from an iterator as it removes entries from allocatedGPUPointers and rmvarGPUPointers
`jcuda.Pointer`
`malloc(String opcode, long size, boolean initialize)`
Allocate pointer of the given size in bytes.
`void`
`removeGPUObject(GPUObject gpuObj)`
Removes the GPU object from the memory manager
`String`
`toString()`
Print debugging information

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

## Constructor Details

### GPUMemoryManager

public GPUMemoryManager(GPUContext gpuCtx)

## Method Details

### getGPUMatrixMemoryManager

public GPUMatrixMemoryManager getGPUMatrixMemoryManager()

### getGPULazyCudaFreeMemoryManager

public GPULazyCudaFreeMemoryManager getGPULazyCudaFreeMemoryManager()

### getSizeAllocatedGPUPointer

public long getSizeAllocatedGPUPointer(jcuda.Pointer ptr)
Get size of allocated GPU Pointer

Parameters:
  `ptr` - pointer to get size of
Returns:
  either the size or -1 if no such pointer exists

### malloc

public jcuda.Pointer malloc(String opcode, long size, boolean initialize)
Allocate pointer of the given size in bytes.

Parameters:
  `opcode` - instruction name
  `size` - size in bytes
  `initialize` - if cudaMemset() should be called
Returns:
  allocated pointer

### guardedCudaFree

public void guardedCudaFree(jcuda.Pointer toFree, boolean noStats)
Note: This method should not be called from an iterator as it removes entries from allocatedGPUPointers and rmvarGPUPointers

Parameters:
  `toFree` - pointer to call cudaFree method on
  `noStats` - do not collect statistics

### guardedCudaFree

public void guardedCudaFree(jcuda.Pointer toFree)

### free

public void free(String opcode, jcuda.Pointer toFree, boolean eager) throws DMLRuntimeException
Deallocate the pointer

Parameters:
  `opcode` - instruction name
  `toFree` - pointer to free
  `eager` - whether to deallocate eagerly
Throws:
  `DMLRuntimeException` - if error occurs

### removeGPUObject

public void removeGPUObject(GPUObject gpuObj)
Removes the GPU object from the memory manager

Parameters:
  `gpuObj` - the handle to the GPU object

### clearMemory

public void clearMemory()
Clear the allocated GPU objects

### clearTemporaryMemory

public void clearTemporaryMemory()
Clears up the memory used by non-dirty pointers that are not inside lineage cache

### toString

public String toString()
Print debugging information

Overrides:
  `toString` in class `Object`