Problem
- For data-dependent large dataset application, it is inefficient to transfer data from host to GPU using host-side file abstraction.
- It is impossible for GPU to directly access file because of lack of file abstraction. A previous work, GPUfs, has provided file access to GPU programs but memory mapped file mechanism is not supported.
Intended users/readers
- GPU developers who write their CUDA program with complex data-dependent access pattern to large dataset.
- GPU architects to provide file system abstraction and/or address translation in GPU device.
Uniqueness of the suggested solution
- GPU-centric application-level virtual memory using Active Pointer
- Pure software address translation for memory-mapped memory region and page fault handler in GPU.
- Adding an additional address translation layer on top of hardware virtual memory
- Memory-mapped file abstraction to allow efficient random access to large dataset.
- Provide address translation in software for memory virtualization with minimal runtime/area overhead.
- Per-thread paging and frequent, concurrent page update
- Efficient hashing for every page in page cache, and page coalescing to transfer between host and device.
- Leveraging latency hiding property of GPUs for address translation and page transfer overhead.
- Enable GPU to access I/O directly and DMA without intervention from CPU
How is the idea evaluated?
- Overhead of Apointer accesses without page fault
- Including latency overhead (single warp), throughput overhead (hundreds of warps)
- Using benchmarks of various compute intensity
- Overhead of software address translation with GPUfs
- Sensitivity studies: number of thread blocks, memory access granularity
- TLB size effect: large entries of TLB is robust against increasing outstanding page access, but no TLB is also good in such a case because of no overhead for TLB updates.
- End-to-end performance with ActivePointers for large dataset application using GPU
- Case: Image collage
- Baselines: CPU, CPU+GPU, and GPUfs only
Was the evaluation in line with the stated user requirements?
- Although accessing active pointers incurs overheads, they show the overall performance degradation is acceptable. However, for large dataset application, the benefit from direct access from GPU to host file system overcomes the active pointer overhead.
- Eliminating unnecessary data transfer between CPU and GPU provides overall benefits in spite of overhead for address translation.
New tools or software techniques
How may users with other requirements be affected?
- Power: what is the additional power consumption resulting from active pointer mechanism? Does that offset the power saved from directly accessing I/O as opposed to accessing I/O via CPU?
- User who does very compute-intensive workloads may not be interested in it, because data movement overhead is relatively very small to the computation in GPU, and computation and data transfer is well-overlapped (latency hiding).
