Continue on the memory
- Memory coalescing in GPU
- DRAM’s burst transfer results in chunks with 64 bytes
- G80: Look up a half WARP / If the threads access the same chunk in a sequence, the accesses are coalesced.
- G92: Look up a half WARP / If the threads access the same chunk, the accesses are coalesced
- Fermi: Dose not do the memory coalescing → rely on the cache to do the coalescing (128bit cache line size from CUDA manual)
no coalescing between the WARPs in G80/G92/Fermi
Before Fermi, there were caches for texture and constants
In Fermi, there are rw caches in L1 and L2. When we allocate a memory, we can ask the device driver to make L1 cachable or not. But we cannot specify non-cachable for L2.
- Control (see the slides)
In G80/G92, each SM has its own WARP scheduler.
In Fermi, each SM has two independent WARP scheduler. Each scheduler is responsible for its own subset of WARPS (left 16 ALUs and right 16 ALUs). If there is a memory access or double type data, only one of the scheduler will use all of the resources.
- Control flow divergence (see the slides)
1) software techniques
- compiler adds predication
- programmer avoids divergence (if (TID > 2) {…} → If(TID / warp_size > 2) {…})
2) hardware techniques
mask stack (see the slides)
- When to predicate and when to diverge
Divergence: no performance penalty if all warp branches the same way (+) / Extra HW cost (-)
Predication: Always execute all paths (-) / Expose more ILP(+)
