Lecture 12 (10/7/2008) - Architecture of NVIDIA GeForce 8 & 9

()

Last time

  • GPU accelerates 3D rendering
  • GPU pipeline
  • Summary of architecture
    • Slim cores
    • Pack cores with SIMD execution units, shared sequencer and execution context
    • Hide stalls by interleaving threads

Comparison of different GPUs

NVIDIA GeForce 8800/9800 (G80/G92)

  • 16 cores (stream multiprocessors, SMs)
  • 8 SIMD functional units (stream processors, SPs) and 2 special functional units (SFUs) per core
  • 1 multiply-add unit and 1 multiply unit per SIMD functional unit
  • 2 cores per texture processor cluster (TPC)
  • 32 threads (called a warp) executed together on one core; takes 4 cycles because there are 8 SIMD functional units per core
  • This is the GPU used in Lab 2.

NVIDIA GeForce GTX 280 (GT200)

  • 30 cores
  • 3 cores per TPC
  • otherwise same as G92

AMD/ATI Radeon 4870 (RV770)

  • 10 cores
  • 16 SIMD functional units per core
  • 4 multiply-add units and 1 special function unit (can also do multiply-add) per SIMD functional unit. See Anandtech for more details.
  • 64 threads executed together on one core; takes 4 cycles because there are 16 SIMD functional units per core

Details on the GeForce 8800/9800 (G80/G92)

Organization

  • 8 TPCs in the streaming processor array (SPA, i.e. the whole chip)
    • 1 texture unit and 2 SMs per TPC
      • 8 SPs, 2 SFUs, sequencer, I & D caches per SM (a warp executes on this level)
        • 1000 registers per SP (allows multiple threads to have private storage)

Diagram of the entire chip (AnandTech)

Diagram of a TPC (AnandTech)

Execution core

There are 32 threads per warp, and they are executed together on one SM. This means all 32 threads must be executing the same instruction, because there’s only one sequencer for the SM, and all 32 move together. Furthermore, for each warp, its instructions are executed sequentially. The SM can execute 8 of these threads per cycle, and will take 4 cycles to execute all 32 threads.

A SM is given one, but potentially up to eight, thread block(s) to execute at a time. Each block consists of 3 to 16 warps (96 to 512 threads). These warps can be executing different instructions, even though within the warp, all the threads execute the same instruction. Every 4 cycles, the SM chooses a warp that is ready to run, and runs it for the next 4 cycles. There is no delay when switching to a different warp at the end of the 4 cycles. (Note, for G80, there is a maximum limit of 768 threads, or 24 warps, per SM.)

All the thread blocks that go to the SMs execute the same program, called a kernel. (Note, this is a different notion from executing the same instruction in the program.) The GPU will execute all of the thread blocks for a given kernel, and then move on to the next kernel. Thread blocks within a kernel are arranged in a grid, and are addressed by two-dimensional indices.

Scheduling warps for execution

Each SM implements a hardware scoreboard scheduler to determine which warp to run next. (Note that because the scheduler chooses between warps, and not threads, it has much less work to do.) Warps which have all of the operands ready for its next instruction are eligible for execution. The scheduler uses a round-robin policy to select an eligible warp to execute.

The scoreboard keeps track of all register operands of all instructions in the instruction buffer to prevent hazards.

Once selected, the same instruction is executed for all 32 threads in the warp. This takes 4 cycles, because the SM has 8 SPs, which can process one thread per cycle.

In the case of a long-latency memory operation, the currently-executing warp can still proceed, as long as it doesn’t use the new data as an operand in its instructions.