Notes on final project
The goal is learn about the hardware by looking at applications and software systems.
• Hardware: Modeling power, area, and performance in term of bandwidth and latency. Do not just think about the ideas, implement things and try to analyze it by modeling and qualify it.
• Application: Pick an application, study the algorithms, pick a subset of algorithms and implement it on an available system such as GPUs and PS3. These are the closest hardware to what we expect from architecture in future.
• Software: Working on compilers, and runtime systems and optimization. Nowadays software systems are making use of data parallelism, you can think of how to modify them for task parallelism.
The final report is due on Dec. 11th. And the tentative presentation days are 12/3 and 12/5 which might be changed because they overlap with MICRO.
GPU Architecture
We will be studying NVIDIA G80. The main competitor is AMD and their GPUs are similar in many ways. The product by AMD is called Radeon 2900, for which the complete specification has been published ( http://ati.amd.com/products/Radeonhd2900/specs.html ). There is a move to expose more GPU internals to programmers.
A Graphic Processor Unit (GPU) takes a description of a scene (colors, light, shape, texture, …) and returns the colored pixels to be displayed on the screen.
A GPU accelerates rendering of 3D scenes.
Review:
A graphics pipeline would comprise the following stages.
- Host creates the scene and transfers it to the GPU.
- Vertex control transforms the scene to positioning vertices. The stream of vertices at this point contains the information about dimension, distance, light, algorithm to be used, etc. It should be noted that elements of this collection of data (stream of vertices) are completely independent. Number of vertices is about Nv.
- VS Transform gets the stream of vertices and runs a small program on each element and generates another stream of vertices with the same number of elements (N¬v).
- VS Geometry runs some other program on the stream of vertices in term of the geometry and generates another stream of vertices (Nv)
- VS Lightening works on how lights will effect the scene. The output is a stream of vertices (Nv).
- VS Setup calculates properties and pixels in the triangle. The output is the stream of vertices (Nv)
- Raster gets the stream of vertices and generates the stream of fragments which has a size of about 10M (Nf). There are more pixels per fragments as they will fall on top of each others.
- FS (Fragment Shader) finally generates the output in a stream of fragments.
- There is also a Frame Buffer Memory at the end.
Question : Which parts of the graphic pipeline tasks are programmable? Answer: Vertex shader, fragment shader, and ROP.
Stream Execution Model
- Streams of data are intrinsically data-parallel.
- A kernel is a function that acts upon streaming data.
- Unit of Execution is processing of one stream element in one kernel –defined as a thread
| kernel | → | kernel |
- The stream is divided into chunks. This is called strip mining and each chunk is called a strip/block/tile.
- Each unit of execution is processing one block of data by one kernel which is defined as thread block.
Programming Model
The future of GPUs is programmable processing so build the architecture around it.
Main parts of the GPU:
- Host
- Input assembler: compiles ad produce a vertex command; blocking is also happening here.
- Vertex thread issue: assembles the commands;
- Stream Processors, Texture units, L1 caches
- L2 caches
- Frame Buffers
Load Balancing
Vertex and fragment processing share unified processing elements.
Load balancing hw is a problem: there should be a fair balance in distribution of hw resources. On the other hand, load balancing sw is easier.
Vertex and Fragment Processing is Dynamically Load Balanced: There is an anti-correlation between pixel workload and vertex workload. When the pixel shaders are busy, vertex shaders are free, so that the overall shader usage is unified.
Alternative Operating Mode Specifically For Computing
Main parts:
- Host
- Input assembler
- Thread execution manager: manage thread blocks (only one kernel at a time)
- Texture Processor Cluster (TPC):
- L2 caches
- Frame Buffers
GeForce-8 Series HW overview
Streaming Processor Array (SPA): SPA is an array of Texture Processor Clusters (TPC). Each TPC has one Texture Unit(TEX) and two Streaming Multiprocessor (SM) which provide 20+ GFLOPS, and have 32KB in registers, DRAM texture and memory access
Streaming Multiprocessor Structure:
- Instruction L1
- Data L1
- Instruction Fetch/Dispatch
- Shared Memory: 16KB
- 8 x Streaming Processor (SP): Scalar ALU for a single thread with 1K registers
- 2 x Super Function Unit (SFU)
Thread Life Cycle in HW
- Kernel is launched on the SPA: the first kernel should be completely done and all the outputs should be stored to the memory before the second kernel can be launched.
- Thread Blocks are serially distributed to all the SM’s
- Each SM launches Warps of Threads
- SM schedules and executes Warps that are ready to run: dynamically out of order scheduler runs the warps.
- As Warps and Thread Blocks complete, resources are freed. Threads are running concurrently.
SM Executes Blocks
- Threads are assigned to SMs in Block granularity
- Threads run concurrently
SM Warp Scheduling
- SM hardware implements zero-overhead Warp scheduling
- 4 clock cycles needed to dispatch the same instruction for all threads in a Warp in G80
