Stream Processing
Stream processing is a computer programming paradigm, related to SIMD, that allows some applications to more easily exploit a limited form of parallel processing. Like other parallel programming paradigms, it also exploits the concept of parallelism and locality though in a bit different way.
Stream processor offer efficiency and performance
From the graph present at slide 2 of lecture 18 it is quite evident that
•GPU and cell processors (both are stream processors) give better performance (measured in GFLOPS) compare to other Intel and AMD multi-core architecture.
•Stream processors give better efficiency (measured in mm^2/GFLOPS or Watt/GFLOPS) compare to Intel and AMD architecture.
•As the technology scales performance gap between Stream processors and general purpose processors (Intel and AMD) is going to increase.
•While going from 90nm to 65nm, more than the VLSI scaling, architectural innovations led to the improvements in the performance and efficiency of all the architectures.
How do stream processors achieve performance and efficiency
Stream processor hardware utilizes the following concepts to its advantage.
1. Parallelism - Due to recent trends in the advancements in VLSI, transistor density on a die is doubling every 18 months( Moore’s law). This means lot of functional units can be placed on a die without compromising on area which can be used to exploit parallel execution.
2. Locality - Partitioning and hierarchical control to improve bandwidth. On chip communication is also expensive in terms of power. So locality is also important for power( an most imporatant efficiency criteria).
3. Bandwidth management - Latencies are getting worse and it is difficult to optimize design for latency. So we expose communication and make a throughput-oriented design.
4. Support stream execution model (bulk kernels and stream loads/stores) - Can overlap bulk operations to hide latency.
Also, the software architecture is a bit bit different for stream processors. Software is given more explicit control to minimize HW scheduling and allocation. This comes with a drawback of SW architecture not supporting backward compatiblity since SW architecture follows the changes in HW architecture. But its is very difficult to maintain Sw backward compatibility and expect HW efficieny from software.
Performance and efficiency improvement comes at cost of higher effort required to get the code work. But once the code works it gives very high performance compare to other general purpose architecures.
Stream Execution Model
1. Stream execution model assumes infinite data. Basically input is assumed of a infinite stream. Bite-sized stream of input stream are processed and output is produced for it.
2. Another way to look at Stream execution would be to view it as bulk operations. These bulk operation can be classified in three categories.
BULK GATHER: Reorganize the loads to achieve higher throughput. In this phase all the input data are brought from the memory(lot of loads) and gathered.
BULK COMPUTE: In this phase gathered data is processed and used in computation of result. Stream execution model assumes lot of data level parallelism during compute phase.
BULK SCATTER: Store the computed result in teh memory so that next bulk operation can use them.
3. Data access are determinable in well in advance. Knowledge of data in well in advance can be used to the advantage by
a. Latency hiding : Hide the main memory access latency/ cache access latency by issuing multiple requests. Also, data can be gathered during the previous compute phase.
b. Blocking : data can be explicitly be made to stay in cache since we know that we are going to use it in near future.
Bulk Operation are good for Hardware
Stream execution model uses bulk operations and bulk operation utilizes the hardware efficiently.
- Explicit parallelism : Bulk operations allows to use all the available functional units. Since hardware is cheap lot of functional units can be used to exploit parallelism.
- Positively Constrained: Always parallel. Always throughput oriented. Stream execution model focuses on finishing big operations efficiently.
- Explicit Locality : In stream execution model, we only touch the data which we are expected/require to use. This increases the locality in data. Also, reuse of data stream reduces global bandwidth. This also improves power.
- Explicit Communication :
- Scalable DLP
- Minimum Control overhead
Software for stream execution model
In the stream execution model, software is given explicit control over execution of the program by the following operations.
- Explicit locality
- Explicit Hierarchical scheduling
- Explicit latency hiding
- Explicit parallelism
