Lecture 23 (11/19/2007) — Data Parallel Memory System

Lecture 23 Slides

()

Lecture’s Outline

  • DRAM Technology
* DRAM Circuit
* DRAM Optimizations
  • Impact on Memory System Design
* Mismatch between processor and memory bandwidth.
* Throughput Sensitivity to Access Pattern (Locality/Parallelism)
  • DRAM Parameters and Trends
  • Stream Memory Systems (SMSs)
* Organization & Mechanism
* Design Space of SMSs
  • Results
  • Conclusion

DRAM Technology

  • High Density Semiconductor Product
  • Store charge in capacitor
  • Each cell consists of 1 Transistor + 1 Capacitor ($/bit = 2×10−9)
  • Memory capacity requirement is increasing
* Total memory cost = 10–50% of system cost

DRAM Cell

  • Write Operation
* WL is selected, BL is driven high or low and the voltage is forced onto the capacitor.
  • Read Operation
* BL is precharged to Vdd/2, WL is selected, capacitor shares its charge with BL, causing a voltage change ∂V that can be sensed.
* According to capacitive voltage divider:
* ∂V = (Vdd/2) * Ccell/(Ccell + CBL)
* CBL is large capacitance compared to Ccell.
* For a reasonable swing, Ccell should be large.
* Trenchcapacitor is used to increase the cell capacitance since Ccell is proportional to parallel plate area of cell.

Sense Amplifier

* Consist of a PSense_amp and NSense_amp.
* Bit lines (D1, D1*) from two different sub arrays share a sense amp. (Note: These two sub arrays can’t be active at the same time).
* Both D1 and D1* are precharged to (Vdd/2). NLAT (NSense Latch) is held at Vdd/2 and ACT (Active pull up) is held low. Because of this, all four transistors are initially OFF.
* During read operation, one bit line will either goes slightly above or slightly below Vdd/2, while the other line will remain at Vdd/2.
* Next NLAT is brought to GND.
* As the difference between NLAT and bit lines (D1 & D1*) approach Vth, NMOS transistor connected to higher voltage bit line starts conduction (why?, because inorder for transistor to conduct, voltage difference between its gate and source should be greater than Vth)
* This brings the lower voltage bit line to GND.
* After a small delay ACT is pulled high, brining the higher voltage bit line to Vdd.
* Sensing operation waveform

DRAM Array

* Large DRAMs are divided into multiple subarrays.
* Sub array size represents tradeoff between density and performance.
* Many small subarrays improves performance (low bit line capacitance, large bit line swing. However, more subarray means more decoders and sense amplifier, thus reducing the effective DRAM cell density.

DRAM Optimization

Optimize For Capacity

* Use small capacitors
* Use larger subarrays => less decoders, sense amplifiers
* Optimization for capacity comes at the cost of increase access time.

Optimize For Bandwidth/pin

* $0.5-$.10 per package pin
* DDR2 requires 80 pins

Impact on Memory System Design

Bandwidth Mismatch

  • Mismatch exist between BW demand of ALU and BW supply from DRAM
* Use hierarchical memory system that exploit locality for achieving higher bandwidth.
* Memory systems have multiple memory channels to get better BW.

Access Pattern Sensitivity

  • Memory System Performance is very sensitive to access pattern. Why?
* DRAM performance is very sensitive to access pattern.
* Load imbalance between multiple memory channels.
  • If all accesses are mapped to only a subset of DRAM:
* Rest of DRAMs idle => under utilization of DRAM Bandwidth.
  • Switching between read and write accesses require extra timing delay.

DRAM Parameters & Trends

  • DRAM Chip Organization
* DRAM Chip consists of number of banks, each bank is a 2D array.
* Each DRAM access specify (Bank, Row, Column)
  • Several resources are shared both within and across banks.
* Row & Column accesses shared by request path.
* Read & Write accesses use shared data path.
* All banks share request and data path.
  • This sharing of resources imposes strict access rules inorder to achieve high throughput from DRAM.
  • Time to read from different banks is smaller than time to read from different column of same bank, which is smaller than time to read from different rows of same bank. Figure
  • Trends Figure
* Technology improvement allows to push more bits/pin
* Burst size is increasing with technology advancement. (Burst size = Data you get for each request from DRAM)
* tRC: activate to activate within the same bank costs more (because there is a precharge phase between two activates) than tRR: activate to activate between different banks.

Stream Memory Systems (SMSs)

  • Stream load & store operations generate related memory accesses.
* SMSs can exploit parallelism & locality by generating multiple references per thread.

Organization & Mechanism

Stream Memory System consists of:

  • Address Generator (AG): Translate a memory access thread into a sequence of individual memory requests.
  • Memory Channels (MC): Memory Channels allow parallel access to different DRAMS. It consists of:
*Channel Buffer: Holds pending memory requests.
*Memory Controller: Decides what command (precharge, activate, read, write) should be generated.
*MSHRs (Miss Status Handling Registers): Allows non-blocking memory accesses by storing the state of outstanding requests.
  • Cross Point Switch: Connets AG with MC. Figure

Design Space of SMSs

  • AG Design Space
* Single wide AG vs Multiple narrow AG => Intra-thread parallelism vs Inter-thread parallelism
* Width of AG: Number of memory accesses AG can generate each cycle.
  • MC Design Space
* Channel Buffer entries: Restrict the number of pending memory requests.
* Memory Access Scheduling: Determines the order in which DRAM commands are generated per cycle.
*Open Scheduling Policy: Precharge a row when there ‘s no pending request to that row and atleast one pending request to different row in same bank.
*Closed Scheduling Policy: Precharge a row as soon as the last pending request to that row is serviced.

Results

  • Reordering row & column commands is important, but specific memory scheduling policy is not very critical. Figure (First bar in the figure does not reorder row or col commands. Second bar reorders row command but not col commands. Rest does both row & col reordering of commands).
  • Performance sensitivity to memory channel buffer size for different number of AG. Figure