VLSI Interconnect and (Unrelated) Locality Optimizations

This lecture deals with two separate topics: properties of interconnect in modern VLSI and a few tips for locality optimizations on a CPU.

()


Announcements

  • Please sign up for scribing.
  • Project presentations in lieu of exam — held at same time and place of exam.
  • Reading assignment can be done in groups
  • Start forming groups for lab 1.
    • Remember that you will not be partnering with same people for labs 2 and 3 (OK for project and reading).
  • Final project presentation tentatively scheduled for final period. May move to last two classes depending on number of students and rate of progress.


Locality Optimizations in a CPU

  • Bypass networks, reservation stations, registers, caches, and memory
  • Bypass networks
    • No control
    • processor designed so that software doesn’t care
    • research on exposing this as registers to improve locality and eliminate writebacks
  • Reservation Stations
    • Control through instruction scheduling (within compiler)
    • processor designed so that software doesn’t care
  • Registers
    • Large potential — reduce number of loads and stores
      • save instructions, BW, and power
      • look at this a bit in the lab
    • generally, try to do more with every word you read from memory — define arithmetic intensity
      • fuse loops
      • loop unrolling
      • more complex computation
    • Matmul example:
      • use temporary sum
      • more after talking about caches
  • Caches
    • Review of cache mechanism
    • Prefetching, spatial, temporal, and micro-arch
    • Prefetching
      • talk about later when discussing streaming
      • a form of parallelism
      • not a big deal for matmul
    • Spatial
      • Cost optimization
      • only “locality” because of cache organization
      • doesn’t mean we can ignore it
      • spatial locality also improves mem bandwidth
      • try to make things stride-1 or at least come in bunches
      • full stride-1 sometimes called streaming algorithm sometimes
    • Temporal
      • same as registers — fusion, unrolling helps, and more arithmetic intensity helps.
      • watch out for inadvertent cache flushes
      • loop interchange/reordering
      • loop tiling / cache blocking
      • matmul example
    • micro-arch
      • array padding to take care of bank conflicts and set pressure.
  • For performance, optimum != optimal.
  • Not the same for power (depending on what is implemented in microarch and gating).


  • blocking applicable to all storage hierarchy levels and often use similar blocking method with different parameters
    • known as cache aware
    • cache-oblivious next time
    • remember to mention that BW can also play a role in blocking parameters and decisions


Is a Cache Part of the Microarchitecture or Architecture?

  • Technically microarchitecture
  • Arch if worried about performance
    • linked list and array-block example
  • Improving cache behavior
    • Prefetching
    • Refactoring to improve layout and reuse
      • Matmul example
    • Add parallelism (streaming, sort of prefetch)


VLSI Interconnect

Points from assignment

  • Medium-range wires have higher bandwidth density, but in the graph they were shown for a smaller block.
  • Wires were optimized for minimum delay-power.
  • Typically try to minimize repeaters while meeting delay or BW to reduce area overhead.


Definitions

FO4
Delay of inverter driving a load equal to 4 inverters identical to itself. Delay expressed as FO4s remains constant across technologies (from ~2 micron to 65nm).
FO1
Like FO4 but fanout of 1. Usually {$\sim {1 \over 3}$}FO4
{$\mathbf{\chi}$}
minimal M1 pitch. Distances remain constant across technologies if expressed in this unit.
{$\lambda$}
used to be technology independent parameter, but not as much now that circuits are wire-limited.
  • Generally {$\chi = 4 \lambda$}.

Wire Properties

  • Delay, BW (area), and power/Energy, but first R, C, L, l, and V

Physical properties

  • Length (dimensions)
    • can scale with technology (local wires) ({$\chi$} shrinks).
    • but global chip-level lengths are constant or slightly growing (chip dimensions grow in terms of {$\chi$}.
    • Aside, why don’t we make gigantic chips?
    • number of metal layers keeps going up.
    • in general, min wire thickness and pitch roughly {$1.5X - 2X$} per step for local → intermediate → global wires.
    • but wire height also grows with higher layers (especially top-level), which affects resistance (later).f
  • Supply voltage
    • Used to scale down with technology (same ratio as length).
    • Not anymore though, voltage has stabilized ~1V or 0.9V.
    • Why? Balance of leakage and speed.\ (Vth can’t keep going down).
  • Resistance
    • Proportional to length and inversely to cross-section
      • Cu resistivity {$2.2\mu\Omega-cm$}
    • length scales down at same rate as width
    • height tends to decrease slower to reduce resistance (increase aspect ratio)
      • counters effects of thin-barrier and electron scattering somewhat.
    • overall, resistance now increases per {$\chi$}
      • M1 minimal
  • Capacitance
    • 4 main contributors, bottom and top plates and sidewalls to other wires.
    • Side-wall is growing because of aspect ratio.
    • overall capacitance is going down though because of slow improvements in k and a bit because of narrower wires (offset by side-wall).
      • side-wall portion is increasing and with it interference.

Delay

  • Delay is RC
    • Can use LRC style for near speed-of-light, but disadvantages in terms of power and area compared to other methods.
    • Can use capacitive feed-forward low-swing — much better than discussed in future of wires paper
  • Model: {$1\mathbf{FO4}+RC$}
    • R and C are proportional to length so delay is quadratic with length.
    • should stay roughly constant per {$\chi$} because width and length scale the same. But effective resistance goes up (electron surface scattering, thin barrier, …) faster than capacitance goes down (sidewall cap + very slow decrease in k).
  • Local wires that are fixed in {$\chi$} scale well for now, but might be a bit problematic in terms of delay in future tech.
  • Longer wires (fixed mm) area a problem because scale quadratically — what do we do?
  • Repeaters
    • scaling is back to linear
    • power is high (talk later)
      • power optimizations can help
    • vias cause major congestion and problems for CAD tools
      • thus also reducing BW
  • RLC
    • speed-of-light delay (1 cycle for entire chip!)
    • power and area problems
  • Low-swing
    • delay is reasonable
    • better power
    • area (BW)

BW

  • unrepeated wires, simply {$1\over\mathit{delay}\mathit{density}$}
    • delay and density go down with wider wires
      • optimal exists for global mostly
  • repeated, same but per wire-segment (which does scale)
    • {$1\over\mathit{segment delay}\mathit{density}$}
    • repeated wires have increasing BW with technology
  • global wires have higher BW per wire, but fewer wires
    • around 500 50K blocks on .18 so 10X more BW in semiglobal wires
    • another order of magnitude for local wires (just as I promised)
  • low-swing techniques can have higher BW per wire
    • but again, fewer wires because of extra circuits, so perhaps ~constant BW.

Energy/Power

  • Power for unrepeated wires is {$P = \alpha C f V \Delta V + P_\mathit{static}$}
    • short wires, again scale with tech, but power per function decreases just due to C (i.e., slowly)
  • repeated wires are problematic — repeaters are power hungry
    • example for repeated wire:
      • roughly speaking constant .25 - .32pF per repeated mm across tech (depending on who is estimating).
      • V is stuck at ~1V
      • power is .25 - .32mW/gbit/s per mm (or .28 - .32pJ/mm/bit)
    • can marginally improve with parameter optimizations.
  • Capacitive low-swing
    • reduce {$\Delta V$}!
    • 10X drop in power or more
    • area goes up a lot, reducing potential BW
    • power consumed when not transmitting
      • state of the art (Mensink, E. , Schinkel, D. and Klumperink, E. (2007) A 0.28 pJ/b 2Gb/s/ch Transceiver in 90nm CMOS for 10mm On-Chip interconnects. Solid-State Circuits Conference, 2007. ISSCC 2007. Digest of Technical Papers. IEEE International. ((URL)) (BibTeX)) is .028 pJ/bit/mm (.12pJ/bit static!) in 90nm
      • but pitch is 15um instead of .5um
  • bottom line — power is proportional to distance, but specific circuit techniques can have huge impact.

What about off-chip?

  • high-speed serial links
    • 2 - 3 pJ/bit for custom specialized designs from Rambus and Intel
    • 20 - 30 pJ/bit more standard
    • efficient designs are ~5–10gbps per pin pair
      • area is high, .3 mm2 per channel or more (and probably not really shrinking).
      • economically do 1 - 4 Tbps chip I/O
  • Proximity
    • currently 3 pJ/bit but can come down
    • very high BW demonstrated, 10s Tbps possible
  • Optical
    • today > 3pJ/bit and not all that competitive
    • maybe good in future if we have on-chip optical for power dissipation on chip.
  • 3D stacking
    • pins → vias !
    • heat dissipation problems and limited scalability.
      • pushes the problem out but doesn’t solve it.
  • aggressive research seems to be optical.