A Dynamic Compilation Framework for Controlling Microprocessor Energy and Performance
Scribe: Michael Z. Lee Date: 2012–09–24 23:56:19 CDT
Problem
The goal of this work is to understand and balance power consumption with performance. The biggest barrier to getting the performance you want is power. You need to cool the chip, so less power means less cooling and less wear means less cost.
This is also a question of how you can take advantage of the knobs given to you in the hardware. Prior work is concentrated on two different areas: Static analysis which looks from high level program structure; and Hardware analysis which looks at run-time processor information.
Users
- Mobile platforms
- All day compute
- Reduced overall cooling cost
Solution
- Look at what the application is doing to get a multi-layer (HW & SW) solution
- A dynamic compiler: software that compiles/optimizes binaries at run-time (Used PIN - extended with dedicated power optimizations)
- Dynamic systems have more knowledge about the running system
- You can use the high-level program structure and the run-time processor information
- Static analysis cannot use knowledge of the inputs/run-time behavior (ie. qsort example, fig 2)
- Hardware analysis cannot use knowledge of the program structure. It doesn’t take advantage of the program structure.
- In some cases (HPC), software is generally predictable and hardware scaling might work better
- But you need to keep this light-weight
- You can use the high-level program structure and the run-time processor information
- Major challenges:
- Your optimization is on the critical path - you need a fast decision because either you need to stall, or you’re inserting code into the instruction stream
- In multi-core: the analysis can be off-loaded onto a different core
- You don’t have access to hardware counters to get a good view of power consumption
- How you can use performance counters to track and understand power consumptions (hueristics)
- On modern processors, you can read a register to find power consumption - Sandy Bridge has a power management unit (PMU)
- Your optimization is on the critical path - you need a fast decision because either you need to stall, or you’re inserting code into the instruction stream
Design
The dispatcher runs and sends it to a classifier to try to group in hot versus cold code. In many programs (especially HPC), 80%−90% of the time is spent in 10%−20% of the program.
Sometimes this fails: GCC does not have a lot of code reuse because it walks straight through the input and doesn’t really go back (one pass for each gcc-optimization). But when you’re running a dynamic compilation system, cycles are wasted analyzing and optimizing code that you rarely repeat.
- Examine functions and loops by instrumenting tests at the entry and exits of a code region
- Functions are nice because it cleanly hot swaps the code - Insert a new jump to make optimzations
- Loops aren’t so clean: they can have multiple exits and register use is indeterminate
- To handle this, generate a new version of the loop during compilation that is amenable to swapping
- How often did you change voltage: Usually on the order of seconds
- The decision model is based on trying to figure out where and when there is some CPU slack time.
- Did not compare to run to idle. The option of shutting down the core is not an option.
- These days, the static power leakage is the real problem. Dynamic power scaling doesn’t make much difference.
Implementation
- Candidate regions: functions and loops
- Adjustable profiling/optimizing granularity
- JIT to generate the code selectively
- And possily apply this to multiple code regions
Eval
- They used it on a real system with real hardware and measurements
- Voltage and current measurement, noise reduction, data acquisition, and data logging
- Baseline starts from a regular gcc -O2 binary
- Lots of different benchmarks (SPEC2K, SPEC95, Olden)
- But willing to compromise a bit of performance (at most 5%) for power savings
- Programs with nice, steady loops give you nice savings
Tech
Yes - more transistors means more power means you need to cool and adjust for the workload. Just running full throttle ends up being wasteful.
Software
Yes - this is a dynamic compiler solution that takes advantage of run-time analysis.
Other Users
Data centers usually try to turn off speed step so they can satisfy latency guarantees.
Bigger Picture
Essentially that you could dynamically insert instructions into the stream of instructions to waste less energy. This paper takes advantage of deep understanding of program execution.
Other comments
How do you handle multi-core systems:
You want to coordinate the frequency across all of the cores. This becomes a scheduling problem. However, it’s possible for the OS/VMM to determine even frequency scheduling.
Data Center:
Often, power management is disabled to protect threads/apps that are latency sensitive.
Processor life-span
Switching frequency often can hurt the processor life-time. Transistors can age and the switch time will slow down.
IO bound workloads
When you reach this level of latency, you want to boot this back to the OS to manage. This work is filling in the dot of micro-lulls that exist due to memory access latency.
