Mars, J., Tang., L, Hundt, R., Skadron, K., Soffa, M.L. (2011). Bubble-up: Increasing utilization in modern warehouse scale computers via sensible co-locations. Micro.

Scribers: Mike Legatt, Haishan Zhu

Problem: Many nodes under-utilized due to rule-of-thumb QoS concerns. Because of latency concerns, current solution is to run in isolation to minimize latency risks.

  • Poor energy efficiency/higher costs.
  • Systems are over-provisioned
  • Difficulties in measuring/predicting interference among jobs. Difficulties in measuring all possible configurations.

Side note:

  • Unknown how many groups worry about QOS concerns and underutilize nodes. Not clear that companies other than Google are having that problem. The whole problem is set up by a single reference - trust the other paper that this is an important problem, and here is a data point to solve.

Users: “Data center”

  • Shared system for processing user requests.
  • Typically large system with many nodes.
  • The applications run includes:
    • crawling/indexing
    • maps,
    • processing,
    • YouTube,
    • BigTable,
    • Web Search, etc.
  • Broader usage/definition of “data center”: servers performing scientific computations, cloud data storage, banking, air traffic control, online shopping, webmail, SQL-based databases, etc.
  • Flexibility in partitioning/resource allocation.
    • Can be dedicated to a single application/user or partitioned to multiple users
    • Can be partitioned in terms of time, and/or space, etc.

Side note:

  • Data centers care about power consumption and energy efficiency.
  • Difference between mainframes and data center (Dr. Erez opinion)- mainframe was original word for data center. Now it refers to be extremely reliable, designed to carry out certain tasks that weren’t included above. Connection model a bit different now - mainframe usage tended to be direct synchronous connection or running of batch jobs, now more asynchronous. Nowadays “data center” tends to focus on a group of a lot of nodes.

Scope of this paper

  • Focused only on Google applications run in a particular data center, a collection relatively few of applications.
  • They’re not user applications, they’re Google applications. Might be processing user data, interacting with users, but not really user-generated applications.
    • For example, using BigTable, can run own queries, but not exactly what they were looking at.
  • Applications were somewhat well-controlled (all Google apps).
  • Data center partitioning was not really discussed, but some things were implied:
    • Fine-grained partitioning in space,
    • Did not discuss time-based partitioning (or was coarse-grained). Idea was a 1:1 mapping of application or its portion to particular core, static for awhile.
    • A phone does fine-grained partitioning, context switching several times a second. The data center approach is to assign an application to a core, which keeps running it for a long period of time.
  • Evaluation in the paper used some 6 core Xeons and Opteron systems. An application or user gets a chunk of space assigned to the user and won’t be switched out frequentyly.

Side note:

  • Amazon potentially does fine-grained resolution, but they don’t disclose.

Application: Roughly speaking, two types of applications.

  • User-facing, latency-sensitive
  • Batch/throughput.
  • 15–20 applications consumed the vast majority of CPU cycles.

Side note:

  • In academic papers, shouldn’t need to emphasize using bold/underline. Useful for skimming. Could be useful in defining a new context, but don’t depend on visual style to get the point across.

Proposed Solution:

  • Trying to predict impact of pairwise interactions between applications that share a node
  • Specifically they used the shared memory system.
  • Other shared resource on a node
    • Network: is a bottleneck, but goes through memory.
    • Disk another I/O bottleneck.
    • Power as a shared resource, voltage groups (one core interacts with another core, by affecting the way it physically modules and manipulates shared lines). Power limit exists for cores on a node.

Side note:

  • This paper only interacts with memory, so doesn’t include Turbo mode, hyper threading, etc. They can’t always disable things, e.g., turbo (unless you’re Intel). Could be their methodology could be applied to other resources but not discussed in detail in the paper. If they put all power-hungry applications running simultaneously, could violate power budget. Other resources are important that could have an impact.

Implementation:

  • Characterize the sensitivity of application to pressure on a shared resource (memory)
  • Characterize pressure exerted by applications.
  • Combine these two creates a lookup table, asking if A & B are combined, what the impact will be.

Side note:

  • The equations were a bit unclear - make sure writing is helpful. An example may have been more helpful than trying to create a formalism.

Evaluation: Measured

  • Prediction accuracy,
  • Utilization (subject to QOS requirements).
  • QoS was reported as fraction of violations of pre-determined maximum degradation.
    • QoS violation - for a particular event, the delay requirements were not met.
  • Measure utilization for different policies - how bad can performance get.
  • Evaluate quality of “bubble” (the response characterization tool). A more qualitative evaluation.
  • Effect of different architectures (Xeon and Opteron)
  • Effect of different policies of QoS limits.

Side note:

  • They showed that they can achieve a very low QOS violations, but it’s a binary process - violation or not, gave no context for “goodness”. They gave a metric, but unclear whether it’s good or not. Doesn’t’ discuss how did they quantify violation.
  • They show they can prove the utilization, but unclear what they can get from the improvement. Worthwhile asking - could there be a simpler/better alternative?

Impact to other users:

  • Affecting QOS (but unclear how severe violations are).
  • Accuracy evaluated, but not much discussion of the input sets, how evaluated.
  • They ran benchmarks, but not necessarily real data. Not clear that these factors were taken into consideration.