Problem
- Decoupling disk from data storage
For large-scale systems, storage is far away from compute
Relatively lower bandwidth between compute and storage results in higher latency
This is also dependant on applications
- Challenging research problem:
How to evaluate a large-scale system is challengeable
Users
- Data centre providers
Need to improve EDP and reduce costs
- Researchers
Need a solid methodology to evaluate the performance of large-scale systems
Performance
- In some cases, whether the performance is improved has a direct impact on users
- What bounds the performance of large-scale systems?
- Cost breakdown
Cores ( compute )
Network bandwidth
Memory
Storage
Infrastructure ( cooling, power delivery, building )
Energy cost
Maintenance ( repairing )
Software ( a huge cost )
- Systems may go about 2 to 5 years
- Maintenance
5 years (4 * 104 hrs), $0.1 per KW/hr → $ 4000 /KW
- Recurring Cost
100 cores / KW
DRAM → 1 TB / KW ( memory chip < 1 W, about 1 GB per chip )
Disk → 100 TB / KW ( 1~10 Watts / disk, about 1 TB per disk )
NVM → 10 TB / KW
| DRAM | NVM | Disk | |
| Static power On | Refresh + interfaces | Interfaces | Spinning + Interfaces |
| Static power Off | Refresh | \ | \ |
| Access R | Activate array, sense | Activate, sense | Move ahead, sense |
| Access W | Activate | State switch | Noise |
| Move data | Higher bandwidth | Longer distances | |
| comm | Cost more |
- Network (not cheap)
20% overhead
10% building
10% cooling
- Non-recurring Cost
Core → $ 10 ~ 100 each
DRAM → $10 / GB
Disk → $ 0.1 / GB
Network → $ 1 /share
- With $ 1000 to spend,
For cores, corresponding to 10 cores, it costs 100 W
For DRAMs, corresponding to 100 GB, it costs 100 W
For Disks, corresponding to 10 TB, it costs 100 W
For NVMs, corresponding to 1 TB, it costs 100 W
- Much cost with the existence of disks and DRAMs
Cost savings when decoupling disks and DRAMs from storage
- Also improve the performance
Data stored in NVMs stacked on chips close to compute, decreasing the latency
- A lot of money put to memory to allow high parallelism
There should be a balance between cost and efficiency
