{"id":22238,"date":"2025-04-15T10:57:56","date_gmt":"2025-04-15T03:57:56","guid":{"rendered":"https:\/\/gcloudvn.com\/?p=22238"},"modified":"2025-04-23T11:04:45","modified_gmt":"2025-04-23T04:04:45","slug":"colossus-under-the-hood-how-we-deliver-ssd-performance-at-hdd-prices","status":"publish","type":"post","link":"https:\/\/gcloudvn.com\/en\/kienthuc\/colossus-under-the-hood-how-we-deliver-ssd-performance-at-hdd-prices\/","title":{"rendered":"Colossus under the hood: How we deliver SSD performance at HDD prices"},"content":{"rendered":"
\n\t
\n\t\t
\n\t\t\t

From YouTube and Gmail to BigQuery and Cloud Storage, almost all of Google\u2019s products depend on Colossus, our foundational distributed storage system. As Google\u2019s universal storage platform, Colossus achieves throughput that rivals or exceeds the best parallel file systems, has the management and scale of an object storage system, and an easy-to-use programming model that\u2019s used by all Google teams. Moreover, it does all this while serving the needs of products with incredibly diverse requirements, be it scale, affordability, throughput, or latency.<\/span><\/p>\n\n\n\n\n\n\n\n\n\n
Application<\/b><\/td>\nI\/O sizes<\/b><\/td>\nExpected performance<\/b><\/td>\n<\/tr>\n
BigQuery scans<\/span><\/td>\nhundreds of KBs to tens of MBs<\/span><\/td>\nTB\/s<\/span><\/td>\n<\/tr>\n
Cloud Storage - standard<\/span><\/td>\nKBs to tens of MBs<\/span><\/td>\n100s of milliseconds<\/span><\/td>\n<\/tr>\n
Gmail messages<\/span><\/td>\nless than hundreds of KBs<\/span><\/td>\n10s of milliseconds<\/span><\/td>\n<\/tr>\n
Gmail attachments<\/span><\/td>\nKBs to MBs<\/span><\/td>\nseconds<\/span><\/td>\n<\/tr>\n
Hyperdisk reads<\/span><\/td>\nKBs to hundreds of KBs<\/span><\/td>\n<1 ms<\/span><\/td>\n<\/tr>\n
YouTube video storage<\/span><\/td>\nMB<\/span><\/td>\nseconds<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Colossus\u2019 flexibility shows up in a number of publicly available Google Cloud products. Hyperdisk ML utilizes Colossus solid state disk (SSD) to support 2,500 nodes reading at 1.2 TB\/s \u2014 impressive scalability. Spanner uses Colossus to address cheap HDD storage with super-fast SSD storage in the same filesystem, the foundation of its tiered storage feature. Cloud Storage uses Colossus SSD caching to deliver the cheapest storage while still supporting the intensive I\/O of demanding AI\/ML applications. Finally, BigQuery\u2019s Colossus-based storage provides super-fast I\/O to extra-large queries.<\/span><\/p>\n

We last wrote about Colossus some time ago and wanted to give you some insights on how its capabilities support Google Cloud\u2019s changing business and what new capabilities we\u2019ve added, specifically around support for SSD.<\/span><\/p>\n

\n
\n

Table of contents<\/p>\nToggle<\/span><\/path><\/svg><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n