IDC published a White Paper about the challenge of Big Data Volume in a data-driven world. IDC expects that the data volume will grow from 45 Zettabyte (ZB) in 2020 to 175 ZB in 2025. The data will be produced in various forms like transactional data, text, voices, pictures, videos, etc.
Annual data size (source: https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf)
What does a Zettabyte mean?
Nowadays, Terabyte, Petabyte, and Exabyte are commonly used data sizes. But soon, Zettabyte will come more into the focus. A Zettabyte is 1.000.000.000.000.000.000.000 Bytes (1021). A Zettabyte would be more than 200 billion DVDs. The following list contains some storage sizes
- 1 Terabyte (TB) = ~1012 B
- 1 Petabyte (PB) = ~1015 B
- 1 Exabyte (EB) = ~1018 B
- 1 Zettabyte (ZB) = ~1021 B
- 1 Yottabyte (YB) = ~1024 B
Data management
The growth of data is still a challenge for proper and cost-effective data management. Some thoughts about handling huge data volumes:
- Today databases can already handle petabytes or even exabytes of data. The bandwidth to bring the data in is becoming more and more a bottleneck. Data can be analyzed on the edge first to avoid transferring all data, especially repetitive data.
- Enterprises make many copies of their production data. Its is said that up to 10 copies are common (e.g. for test or development environments, replication, backups, etc). The number of full copies must be reduced especially for data that is not mission-critical. Technologies like compression are important but not sufficient.
- Data catalogs are essential for working with data: modern metadata management systems help in searching for data, collaboration on data and governing&securing data.
Data management is key to get value from data in a data-driven world with huge volumes, high velocity, broad variety and ambiguous variety.
Data-driven world (Source: https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf)
