0G Storage
What if any Web2 or Web3 project could access data storage infrastructure for vast amounts of data? What if this data could be quickly queried for data availability purposes while being highly secure and custom-purpose?
This post provides an overview of 0G Storage, our infinitely scalable on-chain database that supports any type of Web2 or Web3 data needs.
We’ll provide an overview of our:
- Dual-lane system for data management
- Erasure coding and data certification
- Layered storage architecture
- Economic model and incentives
By the end, it will become clear that 0G provides an extremely performant data storage that can suit any project’s on-chain requirements.
Highly Efficient Data Storage Using 0G
For any party wishing to store data with 0G, the data must first be provided alongside payment using 0G’s token which is fully embedded into 0G’s main chain. To store this data, it is first erasure-coded which means that the data being stored is fragmented into redundant smaller pieces distributed across multiple storage locations.
0G’s system itself has two parts:
- The Data Publishing Lane: For data availability guarantees, whereby data in 0G Storage can be quickly queried and verified by 0G’s consensus network (“0G Consensus”).
- The Data Storage Lane: For large data transfers to 0G Storage.
For data availability purposes, 0G’s Storage Nodes must coordinate with 0G’s consensus network (“0G Consensus”) to certify that the data truly exists in 0G Storage. We use a quorum-based design whereby a random selection of 0G Storage Nodes is chosen.
Multiple Layers of Data Storage
0G Storage uses a layered design, with each layer having a different focus.
The bottom layer is a Log layer. Log entries refer to general records (similar to files in a regular computer program) and are used for unstructured raw data that is append-only.
On top of this is the Key-Value (KV) layer is where structured data is stored, which can be updated via new entries that are added to log entries. For example, if a particular key were to be updated, a new log entry could include this update (alongside other info included in each log entry).
The KV Runtime coordinates with the Log Runtime. "Runtime" refers to the period when a program is executing and includes the environment or framework supporting its operation.
To summarize:
- Log layer: unstructured data that’s append-only.
- KV layer: structured data with mutability.
For example, machine learning models rely upon data sets that would have this tabular data represented within a log entry. In contrast, any data that is in JSON or MongoDB format would be included in KV format.
The key-value storage also allows developers to manage and maintain structured data with full mutability, even building database-like applications on top of the 0G storage system.
Scalable Storage For Any Budget and Need
0G allows users to store any amount of data, and to choose options such as their storage locations, the degree of data replication, and the duration of storage. For example, one could pay extra for additional data backups, elect to use servers in a specific jurisdiction (such as the U.S.), and choose how long the system stores the data.
In return, Miners, who manage 0G Storage, receive tokens (ZG) as a reward from the network. The system has a “storage endowment” which pays these Miners, on top of fees paid by those storing data.
The total storage reward for any given data entry is independent of its popularity, and the reward is split amongst the number of miners storing this data. So a data entry stored by only a few miners would result in a higher payoff to those miners, while a highly stored entry might lead to a lower payoff for those mining that data.
Pricing has two parts:
- Fee: the fee paid to miners who process storage requests and add new data entries. Users can set a higher fee to incentivize more storage nodes to store that data.
- Storage Endowment: the ongoing reward to 0G Storage miners who provide the corresponding data, such as making data available for data availability purposes. This would be paid by those seeking to prove data availability, rather than the original data storer (though they may be the same party).
Proof of Random Access (PoRA)
To incentivize miners to store data, Proof of Random Access (PoRA) is used. This requires miners to answer random queries related to archived data chunks.
Miners must answer each requested query and compute an output until they find a response that satisfies the mining difficulty (i.e. has enough leading zeros).
Once a miner loads archived data chunks and computes a hash representing this, they will see if their response satisfies that target difficulty. As this is random, miners can increase their chances of reward by contributing more computing power and storing more data.
Fairness
To ensure fairness for those with fewer machines, the mining range is limited to 8 TB of data. This means that those with many machines can simultaneously mine for various data ranges (with each range being 8 TB), while single machines can compete on a single 8 TB data range.
Incentivizing Data Sharing
It is challenging to incentivize nodes to share data, as much of this would happen outside of 0G Consensus and it, therefore, makes sense that nodes would prefer to save their available bandwidth.
0G has a built-in royalty mechanism to incentivize data sharing, whereby 0G provides royalties when a new PoRA mining proof is created based on shared data. For example, if Node A shares data with Node B, and Node B generates a correct mining proof, Node A would be rewarded.
0G Storage Will Power Thousands of Use Cases
It should now be clear that 0G Storage has a unique design that can scale exponentially while supporting any form of data availability request. 0G Storage also serves as the underlying storage infrastructure for any other Web3 project that wishes to use 0G, sharing parallels with AWS and traditionally centralized cloud services, yet fully on-chain and decentralized.
If you are interested in partnering with 0G, please reach out via Discord.
For more information, visit 0G.ai.
