Back
0g
Nov 13, 2024
0G Demo: A Technical Walkthrough
In this technical walkthrough, we’ll dive into three demos from DevCon that showcase the powerful capabilities of the 0G Decentralized AI Operating System (dAIOS).
0G’s dAIOS brings decentralized AI to life. Traditional AI systems often lack transparency, security, and user control, essential for an open and trustworthy AI ecosystem. By providing the infrastructure to bring AI on-chain, 0G addresses these issues and empowers developers to build, scale, and monetize AI solutions without relying on centralized providers.
Our architecture has 3 components: Service Marketplace, Compute Network, and Storage Network.
In this article, we will cover:
A brief overview of our Service Network and how anyone can build decentralized AI applications using it.
Three demos illustrating how easy it is to use
Demo 1: Uploading and Downloading AI Models on 0G Storage
Demo 2: Registering as a GPU Provider with 0G Serving
Demo 3: Selecting a Model and Provider for Verifiable Inference
How to get started
The 0G Service Marketplace
The 0G Service Marketplace connects developers and AI service providers in a decentralized peer-to-peer marketplace. Here, “Service Providers” list their offerings, from model inference and fine-tuning to data processing, while developers or "Users" can browse and select the best options for their AI applications.
Service providers have the flexibility to configure their listings, setting dynamic prices based on factors like model complexity, resource demand, and verification requirements.
Users benefit from a range of available models and datasets, as well as the choice of verifiability methods such as OPML, TeeML, or ZKML (with more options like FHEML shortly), allowing them to select the best option for their trust and performance needs.
Apart from 0G Service Marketplace, two other important components of 0G include:
0G Storage: A high-performance decentralized storage solution optimized for large datasets and AI models, featuring advanced sharding and erasure coding for speed, scalability, and data reliability.
0G DA: An infinitely scalable data availability layer that ensures distributed access and verification for on-chain data, supporting the transparency and reliability of AI applications in a decentralized environment.
The 0G Service Marketplace, in combination with 0G Storage (our storage network) and 0G DA (our compute network) form 0G’s dAIOS system that’s bringing decentralized AI to life.
Demo 1: Retrieving an AI Model from 0G Storage for Finetuning
Demo #1 highlights 0G Storage, our decentralized storage network engineered for AI's high-performance demands. Unlike traditional on-chain storage solutions, 0G Storage is optimized for exceptional speed and scalability, seamlessly handling large datasets, intricate AI models, and AI-generated assets.
This brief demo highlights how with just a few lines of code, anyone can quickly and easily upload or download the Llama-70B model, a large-scale AI model. This showcases the capability of 0G Storage to manage extremely large files seamlessly, which is essential for high-performance AI workloads. Built for exceptional throughput, our infrastructure can achieve download speeds of up to 5GB/s, making it ideal for demanding AI applications.
The Storage SDK are CLI Code are easy to get started with, using just a few lines of code.
One key enabler of this is that we use data sharding for accelerated speed, breaking down the file into 4 shards. This is covered in our article on 0G Storage.
This demo also provides a code walkthrough of the 0G Storage SDKs. This includes closely examining our Go SDK functions, with upcoming support for TypeScript and Python SDKs, enabling developers to interact with the storage network efficiently. A CLI option is available for testing, and the open-source repository allows developers to customize and contribute by building their own client interfaces.
Demo 2: Registering as a GPU Provider with 0G Service
In this Demo #2, we walk through the process for a TeeML provider setting up an AI inference service on 0G’s decentralized marketplace, highlighting the ease of deployment and verification steps required for service registration.
Here's what’s covered:
GPU Verification: We begin by verifying that the Service Provider’s GPU is in fact an NVIDIA H200. This ensures compatibility and performance standards for the inference service.
The nvidia-smi command confirms the presence of a NVIDIA H200 GPU.
Running the Llama Model Image: Next, the Llama inference image is launched, with ports exposed to accept external calls. This setup allows the Provider to serve inference requests directly through the 0G platform.
The “docker run ... llama-inference-image” command runs the Llama inference image on the NVIDIA GPU.
Broker Service Setup: The GPU Provider identifies the relevant Docker images for the provider-broker service and activates it, exposing the necessary ports for interaction. This broker acts as an intermediary between the marketplace and the AI model.
The “docker compose -f provider-broker/docker-compose.yml up -d” command starts specified Docker containers to begin running, with confirmations confirming this.
Service Registration: The provider sends a POST request with key parameters for their service: pricing, URL, provider name, model, and verification method. This information registers the service on the 0G Service Marketplace, making it visible to potential users.
Verification and Marketplace Listing: Finally, the provider pings the marketplace to confirm that their service is listed. After verifying the registration, the inference service is live and ready to be accessed by developers on the marketplace.
On the developer side, users currently have three verifiability options: OPML, TeeML, and ZKML. In the future, additional verification methods like secure multi-party ML (spML) and fully homomorphic encryption ML (FHEML) will be supported, expanding options for developers seeking verified and secure AI services.
Demo 3: Selecting a Model and Provider for Verifiable Inference with 0G Service
In Demo #3, we walk through the process of verifiable inference on 0G’s Service Marketplace.
In this example, developers start by selecting from a range of AI models, including popular open-source options and proprietary models exclusive to 0G’s decentralized AI Operating System (dAIOS).
Demo 3 shows available Service Providers, alongside information such as the Uptime %.
The 0G Service Marketplace shows details such as the GPU device used, geolocation, and uptime rate, and verifiability type (like ZKML, TeeML, and OpML). For this demo, we choose a model verified with TeeML, ensuring trusted execution through a TEE (Trusted Execution Environment).
Verified providers are marked with a green check, signaling that 0G has confirmed the integrity of their setup. Once the model and provider are selected, the developer is taken to an interactive playground where they can test the model, such as by prompting it to generate a Haiku. This walkthrough highlights the ease and security 0G’s marketplace provides for developers looking to build reliable, transparent AI applications.
Get Started with 0G
The above demos shows how easy it is for any developer to get started:
Easy Integrations: Developers can easily access 0G’s decentralized storage and OG’s Service Marketplace to upload, retrieve, and manage large datasets and AI models.
Fast Deployment: A straightforward setup process means providers can register AI inference services on the marketplace, making their offerings available to developers within minutes.
Flexible Verifiability Options: Developers have access to multiple verifiability options, such as OPML, TeeML, and ZKML, ensuring secure and transparent interactions with AI services on the platform.
To get started, join our Discord and reach out!
Sign up for our newsletter
