Skip to main content

Decentralized AI samples

Intermediate
Concept
DeAI

Overview​

ICP's unique ability to run compute at scale allows AI and neural networks to run directly onchain within a canister smart contract. Canisters can also interact with LLMs stored off-chain through HTTPS outcalls, or stored locally on a user's device.

Image classification​

To showcase this capability, this demo example displays how an AI that identifies an image can be deployed as a smart contract with a frontend and backend, both running onchain.

You can find the source code for this demo on GitHub.

How this example works​

In this example, the ICP smart contract accepts an image as input from the user and runs an image classification inference. The smart contract has two canisters:

  • The frontend canister that contains HTML, JS, and CSS assets that are served in the web browser.

  • The backend canister that embeds the Tract ONNX inference engine with the MobileNet v2-7 model. This canister provides a classify() endpoint that the frontend canister calls.

AI Demo: how it works

ICP features that make it possible​

To make running AI in a canister possible, two key ICP features are utilized:

  1. WebAssembly virtual machine: Canister code is compiled into Wasm modules to be deployed on ICP. The WebAssembly virtual machine supports standards such as the WebAssembly System Interface, which can be supported through a community tool called wasi2ic.
AI Demo: Wasm
  1. Deterministic time slicing (DTS): DTS splits the execution of very large messages that require billions of Wasm instructions across multiple execution rounds.
AI Demo: DTS

MNIST​

One common machine learning model that is used to train data sets is the Modified National Institute of Standards and Technology (MNIST) database which contains a collection of handwritten numbers. This database is commonly used for training different image processing systems and testing different machine learning platforms.

The Rust Burn crate is a deep learning framework package that can be used for creating machine learning programs. This sample will use the MNIST database and the Rust Burn crate to create a simple app where the user will draw a number, and the machine learning model will calculate the probability that the number drawn is a digit between 0-9.

For example, if the user draws the digit 5, the machine learning model will source the MNIST database, and determine the visual patterns present in both the user's drawn number and the number 5 and calculate the probability that the user drew the number 5.

View or clone the GitHub repo for this sample.

This project is based off of one found within the Rust Burn examples.

Vector database samples​

  • Vectune: Vectune is a lightweight VectorDB with incremental indexing, based on FreshVamana written in Rust. See a forum post that explains how it works.
  • ArcMind Vector DB: A vector database that supports text, image, and audio embedding.
  • KinicDAO Vector DB: A high-performance, completely onchain, tamper-proof vector database specifically built for decentralized apps.
  • Blueband: A vector database built based on Vectra, a local vector database for Node.js. See their forum post for additional details on how it works and demos.
  • ELNA Vector DB: An open-source and fully onchain vector database and vector similarity search engine primarily used to power the ELNA.ai application.