11

Chromadb on fly.io

 4 months ago
source link: https://willschenk.com/labnotes/2024/chromadb_on_fly.io/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

I've been using the model of having all of these fly.io apps as "services", which scale to zero, and have persistent storage. Everything goes to sleep when you don't use it, and takes only a second or to to come up. Pay for what you need. Pretty nifty.

Lets setup a chromadb instance.

Setup the app

fly launch --no-deploy

Lets add a mounts section to have the database stored somewhere, and so we'll end up with something like this:

  app = 'chromadb-on-fly-io'
  primary_region = 'ewr'

  [http_service]
    internal_port = 8000
    force_https = true
    auto_stop_machines = true
    auto_start_machines = true
    min_machines_running = 0
    processes = ['app']

  [[vm]]
    memory = '2gb'
    cpu_kind = 'shared'

  [mounts]
    source = "chromadb"
    destination = "/chroma/chroma"
    initial_size = "100gb"

Then we can create a simple Dockerfile.

FROM chromadb/chroma

And testing

curl https://chromadb-on-fly-io.fly.dev/

{"detail":"Not Found"}

Writing test code to connect

  npm i chromadb
  // no-auth-test.js

  import { ChromaClient } from "chromadb";

  const client = new ChromaClient({
      path: "https://chromadb-on-fly-io.fly.dev"
  });

  const collection = await client.getOrCreateCollection({
      name: "my_collection",
      metadata: {
          description: "My first collection"
      }
  });

  const collections = await client.listCollections();

  console.log( "collections", collections );
  node no-auth-test.js
collections [
  {
    name: 'my_collection',
    id: '95d68c89-5ee6-42f2-9421-8cb57b8f9aeb',
    metadata: { description: 'My first collection' },
    tenant: 'default_tenant',
    database: 'default_database'
  }
]

Adding in auth

This is fully open, so lets add some access control. We'll go with access token method, which is super simple. Add this to the fly.toml file.

  [env]
      CHROMA_SERVER_AUTHN_CREDENTIALS="test-token"
      CHROMA_SERVER_AUTHN_PROVIDER="chromadb.auth.token_authn.TokenAuthenticationServerProvider"
  fly deploy

Try it again:

  node no-auth-test.js
  // with-auth-test.js

  import { ChromaClient } from "chromadb";

  const client = new ChromaClient({
      path: "https://chromadb-on-fly-io.fly.dev",
      auth: {provider: "token", credentials: "test-token"}
  });

  const collection = await client.getOrCreateCollection({
      name: "my_authed_collection",
      metadata: {
          description: "My second collection"
      }
  });

  const collections = await client.listCollections();

  console.log( "collections", collections );
  node with-auth-test.js
collections [
  {
    name: 'my_collection',
    id: '95d68c89-5ee6-42f2-9421-8cb57b8f9aeb',
    metadata: { description: 'My first collection' },
    tenant: 'default_tenant',
    database: 'default_database'
  },
  {
    name: 'my_authed_collection',
    id: '98930e2d-5f72-4a6f-a185-bbe4d606f040',
    metadata: { description: 'My second collection' },
    tenant: 'default_tenant',
    database: 'default_database'
  }
]

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK