Chromadb on fly.io
source link: https://willschenk.com/labnotes/2024/chromadb_on_fly.io/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
I've been using the model of having all of these fly.io apps as "services", which scale to zero, and have persistent storage. Everything goes to sleep when you don't use it, and takes only a second or to to come up. Pay for what you need. Pretty nifty.
Lets setup a chromadb instance.
Setup the app
fly launch --no-deploy
Lets add a mounts
section to have the database stored somewhere, and
so we'll end up with something like this:
app = 'chromadb-on-fly-io'
primary_region = 'ewr'
[http_service]
internal_port = 8000
force_https = true
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
processes = ['app']
[[vm]]
memory = '2gb'
cpu_kind = 'shared'
[mounts]
source = "chromadb"
destination = "/chroma/chroma"
initial_size = "100gb"
Then we can create a simple Dockerfile
.
FROM chromadb/chroma
And testing
curl https://chromadb-on-fly-io.fly.dev/
{"detail":"Not Found"}
Writing test code to connect
npm i chromadb
// no-auth-test.js
import { ChromaClient } from "chromadb";
const client = new ChromaClient({
path: "https://chromadb-on-fly-io.fly.dev"
});
const collection = await client.getOrCreateCollection({
name: "my_collection",
metadata: {
description: "My first collection"
}
});
const collections = await client.listCollections();
console.log( "collections", collections );
node no-auth-test.js
collections [ { name: 'my_collection', id: '95d68c89-5ee6-42f2-9421-8cb57b8f9aeb', metadata: { description: 'My first collection' }, tenant: 'default_tenant', database: 'default_database' } ]
Adding in auth
This is fully open, so lets add some access control. We'll go with
access token method, which is super simple. Add this to the fly.toml
file.
[env]
CHROMA_SERVER_AUTHN_CREDENTIALS="test-token"
CHROMA_SERVER_AUTHN_PROVIDER="chromadb.auth.token_authn.TokenAuthenticationServerProvider"
fly deploy
Try it again:
node no-auth-test.js
// with-auth-test.js
import { ChromaClient } from "chromadb";
const client = new ChromaClient({
path: "https://chromadb-on-fly-io.fly.dev",
auth: {provider: "token", credentials: "test-token"}
});
const collection = await client.getOrCreateCollection({
name: "my_authed_collection",
metadata: {
description: "My second collection"
}
});
const collections = await client.listCollections();
console.log( "collections", collections );
node with-auth-test.js
collections [ { name: 'my_collection', id: '95d68c89-5ee6-42f2-9421-8cb57b8f9aeb', metadata: { description: 'My first collection' }, tenant: 'default_tenant', database: 'default_database' }, { name: 'my_authed_collection', id: '98930e2d-5f72-4a6f-a185-bbe4d606f040', metadata: { description: 'My second collection' }, tenant: 'default_tenant', database: 'default_database' } ]
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK