11

Simple webchat for server by tobi · Pull Request #1998 · ggerganov/llama.cpp · G...

 1 year ago
source link: https://github.com/ggerganov/llama.cpp/pull/1998
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Sponsor

Collaborator

@tobi tobi

commented

Jun 26, 2023

•

edited

I put together a simple web-chat that demonstrates how to use the SSE(ish) streaming in the server example. I also went ahead and served it from the root url, to make the server a bit more approachable.

I tried to match the spirit of llama.cpp and used minimalistic js dependencies and went with the ozempic css style of ggml.ai.

Initially I went for no-js dependencies but gave up and used a few minimal that i'm importing from js cdns instead of adding them here. Let me know if you agree with this approach. I needed microsoft's fetch-event-source for using event-source over POST (super disappointed that browsers don't support that, actually) and preact+htm for keeping my sanity with all this state,. The upshot is that everything is in one small html file. Speaking of- there is probably a better (and less fragile) way to include the server.html in the cpp binary, but it's been 25 years since I worked with cpp tooling.

(updated screenshot)

image
FNsi, disdi, Senjai, antimora, sg3510, mlmabie, franciscogaluppo, thomaspaulmann, langley, sohooo, and 31 more reacted with thumbs up emojiggerganov, koogle, vincentweisser, patw, jaymeh13, TheBlapse, norohind, fgblanch, bramses, starmorph, and 4 more reacted with heart emojiGreen-Sky, WangHaoranRobin, gianpaj, JackJollimore, 2187Nick, badcc, afloresescarcega, mlmabie, lucidrains, phelipetls, and 9 more reacted with rocket emoji

report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK