0

ANTHOLOGY — Open source AI

 1 year ago
source link: https://changelog.com/podcast/541
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Changelog Interviews – Episode #541

ANTHOLOGY — Open source AI

with Beyang Liu, Denny Lee & Stella Biderman at OSS NA 2023

All Episodes

Brought to you by

This week on The Changelog we’re taking you to the hallway track of The Linux Foundation’s Open Source Summit North America 2023 in Vancouver, Canada. Today’s anthology episode features: Beyang Liu (Co-founder and CTO at Sourcegrpah), Denny Lee (Developer Advocate at Databricks), and Stella Biderman (Executive Director and Head of Research at EleutherAI).

Special thanks to our friends at GitHub for sponsoring us to attend this conference as part of Maintainer Month.

Sponsors

DevCycle – Build better software with DevCycle. Feature flags, without the tech debt. DevCycle is a Feature Flag Management platform designed to help you build maintainable code at scale.

Sentry – See the untested code causing errors - or whether it’s partially or fully covered - directly in your stack trace, so you can avoid similar errors from happening in the future. Use the code CHANGELOG and get the team plan free for three months.

Rocky Linux – Enterprise Linux, the open source community way.

Fly.ioThe home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog and check out the speedrun in their docs.

Notes & Links

📝 Edit Notes

The common denominator for these conversations is open source AI.

Beyang Liu and his team at Sourcegraph are focused on enabling more developers to understand code and their approach to a completely open source, model agnostic, coding assistant called Cody has significant interest from us.

Denny Lee and the team at Databricks recently released Dolly 2.0, the first open source, instruction-following LLM, that has been fine-tuned on a human-generated instruction dataset and is licensed for research and commercial use. They want to be the platform of choice the future of AI development.

Stella Biderman gave the keynote address on generative AI at the conference and works at the base layer doing open source research, model training, and AI ethics. Stella trained the EleutherAI pythia model family that Databricks’ used to create Dolly - 2.0.

Chapters

1 00:00

This week on The Changelog

2 01:44

Sponsor: DevCycle

3 04:38

Start the show!

4 05:46

We met Beyang 10 years ago!

5 06:08

The mission of Sourcegraph

6 07:22

Adam still Googles, just less

7 08:30

Plugins make models interesting

8 09:35

When did you start thinking about this?

9 12:16

This is a "Eureka!" momement in time

10 13:11

The gospel of text based input

11 15:44

Is this the future interface of Sourcegraph?

12 17:21

Iterating the interface

13 17:59

How can you access Cody?

14 18:27

Cody is open source

15 20:13

How does it get code intelligence?

16 21:58

What about privacy?

17 26:11

GPT for X

18 26:53

Cody vs Copilot

19 29:25

Open source + model agnostic

20 31:22

What's next?

21 33:19

How high up the stack can AI tooling go?

22 36:07

Is this a step change to plateau?

23 38:21

The ultimate flattener

24 42:56

Will AI awallow all of programing?

25 45:52

Sponsor: Sentry

26 50:08

We're fine-tuned

27 50:51

JIT conference presenter

28 52:32

This time 4 weeks ago

29 53:54

Let's generate our own data

30 55:05

All 15,000 Q&A data is open

31 56:12

Verbose is not always desirable

32 56:42

I want my own Dolly 2.0

33 58:14

How did you collect the Q&A data?

34 1:00:39

We thought we'd need more data

35 1:01:40

Dolly proved it could be done

36 1:03:24

Google's leaked memo

37 1:06:06

Databricks' play in this chess game

38 1:08:45

Turning AI on our transcripts

39 1:11:03

Chain or foundational model?

40 1:12:42

Sponsor: Rocky Linux

41 1:15:19

The base layer

42 1:16:27

What should the world know?

43 1:17:40

Where does the money come from?

44 1:18:13

Training LLMs is NOT that expensive

45 1:22:07

Focused on open source AI research

46 1:25:49

Interpreting LLMs

47 1:28:30

Influencing the properties of the model

48 1:31:40

Do you have fear of where this is going?

49 1:32:58

Connecting with Stella and team

50 1:34:07

Stella's news source is their Discord server

51 1:36:22

Outro

Transcript

⏰ Coming Soon

Changelog

We're hard at work on the transcript for this episode!

Sign in / up to access transcript notifications. 💪


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK