Conferences & Meetups  

Ship First, Model Later: A Short Recap of AI.Dev

By Jessica Kerr  |   Last modified on December 19, 2023

In a keynote at AI.Dev, Robert Nishihara (CEO, Anyscale) described the shift: A year ago, the people working with ML models were ML experts. Now, they’re developers. A year ago, the process was to experiment with building a model, then put a product on top of it. Now, it’s ship a product, find the market fit, then create customized models.

The general-purpose generative AI models available to all of us today (such as ChatGPT) change the way work is done. We can start developing on top of them with a few hours of prompt engineering. Products and features come first, because the models are there for us.

After our product is proven useful, it makes sense to optimize the models underneath it. Create or tweak some models to suit this purpose, and they’ll be faster and cheaper than the big ones at OpenAI or Anthropic.

I say models (plural), because that’s another shift: we aren’t using bare LLMs. At a minimum, we supplement them with relevant info (Retrieval Augmented Generation, or RAG). Many talks at AI.Dev described agents that use LLMs to create instructions for other LLMs, LxMs that generate something other than text (like images), and tools that can do math, code execution, database operations, etc. Building features using generative AI is now all about shipping and iterating.

To iterate, you need observability. Honeycomb’s CEO, Christine Yen, showed how we iterated on our Query Assistant feature by looking at inputs and outputs, plus in-product feedback. 

In Christine’s talk, someone asked the question: How do you measure whether a generated response was good? This is an open question. I talked to a conference sponsor working on that, DeepChecks. Personally, I think we’ll wind up asking yet another LLM!

It’s exciting times in tech. Charles Herring, who spoke about Cassandra this time, said he was considering retiring—and then ChatGPT was released. This added broad new avenues of possibility for his work as CTO of WitFoo.

Lucky us, we get to explore these possibilities! I don’t have to learn the intricacies of models first—just a little prompt engineering. I’ll keep doing my favorite things: ship and iterate.

Enjoy conference recaps? Check out our comprehensive KubeCon NA 2023 recap!


Related Posts

Conferences & Meetups  

AI’s Impact on Cloud-Native at KubeCon 2023

Cloud-native developers and practitioners gathered from around the world to learn, collaborate, and network at KubeCon/CloudNativeCon North America 2023 between November 6th and 9th at...

Culture   Conferences & Meetups  

KubeCon NA 2023: What a Conference!

Now that we’ve had time to decompress from Kubecon, we wanted to do a writeup about our collective experience. Six of us spoke at the...

Observability   Conferences & Meetups  

Top Takeaways from Monitorama 2022

Two of our folks went to Monitorama 2022, and they gleaned a few pearls of wisdom they'd love to share with you, including an unexpected,...