Decision Models Like Jev and Clef Are a Game Changer | ✉️ #101

Hey! 👋

There is a new type of models in town - decision models. The one that made everyone aware of this idea was TypeSafe's Jev, quickly followed by lots of alternatives, including CloudFlare's Clef and OpenAI Decision API. While I work on a bigger article on this topic, I wanted to share some initial thoughts on why these models are a big deal - and why they will eventually be internal part of any future frontier LLM model.

Decision models are useful for... making a decision! It sounds weird, but once you look at actual use cases, you will see how many times you need to make a decision, especially when building AI-powered applications.

For example, it's quite common for AI apps to have a Vector search - but Vector search is not entirely accurate sometimes, so it's also quite common to do some extra verification of search results by using an LLM. You could do it with a big LLM (slow and expensive) or a small LLM (inaccurate and still not really cheap at scale) - or you could do it with a decision model.

Decision model gives you a number of primitives, most simple one being a Noul: how likely something is to happen. You take a piece of text about describing a ranch, and you ask a decision model "is this a ranch?" - and it will tell you that 90% likely "yes", 10% "no". If you are into searching properties to purchase and your vector search returns 200 properties, then you can use decision model to find which of those are more likely to be a ranch. Might be a silly example that came to my mind only because I am binge-watching "The Ranch" series on Netflix, but I think you get the idea.

Another application is routing requests to different AI models. When user asks your AI agent something, you can ask your decision model if this kind of query is most likely to be best processed by Opus, Sonnet or Haiku - and depending on the result, your code can route the request to the right model.

What makes this models a big deal is the performance that is more or less close to what you would expect from a real time application. One of the fun things I built was a 3D shooter, where enemy AI was entirely powered by Jev - it would get inputs from its environment and make a decision to move in certain direction, shoot, hide etc. And Jev is fast enough for this kind of experience to feel real time. Imagine what it can do inside your actual serious agentic workflows.

I know it's probably still confusing a bit, and this is why we are working on a bigger article on the topic - as well as testing how far we can push a decision model inside gnum.io, our new product currently in closed-beta. Stay tuned! But also: don't wait for us and just give those models a try.

The 102nd mkdev dispatch will arrive on Friday, October 23rd. See you next time!