Transcript
Hi Mike Matchett with Small World Big Data and I'm excited to be back here with Couchbase, one of my favorite vendors of IT technology. Uh, we're going to explore how they're managing to bring the future about in AI and other ways of working with your data. It's not just about, uh, what was going on five years ago. There's a lot happening, especially in AI. And let's get started. Hey, Jeff, welcome back to our show. Thanks, Mike. It's great to be here. Great to see you again. So, so last time we were talking Couchbase. I mean, you need to catch us up a little bit. Couchbase JSON database. You guys added SQL plus plus. So it had Acid kind of transactions. You might I think we might have talked a little bit about columnar analytics. Got added in there. Lots of great things. Uh where is what is happening with Couchbase in general. And how are you guys starting to see the world? Well, so in general, number one, we're on a tear. Uh, so for anyone who doesn't know us, we're NoSQL database. Json primarily. We live in applications that you use constantly, every single day. Whether you're booking an airline ticket, scrolling LinkedIn, uh, your credit cards are protected, you know, via Fico through Couchbase. We're everywhere. Um, you know, at scale and, you know, and, you know, huge organizations rely on us. What we're seeing right now and as you just said, we've got that really killer, high performance, very flexible database as our foundation. But we've also got mobile capabilities where we're the only vendor who just recently announced, uh, vector search on a mobile device. You don't see that every day. Every day. Uh, we're also, because of our NoSQL origin, we can run on premises in Kubernetes in a fully managed database as a service called Capella all at the same time. And it's got our entire product line available, uh, in, in those different deployment types, mobile, you know, on prem, all the way to mobile. Um, now for environments that have really low resourcing, like, uh, a 737 that was built 20 years ago but still wants, you know, seatback entertainment installed in them, having an edge server that consolidates the information for, I don't know, 132 passengers. We do that now. Um, we've we've extended our platform. Uh, and this is the fun part to talk about not only in columnar analytics, but also in AI services and the AI services that we've done go well beyond just what you hear from most database vendors. They support vector search. Well, of course we do across all of those different platforms, but we've added a number of AI services that handle things like pre-processing your PDF documents and your images and your JSON data for vectorization. Of course, we have a vectorization service capability. We have a model service that actually manages a couple of things. Your interface to your favorite, uh, LM framework, uh, interface to uh, perhaps a privately hosted model that you might pick or we might offer to you, um, or even things like Nvidia's enterprise AI environment. So it's hardware accelerated access to models and all of their tools. Model service handles, all of that. And then finally, we've got the agent catalog. Now, the agent catalog is really clever in that it looks at your, your developer, uh, repository and vectorize all of that, reads all your code and the suspected operation of what your agents are supposed to be doing and then can help you figure out. Well, let's see if I'm making a reservation system. I got one for airline tickets, but now I want to do the rental cars. Well, the agent catalog will tell you take these components from the reservation system, match them up to this other set of componentry for, uh, for rental cars. And that's your beginning, your start for building agents. But so we're going to explode the agent space here. Yeah. Yeah, yeah. So and but what we're really trying to do is serve both our developer constituencies, um, in the both introduction and automation and use of, uh, the rag workflows. Um, because that's a whole brand new area for, for most organizations right now. And give them means to understand the, the value that we might offer to their organization when it comes to running models, when it comes to reducing complexity of all the different data sources that you're going to be, you know, consolidating or or using and wanting to put into a prompt or maybe just capturing the transcript of what an agent and a model might be talking about while they're talking. Right. You're going to want that for things like governance, observability, Ability. Um, and, you know, make sure that the agent isn't drifting, uh, setting up guardrails. The agent catalog is going to help facilitate a lot of that. I can't say I'm going to do everything in there, but philosophically, a lot of that is what, uh, what we're making available. Okay. My mind's just expanding here a little bit, Jeff, because we went from talking about what we used to talk about, which was a JSON database with some nice extra features on it, like Acid compliance and the rest of it to being more like an AI platform, enabling people to build AI agents of all sorts of stripes, effectively costly. Yeah, I just want to just drive down a little bit on this. Let's just start that front part of it. So rag retrieval augmented generation is a big thing today. And and in order to use Rag you got to have your take your data, chunk it out, create vectors for it that are this and then store those vectors. Be able to search them. How does how does how does this how do you approach that. What's what's the efficiency here of doing vectors in the database, particularly in a JSON database. Sure. Well, you start with the idea that everything you're working in, right, that your your your documents, your data, your your, the prompts you're generating, it's all text. Yeah. And it's all semi-structured text as we talked about in the past. So that's great. That's JSON data right. So if you need if you're building programmatically a prompt like, uh, you know, making a mad lib, that's how I teach my sales reps. It's like making Mad Libs for for llms and, you know, and evolving them over time. Well, the JSON data store is great for that because it's very, very flexible. You can change everything on the fly. It's not like a relational database where you have to roll out a whole new schema to the entirety of the application just to add a column, right? Json says is like, well, do whatever you want. And then when you get it to a state where you really, really like it, you check that and make that part of the application. Um, so let's say just using JSON as a as A's data format. Really kind of obvious, but you can double click on that too. That's that I just said for prompts. But what happens when the when the agent or when your thing is running and running at scale. Right. How do you know if the data that you're feeding it. You know, tomorrow's data is different, right? There might be a variable that's, you know, volatile, um, that might change. You know, what the what the LM says. But what happens on the LM side when it gets smarter and retrained, it might change its opinion about everything. So you've got this concern about, you know, two sides of the two sides of this, this rag process, uh, changing the, the the input and the output data. So how are you going to resolve for that and verify that your agent or your programs are doing what you expect them to do? Well, you need to validate it. How do you validate stuff? How do we always validate stuff. We check the new stuff against the old stuff. So again that's a great JSON problem. Take save yesterday's transcript and compare it to today's. Maybe when you save it, you do other. You gather other metadata about it like sentiment of the response for example. And then you can figure out, well, it's saying exactly the same thing. So that's what you know, it's on subject you know, correctly. So it's okay. But if it's not then you can throw exceptions and notify the programmer about, you know, changing the the guardrails that your agent is running within. So there's a lot of interesting having a, you know, a JSON based consolidated unified developer data platform that controls a lot of this stuff. And because it's in one spot and we handle crazy things like key value and, you know, full text search and vector search and analytics and, you know, and, uh, eventing. Right. And then we also integrate with Kafka and all kinds of other different technologies. So you can end up moving or, you know, or centralizing a lot of this data for AI in a single environment. Well, that's going to save you money, right? Especially if you're running like purpose built databases everywhere. Um, and it's going to, you know, and it's going to make your management of all of that data much, much less complex. Yeah, it does seem to me like you want to get everything that you want to run through an LLM, you know, even with the Rag idea into JSON, into this, into a database where the vectors are built for you and served up. And even as you said before, it sounds like the llms themselves become fungible. It doesn't matter which LLM you're talking to, whether it's one you can actually host within Couchbase. It sounds like you've got some of that or working with, uh, you know, like bedrock or working with, uh, NVIDIA out there, that becomes a secondary concern and something people can play with, uh, if they focus on getting the data level. All right. What's what's the real thing when you're trying to make it, you know, when you're trying to work with AI, you're trying you're worrying about the functionality of the software you're writing, not. You know how cool the model is or how really the speed of the environment. That's a concern for people. But you know, the value as a programmer that you're adding is the context of what the agent is supposed to be doing, right? So, um, you know, it all kind of makes sense, uh, in that regard, I guess the other the last thing is, um, that there are a couple of things that, uh, that we keep observing as we talk to customers. One is, um, you know, if you have a complex data architecture and arguably who does not, um, AI doesn't like data complexity for all the reasons we've just kind of run through. And for development teams, the complexity is going to end up shifting as they really do start to build agents, uh, from that data architecture to just that rag, that rag workflow and your AI architectures, uh, you know, that you're creating. So facilitating that, that, that complexity shift, clean up your data is is it's kind of a recommendation that we end up making. But the last piece that we want, you really want to end up talking about is where are the agents working and for whom are they working? Right. So then that gets into situations of, you know, are there users that are involved? Is there a human interaction involved in the agent? Is it working on behalf of someone? And if that's the case, where is it working? Well, it's got to end up probably working on a mobile device because that's our favorite, our favorite interface to technology right now. And you started with that, right? You just you just told me Couchbase is now going down to the edge and is mobile. So that's so. So we're the only vendor who's got, you know, a vector search, you know, the foundation for this running on mobile devices. Right. Straight away. And and I think that's you know, the game is going to be in mobile. You're going to see small models running on mobile devices. You're going to see specialty models that are, you know, they're already starting to to, to to pop out that have having particular subject matter that they're really, really well informed about. Um, and all of that is going to, you know, the ultimate thing that we're trying to establish for people is still maintain that killer user experience that we all expect, arguably off our phones or off our, our computing environments. Um, but while you're working with models which today are slow, but, you know, nobody tolerates latency for very long. So putting your data close to the model is a big deal, right? Running models on accelerated hardware like Nvidia's is a big is going to be a big deal later on. Balancing you're controlling your costs of how you're running these things. What's the frequency of of them running. That's going to be a concern. And those are all kind of, you know, areas that we can we can help with. So I'm really enjoying the being able to describe all of the AI centric advancements that Couchbase is making. Um, because I think we're knocking down a number of those different trust problems that customers are having. Right. Whether it's, you know, trusting, trusting it doesn't hallucinate. Trusting I don't expose my data. Trusting that the agent doesn't drift. Trusting that I don't overrun my budget. Right. All of those things we've, you know, we've kind of touched on as we're as as we're having this conversation. Yeah. I really like the story that you're coming out with here because you're not saying you're an AI solution. What you're saying is you're you're this enabling platform and you're kind of future proofing an organization's investments in an AI direction to whatever they want to build its agents at the edge. If it's working with large agents, if it's working with larger llms, if it's working with small ones, if it's distributing data, and even if, you know, a year from now, we get back together and you find and people find out, oh, llms are over passé, we're doing something else entirely, right. You know, this this still sounds like the right thing to do today is to get your data into a format like this with the built in embeddings, and get it available everywhere so that whatever you need to do next year, uh, you can do it. Yeah. It's like, you know, for lack of a better word, the data engine that is going to end up either facilitating the exercise so fueling it or, you know, regathering up all of the artifacts from it and recycling it, because that you're going to want to do that with AI, right? Just because of its notion of, uh, of changing its mind, you know, in the future. Um, so, yeah, you know, I think exactly what you said is, you know. I yeah, I think going forward and people ask me, where should I start? Where should I stick the stick, the shovel in? I'm definitely going to say start here. Right. Like, like this is where it makes sense to, uh, to to to get going. And there's a lot of people out there. I mean, we've seen all the studies, stalled projects, projects that didn't work out. You know, people who are like, ah, is this thing going to lose weight? Am I going to lose all my best customers? Am I going to be egg on my face when it misspells strawberry? You know, like whatever it is, right? Uh, and yet and yet here is something practical. Uh, and it will continue to add value. Because it's not. It's not really, uh, just about one thing. It's about getting your data in a usable format that can be used by, you know, like you said, by columnar analytics, new forms of llms, new new ways of doing retrieval, new ways of feeding the data to the edge. Right. You're just this is a data enablement exercise. And definitely, definitely what I'll tell people to start with going forward. Yeah. No, that's a great summary of it is, you know, being the data engine for for all of this activity and, and you know, and having many of the controls, you know, already built in or facilitated of the things that you really, you know, are going to want to do when these things are running autonomously because people are terrified of autonomy of, you know, agentic autonomy, uh. You know, for many good reasons. Yeah, absolutely. Everybody's going. To want to say, okay, on your device to make sure that, uh, the agent's doing what you expected it, but, you know. To do that kind of brings me down to, like, sort of this sort of the last philosophical question I want to ask, and then we'll do something practical. But what do you. What do you see? You talked to a lot of people. Uh, what do you see as some of the more exciting, agentic things that people can do today? Uh, that just just you just want to be part of. Well, okay, so there are a handful. Uh, I have a customer called, um, soft acuity. They are making a, um, a super television, basically, for to create a smart operating room. And what they're doing is the TV set is actually, uh, watching. So it's got multiple cameras and multiple microphones everywhere. It's got to be a TV kind of, you know, a form factor because you don't want to have to retool the entire operating room, right? You want to just hang it on the wall. Um, but it's doing things in the background like, uh, authenticating everybody who's in the room. And it's doing that by, like, voice and facial recognition, even while the surgeons are wearing a mask. Uh, because you can't touch anything in a sterile environment. Right. So you can't use your, your thumbprint or, you know, or other things. So it's got to come up with ways to recognize you. And those are great. I initial building block kind of use cases. Um, interestingly, I think because probably because they are very willing to, to share their data, um, is is the, the medical or the healthcare profession seems to be advancing pretty, pretty well in this regard, you know, because they, they know how to handle PII and, and, you know, and as part of their science. Right. They share everything with everybody. So the data, you know, the the data sharing issues and privacy issues are not quite as acute as, you know, an enterprise whose, you know, whose data is their you know, their gem. Um, and you know, and are much more reluctant to, to jump in. Uh, let me, let me, let me say, I mean, we could talk about this for I mean, we scratched so many topics. It's like Pandora's box. I mean, we could talk about this for hours? Uh, let me just ask for the sake of people who might be watching this if if they're at all interested. And they should be if they're following along here, what would you point them at as next steps to take to say, like, hey, you know, they've been given an initiative to do AI in their company. Maybe some things haven't gone as well as they wanted it to be. Uh, and they're looking at this going like, hey, you know, we should we should maybe start here. Um, what would you recommend they do? So, you know, to me, there's a handful of just practical recommendations, like if you've never tried AI. It's about time, right? And so, you know, even try free ChatGPT or, you know, sign up for a, you know, a personal license for the stuff just so that you get familiar with what it's like to prompt and interact with with a model, and you can end up experiencing the, you know, the common problems that everybody you know, has with models like the longer you talk to them, the stupider they get, right? Because they're still maintaining the original context which you've already worked through. That is not the current context of what you want to be doing. So there's that. There's like a coding assistant or copilot right under the same, uh, the same ilk. Um, from my standpoint of, you know, as a vendor of Couchbase, uh, and if you if you are a developer, you know, building applications, sign up for our free, perpetually free Capella trial, you know, Couchbase Capella instance. You know, call it a trial, because about nine months ago was when we, uh, turned it into perpetually free. So as long as you use it. Right, your your account is, is intact, and you can start to see what it's like to be building in, uh, you know, in our, our own NoSQL environment, um, you know, you'll get to enjoy SQL plus plus the, the, you know, SQL syntax for JSON. It's really easy to get going and you can start to test drive what it's going to be like to, to program in and build an application like this. Um, so, you know, that's one of the areas, but, you know, the other is just keep learning about all this stuff because it's, you know, I've been hawking the, you know, this AI stuff for, I don't know, I'm coming up maybe on a year anniversary. And in that time, the, you know, the advancements have changed and have accelerated, right? It used to be like every three months there was something new. And then it was every month. And then it's like every two and a half weeks. There's something big, deep secret, big coming in that is, you know, is perceived as a potential disruptor. And likewise. Right. The amount of information that we're gathering about what's happening in the market has been, you know, maturing and growing as well, to the point where, you know, where everybody thought everyone was doing building agents, you know, nine months ago, you know, they're not they're still kind of stuck in the particular concerns they have today, like, you know, risk, you know, risk of losing their data, The fear of hallucinations, all those kind of common things are still true. And so come up with ways to work yourself through that anxiety, um, so that you can become comfortable with, uh, with making these things. And, you know, in the meantime, we can certainly help with it. Yeah, I agree that would be that would be the final conclusion here. And again, I think this is where I'm going to recommend people forward. It's like start with your data, end of things. Invest in something that would be a base of a pyramid no matter what tomorrow brings. And if there's if there's anything certain, I totally agree with you that rate of change in this place is just getting faster. So. So you make plans for up here. But that's definitely going to change. Just make keep these things solid. And you can be, uh, agile at the top of that. Um, all right. That's that's pretty cool. Uh, and then, uh, there's, there's, there's a lot more things to talk about. You want to give us just sort of a something to look out for. Well, so, uh, yeah, most recently, I'll go back to the, uh, the mobile, uh, idea we just came out with, uh, an edge server. Right? And what that is, is that it's a consolidation server that runs in your local environment, like your house, your warehouse, your airplane, and consolidates the data from multiple mobile clients. Right. We've had Couchbase Lite, our embeddable database running on mobile devices for years. You probably have half a dozen or more instances of it on your phone. And so that's always been synchronizing back to its parent cluster. Parent cloud. But now it can sync directly within the local, you know, the local environment if it's really, you know, resource constrained, like I mentioned earlier on an old airplane or whether you're, you know, in a warehouse that, you know, loses electricity periodically or, you know, the weather's bad because, you know, because that happens. Um, it's, you know, it's a means to keep everything open. I was talking to a customer, uh, a couple of weeks ago who was saying they represent major, major retailers And when the power goes out in the mall, if you go to the mall anymore. When the power goes out in the mall, they close down for business because they've lost all connectivity to, you know, to, to their, their, their parent operation. Well, this kind of edge server would enable you to stay in business and then sync everything back up when, uh, you know, when power comes on or when the generator comes on. Uh, you know, in that kind of instance. So there are real world use cases for this edge environment. So we're really psyched about that. Um, and that's even before you get to agentic AI use cases. Right. That's just peer to peer peer to peer edge computing, right? Yeah, exactly. Yeah, yeah. And you know, our our install base is, you know, crosses the, you know, uh, you know, the breadth of, you know, I've got huge financial services companies using us on premises and in their own clouds by Kubernetes and whatnot. I've got, you know, I've got lots of customers moving over to Couchbase Capella, our database, as a service. It's really good. And then, you know, tons on mobile. We're the only vendor who's got that kind of vector capabilities and AI capabilities across the board there. Um, which I think is pretty cool. And, you know, and I think a pretty good vision of at least how we can help you get into the whole exercise with cleaner data with, you know, more consistent kind of prompt capabilities with more consistent, uh, kind of, uh, post activity, you know, post-deployment, um, management of, of your different agents and your, your AI programs. And I guess ultimately, I'd say get you beyond if you've done any of this stuff. Right. Congratulations on your chat bot. Right. Because that's about where everybody has stopped right now. Right. Customer service. Chatbot. Documentation. Chat bot. Uh, copilot inside your development tool. I have all, you know, we do all of those things too. But what I'm really hoping for is that next generation of, you know, let's go. Really? Really beyond this, like. Like that, uh, smart operating room example. Um, and do something really, really cool. That's not just a, a modification of the chat interface, but it's, you know, it's my CEO says it's the early innings, you know. Yeah. Well, we'll definitely be here when that happens. And we definitely want to talk to you about it when you uncover those things. So come back around and show us, uh, appreciate you coming around today and talking about this. There's a lot to unpack here. I'd encourage anybody who's who hasn't looked at a document database, who hasn't looked at Couchbase in a long time. You really got to go back and look at this. This is really a foundation for just doing anything rag in an easy, supportable, future proof way. Uh, and get started doing the right things again. I said at the bottom of this I pyramid so that you can, uh, take off with it. Um, yeah. Absolutely. Bright future ahead. Thank you, Jeff, for being here. Thanks a ton, Mike. All right, take care, folks. See you later.
