Transcript
Hi Mike Matchett with Small World Big Data. And today welcome, we are going to talk about bringing the cloud down into your very own local laptop and running stuff against a full cloud implementation in your laptop. Well, let's talk about that. I've got LocalStack here who does something to bring clouds to local environments for development testing. Just hang on a second and we'll get right into it. Yum, yum. So welcome, Waldemar. Welcome to our show. I understand you are CTO and, uh, senior executive for whatever you can explain to us how this runs of LocalStack. I'm just curious, before we dive into some of the details about LocalStack and what it does for people, how you got involved in the cloud, end of things, what what got you into this, and how did you get involved in open source at this scale? So thanks so much for for having me here, Mike. So again, let's welcome our CTO and co-founder of LocalStack. So the whole project started back in 2017. I was working for a company called Atlassian, and we had this use case of a lot of infrastructure running on AWS, and we just tried to find ways to effectively and efficiently test these applications on the local machines. Um, so this is really where it all started. Um, so we created this first version of a local emulator of AWS services, and then put it out as an open source project. Um, it was kind of growing there over a couple of years. And then three years ago, we actually got our seed funding and created a company around LocalStack. So that, in a nutshell, is like a very quick rundown of of the history of LocalStack. I think most people are aware of Atlassian, Jira and all the other great things that that they brought together in the development world. Uh, but tell me a little bit more about this, this challenge or what are the challenges involved with, uh, developing towards the cloud if you're a developer? I know I've got my code. I'm obviously writing it on a local machine on that. But what are some of the things I run into that cause friction? Yeah, 100%. So so one thing like if you look at maybe some more traditional application approaches with like, you know, back in the 2000, you maybe have you had your a Java application or something that runs on your local machine, you were able to, in your IDE, set a breakpoint in the code and really, you know, execute and develop things fully on the local machine. That has kind of changed with the cloud, because essentially every change that a developer makes to the application needs to first be pushed to the cloud and then deployed there, which, you know, takes a couple of seconds, maybe even minutes to deploy. And these, you know, these wait times kind of compound over time. Right? So you have to wait for a couple of minutes several times a day, dozens of times a day. It really slows developers down. So this is the main one of the main value propositions of LocalStack really giving you these these instantaneous, fast, real time feedback cycles on the local machine without having to push things to the cloud. That's the main part. And then we also have some some nice additional features in the, in the, in the platform, which we can talk about teams, uh, features around team collaboration. For example, it's very easy to share the state of an instance of LocalStack with your team members so you can collaborate very effectively. And there's also some more advanced use cases like chaos engineering. And we can obviously deep dive a bit more into those in the conversation. All right. So really that level one value here is improving developer productivity reducing those wait times to get to the cloud. I can appreciate that as a former, you know, capacity planner performance kind of manager person, anytime I hit a button and a user hits a button and it takes more than 10s for the answer to get back, you've disrupted their flow and people don't realize just how bad that can be, because it's not just the 10s you've lost, it's the next ten minutes that takes to get back into that flow, or 15 minutes or whatever. Or if I had to get up and go get a cup of coffee. You've lost me for half an hour, right? And that happens to ten, ten times a day. It's a real issue to take someone out of their out of their heads down flow, and doesn't take that much to do it. So this is really interesting. Just sort of another piece of the technology puzzle here, though, is that you're not just providing a dumb API stub set of stubs for something like deploying to Amazon, right? You're you're making this much more productive and functional to test against. What are you doing under the hood? Yeah. That's right. So we really sort of see LocalStack as a as a high fidelity emulator in a sense. So we really sort of give you like a fully functional version of the local of the cloud APIs running on the local machine. So we put a lot of time and effort into making sure that our implementation is on par with the real system. So we, for example, do things around, um, you know, these these public, uh, cloud services have API specifications, which you automatically scrape and detect any changes. So we can always see if there's a change in the cloud service and then start the implementation on LocalStack side. So making sure that there there is the implementation is on par with the real system. Um, so there's a lot of time and energy that goes into that. Um, it's quite a big endeavor, as you can imagine. Like building a local cloud stack is takes quite some, some engineering resources. On the other hand, it it enables some advanced use cases that are, um, I would even argue hard to achieve in the real cloud environment. So just to give you a few examples, um, the team collaboration, as I mentioned before, you can easily snapshot the entire state of a container and then share it with your team members. It's the equivalent of, you know, copying an AWS account from one place to another, which is really hard to achieve, but we can easily do that with with LocalStack. Um, another interesting example is a feature that we call, uh, chaos Engineering. It's a it's basically the ability to inject some faults and errors and latencies into the APIs to kind of see how your application responds and reacts to that, really building these resilient applications that are able to to handle faults and errors at runtime because things do break at runtime all the time. Um, I think even Werner Vogels mentioned that at some point the CTO of, uh, AWS things break all the time. They literally do in the cloud, and making sure that the application is actually prepared for that is is critical, right? Amazon's not going to break themselves for you to test against. Right. You you can't you can't ask them to inject a whole bunch of errors on their side just to help you debug and prepare and do the rest of the things on there. Uh, that's just that's kind of interesting. You can be your own little agent of chaos on your own, on your own machine and say like, well, I got a I got a code in this faulty world. Uh, let me make sure what I do is bulletproof. More than more than that. Um, so so that's that's curious. Uh, and, you know, just from, just from a spending perspective, if I am running my debugging and development cycles locally. Um, but one, I must have more insight and observability to what's going on. I'm just going to guess that that's true. You can correct me. Uh, but I can also not waste as much money, uh, on cloud resources for dev and test kind of environments. Right? So do you see that? Do you see that in your in your client base? Yeah, 100%. So it's not the main value driver, but it's a nice side effect in a sense. Right. So we see that, uh, oftentimes we we see, you know, customers, uh, you know, you know, having these, these development environments where somebody tries to or forgets to, to shut down a cluster over the weekend, maybe. And you incur these, these cost spikes that are kind of, um, hard to, to get under control, especially with developer environments. Um, and with stack. So our solution basically you use your own local compute, right. So you get the same APIs, but you don't sort of run that on, on, on external hardware. Just run on your, on your machine and really has this nice side effect of not having to clean up the dev environments after yourself. Um, so me, for example, myself as a CTO of of our own team, I also we have some automation in place to monitor our own AWS environment. Right? So we have some cleanup scripts that are running once in a while. And then sometimes screen cleanup script doesn't forget to clean up some of the resources or misses some of them. I need to poke my team and say, you know, can you please clean that up because something is still running there and just, you know, having not having this problem in the first place, I think is a really the value proposition of LocalStack. All right. So I can, I can I can get a LocalStack and I can emulate my, my cloud environment to the point where I can be very productive developing, uh, making my programs more resilient. My my development more resilient. Debugging, uh, possibly saving some money. Uh, but you're not just stopping with, uh, you know, Amazon or big one big cloud provider. You've got some other things coming out. You've got version four, uh, just, uh, here and tell us a little bit about what people can look forward to and how you are expanding your ability to help people develop, develop cloud apps. Yeah, absolutely. So we are in a pretty exciting transition right now where we're actually becoming multi-cloud as a as a company. So we've been focusing a lot on AWS so far in our journey. And we've actually put out the first version of a snowflake, um, local emulator. So really allowing you to to run snowflake data pipelines and queries on the local machine, it's something that's come up with our existing customer base because they had this use case around, okay, we are already using the cloud AWS, but is there something like an equivalent for for snowflake as well? So that's quite exciting. It's currently in preview. So we it's not GA yet. We're working with some early adopters to get some early feedback, but we'd very much encourage the audience out there to to reach out to us and give that a try. And for, for the new version 4.0 release. So that's a major release that we're putting out in end of November this year. Um, so we have a whole bunch of, um, new service implementations that we're putting out there. We put a lot of focus on. Again, parity is a pretty strong focus for us, making sure that the implementation is on par with the real system. And we're also putting out some new, um, new services, including a bedrock that's a first version of bedrock, which is an AI service provided by Amazon. And there's been a lot of requests for being able to run some of these AI models locally, giving you these local experience for for developing AI models and applications. Um, so a lot of exciting developments, I think, especially as we sort of push out V4 and then into the next year becoming multi-cloud. We're going to enable also some advanced use cases like cloud migration, like how do you actually migrate workloads from one cloud to another and have like make sure that the interaction, um, across clouds is actually working properly and you can test that before going into the actual system. I mean, that's going to be really useful as people either need to design their applications from the start to be multi-cloud, capable, or have developed in one direction and now want to consider for any other reason why they might want to go to a different cloud provider, arbitrate and get get some advantage there. Uh, help them to help them figure that out. And I just want to I just want to point out again, if folks are following along here, you're not simply, uh, what we call, like, mocking the APIs. You've got an emulation here. So if I'm a developer and I write, say, for example, some S3 code or some snowflake code, it's actually functionally going to do something along those lines and give me a reasonable answer back. Right. This is this is it's not necessarily scalable as a cloud is itself, but it it actually works. Yeah, 100%. So it's really sort of the we call almost like a drop in replacement for the real cloud. Right. So functionally the same same behavior. Um, and I mean, the reason why we're even able to do this in the first place is that if you look at the implementation of a cloud service, a lot of what the cloud has to invest in is security, scalability, operations, uh, you know, incident management and whatnot. And I guess from a, from a functional, functional point of view, like the the surface area of the services is, uh, is it's manageable, right? It's still a complex service, but you can actually run this and develop this on the local machine. So this is the reason why we can even why it's even feasible to develop an emulator in the first place. And yeah, to your point, definitely it's not it's not marketing. It's an actual emulator. Maybe the closest, um, uh, equivalent would be something like an Android or mobile app emulator that runs on your, on your device, where you can actually emulate a mobile device and just develop your application for that. Right. And it's fairly similar to what we are providing here. Right. It's just a simulated or emulated environment for for, which is a close enough proxy for the real system to do development there. All right. And it's it's very it's very interesting that you're getting beyond things that most people think of. As you know, the 100 Amazon cloud Services to things like snowflake, which takes you not just from the developer teams, but now into data, data people, people who warehousing and building things. And it's I mean, there frictions are going to be a little different. They can't just move petabytes of stuff around and try it out. Right. They, they need to they, they, they have other issues with dealing with cloud services just for testing and debugging and development. So that's awesome stuff. Uh, if you had I'm just going to ask you one more question here. If you had some recommendation for someone who's looking at their cloud development pipeline here with complex CI, CD steps and stuff, like, what would you recommend? They, uh, sort of take a, take a first look at and say, this is where we really need to to get Ahold of things. What if you just had to give some, some best practice advice? What would that be? Yeah. So so I think that a great first starting point is already sort of if there is some kind of automation and testing in place, then I think LocalStack is a great way to, um, uh, sort of use use some of your, your, your, your testing infrastructure and, and run that on LocalStack. So I think generally the, the notion of having tests, repeatable tests and pipelines is already a great first starting point. I think it's becoming pretty standard in the industry today to have like pretty decent test coverage software. Increasingly, we're also seeing this in the data space, which makes again like snowflake and other products so, so exciting for us now. Um, but I think really um, analyzing like when you start from an existing, um, CI CD process, maybe to start analyzing, you know, how long does it take to run these CI builds? Is there any, any, any friction potentially, where multiple CI builds are deploying into the same shared environments? That's another thing that we sometimes see with our user base, where you might have like conflicting changes overwriting each other. If you're basically deploying into a shared environment and again with LocalStack, you get these isolated test sandboxes that just run for the duration of the test, and you can basically just turn them down afterwards. And you basically get this full isolation. Um, but I think, again, like having some sort of automation testing in place is already a great first starting point and then sort of starting to to optimize the setup from there is would be my recommendation for practitioners out there. All right. That's some some great advice. Uh, if someone wants to learn more about LocalStack, maybe do some research on it. Uh, try it out. Where would you what would you where would you recommend people start? I'm assuming they're in this situation where they've got some, some issues that they recognize and they, they want to take the next step. Yeah. Perfect. So, um, we have our website. It's LocalStack cloud. That's our sort of main website where you'll find links to our, our blog posts, um, journal documentation and also a link to our web app. So Applocker Stack cloud is our web experience where you can actually sign up for an account and get some of these functionalities that we talked about today. And we also have a fairly active slack community. So there's also a link on the website. I'd encourage you to to join our slack community that's very active exchange around general best practices for for cloud development, how to use LocalStack. And I think that's a great way to get in touch with the community as well and share some feedback. All right. Thank you so much for being here today. There's more coming. I'm sure you guys are just getting going with this, with this concept. There's a lot of great things I can just picture. It can be done with with this kind of functionality to help optimize not just development, but like I said, you know, the data science world. And we didn't even say the words I hear today. Waldemar thank you for that. But that's out there too. And coming. Uh, so thanks for being here today. Uh, check it out and, uh, hope to see you back again soon. Thanks so much for having me. Thanks, Mike. Yup, yup.
