Episode 91
· 32:05
Make it unintimidating, but also make it usable. And so for the developer, it's make it incredibly configurable. For the other user, it's like make it really easy. He needs to, like, balance both of those in one product. That's the challenge.
Jack Bridger:Hi, everyone. You're listening to scaling DevTools. I'm joined today by someone who I've known for a very long time and have been friends with for a long time, Hamza Chowdhury, who is the cofounder and CEO of Lightdash. And Lightdash is a BI tool, open source BI tool, that is absolutely blowing up right now. So Hamzah, it's so great to have you on the show.
Hamzah Chaudhary:Yeah. Thanks, Jack. It's exciting to be here.
Jack Bridger:So I know that you're bringing a lot of, like, software engineering best practices to BI, but could you talk a bit about what Lightdash does and what BI is for any devs who may not have delved into that realm?
Hamzah Chaudhary:Yeah. Of course. Yeah. Yeah. So I'll tell you a little bit about, BI, and then happy to to tell you about Lightdash as well to set the scene.
Hamzah Chaudhary:So BI, for folks who are wondering, is business intelligence. And, really, what that means is, for most companies, once you start once your data starts getting more complex than just sort of the the sort of out of the box product analytics, that you might start with as a company and you start needing to aggregate multiple different sources of data, So really simple example might be pulling in Stripe data alongside some of that analytics data, potentially a 3rd or 4th source, could be some user data, could be, you know, click ads data from Facebook, something like that. You start in having different requirements on how complex the data model is and then the visualizations that you wanna create on top of that. And that's really where data visualization tools like BI, like business intelligence tools, like Lightdash become more helpful because you wanna be able to, a, create that complex data model and then have a really easy way for people to visualize and interact with it. So that's a little bit about business intelligence, and and kinda where Lightdash fits in is, you know, we're built on 2 things.
Hamzah Chaudhary:Like, one is built on open source. And so, I'm a big believer in open source software, and so it's, you know, I think having particularly when it comes to user event data and that sort of stuff, having the ability to control how that sort of flows through your system, I think is super important. But then the other thing is, you know, we wanna bring a lot of the best practices from software engineering to the world of data. Most BI tools today kinda live in this ClickOps world where everything is UI driven. Actually, we think about it a little bit differently where we want everything to sort of be CLI and code driven, programmatic as much as possible.
Hamzah Chaudhary:And that was really the the kind of kernel for, for Lightdash.
Jack Bridger:Yeah. And when I first met you, you were a software engineer, a lot I feel it was more than 10 years ago, and you were also on the on the weekends occasionally, teaching coding, with with a a bunch of us. And I know that you're also working with Cytora, which was, like, a very, very data intensive company. So I have a little bit of background about, like, where where you got to Lightdash, but could you share a little bit about, like, how you came came to Lightdash?
Hamzah Chaudhary:Yeah. Absolutely. So, yeah, like you mentioned, started off as a software engineer, a long, long time ago. And yet the last company I was at, Cytura, where you and I you know, I met you and I was working there. So I joined as their second engineering hire, and, ultimately, I ended up being responsible for all of our data pipelines and data analytics.
Hamzah Chaudhary:It's not involved everything from data ingestion through to data modeling and then also outputs. So we were building machine learning models at the time, serving those up to folks in sort of the insurance and banking industries. And and what I learned was that our data teams basically were woefully underprepared from a tooling perspective compared to the software engineering side of the business. Interesting. It was kinda like this wild west where, you know, on the data side, it was just sort of tumbleweeds, CSVs being thrown around, things being saved in Google Drive.
Hamzah Chaudhary:It was like no idea about, like, things like Git version control, like, productionization in terms of, like, having preview and dev environments, production environments, all that sort of stuff. So one of my first jobs when I when I sort of took over that team was literally just thinking about what tools can I empower the team with that will help them sort of, you know, get some of that leverage that software engineers have enjoyed for so long, and it makes engineers really good at being able to sort of scale their impact and and kinda gain leverage? It turns out there really isn't that much, like, in the data world. Like, those tools just didn't exist. And so when I left Cytora, it's also where I met my now cofounder, Oliver.
Hamzah Chaudhary:So Oliver and I kind of became really interested in this concept of, like, treating data people more like software engineers. And if you did that, what would the toolset what would the workflow look like? And, you know, after a couple of iterations, we landed on Lightdash as this sort of open source business intelligence, which is, like, very code driven, you know, works in the CLI, works with IDEs, works native to to kinda all those things. And that's kinda always been the mission from day 1 is to try and empower, data people to to be able to work more like software engineers. And what we mean by that is just to gain leverage.
Hamzah Chaudhary:Right? To be able to do more with less. That's that's kind of the the approach that we've had.
Jack Bridger:Yeah. And when you when you kinda landed on it, what did it look like at first? Like, did you just kind of go out and throw something out there? Like, how did it
Hamzah Chaudhary:Yeah. No. So, actually, when we started so when we left, the company, we were we were really we really thought we were gonna do bootstrapped. So we were like, you know, we're gonna do this our way. So we set up a consultancy.
Hamzah Chaudhary:So we were doing data consultancy for companies. We're like building and and, building and managing data stacks for them. This was in January of 2020. It was a great time to to start a new company in in the world. Okay.
Jack Bridger:Yep. Great time. What a great 3 months.
Hamzah Chaudhary:It was an exciting time to say the least. But, yeah, one of the companies that we were working with was actually linked to the department of education here in the UK. Interesting. And so what we're doing for them was basically helping aggregate a lot of data around online curriculums that kids were watching during the pandemic. So, you know, a lot of schooling went online, and so the government is really interested in what were the viewership rates for, like, you know, different subjects, all that sort of stuff.
Hamzah Chaudhary:And so we were one of the companies that were helping to aggregate all that data for the Department of Education. And so we built and managed those data pipelines. And, actually, Lightdash was built as an internal tool in initially just for Oliver and I to actually manage that consultancy business because we didn't wanna pay for one of the really big BI tools. And so I was like, you know, I think we can we can build something better because it's just for us. Inevitably, somebody from our customer side saw that product.
Jack Bridger:What's that?
Hamzah Chaudhary:Yeah. They were like, what's that? It looks a bit it looks a bit janky, but, like, I'm kinda interested. And so, you know, we we kinda spilled the beans on what it was, and, you know, they asked to to use it and kind of like, I guess, like, a fair number of startups, like, the rest is history, and that sort of became, like, our main focus. We're really driven to be like, no.
Hamzah Chaudhary:Like, that's just, like, the site project. We're building this consultancy. But, like, a few people pulled it out of us, and then we kind of realized that maybe this is the thing. And then the consultancy kinda stopped and and Lightdash carried on.
Jack Bridger:Wow. That's very cool. And so did you just, like were you just working with, those kind of existing clients?
Hamzah Chaudhary:Yeah. Initially initially, we started by working with those existing clients, and then we started writing the first lines of, kinda light dash in, I think, towards the end of 2020, sort of beginning of 2021, and then put it onto Hacker News. So we we posted it onto, onto Hacker News, and the first comment we got we gave it, like, a really bad title. We tried to be, like, extremely robust of, like, what it was and very complex to to remember what
Jack Bridger:it was just for people's reference.
Hamzah Chaudhary:I I I actually am pretty too embarrassed to remember, but I think it it is something like the the analytics, like, pipeline tool for your, like, data develop it was, like, really, really long and, like, really unhelpful. Okay. And then someone commented saying, like, oh, this is, like, open source Looker. And I was like, that's a much better name. And so for, like, the next, like, sort of year, we ran with that as, like, the the tagline.
Hamzah Chaudhary:So the open source Looker alternative. That's great. That was that was, like, the story of how Lightdash got shared with the world. And then after after we launched our Hacker News, kind thing started picking up far more organically and, like a lot of open source tools, kinda built a community, after that as well.
Jack Bridger:Yeah. And what was, what was it like, when you got that kind of, like, influx? Like, how did you how did you manage that?
Hamzah Chaudhary:Yeah. It's a good question. I think in the in the earliest days of the product, we, like I think there's there's always an excitement, you know, and you get, like, that I think it's called, like, the the Hacker News hug or maybe you could please support the product hunt hug where you get, like, a little spike, and then hopefully, you know, everything everything kinda stays up. I think for us when when we launched Hacker News initially, I don't even think you you if you could spin up the open source version, I didn't even think there was, like, a login page. Like, it was just like a open source app, and there was there was there was very little that people could do with it.
Hamzah Chaudhary:Yeah. But the exciting bit was how many folks kinda hung around because they liked what we were building. So in the earliest days of the product, you know, very few people could actually productionize it. But I was really surprised by how far people took that current state of the product. I think that was probably the thing that gave us more, conviction that this was something that people needed, was that folks were already taking that early version where you you couldn't even invite anyone to it.
Hamzah Chaudhary:It was just new inside this app. And they were, you know, doing really, really impressive things with it. But in terms of, like, you know, how did we deal with, like, an overwhelmingly large number of of kinda inbound users, I think the answer is we probably didn't handle it incredibly well. Like, we just we try to spin up a community and be like, come and talk to us. Tell us what you think.
Hamzah Chaudhary:Yeah. And and that was kind of it. And then but the main thing was gave us, like, enough conviction to to keep building.
Jack Bridger:Yeah. Yeah. And do you remember, like, how you, because it's something that I I launched something recently, and it's, like, a challenge I have. It's, like, how you, like you know, you there's probably just 2 of you at this time. Right?
Jack Bridger:Mhmm. Yeah. And then
Hamzah Chaudhary:there's There's 3 of us. Yeah. It was there were 3
Jack Bridger:of you. Yeah. And it's, like, you're getting tons of requests, and probably each one of them takes, like, a week to fulfill or something. And it's like Yeah. Just can't possibly do it all.
Jack Bridger:Like, how are you kind of Yeah. That's a good question.
Hamzah Chaudhary:So I think there's there's probably 2 things that matter there. I think one is, like, having a focus is really important because you are right when you share stuff. I think it's, like, one thing that I know a lot of open source founders have kinda mentioned, which is because you have this, like, incredible amount of feedback coming in, it's so important to also still be a little bit focused on, like, well, what was the vision when we started and, you know, making sure you're actually, like, still building a little bit towards that. Because, like, folks will always ask for things, but they don't necessarily it's like kinda seeing the forest through the trees and being like, you know, are you still building the thing that you set out to build, and does this still, like, look and feel close to what you you, you know, you tried to do? Of course, you're gonna take user feedback on board, but I think it's still important to, like, have your opinion of how you want the product to to you know, it's the whole, like, faster horse versus a car thing.
Hamzah Chaudhary:And it's like you really wanna make sure you're still building the car that you set out to build. Yeah. I think the other thing is more around, like, like, the sequencing of how what kinds of users you meet. And we got kinda lucky where I think in the earlier stages of of Lightdash, all the requests were very similar. Like, I actually found it a little bit weird that in a week, like, 7 or 8 people would request the exact same thing.
Hamzah Chaudhary:Yeah. So we never had this problem, you know, where you get, like, extremely disparate feature requests. And I think that had to do with just, like, we're really focused on, like, a very specific type of person that we wanted to target at every single stage. And so as the companies, like, evolved and, you know, if you, like, now target, like, different sizes of companies or different people, like, we found that those feature requests tend to, like, sit in the same buckets. It's quite nice from that perspective.
Jack Bridger:And who who are your target? Who are your targets?
Hamzah Chaudhary:Yeah. So we work with it's sort of like a very technical data analyst, and they're not, you know, they're not sort of fully fledged engineers. The way I describe them is they're folks who are, like, SQL savvy, Python dangerous. And but they're, you know, they're they're definitely not sort of, like, fully fledged devs who are, you know, super comfortable spinning up Kubernetes clusters and and kinda managing everything themselves, but they're definitely, you know, IDE ready, CLI native, that sort of stuff. It's like this new, new sort of breed of data analyst where they are definitely more technical than sort of just working in a spreadsheet, and they kinda need tools that kinda meet them at their workflow.
Jack Bridger:Yeah. And so where where do these people hang out? Like, how have you been kind of growing and yeah. Yeah. That's a good question.
Hamzah Chaudhary:So there's, so we work, with a bunch of other open source tools and communities, and probably the biggest one today for for data for the data folks is kinda DBT, has built this, you know, really large, open source community of data practitioners. And they also coined there's, like, this new term now called analytics engineering, which maybe is a better way of describing that persona that I just mentioned to you. Yeah. Python dangerous. Yeah.
Hamzah Chaudhary:Exactly. And and, and so we actually still work really closely with the Deepsea community, and, you know, those sort of folks have been have been really, really good to us. I think, yeah, we we kinda hung out in a bunch of open source communities like that and tried to find other tools in the space that we thought were complementary or kind of believed in the same vision of what we were doing. And that tends to be where folks kind of found us and and probably still do find us. I think, like, the majority of our inbound today still comes from open source communities.
Hamzah Chaudhary:A lot of that from, you know, teams like PBT as well.
Jack Bridger:Yeah. Yeah. That's, that's really cool. And, like, do you would you have any, like, advice for anyone listening in terms of, like, working with when when you're, like, a startup and you start to work with, like, a really established player as, like, a kind of partner
Hamzah Chaudhary:or within their ecosystem? I think it's it's probably what I mentioned earlier, which is to be, like, super clear about your vision for something. Mhmm. I think everybody and this is, like, you know, investors, friends, family, partners, everybody will have, like, a different vision for where they think your, like, product and and company could go. And it it's pretty easy for it to become diluted quite quickly if if you're not, like, try and please everyone.
Hamzah Chaudhary:Yeah. Because you're trying to please everyone and and also, you know, you kinda like, I think as a as a founder, particularly, you often sometimes value other people's opinion more than your own, either because you think someone's more successful or whatever it might be. And I actually think the opposite is almost always true where, you know, you know your problem better than anyone else. And Mhmm. Actually, a lot of folks, like, give out opinions that I find are, like, very fast opinions, but because they're really intelligent, they sound really good.
Hamzah Chaudhary:You can often, like, over index on them, Whereas, like, in actual fact and that that's not to say, by the way, that those people are kinda doing anything untoward. It's just that, like, they're really smart, and they can come up with, like, opinions very quickly. In actual fact, you spend, like, 100 hours thinking about, like, that problem, that product. So kinda like trusting your own judgment on that, I think, is super important for, like, holding your own vision for it. But, yeah, that that's the same is true for when you're working with a big company.
Hamzah Chaudhary:I think, you know, DBT for us have been great. They're an awesome company to work with. But I think it has been really useful for us to always kinda hold in our minds, like, well, what is really important to us? Like, how do you how do how do, like, we make sure we carve our path? And then, yeah, like, the way you get successful partnerships is by having, like, our incentive align with their incentive, and they kinda wanna do something because it works for them and it works for us.
Hamzah Chaudhary:And if you can find that, then hopefully, you'll get, like, commercial success as well.
Jack Bridger:Yeah. Yeah. That that's it's really interesting that you talk about the vision and kind of, like, holding on to, like, what you know, because that totally resonates with me that it can be really easy to just hear something that sounds, like, so legit and, like, from someone so smart. Like, yeah. I guess I I was thinking about this all wrong.
Jack Bridger:And then in open source in general, I know that, like, you've been doing really well in open source. And, I mean, got, like, a crazy number of stars now and seems like tons of activity. What how have you kinda managed to scale beyond, you know, those first, like, you know, people coming in from Hacker News and liking it to, like, where you are now?
Hamzah Chaudhary:So, like, I think with for us, at least, it's been a couple of things. Like, one is, the open source community kinda keeps growing. And so every time we think that we're like, okay. Like, this is probably the level out moment, that just doesn't seem to happen, which is amazing.
Jack Bridger:But a lot of it
Hamzah Chaudhary:has now been that, like, you know, when Lightdash gets rolled out of the company, it tends to go really wide. So every person at the company will have exposure to Lightdash from the data team, but also all the way through, like, marketing, sales, spin ops. It tends to be you know, when a company starts building their core KPI dashboards in Lightdash, that tends to go to everyone. And so we get this, like, really widespread and, you know, with the natural sort of movement of folks through companies. A lot of what we get now is folks moving to a new organization and saying, like, we used to use this really cool tool called Lightdash.
Hamzah Chaudhary:We're we're probably, like, just about old enough where that is true. You know? We're like we've been around for, like, a couple of few years now, so there's folks who use this, like, 2 years ago and are now, like, in a new company and and kind of bringing us there. That tends to be, like, one re like, that that definitely is happening, like, more and more often, which is really, really nice. That's awesome.
Hamzah Chaudhary:And maybe the the other one is more just I think there's, like, a a natural network effect inside open source that is that is really cool, which is, you know, once folks start developing a feature, they also wanna kinda show it off or kind of they contribute back to the core product. They wanna show that off themselves. And so we get a lot of folks who are, you know, building really cool things on top of Lightdash and then sharing that with their own communities or their own sort of audiences. That kind of,
Jack Bridger:like, naturally brings us, like, more folks back to the core product. That's really cool. So they're, like, just is that actually part of, like, core Lightdash, or is it, like, they'll have another repo that, like, kinda plugs in or, like,
Hamzah Chaudhary:yes. It's a bit of both. Like, I I put put them into, like, 3 buckets. Like, one is folks who've actually contributed absolutely massive features to Lightdash where Interesting. I would not have expected them to be able to, like, do something like that, but they've, yeah, put it off in it.
Hamzah Chaudhary:It's incredible.
Jack Bridger:Some people are just amazing. It's unbelievable.
Hamzah Chaudhary:It's incredibly impressive. I'm always impressed by, like, how much people can do, and it puts me to shape for, like, how like, my my engineering ability, I think. And then the the other two versions are, yeah, like, folks who have some sort of plug in or some sort of other stack that they've built on top of Lightdash, with sort of Lightdash powering it almost as like a, you know, like an OEM type type service. Yeah. And then the third one, we actually get folks who just build really cool open source kinda datasets or data visualizations that they share with the community.
Hamzah Chaudhary:So, like, 2 really cool ones that come to mind. It started as a open source, and this company actually now works with us as well. They sort of monitor ocean health. Oh, really? So this company has, like, the single biggest dataset of, like, ocean health from, like, plastic and microplastic pollution.
Hamzah Chaudhary:They use Lightdash to create this open dashboard that sort of anyone could interact with and and kinda build. That was obviously incredibly cool, like, a a really cool, like, use case for the product. And, you know, that that was, like, one way started as someone who built it as, like, something for a conference that they shared, that they were using used our open source technology and ended up, like, rolling that into their company. And then the other end of the spec well, not the other end of the spectrum, but kinda different. We work with a company that has the largest dataset of, like, auction data for Pokemon cards.
Hamzah Chaudhary:No way.
Jack Bridger:Yeah. How much is the most expensive world?
Hamzah Chaudhary:Oh, man. It's I think it's it's in the 7 figures for sure.
Jack Bridger:It's in the
Hamzah Chaudhary:yeah. And so, you know, they wanted a way to share that dataset with the world, and so they did the same thing. They sort of, like, built, you know, a bunch of interactive dashboards and and sort of datasets in in Lightdash and kinda gave that back to the Swiss community. So I think those sorts of contributions I always find, like, really fascinating because there's there's just lots of interesting datasets in the world that I don't get to see, but I I have the fortune of of being able to work with companies that have them. So it's it's super cool.
Jack Bridger:Yeah. That's really cool. I need to check the Ocean one. That's yeah. UK's, oceans, or UK's, like, sea beaches are getting pretty polluted these days.
Jack Bridger:Yeah.
Hamzah Chaudhary:I think so.
Jack Bridger:Amazing. And so we touched on it very briefly, but you talked a bit about bringing software engineering practices to, analytics to BI. Could you talk a bit about that, about what it's been like? Any things that have been challenging or anything surprising that you've kind of learned in the in that process?
Hamzah Chaudhary:Yeah. Definitely. So, yeah, for sure. I think from a business intelligence perspective, like I mentioned, most tools up until today have kinda relied on being very UI driven. So, like, the the status quo is that everything happens through, like, a drag and drop editor or some cloud based IDE, if there is one.
Hamzah Chaudhary:The problem with that, like, is that it's really hard to, like, build any sort of developer workflow, and I always try to think in terms of, like, what's, like, the flywheel of development that exists in your product. You know, like, what small cog turns a bigger cog. And for business intelligence, it's it's really quite straightforward. It's like a data team's responsibility is to try and, like, build these data models that are reusable. So the example that I often tell folks, like, to to kinda bring this to to life a bit more is take revenue inside a company.
Hamzah Chaudhary:Revenue is like a really easy term to understand, but typically it's very difficult to calculate because, like, data is quirky. And in any single business, they're gonna say, like, hey. Like, actually, you need to, like, you know, join these 2 datasets, deduplicate this, and then, like, you know, take away 10 and multiply by 4, and then that's how you get, like, our revenue calculation.
Jack Bridger:Really? I did not realize that was that hard yet. Yeah.
Hamzah Chaudhary:Data data is funky, and, like, it's it's it always should be really simple. It should just be, like, you know, sum the total of Stripe, but, like, it it's a
Jack Bridger:just check Stripe. Yeah.
Hamzah Chaudhary:But I'll I'll put in, like, businesses. It just doesn't end up being that way because things happen.
Jack Bridger:Yeah. I suppose a lot of it's, like, you take I mean, even Code of Uni back in the day, like, we'd have invoices for, like, some things, and then we'd have Stripe for other things. And I can see Yeah. That was, like, tiny company. I can see how it gets, like
Hamzah Chaudhary:It it can get complex quick. And, but, like, what we wanted to do is we wanted folks to codify those sorts of definitions and basically store them in, like, make them immutable as code. And so with Lightash, what we what we do is we have this notion that those sorts of metrics that are key to your business, those should be, like, get version controlled, stored in GitHub, and, you know, very difficult to for people to change unknowingly. So, like, you really have to, like, know what you're doing to change it, but really easy to pull from. And so in Lightdash, there's kind of 2 workflows, 1 for the developer and one for, you know, the nontechnical nontechnical is unkind.
Hamzah Chaudhary:But, like, you know, the business user who is, like, I'm not necessarily the SQL person. I'm just trying to, like, build the report that I'm interested in to do my job. So I'm pulling revenue. I'm looking at it, you know, cut by region, cut by whatever thing is that we're interested in and, you know, move on and and kinda use it to to do my job. And the way that we found to do that effectively is to give folks access to those prebuilt metrics.
Hamzah Chaudhary:So this notion of, like, a metrics layer where you have a bunch of things that are unique to your business and you have those defined and stored as code, and then everything else kind of happens in the UI. And And so from a developer workflow perspective, we kind of built the ability to use things like your local IDE to leverage CLI tooling to make it really quick for data people to build and manage those metrics. So you actually start from code. You write everything. You can have a sort of, like, hot reloading at the same time.
Hamzah Chaudhary:So you have the light dash UI app and, like, as you're creating metrics, you can see in real time, you know, how changing some calculation in your metric changes a dashboard, changes a chart that you've already built. And then when you're happy with it, you can sort of push it to production, send it to staging, open a PR, have validation, have testing. And it's really just like bringing a level of, like, productionization and maturity to that data workflow that gives people, like, reassurance. That's the main thing. Right?
Hamzah Chaudhary:Like, you don't want your COO to be looking at a report or a dashboard that is, like, out of date or broken because someone did something, and there was just no tracking and no audit trail or no version history that you can, like, roll back on.
Jack Bridger:Yeah. This is so cool. This is this is yeah. That's a it's so so it makes it's so obvious, like, when you say it. It's like, that's, like, clearly a much better way to work than, like, dragging stuff around and just
Hamzah Chaudhary:Yeah. I don't know.
Jack Bridger:I guess that's how how how were people doing it before? It's just, like, dragging.
Hamzah Chaudhary:A lot of it was that. And to be clear, I think that's still the right workflow for a a certain set of people. I think it's more recognizing that there are 2 personas involved. Like, there is you know, and particularly, this is unique to, I guess, tools like Lightdash. They become relevant at companies once you have someone whose full time focus it is to to look after the data inside your company.
Hamzah Chaudhary:And so that person's workflow is always gonna be different to the person who's consuming the data. You know, the person who's interacting with a dashboard, like, yes, for them, go UI driven, make it easy to drag and
Jack Bridger:drop. Small change.
Hamzah Chaudhary:Yeah. Make small changes, that sort of stuff. But, like, changing core parts of, like, your reporting infrastructure, that sort of thing is, like, a different workflow. Yeah. And so think of it as, like, levels of abstraction where, like, the least obstructed is, like, you write a raw SQL query.
Hamzah Chaudhary:And so there, you can do anything you want. Right? You can write any as long as you know SQL, you can create any query that you want. And, like, the least obstructed is, like, I ask you to send me a number. Right?
Hamzah Chaudhary:Like, I tell you, like, hey. Tell me how much money we made last week. And I'm just gonna send you the number and, like,
Jack Bridger:that's the number.
Hamzah Chaudhary:And so, like, there are, like, levels in between those of, like, depending on what abstraction you're at, there are different actions that you should be allowed to do. And that's, like Yeah. How do you give people guardrails without giving them stop signs around how they can interact with data without it being overwhelming? That's that's kind of the trick. That's, like, the the trick that the I tools have to figure out, and, like, Dash is is doing that as well, which is, like, make it unintimidating, but also make it usable.
Hamzah Chaudhary:And so for the developer, it's make it incredibly configurable. For the other user, it's, like, make it really easy. He needs to, like, balance both of those in one product. That's the challenge.
Jack Bridger:Yeah. I think that's the sound bite for the episode of, like, guardrails, without stop signs. That's There you go. I actually feel feel like this is quite an interesting thing, and this is a massive aside. But, like, do you think there's any other things which are, like, kind of ripe to be, like, bring software engineering best practices too?
Jack Bridger:Because I've not seen people, like, think about that, but it's, like, such a interesting thing.
Hamzah Chaudhary:I don't know if I'm, like, qualified to say, like, what are the things, like, you
Jack Bridger:could Just throw out loads of topics.
Hamzah Chaudhary:I think, like, you know, when when we talk about Lightdash, we talk about, like, data, like, bringing it to to the world of data. I think, like, that actually spans, like, lots of different areas that not Lightdash doesn't cover all of them. You know? Like, for example, the one that when we started the company that I was really interested in was, like, the machine learning, data science workflow. Even though, like, Lightdash focuses more on, like, data analytics and, like, business reporting, I think, like, there's the people used to see, like, the sexy side of data as, like, the data science, machine learning, like, the model building.
Hamzah Chaudhary:That was the world that I came from. And that world was, like, even worse than, like, the analytics world. Right? Like, it's full of it's very, like, academic for for the most part. There's very little sort of thought going in necessarily into, like, how the workflows can be super optimized.
Hamzah Chaudhary:That's now changing. Now you have, like, tons of companies doing MLOps, thinking about MLOps pipelines, how you deploy, you know, machine learning models to production, all that sort of stuff, and CLMs have kinda changed that entirely. But I still think that, like, the developer workflows for those, like, you know, it's not about copying software engineering and copy and pasting. It's about, like, the principles. Right?
Hamzah Chaudhary:Like Yeah. How do you copy the principles of, like, version control of, like, you know, dry engineering principles, that sort of stuff, and, like, which ones are relevant to this, like, new area. Yeah. So I think that's probably the most interesting one for me is, like, I I see, like, the data science world and LLMs as technology separate from, like, LLMs as a developer workflow. Like, how do you develop Yeah.
Hamzah Chaudhary:Technology. That bit, I think I'm, like, keenly watching because I'm interested to see, like, what the developer workflow will look like there and, like, what parts of software engineering will come over and and keep making sense and which parts, like, won't or will have to adapt. If there's anything, like, super left field, I can't think of, like, what it might be. It needs, like, software I'm thinking about, like, whether, like, cooking or gardening, you know, is, like, right for
Jack Bridger:for, like, get software engineering.
Hamzah Chaudhary:Yeah. But, like, it's I I can't make it stick right now.
Jack Bridger:Yeah. Well, maybe, like, don't repeat yourself if you're mowing the lawn. Yeah. Yeah. That was terrible.
Jack Bridger:I did all that. Yeah. Okay. So, Hamza, I think we're coming towards the end, but what what's next for Lightdash? What what have you got planned?
Hamzah Chaudhary:Yeah. So I think on the product side, we're working on some really, really exciting stuff. I think we've been thinking a lot about what does the world look like once you sort of commoditized, like, this dashboarding or reporting layer for data people? Like, what are the really cool exciting things you would do if you knew that your data was, like, as reliable as an API? So if you did that, like, what other things would you wanna build?
Hamzah Chaudhary:And there's, like, a really exciting, like, number of exploding use cases that we're seeing from customers, everything from, like, you know, building and training AI analysts to actually developing entire applications off of this, like, back end data API. And I think I'm really excited about that world of we call it, like, operational analytics, where you're actually, like, using your data to do other things, like plug them into other tools and, you know, perform actions in other parts of, like, your your business. That's all possible because we're treating data as code and treating it like a software product. So I think for me, on the product side, like, that's definitely, like, this exciting world that we're moving towards. But I'm, you know, I'm grateful that the LightTouch is hopefully gonna be a part big part of.
Hamzah Chaudhary:Yeah. I think that's on the on the product side. And from a team perspective, it's just, you know, I'm very grateful I get to work real with incredible people every day internally and our partners. And we wanna keep growing them. So, you know, we're hiring on the engineering side, on the data side, everywhere.
Hamzah Chaudhary:So, yeah, it's, it's good times ahead for for Lightdash. Amazing.
Jack Bridger:And where can people learn more about Lightdash and about you?
Hamzah Chaudhary:So, with me, feel free to you could find me on on Twitter or LinkedIn. And then like Dash, we have an open Slack community. So you're more than welcome to to join and and kinda learn more about some of the use cases of what folks are building. And and if you're keen to hear more about the product, find us there and on GitHub. If you star the repo, we will plant a tree or remove a kilogram early from the ocean for you.
Jack Bridger:That's awesome. Okay. I will star it if I haven't already. There you go. Okay.
Jack Bridger:Amazing. Well, thank you so much, Hamza, and thanks everyone for listening. Thanks, Jack.
Listen to Scaling DevTools using one of many popular podcasting apps or directories.
