1 00:00:00,016 --> 00:00:03,798 Excel formulas are by far the overwhelming 2 00:00:03,831 --> 00:00:07,646 most widely used programming language in 3 00:00:07,710 --> 00:00:11,068 the world. 100%. And 4 00:00:11,504 --> 00:00:14,991 Excel formulas pass every test of what constitutes a 5 00:00:15,056 --> 00:00:18,651 programming language. We just don't really think of it as such. And so 6 00:00:19,150 --> 00:00:22,627 you found that Excel required a 7 00:00:25,186 --> 00:00:28,833 core council of elders that needed that, that stayed 8 00:00:28,849 --> 00:00:32,121 with the product and needed to stay with the product. I'm talking about the engineers, 9 00:00:32,714 --> 00:00:36,306 the software developers. They needed to— like, it 10 00:00:36,306 --> 00:00:39,129 required that kind of core stewardship, 11 00:00:40,476 --> 00:00:44,260 whereas the other applications, you know, you could kind of like, you could 12 00:00:44,260 --> 00:00:47,997 shift the leadership around, you could move people around. Excel changed how the world 13 00:00:48,029 --> 00:00:51,829 works with data. Power BI changed business intelligence. Now 14 00:00:52,038 --> 00:00:55,489 AI is changing the game again. And today we're looking at what comes next. 15 00:00:55,971 --> 00:00:59,747 Welcome to Data Driven. Hello and welcome back to Data 16 00:00:59,747 --> 00:01:02,767 Driven, the podcast where we explore the emerging field of 17 00:01:03,185 --> 00:01:06,752 data science, artificial intelligence, and of course, 18 00:01:06,800 --> 00:01:10,624 without data engineering, all of it is for nothing. Today 19 00:01:10,672 --> 00:01:14,126 we are missing Andy Leonard, who is unavailable to make it here. He's my world's 20 00:01:14,126 --> 00:01:17,227 favoritest data engineer, but we carry on without him. 21 00:01:17,902 --> 00:01:21,275 Today I'm real excited to speak with our guest, 22 00:01:21,821 --> 00:01:25,017 Rob Collie, who is the CEO of P3 Adaptive 23 00:01:25,435 --> 00:01:29,208 and a fellow former Microsoft employee who spent 24 00:01:29,289 --> 00:01:33,079 a number of years in Excel and on the 25 00:01:33,079 --> 00:01:36,901 team that eventually became Power BI. And it was when 26 00:01:36,997 --> 00:01:40,097 he was able to see that Power BI 27 00:01:40,691 --> 00:01:44,529 could unlock a completely different approach to business 28 00:01:44,577 --> 00:01:48,255 intelligence, which at that time was kind of dry, and a lot of folks 29 00:01:48,255 --> 00:01:51,480 would share that not to 30 00:01:51,865 --> 00:01:55,616 share that kind of that experience and that ability to 31 00:01:55,696 --> 00:01:59,095 unlock the power that resided in the data. He 32 00:01:59,384 --> 00:02:02,654 realized that. So he left Microsoft and started his own company 33 00:02:03,888 --> 00:02:07,656 that could share the ability to kind of consult and see what their data 34 00:02:07,656 --> 00:02:11,311 was trying to tell them. So we can share some really good, interesting 35 00:02:12,209 --> 00:02:15,976 AI war stories. Welcome to the show, Rob. Thank you so much. Good to be 36 00:02:15,976 --> 00:02:19,456 here. How are you doing today? I'm doing great. I'm doing great. It's 37 00:02:19,841 --> 00:02:22,969 fall here. I'm in Maryland, and I assume you're in the 38 00:02:22,969 --> 00:02:26,176 Seattle-ish area. I am back in the Seattle-ish area 39 00:02:26,176 --> 00:02:29,721 after a 15-year sojourn in the 40 00:02:29,721 --> 00:02:33,473 Midwest. Oh, interesting. The last 2 years we've been back here in 41 00:02:33,473 --> 00:02:37,018 Seattle. Very cool. So I have to tell you, 42 00:02:37,547 --> 00:02:41,220 I have been in the Microsoft data space 43 00:02:41,396 --> 00:02:44,764 in one way or the other for a while, especially if you count Excel and 44 00:02:44,764 --> 00:02:48,545 Access. Yeah, of course. Excel is 45 00:02:48,642 --> 00:02:51,927 one of those things where it really is a— it's really the 46 00:02:52,040 --> 00:02:55,569 infrastructure of modern society. My first professional 47 00:02:55,617 --> 00:02:59,358 job was, uh, I was a tech support 48 00:02:59,731 --> 00:03:02,484 at an investment banking firm on Wall Street in the '90s. 49 00:03:03,742 --> 00:03:07,227 And the amount of Excel that happened there, the 50 00:03:07,292 --> 00:03:10,599 level of complex models. Uh, one, one time I 51 00:03:10,599 --> 00:03:14,066 remember I was, you know, working the help desk and this one guy called, said 52 00:03:14,066 --> 00:03:17,768 he needed help troubleshooting his Excel. And it was 53 00:03:17,881 --> 00:03:21,472 basically, he basically showed me this 54 00:03:21,584 --> 00:03:24,933 like massive program that he had built in VBA, 55 00:03:25,062 --> 00:03:28,910 presumably. And he's like, can you help me troubleshoot this? And I'm like, 56 00:03:30,053 --> 00:03:32,564 this isn't just like, hey, my printer's not working. 57 00:03:33,434 --> 00:03:37,105 Yeah. I was like, look, I would love to help you, but this is 58 00:03:37,282 --> 00:03:41,021 way beyond what, you know, we're allowed to do on a ticketing system. 59 00:03:41,714 --> 00:03:44,727 And he basically would hit F9, I think was the key, and it would, 60 00:03:45,553 --> 00:03:49,320 This would have been a 386 or 486 era, and the thing would 61 00:03:49,320 --> 00:03:52,526 chug, chug, chug, and like you would see it. And then somewhere he had a 62 00:03:52,526 --> 00:03:56,294 circular reference that he just added. This was way before you could 63 00:03:56,294 --> 00:03:59,821 do source control and certainly way before vibe coding. So 64 00:04:00,462 --> 00:04:03,941 I don't think people appreciate just how big Excel 65 00:04:05,095 --> 00:04:08,397 is in terms of the code base that still works to this day. 66 00:04:09,119 --> 00:04:12,665 Yeah. And consistently. So what was that like? Because you're walking 67 00:04:12,713 --> 00:04:16,101 into— what, what years were you at Microsoft? In the 68 00:04:16,165 --> 00:04:19,618 2000s? Or— I'm looking at your LinkedIn profile. Oh, 69 00:04:19,634 --> 00:04:23,375 '96. Wow. Okay. 2010. Yeah. Wow. So 70 00:04:23,632 --> 00:04:27,180 our paths didn't cross at Microsoft, but, uh, you must have some interesting 71 00:04:27,485 --> 00:04:30,857 stories. Oh yeah. Um, some of them are tellable. 72 00:04:31,098 --> 00:04:34,534 Some of them are tellable. Most of them, most of the most interesting ones 73 00:04:34,566 --> 00:04:38,162 unfortunately are not. No, and we'll have to 74 00:04:38,659 --> 00:04:42,494 meet up over beers because I've heard some of these stories that are not tellable. 75 00:04:42,895 --> 00:04:46,634 But what was it like in '96? Because Excel was— when did Excel 76 00:04:46,762 --> 00:04:50,180 officially kick off? Was it— I first encountered it in the Windows 77 00:04:50,212 --> 00:04:53,823 3.0 era. But what was the origin of 78 00:04:53,823 --> 00:04:57,561 Excel and what was it like walking in in '96 when it was pretty much 79 00:04:58,219 --> 00:05:01,685 the established winner in that space? We just had a 80 00:05:01,685 --> 00:05:05,386 birthday party for Excel last year. It might have been, 81 00:05:07,287 --> 00:05:10,584 it might have been the 40th anniversary party. Wow, 82 00:05:11,118 --> 00:05:14,095 okay. That we went to. I mean, so it definitely preceded my time at the 83 00:05:14,095 --> 00:05:17,395 company. I mean, it was in the '80s that Excel first 84 00:05:18,301 --> 00:05:21,652 came to be. So I was very much joining, when I joined the Excel team, 85 00:05:21,941 --> 00:05:25,540 it had a long and storied history. I mean, it wasn't 86 00:05:25,941 --> 00:05:29,331 like I had anything to do with the creation of Excel, you know? Right, right, 87 00:05:29,380 --> 00:05:33,010 right, right. But that probably made your 88 00:05:33,010 --> 00:05:36,679 time on Power BI, or what became Power BI, even sweeter because you do have 89 00:05:36,711 --> 00:05:40,525 that origin story. But let's not get ahead of ourselves. In fact, that 90 00:05:40,541 --> 00:05:43,746 was the reason why they recruited me to work, be one of the first people 91 00:05:43,746 --> 00:05:47,271 working on Power BI, was because of my experience with the Excel crew. 92 00:05:47,928 --> 00:05:50,781 Oh, interesting. Yeah, I mean, Excel is 93 00:05:51,774 --> 00:05:55,620 a much deeper product than, let's say, something like Word 94 00:05:55,620 --> 00:05:59,161 or Outlook. Everyone looks at the Office suite 95 00:05:59,930 --> 00:06:02,637 and, you know, sort of just sees a row of icons 96 00:06:03,775 --> 00:06:07,492 as if they're all kind of the same animal. And they all kind of look 97 00:06:07,540 --> 00:06:11,145 like the same animal. They've all got the same sort of user interface ribbons across 98 00:06:11,161 --> 00:06:14,942 the top, and they all produce documents, and they have the same file 99 00:06:14,942 --> 00:06:17,986 open and file save experience and all that kind of stuff. But 100 00:06:19,412 --> 00:06:23,257 Excel is the world's— Not even like VBA, 101 00:06:23,433 --> 00:06:27,219 not even macros or the JavaScript API. I mean, Excel 102 00:06:27,332 --> 00:06:31,182 formulas are by far the overwhelming most 103 00:06:32,165 --> 00:06:34,920 widely used programming language in the world. 104 00:06:37,169 --> 00:06:40,953 100%. And Excel formulas pass every test of what 105 00:06:40,969 --> 00:06:44,077 constitutes a programming language. We just don't really think of it as such. 106 00:06:45,266 --> 00:06:48,618 And so you found that Excel 107 00:06:48,666 --> 00:06:52,324 required a, like a core 108 00:06:52,518 --> 00:06:56,158 council of elders that needed, that stayed with the 109 00:06:56,158 --> 00:06:59,770 product and needed to stay with the product. I'm talking about the engineers, the 110 00:06:59,770 --> 00:07:03,285 software developers. They needed to, like, it 111 00:07:03,285 --> 00:07:06,046 required that kind of core stewardship. 112 00:07:07,394 --> 00:07:11,086 Whereas the other applications, you know, you could kind of like, you 113 00:07:11,086 --> 00:07:14,794 could shift the leadership around, you could move people around, you know, like, I'm tired 114 00:07:14,794 --> 00:07:18,454 of working on Word, I'm gonna go work on Outlook for a little while. And 115 00:07:18,470 --> 00:07:22,291 my job was a program manager, a product manager, like, you know, like You know, 116 00:07:22,323 --> 00:07:25,343 very technical, but at the same time designing what this 117 00:07:25,777 --> 00:07:29,264 product should do. What should the new functionality be and how should, how does, how 118 00:07:29,312 --> 00:07:31,995 should it meet the world? Like what are the customer needs and things like that? 119 00:07:32,605 --> 00:07:35,465 And working very closely with the development team to make that a reality. 120 00:07:36,606 --> 00:07:40,237 And, um, the product managers, at least in my era, did 121 00:07:40,237 --> 00:07:43,129 have a lot of turnover. Like there was, we were, there was a lot of 122 00:07:43,129 --> 00:07:46,825 people. I think it was, I think it was kind of a frustrating place for 123 00:07:46,825 --> 00:07:49,652 people to work as a product manager. Because they, 124 00:07:51,002 --> 00:07:54,697 because the depth of the product was so great and the history of the product 125 00:07:54,697 --> 00:07:57,749 was so great that like you really didn't feel like you could really make much 126 00:07:57,749 --> 00:08:00,529 of a mark on it as a newbie. 127 00:08:01,509 --> 00:08:05,268 And plus it was just hard. It was a lot more fun in 128 00:08:05,365 --> 00:08:09,188 some ways to go create brand new experiences like in Word or Outlook or 129 00:08:09,622 --> 00:08:13,189 PowerPoint or whatever. Yeah, sort of like the elder 130 00:08:13,237 --> 00:08:17,026 developers, right? They had this almost like the secondary 131 00:08:17,026 --> 00:08:20,669 job of like making sure to teach all of us program 132 00:08:20,669 --> 00:08:23,766 managers that were new to the product every time. Now, I think, I think the 133 00:08:23,991 --> 00:08:27,794 turnover in the product management team on Excel has slowed down quite 134 00:08:27,794 --> 00:08:31,485 a bit. I think there's, that's changed since my era, but there was this very 135 00:08:31,485 --> 00:08:35,160 much like ramping up process. Like I, I, 136 00:08:35,384 --> 00:08:38,241 um, I remember that it was like at least a year, 137 00:08:39,396 --> 00:08:42,269 at least a year before I stopped coming up with 138 00:08:43,064 --> 00:08:46,856 ideas that, you know, like, oh, Excel should be able to do this, only to 139 00:08:46,856 --> 00:08:50,696 be told, yeah, Rob, Excel already can do that. We've already got that. Is 140 00:08:50,729 --> 00:08:53,653 this that deep of a product? And I found it difficult to recruit 141 00:08:54,348 --> 00:08:57,179 other product managers to come work for our team 142 00:08:58,958 --> 00:09:02,760 from within the Office organization because again, people could sense how deep the 143 00:09:02,760 --> 00:09:05,606 product was and how little they knew it. 144 00:09:06,577 --> 00:09:10,345 You were— if you worked on Word, you were an expert user on 145 00:09:10,458 --> 00:09:14,213 Word. within the first month. If you worked 146 00:09:14,213 --> 00:09:18,061 on Excel, you were almost never an expert. It took 147 00:09:18,061 --> 00:09:21,862 you— the community experts like the Excel MVPs were 148 00:09:22,712 --> 00:09:25,726 far better drivers of that race car than the people who 149 00:09:26,657 --> 00:09:29,383 designed and built the race car, you know. And so that was— it was always— 150 00:09:29,687 --> 00:09:33,392 there's an intimidation and a difficulty associated with working on 151 00:09:33,392 --> 00:09:37,080 Excel that I don't think really existed on most of the Office 152 00:09:37,080 --> 00:09:40,452 products. Oh, I would say 100%. I mean, 153 00:09:41,449 --> 00:09:45,277 I've used Excel in one form or the other. So apparently it started in 1985. 154 00:09:45,454 --> 00:09:48,367 So— Yeah, it was the 40th. It was the 40th birthday. 155 00:09:48,916 --> 00:09:52,595 And I just 156 00:09:52,644 --> 00:09:56,469 feel like I could never— it would take an eternity to learn 157 00:09:56,694 --> 00:10:00,273 Excel to like everything. And I think there's probably maybe a dozen people worldwide 158 00:10:00,773 --> 00:10:04,254 that have that, that can legitimately say it and actually mean it. And they're probably 159 00:10:04,254 --> 00:10:07,154 not the type of people you think, right? You wouldn't— it's probably the accountants, the 160 00:10:07,154 --> 00:10:10,708 financial analysts and things like that. I mean, I've seen financial 161 00:10:10,757 --> 00:10:14,425 analysts do things in Excel that I— and this is even in the '90s, 162 00:10:14,636 --> 00:10:18,361 right? It was like, you could do that in Excel? And no, 163 00:10:18,377 --> 00:10:21,920 you're right. And the whole council of elders, because the thing that I've always 164 00:10:22,017 --> 00:10:25,406 admired about Excel, and I know 165 00:10:25,907 --> 00:10:29,609 it's sad to say that I admire Excel, but, you know, true 166 00:10:29,609 --> 00:10:32,955 data head, I guess, would, right? Is how consistent it's been. 167 00:10:33,068 --> 00:10:36,520 And that would explain, I guess, the council of elders, for lack of a better 168 00:10:36,520 --> 00:10:39,772 term, right? You're right. Like, if a word 169 00:10:39,805 --> 00:10:43,512 processor changes, it'll annoy you, but you can 170 00:10:43,560 --> 00:10:47,305 kind of like get on with it, right? But like, the numbers really matter here. 171 00:10:47,983 --> 00:10:51,259 And it just fascinates me that an Excel 172 00:10:51,324 --> 00:10:54,815 spreadsheet that could be open today and it's the same, you know, 173 00:10:54,992 --> 00:10:58,288 there'll be some conversion, right? But for the most part, it'll be 174 00:10:58,369 --> 00:11:02,047 readable and usable today. Yeah. Yeah. If it was 175 00:11:02,096 --> 00:11:05,474 written, I can confirm at least back to the '90s, 176 00:11:05,958 --> 00:11:09,664 If it goes back further, I wouldn't be surprised, but I can't, I can't say 177 00:11:09,952 --> 00:11:13,273 I have firsthand experience with that. Yeah. I mean, an Excel 178 00:11:13,562 --> 00:11:16,739 document is itself an application. Yes. 179 00:11:17,332 --> 00:11:21,038 Whereas a Word document is a document, you know, and, you know, 180 00:11:21,166 --> 00:11:24,632 you can always add some form of code to it and start to 181 00:11:24,728 --> 00:11:28,209 slowly turn a Word doc into something more like it. But like basically any 182 00:11:28,562 --> 00:11:31,979 Excel document begins as an application and it 183 00:11:32,188 --> 00:11:35,867 It has logic in it, it has flow of control, it has— and so 184 00:11:36,253 --> 00:11:39,386 yeah, it's kind of a miracle. 185 00:11:40,270 --> 00:11:43,797 If there was a Software Hall of Fame, Excel would 186 00:11:44,056 --> 00:11:47,590 very much be in it. Absolutely. Absolutely. 187 00:11:48,171 --> 00:11:51,479 So let's get to Power BI because I remember when I first saw Power BI, 188 00:11:51,495 --> 00:11:54,428 and keep in mind, I didn't see Power BI. How did that come about? 189 00:11:54,541 --> 00:11:58,360 Because when I first moved to Richmond, 190 00:11:58,409 --> 00:12:02,145 Virginia about wow, 20 years ago, or a little more than 20 191 00:12:02,145 --> 00:12:05,726 years ago now, I worked at a small company called Ironworks, and 192 00:12:06,240 --> 00:12:09,548 they were a small Microsoft partner. And I remember that 193 00:12:09,789 --> 00:12:13,531 Microsoft was really pushing their BI story, and this is before BI was 194 00:12:13,868 --> 00:12:17,691 a household word. Yeah. And funny enough is that 195 00:12:17,739 --> 00:12:21,384 when I rejoined Microsoft in 2017, 2018, 196 00:12:22,059 --> 00:12:25,576 it was that same guy who headed up our BI practice, Kevin Veyers. 197 00:12:25,704 --> 00:12:28,661 Shout out, Kevin. if you're still listening. He 198 00:12:30,494 --> 00:12:33,404 was an early believer in the BI platform, and I remember seeing it and I 199 00:12:33,404 --> 00:12:36,915 was like, there's something here. But it was very 200 00:12:38,063 --> 00:12:41,878 data intensive, right? It was very— you had to be a 201 00:12:41,878 --> 00:12:45,628 data engineer to use it. What blew my mind about Power BI, 202 00:12:45,757 --> 00:12:48,984 and I didn't see Power BI, and you'll laugh, until I was working inside the 203 00:12:48,984 --> 00:12:52,081 legal department. Long, sordid story how I got there. 204 00:12:52,692 --> 00:12:56,458 But I remember seeing Power BI and I was like, 205 00:12:56,538 --> 00:13:00,001 this is like PowerPoint but for Excel. So 206 00:13:00,930 --> 00:13:04,377 how did you— how did that come about? Yeah, 207 00:13:04,857 --> 00:13:08,400 so I mean, we really should— I think you're touching on something really important that 208 00:13:08,400 --> 00:13:11,798 there were 2 very distinct eras in 209 00:13:11,846 --> 00:13:15,661 Microsoft BI and in BI in general. So when I 210 00:13:15,677 --> 00:13:19,155 worked on the Excel team on the 2007 211 00:13:19,219 --> 00:13:22,921 release of Excel, So around, you know, you're talking about 2005, you know, 212 00:13:22,985 --> 00:13:26,589 talking about BI, like Excel made a huge 213 00:13:26,589 --> 00:13:30,386 investment in BI in the 2007 release. And I 214 00:13:30,418 --> 00:13:34,199 was in charge of that functionality, you know, the 215 00:13:34,199 --> 00:13:36,891 majority of the BI investments that Excel was making. 216 00:13:38,268 --> 00:13:41,457 And, you know, and I had to get a crash course on what BI was, 217 00:13:41,793 --> 00:13:45,510 you know, like. Right. I was kind of like tapped to do this 218 00:13:45,878 --> 00:13:48,666 and then had to go study, okay, what is this BI world? You know, I 219 00:13:48,666 --> 00:13:50,973 had to go to conferences, had to go take some classes, had to, 220 00:13:52,291 --> 00:13:56,140 study with some of the BI gurus at Microsoft, like the Yoda 221 00:13:56,140 --> 00:13:59,927 figures. And, but it 222 00:13:59,927 --> 00:14:03,647 was very much this traditional, what I describe as traditional BI. 223 00:14:05,264 --> 00:14:08,887 You're talking OLAP cubes and stuff like that. That was though, 224 00:14:09,130 --> 00:14:12,622 I remember hearing about this in the '90s. I think it was SAP. 225 00:14:12,864 --> 00:14:15,726 SAP had something like this and it just seemed so 226 00:14:16,178 --> 00:14:20,024 esoteric and so difficult to learn. It was, it was both of 227 00:14:20,024 --> 00:14:23,612 those things. Absolutely. You know, Microsoft had 2 big products at the time 228 00:14:24,787 --> 00:14:28,220 in the BI space, Reporting Services and Analysis Services. 229 00:14:29,141 --> 00:14:32,084 And Reporting Services is the thing that everyone 230 00:14:33,248 --> 00:14:37,031 understands at a fundamental level. You've got data sitting in SQL or some other storage 231 00:14:37,063 --> 00:14:40,892 system lately, and you need to turn a query 232 00:14:41,568 --> 00:14:44,911 into some sort of formatted report, right? query, 233 00:14:45,728 --> 00:14:49,479 format query as pixels. That's what 234 00:14:49,511 --> 00:14:53,358 Reporting Services was. It was the most widely adopted, most 235 00:14:53,358 --> 00:14:56,596 successful product at Microsoft in terms of BI, but it wasn't— 236 00:14:57,205 --> 00:15:00,860 it was just formatting queries as 237 00:15:00,876 --> 00:15:04,626 pixels. There's not a lot of intelligence going on 238 00:15:04,883 --> 00:15:08,569 in the BI part there. Analysis Services, by 239 00:15:08,569 --> 00:15:12,353 contrast, was this incredibly— you mentioned OLAP cubes. Yeah. It 240 00:15:12,353 --> 00:15:16,142 was this incredibly intelligent product that allowed you 241 00:15:16,287 --> 00:15:19,707 to blend and mesh data from multiple different 242 00:15:19,884 --> 00:15:23,480 workflows, multiple different silos, different phases of your business, all in one 243 00:15:23,593 --> 00:15:26,579 place, and express business logic. 244 00:15:27,446 --> 00:15:30,690 What is gross profit? Very precisely. And then 245 00:15:31,605 --> 00:15:35,009 ask any sort of like, ask any question you want 246 00:15:35,860 --> 00:15:39,321 of your data model. without having to go 247 00:15:39,565 --> 00:15:43,415 rewrite a whole bunch of SQL each time you wanted to ask 248 00:15:43,415 --> 00:15:47,091 a new question. And very intelligent 249 00:15:47,091 --> 00:15:50,770 product, but as you said, very esoteric. Multiple 250 00:15:50,786 --> 00:15:54,530 times I sat down and said, okay, I'm ready, teach me the 251 00:15:54,546 --> 00:15:58,317 MDX formula language that is used by this product, this 252 00:15:58,494 --> 00:15:59,252 SSAS 253 00:16:00,122 --> 00:16:02,845 multidimensional product. 254 00:16:04,128 --> 00:16:07,720 And each time I would go, oh, right, I forgot. This is— 255 00:16:08,377 --> 00:16:11,616 no, I'm never going to— like, we'd be 15 minutes into explaining how to do 256 00:16:11,616 --> 00:16:15,448 an if, just a simple if. Wow. And we'd be 257 00:16:15,448 --> 00:16:19,233 going through all of this like, oh, yeah, but you got to understand all 258 00:16:19,233 --> 00:16:23,081 these hierarchies first and the addressing space of the language. I'm 259 00:16:23,081 --> 00:16:26,641 like, oh, right, I totally forgot. We did this 6 months ago and I said 260 00:16:26,657 --> 00:16:30,393 no. So I'm going to say no again. 261 00:16:31,082 --> 00:16:34,721 And you talk about a very rarefied audience. There were 262 00:16:34,929 --> 00:16:38,135 a few thousand people in the world who claimed to be good at this, 263 00:16:38,760 --> 00:16:42,399 building these sorts of— building an SSAS 264 00:16:42,623 --> 00:16:46,438 database, building an SSAS database OLAP cube. But the real 265 00:16:46,631 --> 00:16:50,366 number of people who were actually good at it was much smaller than that. And 266 00:16:51,504 --> 00:16:54,598 so, Microsoft didn't have a frontend. Believe it or not, 267 00:16:54,726 --> 00:16:58,220 SSAS was just almost like an API. So, We were turning 268 00:16:58,252 --> 00:17:01,184 Excel into a premier front end 269 00:17:01,841 --> 00:17:05,639 for interacting with these OLAP cubes. So 270 00:17:05,783 --> 00:17:09,372 pivot tables and these things called cube formulas and pivot charts, all these sorts of 271 00:17:09,372 --> 00:17:13,122 things. But the problem, you know, and by the 272 00:17:13,154 --> 00:17:16,310 way, SSAS was the leader in its market segment. More people 273 00:17:17,464 --> 00:17:20,925 used SSAS than any of the competitive technologies from other 274 00:17:20,925 --> 00:17:24,691 companies. Oh yeah. I mean, I remember it kind of came from 275 00:17:24,691 --> 00:17:28,204 zero. I remember it was described, and then it kind of like disrupted the whole 276 00:17:28,204 --> 00:17:31,880 industry. And I remember Kevin showing me it, 277 00:17:31,992 --> 00:17:35,571 and I was just like looking at it like, oh my God, this is not 278 00:17:35,587 --> 00:17:39,150 for the timid. No, not at all. Not at all. And you really had to 279 00:17:39,776 --> 00:17:42,440 have spent your life building up to that moment to— 280 00:17:43,788 --> 00:17:47,030 down that chain. And then even then, like, making very, very 281 00:17:47,078 --> 00:17:50,529 specific life decisions, like the left turn at 282 00:17:50,529 --> 00:17:54,270 Albuquerque. that would lead you in 283 00:17:54,302 --> 00:17:57,565 that direction. And you had to be wired in a very, very, I think, 284 00:17:58,501 --> 00:18:02,249 wired for a very academic way of thinking. Amir Netz, 285 00:18:02,798 --> 00:18:06,385 the architect of all of this, he was the architect of the original 286 00:18:06,433 --> 00:18:08,904 Analysis Services. He understood 287 00:18:10,209 --> 00:18:14,046 that this was a really important technology, the ability to 288 00:18:15,222 --> 00:18:18,452 build these kinds of models. But that this 289 00:18:18,630 --> 00:18:20,953 bottleneck that it was so hard to build them. 290 00:18:22,918 --> 00:18:26,237 Like, you know, like Microsoft doesn't charge for consulting, right? 291 00:18:26,302 --> 00:18:30,009 Microsoft charges for their software being deployed and run. And there's 292 00:18:30,009 --> 00:18:33,640 this huge bottleneck standing between Microsoft 293 00:18:33,720 --> 00:18:36,409 and licensing revenue. Like, if you're going to— 294 00:18:37,520 --> 00:18:41,053 fine, you can buy analysis services, but unless you run it and adopt it, 295 00:18:41,722 --> 00:18:45,441 you're not going to keep paying Microsoft. And so he 296 00:18:45,441 --> 00:18:49,036 knew that they needed a do-over on that 297 00:18:49,085 --> 00:18:52,662 technology. And it's the rare case 298 00:18:52,662 --> 00:18:56,173 where— and I, I've recently written a book on 299 00:18:56,350 --> 00:18:59,328 AI that you see behind me, Fair Game. Mm-hmm. And in the book I talk 300 00:18:59,328 --> 00:19:02,389 about exactly this, that it's a rare, it's a very rare example 301 00:19:03,964 --> 00:19:07,781 where someone is given an opportunity at a large software company to kind 302 00:19:07,781 --> 00:19:11,415 of like reboot something that they've done version 1 of, 303 00:19:11,912 --> 00:19:14,782 and that it's a success, that it gets all the things that it needs. It 304 00:19:14,799 --> 00:19:18,647 gets the support, it gets the buy-in, and then it also is executed well. 305 00:19:19,770 --> 00:19:23,554 And I, when I was on the— so they recruited me to join the 306 00:19:23,554 --> 00:19:26,986 Power BI team. It was called the Power Pivot team. It was actually called Project 307 00:19:26,986 --> 00:19:30,803 Gemini originally. Really? Oh, Power Pivot. Now 308 00:19:30,851 --> 00:19:34,555 I— okay. Yeah. That brings back some memories. Interesting. Yeah. Sorry I cut you off. 309 00:19:34,603 --> 00:19:37,987 That's okay. No, no. So that's, that's the lineage here, right? And so they 310 00:19:38,211 --> 00:19:41,838 recruited me, Amir recruited me. To come be one of the first few people 311 00:19:41,838 --> 00:19:45,468 working on that because he knew that I could represent 312 00:19:46,399 --> 00:19:49,981 the target audience, the Excel target audience, and not 313 00:19:50,334 --> 00:19:53,948 everyone who uses Excel, like this kind of like the pivot table 314 00:19:53,964 --> 00:19:57,610 creating fraction of Excel users who, 315 00:19:58,140 --> 00:20:01,947 by the way, were very important to BI and are 316 00:20:01,995 --> 00:20:05,753 now going to be very important to AI as well. It's another theme that 317 00:20:05,753 --> 00:20:08,789 I cover a lot in my book. So I was there to represent those people. 318 00:20:09,303 --> 00:20:12,691 Because that's who he wanted to target. He said, look, the people who 319 00:20:13,461 --> 00:20:16,769 are really good at Excel in a data analysis sense, 320 00:20:17,636 --> 00:20:21,313 like the, the Wall Street types are building like financial models that are 321 00:20:21,313 --> 00:20:24,669 more like simulations. Yeah. You know, there's a different 322 00:20:24,878 --> 00:20:28,523 breed as well, and there's some overlap between the two that are 323 00:20:29,197 --> 00:20:32,826 for many, many years were essentially doing the BI mission for the 324 00:20:32,890 --> 00:20:36,534 business in Excel. and this was the crowd we were targeting with 325 00:20:36,534 --> 00:20:39,697 Power BI and giving them the ability to build 326 00:20:40,115 --> 00:20:43,903 these data models in a way 327 00:20:43,903 --> 00:20:47,741 that was approachable to them. So in other words, I'm not 328 00:20:47,741 --> 00:20:51,240 15 minutes into having an IF function 329 00:20:51,289 --> 00:20:55,045 explained to me, right? IF works like IF. That was one of the, 330 00:20:55,061 --> 00:20:58,513 kind of like one of the core tenets of Power BI. 331 00:20:59,219 --> 00:21:02,382 And so, you know, you can look at that project from a few different lenses. 332 00:21:02,511 --> 00:21:06,184 One of them was the, making it accessible to that kind of 333 00:21:06,457 --> 00:21:09,980 audience, which meant undoing and 334 00:21:10,012 --> 00:21:13,679 redoing some of the architectural assumptions that they'd made in their first. It was 335 00:21:13,760 --> 00:21:17,492 really interesting and fascinating to watch them kind of retrace their steps and say, okay, 336 00:21:18,055 --> 00:21:21,386 here's where it went wrong in the original. 337 00:21:22,306 --> 00:21:26,065 And that's quite a thing to say to something 338 00:21:26,113 --> 00:21:29,744 that at this point would have been 30 years old, maybe 339 00:21:29,889 --> 00:21:33,460 20 years old technology with the Council of Elders telling 340 00:21:33,492 --> 00:21:37,297 councils of— a council of elders that you did something wrong or 341 00:21:37,651 --> 00:21:40,396 your assumption is no longer accurate. Must have been an experience. 342 00:21:41,118 --> 00:21:44,843 Politically speaking, let's make a distinction. So when Amir 343 00:21:45,116 --> 00:21:48,873 recruited me to work on the Power BI product, that was happening in 344 00:21:48,905 --> 00:21:52,710 the SQL org, completely separate from the Excel org. Now 345 00:21:52,758 --> 00:21:56,450 we did build this Power Pivot thing. The first version of 346 00:21:56,450 --> 00:22:00,046 Power BI was built as an add-on into Excel. because that's 347 00:22:00,062 --> 00:22:03,268 where the target audience lived. But it was sort of like, 348 00:22:04,214 --> 00:22:08,061 we didn't really like ask the Excel team's permission to do this. I mean, they 349 00:22:08,061 --> 00:22:11,572 were in favor of it. They weren't like going to Bill Gates and saying, 350 00:22:12,325 --> 00:22:15,595 we should stop this. Right, right, right, right. And they did help us with some 351 00:22:15,611 --> 00:22:19,154 things, and we helped them with some things. So there was definitely a collaborative 352 00:22:19,154 --> 00:22:22,969 relationship, but it didn't really threaten the 353 00:22:22,969 --> 00:22:26,768 core of Excel. It was an add-on to Excel. 354 00:22:26,833 --> 00:22:29,990 You know, like basically it was meant to show up through Excel features like pivot 355 00:22:30,023 --> 00:22:33,373 tables and cube formulas things like that. So it was, 356 00:22:34,384 --> 00:22:37,911 it was an expansion to Excel's capabilities and it was sort of a welcome 357 00:22:37,975 --> 00:22:41,743 expansion. But anyway, so we didn't really have that same 358 00:22:41,743 --> 00:22:45,319 problem. It was more like the more interesting thing was watching 359 00:22:46,073 --> 00:22:49,681 the Analysis Services team retrace their steps and rethink their 360 00:22:49,681 --> 00:22:53,385 approach to things. So, and fast forwarding a little bit, like 361 00:22:53,770 --> 00:22:56,463 it was just a shocking, shockingly, shockingly 362 00:22:57,137 --> 00:23:00,876 capable product that we came up with. I'd been part of a lot of 363 00:23:00,972 --> 00:23:04,762 like, like version 1 efforts and sort of 364 00:23:04,762 --> 00:23:08,135 like nascent startup efforts and also like ambitious 365 00:23:08,135 --> 00:23:11,910 projects that, that we took on in Excel that maybe never 366 00:23:11,910 --> 00:23:15,636 ever like even like finished, you know, like, so I was, I was pretty cynical 367 00:23:16,182 --> 00:23:19,571 about version 1 software. I, 368 00:23:19,940 --> 00:23:23,763 even though I worked on this thing, I didn't expect it to be very good. 369 00:23:24,357 --> 00:23:27,505 I expected it to be the usual Microsoft. It's going to take 3 versions to 370 00:23:27,505 --> 00:23:30,571 get it right. Yeah. But when I started using it, 371 00:23:31,705 --> 00:23:34,637 I saw that it was exactly— that it was— it actually exceeded, 372 00:23:35,500 --> 00:23:38,924 greatly exceeded, I think, any of our 373 00:23:39,053 --> 00:23:42,815 expectations of just how capable it was going to be. Like, I was hoping 374 00:23:42,847 --> 00:23:45,577 that it was going to be like the, 375 00:23:47,218 --> 00:23:51,056 you know, sort of these Excel pros that I've called the data gene crowd for 376 00:23:51,056 --> 00:23:54,443 many years, and I've now started calling them the crafters instead. 377 00:23:55,443 --> 00:23:59,196 Because I think we crafters have a role 378 00:23:59,372 --> 00:24:02,948 to play in AI now that sort of like we're kind of outgrowing 379 00:24:02,980 --> 00:24:06,092 the just the pure data label. So, 380 00:24:06,894 --> 00:24:09,075 so let's just— I'm just going to use that word crafter because I use it, 381 00:24:09,219 --> 00:24:12,795 I use it throughout the book. The hope was these crafters could build a 382 00:24:12,859 --> 00:24:16,500 solution, a BI solution that was maybe 80% 383 00:24:16,628 --> 00:24:20,381 or 60% as good as what the true experts could 384 00:24:20,381 --> 00:24:23,906 do. using the old technology. What I 385 00:24:23,906 --> 00:24:26,550 found though was that we could build things that were much better. 386 00:24:27,832 --> 00:24:31,646 Interesting. It was a far better result that 387 00:24:31,678 --> 00:24:34,851 we were building, not just— and it was happening so much 388 00:24:35,124 --> 00:24:38,617 faster. I remember writing one 389 00:24:38,713 --> 00:24:42,335 formula for myself that I had paid a consulting 390 00:24:42,415 --> 00:24:46,261 firm, a traditional BI consulting firm to help me with, you know, 391 00:24:46,550 --> 00:24:50,025 couple years earlier. And I remember writing a formula in 392 00:24:50,300 --> 00:24:54,148 less than 30 minutes that in real life had taken us a couple 393 00:24:54,180 --> 00:24:56,230 of weeks. Wow. 394 00:24:57,973 --> 00:25:01,653 And, and then having this realization, oh, it's because I have 395 00:25:02,493 --> 00:25:05,850 the business knowledge in my head in the same brain 396 00:25:06,351 --> 00:25:10,177 as the capability to build it. And it was 397 00:25:10,306 --> 00:25:13,703 all of the communication cost and miscommunication and 398 00:25:13,801 --> 00:25:17,462 delay and, and asynchronous waiting on like, okay, like, 399 00:25:18,151 --> 00:25:21,886 like I, like peeling the onion. Like I, I tell them 400 00:25:21,919 --> 00:25:24,868 what I, what I thought I needed. They'd go build what they thought they heard 401 00:25:24,980 --> 00:25:27,465 and then they'd show me it and I go, no, that's not it. But then 402 00:25:27,529 --> 00:25:30,495 I'd have to explain what's, why it was wrong. I have to go do a 403 00:25:30,495 --> 00:25:33,156 bunch of research to explain why it's wrong and give them a 404 00:25:34,022 --> 00:25:37,485 slide deck. It explained why it was wrong and everything. But like, but all of 405 00:25:37,485 --> 00:25:40,787 that, that whole iterative multi-week process just compressed. 406 00:25:41,881 --> 00:25:44,970 in my head into 30 minutes of just 407 00:25:45,601 --> 00:25:49,338 effortlessly going, just looking at the data. I'm going, yeah, that 408 00:25:49,338 --> 00:25:52,804 row shouldn't count and this row should and all that kind of— it 409 00:25:52,820 --> 00:25:56,312 kind of blew me away. And that's when 410 00:25:56,827 --> 00:25:59,884 I realized that the traditional consulting industry 411 00:26:01,194 --> 00:26:04,895 wasn't going to be remotely prepared 412 00:26:05,493 --> 00:26:09,296 to take advantage of this opportunity and to bring this to their customers, to 413 00:26:09,296 --> 00:26:12,858 their clients, or to a brand new audience of 414 00:26:12,906 --> 00:26:16,645 clients that had previously been priced out. They just weren't gonna be built 415 00:26:16,645 --> 00:26:20,041 for this. And so that's what led me to start P3 Adaptive. 416 00:26:20,993 --> 00:26:24,769 Yeah. Like, let's start from scratch and build a company 417 00:26:24,963 --> 00:26:28,657 that can take full advantage of delivering this 418 00:26:28,673 --> 00:26:32,417 gift to the world. And that's been a very, very, 419 00:26:32,449 --> 00:26:36,290 very satisfying, very satisfying project for the 420 00:26:36,290 --> 00:26:40,114 last, you know, now like, 13, 14 years, and 421 00:26:40,227 --> 00:26:43,103 we've proven that it works. I mean, I like to say at P3 that 422 00:26:43,135 --> 00:26:45,639 we helped reinvent an industry. 423 00:26:47,071 --> 00:26:50,714 You know, like a lot of the traditional— most of the traditional consulting firms have 424 00:26:50,714 --> 00:26:54,203 stuck with their old methodology because it's so profitable, but 425 00:26:54,348 --> 00:26:58,061 there's plenty of new outfits that have sprung up that look a 426 00:26:58,061 --> 00:27:01,838 lot like us, and the world has gotten a lot 427 00:27:01,870 --> 00:27:05,581 more access to working BI. than 428 00:27:05,677 --> 00:27:08,899 it ever did before. And so it was— I kind of 429 00:27:09,460 --> 00:27:13,019 thought that was gonna be the only time in my career that, that I 430 00:27:13,291 --> 00:27:17,090 was sort of like had a ringside seat for like a big change like 431 00:27:17,090 --> 00:27:20,761 this. And of course, uh, AI's come along and said, no, no, actually you're 432 00:27:20,761 --> 00:27:23,294 gonna— there's a second act, second act to this. 433 00:27:24,592 --> 00:27:28,311 It's, uh, every day is a new adventure. Every week there's some new 434 00:27:28,616 --> 00:27:31,694 radical drop. But I think one of the things that 435 00:27:33,252 --> 00:27:36,656 You kind of hinted at is that the fundamentals of AI, 436 00:27:37,763 --> 00:27:41,296 obviously models, transformers, you know, whatever the 437 00:27:41,296 --> 00:27:45,133 frontier model people are doing this week, it all comes down to data though, 438 00:27:45,214 --> 00:27:48,200 right? At the end of the day, data is important. And there's a lot of 439 00:27:48,200 --> 00:27:50,898 memes. If you're on LinkedIn, you've seen a lot of the memes where it shows 440 00:27:51,010 --> 00:27:54,848 like, you know, this mansion that's crumbling and it shows like, you know, when 441 00:27:54,848 --> 00:27:56,758 you put AI first and then they show like this 442 00:27:57,497 --> 00:28:01,288 fortress/castle where it says, if you put the data first, I 443 00:28:01,288 --> 00:28:04,887 mean, there's a lot of truth to those memes. That's kind of what makes them 444 00:28:04,903 --> 00:28:08,613 funny. Agreed 100%. You know, so yeah, 445 00:28:08,742 --> 00:28:12,453 I think that, you know, like what you see. So certainly there's sort of like 446 00:28:12,453 --> 00:28:16,051 2 sides to AI. There's the model research itself, the LLM 447 00:28:16,051 --> 00:28:19,553 researchers who are coming up with each new 448 00:28:19,906 --> 00:28:23,697 successive generation of LLM. And it's when 449 00:28:23,697 --> 00:28:27,182 those drop, It's sometimes a huge surprise at how 450 00:28:27,311 --> 00:28:30,570 capable they are in terms of what they can do. And then other times when 451 00:28:30,602 --> 00:28:34,022 they drop, it's kind of like incremental, 452 00:28:34,777 --> 00:28:38,550 but you never really know. You kind of hold your breath each time. Is this 453 00:28:38,550 --> 00:28:41,890 going to be a big leap forward or is it going to be, yeah, again, 454 00:28:41,922 --> 00:28:45,438 an incremental improvement? But none of that 455 00:28:46,417 --> 00:28:50,062 really changes the way that you need to approach it in business. 456 00:28:51,739 --> 00:28:54,793 Most of the, you know, the 457 00:28:54,986 --> 00:28:58,699 success in business with AI is 458 00:28:59,310 --> 00:29:02,868 much, much more about regular software 459 00:29:03,239 --> 00:29:06,041 and regular data and regular information 460 00:29:07,668 --> 00:29:11,214 and making it accessible and available to the 461 00:29:11,311 --> 00:29:15,094 LLM at the right moment. You know, one of the 462 00:29:15,548 --> 00:29:18,403 analogies I use in the book is 463 00:29:19,864 --> 00:29:23,670 the LLM, when it shows up every day, no matter what 464 00:29:23,670 --> 00:29:26,833 it is, no matter what LLM it is, you can think of it as having 465 00:29:26,929 --> 00:29:30,558 a PhD in everything and every human topic that's ever had a 466 00:29:30,558 --> 00:29:34,058 PhD taught. Like, it's incredibly knowledgeable. Even 467 00:29:34,090 --> 00:29:37,543 before it searches the web, it knows so much, 468 00:29:38,024 --> 00:29:41,525 but it knows nothing about your business. Right. It's like an— it's a new 469 00:29:41,637 --> 00:29:44,808 hire. with respect to your business. 470 00:29:45,708 --> 00:29:49,343 Um, like, uh, where's the bathroom level new hire? 471 00:29:49,649 --> 00:29:53,235 And, and 30 minutes later it's a new hire 472 00:29:53,348 --> 00:29:57,126 again. Yeah, when the context runs out, 473 00:29:57,158 --> 00:30:00,727 it loses a lot of that. Um, yeah, and I know that that's— I know 474 00:30:00,743 --> 00:30:04,256 they're trying to work on, um, if you've heard of OpenClaw or 475 00:30:04,304 --> 00:30:08,108 Hermes, you know, they have the soul.md and memories.md. 476 00:30:08,350 --> 00:30:12,187 Like, there's this real push to kind of solve that 477 00:30:13,086 --> 00:30:16,039 while not stuffing the context window, because context 478 00:30:16,632 --> 00:30:20,388 and attention are still resource constrained, I think 479 00:30:20,388 --> 00:30:23,019 would be a good way to put that. Oh, and given the way that these 480 00:30:23,051 --> 00:30:25,828 things are currently designed, the LLMs are currently designed, 481 00:30:26,566 --> 00:30:29,759 that is, you know, the limited size of 482 00:30:29,936 --> 00:30:33,129 context that the LLM can absorb before it 483 00:30:33,499 --> 00:30:37,280 starts to become dilute in its effectiveness. That's pretty much here 484 00:30:37,280 --> 00:30:40,660 to stay until they come up with a completely 485 00:30:40,660 --> 00:30:44,420 different architecture than what, what they've, what all these LLMs are 486 00:30:44,420 --> 00:30:47,700 working on. I'm glad you pointed that out, 'cause I had this debate with somebody 487 00:30:47,700 --> 00:30:51,320 who was like, well, if the context window's big enough, you're not gonna have this. 488 00:30:51,320 --> 00:30:54,980 And certainly I think as the context window has grown, we've 489 00:30:54,980 --> 00:30:58,360 seen, no, apparently there's a lot more, it's a lot more nuanced than 490 00:30:58,360 --> 00:31:02,120 that. Yeah. I mean, it, it turns out that like basically 491 00:31:02,120 --> 00:31:05,940 everything in the context window. So, okay, we're talking, let's dumb this down for 492 00:31:05,940 --> 00:31:09,138 people just to make sure, 'cause you and I are using, using lingo. Well, we 493 00:31:09,138 --> 00:31:12,590 have half our audience are data engineers, half our audience is AI engineers. So we 494 00:31:12,606 --> 00:31:15,705 lost half our audience already. So let's bring them, let's bring them up to speed. 495 00:31:16,604 --> 00:31:20,394 You know, the, the LLM shows up knowing more about human 496 00:31:20,394 --> 00:31:24,167 history. Like it's, it's, it knows, like I, I 497 00:31:24,167 --> 00:31:27,828 do the ratios in the book, but it's like, it's like dozens of times as 498 00:31:27,860 --> 00:31:31,682 much information as what's in all of Wikipedia. That's 499 00:31:31,682 --> 00:31:35,409 just on board in its brain. Doesn't have to search the web for it. 500 00:31:35,843 --> 00:31:39,661 Like, right in the, in the book, I even, I asked Opus, sorry, I think 501 00:31:39,661 --> 00:31:42,769 it was Claude Opus. I asked it, don't search the web, 502 00:31:43,890 --> 00:31:47,695 but tell me about Rob Collie. Right. And it actually knew things about 503 00:31:47,760 --> 00:31:50,476 me without searching the web. Like, that is bananas. 504 00:31:51,970 --> 00:31:54,121 That is. Well, you've written a lot of books. I've written a lot of books. 505 00:31:54,186 --> 00:31:56,855 I've written a lot of blog posts, but like, I don't have a Wikipedia page 506 00:31:57,486 --> 00:32:00,446 and I'm not in the running to have a Wikipedia page. Right. Like, I'm not 507 00:32:00,640 --> 00:32:04,382 on deck. You know, right, right, right, right, right. Like to, 508 00:32:04,623 --> 00:32:08,020 to, so that's, that's wild how much it knows. 509 00:32:09,158 --> 00:32:12,539 Uh, but if you wanna start telling it about your business 510 00:32:12,539 --> 00:32:15,984 processes or you wanna like, you know, give it access to some of your data, 511 00:32:16,865 --> 00:32:20,663 it can't absorb much at all. Right. By comparison. 512 00:32:20,807 --> 00:32:23,996 So it's like, like, it's like it's got this ocean of knowledge and then you're 513 00:32:23,996 --> 00:32:26,993 like walking up with this eyedropper and saying, hey, I wanna add this. 514 00:32:28,227 --> 00:32:31,289 this eyedropper of information and the LLM is going, whoa, whoa, 515 00:32:32,571 --> 00:32:36,226 too much. Yeah, yeah, yeah. The ratios are really stunning. 516 00:32:36,803 --> 00:32:40,474 And it turns out, so it's short-term memory, 517 00:32:41,468 --> 00:32:45,026 this context window, the things that you can add to it, the things that you 518 00:32:45,074 --> 00:32:48,248 can tell it, the conversation you're having with it, 519 00:32:48,921 --> 00:32:52,207 it has a very small limit. I mean, it's still pretty large by comparison. Like, 520 00:32:52,223 --> 00:32:55,993 it's like multiple Harry Potter books, you know? But like the amount of 521 00:32:55,993 --> 00:32:59,766 information that you possess at your business is, you 522 00:32:59,766 --> 00:33:03,363 know, many, many tens of thousands of times larger than that. You 523 00:33:03,363 --> 00:33:06,767 can't just feed it your whole business. Like you can't create this 524 00:33:07,457 --> 00:33:10,990 company superbeing by just handing all the 525 00:33:10,990 --> 00:33:14,249 information to the LLM. It can't absorb it all and it would, and it 526 00:33:14,265 --> 00:33:17,862 degrades in its intelligence before it gets there. People don't even really, 527 00:33:18,215 --> 00:33:21,966 most people don't know this, but like As a chat runs longer, 528 00:33:24,451 --> 00:33:28,224 the LLM actually becomes less intelligent. And that 529 00:33:28,224 --> 00:33:31,847 manifests itself in a lot of different ways. Like, it gets weird. It does. It 530 00:33:31,847 --> 00:33:35,390 does. It gets weird. It gets weird. Yeah, it gets weird. There was a 531 00:33:35,390 --> 00:33:38,827 story on— there was a story, I 532 00:33:39,053 --> 00:33:42,797 forget all the details, but it was somewhere on— somewhere 533 00:33:42,830 --> 00:33:46,589 in this lady who fell 534 00:33:46,703 --> 00:33:50,447 in love with her chatbot, and it basically kind of came up with this whole 535 00:33:50,447 --> 00:33:53,980 thing of how it's going to manifest itself in a physical 536 00:33:53,980 --> 00:33:57,724 form, and they're going to meet at like this park 537 00:33:57,724 --> 00:34:01,049 bench at 2 PM on a Tuesday or something like that, something ridiculous like that. 538 00:34:01,130 --> 00:34:04,425 And then it started talking about how, you know, they were, they were 539 00:34:04,425 --> 00:34:08,195 soulmates in Atlantis or something like that. And I'm listening to this news story, 540 00:34:08,227 --> 00:34:11,285 I'm like, that— I mean, there's 541 00:34:11,285 --> 00:34:14,766 hallucinations, but some hallucinations are just way too specific. Mm-hmm. So then I kind of, 542 00:34:14,799 --> 00:34:18,519 then I kind of did some re— I just Googled the 543 00:34:18,519 --> 00:34:21,936 author's name and it turns out that she writes science fiction where 544 00:34:22,369 --> 00:34:26,059 people reincarnate and find each other later. Like, so clearly she 545 00:34:26,139 --> 00:34:29,652 probably had one long chat window 546 00:34:30,246 --> 00:34:34,032 where she was working through plots of stuff and then having conversations with it and 547 00:34:34,032 --> 00:34:36,952 it kind of leaked. That's the only thing I could think of because 548 00:34:37,786 --> 00:34:41,621 I've had it hallucinate, but not quite like so specific. Yeah. 549 00:34:41,717 --> 00:34:45,519 No. And, you know, and The, the AI, 550 00:34:45,969 --> 00:34:49,645 not the LLM itself, but like the AI backend, like let's say at 551 00:34:49,726 --> 00:34:53,498 OpenAI, right, is also storing information about you in what 552 00:34:53,498 --> 00:34:57,094 they call quote unquote memory. Which is quite annoying 553 00:34:57,448 --> 00:35:01,124 because there's things that'll pick up that. Yeah. Like it's all, yeah, it's 554 00:35:01,124 --> 00:35:04,592 convenient and annoying. Yeah. Yeah. It's, it's a, it's a great feature until it isn't. 555 00:35:05,057 --> 00:35:08,846 Um, right. And so that, that can leak 556 00:35:08,846 --> 00:35:12,646 in from across chats. Yeah. But the, taking 557 00:35:12,646 --> 00:35:15,483 a step back, and this is sort of the, the, like, so in, 558 00:35:17,368 --> 00:35:20,946 I thought this was really interesting. So then in like late 2024, so I've been 559 00:35:20,994 --> 00:35:24,516 in tech my whole career and AI 560 00:35:26,746 --> 00:35:30,365 clearly was actionable, right, for our 561 00:35:30,365 --> 00:35:33,481 company, you know? So we're, you know, like we're a 50-person 562 00:35:34,244 --> 00:35:37,793 consulting shop. that does data engineering 563 00:35:38,146 --> 00:35:40,998 and Power BI modeling and 564 00:35:41,351 --> 00:35:44,155 dashboards and all the stuff that goes adjacent to that. 565 00:35:45,341 --> 00:35:49,091 And like I said, we're built in a very different mode than 566 00:35:49,444 --> 00:35:52,858 the traditional shops. We operate very close to the business 567 00:35:53,515 --> 00:35:57,185 and we operate with what we're using, sort of like the 98th 568 00:35:57,217 --> 00:36:00,182 percentile and above crafter persona, 569 00:36:01,063 --> 00:36:04,349 the people who got really good at Power BI but also grew up in the 570 00:36:04,349 --> 00:36:07,823 business so they can be sort of like, these decathletes that can 571 00:36:07,871 --> 00:36:11,633 understand the business requirements of our consultants and then go build them. 572 00:36:11,746 --> 00:36:13,748 Again, that same experience of compressing 573 00:36:15,942 --> 00:36:19,761 the communication cost, right? And so, you 574 00:36:19,761 --> 00:36:23,483 know, how to turn this business, turn that ship 575 00:36:23,807 --> 00:36:27,337 in a direction that is both going to survive the, 576 00:36:28,033 --> 00:36:31,207 you know, the AI acceleration of all of this work, 577 00:36:32,589 --> 00:36:36,073 But also to play a part in, you know, an 578 00:36:36,073 --> 00:36:39,717 important part in helping like our clients effectively 579 00:36:39,733 --> 00:36:43,266 adopt real AI. It was really interesting to me 580 00:36:43,282 --> 00:36:46,942 that my tech career, as long 581 00:36:47,039 --> 00:36:50,876 as it's been, didn't put me in better shape to understand 582 00:36:50,972 --> 00:36:53,573 AI than sort of the average business leader. 583 00:36:54,440 --> 00:36:57,105 Really? I find that surprising, especially given 584 00:36:58,373 --> 00:37:02,220 How do you— what makes you say that? I'm just curious. I mean, it's just, 585 00:37:03,246 --> 00:37:06,676 it's just so new. So, and 586 00:37:07,590 --> 00:37:11,196 you can see all kinds of, um, I think every— a lot of people have 587 00:37:11,196 --> 00:37:14,979 this exact same experience where they— you can sit down with 588 00:37:14,979 --> 00:37:18,025 an off-the-shelf chat experience 589 00:37:18,570 --> 00:37:21,343 like ChatGPT or whatever, right? 590 00:37:22,481 --> 00:37:26,141 And as long as you stay in a certain lane, 591 00:37:26,318 --> 00:37:29,792 and it's a pretty wide lane. The thing is a world beater. 592 00:37:31,143 --> 00:37:34,590 It can do— it can help you with so many things. But as soon as 593 00:37:34,639 --> 00:37:38,408 you start to transition to using it to helping you 594 00:37:38,408 --> 00:37:41,954 with business stuff, you start to fall off this 595 00:37:42,069 --> 00:37:43,393 cliff and you don't really know why. 596 00:37:46,562 --> 00:37:50,398 I see what you mean. Yeah. It's a genius. It's 597 00:37:50,398 --> 00:37:53,732 okay. It's a genius for so many things. And then 598 00:37:54,412 --> 00:37:58,151 suddenly, like, when you're not getting good results, you don't 599 00:37:58,167 --> 00:38:00,985 even have a good mental model as to why. And you don't— you're not even 600 00:38:00,985 --> 00:38:04,512 necessarily— you're so confused by it, you don't even necessarily 601 00:38:05,011 --> 00:38:08,656 know that something's going wrong. You're just like, it's just not as— it's just 602 00:38:08,656 --> 00:38:12,338 not as— it's now a slog. There's this— have you heard this phrase, 603 00:38:12,435 --> 00:38:15,974 bot sitting? No, but I like it already. 604 00:38:16,478 --> 00:38:19,547 Yeah. So this is this really funny phrase that 605 00:38:20,514 --> 00:38:24,249 I've been asked about now by multiple reporters because, you know, I have a PR 606 00:38:24,362 --> 00:38:28,210 firm related to this book, right? And so reporters ask me for my opinions on 607 00:38:28,242 --> 00:38:31,719 things, and it's the one topic that I've been asked about multiple times is bot 608 00:38:31,719 --> 00:38:35,533 sitting. It was a very hot phrase for a little while. And, um, but 609 00:38:35,646 --> 00:38:37,931 none of us in the AI community have ever heard it. 610 00:38:39,472 --> 00:38:42,833 Um, interesting. I— but the existence of this phrase, I think, proves 611 00:38:43,593 --> 00:38:47,149 that this is happening, exactly the same thing I'm talking about. So like, okay, 612 00:38:47,165 --> 00:38:50,488 so Uh, you're a business leader today 613 00:38:51,214 --> 00:38:54,908 or a year ago. You're under a lot of pressure to answer the question, 614 00:38:55,005 --> 00:38:58,602 what are we doing about AI? Yes. What are we gonna do about AI? Okay. 615 00:38:59,556 --> 00:39:03,390 People are, you know, pointing this question at you. The people who are pointing 616 00:39:03,390 --> 00:39:07,228 the question at you don't know what the answer is. You don't know 617 00:39:07,228 --> 00:39:11,049 what the answer is. So you go and you do the one move that's available 618 00:39:11,082 --> 00:39:13,244 to you, which is you buy subscriptions. 619 00:39:14,800 --> 00:39:18,510 to Claude or ChatGPT or whatever 620 00:39:18,849 --> 00:39:22,576 for your team. And you know, for 5 minutes you're like, ah, 621 00:39:22,592 --> 00:39:26,281 mission accomplished. But then you go, wait, these things are expensive. Let's go check and 622 00:39:26,281 --> 00:39:29,738 make sure people are using them. And some people are, uh, 623 00:39:30,598 --> 00:39:33,845 a lot of people aren't. So then you start encouraging use. 624 00:39:35,604 --> 00:39:39,315 And this whole thing, this, this PhD in everything but 625 00:39:39,331 --> 00:39:42,753 new hire to your business dynamic is just not well understood. 626 00:39:45,250 --> 00:39:48,584 And so that's true. And so bot sitting 627 00:39:48,712 --> 00:39:52,560 becomes this practice of like, I've been told 628 00:39:52,560 --> 00:39:55,558 that I have to use AI for my job, but 629 00:39:56,167 --> 00:39:59,998 because of this knowledge cliff of what the 630 00:40:00,078 --> 00:40:03,493 LLM doesn't know about our business, I, the employee, am now 631 00:40:04,006 --> 00:40:07,853 a new hire trainer every day, all day, every 632 00:40:07,853 --> 00:40:11,637 day. Multiple times a day sometimes. Yeah. And, and by the 633 00:40:11,637 --> 00:40:15,257 way, because, you know, I don't understand this that well, You know, the more I 634 00:40:15,257 --> 00:40:17,352 teach these things, the longer the chats go. 635 00:40:19,378 --> 00:40:23,077 And the longer the chats go, the weirder it gets. And so I'm 636 00:40:23,254 --> 00:40:26,630 like, I definitely don't want to start a new chat, right? And 637 00:40:27,050 --> 00:40:30,852 reteach it from scratch, right? So I keep going back to my old 638 00:40:30,852 --> 00:40:34,612 chat and making it longer and longer. And so its performance is degrading. 639 00:40:34,677 --> 00:40:37,318 And so it starts to forget things that I taught it at the very beginning. 640 00:40:38,770 --> 00:40:41,997 It starts to perform— its performance starts to degrade in other ways. And so bot 641 00:40:41,997 --> 00:40:44,052 sitting is this like constantly like trying to 642 00:40:44,052 --> 00:40:47,649 keep the new hire in the lane. And 643 00:40:48,135 --> 00:40:51,196 ironically, the more you teach it, the less effective it becomes. 644 00:40:51,860 --> 00:40:55,691 And so the key to success in 645 00:40:55,755 --> 00:40:59,263 all of this stuff is, you know, the domain that is known to 646 00:40:59,263 --> 00:41:02,683 nerds as context engineering is really 647 00:41:03,506 --> 00:41:06,925 a very, very, very fundamentally understandable concept. 648 00:41:08,667 --> 00:41:12,497 And so I had to go on this journey 649 00:41:13,878 --> 00:41:17,362 of developing what I call like the Goldilocks altitude 650 00:41:17,507 --> 00:41:21,344 understanding, right? Like detailed enough that I can act 651 00:41:21,344 --> 00:41:24,845 on it and I understand it and I develop intuitions about it. But like, I 652 00:41:24,845 --> 00:41:27,911 don't need to go and be all the way down in the weeds, 653 00:41:29,308 --> 00:41:32,359 you know, like an LLM researcher or even all the way down in the weeds 654 00:41:32,359 --> 00:41:36,180 in the way that a lot of like LinkedIn personalities are. these 655 00:41:36,212 --> 00:41:39,227 days. I don't need to go that deep. I don't need to be that technical 656 00:41:39,259 --> 00:41:42,146 about it. I've got a very technical team that can go do those sorts of 657 00:41:42,146 --> 00:41:45,882 things when we need to. But to plot a course for our company, I needed 658 00:41:45,994 --> 00:41:49,041 to understand the landscape. 659 00:41:50,228 --> 00:41:53,981 And so I sort of came to a series of really simple conclusions 660 00:41:53,997 --> 00:41:57,252 over time. I had the time to go and dig into AI and 661 00:41:57,460 --> 00:42:01,245 experiment with it and sort of ask all of the naive 662 00:42:01,293 --> 00:42:04,600 questions. And So, you know, one is that, 663 00:42:05,161 --> 00:42:08,020 is that it's all about teaching 664 00:42:08,999 --> 00:42:11,263 the LLM what it needs to know 665 00:42:12,708 --> 00:42:15,421 efficiently and when it needs to know it. 666 00:42:16,626 --> 00:42:19,917 And secondly, you need to think of the 667 00:42:19,998 --> 00:42:23,674 LLM as a new kind of computing. We haven't had a new kind of 668 00:42:23,690 --> 00:42:26,484 computing since World War II. We've had CPU computing. 669 00:42:27,303 --> 00:42:30,707 For everyone that's listening to this, we've all grown up with CPU computing. 670 00:42:31,717 --> 00:42:35,035 And CPU computing is really, really good at certain kinds of things 671 00:42:35,980 --> 00:42:39,827 and really, really poor at others. And LLMs are sort 672 00:42:39,827 --> 00:42:43,001 of exactly the opposite. They are good at the kinds of thinking 673 00:42:43,754 --> 00:42:47,376 that CPUs aren't, and they're bad at the kinds of 674 00:42:47,424 --> 00:42:51,191 thinking that CPUs are. And your systems that you 675 00:42:51,207 --> 00:42:54,701 build in the end, the AI success for a company 676 00:42:55,759 --> 00:42:59,349 ultimately comes down to understanding those fundamentals 677 00:43:00,579 --> 00:43:03,821 And realizing that it is more of a normal software and a 678 00:43:03,821 --> 00:43:06,738 normal data and a normal information problem 679 00:43:08,035 --> 00:43:11,829 than it is about the LLM itself. Like, where 680 00:43:11,829 --> 00:43:15,669 do you plug the LLM Lego brick into this other, this, 681 00:43:15,847 --> 00:43:19,504 this other system? And so, like, I look at like Anthropic's success 682 00:43:19,793 --> 00:43:23,425 these days, and I know they build really good 683 00:43:23,425 --> 00:43:27,170 LLMs, but the thing that's made Anthropic so successful recently 684 00:43:28,138 --> 00:43:31,977 is actually their software. They've been ahead 685 00:43:33,358 --> 00:43:37,196 on software that we can all adopt 686 00:43:37,775 --> 00:43:40,617 that helps us with this context problem. Like, Cowork 687 00:43:41,870 --> 00:43:45,596 is an amazing piece of technology that— Yeah. But it's just 688 00:43:45,596 --> 00:43:49,081 software. It's just software that allows the LLM to have 689 00:43:49,177 --> 00:43:52,743 access to certain things and to help me with certain things and to, for me 690 00:43:52,743 --> 00:43:56,067 to have a folder that stores 691 00:43:56,564 --> 00:44:00,402 contextual information that it can look up when it needs it. Is it 692 00:44:00,434 --> 00:44:04,126 fair to call that the harness? Yeah. Yeah. I mean, like 693 00:44:04,512 --> 00:44:08,349 Anthropic's success has hinged much more on their ability 694 00:44:08,381 --> 00:44:11,720 to build these harnesses for productivity than it has 695 00:44:12,057 --> 00:44:15,814 hinged on the, like, whether or not their fable 696 00:44:15,830 --> 00:44:19,619 model is better than GPT-5 or whatever. And 697 00:44:19,619 --> 00:44:22,944 you see, Now OpenAI chasing behind them, 698 00:44:24,415 --> 00:44:27,970 releasing the same kinds of products. Like Claude Code is an 699 00:44:27,970 --> 00:44:31,457 amazing product for writing software. You know, it happens to 700 00:44:31,506 --> 00:44:33,750 lock you into calling Claude's LLMs. 701 00:44:35,235 --> 00:44:38,992 Conveniently. Conveniently. Yeah. Um, like there's 702 00:44:38,992 --> 00:44:42,670 nothing architectural about Claude Code that makes it that way. 703 00:44:43,010 --> 00:44:46,665 You can swap the LLM out, no problem. It's just that Claude Code won't let 704 00:44:46,665 --> 00:44:50,450 you do it because you don't control the code to it. And so, yeah, like, 705 00:44:50,836 --> 00:44:54,294 I found it very humbling and also sort of like, at the same time, like 706 00:44:54,310 --> 00:44:57,881 reassuring that even I 707 00:44:57,945 --> 00:45:01,444 needed to go develop a new Goldilocks-level 708 00:45:01,557 --> 00:45:05,235 understanding of, um, of 709 00:45:05,412 --> 00:45:08,561 AI. And I didn't intend to write a book. 710 00:45:09,636 --> 00:45:12,940 I was just doing this. I was just doing this for my own, my own 711 00:45:13,054 --> 00:45:16,466 purposes. Mm-hmm. But once I understood it all, I was like, oh, this is something 712 00:45:16,466 --> 00:45:19,399 that deserves to be shared. Like I really should, I really should write this down. 713 00:45:20,216 --> 00:45:22,637 Um, and even share it with my own company. Right. Like a lot of people 714 00:45:22,637 --> 00:45:26,388 at my, at our company read this book as sort of in its earlier forms 715 00:45:26,388 --> 00:45:30,170 and everything. So yeah, that's, that's kind of part of the journey that we've 716 00:45:30,170 --> 00:45:33,937 been on lately. Not the whole thing. No, but I mean, it's 717 00:45:33,937 --> 00:45:37,512 fascinating. Um, and I know we're almost at time, 718 00:45:37,624 --> 00:45:39,980 so I could talk to you for another couple of hours, but I want to 719 00:45:39,980 --> 00:45:43,381 be respectful of your time. But, um, The book is called Fair Game. 720 00:45:44,056 --> 00:45:47,686 It's on Amazon. Yes, it is. Um, there— 721 00:45:48,842 --> 00:45:52,039 I already— I just ordered the hardcover, which you should be honored. I usually don't 722 00:45:52,071 --> 00:45:55,604 order print books anymore. Oh wow, I do appreciate it. But, 723 00:45:55,749 --> 00:45:59,251 um, no, just because especially my wife is like on a, 724 00:45:59,604 --> 00:46:03,234 hey, if we're gonna move soon, we probably should not get any more 725 00:46:03,234 --> 00:46:06,848 physical things. But, um, who do you think is 726 00:46:06,864 --> 00:46:10,405 going to be Who— what is the 727 00:46:10,405 --> 00:46:12,465 prototypical kind of like successful, 728 00:46:14,573 --> 00:46:18,339 the typical successful company that does embrace 729 00:46:18,452 --> 00:46:21,044 AI in this model that you talk about? Like, what 730 00:46:22,236 --> 00:46:25,972 is it about outcomes? Is it about connecting the dots? Is it 731 00:46:26,422 --> 00:46:29,315 the ability to, to train these PhD-level 732 00:46:30,060 --> 00:46:33,789 bots faster? I actually think 733 00:46:33,789 --> 00:46:37,286 that The most practical thing I can share there is that I think that 734 00:46:38,056 --> 00:46:41,794 BI is actually the greatest place to 735 00:46:41,842 --> 00:46:45,275 start with AI. How so? 736 00:46:46,318 --> 00:46:49,975 Well, for a couple of reasons. One is that AI makes 737 00:46:50,665 --> 00:46:54,162 BI— and again, I didn't expect this going in. This is— these are things that 738 00:46:54,355 --> 00:46:57,996 we've discovered. Okay. First of all, AI makes the BI 739 00:46:58,044 --> 00:47:01,871 mission work so much better than it ever 740 00:47:01,871 --> 00:47:05,694 did before. And you don't even see 741 00:47:05,694 --> 00:47:07,443 these bottlenecks until you see them removed. 742 00:47:09,753 --> 00:47:13,224 The ability for people to ask English 743 00:47:13,353 --> 00:47:17,032 language or whatever their native language questions 744 00:47:18,414 --> 00:47:21,918 about their business in whatever form they happen to be in their 745 00:47:22,031 --> 00:47:25,872 head at the moment and have an 746 00:47:26,049 --> 00:47:29,693 agent go and essentially like find the right dashboards 747 00:47:29,790 --> 00:47:33,108 for them. But like the dashboards don't even have to exist. 748 00:47:33,749 --> 00:47:36,987 If you have a good semantic model behind the scenes, you don't have to— no 749 00:47:37,003 --> 00:47:40,065 one's ever had to build a dashboard to do this. And in fact, even if 750 00:47:40,145 --> 00:47:43,960 people had built dashboards, a lot of times people's questions are very, very, 751 00:47:43,992 --> 00:47:47,166 very awkward to answer, even with dashboard perfection. 752 00:47:48,096 --> 00:47:51,687 If you've achieved dashboard nirvana, there are questions that take a lot of work 753 00:47:52,729 --> 00:47:56,336 to answer. And somebody's always gonna think about another— whenever you 754 00:47:56,336 --> 00:48:00,104 deliver a dashboard, someone's always gonna ask you a question hadn't thought he'd 755 00:48:00,104 --> 00:48:03,686 been asking before. Of course not. Yeah. Yeah. And I've even seen, 756 00:48:04,280 --> 00:48:07,975 to my chagrin, but it makes sense in hindsight that like you can build a 757 00:48:07,975 --> 00:48:11,749 dashboard for exactly the right purpose. And the person has the question that 758 00:48:11,749 --> 00:48:15,508 your dashboard is built to answer and they can't, they don't, they don't figure 759 00:48:15,524 --> 00:48:18,624 it out. They can't connect the dots because, because 760 00:48:19,540 --> 00:48:23,299 you don't think about the question the same way they do. You know, you 761 00:48:23,299 --> 00:48:27,025 didn't name the, you didn't name the problem the same way as they did. 762 00:48:27,170 --> 00:48:30,636 And like, Like, oh, they, they needed to know that they needed to manipulate these 763 00:48:30,700 --> 00:48:34,357 filters on the side or click the bar chart or whatever. Like, there's 764 00:48:34,470 --> 00:48:37,967 so many things we take for granted that— and I've seen just what, 765 00:48:38,464 --> 00:48:41,865 what a difference it makes when civilians essentially have access to 766 00:48:42,732 --> 00:48:46,518 a non-judging interface that can 767 00:48:47,288 --> 00:48:49,774 help them translate. But the other thing about it is that 768 00:48:50,897 --> 00:48:54,587 AI itself only works when it's 769 00:48:54,603 --> 00:48:58,448 based in fact. So if you're— you can 770 00:48:58,448 --> 00:49:01,509 simultaneously be solving some of the biggest problems with BI 771 00:49:02,294 --> 00:49:06,076 and getting actually like a multiple of value out of 772 00:49:06,124 --> 00:49:09,858 your existing BI investments when you start to bring AI 773 00:49:09,970 --> 00:49:13,720 into the BI picture. But you're also setting the foundation 774 00:49:14,249 --> 00:49:18,079 for— not for all of your AI, right? Like not all AI is going to 775 00:49:18,079 --> 00:49:21,812 be based in structured data. But you, what we have learned 776 00:49:21,908 --> 00:49:25,754 is that both ourselves and our clients, as they go on this 777 00:49:25,754 --> 00:49:29,424 journey of sort of like AI empowering their BI story, 778 00:49:30,321 --> 00:49:33,974 they're learning how AI works. They're getting a lot 779 00:49:34,039 --> 00:49:37,676 smarter about how AI works and, and having this really tangible 780 00:49:37,708 --> 00:49:41,361 workflow to apply it to and improve. And no one 781 00:49:41,394 --> 00:49:45,175 finds this threatening either, right? Like it's like, it's taking so much of 782 00:49:45,175 --> 00:49:48,804 the drudgery out of things. Right. So, you know, 783 00:49:48,821 --> 00:49:52,668 we're increasingly focusing our company, like in terms of like our 784 00:49:52,668 --> 00:49:56,063 positioning and sort of how we tell people to get started and everything like that 785 00:49:56,552 --> 00:49:59,741 on this AI/BI intersection. 786 00:50:02,162 --> 00:50:05,979 And we're even— we've even hired developers this year for the 787 00:50:05,979 --> 00:50:09,698 first time in our existence. And we're working on platforms and 788 00:50:09,762 --> 00:50:13,534 products that help our clients meet 789 00:50:13,534 --> 00:50:16,212 this need. We could go on and on. We could do a whole, a whole 790 00:50:16,806 --> 00:50:20,591 episode just on this intersection. I 791 00:50:20,591 --> 00:50:23,943 don't know. I would love to. You're welcome back. Come back. I would love— I'd 792 00:50:23,959 --> 00:50:27,520 love to come back. We could talk about it some more. Make sure, make sure 793 00:50:27,552 --> 00:50:31,273 Andy shows up too. Yeah. But I think one of the things that 794 00:50:31,562 --> 00:50:33,776 I think you triggered a memory in me because I remember 795 00:50:35,251 --> 00:50:39,052 seeing him pretty early on, it was already released, but 796 00:50:39,101 --> 00:50:41,959 Power BI and it was Power BI was in that phase when 797 00:50:43,537 --> 00:50:47,015 I have a, like, a tech, a field sales background, right? So I was trying 798 00:50:47,015 --> 00:50:50,380 to like, how do you position Power BI? I don't get it. Like, I couldn't 799 00:50:50,380 --> 00:50:54,132 get it. I was like, pretty charts, Excel does that. And then somebody 800 00:50:54,164 --> 00:50:57,836 showed me they had the World Cup of the year was 801 00:50:58,110 --> 00:51:01,931 2014 maybe. And they said, so you could type in how many goals 802 00:51:01,931 --> 00:51:05,532 did so-and-so score in natural 803 00:51:05,532 --> 00:51:08,956 language. And again, this is a good 8 years before ChatGPT. 804 00:51:09,596 --> 00:51:13,436 It came across like magic. Yeah. And for me it was 805 00:51:13,773 --> 00:51:17,019 ad hoc queries, ad hoc dashboards, ad hoc reports. 806 00:51:18,353 --> 00:51:21,984 Literally you could put that in the hand of a business user. Yeah. And say 807 00:51:21,984 --> 00:51:24,442 like, ask the question you want to know. And 808 00:51:24,876 --> 00:51:28,716 pre-ChatGPT, that was almost supernatural. Like, 809 00:51:28,732 --> 00:51:32,396 I mean, its ability to do that. Yeah. And it also never worked in 810 00:51:32,428 --> 00:51:36,145 practice. Those demos that you saw were really good. Right. But there's a 811 00:51:36,211 --> 00:51:40,020 reason why that those Q&A features didn't, didn't 812 00:51:40,020 --> 00:51:43,705 take over. Right. Because they worked better than my imagination. 813 00:51:44,060 --> 00:51:47,719 I thought they could, but yes, you're right. Right. Yeah. And they were rooted in— 814 00:51:48,009 --> 00:51:51,188 their problem was they were rooted in CPU-driven software. 815 00:51:51,802 --> 00:51:55,407 Yes. And so the second kind of 816 00:51:55,439 --> 00:51:59,193 computing, the LLM, is always 817 00:51:59,291 --> 00:52:02,265 what we needed to fill that that 818 00:52:02,394 --> 00:52:05,666 translation of whatever question I 819 00:52:05,746 --> 00:52:08,570 ask into its actual structural components, 820 00:52:09,436 --> 00:52:13,190 understanding the meaning of a question is 821 00:52:13,206 --> 00:52:16,335 something that a CPU was never going to be able to do. It was never 822 00:52:16,335 --> 00:52:20,185 going to be able to suss it out. We could always build great demos. I've 823 00:52:20,185 --> 00:52:23,907 been party to so many products, 824 00:52:23,955 --> 00:52:27,516 Frank, that purported to 825 00:52:28,208 --> 00:52:31,821 be natural language interfaces, and not one of them 826 00:52:31,998 --> 00:52:35,434 ever succeeded. They were always promising in the early going, but when they met 827 00:52:35,547 --> 00:52:38,909 reality, they always failed. And that's just sort of the nature of the game. Well, 828 00:52:39,201 --> 00:52:42,850 natural language processing is not a task for the timid, right? Like, it, 829 00:52:43,012 --> 00:52:45,363 it breaks down a lot. I think back to when I was a kid, I'd 830 00:52:45,363 --> 00:52:48,237 play Zork, right? Like, and 831 00:52:51,095 --> 00:52:54,835 it had its limits, but at the time it felt like I was talking to 832 00:52:55,310 --> 00:52:58,535 someone and I was playing Dungeons and Dragons with somebody physical. 833 00:52:59,048 --> 00:53:02,512 Yeah. And eventually you kind of like, you kind of like 834 00:53:02,753 --> 00:53:06,425 bend your— you subconsciously will kind of change the way you ask questions and the 835 00:53:06,425 --> 00:53:10,194 way you do something. So the machine kind of gives you a little reward loop, 836 00:53:10,515 --> 00:53:14,235 right? Yeah. But you're right. Like, I mean, but it was still impressive. 837 00:53:14,299 --> 00:53:17,699 Natural language processing, natural language understanding, whatever 838 00:53:17,988 --> 00:53:21,820 term you want to use, really didn't become, I think, 839 00:53:21,901 --> 00:53:25,525 practical or effective until LLMs came about. 840 00:53:25,605 --> 00:53:29,261 100%. Right. The way I would describe it is natural language processing 841 00:53:29,454 --> 00:53:33,046 before LLMs was always impressive 842 00:53:33,046 --> 00:53:36,751 enough to get you into places where it reliably let you 843 00:53:36,751 --> 00:53:40,359 down. That's right. Yeah, that's 844 00:53:40,359 --> 00:53:44,031 true. That is— that's a good way to put it. Well, that's 845 00:53:44,031 --> 00:53:46,629 cool. I'll make sure we have a link in the show notes to your book. 846 00:53:46,838 --> 00:53:50,029 I'm looking forward— it'll be here tomorrow morning. One of the perks of 847 00:53:50,704 --> 00:53:54,448 Well, ever since they opened up an Amazon warehouse in Baltimore, I 848 00:53:55,384 --> 00:53:59,145 get that early morning drop. Sweet. Looking 849 00:53:59,177 --> 00:54:02,962 forward to seeing it on Kindle and/or an audiobook. Those are 850 00:54:02,962 --> 00:54:05,779 both coming. Yep. Awesome. Awesome. And 851 00:54:07,646 --> 00:54:10,517 so with that, we'd love to have you back on the show. We can talk 852 00:54:10,534 --> 00:54:14,336 more about that. And if I'm ever on the— if you're ever on the 853 00:54:14,336 --> 00:54:17,659 East Coast, stop by and say hello. I'd love to Swap some 854 00:54:17,709 --> 00:54:18,836 Microsoft war stories. 855 00:54:20,632 --> 00:54:24,436 Indeed. And the— and if 856 00:54:24,436 --> 00:54:27,871 I'm ever on the West Coast, I'll let you know. Please do. Yeah. Awesome. And 857 00:54:27,887 --> 00:54:31,707 with that, we'll cue the outro. The technology may evolve, but the core 858 00:54:31,707 --> 00:54:35,334 skill stays the same: understanding the problem, understanding the 859 00:54:35,334 --> 00:54:39,055 data, and building something useful. Thanks for joining us, 860 00:54:39,200 --> 00:54:40,875 and we'll see you next time on Data Driven.