1 00:00:00,080 --> 00:00:03,440 It's very much based off of your requirements, your 2 00:00:03,440 --> 00:00:07,200 skills, your knowledge, your processes 3 00:00:07,200 --> 00:00:10,800 that now need to be defined within 4 00:00:11,520 --> 00:00:15,360 your AI stack. And that really is the last mile. And 5 00:00:15,360 --> 00:00:19,120 I think that's even I saw that from both conferences where 6 00:00:19,680 --> 00:00:23,360 the realization that there's still a lot of work that needs to be done to 7 00:00:23,360 --> 00:00:27,050 get AI to a point where it's actually very fine 8 00:00:27,050 --> 00:00:30,410 tuned, very functional, very efficient. 9 00:00:30,730 --> 00:00:34,490 And right now it may work, but it may not be very efficient for scaling, 10 00:00:34,490 --> 00:00:38,250 it may not be efficient for cost, it may not be efficient for the new 11 00:00:38,330 --> 00:00:42,090 token economy that we're seeing. And the last mile is 12 00:00:42,490 --> 00:00:46,010 historically the biggest problem to crack. Right. And once you solve that problem, 13 00:00:46,250 --> 00:00:50,050 Amazon as a physical last mile in terms of 14 00:00:50,050 --> 00:00:53,370 how they actually execute on delivery, right, because you can have 15 00:00:53,370 --> 00:00:57,160 warehouses, but everybody lives in a different house, right? 16 00:00:57,160 --> 00:01:00,760 So there's a lot of little last miles. It's, it's 17 00:01:00,760 --> 00:01:04,480 death by a thousand paper cuts, if you will. Proof of concept 18 00:01:04,480 --> 00:01:08,240 projects are everywhere. Real business value, that's 19 00:01:08,240 --> 00:01:11,120 the hard part. This is data Driven. 20 00:01:21,530 --> 00:01:25,130 I'm Frank Lavinia and with me I have a very special guest, 21 00:01:25,450 --> 00:01:29,290 Christopher Newland, who is a colleague of mine at Red Hat. And 22 00:01:30,810 --> 00:01:34,610 we're gonna do a deep dive. You, you travel all the time. 23 00:01:34,610 --> 00:01:38,290 I know that I travel two weeks back to back and that was a been 24 00:01:38,290 --> 00:01:42,050 a while since I had to do that. But it is conference 25 00:01:42,050 --> 00:01:45,820 season and so you 26 00:01:45,820 --> 00:01:48,580 were at IBM think last week 27 00:01:49,860 --> 00:01:53,580 and you were also at Red Hat Summit this 28 00:01:53,580 --> 00:01:57,340 past week, as was I. I have my Atlanta I heart I Red Hat 29 00:01:57,340 --> 00:02:00,260 Atlanta T shirt on and 30 00:02:00,900 --> 00:02:04,540 so how's it going, Christopher? But yeah, it was nice 31 00:02:04,540 --> 00:02:08,300 because one of those two IBM think was 32 00:02:08,300 --> 00:02:12,059 actually in my area of Boston, so I was able 33 00:02:12,059 --> 00:02:15,779 to attend that locally. Still, still a lot 34 00:02:15,779 --> 00:02:19,139 though. Like, you know, you're going in at like 7 in the morning to be, 35 00:02:19,139 --> 00:02:22,619 try to beat traffic and then you're leaving at like 10 o' clock at night. 36 00:02:22,619 --> 00:02:25,619 But it's very poor. Yeah, 37 00:02:27,219 --> 00:02:30,939 yeah. And those conferences were very, very different, targeting very 38 00:02:30,939 --> 00:02:34,539 different audiences. So it was, I felt like I got kind of 39 00:02:34,539 --> 00:02:38,240 two perspectives of the AI world and 40 00:02:38,240 --> 00:02:41,960 what people are concerned about. One from a very like executive 41 00:02:42,200 --> 00:02:45,520 lens and another one from more the day to day users, 42 00:02:45,520 --> 00:02:49,160 developers, engineers who are actually implementing the AI. 43 00:02:50,040 --> 00:02:53,840 So which is which? I think I know the answer, but yeah. So 44 00:02:53,840 --> 00:02:57,080 I IBM think is an executive conference. 45 00:02:58,280 --> 00:03:02,120 So I think it's normally director level or above. 46 00:03:02,910 --> 00:03:06,390 So it's targeting a lot of C suites, senior 47 00:03:06,390 --> 00:03:10,070 directors, I think the 48 00:03:10,070 --> 00:03:13,510 most that you would ever, lowest you would See would be like a senior manager 49 00:03:13,510 --> 00:03:16,910 of some sort. But for the most part it's a C suite type of 50 00:03:16,910 --> 00:03:20,550 conference. And a lot of the conversation 51 00:03:20,550 --> 00:03:24,190 there is more about the business return of 52 00:03:24,190 --> 00:03:27,950 AI and what does that look like this year. And then 53 00:03:29,000 --> 00:03:31,080 Red Hat Summit is very much about 54 00:03:32,840 --> 00:03:36,480 the system administrator, the cluster administrator, the 55 00:03:36,480 --> 00:03:40,200 sre, the developer who's actually 56 00:03:40,200 --> 00:03:43,920 utilizing these technologies and actually like implementing something 57 00:03:43,920 --> 00:03:47,600 with it or managing something with it. So very like two 58 00:03:47,600 --> 00:03:51,080 different lenses to the same challenge within the industry. 59 00:03:52,600 --> 00:03:56,120 Yeah, no, it was interesting. And I don't know about you and what you, 60 00:03:56,820 --> 00:04:00,100 the attendees this year had much better questions, I think, than any other 61 00:04:00,500 --> 00:04:04,260 Red Hat event than I've ever AI questions than I ever seen before. 62 00:04:04,900 --> 00:04:08,420 Right. It seems like people are struggling to 63 00:04:08,420 --> 00:04:11,540 implement this in a way that is secure, 64 00:04:11,860 --> 00:04:13,380 stable, scalable. 65 00:04:16,020 --> 00:04:19,420 And I think we also have a much better platform story this year than we 66 00:04:19,420 --> 00:04:23,100 had in previous years. Absolutely. So the way I've been 67 00:04:23,100 --> 00:04:26,840 framing it to people, it kind of goes into two terms. So the first 68 00:04:26,840 --> 00:04:30,040 term I've been using with people is that last mile. 69 00:04:30,680 --> 00:04:34,120 And that then kind of feeds into this 70 00:04:34,120 --> 00:04:37,720 concept that you hear a lot about in business and other 71 00:04:37,720 --> 00:04:41,440 industries called the 8020 rule. I think a lot 72 00:04:41,440 --> 00:04:45,040 of people are finding that 80, 20, what 73 00:04:45,040 --> 00:04:48,480 is 80% of the returns or 74 00:04:48,480 --> 00:04:52,240 20% of the effort? And then what we find is that it 75 00:04:52,240 --> 00:04:55,560 flips for that remaining 20% 76 00:04:56,760 --> 00:05:00,520 of returns is now going to be 80% of 77 00:05:00,520 --> 00:05:04,280 the effort. And that 20% is what I've defined really as 78 00:05:04,280 --> 00:05:07,760 the last mile. And I think the 79 00:05:07,760 --> 00:05:11,400 conversations I'm having with people is that they now have the tools and they've had 80 00:05:11,960 --> 00:05:15,680 POCs and they're seeing results and they're seeing 81 00:05:15,680 --> 00:05:19,380 even a lot of times good results. They just don't know how do I get 82 00:05:19,380 --> 00:05:22,940 it to the point where it's actually returning investment, whereas 83 00:05:22,940 --> 00:05:26,580 roi. And this is a question that was happening at both conferences, 84 00:05:26,900 --> 00:05:30,540 both from an executive lens point and from the, you know, the 85 00:05:30,540 --> 00:05:34,300 general day to day developers. And this is where I think open source 86 00:05:34,300 --> 00:05:37,780 is set for in a great position because 87 00:05:38,260 --> 00:05:41,980 there's so many open source tools out there that we 88 00:05:41,980 --> 00:05:45,360 can work with people on, you know, finalizing that last 89 00:05:45,360 --> 00:05:49,200 mile. I think what people are most annoyed about though is 90 00:05:49,200 --> 00:05:52,960 that there's not a magic button that's going to fix it because it's 91 00:05:52,960 --> 00:05:56,160 very much based off of your requirements, your 92 00:05:56,160 --> 00:05:59,920 skills, your knowledge, your processes 93 00:06:00,000 --> 00:06:03,520 that now need to be defined within 94 00:06:04,160 --> 00:06:07,520 your AI stack. And that really is the last mile. 95 00:06:07,760 --> 00:06:11,360 And I think that's even I saw that from Both conferences 96 00:06:11,360 --> 00:06:15,160 where the realization that there's still a lot of work that needs to be 97 00:06:15,160 --> 00:06:18,960 done to get AI to a point where it's actually 98 00:06:19,040 --> 00:06:22,480 very fine tuned, very functional, very 99 00:06:22,480 --> 00:06:26,000 efficient right now. It may work, but it may not be very 100 00:06:26,000 --> 00:06:29,800 efficient for scaling, it may not be efficient for cost, it may not 101 00:06:29,800 --> 00:06:32,640 be efficient for the new token economy that we're seeing. 102 00:06:33,520 --> 00:06:37,320 And the last mile is historically the biggest problem to crack. Right. 103 00:06:37,320 --> 00:06:40,910 And once you solve that problem, Amazon as a 104 00:06:40,910 --> 00:06:44,510 physical last mile. Right. In terms of how they actually execute on 105 00:06:44,510 --> 00:06:47,710 delivery. Right. Because you can have warehouses, but 106 00:06:47,870 --> 00:06:51,510 everybody lives in a different house. Right. So there's a lot of little 107 00:06:51,510 --> 00:06:55,350 last miles. It's death by a thousand paper cuts, if 108 00:06:55,350 --> 00:06:59,150 you will. Absolutely. And we saw the same thing with microservices 109 00:06:59,470 --> 00:07:03,310 back in the 2010s where there are a lot 110 00:07:03,310 --> 00:07:07,160 of organizations that developed microservices but 111 00:07:07,160 --> 00:07:10,920 then had a lot of challenges and had to overcome a lot of 112 00:07:10,920 --> 00:07:14,440 that last mile when it came to data domain. 113 00:07:14,440 --> 00:07:18,240 And you know, where, where does your data exist within this 114 00:07:18,320 --> 00:07:22,040 microservice architecture? How do you do contracts and 115 00:07:22,040 --> 00:07:25,760 handshakes between services? How do you orchestrate these services? How do you scale 116 00:07:25,760 --> 00:07:29,600 them? You know, in many ways this is the 117 00:07:29,600 --> 00:07:33,200 problems that we saw kubernetes kind of develop out of. 118 00:07:34,160 --> 00:07:37,960 And now we're seeing being embraced now by a lot 119 00:07:37,960 --> 00:07:41,680 of the same challenges we're seeing with agentic systems and 120 00:07:41,680 --> 00:07:45,360 AI and how do we scale that out efficiently? 121 00:07:45,440 --> 00:07:49,080 So I love what you said. It's not a new problem. It's 122 00:07:49,080 --> 00:07:52,080 just the same problem we've seen reiterating over 123 00:07:52,720 --> 00:07:56,400 50 plus years of compute history that now 124 00:07:56,400 --> 00:08:00,160 just has a different lens to it of the 125 00:08:00,160 --> 00:08:03,680 AI problem now. But a lot of the same solutions are still the 126 00:08:03,680 --> 00:08:07,400 solutions that we had for many of those advancements in technology 127 00:08:07,560 --> 00:08:11,120 that we saw, you know, over the last few decades. That is 128 00:08:11,120 --> 00:08:14,600 interesting because like, you know, you know, Kubernetes does 129 00:08:14,680 --> 00:08:18,000 has solved a lot of the same problems and it doesn't solve them all. But 130 00:08:18,000 --> 00:08:21,760 there's a significant overlap and I, and I got that sense from the conference 131 00:08:21,760 --> 00:08:25,600 that people are finally starting to get it. Like, why OpenShift AI? 132 00:08:25,600 --> 00:08:28,760 Well, because OpenShift solves a lot of these problems. You just 133 00:08:29,810 --> 00:08:33,370 put AI on top of those solved problems and it 134 00:08:33,370 --> 00:08:37,050 doesn't fix everything. There's still going to be a lot more room 135 00:08:37,050 --> 00:08:40,890 for improvement in terms of how you implement that 136 00:08:40,890 --> 00:08:43,330 on your last mile. But it gets you 137 00:08:44,929 --> 00:08:48,130 halfway there from a get go easily. Yes, 138 00:08:48,850 --> 00:08:52,690 absolutely. A lot of the questions that 139 00:08:52,690 --> 00:08:56,470 IBM think were it was Actually funny, 140 00:08:56,470 --> 00:08:58,750 a lot of them are about IBM Bob and I know you and I have 141 00:08:58,750 --> 00:09:02,430 been kind of talking about this for like last two weeks, but 142 00:09:02,670 --> 00:09:06,350 at the IBM think IBM Bob was a very serious conversation of 143 00:09:06,430 --> 00:09:09,630 executives wanting to know how can they 144 00:09:10,110 --> 00:09:13,830 mimic tools like Claude code. Right. But 145 00:09:13,830 --> 00:09:17,390 within their enterprise setting. And the biggest thing about IBM 146 00:09:17,390 --> 00:09:20,990 Bob, that I learned actually at IBM think from both the engineers there and 147 00:09:21,500 --> 00:09:25,100 those who are interested in it, is that a big thing here is what they 148 00:09:25,100 --> 00:09:28,900 want for institutional knowledge. They want to keep a record 149 00:09:28,900 --> 00:09:32,460 of all that institutional knowledge from the prompts 150 00:09:32,620 --> 00:09:36,460 and the context and all the things that you 151 00:09:36,460 --> 00:09:39,340 know, are built out of IBM Bob, 152 00:09:40,220 --> 00:09:44,060 so that they can keep that information as institutional 153 00:09:44,060 --> 00:09:47,700 knowledge. Really being able to then take that knowledge and 154 00:09:47,700 --> 00:09:51,300 then kind of re injected into their broader agentic 155 00:09:51,300 --> 00:09:54,980 engineering. And I think that's actually the, you know, I don't think 156 00:09:54,980 --> 00:09:58,660 IBM Bob is actually really meant to be a clone of cloud code. I think 157 00:09:58,660 --> 00:10:02,300 it's really meant to be a manager of institutional knowledge across 158 00:10:02,700 --> 00:10:06,500 many different. Yeah, so we have a 159 00:10:06,500 --> 00:10:09,860 special guest, a second special guest show up. This is 160 00:10:09,860 --> 00:10:13,450 Crystal, my little dachshund pup. And I had a pick her up from 161 00:10:13,450 --> 00:10:17,250 chewing wires, but I was listening. But 162 00:10:17,250 --> 00:10:18,930 you're right though, there is definitely a 163 00:10:21,730 --> 00:10:24,450 Bob feels different. I don't know how to describe it. 164 00:10:26,050 --> 00:10:29,890 I had issues getting authenticated into it but the folks at 165 00:10:29,890 --> 00:10:33,410 the Bob booth, at the my IBM booth did help with that. But 166 00:10:33,810 --> 00:10:37,530 it's unfortunately named honestly I think because I think 167 00:10:37,530 --> 00:10:41,260 of Microsoft Bob and that was not 168 00:10:41,260 --> 00:10:44,620 exactly a winning product. Right. But, 169 00:10:45,980 --> 00:10:48,340 but I, I've been playing around with it and I, you know, I had to 170 00:10:48,340 --> 00:10:51,860 do kind of the init process on, on, on a couple 171 00:10:51,860 --> 00:10:55,580 projects and was interesting because it suggested how to take those 172 00:10:55,580 --> 00:10:59,420 projects and turn them into MCP servers and agents, 173 00:11:00,140 --> 00:11:03,020 which the other ones Codex and Claude 174 00:11:03,900 --> 00:11:07,100 have not. I thought that was interesting and I didn't prompt it to do that. 175 00:11:07,100 --> 00:11:10,830 It just basically said on its own like, you know, you could turn 176 00:11:10,830 --> 00:11:14,430 this process into an agent MCP server and things like that. 177 00:11:14,590 --> 00:11:17,750 While that was in the back of my mind as I, you know, built these 178 00:11:17,750 --> 00:11:21,430 various projects, it was not top of mind. So I thought that was interesting. 179 00:11:21,430 --> 00:11:25,150 Yeah, it definitely is not a clone. It's. It's meant to solve a new 180 00:11:25,229 --> 00:11:28,590 problem. Yes, 181 00:11:28,990 --> 00:11:32,790 I agree, I agree. And I think it really 182 00:11:32,790 --> 00:11:36,430 starts feeding into this bigger scope of things like 183 00:11:36,750 --> 00:11:40,550 sports perspective and development. Right. And these other tools 184 00:11:40,550 --> 00:11:43,630 of like how do we get the knowledge out of the project managers, how do 185 00:11:43,630 --> 00:11:47,150 we get it out of the JIRA and how do we get it 186 00:11:47,150 --> 00:11:50,670 into a way that the AI can 187 00:11:50,670 --> 00:11:54,430 interpret it but not lose that knowledge along the way? 188 00:11:55,070 --> 00:11:58,910 So as the prompts are coming in, as the context are coming in, it then 189 00:11:58,910 --> 00:12:02,150 comes part of that institutional knowledge. And I think that is 190 00:12:02,150 --> 00:12:05,740 ultimately what Bob is trying to achieve. That is very 191 00:12:05,740 --> 00:12:09,580 different than I think, what a lot of the other alternatives out there are. 192 00:12:10,220 --> 00:12:14,020 My hope is that as this grows, we see more opportunities for 193 00:12:14,020 --> 00:12:17,860 it to become more open source. That's probably one area 194 00:12:17,860 --> 00:12:20,780 where it's a little different than what we do here at Red Hat, where I 195 00:12:20,780 --> 00:12:24,460 think we, I mean, we're not supporting the project, 196 00:12:24,540 --> 00:12:28,260 but a project that we're very closely buying is things like 197 00:12:28,260 --> 00:12:31,730 open code, for example, which is an open source alternative 198 00:12:31,810 --> 00:12:35,530 to cloud code. It's really interesting to see all these 199 00:12:35,530 --> 00:12:39,370 different solutions right now. I also like the fact that Bob can be 200 00:12:39,370 --> 00:12:43,170 an IDE and mimic more cursor, or it 201 00:12:43,170 --> 00:12:46,930 can be a CLI and mimic more of a cloud code, which obviously 202 00:12:47,010 --> 00:12:50,130 with my background I'm more comfortable with the CLI side 203 00:12:51,650 --> 00:12:55,450 now. That was a big one. And I would say then obviously agents would 204 00:12:55,450 --> 00:12:59,210 be the second biggest thing. Just in general, that was the theme last 205 00:12:59,210 --> 00:13:02,970 year at IBM Think. And that didn't change this year. I 206 00:13:02,970 --> 00:13:06,730 think we're just seeing the experimentation of 207 00:13:06,730 --> 00:13:09,610 agents now, moving into the 208 00:13:09,770 --> 00:13:12,410 solidification of agents in the industry. 209 00:13:13,930 --> 00:13:17,610 And I think we heard about agents a little bit at a high level 210 00:13:17,850 --> 00:13:21,330 at IBM Think. But then for Summit, everything was about 211 00:13:21,330 --> 00:13:24,890 agents. Everything went down to 212 00:13:25,050 --> 00:13:28,810 how does this implement to the agent, how does the inference of AI 213 00:13:28,810 --> 00:13:32,490 implement to the agent, how does the data implement 214 00:13:32,490 --> 00:13:35,970 to the agent? You know, the orchestration layer, 215 00:13:36,130 --> 00:13:39,890 kubernetes, all these things. It all had to do with the agent. 216 00:13:39,970 --> 00:13:43,650 And that was really interesting to see how the conversation 217 00:13:43,730 --> 00:13:47,490 over the last two years has shifted from all 218 00:13:47,490 --> 00:13:51,290 of these individual parts. I think the last time I was on, on your show 219 00:13:51,290 --> 00:13:54,800 and I know you and I have talked a lot about, about how AI. There's 220 00:13:54,800 --> 00:13:58,080 been a lot of these parts, but nothing has kind of unified them. 221 00:13:58,560 --> 00:14:02,400 I think what we're seeing with AI agents is going 222 00:14:02,400 --> 00:14:06,200 to be that unification. The agent will become the unification part of all these 223 00:14:06,200 --> 00:14:09,920 different parts of the AI industry where all these tools now will come 224 00:14:09,920 --> 00:14:13,680 together. And we saw a lot of that at Red 225 00:14:13,680 --> 00:14:17,320 Hat Summit. You don't think that'll be. Harnesses will ultimately 226 00:14:17,320 --> 00:14:21,050 be the container for that, where all these things will live and harnesses will be 227 00:14:21,050 --> 00:14:24,690 kind of like top level abstraction. This is a really good 228 00:14:24,690 --> 00:14:28,410 question because this is the big debate within the AI labs 229 00:14:28,970 --> 00:14:32,730 and the AI community, are you invested in 230 00:14:32,730 --> 00:14:36,530 harness engineering or do you think the models 231 00:14:36,530 --> 00:14:40,090 themselves will just supersede 232 00:14:40,250 --> 00:14:43,890 the harness and that they can be knowledgeable 233 00:14:43,890 --> 00:14:47,010 enough to basically function agentically without. 234 00:14:48,850 --> 00:14:52,250 So obviously the open AIs and the 235 00:14:52,250 --> 00:14:55,930 clouds of the world and anthropic. They're probably a little bit 236 00:14:55,930 --> 00:14:59,650 more on the model side because that would ultimately benefit them. 237 00:14:59,810 --> 00:15:03,050 Right where I think the IBMs and the 238 00:15:03,050 --> 00:15:06,690 Nvidias and I would say the majority of the industry 239 00:15:07,090 --> 00:15:10,530 is probably a little bit more on the harness side because that allows 240 00:15:11,430 --> 00:15:15,270 a larger ecosystem of third party tools and something 241 00:15:15,270 --> 00:15:19,110 that's a little bit more familiar to people. I don't know. 242 00:15:19,510 --> 00:15:22,790 I think over the next year or two it'll definitely be harnessed because that's where 243 00:15:22,790 --> 00:15:26,350 we've seen the most advancement. But with things like mixture of 244 00:15:26,350 --> 00:15:29,510 expert models just continuing to advance and how 245 00:15:29,990 --> 00:15:32,950 they can do reasoning and they can do a lot of agentic. 246 00:15:33,590 --> 00:15:37,110 It could be that we see the model layer 247 00:15:37,430 --> 00:15:40,430 chip away at the harness layer and is this going to be a back and 248 00:15:40,430 --> 00:15:44,030 forth and it really just gets also into 249 00:15:44,030 --> 00:15:47,830 how do you inject the context. And this is closely 250 00:15:47,830 --> 00:15:50,870 related to the same argument of is RAG still needed? 251 00:15:52,150 --> 00:15:55,950 With context size growing so much, why would you need rag? And 252 00:15:55,950 --> 00:15:59,670 I think from an enterprise standpoint, and I think Red Hat is 253 00:15:59,670 --> 00:16:03,390 very big on the harness side because we see the 254 00:16:03,390 --> 00:16:06,950 need for different security layers, different different integrations into third party 255 00:16:06,950 --> 00:16:09,630 tools, different 256 00:16:10,270 --> 00:16:13,390 authorization layers, routing, 257 00:16:14,030 --> 00:16:17,830 networking that the model 258 00:16:17,830 --> 00:16:21,589 will not be able to manage completely, at least for 259 00:16:21,589 --> 00:16:25,190 a while. And that's where I think the harness engineering layer will 260 00:16:25,190 --> 00:16:28,590 exist because there are all these existing technologies 261 00:16:29,230 --> 00:16:32,970 that the agent needs to integrate with and that's all going to 262 00:16:32,970 --> 00:16:36,370 happen at that harness layer and then be 263 00:16:36,370 --> 00:16:40,210 executed within that runtime layer. Yeah, that's how I see it 264 00:16:40,210 --> 00:16:42,890 too. I think the harness layer is really going to be. 265 00:16:45,930 --> 00:16:49,730 It may not be a foundational type situation where you build on 266 00:16:49,730 --> 00:16:52,730 top of it. I see it more as the mortar between the bricks. 267 00:16:53,610 --> 00:16:57,410 I agree. Right. Like, and it's not 268 00:16:57,410 --> 00:17:01,170 that the mortar is more important than the bricks, but 269 00:17:01,170 --> 00:17:04,170 the bricks are kind of pile of rubble 270 00:17:04,970 --> 00:17:08,090 unless you have mortar kind of holding in place. That's kind of how I see 271 00:17:08,090 --> 00:17:10,250 the harness story evolving. 272 00:17:12,010 --> 00:17:15,370 But I have a hard, I, I have a hard time 273 00:17:15,370 --> 00:17:19,129 imagining models ever being able to be that far advanced. 274 00:17:19,130 --> 00:17:22,850 However, you know, we've gotten 275 00:17:22,850 --> 00:17:26,090 further with the LLM architecture than I ever thought we would. 276 00:17:27,050 --> 00:17:30,780 Synthetic data has been more. And 277 00:17:30,780 --> 00:17:34,260 distillation has worked better than I ever thought it would. So Take. 278 00:17:34,420 --> 00:17:38,260 Take my thoughts with that in mind. Right. You know, 279 00:17:38,260 --> 00:17:41,060 when. When I looked at synthetic data and kind of 280 00:17:42,740 --> 00:17:46,099 distillation in particular. Right. There's a meme where they show, 281 00:17:46,900 --> 00:17:50,700 you know, somebody fishing in the. In the water, and then somebody is 282 00:17:50,700 --> 00:17:54,380 fishing from that guy's pot, and then from. Somebody was fishing from that 283 00:17:54,380 --> 00:17:57,850 guy's pail. Right. And then they show each subsequent 284 00:17:57,850 --> 00:18:01,410 fisherman was like, more and more distorted. We've not really seen 285 00:18:01,410 --> 00:18:05,250 that come about. Right. It's not like you're copying VHS 286 00:18:05,250 --> 00:18:08,730 tapes where subsequent generation gets 287 00:18:08,730 --> 00:18:12,570 worse. I'm sure that if you don't do it carefully, you'll 288 00:18:12,570 --> 00:18:15,970 get some weird artifacts. But it's not been. 289 00:18:16,770 --> 00:18:20,570 That has not been a default case, which I think is interesting. It 290 00:18:20,570 --> 00:18:24,150 is interesting too, because most of the models that are out 291 00:18:24,150 --> 00:18:26,230 right now are 292 00:18:27,190 --> 00:18:30,870 distillations of actually GPT4 family. Right. 293 00:18:30,950 --> 00:18:34,230 Even the GPT5 is still a direct 294 00:18:34,470 --> 00:18:38,310 distillation of 4. It was not completely retrained. 295 00:18:39,350 --> 00:18:42,990 And Anthropic obviously has their first generations and second 296 00:18:42,990 --> 00:18:46,550 generations, but we actually haven't seen very much 297 00:18:47,190 --> 00:18:50,830 new generation just because how expensive it is to create 298 00:18:50,830 --> 00:18:54,550 from. From fresh. And from. What I'm imagining is that they've 299 00:18:54,550 --> 00:18:57,790 tried and it's just they haven't gotten the results that they wanted. 300 00:18:58,990 --> 00:19:02,390 So I think that will be what we see. I don't know. I haven't heard 301 00:19:02,390 --> 00:19:05,870 if Mythos is a. So if people aren't following the. With Mythos 302 00:19:05,870 --> 00:19:08,270 model from Anthropic, it's a. 303 00:19:09,790 --> 00:19:13,310 It's a model that they've withheld because supposedly it's too 304 00:19:13,310 --> 00:19:16,670 risky. I don't know if that model is. 305 00:19:17,400 --> 00:19:21,240 Is a whole new generation. I would imagine that it probably 306 00:19:21,240 --> 00:19:24,960 is. But to your point, most of the models are out there now, 307 00:19:24,960 --> 00:19:28,360 and what we know from the Chinese models, that they're all just distillations of 308 00:19:28,600 --> 00:19:32,280 the American models. We have proof now that they've been 309 00:19:32,920 --> 00:19:34,840 mass API, hitting the 310 00:19:36,360 --> 00:19:40,160 GPT and Anthropic and Gemini to 311 00:19:40,160 --> 00:19:43,320 create the generation of Chinese models that we have now. 312 00:19:43,960 --> 00:19:47,800 So that's something. And they get. They're very performant. Like, those models are very, extremely 313 00:19:48,040 --> 00:19:51,760 good. Very good. I mean, it just shows you like this. This is 314 00:19:51,760 --> 00:19:55,520 not the paradigm of, you know, analog VHS copying. Right. This 315 00:19:55,520 --> 00:19:58,520 is more. More, I guess, in the style of, you know, 316 00:19:58,920 --> 00:20:02,640 remixing an old song digitally. Right. You 317 00:20:02,640 --> 00:20:06,400 don't really get it's. It's not 318 00:20:06,400 --> 00:20:10,240 a well thought out analogy, Christopher, but. But, you know, you'll hear 319 00:20:10,240 --> 00:20:13,480 like, you know, a lot of techno songs in the early 2000s you will hear 320 00:20:13,480 --> 00:20:17,230 them on I don't go to Clubs Anymore, but on my, on my what's 321 00:20:17,230 --> 00:20:20,390 New and what's Hot techno playlists on Spotify. 322 00:20:20,790 --> 00:20:24,350 Right. I, I recognize the same backbeat, I recognize the same 323 00:20:24,350 --> 00:20:27,910 chorus, right. Like from songs from like 20, 30 years ago, right. Like, 324 00:20:29,110 --> 00:20:32,870 and even sampling and rap music, right. Like, it's a bit more like that 325 00:20:32,870 --> 00:20:36,470 where you do get a completely fresh perspective based on older parts. 326 00:20:37,510 --> 00:20:41,150 And that's something that I did not expect. I, I just assumed that it would 327 00:20:41,150 --> 00:20:44,860 be some kind of. You would start getting really bizarre artifacts after 328 00:20:45,260 --> 00:20:48,460 so many generations. But that's not been the case. So, 329 00:20:49,660 --> 00:20:53,300 you know, I think it's interesting because we really don't. This is really uncharted 330 00:20:53,300 --> 00:20:57,140 territory, right? These are. Yes, they're based on very well known mathematical 331 00:20:57,140 --> 00:21:00,060 principles. But like, as these systems get more complex, 332 00:21:01,340 --> 00:21:05,060 it's getting harder and harder to predict not just their behavior, but 333 00:21:05,060 --> 00:21:08,910 the range of their behaviors. Yep. One second. I'm going to grab 334 00:21:08,910 --> 00:21:11,670 something because we'll do a little bit of show and tell as well. Cool, cool, 335 00:21:11,670 --> 00:21:14,630 cool. So while you're away, I will 336 00:21:15,910 --> 00:21:19,750 maybe I can interview a dachshund. So what is, what do dogs think about AI? 337 00:21:21,190 --> 00:21:24,950 Everybody and their cousin and their dog has a AI startup now. So what's your 338 00:21:24,950 --> 00:21:28,150 AI startup? Oh, a link shortener. 339 00:21:28,230 --> 00:21:31,750 Okay, cool. Because I get it. Your short 340 00:21:31,750 --> 00:21:33,590 legs. I get it. That's cool. 341 00:21:35,770 --> 00:21:39,530 While we're waiting for Christopher to come back, you all know I'm a big 342 00:21:39,530 --> 00:21:42,970 fan of Humble Bundle, so. Humble 343 00:21:42,970 --> 00:21:46,690 Bundle. Oh, you're back. Cool. Oh, you can finish your thought. 344 00:21:46,690 --> 00:21:50,290 Humble oh, so Humble Bundle. I actually, so I 345 00:21:50,290 --> 00:21:54,130 worked the booth. I had a, A talk on day one and I worked 346 00:21:54,130 --> 00:21:57,130 the booth on the subsequent days and 347 00:21:58,490 --> 00:22:01,770 you know, a lot of people came by and 348 00:22:02,170 --> 00:22:05,460 other Red Hatters. Actually, I, I was showing them 349 00:22:05,780 --> 00:22:09,620 Humble Bundle. I'm sorry, go ahead. 350 00:22:09,860 --> 00:22:13,620 No, I just said that looks really cool. Yeah, yeah. So if 351 00:22:13,620 --> 00:22:17,060 you're not Familiar with it, humblebundle.com it started as games, 352 00:22:17,940 --> 00:22:21,780 but if you go and you pick store. 353 00:22:22,980 --> 00:22:25,460 Not store, I'm sorry, bundle, 354 00:22:27,380 --> 00:22:31,180 you can pick books and there's comic books there. But there's 355 00:22:31,180 --> 00:22:34,860 also a lot of stuff here that is particular around 356 00:22:35,260 --> 00:22:38,620 software. Right. So in this example here, 357 00:22:38,940 --> 00:22:42,620 this is the books on practice 358 00:22:42,620 --> 00:22:45,500 exams for AWS and gen AI, 359 00:22:46,460 --> 00:22:48,380 all sorts of interesting stuff here. Security. 360 00:22:50,140 --> 00:22:53,820 This is actually a hybrid of like courseware. 361 00:22:54,460 --> 00:22:57,660 So they also have software oops. 362 00:22:58,950 --> 00:23:02,790 Bundles that are, you know, sometimes it's kind of like image editing 363 00:23:02,790 --> 00:23:06,150 tools and things like that. But very often 364 00:23:06,470 --> 00:23:10,270 they Will have courses for, you know, how to get into Open Claw and things 365 00:23:10,270 --> 00:23:14,030 like that. And I know if you don't know, Christopher is really into openclaw, 366 00:23:14,030 --> 00:23:17,790 he helped me get my Claudia kind of up and working. 367 00:23:17,790 --> 00:23:21,590 But if you go here, I know a lot of listeners Data Driven are 368 00:23:21,670 --> 00:23:25,430 big into Power bi. These are basically courses on Power Bi 369 00:23:25,430 --> 00:23:28,920 and things like that. And the cool thing is it's 370 00:23:28,920 --> 00:23:32,720 $20 for 17 courses and a 371 00:23:32,720 --> 00:23:36,040 portion of your cost goes to a charity. 372 00:23:37,000 --> 00:23:40,640 So it's really cool. You get a lot of material and you know, a 373 00:23:40,640 --> 00:23:44,479 charity gets funded and things like that. So definitely 374 00:23:44,479 --> 00:23:48,280 check it out. They often have AI books or, 375 00:23:48,280 --> 00:23:50,680 you know, app development books. A lot of things around 376 00:23:52,920 --> 00:23:56,080 game development too because that's kind of where Humble Bundles started. 377 00:23:56,640 --> 00:24:00,320 Nice. That's a great segue too because speaking of 378 00:24:00,400 --> 00:24:03,680 openclaw, so when I got home from 379 00:24:05,280 --> 00:24:08,960 Red Hat Summit, this arrived. Oh, nice. 380 00:24:10,319 --> 00:24:13,960 So I haven't gotten a Mac Mini in 381 00:24:13,960 --> 00:24:17,720 probably like over 10 plus years. So when this came I was 382 00:24:17,720 --> 00:24:21,430 kind of like. Because I don't know if you remember, the Mac Mini 383 00:24:21,430 --> 00:24:25,230 was maybe about the same size, but it was much bigger than this. 384 00:24:25,550 --> 00:24:28,350 And I'm actually holding this with like one hand right now. 385 00:24:29,310 --> 00:24:31,390 But the reason why I got this is because 386 00:24:33,150 --> 00:24:36,750 Red Hat in particular wants to make sure that all of 387 00:24:36,830 --> 00:24:40,510 the agents that I'm running for Red Hat are isolated 388 00:24:40,510 --> 00:24:43,870 for runtime. So I could use 389 00:24:44,510 --> 00:24:47,640 my. Let me see if I can pull it over. You have one of those 390 00:24:47,640 --> 00:24:51,280 framework things, right? This is the framework. Yeah. This is the 391 00:24:51,280 --> 00:24:53,640 size of it. So that is actually 392 00:24:54,600 --> 00:24:58,240 powering my home Lab that has OpenShift in it. I could 393 00:24:58,240 --> 00:25:01,480 do that. And that's actually where a lot of our tooling are going 394 00:25:01,560 --> 00:25:05,320 towards. But I also need a 395 00:25:06,440 --> 00:25:10,240 agent to have access to my email, have access to like more 396 00:25:10,240 --> 00:25:13,750 like my day to day tooling which actually exists more. More on a 397 00:25:13,750 --> 00:25:17,310 desktop. And that's where this guy comes into play. 398 00:25:18,190 --> 00:25:21,950 It's interesting. Now we're separating 399 00:25:21,950 --> 00:25:25,750 the harness from its runtime and now I'm dealing with 400 00:25:25,750 --> 00:25:29,070 multiple runtimes. I'm going to have runtimes that probably run on the home lab 401 00:25:29,550 --> 00:25:33,070 and now we have runtimes that are going to run on this. 402 00:25:34,350 --> 00:25:37,070 This one would be. I need it to do something that actually 403 00:25:38,110 --> 00:25:41,640 involves some kind of GUI or something that's already on my 404 00:25:41,640 --> 00:25:45,120 desktop, which just there's no API set up for me to do. 405 00:25:45,760 --> 00:25:49,400 Or I need it to do something, maybe some, something basic that's really easy to 406 00:25:49,400 --> 00:25:53,200 do within the Mac ecosystem where this 407 00:25:53,360 --> 00:25:56,880 like my home Lab, maybe it's an agent that's running diagnostics 408 00:25:56,960 --> 00:26:00,640 on like AI ops diagnostics on my 409 00:26:00,640 --> 00:26:04,480 home lab. Why isn't something up? Why isn't it working correctly? 410 00:26:05,280 --> 00:26:08,170 And this is where the whole concept of 411 00:26:09,130 --> 00:26:12,330 runtime now has become such a big thing. And I think it will continue to 412 00:26:12,330 --> 00:26:15,890 become more important this year. Harness is kind of getting the 413 00:26:15,890 --> 00:26:19,210 spotlight, but we need to move more into this runtime conversation of 414 00:26:19,450 --> 00:26:22,810 okay, now the harness has put the context together, it's put all the knowledge together 415 00:26:22,810 --> 00:26:26,530 and the skills. The agent is running the agentic loop with 416 00:26:26,530 --> 00:26:29,610 the model. But now where does the output actually run? 417 00:26:30,330 --> 00:26:34,070 Does it run on your, your personal computer 418 00:26:34,070 --> 00:26:37,870 where it has access to sensitive information and you know, it 419 00:26:37,870 --> 00:26:41,550 could do things that it shouldn't or does it run in an 420 00:26:41,550 --> 00:26:45,030 isolated environment? So this is probably going to act more as like a little server 421 00:26:45,030 --> 00:26:48,590 that runs here in my office where 422 00:26:48,590 --> 00:26:52,310 agents, this is just for agents. This box. Where is the inference run? 423 00:26:52,390 --> 00:26:55,750 Where does the inference run for your agents? Is it run? Yeah. 424 00:26:56,070 --> 00:26:59,430 Does it run on that? Does it run on your framework or does it run 425 00:26:59,430 --> 00:27:03,050 in a hypercloud service? So I'm actually 426 00:27:03,050 --> 00:27:05,850 doing a new technique called semantic routing. 427 00:27:07,210 --> 00:27:10,850 All my requests go to my home lab first. Within what we would call the 428 00:27:10,850 --> 00:27:14,610 control plane for the agent, there's a router that 429 00:27:14,610 --> 00:27:18,130 exists that actually evaluates the information that's coming in and 430 00:27:18,130 --> 00:27:21,770 decides based off of sensitivity and complexity where 431 00:27:21,770 --> 00:27:25,530 this route should go. About 80% of my traffic actually hits 432 00:27:25,530 --> 00:27:29,300 the framework for a model that's running within Vllm 433 00:27:29,460 --> 00:27:32,740 on the framework device itself on 434 00:27:33,140 --> 00:27:36,980 OpenShift and then about 20% where I've deemed kind of high 435 00:27:36,980 --> 00:27:40,620 reasoning. Then we'll get sent off 436 00:27:40,620 --> 00:27:44,180 to our corporate Gemini account that we have within Red 437 00:27:44,180 --> 00:27:47,860 Hat. So this way it's also really 438 00:27:47,860 --> 00:27:51,460 nice because when I first started working with agents all the way back, I mean 439 00:27:51,460 --> 00:27:54,930 I've been working with agents for, for years and years. But our current 440 00:27:54,930 --> 00:27:58,770 modern day idea of what agents look like, back at 441 00:27:58,770 --> 00:28:02,530 the beginning of this year I was running out of 442 00:28:02,530 --> 00:28:06,210 tokens. I was getting throttled by, by Google and there was 443 00:28:06,210 --> 00:28:08,809 nothing I could do about that because that was part of our corporate account. It 444 00:28:08,809 --> 00:28:12,650 wasn't anything to do that I could go and change the knobs. So moving to 445 00:28:12,650 --> 00:28:16,370 this semantic routing approach allowed me to not 446 00:28:16,370 --> 00:28:20,000 run into that throttling anymore. Most of my things go. So right now I'm running 447 00:28:20,000 --> 00:28:23,038 a quinn, the quinn 3.6 448 00:28:23,402 --> 00:28:27,200 35B mixture of experts model. Nice. 449 00:28:27,200 --> 00:28:31,040 And that's running right now and doing all of my local agentic work. 450 00:28:31,040 --> 00:28:34,840 It's doing most of the low reasoning tasks and then all the high 451 00:28:34,840 --> 00:28:37,880 reasoning tasks then get sent off to Gemini. 452 00:28:38,760 --> 00:28:41,920 So do you ever have it set up where the high reasoning task will divvy 453 00:28:41,920 --> 00:28:45,640 up a bunch of low reasoning tasks and then send that down to your Quinn? 454 00:28:46,830 --> 00:28:50,350 Or is that something in the works? I have 455 00:28:50,990 --> 00:28:53,630 experimented some with that. So 456 00:28:54,670 --> 00:28:58,270 that gets into some like post inference type of techniques that 457 00:28:58,430 --> 00:29:01,950 we've been experimenting with, myself included. 458 00:29:02,350 --> 00:29:05,630 I haven't gotten that far yet. This is where 459 00:29:06,350 --> 00:29:10,030 areas such as like speculative decoding kind of come into play or 460 00:29:11,150 --> 00:29:14,900 post inference technique. Why would speculative 461 00:29:14,980 --> 00:29:17,300 decoding come into play here? 462 00:29:18,740 --> 00:29:22,100 Yeah, because there could be a speculator that sits 463 00:29:22,820 --> 00:29:26,500 at the local model that actually 464 00:29:26,900 --> 00:29:30,580 acts as kind of almost like a guardrail to the 465 00:29:32,100 --> 00:29:35,700 larger model where it can actually start reasoning about 466 00:29:35,860 --> 00:29:38,710 some of the things earlier on and decide 467 00:29:39,910 --> 00:29:43,750 basically acts as a breaker. I got you. And that makes sense. 468 00:29:43,750 --> 00:29:47,430 That's where speculative decoding would be kind of the 469 00:29:47,430 --> 00:29:50,390 next iteration on that where 470 00:29:51,190 --> 00:29:54,950 it's really the management of knowledge and memory and cache at that 471 00:29:54,950 --> 00:29:58,670 point. I really haven't gotten into that with my local setup, but that's 472 00:29:58,670 --> 00:30:02,350 part of that whole last mile where memory I 473 00:30:02,350 --> 00:30:06,200 think will be the last portion of the last mile for 474 00:30:06,200 --> 00:30:09,680 everybody. It's going to be memory management, it's going to be cache management. 475 00:30:10,720 --> 00:30:14,480 When you say memory, organizational memory, not necessarily the physical memory. 476 00:30:15,120 --> 00:30:18,920 When I'm talking about memory, I'm talking about the memory of 477 00:30:18,920 --> 00:30:21,840 the agent itself. For 478 00:30:22,240 --> 00:30:25,600 OpenClaw, for example, every time it makes decisions, 479 00:30:25,920 --> 00:30:29,640 it keeps a compressed record of what it's done in these 480 00:30:29,640 --> 00:30:32,700 JSON files and then it will reference that 481 00:30:35,100 --> 00:30:38,700 your cloud code does something very similar. Every time you hit your token 482 00:30:39,420 --> 00:30:42,820 context window maximum, you'll see that it's doing a bunch of 483 00:30:42,820 --> 00:30:46,500 compressions and it takes a little thought. That's actually what 484 00:30:46,500 --> 00:30:49,980 we call a form of memory. If you've actually been following the news. 485 00:30:50,140 --> 00:30:51,020 Even just today, 486 00:30:53,820 --> 00:30:56,930 Google IE just announced a whole new 487 00:30:56,930 --> 00:31:00,650 agentic memory platform, a 488 00:31:00,650 --> 00:31:04,410 framework that fits right into this. And that's why I think memory 489 00:31:04,410 --> 00:31:08,210 is going to be the next iteration on. On, 490 00:31:08,370 --> 00:31:12,050 you know, improving the agentic system. And that's not the KV cache, 491 00:31:12,050 --> 00:31:15,490 that's not your physical memory. It's not the 492 00:31:15,650 --> 00:31:19,250 agentic memory would be a. Yeah, it's like a gentic memory. It's how your 493 00:31:19,250 --> 00:31:22,570 agent has recogn 494 00:31:22,570 --> 00:31:26,360 reconciling what it's doing doing and has. It's. It's 495 00:31:26,360 --> 00:31:29,840 outside of the context window, but it's not the KV 496 00:31:29,840 --> 00:31:33,480 cache. It's something that's like, oh, this is what I've done in the 497 00:31:33,480 --> 00:31:37,240 past and this is the context I need that I just 498 00:31:37,240 --> 00:31:40,880 need to keep carrying forward in my conversations. 499 00:31:41,920 --> 00:31:45,560 It's something that maybe it's not an MD 500 00:31:45,560 --> 00:31:48,960 file, it's not like permanent knowledge. It could get flushed. You could just say 501 00:31:49,530 --> 00:31:51,450 go ahead and flush your, your memory 502 00:31:53,450 --> 00:31:56,490 and that may actually be what you need to do because maybe it's. There's a 503 00:31:56,490 --> 00:32:00,210 lot of nonsense in there or something that's doing something wrong. It's not 504 00:32:00,210 --> 00:32:03,330 meant to be long term. Think of it like human short term memory. Exactly what 505 00:32:03,330 --> 00:32:07,130 it is. Interesting. Not everything that we do is long term. 506 00:32:07,130 --> 00:32:10,890 So long term memory in this case would be the, your 507 00:32:10,890 --> 00:32:14,090 MD files, it would be your kv, 508 00:32:14,250 --> 00:32:18,090 potentially even like some layers of your KV cache where I 509 00:32:18,090 --> 00:32:21,930 would actually consider that more like intermediate. But it's really that 510 00:32:21,930 --> 00:32:25,450 long lasting context that just keeps getting injected in where 511 00:32:25,690 --> 00:32:28,970 this concept of, of memory that we keep hearing about 512 00:32:29,210 --> 00:32:32,890 is more of that short term memory of what knowledge 513 00:32:32,890 --> 00:32:36,290 do you need to have right now to make the decisions that you need to 514 00:32:36,290 --> 00:32:40,090 make based off of the reasoning and the topics that you're working with 515 00:32:40,090 --> 00:32:43,930 right now? So a good example would be. I'm sorry, go ahead. No, no, 516 00:32:43,930 --> 00:32:47,090 no, no, go ahead. No, good, good example. You like your hotel room number when 517 00:32:47,090 --> 00:32:50,610 you go to a conference, right. Like you're never going to cancel, you're having, you 518 00:32:50,610 --> 00:32:54,290 know, needing to remember that beyond once you check out is very low. 519 00:32:54,450 --> 00:32:58,170 Or when you get the two factor authentication, the six digit code. Right. You only 520 00:32:58,170 --> 00:33:00,930 need to remember that for a very short window of time. 521 00:33:03,490 --> 00:33:07,130 Yes, exactly. And that's a prime example where 522 00:33:07,130 --> 00:33:10,860 you could long term forget that information. But in the 523 00:33:10,860 --> 00:33:14,260 short term it would be very detrimental if you forget your hotel room, you have 524 00:33:14,260 --> 00:33:18,020 to go and ask somebody and that, that takes time. Yeah, it takes time and 525 00:33:18,020 --> 00:33:21,460 that's exactly the same narrative. It's not that the agent couldn't get that information, 526 00:33:21,620 --> 00:33:25,420 it's just that it's faster for the agent to get that information if it's 527 00:33:25,420 --> 00:33:29,180 located in some type of short term memory. And that's 528 00:33:29,180 --> 00:33:32,580 where we're seeing so much advancement in, in these 529 00:33:32,580 --> 00:33:36,380 agentic platforms. Did you want to, did you want to add 530 00:33:36,380 --> 00:33:38,940 anything to that? I know we're coming up to time, so I just. Oh no, 531 00:33:38,940 --> 00:33:42,440 I mean, no, I appreciate your time. I see the, I see that we' up 532 00:33:42,440 --> 00:33:46,160 on time and you know the. 533 00:33:46,960 --> 00:33:50,480 No, I think there's a lot. I think, I think the one thing I learned 534 00:33:50,560 --> 00:33:54,320 this week was it's very easy to think that you're behind 535 00:33:54,320 --> 00:33:58,159 everyone else. But you know, we've had people, we had people come in the booth, 536 00:33:58,159 --> 00:34:01,520 like, I don't know anything about this, tell me where to get started. And I 537 00:34:01,520 --> 00:34:05,280 was like, you know, to hear that in 2026 was 538 00:34:05,280 --> 00:34:09,080 both shocking and, and 539 00:34:09,080 --> 00:34:12,760 refreshing. Right. You know, there were, there were people. I'm not going to name 540 00:34:12,760 --> 00:34:16,480 any names, but like, you know, there are people who are in our 541 00:34:16,480 --> 00:34:19,240 division and they've not even installed 542 00:34:19,720 --> 00:34:23,520 Claude yet. Open Claw. I mean, I 543 00:34:23,520 --> 00:34:27,160 always get those two confused, even though I know they're very different things. 544 00:34:27,320 --> 00:34:31,000 But, you know, who've not installed Open Claw, like 545 00:34:31,000 --> 00:34:34,680 on their own? And it's just like I feel behind because I 546 00:34:34,680 --> 00:34:37,859 have Open Claw, but I don't have it as set up. Well, set up as 547 00:34:37,859 --> 00:34:41,699 you. Right. But I do have it, you know, so it's kind of 548 00:34:41,699 --> 00:34:45,379 like it's, it's, it's, you know, don't be afraid of being behind because 549 00:34:45,379 --> 00:34:49,179 chances are you're probably not. No, no. Part of the reason why 550 00:34:49,179 --> 00:34:52,499 I have the dog do that intro now, which of course is, you know, obviously 551 00:34:52,499 --> 00:34:56,099 AI generated was part of the joke of that was everybody on their dog is 552 00:34:56,099 --> 00:34:59,939 an AI expert now. And there's not 553 00:34:59,939 --> 00:35:03,420 really any experts. There's probably about half a dozen people 554 00:35:03,420 --> 00:35:06,940 worldwide that really are on a whole other level. 555 00:35:07,260 --> 00:35:10,700 I mean, the Andrew Angs. The Andrew Angs of the world. The 556 00:35:12,940 --> 00:35:16,380 Jeffrey Hinton's of the world. Right. Like those are the people. 557 00:35:16,860 --> 00:35:20,540 Yan Lecun for sure. You don't 558 00:35:20,540 --> 00:35:24,380 hear much from Yahshua Bengio anymore. But you know, 559 00:35:24,380 --> 00:35:28,180 like people at that level, right. At that, that strata, like 560 00:35:28,180 --> 00:35:31,680 they are, they really are like that far ahead. 561 00:35:32,240 --> 00:35:35,760 And it's always interesting seeing like what problems they're trying to solve. 562 00:35:36,000 --> 00:35:39,760 I think is very interesting. What is particularly interesting, I think it was. John Lecun 563 00:35:39,760 --> 00:35:43,360 is very skeptical of LLMs getting any further 564 00:35:43,360 --> 00:35:46,880 along. Yeah. Which I think is 565 00:35:46,880 --> 00:35:50,320 interesting. I mean, it's, you know, at this point almost a 8 or 9 year 566 00:35:50,320 --> 00:35:53,680 old concept of LLM 567 00:35:53,680 --> 00:35:57,250 transformers. The concept that he created. The concept that he created. Right. 568 00:35:57,250 --> 00:36:01,050 So the underlining layers. Yeah, yeah. So like a lot 569 00:36:01,050 --> 00:36:04,650 of. Go ahead. I was gonna say there's a lot of new 570 00:36:04,650 --> 00:36:08,250 interviews that he has out in the last couple weeks about, you know, his 571 00:36:08,250 --> 00:36:11,849 new approach to AI and how he 572 00:36:11,849 --> 00:36:15,570 sees it superseding LLMs. And that'll be interesting 573 00:36:15,570 --> 00:36:19,170 too because he's looking at it from a whole new direction than just 574 00:36:19,570 --> 00:36:23,320 how LLMs just, they're just, 575 00:36:23,560 --> 00:36:27,200 they're just building the next pixel the 576 00:36:27,200 --> 00:36:30,760 next text where he's looking at it from a whole new 577 00:36:30,760 --> 00:36:34,440 direction of, you know, maybe we built this 578 00:36:34,440 --> 00:36:38,120 house of cards wrong. We need to just kind of start over and basically 579 00:36:38,120 --> 00:36:41,920 like, stop, start at the basics and, and build something better from what we've 580 00:36:41,920 --> 00:36:45,600 learned. And it'll be very interesting to see what he comes up with out of 581 00:36:45,600 --> 00:36:49,320 all this. Yeah, I, I, because I, I'm surprised we've gotten 582 00:36:49,320 --> 00:36:52,920 this far this fast with LLMs. I, I 583 00:36:52,920 --> 00:36:56,720 really thought, like, the whole reasoning aspect to LLMs is something I 584 00:36:56,720 --> 00:37:00,040 did not see that they were, I did not, I would not have bet real 585 00:37:00,040 --> 00:37:03,760 money on them being able to do that. Right. But here we are. Like, they 586 00:37:03,760 --> 00:37:07,280 clearly can do some level of reasoning. How much is probably 587 00:37:07,280 --> 00:37:11,040 debatable, but the fact that they're just, you 588 00:37:11,040 --> 00:37:14,800 know, you hear that they're just like text prediction thing algorithms on your 589 00:37:14,800 --> 00:37:18,170 phone where they predict the next word. Well, technically true, 590 00:37:19,770 --> 00:37:23,250 I think doesn't really tell the whole story. Right. Like, that's like saying that the, 591 00:37:23,250 --> 00:37:27,010 the F35 fighter is the same thing as a 592 00:37:27,010 --> 00:37:30,730 paper airplane. Right. Like, they do have to apply, they do have to obey the 593 00:37:30,730 --> 00:37:34,330 same laws of physics, thrust, lift, gravity, blah, blah, blah. 594 00:37:35,450 --> 00:37:38,650 But they are very different animals in that sense. 595 00:37:39,050 --> 00:37:42,890 Very much. I agree with that analogy. It's really good. Cool, man. I love to 596 00:37:42,890 --> 00:37:46,410 have you on the show again. We could talk open claw. Yeah. You've done some 597 00:37:46,410 --> 00:37:50,050 crazy cool stuff with that. Definitely. I know some of the 598 00:37:50,050 --> 00:37:53,330 agents that you've built that people probably don't want me talking about because I know 599 00:37:53,330 --> 00:37:57,050 you made a lot of it. Security people very nervous. 600 00:37:57,850 --> 00:38:01,410 That's true. But the stuff that you've been able to 601 00:38:01,410 --> 00:38:05,050 automate has been nothing short of like, oh, my God, that's amazing. 602 00:38:05,690 --> 00:38:09,450 And also super useful. Crazy too, for me is that I've 603 00:38:09,450 --> 00:38:12,660 been so busy that so much of the stuff that I did that people were 604 00:38:12,660 --> 00:38:16,500 talking about was like one to two months ago 605 00:38:16,900 --> 00:38:20,340 and I think this summer. So there was actually a, 606 00:38:20,500 --> 00:38:24,140 a really popular podcast out of the the 607 00:38:24,140 --> 00:38:26,500 AI Daily Brief that went out where he was talking about how 608 00:38:27,940 --> 00:38:31,420 everything that's happened over the last six months basically came out of 609 00:38:31,420 --> 00:38:35,220 Christmas break. So, like, everyone went home and 610 00:38:35,460 --> 00:38:38,700 had a few weeks to just like, play around with this stuff. I was one 611 00:38:38,700 --> 00:38:41,660 of those people. So, like, so much of what I did came out of those, 612 00:38:41,660 --> 00:38:45,420 like, experimentation phases. And I think I have to repeat 613 00:38:45,420 --> 00:38:48,980 that this summer because there's so much new things that we've learned. 614 00:38:49,220 --> 00:38:53,060 Right. That, But I still haven't built on top of that yet. And I think 615 00:38:53,939 --> 00:38:57,620 so for me right now, so many of my agents are doing very simple tasks. 616 00:38:57,620 --> 00:39:01,380 They're doing information gathering, they might be looking at 617 00:39:01,380 --> 00:39:05,190 meetings and suggesting that I read certain articles 618 00:39:05,190 --> 00:39:08,830 correlating to something I'm about to talk about. But I want to go the 619 00:39:08,830 --> 00:39:12,470 next level where I get into like a multi agent system where 620 00:39:12,470 --> 00:39:16,070 I have like a chief of staff agent who's got one 621 00:39:16,070 --> 00:39:19,430 that's doing programming demos and then I have another one that's doing 622 00:39:19,990 --> 00:39:23,550 like general administrative assistant work or another 623 00:39:23,550 --> 00:39:27,110 one that's front facing, you know, I 624 00:39:27,510 --> 00:39:31,250 a model that's on our slack that people can just ask questions to 625 00:39:31,250 --> 00:39:35,090 based off of my institutional knowledge that I have of, of 626 00:39:35,090 --> 00:39:38,890 our company and our industry. So that's the next phase and that's 627 00:39:38,890 --> 00:39:41,970 where the memory stuff has to come into play and the multi agent kind of 628 00:39:41,970 --> 00:39:45,690 orchestration and all these things are things that are being worked on now. So there's 629 00:39:45,690 --> 00:39:49,530 not like a clear winner or a clear understanding of what's what 630 00:39:49,530 --> 00:39:52,250 that looks like right now. But we're all kind of playing around with it. So 631 00:39:52,250 --> 00:39:55,810 I think that's kind of the next phase. And yeah, I look forward to coming 632 00:39:55,810 --> 00:39:59,480 back. And I think that's probably part two of this conversation will be absolutely. 633 00:39:59,640 --> 00:40:02,280 What does that look like? What are these tools? How do we kind of build 634 00:40:02,280 --> 00:40:05,400 on top of this thing called openclaw or 635 00:40:06,760 --> 00:40:10,520 Hermes or all these other ones that are out these days. Yeah, that'd be 636 00:40:10,520 --> 00:40:14,160 awesome. And even if we just do a deep dive on like kind of what's 637 00:40:14,160 --> 00:40:17,880 exactly, you know, what's what. Because I know you mentioned a couple of things that 638 00:40:18,520 --> 00:40:22,280 maybe most of our listeners don't fully grok, right. Because 639 00:40:22,280 --> 00:40:26,060 we have a lot of data engineers here too. Right. So and the 640 00:40:26,060 --> 00:40:29,540 other thing too that really came out was people would ask me questions about because 641 00:40:29,700 --> 00:40:33,220 we have something that Microsoft folks may know as TFAs or technical focus 642 00:40:33,220 --> 00:40:36,980 areas, call them pillars. So you're the agentic lead, I 643 00:40:36,980 --> 00:40:40,740 believe, and I'm the connecting models to data. Right. 644 00:40:40,740 --> 00:40:44,100 So the rag and that sort of thing and you know, a lot of the 645 00:40:44,100 --> 00:40:47,700 conversations I had was, you know, data engineering is 646 00:40:48,020 --> 00:40:51,790 more important now in AI systems than they were in the past 647 00:40:51,870 --> 00:40:54,910 because I don't know exactly how 648 00:40:55,870 --> 00:40:58,670 rag agentic systems would fail but, 649 00:41:00,030 --> 00:41:02,750 but when they fail, they probably fail very spectacularly. 650 00:41:04,110 --> 00:41:07,830 But I know with, with rag systems, right, you know, if 651 00:41:07,830 --> 00:41:10,750 your data chunking strategy and your data kind of 652 00:41:11,150 --> 00:41:14,990 indexing strategy is not, I wouldn't say perfect because you'll 653 00:41:14,990 --> 00:41:18,830 never really get there, but appropriate to the data source documents that you're dealing with, 654 00:41:19,530 --> 00:41:23,050 you're not gonna. It's gonna fail in a way that is subtle and is only 655 00:41:23,050 --> 00:41:26,690 gonna amplify get worse down the road. Right. So you 656 00:41:26,690 --> 00:41:30,130 really have to think through a lot of these things. Right. The, the one sentence 657 00:41:30,130 --> 00:41:33,850 I said most of all was, you know, chunking 658 00:41:33,850 --> 00:41:37,610 is an architectural decision. Yes. It's 659 00:41:37,610 --> 00:41:41,450 an important one. Treat it with that importance as opposed to just whatever, 660 00:41:41,450 --> 00:41:44,970 you know, paragraph by paragraph or blah, blah, blah, blah, blah. 661 00:41:45,310 --> 00:41:49,030 So that was other consistent theme. But I will say that the, the questions that 662 00:41:49,030 --> 00:41:52,830 I get are far more evolved than I haven't gotten at any 663 00:41:52,830 --> 00:41:56,110 other conferences in a while. I agree. 664 00:41:56,430 --> 00:42:00,190 Especially this year. It's just a step up from where we were. So. Yeah. Cool. 665 00:42:00,190 --> 00:42:04,030 This is great. Thank you for having me on. No problem. We'd love to 666 00:42:04,030 --> 00:42:07,070 have you back. And since the recordings for the podcast, we'll let the music play. 667 00:42:12,620 --> 00:42:12,860 It.