1 00:00:39,417 --> 00:00:41,217 W. Curtis Preston: Hi, and welcome to Backup Central's. 2 00:00:41,217 --> 00:00:42,367 Restore it all podcast. 3 00:00:42,807 --> 00:00:43,657 I'm your host, W. 4 00:00:43,677 --> 00:00:45,567 Curtis Preston, AKA Mr. 5 00:00:45,572 --> 00:00:45,767 Backup. 6 00:00:46,527 --> 00:00:52,007 And I have with me my continuing advisor on my consumer backup project 7 00:00:52,007 --> 00:00:55,077 Prasanna Malaiyandi, how's it going? 8 00:00:55,077 --> 00:00:55,637 Prasanna. 9 00:00:57,267 --> 00:00:57,837 Prasanna Malaiyandi: I'm good. 10 00:00:57,837 --> 00:00:58,227 Curtis. 11 00:00:58,227 --> 00:00:58,707 You know what? 12 00:00:58,737 --> 00:01:02,547 We have the expert, the backup anorak Daniel Rosehill coming 13 00:01:02,547 --> 00:01:05,367 next week on the podcast, so 14 00:01:05,877 --> 00:01:06,417 W. Curtis Preston: yeah. 15 00:01:06,627 --> 00:01:07,607 Prasanna Malaiyandi: can definitely pick his brains. 16 00:01:08,427 --> 00:01:08,817 W. Curtis Preston: Yeah. 17 00:01:09,027 --> 00:01:11,427 You know, it's been interesting cuz, and it was funny, it was on the 18 00:01:11,427 --> 00:01:15,237 podcast where I really, I really, I had a moment where I was like, 19 00:01:15,867 --> 00:01:17,967 I'm not really backing up I photos. 20 00:01:18,357 --> 00:01:18,657 Right? 21 00:01:18,657 --> 00:01:24,327 Like, I'm, I'm backing up, you know, I, I use iCloud, but as, as we have 22 00:01:24,332 --> 00:01:26,607 discussed, iCloud is not a backup. 23 00:01:26,937 --> 00:01:28,467 iCloud is a sync. 24 00:01:29,067 --> 00:01:29,487 Right. 25 00:01:29,547 --> 00:01:35,097 And if something catastrophic, if you, if I ever got hacked, um, and somebody 26 00:01:35,097 --> 00:01:40,317 got a hold of my, my iCloud password or my iPhone and then just decided to 27 00:01:40,317 --> 00:01:45,267 massively delete everything, I, if I caught it soon enough, I would be okay. 28 00:01:45,267 --> 00:01:47,397 Cuz I do have like a deleted items thing. 29 00:01:47,607 --> 00:01:47,967 Right. 30 00:01:48,417 --> 00:01:51,867 And as we discussed, I have, well, I don't think we'd discuss on the pod, 31 00:01:52,137 --> 00:01:56,847 but as part of this project I found out I have 11,000 photos in, in iCloud. 32 00:01:58,002 --> 00:01:58,722 So, 33 00:01:59,022 --> 00:02:01,242 Prasanna Malaiyandi: isn't as much as a lot of other people, you know. 34 00:02:01,242 --> 00:02:02,142 I'm sure there are 35 00:02:02,202 --> 00:02:03,132 W. Curtis Preston: know, I am not, 36 00:02:03,227 --> 00:02:03,342 Prasanna Malaiyandi: you. 37 00:02:04,152 --> 00:02:04,542 W. Curtis Preston: yeah. 38 00:02:04,542 --> 00:02:06,972 As, as I, as you and I were talking earlier, I'm not, you 39 00:02:06,972 --> 00:02:10,632 know, on one end, I'm not Cecil b Dilla and I'm not, you know, 40 00:02:10,632 --> 00:02:12,102 photographing and filming everything. 41 00:02:12,252 --> 00:02:15,992 On the other hand, I'm not Prasanna because you use your phone camera, 42 00:02:16,362 --> 00:02:18,312 like you use your, uh, Tesla, 43 00:02:19,467 --> 00:02:20,607 Prasanna Malaiyandi: Yeah, pretty much. 44 00:02:20,612 --> 00:02:21,057 Yeah. 45 00:02:21,477 --> 00:02:22,887 Almost never 46 00:02:23,442 --> 00:02:23,742 W. Curtis Preston: Okay. 47 00:02:23,742 --> 00:02:25,722 So you've had your Tesla, how long now 48 00:02:26,457 --> 00:02:27,597 Prasanna Malaiyandi: four years, 49 00:02:27,942 --> 00:02:29,322 W. Curtis Preston: and how many miles do you have on it? 50 00:02:30,087 --> 00:02:34,437 Prasanna Malaiyandi: I think like 11,600, 700, something like that. 51 00:02:35,962 --> 00:02:38,422 And it works great because my maintenance cost has been zero. 52 00:02:38,427 --> 00:02:41,302 My electricity cost is really minimal versus gas, 53 00:02:41,557 --> 00:02:44,077 W. Curtis Preston: Cars are incredibly reliable when you don't use them. 54 00:02:44,527 --> 00:02:46,147 Um, anyway, yeah, 55 00:02:46,342 --> 00:02:48,202 Prasanna Malaiyandi: powered cars, even if you don't use them, 56 00:02:48,202 --> 00:02:49,672 you still gotta change the oil. 57 00:02:49,672 --> 00:02:51,352 You still gotta do everything else, you know? 58 00:02:51,352 --> 00:02:51,802 So, 59 00:02:52,702 --> 00:02:54,232 W. Curtis Preston: I'm not, we're not doing a, we're not 60 00:02:54,232 --> 00:02:57,142 doing an e an ecar thing. 61 00:02:57,292 --> 00:03:01,312 But anyway, but yeah, we've, we've been having some fun with, uh, with this 62 00:03:01,317 --> 00:03:04,552 project of figuring out the various ways. 63 00:03:04,552 --> 00:03:05,032 Right. 64 00:03:05,122 --> 00:03:05,602 Uh, 65 00:03:05,632 --> 00:03:07,132 Prasanna Malaiyandi: and, and I think, yeah. 66 00:03:07,132 --> 00:03:09,652 I was just gonna say, I think you should mention to the listeners what you're 67 00:03:09,652 --> 00:03:14,242 current, what you are currently trying to do for backing up your iCloud photo 68 00:03:14,302 --> 00:03:19,432 W. Curtis Preston: my current, uh, uh, uh, I don't know what, 69 00:03:19,432 --> 00:03:22,192 I don't know what, what, what, I don't know these different methods. 70 00:03:22,192 --> 00:03:27,262 My current method that I am trying is Google Photos, and it turns 71 00:03:27,262 --> 00:03:34,132 out Google photos, it, it's the only one that I've found so far. 72 00:03:34,462 --> 00:03:36,022 Uh, well, the only one that I've. 73 00:03:37,072 --> 00:03:42,022 Well, there's maybe one other, which is iDrive, but Google Photos, cuz 74 00:03:42,027 --> 00:03:47,452 the problem is that on an iPhone you can turn on optimized storage. 75 00:03:47,902 --> 00:03:52,742 And so I have like, I don't know, somewhere between sixty 76 00:03:52,762 --> 00:03:53,482 and a hundred gigabytes. 77 00:03:53,482 --> 00:03:59,812 We're not quite sure of photos up in, uh, iCloud and, but I only have four 78 00:03:59,812 --> 00:04:03,052 and a half gigabytes on my phone because it's, I'm using the optimized storage. 79 00:04:03,362 --> 00:04:09,212 But apparently Google Cloud photo, Google Photos pulls down a high res 80 00:04:09,242 --> 00:04:11,612 whatever high, the original version from. 81 00:04:12,947 --> 00:04:14,477 iCloud and then backs 82 00:04:14,642 --> 00:04:15,602 Prasanna Malaiyandi: that, that's our theory. 83 00:04:15,632 --> 00:04:16,322 That's our theory. 84 00:04:16,427 --> 00:04:17,087 W. Curtis Preston: That's the theory. 85 00:04:17,087 --> 00:04:19,907 Well, it's, I, that's what it says in documentation. 86 00:04:20,117 --> 00:04:21,467 We shall see what we shall see. 87 00:04:22,097 --> 00:04:25,397 Um, and then we will report on the results here and then, 88 00:04:25,397 --> 00:04:26,627 and, and I'll blog about it. 89 00:04:26,632 --> 00:04:32,987 I'll backup Central, uh, because IO iCloud is not a backup. 90 00:04:33,197 --> 00:04:36,197 The number of articles that I read that told me to use iCloud to 91 00:04:36,197 --> 00:04:39,047 back up my iPhone pissed me off. 92 00:04:39,707 --> 00:04:40,157 Right? 93 00:04:40,217 --> 00:04:45,347 Like, it, it was like 95% of the articles that I found on how to back up, uh, 94 00:04:45,377 --> 00:04:48,677 my photos basically said, OI cloud. 95 00:04:48,797 --> 00:04:49,127 I'm like, ah. 96 00:04:49,352 --> 00:04:49,832 Prasanna Malaiyandi: Because. 97 00:04:49,862 --> 00:04:54,272 Because for most consumers, right, they're probably not going to do what 98 00:04:54,272 --> 00:04:56,222 you're about to do, and they don't care. 99 00:04:56,282 --> 00:04:57,812 And so turn on iCloud. 100 00:04:57,812 --> 00:04:59,912 At least you have something else other than whatever's on your phone, 101 00:05:00,197 --> 00:05:00,557 W. Curtis Preston: Yeah. 102 00:05:00,647 --> 00:05:04,667 So we're gonna, we're gonna have an answer for the three people in the world, 103 00:05:04,697 --> 00:05:08,207 all of which are probably already on this recording, the three people in the 104 00:05:08,212 --> 00:05:14,477 world that actually care about having an actual backup of their, of their photos. 105 00:05:14,482 --> 00:05:15,677 Anyway, all right. 106 00:05:15,857 --> 00:05:19,517 Well, we're gonna bring back a, a longtime friend and a 107 00:05:19,517 --> 00:05:21,627 returned guest to our podcast. 108 00:05:22,607 --> 00:05:28,667 Uh, he is one of the few people in this industry that, um, make me feel young. 109 00:05:29,027 --> 00:05:29,777 Uh, we welcome. 110 00:05:30,197 --> 00:05:31,037 We welcome. 111 00:05:31,457 --> 00:05:35,417 And he's also the, uh, the technologist, extraordinary and 112 00:05:35,417 --> 00:05:37,627 plenty of potentially at Vast Data. 113 00:05:38,217 --> 00:05:41,087 Welcome to the podcast, Howard Marks. 114 00:05:41,597 --> 00:05:42,377 How's it going, Howard? 115 00:05:43,412 --> 00:05:48,632 Howard Marks: I'm really happy to be here cuz you know you guys went on your 116 00:05:48,632 --> 00:05:54,982 little podcast and you said something about using flash for backup being stupid 117 00:05:55,487 --> 00:05:55,877 W. Curtis Preston: Yeah. 118 00:05:55,967 --> 00:05:56,117 Yeah. 119 00:05:56,117 --> 00:05:56,477 I might have 120 00:05:56,512 --> 00:05:59,072 Prasanna Malaiyandi: But, but, but, but wait, I wanna clarify, 121 00:05:59,072 --> 00:06:01,142 Howard, that was from Curtis. 122 00:06:01,532 --> 00:06:03,842 I was the one who was like, yes. 123 00:06:04,787 --> 00:06:05,237 W. Curtis Preston: Oh, he's 124 00:06:05,552 --> 00:06:06,212 Prasanna Malaiyandi: What about Howard? 125 00:06:07,152 --> 00:06:07,372 Oh, 126 00:06:07,397 --> 00:06:08,477 W. Curtis Preston: throwing me right under the 127 00:06:08,522 --> 00:06:09,152 Prasanna Malaiyandi: throw you under the, 128 00:06:09,887 --> 00:06:10,727 W. Curtis Preston: So, we'll, we'll, we'll 129 00:06:10,912 --> 00:06:12,332 Howard Marks: you don't have to explain that to me. 130 00:06:12,332 --> 00:06:14,162 I've known Curtis 35 years. 131 00:06:15,227 --> 00:06:16,187 W. Curtis Preston: that's that. 132 00:06:16,187 --> 00:06:20,527 We will, we will, uh, we will, we'll get to that topic. 133 00:06:21,047 --> 00:06:24,467 I will give you a chance to defend your, your, your Honor. 134 00:06:25,067 --> 00:06:28,577 Um, why don't we start with an update. 135 00:06:28,577 --> 00:06:30,467 It's been a while since we've had you on the pod. 136 00:06:30,467 --> 00:06:35,747 Why don't we start with an update on, uh, how much more vast, vast 137 00:06:35,747 --> 00:06:38,327 data is, uh, since we had you on. 138 00:06:40,052 --> 00:06:44,762 Howard Marks: Well, you know, from the financial side, um, we announced at 139 00:06:44,762 --> 00:06:48,902 the beginning of this year that we've hit a hundred million a year in annual 140 00:06:48,902 --> 00:06:54,362 recurring revenue cuz we've organized ourselves as a software company even 141 00:06:54,362 --> 00:06:59,282 though the experience customers have is look, appliances on my data center 142 00:06:59,462 --> 00:07:01,532 call vast when something goes wrong. 143 00:07:02,072 --> 00:07:07,292 Um, we arrange for customers to buy the hardware so that we are a software 144 00:07:07,292 --> 00:07:09,062 company, makes life easier for us. 145 00:07:09,632 --> 00:07:14,942 Um, the other big thing is that our friends at HPE 146 00:07:15,612 --> 00:07:20,762 just made announcement of a product of theirs called GreenLake Files 147 00:07:21,392 --> 00:07:24,302 that will be powered by our software. 148 00:07:24,967 --> 00:07:32,522 So before today, if you wanted a scale out, expandable, low cost all flesh file 149 00:07:32,522 --> 00:07:39,152 and object system, we would facilitate your buying hardware from the OEMs that we 150 00:07:39,152 --> 00:07:43,072 deal with, and we'd sell you the software and you'd have a system that was running. 151 00:07:43,692 --> 00:07:52,142 Now you can buy that from HPE as part of GreenLake, and that includes management 152 00:07:52,147 --> 00:07:54,902 through the GreenLake Cloud front end. 153 00:07:55,442 --> 00:08:00,332 So you can manage the GreenLake for files along with GreenLake for Block 154 00:08:00,332 --> 00:08:04,142 and the Compute and all the other servers that are part of GreenLake. 155 00:08:04,172 --> 00:08:08,762 So they've taken our software, married it to their control plane 156 00:08:08,767 --> 00:08:10,232 and run it on their hardware. 157 00:08:10,562 --> 00:08:12,632 Prasanna Malaiyandi: And is for, sorry, for those who may 158 00:08:12,637 --> 00:08:13,862 not be familiar with GreenLake. 159 00:08:14,102 --> 00:08:20,822 GreenLake is more of a, I don't know, a managed or a hosted environment done by 160 00:08:20,847 --> 00:08:24,017 Howard Marks: It, it, it, it's an as a servicey. 161 00:08:24,587 --> 00:08:29,867 So there, there are both consumption and CapEx models the way I understand it. 162 00:08:30,437 --> 00:08:39,337 But you know, you don't log into the block array that and create a LUN. 163 00:08:39,677 --> 00:08:44,387 You go to the cloud website and you create a LUN and their 164 00:08:44,387 --> 00:08:47,507 control plane does that for you. 165 00:08:48,107 --> 00:08:53,507 And so it's got more controls and you do don't have to keep the detail. 166 00:08:53,582 --> 00:08:53,792 Prasanna Malaiyandi: Yeah. 167 00:08:54,362 --> 00:08:56,132 W. Curtis Preston: So then you're, you're probably paying 168 00:08:56,132 --> 00:08:57,722 for what you provision then, 169 00:08:58,997 --> 00:09:00,768 Howard Marks: Uh, you can do that or you either way. 170 00:09:01,547 --> 00:09:05,687 And, you know, that's, that's, you know, kind of an H p E finance question. 171 00:09:06,407 --> 00:09:08,627 But my understanding is they do it either way. 172 00:09:08,957 --> 00:09:10,847 Prasanna Malaiyandi: And I'm sure for vast data, right. 173 00:09:10,847 --> 00:09:14,147 That's a huge win, being part of that offering. 174 00:09:14,582 --> 00:09:18,782 Howard Marks: Well, it's, you know, first of all, it just gets hundreds and hundreds 175 00:09:18,782 --> 00:09:26,102 of boots on the ground out selling our software and our whole concept of you 176 00:09:26,102 --> 00:09:31,052 can do all flash for as cheaper, cheaper than other guys can do spinning disk. 177 00:09:31,412 --> 00:09:35,192 And why do you want spinning discs as opposed to flash? 178 00:09:35,492 --> 00:09:36,812 Other than that, they're cheaper. 179 00:09:37,472 --> 00:09:39,312 You know, I can't think of another advantage. 180 00:09:40,022 --> 00:09:45,572 Um, and so we, you know, for most workloads have narrowed 181 00:09:45,577 --> 00:09:52,037 that down or reversed it, and so, All flash cheaper than disk. 182 00:09:52,037 --> 00:09:52,997 What a great idea. 183 00:09:53,867 --> 00:09:58,967 Um, so we've got, you know, a, all those HPE sales guys going out there selling it 184 00:09:58,967 --> 00:10:06,587 as an HPE product, you know, it's not like Qumulo or Scality, where HPE was reselling 185 00:10:06,587 --> 00:10:09,077 those products to run on HPE servers. 186 00:10:09,827 --> 00:10:10,107 Prasanna Malaiyandi: Mm-hmm. 187 00:10:10,577 --> 00:10:14,387 Howard Marks: Um, and you know, who you called for support was. 188 00:10:15,017 --> 00:10:18,317 Well, is this a server problem or is this a software problem? 189 00:10:18,737 --> 00:10:21,077 It's HPE GreenLake for files. 190 00:10:21,377 --> 00:10:23,537 HPE takes the support calls. 191 00:10:23,537 --> 00:10:25,157 It's a full HPE product. 192 00:10:26,177 --> 00:10:28,367 Um, it's our software underneath it. 193 00:10:29,727 --> 00:10:30,827 W. Curtis Preston: Yeah, it's interesting. 194 00:10:30,832 --> 00:10:36,197 So you know, you talked about the, you, you know, you said you, you. 195 00:10:36,707 --> 00:10:38,447 You, you have a r r. 196 00:10:38,447 --> 00:10:44,747 So basically your customers are paying an annual fee to you based 197 00:10:44,747 --> 00:10:47,177 on the size of their storage 198 00:10:47,177 --> 00:10:50,357 Howard Marks: we, so, so we make, we make our money on a, what 199 00:10:50,362 --> 00:10:51,947 we call a Gemini subscription. 200 00:10:52,637 --> 00:10:57,407 That is, you know, in capacity units, we sell it at a hundred terabytes. 201 00:10:57,407 --> 00:11:02,147 HP can sell it in different ways, um, and it's per year. 202 00:11:02,937 --> 00:11:08,417 And we guarantee that we'll write that agreement for any piece 203 00:11:08,417 --> 00:11:10,967 of hardware for 10 years at the 204 00:11:10,997 --> 00:11:11,987 W. Curtis Preston: right, right. 205 00:11:12,047 --> 00:11:12,377 right. 206 00:11:12,377 --> 00:11:12,707 I remember 207 00:11:12,767 --> 00:11:15,467 Howard Marks: Because, you know, it's not spinning disks. 208 00:11:15,467 --> 00:11:19,637 They don't start failing a lot more often in year five and six. 209 00:11:20,147 --> 00:11:22,927 And so if you want to keep it for 10 years, keep it for 10 years. 210 00:11:23,747 --> 00:11:27,917 If you decide you want to replace some of your hardware in your five 211 00:11:27,917 --> 00:11:33,557 or six, because the new denser or faster hardware is more attractive to 212 00:11:33,562 --> 00:11:38,837 you, uh, but you bought seven years of support, we'll transfer it on the, 213 00:11:39,047 --> 00:11:40,907 you know, terabyte per terabyte basis. 214 00:11:41,162 --> 00:11:41,522 W. Curtis Preston: Right. 215 00:11:41,527 --> 00:11:41,752 Gotcha. 216 00:11:43,112 --> 00:11:45,272 Um, yeah, that's a pretty good deal for you. 217 00:11:45,272 --> 00:11:50,642 And by the way, I'll, I'll, um, I, I was gonna, uh, compare it to something 218 00:11:50,642 --> 00:11:55,182 else, but, but it, it made me remind me of our, uh, disclaimer Prasanna. 219 00:11:55,202 --> 00:11:56,522 And I work for different companies. 220 00:11:56,522 --> 00:11:58,419 I work for myself, he works for Zoom. 221 00:11:58,739 --> 00:12:01,269 And, uh, these are our opinions, not theirs. 222 00:12:01,659 --> 00:12:05,589 And, uh, be sure to rate us at, uh, your favorite podcast or give 223 00:12:05,589 --> 00:12:07,029 us all the stars and comments. 224 00:12:07,034 --> 00:12:09,519 It helps other people find us. 225 00:12:09,519 --> 00:12:14,319 If you think we're amazing, then maybe other people will do so as well. 226 00:12:14,739 --> 00:12:19,779 Uh, reach out to me, uh, @wcpreston on Twitter, or w Curtis Preston 227 00:12:19,779 --> 00:12:22,629 at gmail, and, you know, to be part of the conversation. 228 00:12:22,749 --> 00:12:24,489 And we'll see. 229 00:12:24,579 --> 00:12:26,169 Um, you know, we'll get you on. 230 00:12:26,959 --> 00:12:32,319 So your arrangement with HP reminds me of our arrangement with Dell. 231 00:12:32,379 --> 00:12:34,809 Basically it's the whole boots on the ground thing. 232 00:12:35,319 --> 00:12:40,179 Uh, you get to put your product in front of a whole, you know, giant number of 233 00:12:40,179 --> 00:12:43,209 other people and it's great for you. 234 00:12:43,214 --> 00:12:44,019 It's good for them. 235 00:12:44,019 --> 00:12:49,659 Their customers get the benefit of your, uh, technology with, with the company 236 00:12:49,659 --> 00:12:51,369 that they already, you know, know and 237 00:12:51,654 --> 00:12:54,984 Howard Marks: And you know, and, and we all know that there are loyal H P E 238 00:12:54,989 --> 00:12:59,994 customers who you know now it's a lot more likely they'll buy this product cuz 239 00:12:59,994 --> 00:13:02,214 it's got that stamp of approval on it, 240 00:13:03,024 --> 00:13:04,644 all of which works for us. 241 00:13:05,949 --> 00:13:06,369 W. Curtis Preston: Yeah. 242 00:13:06,459 --> 00:13:06,909 Yeah. 243 00:13:07,299 --> 00:13:09,549 Well, congratulations on hitting a hundred million. 244 00:13:09,609 --> 00:13:12,999 Um, wish you the best of luck on your way to the, you know, 245 00:13:12,999 --> 00:13:14,349 doubling that and triple in that. 246 00:13:15,069 --> 00:13:22,869 Um, last time you were on, we talked about, we, we alluded to, I think, 247 00:13:22,869 --> 00:13:31,179 a little bit about how you do dedupe or dedupe-like stuff that's a little 248 00:13:31,179 --> 00:13:33,399 different than the rest of the world. 249 00:13:34,229 --> 00:13:39,459 And, and, and that it's better, you know, these are the, you know, the, 250 00:13:39,609 --> 00:13:40,689 the, you're saying it's better. 251 00:13:40,689 --> 00:13:42,879 So I, I want to give you a chance to talk about that, and then 252 00:13:42,959 --> 00:13:46,244 Howard Marks: we, we guarantee it's, we guarantee it's better because I'm 253 00:13:46,494 --> 00:13:49,134 a vendor and without a guarantee you shouldn't believe anything I say. 254 00:13:49,584 --> 00:13:50,034 W. Curtis Preston: Okay. 255 00:13:50,154 --> 00:13:50,664 All right. 256 00:13:50,664 --> 00:13:51,594 That sounds good. 257 00:13:51,864 --> 00:13:54,774 So how so how, so first off, how is it 258 00:13:54,774 --> 00:13:55,734 better, and then why? 259 00:13:56,394 --> 00:13:59,454 Howard Marks: It's better cause it reduces data further. 260 00:14:00,324 --> 00:14:02,934 Um, And the why is how it works. 261 00:14:03,384 --> 00:14:06,474 So, you know, at the beginning it's really pretty simple. 262 00:14:06,474 --> 00:14:13,284 We do variable chunk deduplication with a variation of the rock soft method. 263 00:14:13,854 --> 00:14:20,604 So if there are insertions, we re re re-sync relatively quickly and the 264 00:14:20,874 --> 00:14:22,644 deduplication gets more effective. 265 00:14:22,924 --> 00:14:23,344 W. Curtis Preston: Mm-hmm. 266 00:14:23,784 --> 00:14:27,924 Howard Marks: Um, we do z standard compression on the data. 267 00:14:29,274 --> 00:14:36,834 We then throw some data specific encryption algorithms at the data for 268 00:14:36,834 --> 00:14:38,935 things like, oh look, it's numeric data. 269 00:14:39,254 --> 00:14:41,844 Well that means it's only gonna vary within this range. 270 00:14:41,844 --> 00:14:42,684 We'll store deltas. 271 00:14:45,234 --> 00:14:50,004 And so whichever of those compression methods reduces this block of data most. 272 00:14:50,259 --> 00:14:50,679 W. Curtis Preston: Mm-hmm. 273 00:14:50,849 --> 00:14:51,264 Howard Marks: we run. 274 00:14:52,554 --> 00:14:59,604 Um, because we're doing so the, the data path is writes go to storage class memory. 275 00:15:00,499 --> 00:15:00,919 W. Curtis Preston: Mm-hmm. 276 00:15:01,014 --> 00:15:04,764 Howard Marks: then get act and then all of this data reduction happens 277 00:15:04,764 --> 00:15:10,644 as we migrate from that writebuffer to the capacity flash layer. 278 00:15:11,514 --> 00:15:17,304 And since it's after the act, as long as we're draining that buffer fast 279 00:15:17,309 --> 00:15:23,604 enough, l how long in time it takes to move any piece is irrelevant. 280 00:15:24,294 --> 00:15:27,894 And so we have time to go, ah, let's try five different compression algorithms. 281 00:15:28,194 --> 00:15:29,674 Use whichever one works best. 282 00:15:30,639 --> 00:15:30,919 W. Curtis Preston: Interesting. 283 00:15:31,389 --> 00:15:31,679 Yeah. 284 00:15:31,704 --> 00:15:32,394 Prasanna Malaiyandi: do you do, 285 00:15:32,659 --> 00:15:32,999 W. Curtis Preston: go ahead. 286 00:15:33,299 --> 00:15:33,639 Go ahead. 287 00:15:33,924 --> 00:15:34,494 Prasanna Malaiyandi: do you? 288 00:15:34,644 --> 00:15:39,144 And that's actually very interesting how you can, like you said, by 289 00:15:39,144 --> 00:15:40,284 storing it in memory, right. 290 00:15:40,284 --> 00:15:42,324 You're not impacting client latencies at all. 291 00:15:42,324 --> 00:15:42,564 Right. 292 00:15:42,564 --> 00:15:43,584 For them it's like, Hey, right. 293 00:15:43,589 --> 00:15:44,184 Went through. 294 00:15:44,484 --> 00:15:47,634 And then you have this time to do the parallel, uh, computation. 295 00:15:48,314 --> 00:15:49,724 Howard Marks: Yeah, just, just accept it. 296 00:15:49,854 --> 00:15:54,864 It's storage class memory, so it's an S s D, so it's persistent and 297 00:15:54,864 --> 00:15:59,304 there's no batteries and protection and you know, a panic when power goes 298 00:15:59,484 --> 00:15:59,784 Prasanna Malaiyandi: Yeah. 299 00:15:59,964 --> 00:16:04,464 Now, when you are running these algorithms, like I know AI and ML 300 00:16:04,464 --> 00:16:07,224 is all the hot topic everywhere you look these days, right? 301 00:16:07,464 --> 00:16:13,794 Are you guys doing anything around that in terms of trying to smartly detect 302 00:16:13,799 --> 00:16:16,134 which compression algorithms based on 303 00:16:16,479 --> 00:16:20,829 Howard Marks: We we're, we're not doing that in the data path right now. 304 00:16:21,279 --> 00:16:25,239 You know, frankly, the running the five doesn't use that much 305 00:16:25,239 --> 00:16:26,979 compute that it's worth it. 306 00:16:27,549 --> 00:16:33,069 Um, we're using AI in our cloud platform, so if you have multiple 307 00:16:33,069 --> 00:16:37,359 clusters, there's a cloud site you can go to and see one dashboard. 308 00:16:37,929 --> 00:16:41,619 Um, and we're using it for the capacity projections. 309 00:16:42,099 --> 00:16:45,309 So it's like, oh look, here's how much capacity you're gonna need 310 00:16:45,309 --> 00:16:48,639 six months from now while you're filling out your budget request. 311 00:16:48,669 --> 00:16:52,989 Let me tell you what you're gonna, there's AI behind that so that it 312 00:16:52,989 --> 00:16:56,139 smooths things like, oh look, every three months they do a cleanup. 313 00:16:56,529 --> 00:17:00,549 And so let me factor that the AI is good enough to factor that kind of thing 314 00:17:00,549 --> 00:17:03,249 in, but not in the data path. 315 00:17:04,029 --> 00:17:05,679 But let's get back to the data path. 316 00:17:05,839 --> 00:17:06,059 Prasanna Malaiyandi: Yep. 317 00:17:06,264 --> 00:17:11,034 W. Curtis Preston: yeah, your, um, your comment when you, you know, there 318 00:17:11,039 --> 00:17:17,044 was a comment you were like, as long as we're clearing the buffer quick enough. 319 00:17:17,824 --> 00:17:24,834 Um, and, and I would agree with you, um, you know, how, how do you ensure that 320 00:17:24,839 --> 00:17:27,024 that happens, I guess is, is one question. 321 00:17:28,284 --> 00:17:30,744 Howard Marks: Well, first of all, it becomes a parallelism issue. 322 00:17:31,674 --> 00:17:37,164 So we have a large number of compute nodes, all of which are 323 00:17:37,164 --> 00:17:39,414 draining this buffer in parallel. 324 00:17:40,194 --> 00:17:46,134 And so when the buffer hits a high water mark, more threads 325 00:17:46,134 --> 00:17:51,564 to D stage, it gets spawned and allocated across the parallel system. 326 00:17:52,164 --> 00:17:57,984 Now, if there's a huge influx of writes, and you know, we're talking. 327 00:18:00,474 --> 00:18:06,234 Tens of gigabytes per second for hours on the smallest system. 328 00:18:06,424 --> 00:18:06,844 W. Curtis Preston: Mm-hmm. 329 00:18:07,704 --> 00:18:11,574 Howard Marks: Um, then we'll start introducing latency into the 330 00:18:11,574 --> 00:18:13,224 writes and apply back pressure. 331 00:18:14,504 --> 00:18:14,794 W. Curtis Preston: Okay. 332 00:18:14,944 --> 00:18:15,774 Okay, that makes 333 00:18:16,224 --> 00:18:18,784 Howard Marks: But, but you know, that's, you know, literally, 334 00:18:19,884 --> 00:18:21,864 you know, it, it don't 335 00:18:21,964 --> 00:18:26,034 ha it, you know, the mechanism is there just in case, 336 00:18:26,074 --> 00:18:26,424 W. Curtis Preston: right. 337 00:18:26,664 --> 00:18:27,204 Howard Marks: happen. 338 00:18:27,639 --> 00:18:32,079 Prasanna Malaiyandi: And when you, and because you have more than more 339 00:18:32,169 --> 00:18:39,159 capacity, I guess, more throughput at the capacity level than at the storage 340 00:18:39,159 --> 00:18:42,729 media level, is that why you can just increase the number of parallel 341 00:18:42,729 --> 00:18:45,579 threads and you don't have to worry about the backend being a bottleneck? 342 00:18:46,194 --> 00:18:50,604 Howard Marks: so in, in, so our, our building block, we call a D Box 343 00:18:50,604 --> 00:18:54,204 or a data box, and it's got some. 344 00:18:54,894 --> 00:18:56,094 S scm SSDs. 345 00:18:56,184 --> 00:18:57,384 We started with Opta. 346 00:18:58,014 --> 00:19:05,814 We now mostly use K oxia FL six, and then a larger number of capacity SSDs. 347 00:19:06,774 --> 00:19:11,574 And they, you know, it's whatever the cheapest we can get or the cheapest that 348 00:19:11,574 --> 00:19:18,294 our OEMs use is, um, the P C I E lanes. 349 00:19:18,294 --> 00:19:24,414 Feeding the small number of S C m SSDs is generally the bottleneck. 350 00:19:24,904 --> 00:19:25,194 Prasanna Malaiyandi: Okay. 351 00:19:26,064 --> 00:19:30,894 Howard Marks: And so we can paralyze reading data out of s C m, the 352 00:19:30,899 --> 00:19:32,794 writing to cut to the capacity. 353 00:19:33,614 --> 00:19:37,614 We have a lot more capacity ssd, so there's plenty of bandwidth to write 354 00:19:37,904 --> 00:19:38,324 Prasanna Malaiyandi: Gotcha. 355 00:19:39,894 --> 00:19:42,685 W. Curtis Preston: So what, why'd you stop using OC Octane? 356 00:19:44,544 --> 00:19:49,614 Howard Marks: Um, well first we decided just to get a second 357 00:19:49,614 --> 00:19:51,324 source because it's a good idea. 358 00:19:52,209 --> 00:19:54,999 Um, and then I, Intel 359 00:19:55,104 --> 00:19:56,214 W. Curtis Preston: turned out to be a really good 360 00:19:56,499 --> 00:20:00,909 Howard Marks: of, then, then Intel decided to get out of the business. 361 00:20:01,749 --> 00:20:02,739 Um, 362 00:20:03,459 --> 00:20:07,359 and, you know, we have supply agreements with Intel. 363 00:20:07,359 --> 00:20:09,399 They still have a warehouse full of wafers. 364 00:20:10,179 --> 00:20:16,239 Um, but it, you know, the, the performance advantage wasn't worth the complexity. 365 00:20:16,399 --> 00:20:22,639 So we've chunked on these variable sized 32 K average blocks and we de-dupe them. 366 00:20:24,199 --> 00:20:27,979 But in addition to running a strong hash. 367 00:20:28,774 --> 00:20:36,064 To validate identical, we run a series of weaker hashes against the same 368 00:20:36,064 --> 00:20:42,034 data blocks, and these weaker hashes are designed to generate the same 369 00:20:42,034 --> 00:20:49,294 hash value for inputs across a narrow range of cryptographic distance. 370 00:20:50,464 --> 00:20:54,454 So if two blocks have a sm, so cryptographic distance is the 371 00:20:54,454 --> 00:20:58,444 number of bits you have to flip to turn block A into block B. 372 00:20:59,704 --> 00:21:06,004 If block A is within X bits of block B, this hash will 373 00:21:06,004 --> 00:21:07,834 generate the same hash value 374 00:21:09,754 --> 00:21:10,044 W. Curtis Preston: Okay. 375 00:21:11,164 --> 00:21:12,994 Howard Marks: from a data reduction point of view. 376 00:21:12,994 --> 00:21:17,254 If two blocks generate the same hash value and are a small cryptographic 377 00:21:17,259 --> 00:21:22,834 distance part, they have long common strings between them. 378 00:21:23,974 --> 00:21:28,819 And will therefore re compress with the same compression dictionary. 379 00:21:31,969 --> 00:21:37,879 So the first block that generates one of these similarity hashes, we just 380 00:21:37,909 --> 00:21:44,779 compress and store when the second through MTH block generates the same hash. 381 00:21:45,079 --> 00:21:49,729 We recall the first one and we used the dictionary from the first 382 00:21:49,729 --> 00:21:51,289 one to compress the second one 383 00:21:51,604 --> 00:21:53,374 Prasanna Malaiyandi: So you get better compression 384 00:21:53,509 --> 00:21:55,969 Howard Marks: we can store it compressed without the overhead of 385 00:21:55,969 --> 00:21:57,829 storing the dictionary a second time. 386 00:21:58,264 --> 00:21:58,474 Prasanna Malaiyandi: yeah. 387 00:21:59,494 --> 00:22:00,094 W. Curtis Preston: Yeah. 388 00:22:00,364 --> 00:22:00,724 Yeah. 389 00:22:01,279 --> 00:22:03,499 Howard Marks: and it becomes essentially the difference, 390 00:22:03,604 --> 00:22:03,904 Prasanna Malaiyandi: Yeah. 391 00:22:04,564 --> 00:22:08,434 So instead of storing a bigger block, it's like just very, very small Deltas 392 00:22:08,434 --> 00:22:11,254 because they are cryptographically 393 00:22:13,129 --> 00:22:13,429 Howard Marks: right? 394 00:22:13,534 --> 00:22:13,984 W. Curtis Preston: that, 395 00:22:14,164 --> 00:22:14,464 that's 396 00:22:14,509 --> 00:22:15,139 Howard Marks: similar. 397 00:22:15,439 --> 00:22:18,889 The mathematicians would say it's a limited cryptographic distance. 398 00:22:19,124 --> 00:22:19,984 Prasanna Malaiyandi: That's unique. 399 00:22:20,374 --> 00:22:23,764 I've never heard of someone doing that. 400 00:22:23,824 --> 00:22:24,744 Have you, Curtis? 401 00:22:27,124 --> 00:22:29,284 W. Curtis Preston: just this guy that we had on the podcast a little 402 00:22:29,374 --> 00:22:29,524 Prasanna Malaiyandi: Yeah. 403 00:22:29,734 --> 00:22:30,844 W. Curtis Preston: that looks a lot like Howard. 404 00:22:32,779 --> 00:22:36,409 Howard Marks: It, it, you know, nobody else is doing it now. 405 00:22:37,099 --> 00:22:37,849 Um, 406 00:22:38,274 --> 00:22:40,684 W. Curtis Preston: Is it, are you patenting it or, 407 00:22:41,359 --> 00:22:43,069 Howard Marks: there are patents around it. 408 00:22:43,069 --> 00:22:45,879 I don't, I haven't looked to see exactly 409 00:22:46,324 --> 00:22:46,744 W. Curtis Preston: gotcha. 410 00:22:46,924 --> 00:22:47,344 Gotcha. 411 00:22:47,449 --> 00:22:47,450 Howard Marks: to. 412 00:22:48,124 --> 00:22:48,304 W. Curtis Preston: Yeah. 413 00:22:48,304 --> 00:22:48,454 That 414 00:22:48,499 --> 00:22:48,719 Howard Marks: Um, 415 00:22:49,399 --> 00:22:52,519 cause reading patent applications makes my brain hurt. 416 00:22:53,719 --> 00:22:56,179 W. Curtis Preston: I see, I thought that you would, let's 417 00:22:56,179 --> 00:22:57,769 say you got two chunks, right? 418 00:22:57,829 --> 00:23:03,559 And you run the really weak, but much faster, I'm assuming, uh, hashing 419 00:23:03,559 --> 00:23:09,049 algorithm, and that you would say these two blocks definitely aren't the same. 420 00:23:09,229 --> 00:23:11,209 And so let's not do anything else other than com. 421 00:23:11,389 --> 00:23:12,439 They're not, they're nowhere. 422 00:23:12,469 --> 00:23:14,509 They're, they're, they're cryptographic distance. 423 00:23:14,509 --> 00:23:18,109 I think you said so far apart, there's no point in running the 424 00:23:18,109 --> 00:23:21,559 stronger ddu, uh, thing on it. 425 00:23:22,219 --> 00:23:23,149 Um, that's 426 00:23:23,149 --> 00:23:23,389 where I 427 00:23:23,479 --> 00:23:27,859 Howard Marks: it, it turns out, it turns out even with a weak hash, the 428 00:23:27,859 --> 00:23:36,169 number of identical hashes that are not identical data is so small that 429 00:23:36,174 --> 00:23:39,019 the cost of testing is ignorable. 430 00:23:39,644 --> 00:23:39,919 Prasanna Malaiyandi: Yeah. 431 00:23:40,909 --> 00:23:43,459 And especially if it's in flash, like probably 432 00:23:43,759 --> 00:23:43,979 Howard Marks: the, 433 00:23:44,509 --> 00:23:45,169 Prasanna Malaiyandi: is 434 00:23:45,319 --> 00:23:47,719 Howard Marks: compare is so rare. 435 00:23:47,724 --> 00:23:49,549 It doesn't matter that it's expensive. 436 00:23:49,699 --> 00:23:49,939 Prasanna Malaiyandi: Yeah. 437 00:23:51,499 --> 00:23:54,139 And the fact that you're not doing this in line, right. 438 00:23:54,139 --> 00:23:55,069 So it's all been de 439 00:23:55,429 --> 00:23:55,549 Or 440 00:23:56,089 --> 00:23:59,449 Howard Marks: it, it's not trench. 441 00:23:59,479 --> 00:24:02,059 It's in line, but it's not. 442 00:24:03,159 --> 00:24:03,579 Prasanna Malaiyandi: client. 443 00:24:03,589 --> 00:24:03,949 Howard Marks: act, 444 00:24:04,129 --> 00:24:04,699 Prasanna Malaiyandi: Yeah, yeah. 445 00:24:04,704 --> 00:24:05,019 Exactly. 446 00:24:05,239 --> 00:24:10,009 Howard Marks: it's, it's it's post act, so it doesn't have any impact on latency. 447 00:24:10,579 --> 00:24:14,999 But you know, the, the S CM is a one-way writebuffer. 448 00:24:15,019 --> 00:24:20,289 We write new data into it, it gets demoted to the capacity flash layer 449 00:24:21,169 --> 00:24:25,579 and there's so much bandwidth in the capacity flash layer that reads from 450 00:24:25,579 --> 00:24:28,549 there actually faster than from the scm. 451 00:24:29,029 --> 00:24:31,339 So there's no reason ever to promote it back. 452 00:24:32,284 --> 00:24:32,674 W. Curtis Preston: Right, 453 00:24:32,779 --> 00:24:38,839 Howard Marks: Um, but the other thing is we keep all the metadata in that s scm. 454 00:24:39,949 --> 00:24:44,029 So as you expand the system, you add another enclosure that's got more s cm 455 00:24:44,029 --> 00:24:49,579 and more capacity, the DUP hash table and the similarity hash tables all grow 456 00:24:49,579 --> 00:24:50,029 with it. 457 00:24:50,749 --> 00:24:56,149 So it's one data reduction realm regardless of how big a cluster is. 458 00:24:56,779 --> 00:24:59,559 We don't have to store that DUP table in memory. 459 00:25:00,514 --> 00:25:00,844 Prasanna Malaiyandi: Yep. 460 00:25:01,369 --> 00:25:06,319 Howard Marks: And so you know the whole, well, flash would be great for backup, 461 00:25:06,319 --> 00:25:08,599 except I can't afford it as well. 462 00:25:08,599 --> 00:25:14,219 If you've got three or four conventional PBBAs, 463 00:25:14,534 --> 00:25:14,954 Prasanna Malaiyandi: Mm-hmm. 464 00:25:15,349 --> 00:25:18,859 Howard Marks: you know, first of all, the vendors of PBBAs charged 465 00:25:18,889 --> 00:25:21,199 you a lot for that disc storage. 466 00:25:22,249 --> 00:25:24,509 You know, they, that's a high margin product. 467 00:25:25,564 --> 00:25:25,854 Prasanna Malaiyandi: Yeah. 468 00:25:26,209 --> 00:25:31,249 Howard Marks: as soon as you have two of them, you have two deduplication realms. 469 00:25:31,689 --> 00:25:32,039 W. Curtis Preston: Right, 470 00:25:32,299 --> 00:25:36,229 Howard Marks: And we might talk about data duping 10 to one. 471 00:25:36,829 --> 00:25:39,589 That doesn't mean all your data dupes 10 to one 472 00:25:40,059 --> 00:25:40,409 W. Curtis Preston: right. 473 00:25:41,059 --> 00:25:44,059 Howard Marks: 50% of your data at least is unique. 474 00:25:44,264 --> 00:25:44,484 Prasanna Malaiyandi: Yep, 475 00:25:45,259 --> 00:25:50,239 Howard Marks: Some of your data ddus a hundred or a thousand to one, and most 476 00:25:50,239 --> 00:25:54,079 of the benefits you get is from that data that ddus a hundred or a thousand to one. 477 00:25:54,079 --> 00:25:58,239 Well, when you got two boxes, it's not a hundred or a thousand, it's 50 to 500. 478 00:25:58,924 --> 00:25:59,374 Prasanna Malaiyandi: yep. 479 00:25:59,674 --> 00:26:03,784 And every time you add a new box, you're, you lose some of that benefit as well. 480 00:26:03,784 --> 00:26:04,144 Right. 481 00:26:04,144 --> 00:26:04,444 So, 482 00:26:05,204 --> 00:26:07,714 W. Curtis Preston: you can dup all our episodes down to 483 00:26:07,714 --> 00:26:09,304 like four or five comments. 484 00:26:09,394 --> 00:26:09,754 Right? 485 00:26:09,754 --> 00:26:12,454 Like back up, back up, all the things. 486 00:26:12,724 --> 00:26:13,294 Howard Marks: Well that 487 00:26:13,384 --> 00:26:14,134 W. Curtis Preston: 3, 2, 1. 488 00:26:14,134 --> 00:26:15,124 Just 3, 2, 1. 489 00:26:16,114 --> 00:26:19,534 Howard Marks: that requires the next version of, uh, AI deduplication 490 00:26:19,534 --> 00:26:21,454 that can take out the idle banter. 491 00:26:22,564 --> 00:26:24,004 W. Curtis Preston: Exactly, exactly. 492 00:26:24,004 --> 00:26:25,804 Our episodes will be like five minutes long. 493 00:26:26,489 --> 00:26:29,129 Prasanna Malaiyandi: So just to summarize or just to close on that, so 494 00:26:29,129 --> 00:26:31,049 we talked about the how you guys do it. 495 00:26:31,054 --> 00:26:35,279 So because of all these technologies that you're leveraging or mechanisms, 496 00:26:35,279 --> 00:26:40,619 right, that's how you're able to offer that guarantee, right? 497 00:26:40,619 --> 00:26:42,499 That's better than anyone else. 498 00:26:43,169 --> 00:26:45,449 Howard Marks: Yeah, we, you know, we use Z Standard. 499 00:26:45,449 --> 00:26:48,569 It's a slightly newer compression algorithm than anybody else 500 00:26:48,869 --> 00:26:52,619 does, cuz we started a little bit later than everybody else. 501 00:26:52,619 --> 00:26:54,299 So we got to pick the latest one. 502 00:26:54,959 --> 00:26:59,459 Um, and then we have those, you know, the additional, well, oh, they're numbers, 503 00:26:59,459 --> 00:27:01,709 let's just store the differences, tricks. 504 00:27:02,879 --> 00:27:06,029 And then we do deduplication on variable block, which is 505 00:27:06,034 --> 00:27:07,529 as well as anybody does it. 506 00:27:08,099 --> 00:27:10,979 And then we throw in similarity as, oh, here's another unique 507 00:27:10,979 --> 00:27:12,449 trick nobody else does. 508 00:27:13,049 --> 00:27:17,909 And so the combination is, we are confident that as long as you're send, 509 00:27:17,914 --> 00:27:21,299 you know, we guarantee as long as you're sending us unencrypted data, 510 00:27:22,529 --> 00:27:26,999 that will reduce it better than the other guy, whoever the other guy is. 511 00:27:27,479 --> 00:27:31,769 And if we don't, we'll provide the capacity so that you 512 00:27:31,769 --> 00:27:32,849 didn't pay any more money. 513 00:27:32,849 --> 00:27:33,059 Cuz 514 00:27:33,194 --> 00:27:33,524 Prasanna Malaiyandi: Yeah. 515 00:27:34,034 --> 00:27:34,244 Yeah. 516 00:27:34,514 --> 00:27:37,754 No, that's a great guarantee for end users and customers, especially 517 00:27:37,754 --> 00:27:38,864 with budgets these days, right? 518 00:27:38,864 --> 00:27:40,894 It's like, Hey, I bought this system. 519 00:27:41,354 --> 00:27:43,304 It doesn't quite meet my expectations. 520 00:27:43,304 --> 00:27:46,214 I can't go back to my boss and ask for more money. 521 00:27:46,214 --> 00:27:46,694 So 522 00:27:47,204 --> 00:27:50,864 Howard Marks: Well, you know, the other side of that is, you 523 00:27:50,869 --> 00:27:54,794 know, just, it's really a very simple scale out architecture. 524 00:27:55,274 --> 00:27:59,294 So you don't buy today what you think you're gonna need in three years. 525 00:27:59,294 --> 00:28:03,534 You buy today what you think you're gonna need in a year, and then you 526 00:28:03,534 --> 00:28:05,654 can buy more when you need it later. 527 00:28:05,654 --> 00:28:09,524 Or if, as some of our customers have found out much to therin of 528 00:28:09,529 --> 00:28:14,084 our sales guys, their data reduces better than they expected and they 529 00:28:14,084 --> 00:28:16,394 don't need anymore in the next year. 530 00:28:16,904 --> 00:28:18,734 Well then you're just ahead of the game. 531 00:28:19,259 --> 00:28:23,159 Prasanna Malaiyandi: So, and I know maybe you could talk in gener generalities, 532 00:28:23,159 --> 00:28:29,549 but sort of like if I was a customer who had one of the competition, PBBAs, right. 533 00:28:29,549 --> 00:28:31,469 And I now use Vast, right. 534 00:28:31,469 --> 00:28:36,539 I buy a vast system, sort of like, is there, like what is the savings that 535 00:28:36,539 --> 00:28:38,129 I normally see in terms of storage? 536 00:28:38,129 --> 00:28:42,089 Like if I had like a hundred terabyte P B B A, actual 537 00:28:42,299 --> 00:28:45,479 Howard Marks: if, if you, you know, a hundred terabytes is small for us. 538 00:28:45,599 --> 00:28:45,959 Prasanna Malaiyandi: okay. 539 00:28:46,049 --> 00:28:46,409 Or say 540 00:28:46,414 --> 00:28:46,439 a 541 00:28:46,499 --> 00:28:51,239 Howard Marks: So if you had, if you had a petabyte P B B A, um, then you 542 00:28:51,239 --> 00:28:55,439 know, you're probably storing four or five petabytes of logical data on 543 00:28:55,439 --> 00:28:55,639 it. 544 00:28:56,909 --> 00:29:04,199 Um, and you bought a, you know, a petabyte of usable from us and you'd 545 00:29:04,204 --> 00:29:07,739 probably store 25 or 30% more on it. 546 00:29:08,264 --> 00:29:08,744 Prasanna Malaiyandi: Okay. 547 00:29:09,139 --> 00:29:14,609 Howard Marks: But that petabyte, P B B A is as big as you can buy that P B B 548 00:29:14,609 --> 00:29:16,829 A, there isn't a two petabyte P B B A. 549 00:29:17,384 --> 00:29:17,714 Prasanna Malaiyandi: Yep. 550 00:29:18,659 --> 00:29:24,269 Howard Marks: And the real difference is at restore time 551 00:29:24,824 --> 00:29:25,364 Prasanna Malaiyandi: Hmm. 552 00:29:26,029 --> 00:29:29,339 Howard Marks: because PBBAs are scaled. 553 00:29:29,819 --> 00:29:31,859 For backup speed, not restore speed. 554 00:29:31,919 --> 00:29:34,949 They don't even have restore speed on the spec sheet anymore. 555 00:29:36,419 --> 00:29:43,379 And backups are not sequential operations nearly as much as 556 00:29:43,379 --> 00:29:44,999 you think they used to be. 557 00:29:45,449 --> 00:29:45,869 And you 558 00:29:45,869 --> 00:29:46,349 know, when 559 00:29:46,349 --> 00:29:53,609 Curtis, when when Curtis changed block, block tracking, incremental forever, 560 00:29:53,834 --> 00:29:54,314 W. Curtis Preston: Right? 561 00:29:54,329 --> 00:29:59,459 Howard Marks: of those things make the backup and the restore much more random. 562 00:29:59,764 --> 00:29:59,984 Prasanna Malaiyandi: yep, 563 00:30:00,599 --> 00:30:04,379 Howard Marks: And so if you're backing up to a disc based p v a, 564 00:30:04,384 --> 00:30:07,879 your restore speed is like a fourth or a fifth, you're backup speed. 565 00:30:09,054 --> 00:30:09,274 Prasanna Malaiyandi: yep. 566 00:30:09,779 --> 00:30:13,109 Howard Marks: If you're backing up to a vast, your restore speed 567 00:30:13,319 --> 00:30:15,329 is five times your backup speed. 568 00:30:17,279 --> 00:30:20,279 Cause we're, cuz we are designed to serve. 569 00:30:20,894 --> 00:30:24,914 Re primary storage applications where reads happen much more frequently 570 00:30:24,914 --> 00:30:28,814 than writes cuz the reads come from all the capacity SSDs, the 571 00:30:28,814 --> 00:30:30,374 writes have to go to the s scm. 572 00:30:31,274 --> 00:30:37,244 Um, and so what, where that really starts to get important is when, when we start 573 00:30:37,244 --> 00:30:42,464 talking about ransomware attack, cuz 10 years ago Curtis and I used to teach 574 00:30:42,464 --> 00:30:47,594 seminars and we'd go, yeah, 90, 95% of your restorers are, you know, the file. 575 00:30:47,594 --> 00:30:48,854 Somebody screwed up. 576 00:30:49,574 --> 00:30:53,804 And you know, if it's on A P B B A it'll be restored in a couple of minutes. 577 00:30:53,804 --> 00:30:54,434 And if it was on 578 00:30:54,439 --> 00:30:57,884 tape, you'd go find the tape and then a couple of minutes. 579 00:30:57,884 --> 00:31:02,294 And so, but you don't know you've been ransomware attacked till 580 00:31:02,294 --> 00:31:05,594 thousands or hundreds of thousands of files have been encrypted. 581 00:31:05,929 --> 00:31:06,149 Prasanna Malaiyandi: Yep. 582 00:31:07,244 --> 00:31:12,014 Howard Marks: And so now you have to like use something like instant recovery to 583 00:31:12,014 --> 00:31:15,074 check back, you know, is this backup good? 584 00:31:16,304 --> 00:31:20,594 You gotta do three or four quick looks without restoring, which 585 00:31:20,594 --> 00:31:24,824 is a great feature, but you know, requires a relatively high speed 586 00:31:24,824 --> 00:31:26,864 backend to work relatively well. 587 00:31:27,584 --> 00:31:30,734 And then you're gonna find, okay, this is my last non good point. 588 00:31:31,574 --> 00:31:34,484 And then you have to restore and you are gonna have to restore a lot 589 00:31:34,484 --> 00:31:39,254 of data and restore speed starts to become really important then. 590 00:31:39,764 --> 00:31:40,214 Prasanna Malaiyandi: Mm-hmm. 591 00:31:40,634 --> 00:31:45,034 Howard Marks: And then the kicker is, and the lawyers in the insurance company 592 00:31:46,184 --> 00:31:50,504 won't let you use the, the system that was infected for another couple of 593 00:31:50,504 --> 00:31:55,514 weeks cuz it's evidence or we have to get somebody in to clean it and certify 594 00:31:55,514 --> 00:32:02,054 that it's cleaned well, if you know you can run a VMware NFS data store 595 00:32:02,054 --> 00:32:06,254 on VAs, you can just restore to VASc. 596 00:32:06,259 --> 00:32:09,344 Now it's a bad idea to run your primary and your backup on the same 597 00:32:09,344 --> 00:32:11,534 system for more than a day or two, 598 00:32:12,224 --> 00:32:12,524 W. Curtis Preston: Right, 599 00:32:12,554 --> 00:32:16,619 Howard Marks: but, Compared to not running your primary and just, you 600 00:32:16,619 --> 00:32:21,329 know, if your choice is backup only or primary and backup, and if this one 601 00:32:21,334 --> 00:32:23,459 system dies, I'm really in trouble. 602 00:32:23,969 --> 00:32:26,309 Not that part of choice for me. 603 00:32:26,309 --> 00:32:28,589 I want my users back up and running. 604 00:32:29,009 --> 00:32:32,369 As soon as the lawyers let me get tacked to the old system, or my 605 00:32:32,369 --> 00:32:36,359 VAR gives me a new system, or I have someplace else to storage, VMO 606 00:32:36,359 --> 00:32:39,479 to, I'm getting that stuff off there right away. 607 00:32:39,809 --> 00:32:43,709 But that might mean I'm up a week earlier and a week earlier is a lot of time. 608 00:33:13,945 --> 00:33:19,245 W. Curtis Preston: my objection to flash for backup has 609 00:33:19,245 --> 00:33:21,495 been for two primary reasons. 610 00:33:21,495 --> 00:33:26,415 One is, is expensive af right second. 611 00:33:27,600 --> 00:33:29,040 Do I really need it? 612 00:33:29,520 --> 00:33:30,000 Right? 613 00:33:30,300 --> 00:33:35,130 Like, because there's, there are a lot of things that we can buy in life, right? 614 00:33:35,520 --> 00:33:38,520 Uh, like I, I need to move fertilizer. 615 00:33:38,520 --> 00:33:44,610 I can totally borrow Prasannas, uh, Tesla and it will do it, right? 616 00:33:44,950 --> 00:33:47,730 But, but is that what I should be using for that? 617 00:33:47,730 --> 00:33:51,960 Do I need, do I need a Tesla to move fertilizer or will 618 00:33:51,960 --> 00:33:53,160 my Prius do 619 00:33:53,265 --> 00:33:54,485 Howard Marks: need Prasannas. 620 00:33:55,155 --> 00:33:59,205 Tesla to move fertilizer if you ever want prasanna to speak to you again. 621 00:33:59,700 --> 00:34:01,050 W. Curtis Preston: no, that's, that's true. 622 00:34:01,260 --> 00:34:04,800 By the way, the Prius has been used to move fertilizer just for the record. 623 00:34:05,310 --> 00:34:08,850 Um, but so, so that's the thing. 624 00:34:08,850 --> 00:34:11,880 It's like, there, there are a lot of things, like, this goes back 625 00:34:11,880 --> 00:34:15,180 to the c d P, the c d P argument that I made back in the day. 626 00:34:15,330 --> 00:34:18,420 It was the same thing, the same two arguments. 627 00:34:18,570 --> 00:34:21,620 One was c D P was too damn expensive, right? 628 00:34:22,410 --> 00:34:24,780 And then the other was, does anybody actually need. 629 00:34:25,980 --> 00:34:29,610 The, the, the, the functionality that C D P provided. 630 00:34:29,610 --> 00:34:30,870 And the answer is yes. 631 00:34:31,200 --> 00:34:34,800 0.1% of the population needed what C D P provided. 632 00:34:34,800 --> 00:34:39,810 And that's why you don't really see c D P as a choice very, very often these days. 633 00:34:39,810 --> 00:34:44,280 Right there, there, there's a one or two companies that do it now, um, 634 00:34:44,860 --> 00:34:46,710 and all the other products have died. 635 00:34:46,710 --> 00:34:49,440 So those are my two arguments. 636 00:34:49,860 --> 00:34:53,310 It's, I, I already know what your argument to the second one is gonna 637 00:34:53,310 --> 00:34:55,410 be because you just gave it, I think. 638 00:34:56,340 --> 00:34:57,960 Um, so 639 00:34:57,965 --> 00:34:58,320 why 640 00:34:58,350 --> 00:34:59,130 Prasanna Malaiyandi: about for cost? 641 00:34:59,745 --> 00:35:00,165 Howard Marks: Well, 642 00:35:00,185 --> 00:35:00,760 W. Curtis Preston: talk about costs? 643 00:35:01,335 --> 00:35:05,235 Howard Marks: so, for cost, it depends what flash systems you're talking about. 644 00:35:05,470 --> 00:35:05,890 W. Curtis Preston: Mm-hmm. 645 00:35:06,705 --> 00:35:12,135 Howard Marks: Um, I will give you that most all flash systems are 646 00:35:12,135 --> 00:35:19,355 designed to be fast as possible for a small amount of data, because that's 647 00:35:19,355 --> 00:35:23,595 what you need to run the Oracle databases that make companies work. 648 00:35:24,015 --> 00:35:29,625 And so if you, you know, it's a block, you know, it's block storage to be low 649 00:35:29,625 --> 00:35:35,685 latency to support O L T P and therefore expensive because that's, you know, if 650 00:35:35,685 --> 00:35:40,725 that system goes down, you count by the second how much money you're losing. 651 00:35:40,730 --> 00:35:43,815 And so you have always bought expensive storage for that. 652 00:35:45,105 --> 00:35:45,705 Um, 653 00:35:45,735 --> 00:35:50,625 W. Curtis Preston: sort of the, sort of the true normal sort of, if I, if 654 00:35:50,625 --> 00:35:54,225 I can use this word, pure flash array. 655 00:35:55,380 --> 00:35:56,070 Howard Marks: Yes. 656 00:35:56,325 --> 00:35:59,745 W. Curtis Preston: the, that type is designed for that, right? 657 00:35:59,865 --> 00:36:05,205 Um, that's technically pure with a small p, but it works the other way as well. 658 00:36:05,805 --> 00:36:06,495 Um, 659 00:36:06,635 --> 00:36:08,880 Howard Marks: talking either way, you know? 660 00:36:08,940 --> 00:36:09,330 Yeah. 661 00:36:09,510 --> 00:36:12,840 You know, I could name half a dozen other products, but 662 00:36:14,475 --> 00:36:15,885 W. Curtis Preston: And they're just too expensive. 663 00:36:16,500 --> 00:36:21,330 Howard Marks: of it, you know, it's, we're gonna design a system based on 664 00:36:21,330 --> 00:36:24,180 having a, a pyramidal tiered system. 665 00:36:25,055 --> 00:36:26,100 this is the one at the top. 666 00:36:26,695 --> 00:36:26,985 Prasanna Malaiyandi: Yeah. 667 00:36:27,870 --> 00:36:31,530 Howard Marks: And if you assume you're gonna build a tier system, then you 668 00:36:31,530 --> 00:36:35,340 want the one at the top to be as fast as possible, and you kind of 669 00:36:35,345 --> 00:36:37,110 don't care how much it costs because 670 00:36:37,110 --> 00:36:39,600 you'll just put stuff that doesn't deserve it on the next tier. 671 00:36:40,085 --> 00:36:40,305 Prasanna Malaiyandi: Yep. 672 00:36:40,650 --> 00:36:44,550 Howard Marks: Philosophically our idea was we're gonna make something that 673 00:36:44,550 --> 00:36:49,140 delivers performance for everything but the very, very top there. 674 00:36:50,235 --> 00:36:50,415 W. Curtis Preston: Mm-hmm. 675 00:36:50,460 --> 00:36:57,180 Howard Marks: And goes down in cost to where Well, if you use enough 676 00:36:57,180 --> 00:37:00,660 of it, you don't need those tiers. 677 00:37:01,020 --> 00:37:02,370 You don't need the complexity. 678 00:37:04,110 --> 00:37:04,440 Right. 679 00:37:04,650 --> 00:37:09,570 So, you know, part of our story is as you consolidate workloads, you have 680 00:37:09,570 --> 00:37:13,650 workloads that need performance, and you have workloads that need capacity. 681 00:37:14,790 --> 00:37:19,350 When you add capacity, performance comes with because in, 682 00:37:19,680 --> 00:37:23,130 you know, spindles, how many SSDs? 683 00:37:24,090 --> 00:37:24,330 Yeah. 684 00:37:24,330 --> 00:37:26,130 A hundred SDS is so much performance. 685 00:37:26,130 --> 00:37:28,260 200 SDS is twice that much performance. 686 00:37:29,340 --> 00:37:34,440 So if you take the applications that need capacity, And you put them on the 687 00:37:34,440 --> 00:37:37,770 same system as the applications that need performance but don't need capacity. 688 00:37:38,280 --> 00:37:42,120 The performance that the capacity creates is used by the applications 689 00:37:42,120 --> 00:37:46,680 that need the performance and the cost of the performance is brought down 690 00:37:46,680 --> 00:37:51,480 because you've used that much capacity and you get in a vir virtuous cycle. 691 00:37:53,990 --> 00:37:55,320 W. Curtis Preston: I think I followed that. 692 00:37:56,370 --> 00:37:58,380 Prasanna Malaiyandi: yeah, it, it, it's basically 693 00:37:58,385 --> 00:37:58,680 by 694 00:37:59,020 --> 00:37:59,760 Howard Marks: if you, if 695 00:37:59,765 --> 00:38:00,110 Prasanna Malaiyandi: that's 696 00:38:00,110 --> 00:38:00,430 common. 697 00:38:00,780 --> 00:38:07,920 Howard Marks: if you, you're, paying 10 x for 10% and one x for 698 00:38:07,925 --> 00:38:11,790 90%, then you're paying a hundred. 699 00:38:13,740 --> 00:38:17,060 If you have one tier that costs a hundred, 700 00:38:18,420 --> 00:38:18,840 W. Curtis Preston: Mm-hmm. 701 00:38:19,020 --> 00:38:20,160 Howard Marks: why have two tiers? 702 00:38:20,510 --> 00:38:20,800 Prasanna Malaiyandi: Yeah. 703 00:38:22,470 --> 00:38:28,800 Howard Marks: And when you use capacity, that capacity comes with performance. 704 00:38:29,350 --> 00:38:29,770 W. Curtis Preston: Mm-hmm. 705 00:38:29,985 --> 00:38:31,995 Howard Marks: And that means that performance is available 706 00:38:31,995 --> 00:38:34,695 to other applications that didn't need the capacity. 707 00:38:36,075 --> 00:38:39,075 So you don't need to have separate systems, you just have 708 00:38:39,110 --> 00:38:42,660 W. Curtis Preston: So, so if I could, if I could try to put this 709 00:38:42,660 --> 00:38:46,350 in, in, in just different words, but it'll say the same thing. 710 00:38:46,720 --> 00:38:49,690 If I've got a hundred QLC disks, right. 711 00:38:50,290 --> 00:38:52,660 Um, and, and these are how big 712 00:38:53,875 --> 00:38:55,735 Howard Marks: 15 or 30 terabytes. 713 00:38:57,010 --> 00:38:59,170 W. Curtis Preston: the each, each one, right? 714 00:38:59,260 --> 00:38:59,770 Howard Marks: Each one 715 00:39:00,250 --> 00:39:02,710 W. Curtis Preston: So if I've got a hundred, I've got one and a half 716 00:39:02,890 --> 00:39:04,600 tear, one and a half petabytes. 717 00:39:05,350 --> 00:39:05,860 Did I 718 00:39:05,860 --> 00:39:06,220 do that 719 00:39:06,220 --> 00:39:07,900 right of raw? 720 00:39:07,960 --> 00:39:08,230 Yeah. 721 00:39:08,230 --> 00:39:08,530 Okay. 722 00:39:08,590 --> 00:39:08,950 All right. 723 00:39:09,400 --> 00:39:13,030 So I've got one and a half petabytes of raw capacity. 724 00:39:13,030 --> 00:39:17,710 And what you're saying is we can just take a slice off the top, if you will. 725 00:39:17,860 --> 00:39:20,350 You know, we used to call short stroking the discs. 726 00:39:20,350 --> 00:39:25,180 The, obviously you don't need to short stroke a, a flash, but you're 727 00:39:25,180 --> 00:39:29,080 basically saying, we're just gonna take a slice off the top, uh, of these 728 00:39:29,080 --> 00:39:35,410 150 discs and we're gonna get this massive performance slice, uh, for, 729 00:39:35,500 --> 00:39:37,990 for the 10% that need that performance. 730 00:39:38,110 --> 00:39:40,180 And then the rest will just put wherever we need to put it. 731 00:39:40,330 --> 00:39:42,070 Is that, Does that sound about 732 00:39:42,070 --> 00:39:42,370 right? 733 00:39:42,460 --> 00:39:46,660 Howard Marks: I'm, saying all those SSDs create one pool of 734 00:39:46,660 --> 00:39:49,000 performance and one pool of capacity, 735 00:39:49,600 --> 00:39:49,950 W. Curtis Preston: Right. 736 00:39:50,080 --> 00:39:52,800 Howard Marks: and a workload can draw from either one as much as it needs. 737 00:39:53,780 --> 00:39:54,200 W. Curtis Preston: Gotcha. 738 00:39:56,245 --> 00:40:02,035 Prasanna Malaiyandi: Is it separate capacity and performance pools that you 739 00:40:02,035 --> 00:40:03,185 then assign to Applic? 740 00:40:03,415 --> 00:40:03,705 Okay. 741 00:40:03,745 --> 00:40:04,255 It's just one 742 00:40:04,255 --> 00:40:05,875 pool that includes both 743 00:40:06,025 --> 00:40:09,595 Howard Marks: that, and now, you know, and you can use q o s, you can say, 744 00:40:09,595 --> 00:40:15,685 okay, this workload gets a hundred thousand iops, or, or 50 gigabytes 745 00:40:15,685 --> 00:40:18,235 per second, and this other one gets 746 00:40:18,235 --> 00:40:18,745 different. 747 00:40:19,055 --> 00:40:19,345 Prasanna Malaiyandi: yeah, 748 00:40:19,410 --> 00:40:19,700 W. Curtis Preston: yeah, 749 00:40:19,840 --> 00:40:20,060 And 750 00:40:20,320 --> 00:40:20,780 you'll just use 751 00:40:20,845 --> 00:40:21,325 Howard Marks: And so you 752 00:40:21,370 --> 00:40:21,760 W. Curtis Preston: you need to 753 00:40:22,135 --> 00:40:24,145 Howard Marks: performance, right? 754 00:40:24,835 --> 00:40:33,745 But you know, if you've got, um, you know, your backups and you've got the developers 755 00:40:33,745 --> 00:40:41,065 who wanna do run, live copies of the database, well run it all on one system. 756 00:40:41,185 --> 00:40:42,775 It's, it's an all flash system. 757 00:40:42,780 --> 00:40:44,545 It's fast enough to run the database. 758 00:40:44,855 --> 00:40:49,115 Prasanna Malaiyandi: It's almost as if you're saying, You've built a 759 00:40:49,115 --> 00:40:54,125 system that works for all workloads except that 1% or whatever, that's 760 00:40:54,125 --> 00:40:55,805 like that very, very, very high end. 761 00:40:56,165 --> 00:40:59,705 And you're saying you have one common architecture that allows it 762 00:40:59,705 --> 00:41:04,145 to deal with, regardless of if your workload is capacity focused and not 763 00:41:04,150 --> 00:41:07,925 very performance, it doesn't need a lot of performance or it's high 764 00:41:07,925 --> 00:41:09,515 performance and maybe a little capacity. 765 00:41:09,515 --> 00:41:09,845 It's all a 766 00:41:09,920 --> 00:41:12,650 Howard Marks: and and it doesn't matter whether your definition of 767 00:41:12,650 --> 00:41:14,430 performance is bandwidth or iops. 768 00:41:15,410 --> 00:41:17,930 You know, it's like all but that very lowest. 769 00:41:17,930 --> 00:41:20,990 You know, we, you know, we're an all flash system lightly loaded. 770 00:41:20,990 --> 00:41:22,880 We deliver one millisecond latency. 771 00:41:23,385 --> 00:41:23,675 Prasanna Malaiyandi: yeah, 772 00:41:23,900 --> 00:41:27,710 Howard Marks: You know, some systems can deliver half that 773 00:41:27,710 --> 00:41:29,870 and some rare applications care. 774 00:41:30,830 --> 00:41:36,380 But you know, between that and the 10, Tencent, a gigabyte, well, 775 00:41:36,380 --> 00:41:40,970 there are 20 terabyte hard drives and super micro servers and you 776 00:41:40,970 --> 00:41:44,240 know, they don't do any iops, but you can write to 'em pretty fast. 777 00:41:44,840 --> 00:41:45,050 You know, 778 00:41:45,135 --> 00:41:45,425 Prasanna Malaiyandi: Yeah. 779 00:41:45,590 --> 00:41:47,000 Howard Marks: in between we can cover. 780 00:41:47,890 --> 00:41:50,610 W. Curtis Preston: So we're dancing around. 781 00:41:52,160 --> 00:41:58,070 You're saying why you could be cheaper, but let me, 782 00:41:58,100 --> 00:42:02,300 let me just put a, lemme just put it right, you know, sort of, I'm 783 00:42:02,300 --> 00:42:07,800 assuming that you get into competitive bids with PBBAs on a regular basis. 784 00:42:08,730 --> 00:42:09,230 Howard Marks: Yes, sir. 785 00:42:09,980 --> 00:42:10,400 W. Curtis Preston: Okay. 786 00:42:10,880 --> 00:42:11,930 How do you do there? 787 00:42:14,165 --> 00:42:14,975 Howard Marks: They're easy. 788 00:42:16,235 --> 00:42:18,295 Those are very high profit margin products for 789 00:42:18,295 --> 00:42:18,455 the 790 00:42:19,025 --> 00:42:21,485 W. Curtis Preston: so you're, so you're, saying you can come in 791 00:42:21,485 --> 00:42:27,455 less expensive than the effective price of the typical P B B A, even 792 00:42:27,455 --> 00:42:29,525 though you're using all this flash. 793 00:42:29,755 --> 00:42:30,255 Howard Marks: Yes, sir. 794 00:42:31,005 --> 00:42:31,395 W. Curtis Preston: Okay. 795 00:42:31,665 --> 00:42:33,405 Because that, that's the short answer. 796 00:42:34,005 --> 00:42:35,005 I like the long answer. 797 00:42:35,505 --> 00:42:36,646 That's, I like the long answer. 798 00:42:36,665 --> 00:42:37,335 You and I 799 00:42:37,335 --> 00:42:39,105 live in long, right. 800 00:42:39,435 --> 00:42:43,785 Um, yeah, but in the end, it doesn't matter if it's still more expensive. 801 00:42:44,555 --> 00:42:47,865 Howard Marks: yeah, the, you know, the long answer is, you know, we use the 802 00:42:47,865 --> 00:42:53,655 cheapest flash we can get because we designed the system to treat flash well 803 00:42:53,985 --> 00:42:56,175 and understand how to minimize wear. 804 00:42:56,925 --> 00:43:03,095 We ha our erasure codes have 3% overhead at I at large scale, so we're not wasting. 805 00:43:04,530 --> 00:43:09,090 Space on raid, we reduce data better than anybody else does. 806 00:43:09,090 --> 00:43:12,780 So you know, we're getting as much capacity in there. 807 00:43:13,410 --> 00:43:19,320 Um, and then when you start saying, okay, it's 30 terabyte SSDs, so you get a lot 808 00:43:19,320 --> 00:43:24,270 of capacity and a little bit of space and a little bit of power, and the power 809 00:43:24,270 --> 00:43:27,420 and Rackspace start to add up as costs. 810 00:43:28,140 --> 00:43:34,020 Um, especially when you start looking at the fact that the leading PBBAs are 811 00:43:34,020 --> 00:43:39,890 still using eight terabyte hard drives because that rehydration tax of turning, 812 00:43:40,425 --> 00:43:40,815 W. Curtis Preston: Hmm. 813 00:43:41,830 --> 00:43:44,790 Howard Marks: making everything random, well the bigger the hard 814 00:43:44,790 --> 00:43:46,770 drive gets, the worse it is cause. 815 00:43:47,685 --> 00:43:49,650 One hard drive is a hundred iops. 816 00:43:49,960 --> 00:43:50,180 Prasanna Malaiyandi: Yep. 817 00:43:50,250 --> 00:43:51,870 Howard Marks: Doesn't matter whether it's a one terabyte hard 818 00:43:51,870 --> 00:43:53,340 drive or a 20 terabyte hard drive. 819 00:43:54,120 --> 00:43:56,280 And so they're just reaching the, the world. 820 00:43:56,310 --> 00:44:01,230 The land of diminishing returns on IO density. 821 00:44:01,230 --> 00:44:03,240 They can't go any lower. 822 00:44:03,900 --> 00:44:09,390 And now the sheet metal and the power supplies and the SaaS 823 00:44:10,020 --> 00:44:15,570 expanders are becoming a larger and larger percentage of their cogs. 824 00:44:17,070 --> 00:44:21,090 And they mark 'em up a lot cuz there's a lot of IP in there in terms of 825 00:44:21,095 --> 00:44:23,160 software and they have to make a margin. 826 00:44:24,060 --> 00:44:28,470 Um, and so we just don't have most of those problems. 827 00:44:30,270 --> 00:44:30,510 Prasanna Malaiyandi: Yep. 828 00:44:30,870 --> 00:44:34,500 W. Curtis Preston: you're, you're, also marking up due to your ddo, right? 829 00:44:34,500 --> 00:44:36,180 I mean, you 830 00:44:36,480 --> 00:44:36,810 Howard Marks: Yeah. 831 00:44:36,820 --> 00:44:41,310 Or you know, some of it, you know, some small portion of the difference is, you 832 00:44:41,310 --> 00:44:45,240 know, compared to the guys, those the best P VBAs, we still do a little bit better. 833 00:44:46,425 --> 00:44:52,005 but but when we go into a customer who says, no, no, no price, this as 834 00:44:52,010 --> 00:44:56,355 if you de-dupe the same, we're still coming in with a lower selling price. 835 00:44:58,275 --> 00:45:00,495 Prasanna Malaiyandi: Yeah, I, I'm not surprised about that. 836 00:45:00,495 --> 00:45:02,295 The other thing, Howard, I wanted to bring up, I know you 837 00:45:02,295 --> 00:45:06,585 mentioned sort of dis drives and the a hundred iops limit, right? 838 00:45:06,585 --> 00:45:08,145 That each of them typically have. 839 00:45:08,475 --> 00:45:13,185 The other thing that I've also seen is as the drives get larger and larger, anytime 840 00:45:13,185 --> 00:45:15,125 you have to do a raid, rebuild, right? 841 00:45:15,435 --> 00:45:19,455 And you're talking like a 20 terabyte drive and it just takes longer and longer, 842 00:45:19,455 --> 00:45:22,155 and now there's a potential for failure, 843 00:45:22,505 --> 00:45:22,795 Howard Marks: Yeah. 844 00:45:22,905 --> 00:45:23,195 Well, 845 00:45:23,655 --> 00:45:25,035 Prasanna Malaiyandi: becomes a lot worse. 846 00:45:25,405 --> 00:45:30,015 Howard Marks: you know, I do a lot of, you know, resilience calculations 847 00:45:30,735 --> 00:45:37,305 and people just don't realize how big a factor rebuild time is 848 00:45:37,575 --> 00:45:40,365 in the probability of data loss. 849 00:45:40,845 --> 00:45:43,125 Uh, we had one customer share with us. 850 00:45:43,125 --> 00:45:45,135 They ran, you know, the leading. 851 00:45:45,765 --> 00:45:53,295 Scale out system before us and the average for their rebuilds was 53 days. 852 00:45:54,305 --> 00:45:54,525 Prasanna Malaiyandi: Oh 853 00:45:56,295 --> 00:45:56,515 wow. 854 00:45:57,225 --> 00:45:58,275 W. Curtis Preston: That's two months. 855 00:45:59,175 --> 00:46:04,395 Howard Marks: yeah, that's two months during which time your data is exposed 856 00:46:05,425 --> 00:46:10,305 and you know, could be slightly exposed if you are already running. 857 00:46:10,305 --> 00:46:14,385 N plus three could be really exposed if you're running n plus 858 00:46:14,385 --> 00:46:17,505 one, like some vendors recommend. 859 00:46:18,525 --> 00:46:19,245 So it all depends. 860 00:46:20,325 --> 00:46:20,715 W. Curtis Preston: right. 861 00:46:22,245 --> 00:46:22,535 Okay. 862 00:46:22,575 --> 00:46:25,605 So, so I think, I think, you know, you, you've definitely 863 00:46:25,605 --> 00:46:27,555 covered the cost argument. 864 00:46:28,425 --> 00:46:32,925 Um, the, and, and it's, I think if we just back up, you've 865 00:46:32,930 --> 00:46:35,565 already covered the why, right? 866 00:46:35,565 --> 00:46:36,705 The why. 867 00:46:36,705 --> 00:46:38,085 would, why 868 00:46:38,085 --> 00:46:39,765 is today's Restore different? 869 00:46:41,025 --> 00:46:41,535 Howard Marks: Ransomware. 870 00:46:42,735 --> 00:46:43,155 W. Curtis Preston: Yeah. 871 00:46:43,275 --> 00:46:43,725 Howard Marks: stores. 872 00:46:43,815 --> 00:46:44,655 The stores bigger. 873 00:46:45,960 --> 00:46:48,990 And the restore location is less well known 874 00:46:49,168 --> 00:46:50,068 W. Curtis Preston: What do you mean by that? 875 00:46:50,473 --> 00:46:52,873 Howard Marks: you may not be able to restore back to the infected 876 00:46:52,873 --> 00:46:54,583 system cuz it's still evidence, 877 00:46:54,868 --> 00:46:55,288 W. Curtis Preston: okay. 878 00:46:55,678 --> 00:46:56,278 Understood. 879 00:46:56,338 --> 00:46:56,728 Okay. 880 00:46:57,103 --> 00:46:57,313 Howard Marks: right? 881 00:46:57,313 --> 00:46:59,023 You need someplace to restore to. 882 00:46:59,113 --> 00:47:04,933 And you know, having it where the primary in the backup are duped to each other 883 00:47:05,923 --> 00:47:08,413 probably gives you that in a pinch. 884 00:47:08,813 --> 00:47:09,103 Prasanna Malaiyandi: Yeah. 885 00:47:09,413 --> 00:47:09,863 Have you 886 00:47:09,863 --> 00:47:10,143 seen. 887 00:47:10,243 --> 00:47:12,223 Howard Marks: emphasize in a pinch, cuz 888 00:47:12,568 --> 00:47:12,858 W. Curtis Preston: Yeah, 889 00:47:13,303 --> 00:47:15,883 Howard Marks: you know, we've, we've all been trained, no, no bad idea. 890 00:47:15,888 --> 00:47:17,773 Don't mix the strip, don't cross the streams. 891 00:47:18,253 --> 00:47:23,173 Um, but, but that's the, if you have a backup on the primary, you 892 00:47:23,173 --> 00:47:25,183 don't, you don't have a backup. 893 00:47:25,843 --> 00:47:29,533 But if you have to choose between backup and primary, I'd rather have primary. 894 00:47:30,733 --> 00:47:34,123 Prasanna Malaiyandi: Have you seen customers actually do this 895 00:47:34,123 --> 00:47:36,703 in the field with fast systems? 896 00:47:36,928 --> 00:47:39,118 Howard Marks: Oh, we have several customers doing really 897 00:47:39,118 --> 00:47:41,458 large scale backup to Vasst. 898 00:47:42,238 --> 00:47:45,958 Um, we had one customer who was kind of shocked cuz they were 899 00:47:45,958 --> 00:47:48,898 doing encryption in net backup. 900 00:47:49,618 --> 00:47:55,768 And so they expected us to not reduce data at all, uh, but they were doing encrypted 901 00:47:55,768 --> 00:48:01,618 net backup backups of Oracle dumps of the same database over and over again, 902 00:48:01,623 --> 00:48:03,448 encrypted with the same encryption key. 903 00:48:04,228 --> 00:48:11,458 And we started seeing about 20% reduction just because even when you encrypted, if 904 00:48:11,458 --> 00:48:13,148 you're backing up the same data, it looks 905 00:48:13,468 --> 00:48:14,818 the same encrypted as it 906 00:48:15,688 --> 00:48:16,168 W. Curtis Preston: right. 907 00:48:16,223 --> 00:48:19,158 There's like sort of two questions in my head here. 908 00:48:19,998 --> 00:48:23,198 One is, and, and they're, they're very much related. 909 00:48:24,078 --> 00:48:29,658 One is the, the whole backup container problem, right? 910 00:48:29,658 --> 00:48:32,238 Meaning that you get the net backup container and the. 911 00:48:32,823 --> 00:48:37,203 Arc serve container and the backup exec container, you know, and they 912 00:48:37,203 --> 00:48:38,823 all stored backup data differently. 913 00:48:38,823 --> 00:48:40,173 And you have that issue. 914 00:48:40,863 --> 00:48:44,373 And then you, but you, there was something that you alluded 915 00:48:44,378 --> 00:48:46,443 to that I found interesting. 916 00:48:46,443 --> 00:48:49,293 You said commonality between the backup and the primary, but the 917 00:48:49,293 --> 00:48:51,283 backup is in some weirdo format. 918 00:48:52,143 --> 00:48:55,263 So are you able to get backup or commonality between the 919 00:48:55,263 --> 00:48:56,283 backup and the primary? 920 00:48:56,460 --> 00:48:57,620 Howard Marks: Now in that case, it's 921 00:48:57,620 --> 00:49:01,160 more likely we'll see commonality between multiple primaries. 922 00:49:01,730 --> 00:49:02,690 You know, it's more like you 923 00:49:02,690 --> 00:49:04,670 restored 17 windows VMs 924 00:49:04,720 --> 00:49:06,880 W. Curtis Preston: Does the way that you're doing ddo make the 925 00:49:06,885 --> 00:49:10,420 format problem any less problematic? 926 00:49:10,660 --> 00:49:11,110 Right. 927 00:49:11,305 --> 00:49:18,175 Howard Marks: O Only in that we reduce them all as opposed to if you were relying 928 00:49:18,175 --> 00:49:20,155 on the data movers to do reduction. 929 00:49:20,905 --> 00:49:26,995 So, you know, kind of the most common case is the storage guys like backup, 930 00:49:27,565 --> 00:49:31,915 you know, com Vault or net backup or Veritas or Veeam or whatever they use. 931 00:49:32,425 --> 00:49:36,925 And the Oracle DBAs don't trust them and insist on doing, doing, dumps. 932 00:49:37,750 --> 00:49:38,200 W. Curtis Preston: Right. 933 00:49:38,335 --> 00:49:42,955 Howard Marks: And so, you know, if you're doing both to, you know, they 934 00:49:42,960 --> 00:49:46,875 just give the Oracle DBAs, okay dump to this NFS mount point on the vast. 935 00:49:47,815 --> 00:49:51,775 Well then we'll reduce all of those dumps as well as anybody could reduce 936 00:49:51,775 --> 00:49:54,505 all of those dumps and your data mover. 937 00:49:54,505 --> 00:49:58,775 You'll do data reduction at multiple stages to manage the network traffic. 938 00:49:59,515 --> 00:50:05,095 And then we'll do the final dup at the end cuz we're finer grained. 939 00:50:05,095 --> 00:50:09,955 And the sim similarity works really well for things that are duped course 940 00:50:09,955 --> 00:50:14,485 grain, cuz the edges all look similar. 941 00:50:15,325 --> 00:50:15,985 And so when 942 00:50:15,985 --> 00:50:20,665 we, when we run, you know, we have a probe you can get as a VM that 943 00:50:20,815 --> 00:50:24,595 scans your data and reports back, this is how much it would reduce. 944 00:50:24,595 --> 00:50:27,835 And this is how much of that comes from each of these techniques. 945 00:50:28,525 --> 00:50:35,305 And so when we run, when we do that with data from a data mover, 946 00:50:35,305 --> 00:50:42,595 d duper, those are usually, you know, 128 K or big blocks because 947 00:50:42,600 --> 00:50:44,115 they have limited memory available. 948 00:50:45,550 --> 00:50:52,120 And so we see more similarity cuz we're finding those pieces Finer 949 00:50:52,135 --> 00:50:55,705 W. Curtis Preston: just to, just to make sure I understood correctly. 950 00:50:55,705 --> 00:51:00,715 So the, one of the question that I didn't really ask was, you know, 951 00:51:01,015 --> 00:51:06,085 when you buy a, you know, pick your favorite P V B A, they tend to support. 952 00:51:07,015 --> 00:51:10,915 These five backup products, and if you buy a different backup 953 00:51:10,915 --> 00:51:14,725 product, well, they're like, well, we don't understand that format yet. 954 00:51:15,355 --> 00:51:17,905 And so then they have to go and do some development work to figure 955 00:51:17,905 --> 00:51:19,825 out how to crack that container. 956 00:51:20,185 --> 00:51:23,005 Do you not have that problem or have you done that 957 00:51:23,020 --> 00:51:28,870 Howard Marks: We, We, have not optimized for any of these backup applications, 958 00:51:29,695 --> 00:51:30,205 W. Curtis Preston: And yet you 959 00:51:30,205 --> 00:51:32,755 still get better duped than the other guys. 960 00:51:33,355 --> 00:51:39,265 Howard Marks: Our, our general case data reduction against all of these reduced 961 00:51:39,265 --> 00:51:42,115 data types still gets better reduction. 962 00:51:42,595 --> 00:51:47,905 You know, we are not, you know, scanning for the timestamps in Oracle rack 963 00:51:47,965 --> 00:51:50,575 dumps and, you know, that level stuff. 964 00:51:51,175 --> 00:51:51,595 W. Curtis Preston: Right. 965 00:51:51,685 --> 00:51:52,075 Howard Marks: Not to 966 00:51:52,130 --> 00:51:52,810 Prasanna Malaiyandi: agnostic, right? 967 00:51:52,855 --> 00:51:57,625 Howard Marks: to say we won't in the future, but our, you know, our data 968 00:51:57,625 --> 00:51:59,785 reduction was written for primary storage. 969 00:52:00,865 --> 00:52:03,205 It just so happens that being an 970 00:52:03,210 --> 00:52:09,905 N F S or an S3 target for a backup data mover is a simple case of primary storage, 971 00:52:11,125 --> 00:52:11,335 Prasanna Malaiyandi: Yeah. 972 00:52:11,605 --> 00:52:12,205 Howard Marks: it just works. 973 00:52:13,150 --> 00:52:13,570 W. Curtis Preston: Right. 974 00:52:13,660 --> 00:52:13,990 Right. 975 00:52:15,730 --> 00:52:16,180 Hmm. 976 00:52:16,240 --> 00:52:17,490 What do you think Prasanna, 977 00:52:17,515 --> 00:52:18,295 Prasanna Malaiyandi: in the future? 978 00:52:19,955 --> 00:52:23,005 So I, I had no complaints to start with. 979 00:52:23,035 --> 00:52:24,475 Uh, the one question 980 00:52:24,760 --> 00:52:25,900 W. Curtis Preston: I lost this argument? 981 00:52:25,900 --> 00:52:26,620 I think I might 982 00:52:26,815 --> 00:52:28,945 Prasanna Malaiyandi: I think, I think you lost this one. 983 00:52:29,005 --> 00:52:35,035 Uh, Howard, the one last question I had was, I know some of these backup vendors 984 00:52:35,035 --> 00:52:39,355 support the ability to do source side due duplication by integrating with 985 00:52:39,475 --> 00:52:41,395 the purpose-built backup appliances. 986 00:52:42,205 --> 00:52:43,465 Does VAs support that? 987 00:52:43,465 --> 00:52:45,145 Are you guys planning to support that? 988 00:52:45,145 --> 00:52:46,615 I know you're looking, you just 989 00:52:46,620 --> 00:52:47,815 previously said, right, that you're 990 00:52:47,905 --> 00:52:56,245 Howard Marks: don't, we don't, um, I've never been really comfortable with the 991 00:52:56,245 --> 00:53:01,555 use of client side CPU for that cuz client side CPU is valuable for other things. 992 00:53:02,425 --> 00:53:08,155 Um, I think, you know, doing a pass at some level in the data mover. 993 00:53:08,580 --> 00:53:09,000 Prasanna Malaiyandi: Mm-hmm. 994 00:53:09,295 --> 00:53:12,535 Howard Marks: It's like, okay, we'll we'll de-dupe at the media server at some 995 00:53:12,535 --> 00:53:17,945 large grain so that we're not transferring 50 copies of windows over the network. 996 00:53:19,165 --> 00:53:22,855 Um, is perfectly reasonable thing to do cuz it's a network 997 00:53:22,855 --> 00:53:25,135 bandwidth management technique. 998 00:53:25,965 --> 00:53:32,305 Um, things like Didi Boost are, you know, let's offload this from the, 999 00:53:32,425 --> 00:53:38,365 the P B B A to the client and we'd just rather do the work ourselves. 1000 00:53:39,025 --> 00:53:44,095 Um, and in our architecture, since you can just add more servers at the front end 1001 00:53:44,755 --> 00:53:48,085 and you just have to buy the servers, we don't even charge for that software. 1002 00:53:48,715 --> 00:53:50,155 If you need more compute. 1003 00:53:50,815 --> 00:53:51,535 To do more 1004 00:53:51,565 --> 00:53:52,375 Prasanna Malaiyandi: you just scale out. 1005 00:53:52,495 --> 00:53:57,775 Howard Marks: to more dup, you just add a few more servers as opposed to stealing 5% 1006 00:53:57,775 --> 00:54:02,695 of the cycles of all of your VMware hosts, which means you now have to not just 1007 00:54:02,700 --> 00:54:05,935 buy servers, but you have to buy another VMware host, another VMware license. 1008 00:54:06,175 --> 00:54:08,675 All the other stuff you put on a VMware host starts to add up. 1009 00:54:08,875 --> 00:54:09,115 Prasanna Malaiyandi: Yeah. 1010 00:54:09,965 --> 00:54:10,385 Gotcha. 1011 00:54:10,525 --> 00:54:13,285 W. Curtis Preston: Well since, well, since you stepped into my neighborhood 1012 00:54:13,285 --> 00:54:19,585 now Howard, I will have to say that source side, DUP done correctly, 1013 00:54:20,335 --> 00:54:25,675 speeds up the backup and reduces the C P U utilization on the client. 1014 00:54:26,635 --> 00:54:29,905 But I, I want, I can't speak to the, to the implementations 1015 00:54:29,905 --> 00:54:30,835 you were talking about. 1016 00:54:31,285 --> 00:54:35,065 Um, I can only speak to the one that I am obviously very familiar with. 1017 00:54:35,545 --> 00:54:39,415 Um, cuz there, you know, there that, that is the off discussed 1018 00:54:39,420 --> 00:54:41,275 thing of like, well there is a, 1019 00:54:41,875 --> 00:54:42,115 Howard Marks: It's 1020 00:54:42,145 --> 00:54:42,745 W. Curtis Preston: know, there's a 1021 00:54:42,745 --> 00:54:42,985 pen. 1022 00:54:43,945 --> 00:54:46,375 Howard Marks: it's also a different case because of the assumed 1023 00:54:46,375 --> 00:54:47,605 bandwidth at all the stages. 1024 00:54:48,745 --> 00:54:49,075 W. Curtis Preston: right. 1025 00:54:49,225 --> 00:54:51,685 Howard Marks: You know, I'm, I'm kind of assuming that there's 1026 00:54:51,685 --> 00:54:53,755 a lot of bandwidth for short 1027 00:54:53,755 --> 00:54:55,345 distances in the data center 1028 00:54:56,095 --> 00:54:57,385 W. Curtis Preston: Well, all right. 1029 00:54:57,385 --> 00:55:03,715 I, I concede this battle, Howard, I lay down my sword. 1030 00:55:04,435 --> 00:55:05,365 Um, 1031 00:55:05,985 --> 00:55:06,275 Howard Marks: Okay. 1032 00:55:06,280 --> 00:55:06,385 I 1033 00:55:06,475 --> 00:55:08,965 W. Curtis Preston: you know, I mean, you, what's that 1034 00:55:09,255 --> 00:55:09,795 You expect 1035 00:55:10,105 --> 00:55:10,405 to what? 1036 00:55:10,825 --> 00:55:11,485 Howard Marks: in the mail, 1037 00:55:13,825 --> 00:55:17,215 W. Curtis Preston: Um, yeah, I'll send you, I'll send you something. 1038 00:55:20,365 --> 00:55:24,265 Um, alright, well, uh, Howard's been great. 1039 00:55:24,295 --> 00:55:28,795 Uh, glad to hear the update and glad to, you know, I, I remember we did, 1040 00:55:28,795 --> 00:55:32,855 now that I heard you describe it, I, I think we did cover it in the last one, 1041 00:55:32,855 --> 00:55:37,405 but I think you went deeper this time and that, that's good to hear this idea 1042 00:55:37,435 --> 00:55:37,555 Howard Marks: probably. 1043 00:55:37,585 --> 00:55:42,985 W. Curtis Preston: you can, that you have the, that you have the, the bandwidth 1044 00:55:43,435 --> 00:55:48,085 to, to, to, to how many different ways did you say you try each block 1045 00:55:49,395 --> 00:55:49,615 for 1046 00:55:49,825 --> 00:55:56,065 Howard Marks: there's five compression algorithms and, and then there's a strong 1047 00:55:56,065 --> 00:56:01,045 hash and a number of similarity hashes. 1048 00:56:01,045 --> 00:56:04,555 I can't remember offhand what they are, 1049 00:56:05,290 --> 00:56:05,710 W. Curtis Preston: Gotcha. 1050 00:56:06,190 --> 00:56:10,150 I got, I thought I heard you say 15 total ways. 1051 00:56:10,210 --> 00:56:10,630 I thought I 1052 00:56:10,630 --> 00:56:11,230 heard you say 1053 00:56:11,230 --> 00:56:11,560 that. 1054 00:56:11,795 --> 00:56:14,035 Howard Marks: it on that order 1055 00:56:14,370 --> 00:56:14,790 W. Curtis Preston: Gotcha. 1056 00:56:15,430 --> 00:56:19,540 So the fact that you can take each chunk and try 15 different ways to 1057 00:56:19,540 --> 00:56:24,250 compress it and pick the one that works the best is pretty damn cool. 1058 00:56:24,700 --> 00:56:27,160 Um, and, um, you 1059 00:56:27,160 --> 00:56:27,970 know, it's just on 1060 00:56:28,195 --> 00:56:30,475 Howard Marks: of it's just cuz we can parallelize it so well 1061 00:56:30,740 --> 00:56:31,030 Prasanna Malaiyandi: Yeah. 1062 00:56:31,150 --> 00:56:32,350 W. Curtis Preston: that too, right? 1063 00:56:32,380 --> 00:56:33,040 Uh, the 1064 00:56:33,040 --> 00:56:35,290 fact that, you can, you know, that's a. 1065 00:56:35,455 --> 00:56:36,085 Howard Marks: one takes. 1066 00:56:36,085 --> 00:56:36,835 We're doing a lot. 1067 00:56:37,720 --> 00:56:38,010 Prasanna Malaiyandi: Yeah, 1068 00:56:38,245 --> 00:56:40,885 W. Curtis Preston: it's that, that beauty of the scale out architecture, right? 1069 00:56:40,885 --> 00:56:43,795 That you can just pass that out, um, like that. 1070 00:56:44,845 --> 00:56:50,245 All right, well, thanks for coming back, especially to, it's really funny 1071 00:56:50,245 --> 00:56:54,205 how we had you, you're like, you're listening and you're like, Hey, you said 1072 00:56:54,210 --> 00:56:56,495 mean things about the way I do things. 1073 00:56:57,325 --> 00:56:59,395 I will, I accept your challenge. 1074 00:56:59,395 --> 00:57:01,075 And I'm like, all right, come on back. 1075 00:57:01,765 --> 00:57:03,115 Uh, happy to do that. 1076 00:57:03,535 --> 00:57:04,855 And we'll do that with other people. 1077 00:57:04,860 --> 00:57:07,795 By the way, if you're out there listening and you're like, the thing that Curtis or 1078 00:57:07,795 --> 00:57:14,515 Prasanna said is wrong, we will be happy to have you on and have us prove to you 1079 00:57:14,520 --> 00:57:18,505 why you're wrong or, or in this case, 1080 00:57:19,450 --> 00:57:21,760 In this case, uh, right. 1081 00:57:21,760 --> 00:57:23,080 Yeah, we concede. 1082 00:57:23,170 --> 00:57:27,250 Well, I mean, I mean, I think my concerns are certainly valid and 1083 00:57:27,250 --> 00:57:31,060 there are certainly vendors out there that are like, yes, we can 1084 00:57:31,065 --> 00:57:35,050 certainly sell you this appliance for the purposes of backup, because 1085 00:57:35,050 --> 00:57:36,880 recovery speed is really important. 1086 00:57:36,885 --> 00:57:42,460 I'm like, but it costs five times the cost of this thing over here. 1087 00:57:42,670 --> 00:57:47,050 I don't like how much better could it possibly be Anyway, 1088 00:57:47,050 --> 00:57:49,660 so that's, that's where those arguments tend to come from, so. 1089 00:57:49,780 --> 00:57:51,160 Alright, well thanks. 1090 00:57:51,400 --> 00:57:52,120 Thanks Howard for 1091 00:57:52,285 --> 00:57:54,355 Howard Marks: and given, given the marketplace, they're 1092 00:57:54,355 --> 00:57:55,795 not completely unreasonable. 1093 00:57:56,995 --> 00:58:00,985 Uh, you know, we, we just do things sufficiently different that, 1094 00:58:01,165 --> 00:58:04,405 you know, if you think restore speed's important than we do, 1095 00:58:05,310 --> 00:58:05,940 W. Curtis Preston: right, right. 1096 00:58:06,255 --> 00:58:06,475 Prasanna Malaiyandi: Yep. 1097 00:58:07,240 --> 00:58:07,690 W. Curtis Preston: Yeah. 1098 00:58:07,780 --> 00:58:08,170 Yeah. 1099 00:58:08,175 --> 00:58:08,700 Ransomware. 1100 00:58:08,700 --> 00:58:09,260 Ransomware. 1101 00:58:09,790 --> 00:58:10,540 All right. 1102 00:58:10,600 --> 00:58:15,070 Once again, ransomware, you know, I don't know. 1103 00:58:15,070 --> 00:58:15,910 What do you, what do you call it? 1104 00:58:15,970 --> 00:58:21,730 Uh, trump's all, um, although I don't enjoy that word as much 1105 00:58:21,730 --> 00:58:23,080 as I used to for some reason. 1106 00:58:23,470 --> 00:58:27,370 Um, anyways, thanks for, thanks for your questions for, 1107 00:58:30,475 --> 00:58:31,825 Prasanna Malaiyandi: Uh, I try. 1108 00:58:31,825 --> 00:58:32,515 I try. 1109 00:58:32,515 --> 00:58:34,165 And Howard, great to have you back on the podcast. 1110 00:58:34,195 --> 00:58:35,445 Hopefully you'll come again. 1111 00:58:35,635 --> 00:58:36,655 Howard Marks: Always pleasure. 1112 00:58:36,805 --> 00:58:39,085 As long as I keep winning, I'll keep coming back. 1113 00:58:40,810 --> 00:58:42,670 W. Curtis Preston: and we thank you to our listeners. 1114 00:58:42,670 --> 00:58:44,860 Uh, you know, we're nothing without you. 1115 00:58:45,040 --> 00:58:48,670 Remember to subscribe so that you can restore it all.