1 00:00:00,160 --> 00:00:03,880 I grew up in a mining town, and I don't recommend 2 00:00:03,880 --> 00:00:07,320 that actually, as a place to grow up. Got 3 00:00:07,320 --> 00:00:11,120 partially homeschooled by my mom and dad because the quality 4 00:00:11,120 --> 00:00:14,800 of the local schools was not what they wanted to see. I 5 00:00:14,800 --> 00:00:18,320 saw resource production up close and personal. I saw data 6 00:00:18,400 --> 00:00:22,080 coming in up close and personal before anybody knew what to do with it 7 00:00:23,120 --> 00:00:25,840 or how to use it or analyze it at all. 8 00:00:26,900 --> 00:00:30,540 Statistics on operations, on interruptions, what 9 00:00:30,540 --> 00:00:34,300 causes machines to go down and be unavailable for all 10 00:00:34,300 --> 00:00:37,380 these kinds of things. Data can help us understand that. 11 00:00:47,380 --> 00:00:50,900 Hello, and welcome back to Data Driven, the podcast. We explore the 12 00:00:50,900 --> 00:00:54,750 emerging industry of AI, data science, and, of course, data 13 00:00:54,750 --> 00:00:58,550 engineering. You may notice that my favorite data engineer in the world, 14 00:00:58,550 --> 00:01:01,670 Andy Leonard, is not here. However, I brought the most quant, 15 00:01:02,070 --> 00:01:05,830 most quantum curious person I know. I don't like calling her curious, although 16 00:01:06,950 --> 00:01:10,470 one could argue. And Candace Cooley. How's it going, 17 00:01:10,470 --> 00:01:13,710 Candice? It's great. Thank you so much. I'm really excited to be part of Data 18 00:01:13,710 --> 00:01:17,550 Driven today. Awesome. We're happy to have you. And we're also very 19 00:01:17,550 --> 00:01:20,070 happy to have Mr. Dale Nesbitt, who is, 20 00:01:21,910 --> 00:01:25,690 in addition to being at Arrowhead Economics, he's also a 21 00:01:25,690 --> 00:01:29,090 lecturer at Stanford, and I guess he has his first summer class today, 22 00:01:29,730 --> 00:01:32,690 and we're happy to have him. Welcome to the show, Dale. 23 00:01:33,650 --> 00:01:37,090 Thank you. I appreciate it. And thanks for the opportunity to speak with you. 24 00:01:37,730 --> 00:01:41,290 No problem. No problem. What exactly does 25 00:01:41,290 --> 00:01:45,130 Arrowhead Economics do? It's implicit in the name 26 00:01:45,130 --> 00:01:48,610 we do economics. We named after 27 00:01:48,610 --> 00:01:52,450 ourselves after the second Nobel laureate in economics, 28 00:01:52,450 --> 00:01:56,210 Kenneth Arrow, who was riding his bike around Stanford 29 00:01:56,210 --> 00:01:59,890 happily until his 96th birthday. And then. Then we 30 00:01:59,890 --> 00:02:03,170 lost him about five years ago. So we do 31 00:02:03,170 --> 00:02:07,010 economics in the energy patch, critical materials 32 00:02:07,010 --> 00:02:10,610 patch, any commodity that's. That's 33 00:02:10,610 --> 00:02:14,250 produced or traded. Interesting. And that's a. 34 00:02:14,410 --> 00:02:18,130 That's a very large field. I don't think. If were it 35 00:02:18,130 --> 00:02:21,980 not for Eddie Murphy's movie Trading Places, most 36 00:02:21,980 --> 00:02:25,420 people probably wouldn't know a thing about it. I was a kid when that came 37 00:02:25,420 --> 00:02:29,220 out, and I was just fascinated. So sorry. I'm sure that's not the first 38 00:02:29,220 --> 00:02:32,500 time you've heard that, and I hate to tell you, it's probably not gonna be 39 00:02:32,500 --> 00:02:35,780 the last time you've heard that. Well, yeah, I've heard that. 40 00:02:36,900 --> 00:02:40,340 I had an auspicious start. I grew up in a mining town. 41 00:02:41,140 --> 00:02:44,940 Okay. I don't recommend that, actually, as a place to 42 00:02:44,940 --> 00:02:48,580 grow up. Got partially homeschooled by 43 00:02:48,580 --> 00:02:52,380 my mom and dad because the quality of the local schools was not what 44 00:02:52,380 --> 00:02:56,080 they want. I saw resource production up 45 00:02:56,080 --> 00:02:59,600 close and personal. I saw data coming in up close and personal 46 00:02:59,600 --> 00:03:03,440 before anybody knew what to do with it or how to use 47 00:03:03,440 --> 00:03:07,200 it or analyze it at all. Statistics on 48 00:03:07,200 --> 00:03:10,520 operations, on interruptions, what causes 49 00:03:10,520 --> 00:03:14,280 machines to go down and be unavailable for all these kinds of 50 00:03:14,280 --> 00:03:16,800 things. Data can help us understand that. 51 00:03:17,920 --> 00:03:21,480 Interesting. So you were the data. Life really found you? It sounds. 52 00:03:21,480 --> 00:03:25,250 Yeah. Yes and no. Yes. My background is actually in 53 00:03:25,250 --> 00:03:28,810 probability and decision analysis. That's where I did my research. 54 00:03:29,610 --> 00:03:33,250 And what. One of the things that you're going to find the data is really 55 00:03:33,250 --> 00:03:36,810 good for is developing probability distributions over 56 00:03:36,890 --> 00:03:40,410 phenomena that you don't understand the uncertainty about. 57 00:03:41,450 --> 00:03:44,970 We want to understand the uncertainty about certain phenomena. 58 00:03:45,050 --> 00:03:48,900 Data is a way to do that. Interesting. Sorry, 59 00:03:48,900 --> 00:03:52,660 Candice, I cut you off. No, I was curious. So many organizations, they 60 00:03:52,660 --> 00:03:56,500 collect enormous amounts of data, but why 61 00:03:56,500 --> 00:04:00,220 do so few of them seem to not be making better 62 00:04:00,220 --> 00:04:03,460 decisions based upon their data? As a professor of decision 63 00:04:03,540 --> 00:04:07,220 analysis, you need more than data to make decisions. 64 00:04:08,100 --> 00:04:11,540 You need alternatives. You need information, probability 65 00:04:11,620 --> 00:04:15,420 distributions, not data. And you need a notion of your 66 00:04:15,420 --> 00:04:18,900 values, your objectives, what do you like and what do you not like. So if 67 00:04:18,900 --> 00:04:22,640 you're coming like profits, there's a lot of uncertainty 68 00:04:22,640 --> 00:04:26,320 in the middle. If you're producing oil, there's a probability distribution 69 00:04:26,320 --> 00:04:29,920 over oil price, over supply chain costs and those things. 70 00:04:30,320 --> 00:04:33,520 You can't make a good decision unless you have those probability 71 00:04:33,600 --> 00:04:36,800 distributions. I get them in part from data. 72 00:04:37,440 --> 00:04:41,160 The data is out there telling you what these probability distributions 73 00:04:41,160 --> 00:04:44,880 are, if you know how to process it. The reason is because 74 00:04:44,880 --> 00:04:48,510 the notion data driven decisions is a non 75 00:04:48,510 --> 00:04:52,150 sequitur. It really is. Data is not 76 00:04:52,150 --> 00:04:55,310 enough to drive your decisions. You need intelligence, 77 00:04:56,270 --> 00:04:59,870 you need an understanding of uncertainty. What's the on 78 00:04:59,870 --> 00:05:03,150 average value of these variables that you're going to see? 79 00:05:03,469 --> 00:05:07,030 How much spread is there in those variables? You can't make a decision under 80 00:05:07,030 --> 00:05:09,870 uncertainty unless you understand the uncertainty. 81 00:05:10,750 --> 00:05:14,590 So data driven is not data driven, it's analytic driven. And 82 00:05:14,590 --> 00:05:18,300 the data helps you. If you, some of the techniques that, that 83 00:05:18,300 --> 00:05:22,020 we use in practice get what those uncertainties are 84 00:05:22,020 --> 00:05:25,860 to feed into a decision analysis 85 00:05:26,180 --> 00:05:28,660 framework, then you make good decisions. 86 00:05:30,100 --> 00:05:33,780 Yeah, I mean, that makes sense, right? Because data would be the raw 87 00:05:33,780 --> 00:05:37,500 commodity, if you will, the raw oil. But the thing that 88 00:05:37,500 --> 00:05:41,180 you put in your car, gasoline, petrol, whatever you want to call it, 89 00:05:41,180 --> 00:05:44,810 is the refined product from the raw material, you know, Frank? 90 00:05:44,810 --> 00:05:48,610 Absolutely. Data is the crude oil and 91 00:05:48,930 --> 00:05:52,290 the probability distribution of the finished product, gasoline, 92 00:05:52,290 --> 00:05:55,650 distillate, so forth that you put into your machinery that makes it go 93 00:05:55,650 --> 00:05:58,450 absolutely Data is the raw material. 94 00:05:59,330 --> 00:06:03,090 And one of the problems there, a Candace, pursuant to your remark, 95 00:06:03,570 --> 00:06:07,090 is this is Nesbitt's maxim number three, people 96 00:06:07,090 --> 00:06:09,170 only gather data that's easy to gather. 97 00:06:11,020 --> 00:06:14,780 And maximum number two is the process of gathering 98 00:06:14,780 --> 00:06:16,780 data is itself stochastic. 99 00:06:18,300 --> 00:06:21,820 Watch how data is gathered. It's a noisy, noisy process 100 00:06:22,460 --> 00:06:26,020 to observe anything and to write down the 101 00:06:26,020 --> 00:06:29,780 correct, say, operation of machine. We 102 00:06:29,780 --> 00:06:33,180 don't even know what the price of crude oil is because everybody reports 103 00:06:33,340 --> 00:06:37,150 it differently. So data itself isn't 104 00:06:37,150 --> 00:06:40,750 gathering itself is noisy. You have to take that into account when you 105 00:06:40,750 --> 00:06:44,590 analyze it. So this idea that data is. It's like 106 00:06:44,590 --> 00:06:48,430 Rumpelstiltskin, right? Spin straw into gold. You can't. 107 00:06:48,430 --> 00:06:51,790 The data is not straw. It has to be scrupulously 108 00:06:51,790 --> 00:06:55,430 worked. Frank, just exactly what you alluded to to get 109 00:06:56,550 --> 00:06:59,430 proper decision making, to get proper 110 00:06:59,510 --> 00:07:02,130 forecasting and those things. 111 00:07:03,330 --> 00:07:06,930 Well, then doesn't that explain where a lot of data projects or 112 00:07:07,090 --> 00:07:10,850 AI projects go wrong, is because they don't take into account the random. 113 00:07:11,250 --> 00:07:15,090 The randomness. Because you said data collection is stochastic. Right. So 114 00:07:15,410 --> 00:07:19,010 it is. It's inherently unpredictable. 115 00:07:19,010 --> 00:07:22,690 Right. And doesn't that kind of echo throughout the training of these models 116 00:07:22,690 --> 00:07:26,050 that we have and cause weird outliers? 117 00:07:26,620 --> 00:07:30,260 Or does is there magic that can happen that can cancel that 118 00:07:30,260 --> 00:07:33,740 out? Yeah, there's. Well, there's magic. You try to measure 119 00:07:34,300 --> 00:07:38,020 through observation the intrinsic uncertainty in the data, like 120 00:07:38,020 --> 00:07:41,740 the difference between predicted and actual, maybe due 121 00:07:41,740 --> 00:07:45,180 to measurement error, just observing the data. And 122 00:07:45,500 --> 00:07:49,060 the great statistician Fisher told us that data was 123 00:07:49,060 --> 00:07:52,910 generated by a stochastic process. And what 124 00:07:52,910 --> 00:07:56,710 you're trying to do is figure out what that stochastic process must 125 00:07:56,710 --> 00:08:00,430 have been very subtle. So you, 126 00:08:00,430 --> 00:08:04,030 you, at the same time you're estimating, say, coefficients for your 127 00:08:04,030 --> 00:08:07,590 model or something you want to estimate, you're also estimating what 128 00:08:08,070 --> 00:08:11,350 stochastic process was at work when the data were 129 00:08:11,350 --> 00:08:14,070 gathered. And you can infer that 130 00:08:15,350 --> 00:08:18,310 it's very sophisticated. AI doesn't do any of that. 131 00:08:18,950 --> 00:08:22,440 Regression analysis does that. AI 132 00:08:22,440 --> 00:08:26,280 doesn't deal with uncertainty at all. No, 133 00:08:26,280 --> 00:08:29,480 it actually gets really wonky when it's uncertain. 134 00:08:30,360 --> 00:08:33,080 Yeah, wonky being a technical term, of course. 135 00:08:35,480 --> 00:08:39,000 But I think it's really important when you think of data analysis. You want to 136 00:08:39,000 --> 00:08:42,840 be careful to try to measure intrinsically what sort of 137 00:08:42,840 --> 00:08:46,400 stochastic variation, what sort of process by which people 138 00:08:46,400 --> 00:08:50,020 observe data. What was it? When we 139 00:08:50,020 --> 00:08:53,780 do regression analysis, simple linear regression is a term, 140 00:08:53,780 --> 00:08:57,580 sigma in there, and that sigma has to do with the random nature 141 00:08:57,580 --> 00:09:01,220 of the data. So 142 00:09:01,220 --> 00:09:04,700 interesting. So, yeah, it really goes back to the fundamentals of 143 00:09:04,700 --> 00:09:08,380 statistics, doesn't it? Yeah. I mean, think of the, you know, 144 00:09:08,380 --> 00:09:11,900 the clock's always right. Suppose you're measuring the time 145 00:09:12,460 --> 00:09:16,080 and you fail to notice that the clock broke. 146 00:09:16,400 --> 00:09:20,080 And it gives you the same time all the time. Your sensor 147 00:09:20,080 --> 00:09:23,680 broke and it gave you the same reading, no matter what the state of the 148 00:09:23,680 --> 00:09:27,400 machinery was. Your data set ain't too good with 149 00:09:27,400 --> 00:09:31,079 that in there. And yet, wow. No matter what, I get the 150 00:09:31,079 --> 00:09:34,880 same answer. That's really predictive. Fact is, it's a broken 151 00:09:34,880 --> 00:09:38,320 data machine. Right. So you get just 152 00:09:38,560 --> 00:09:42,400 industrial measurements. Your, your process of measuring, 153 00:09:42,400 --> 00:09:46,040 be it manual or sometimes be it automated, you have to 154 00:09:46,040 --> 00:09:49,660 be very, very cognizant of 155 00:09:49,660 --> 00:09:51,580 that and the stochastics of that 156 00:09:53,660 --> 00:09:57,500 and take that into account when you try to understand what that data 157 00:09:57,500 --> 00:10:00,700 implies about your probability distribution over something 158 00:10:02,300 --> 00:10:06,060 interesting. Candice looks like she's 159 00:10:06,060 --> 00:10:09,820 going to say something. I'm thinking about uncertainty, 160 00:10:09,820 --> 00:10:13,660 and I'm also thinking about risk. And I'm trying to understand 161 00:10:14,060 --> 00:10:17,680 what's the distinction that matters between risk and 162 00:10:17,680 --> 00:10:21,240 uncertainty. Well, it's, it's kind of interesting. So suppose 163 00:10:21,240 --> 00:10:24,960 that. Let's take the price of gold. You're going to open a gold mine. Price 164 00:10:24,960 --> 00:10:28,480 of gold right now is $4200 an ounce. It's 165 00:10:28,480 --> 00:10:31,560 infinity minus three. It's really, really high. 166 00:10:32,280 --> 00:10:35,000 Wars and things like that caused that to happen. 167 00:10:35,880 --> 00:10:39,640 But you're uncertain about it. But if you're going to open 168 00:10:39,640 --> 00:10:43,080 that gold mine and it's profitable at a gold price of 169 00:10:43,160 --> 00:10:46,600 fifteen hundred dollars an ounce, there's uncertainty, but there' 170 00:10:48,050 --> 00:10:51,450 risk has to do with loss. So suppose you were 171 00:10:51,450 --> 00:10:55,130 uncertainty. One day I grew up in Reno. One day I walked into the 172 00:10:55,130 --> 00:10:58,930 nugget, I was underage, and I inadvertently pulled the handle on the 173 00:10:58,930 --> 00:11:02,770 slot machine and it played. You didn't have to put money 174 00:11:02,770 --> 00:11:06,130 in it and it would just play. It was broken. 175 00:11:06,610 --> 00:11:10,170 That's a pretty good lottery. I played it until the security guard came and 176 00:11:10,170 --> 00:11:13,530 kicked me out. Right. So there was 177 00:11:13,530 --> 00:11:17,050 uncertainty. I didn't know if it was going to give me cherries or clowns or 178 00:11:17,050 --> 00:11:20,590 whatever, but there was no risk. I didn't have anything at risk. 179 00:11:21,070 --> 00:11:24,830 Risk happens when you have losses that have to be balanced against 180 00:11:24,830 --> 00:11:28,510 gains. Now all of a sudden, your elementary tech 181 00:11:28,510 --> 00:11:32,190 gets a lot tighter when you have losses. I had to put 182 00:11:32,270 --> 00:11:35,990 a quarter in that machine. And so if it didn't come up, cherries 183 00:11:35,990 --> 00:11:39,710 are better. My quarter was gone. So risk has to do 184 00:11:39,710 --> 00:11:43,390 with the probability of loss and how you, 185 00:11:43,390 --> 00:11:46,950 you trade that off with Your preferences against probability of 186 00:11:46,950 --> 00:11:50,700 gain and uncertainty just has to do with how 187 00:11:50,700 --> 00:11:54,140 sure are you. If you had to do an over under on your 188 00:11:54,140 --> 00:11:57,780 profitability, what's the 50, 50 point 189 00:12:00,020 --> 00:12:03,860 risk estimate? That makes a lot of sense. Yeah, that makes a lot of 190 00:12:03,860 --> 00:12:06,580 sense. That's interesting. But 191 00:12:08,180 --> 00:12:11,860 what's your take on. Obviously the commodity that everyone 192 00:12:12,660 --> 00:12:15,860 is most impacted by, at least in the obvious sense, is oil. 193 00:12:16,490 --> 00:12:18,810 Crude oil. Yep, Crude oil. So 194 00:12:20,170 --> 00:12:23,770 I've noticed obviously there's been some, you know, the elephant in the room, there's been 195 00:12:23,770 --> 00:12:27,570 a lot of instability in that, that space, but somebody had said something and 196 00:12:27,570 --> 00:12:30,970 I didn't quite get it, is that the oil 197 00:12:30,970 --> 00:12:34,650 contracts are down a decade out or five to ten years 198 00:12:34,650 --> 00:12:38,330 out. Yeah. So the fluctuation in 199 00:12:38,330 --> 00:12:42,090 prices that we see at the gas pump have more 200 00:12:42,090 --> 00:12:43,610 to do with 201 00:12:45,600 --> 00:12:49,360 speculation. Is that true? No, Did I mishear it? 202 00:12:49,600 --> 00:12:52,640 I don't think it's true. This has to do with the short term supply demand 203 00:12:52,720 --> 00:12:56,360 balance. When the strait of Horamuz was closed, you had 10 204 00:12:56,360 --> 00:13:00,160 million barrels a day less supply at the gate of 205 00:13:00,160 --> 00:13:04,000 the straight of horror moves. So draw yourself a supply 206 00:13:04,000 --> 00:13:07,680 curve and a demand curve and shift that supply curve 10 million 207 00:13:07,680 --> 00:13:11,280 barrels of the day to the left. Your price is going to go high. 208 00:13:13,770 --> 00:13:17,490 Yeah. No matter what we have. And I do this and 209 00:13:17,490 --> 00:13:21,290 there's a different kind of data I'd like to chat about. We have a world 210 00:13:21,290 --> 00:13:25,090 oil model, multi regional world oil model. We forecast this stuff for 211 00:13:25,090 --> 00:13:28,730 the industry short term and long term. Okay. And 212 00:13:28,730 --> 00:13:32,250 so short term is different from the long term. All the oil that's going to 213 00:13:32,250 --> 00:13:36,090 be produced is sitting right there at the wellhead and you have to have 214 00:13:36,090 --> 00:13:39,610 logistics to get it to market. So the supply curve and the demand curve and 215 00:13:39,610 --> 00:13:43,130 the short term are fixed. In the long term they're not fixed. 216 00:13:43,290 --> 00:13:46,970 People can invest capital and go produce some tar sands in 217 00:13:46,970 --> 00:13:50,650 Venezuela or produce some more oil in the Middle East. 218 00:13:51,450 --> 00:13:55,170 And so longer term price effects tend to look a lot different 219 00:13:55,170 --> 00:13:57,930 than shorter term price effects. But both are uncertain. 220 00:13:59,050 --> 00:14:02,490 Both are. So if we gather data on some 221 00:14:02,490 --> 00:14:06,010 phenomenon, if it affects long term prices, 222 00:14:06,410 --> 00:14:10,000 then that data will have a stochastic effect. Right. 223 00:14:10,080 --> 00:14:13,560 It helps you understand them better, helps you reduce your, your 224 00:14:13,560 --> 00:14:17,360 uncertainty of what those prices are going to be. The more data that you gather, 225 00:14:17,440 --> 00:14:20,880 the more definitive, I. E. The lower variance your look. 226 00:14:22,720 --> 00:14:26,480 So a lot of these comments that you hear, they give me 227 00:14:26,480 --> 00:14:29,280 hives. I thought I was back in the 1970s. 228 00:14:30,080 --> 00:14:33,760 The stupidity in these comments, number 229 00:14:33,840 --> 00:14:37,680 one, the biggest stupidity in the world, is that the future price 230 00:14:38,640 --> 00:14:42,440 is an extrapolation of the past price. And people do 231 00:14:42,440 --> 00:14:46,080 that statistically, it's dead wrong. Economics teaches us that 232 00:14:46,400 --> 00:14:50,000 the future occurs because of future 233 00:14:50,000 --> 00:14:53,600 supply and demand has absolutely nothing to do with the past. 234 00:14:54,640 --> 00:14:58,480 So AI is going to have a really hard time with any economic 235 00:14:58,640 --> 00:15:02,340 problem because the future price that you're going to 236 00:15:02,340 --> 00:15:06,140 see, say right at the gate of the straight of horror moves, it's going to 237 00:15:06,140 --> 00:15:09,420 be a function of what's happening then, not what happened now. 238 00:15:10,220 --> 00:15:14,020 People will speculate into the future. They will solve. They will sign 239 00:15:14,020 --> 00:15:17,620 a long term buy or sell contract. I'll give you a million 240 00:15:17,620 --> 00:15:21,380 barrels a day in 2040, okay, I 241 00:15:21,380 --> 00:15:25,190 better have that crude oil in 2040 for you and 242 00:15:25,350 --> 00:15:29,150 you'll sign a contract that I'll buy it. They sell, buy and sell 243 00:15:29,150 --> 00:15:32,870 futures. That just makes the price more and more available 244 00:15:33,190 --> 00:15:34,870 and visible to everybody. 245 00:15:37,590 --> 00:15:41,430 Speculation is a good thing. It's a very, very good thing. 246 00:15:41,750 --> 00:15:44,230 The best thing is speculation trading. 247 00:15:45,350 --> 00:15:48,790 Because it takes risk away from the producers, right? Or no, 248 00:15:49,830 --> 00:15:53,140 it shows you the price. If you watch how people make these 249 00:15:53,140 --> 00:15:56,940 trades, if all of them are made at an implicit price of $80 a 250 00:15:56,940 --> 00:16:00,220 barrel, there's your best guess at the price. Speculation 251 00:16:00,380 --> 00:16:03,900 shows us the price. So this idea that speculation 252 00:16:03,980 --> 00:16:07,820 is bad is really stupid. It's so stupid. 253 00:16:08,140 --> 00:16:11,780 You want more and more and more and more trading. So everybody knows the 254 00:16:11,780 --> 00:16:15,100 price. So the price in the fair, the free market is known to all. 255 00:16:16,700 --> 00:16:20,410 Interesting. That's why people 256 00:16:20,410 --> 00:16:24,170 trade, right? So you don't have to, you 257 00:16:24,170 --> 00:16:27,570 don't have to be data driven about the future price. It's transactional. 258 00:16:28,370 --> 00:16:32,090 Right? Okay. We have short 259 00:16:32,090 --> 00:16:35,730 term disruptions, there's not enough trading. And 260 00:16:35,730 --> 00:16:39,170 until a point, because you're trading on the probability 261 00:16:39,330 --> 00:16:43,050 that the straight remains open versus vibrates 262 00:16:43,050 --> 00:16:45,410 between open and closed versus closed. 263 00:16:47,790 --> 00:16:51,150 And people will speculate that. Thank God for 264 00:16:51,150 --> 00:16:54,030 speculators. Interesting. 265 00:16:55,230 --> 00:16:58,750 What's your take on these poly market type 266 00:16:59,710 --> 00:17:03,230 marketplaces? What's your take on that? 267 00:17:03,950 --> 00:17:07,390 People are speculating. You do? Okay, I love it, I love it. 268 00:17:07,790 --> 00:17:11,590 I think people are betting their probability assessment against yours. They 269 00:17:11,590 --> 00:17:15,390 think they got a better probability distribution of something over yours and 270 00:17:15,390 --> 00:17:18,970 they put their money where their mouth is, they trade on it. And if you 271 00:17:18,970 --> 00:17:22,810 look at the volume of trades that people lay down, that gives 272 00:17:22,810 --> 00:17:26,450 you some notion of the cons, consensus probability of those events 273 00:17:26,450 --> 00:17:29,530 occurring. So the what like on the election? 274 00:17:29,690 --> 00:17:33,370 Speculating on the election. I love those things. I don't play them, but I love 275 00:17:33,370 --> 00:17:37,010 them. Sportsbook is that sports books are 276 00:17:37,010 --> 00:17:40,770 great. They're so much fun to play too. 277 00:17:40,770 --> 00:17:43,770 Bad they rake off so much of the pot as their fee. 278 00:17:45,290 --> 00:17:48,970 But you can watch the data on those and I think 279 00:17:48,970 --> 00:17:52,290 the data suggests that they're very pretty. Pretty 280 00:17:52,290 --> 00:17:55,770 predictive of what happens in a probabilistic sense. 281 00:17:57,050 --> 00:18:00,730 No, I remember one of the. There's a paper from a long 282 00:18:00,730 --> 00:18:04,570 time ago and basically discussed how if you remember the show 283 00:18:04,570 --> 00:18:08,290 who Wants to Be a Millionaire, the crowd, when you ask the crowd a question, 284 00:18:08,290 --> 00:18:11,610 the crowd was more accurate than any of the other mechanisms 285 00:18:12,460 --> 00:18:16,140 that they had, which was phone of friends pass 286 00:18:16,140 --> 00:18:19,500 and something else. But the 287 00:18:19,660 --> 00:18:23,500 crowd actually got the answer right. Far and above. 288 00:18:24,060 --> 00:18:27,859 Yeah. That book, the Wisdom of Crowds. That was it. 289 00:18:27,859 --> 00:18:31,580 Yeah. And it's very good. See, the market is the ultimate wisdom of 290 00:18:31,580 --> 00:18:35,380 crowds. The crude oil market is so heavily traded. There's 291 00:18:35,380 --> 00:18:39,140 no misinformation in that price. There's even a 292 00:18:39,140 --> 00:18:42,420 theory which I subscribe to, that there's nothing the matter with insider 293 00:18:42,420 --> 00:18:46,180 trading. It gets more information into the price and 294 00:18:46,180 --> 00:18:49,820 more correct information into the price. It's 295 00:18:49,820 --> 00:18:53,100 illegal because people say it's unfair. It's. 296 00:18:53,260 --> 00:18:57,100 It's capitalizing on. On insider knowledge and all that. 297 00:18:57,100 --> 00:19:00,660 I'm not so sure I buy that. I want the. When I buy somebody's 298 00:19:00,660 --> 00:19:04,180 stock like SpaceX, I want all the insider and 299 00:19:04,180 --> 00:19:08,020 outsider knowledge involved in my decision whether or 300 00:19:08,020 --> 00:19:11,740 not to purchase that st. And it wasn't all 301 00:19:11,740 --> 00:19:15,140 the insider knowledge was concealed from the market by law. 302 00:19:16,820 --> 00:19:20,540 Yeah, no, that. It's funny you mentioned that because insider trading 303 00:19:20,540 --> 00:19:24,180 has only been illegal since the Great 304 00:19:24,180 --> 00:19:27,940 Depression. Yeah, I remember one of the 305 00:19:27,940 --> 00:19:31,660 most interesting books. It was called the Patriarch and it was 306 00:19:31,660 --> 00:19:34,340 basically about Joe Kennedy. Oh yeah. And 307 00:19:35,220 --> 00:19:38,860 FDR put him in charge of that. But he was a notorious insider 308 00:19:38,860 --> 00:19:42,710 trader. Oh yeah. As you would point out later on, it wasn't 309 00:19:42,710 --> 00:19:46,430 illegal then. Right? It wasn't illegal. It's not even. It's not 310 00:19:46,430 --> 00:19:49,830 illegal in Congress today. Well, yeah, 311 00:19:49,830 --> 00:19:53,030 exactly. I mean there are people who are financial 312 00:19:53,110 --> 00:19:56,910 geniuses. This is crazy. I mean, really. Yeah. These 313 00:19:56,910 --> 00:20:00,390 guys have confident insider information, it's 314 00:20:00,390 --> 00:20:04,070 confidentially delivered. Confidentially. And you can go off 315 00:20:04,070 --> 00:20:07,860 and trade on it. Come on, guys. It's 316 00:20:07,860 --> 00:20:11,580 not a practice that be condoned. Yeah. Shutting down insider 317 00:20:11,580 --> 00:20:14,420 information is about fairness. Then they should be included here. 318 00:20:15,380 --> 00:20:19,020 But we're a little far afield. Getting back to data science, there's a couple of 319 00:20:19,020 --> 00:20:22,820 other things that. One of my. One of my favorite 320 00:20:22,820 --> 00:20:26,420 themes. I saw this during COVID go back to the beginning of COVID 321 00:20:26,980 --> 00:20:30,500 How much data was there on what was curative 322 00:20:30,580 --> 00:20:34,300 or what was ameliorating of symptoms about any treatment 323 00:20:34,300 --> 00:20:37,890 we had, there was no data. Not a lot, but 324 00:20:38,210 --> 00:20:41,890 there was a lot that was suppressed. If it's suppressed, it 325 00:20:41,890 --> 00:20:45,570 doesn't exist. Okay. If you can't analyze with it, 326 00:20:45,570 --> 00:20:48,850 then there is no data. It's like if I flip a coin and put a 327 00:20:48,850 --> 00:20:51,810 piece of paper over it and ask you to call the coin, 328 00:20:52,930 --> 00:20:56,730 it's still 50, 50 to you. I can look under that paper, it's 100 to 329 00:20:56,730 --> 00:21:00,414 me. I know what it is. Your state of information is still 50, 330 00:21:00,486 --> 00:21:03,930 50. You don't have any information when I have it. 331 00:21:05,130 --> 00:21:08,650 So you're going to have to take that into account. So 332 00:21:08,970 --> 00:21:12,410 people were coming out and saying, oh, hydroxychloroquine is going to have 333 00:21:12,410 --> 00:21:15,850 prophylactic effects and curative effects. That's my 334 00:21:15,850 --> 00:21:19,450 subjective judgment. And my brand of data analysis is called 335 00:21:19,450 --> 00:21:22,970 Bayesian, where you come to it with a judgment in 336 00:21:22,970 --> 00:21:25,850 advance and then what data does is 337 00:21:26,090 --> 00:21:29,780 revise your judgment. So, 338 00:21:30,020 --> 00:21:33,780 so people were treating people with hydroxychloroquine and they were 339 00:21:33,940 --> 00:21:37,380 on TV to the extent they could get on. This is great. 340 00:21:37,700 --> 00:21:41,340 60 year old medicine is going to cure it all. As the data began 341 00:21:41,340 --> 00:21:44,820 to came in, come in. If you'd have been a Bayesian, 342 00:21:45,140 --> 00:21:48,340 you'd have said, my initial statistics on 343 00:21:48,740 --> 00:21:52,380 the curative powers of high gloxychloroquine are being 344 00:21:52,380 --> 00:21:56,220 totally not supported by the data. I would have changed 345 00:21:56,220 --> 00:22:00,060 my judgment. That's what data analysis is for, 346 00:22:00,060 --> 00:22:02,220 to get you to change your judgment. 347 00:22:03,660 --> 00:22:07,460 So valuable. You know, you start off an AI 348 00:22:07,460 --> 00:22:10,860 program with judgment as to what all the parameters are. Data 349 00:22:10,940 --> 00:22:14,340 stops the judgment process and makes those parameters 350 00:22:14,340 --> 00:22:17,260 consistent with observations. That's powerful, isn't it? 351 00:22:18,060 --> 00:22:20,820 Yeah, very powerful. And so 352 00:22:20,820 --> 00:22:24,250 hydroxychloroquine fairly quickly started 353 00:22:24,330 --> 00:22:28,090 the, the curative and the, and the amelioration 354 00:22:28,330 --> 00:22:30,810 properties of that drug were pretty quickly 355 00:22:32,250 --> 00:22:35,370 contradicted by the data. Okay, Same with 356 00:22:35,370 --> 00:22:38,890 Ivermectin, the horse tranquilizer. All right. Horse 357 00:22:38,890 --> 00:22:42,410 tranquilizer is going to cure everybody. And, and 358 00:22:42,970 --> 00:22:46,690 pretty quickly contradicted when they tried a little bit of that in small 359 00:22:46,690 --> 00:22:49,680 case studies. It, you know, you might as well have been taking a sugar pill. 360 00:22:50,790 --> 00:22:54,630 And so then we get to the shots and all the people said, 361 00:22:54,630 --> 00:22:58,230 these shots, man, they're going to be magic. They're going to stop transmission. 362 00:22:59,110 --> 00:23:02,790 That's my judgment. We're going to stop transmission. We began to gather 363 00:23:02,870 --> 00:23:06,510 data. The shots were extremely valuable. They didn't stop 364 00:23:06,510 --> 00:23:09,990 transmission, but they cut the severity of the symptoms by three 365 00:23:09,990 --> 00:23:13,830 orders of magnitude. And so they changed our judgment. 366 00:23:14,070 --> 00:23:17,810 The point of data science is get the data to give you 367 00:23:17,810 --> 00:23:21,650 the best possible judgment of probability. That's 368 00:23:21,650 --> 00:23:25,330 what data science is all about. Gather the right data that 369 00:23:25,330 --> 00:23:29,090 can have the most effect on your judgment. That's what we want 370 00:23:29,090 --> 00:23:32,210 from data, is our judgment as to how these 371 00:23:32,210 --> 00:23:35,810 processes work. That's cool, isn't 372 00:23:35,810 --> 00:23:39,130 it? So very valuable raw 373 00:23:39,130 --> 00:23:41,930 material to making me a smarter person. 374 00:23:43,700 --> 00:23:47,540 Having better judgment is consistent with a gazillion observations, 375 00:23:48,100 --> 00:23:51,620 each of which is stochastic in its own right, but they're not 376 00:23:51,780 --> 00:23:55,420 completely stochastic. It's not random. There's a lot of systematic 377 00:23:55,420 --> 00:23:59,220 stuff hidden in there. So the idea of data science is, is 378 00:23:59,460 --> 00:24:02,780 gathering data. If you don't just go out and gather the data, that's easiest to 379 00:24:02,780 --> 00:24:06,260 gather. Gather data that you really need to have 380 00:24:06,740 --> 00:24:10,510 to figure out whether the shot was going to 381 00:24:10,510 --> 00:24:13,630 be strictly prophylactic or whether it's going to cut the 382 00:24:13,630 --> 00:24:17,470 transmission. Well, we figured it out after a while, got enough data that 383 00:24:17,470 --> 00:24:21,270 said it's not going to change the transmission. And you've heard all the political 384 00:24:21,270 --> 00:24:25,069 bitching and moaning about that. Oh God, it's a lousy shot. It doesn't cut 385 00:24:25,069 --> 00:24:28,870 the transmission. It saved lives by cutting the severity. 386 00:24:29,910 --> 00:24:33,590 And that's statistically valid. And so 387 00:24:33,590 --> 00:24:37,150 big success. The data gathering in places like Israel where shots were 388 00:24:37,150 --> 00:24:40,950 mandatory was really interesting. Really showed the prophylactic effect. 389 00:24:42,450 --> 00:24:45,570 God, that's valuable data, isn't it? That's 390 00:24:45,890 --> 00:24:49,690 trillion dollar data. Is it allowed people to 391 00:24:49,690 --> 00:24:53,410 make better decisions? Candace, you're right on the mark. Once you know 392 00:24:53,410 --> 00:24:57,209 that, you can make a lot better decisions both collectively 393 00:24:57,209 --> 00:25:00,770 and individually. I decided to take the COVID 394 00:25:00,770 --> 00:25:04,130 shot because I did a decision analysis of it using my 395 00:25:04,130 --> 00:25:07,700 subjective probabilities. It was a no brainer. How could you be an 396 00:25:07,700 --> 00:25:10,860 anti vaxxer when you made that kind of assessment? 397 00:25:11,500 --> 00:25:15,220 I've done that with allergy shots. Great example. I take 398 00:25:15,220 --> 00:25:19,060 allergy shots, I'm a severe allergy patient. I went in to take 399 00:25:19,060 --> 00:25:22,180 these things. Well, you know, they know what you're allergic to and they stick it 400 00:25:22,180 --> 00:25:25,700 directly into your bloodstream. That's what an allergy shot is. If you're allergic to 401 00:25:25,700 --> 00:25:29,460 peanuts, you get peanuts stuck in your body. And so I asked, 402 00:25:29,460 --> 00:25:32,620 well, what are the, what are the problems here? Well, you know, you can die 403 00:25:32,620 --> 00:25:36,180 of anaphylactic shock. You can drop off the bench and be dead in 30 404 00:25:36,180 --> 00:25:40,010 seconds. Good. That's really a good thing to do. And 405 00:25:40,010 --> 00:25:43,690 they might not work. So I looked at the statistics 406 00:25:44,250 --> 00:25:47,530 and as near as I could tell, about 70% of the time 407 00:25:47,850 --> 00:25:51,210 they Provided significant relief and it was about 408 00:25:51,370 --> 00:25:54,930 1 in 10 to the 7th. So one 10 409 00:25:54,930 --> 00:25:58,690 millionth probability that was going to drop dead and so 410 00:25:58,690 --> 00:26:02,410 more likely to win the lottery or find another. You know 411 00:26:02,660 --> 00:26:05,780 I looked at my values and I rolled back my decision tree. It was a 412 00:26:05,780 --> 00:26:09,540 no brainer. 50 years I've been stepping up to the lady and 413 00:26:09,540 --> 00:26:12,980 having her stick peanuts in my arm which I'm deathly allergic to. 414 00:26:14,500 --> 00:26:18,340 And so you there also other things too that you're 415 00:26:18,340 --> 00:26:21,980 in the doctor's office when that happens. So like if you do like they could 416 00:26:21,980 --> 00:26:25,820 administer isn't that kind of alter the risk profile too? It's not 417 00:26:25,820 --> 00:26:29,590 like you're. I carry an amputee. You wouldn't. I carry. Yeah, like so 418 00:26:29,590 --> 00:26:32,470 you wouldn't go on a rowboat in the middle of the Ocean without your EpiPen 419 00:26:32,470 --> 00:26:36,030 and then take that shot. I did but you know you're not supposed to. All 420 00:26:36,030 --> 00:26:39,550 right, but, but I mean that would also. Yeah but in that 421 00:26:39,550 --> 00:26:43,150 simple example the statistics of 10 to the -7 422 00:26:43,790 --> 00:26:47,550 and the statistics of 70 probability of 423 00:26:47,790 --> 00:26:51,550 amelioration of symptoms, that's the billion dollar data 424 00:26:51,710 --> 00:26:55,560 right there. Simple minded data but 425 00:26:55,560 --> 00:26:59,320 it helps you make can the right decision. I got that out 426 00:26:59,320 --> 00:27:02,800 of the data but I also had to analyze it to find out 427 00:27:03,040 --> 00:27:06,880 how that interacted with my preferences. So it 428 00:27:07,040 --> 00:27:10,880 data analysis is critical to getting the information to 429 00:27:10,880 --> 00:27:14,600 make a good decision. That's what I think. I didn't even 430 00:27:14,600 --> 00:27:18,400 do any analysis. I just looked in books and saw those probabilities ain't 431 00:27:18,400 --> 00:27:22,120 that great. So every time I go in there, hey, I might never walk out 432 00:27:22,120 --> 00:27:24,000 of here again but the odds are pretty low. 433 00:27:27,050 --> 00:27:30,490 That's a great point. So we want to be respectful of your time. It's 434 00:27:30,810 --> 00:27:34,530 almost 40 after the hour. Where can folks find out about more about 435 00:27:34,530 --> 00:27:37,730 you? And I know Stanford does a lot of these lectures. They post them online 436 00:27:37,730 --> 00:27:41,370 or any of yours online. They haven't 437 00:27:41,370 --> 00:27:44,970 been. No, they haven't been but that's why I'm talking to you. Maybe I'll do 438 00:27:44,970 --> 00:27:48,570 that. And if you want to continue. I haven't told you much about data 439 00:27:48,650 --> 00:27:52,470 analysis per se. If you want to fire up tomorrow, I'm 440 00:27:52,470 --> 00:27:56,070 glad to do that. Yeah, let's. Let's talk again soon. We 441 00:27:56,070 --> 00:27:59,870 want to know the uncertainty we're looking at. And the more data you 442 00:27:59,870 --> 00:28:03,510 get, the less uncertainty you have. It gets pretty definitive if you 443 00:28:03,510 --> 00:28:07,150 got 60 terabytes of data but we don't have that 444 00:28:07,150 --> 00:28:10,630 candice. The other thing we'll talk about next time is in real decisions you don't 445 00:28:10,630 --> 00:28:14,470 have any or very little data that's relevant to the decision you're 446 00:28:14,470 --> 00:28:17,870 making. You get the data that somebody gathered on a government 447 00:28:17,950 --> 00:28:21,660 grant, like crime statistics. There's 448 00:28:21,660 --> 00:28:25,380 nothing in it. It's just data that's easy to gather. Okay. 449 00:28:25,860 --> 00:28:29,620 You are awesome. I look forward to speaking with you 450 00:28:29,620 --> 00:28:33,460 again. Yeah. With that, we'll let the 451 00:28:33,460 --> 00:28:34,340 outro music play.