Welcome back to Data Driven, the podcast where we talk about how data
and AI are changing the world. And sometimes we
even understand it. Today's guest is the brilliant Carmen
Lee, CEO of Silicon Data and former Bloomberg brainiac
who's now on a mission to bring financial grade transparency to the wild west
of GPU compute markets. If you've ever wondered how to hedge
your AI infrastructure costs the way airlines hedge fuel, or what
a futures market for GPUs even looks like, you're in for a
treat. Carmen's turning raw compute into a tradable
commodity, normalizing chaos, and possibly building the
Bloomberg terminal for AI infrastructure. Minus the beige
keyboard, we cover everything from tokenomics and TSMC
to why your AI startup's margins are flatter than the earth in a
conspiracy forum. Oh, and there's a used GPU car
lot somewhere in Virginia. Stick around. This one's a data
geek's fever dream in the best way.
Hello and welcome back to Data Driven, the podcast where we explore the
emerging field of data science, artificial intelligence, and
this crazy AI world we live in. But it's all underpinned by data
engineering. And with me, as always, is my favoritest data
engineer in the world. Even my dog is barking, giving you a shout out.
Andy Leonard. How's it going, Andy? It's going well, Frank. How are you?
I'm doing well, I'm doing well. I'm keeping busy.
We were talking about other podcasts that we have and
the other one is Impact Quantum. So go to impactquantum.com
definitely check it out. And had a very fascinating
conversation with our guest in the virtual green room. So without
further ado, let's welcome Carmen Lee to the show. She's
the CEO of Silicon Data and she is driven by a
passion for developing and delivering cutting
edge derivative products and data solutions that
provide essential data, intelligence and efficiency to compute
markets worldwide. Her company's vision is to
revolutionize these markets through unparalleled data transparency
and financial innovation. Welcome to the show, Carmen.
Thank you. You deliver up my tagline so well I might want to
hire you to do the whatever. Thank you. This is like.
Thank you. This is like I was looking the other day. This is almost our
400th show, so I do have a face for radio and
apparent thankfully. But a voice for radio. So good for me.
This is great. And speaking of radio, we were geeking out because
I started my career in New York in finance
and Bloomberg. Having a Bloomberg terminal on your desk was
a status symbol. There were the ones who had it and the ones who didn't
and the ones who wanted it. And you know radio,
right? Bloomberg radio, which we also get here in dc. And you used to work
for Bloomberg, so that's really cool. That's right. I had a great time
working for Bloomberg and my team was part of the
data team I thought is
one of the most cutting edge data company especially in the
financial services industry. Back then I cover all content,
all product data integrations with any third
party ecosystems. So think about any training
cycles from Fedmin back offices, think about any
cloud providers and database Systems and
even AI, LLMs, whatever you call them,
different use cases, real time
reference data, aesthetic data, anything. It's
really fascinating. I learned a lot my background before that I
was in all financial services and I don't know if I bore your audience at
this point. I started my career in trading, high frequency trading
in Chicago. So to me transparency,
efficiency and free market is sort of in my blood.
100% brainwashed at this point in life. So one of
the things I noticed when I was a Bloomberg is there's a
lot of interesting ecosystem
platform came up last year, right? So they all leveraging gen
AI. You're the first few adopters which is good for them
and their client basis sometimes can be financial institutions. So boom, client
basis. So one of the things I noticed is it was a really fascinating conversation.
So those startups, they're gaining a lot of tractions. Good for them. So
obviously I was like oh you're doing so well. And they will complain to me
saying that they were sassed, right? They were 100% SaaS
revenue so static and then it's pivoting to
AI driven SaaS. So their cost, think about last year The
GPU per GPU per hour was like $9 or
6, 7, 9. Back to like 3 if
you own interruptible instances, right? So the swing is like
300% within the same day but then their revenue
is static, right? So their margin like
positive 40% to negative, 60 to
positive and there's no way for them to manage it. And also
same time it's not like they bring on more clients. They
can enjoy the scalability. It's like again
same thing, the margin is uncontrollable and they have this problem say how
do they actually coming out a cash flow plan for next year and then
they obviously complain. Totally strikes me to be
hey, this industry needs financial
infrastructure layer, right? It's almost like talking to American
Airlines. Say hey airline, you cannot hatch your oil prices
fluctuation. How are they Going to price their tickets. They can't, right?
And it's not like American Airline cost OPEX in like give me five year long
contract. They don't do that. Every single of those commodities
pricing discovery and hedging happens in divers market. So
futures options because there's a few reasons, right? Number one is it's just
efficient. Number two is cheap. Both is flexible and then you and me,
we can do the same thing. We have oil exposures, we don't have to be
American airline. But today if you are crowing for
hyperscalers, you can go to those, you know,
whoever, right? Produce chips, right? And get a long term contract.
But you, if you and me start Neo Cloud, guess what? We don't have access
to kind of pricing. It's not good. We you have a
few players who have the pricing, who have that way to hedge it. But
then the smaller Prius just couldn't get in the game, right? It's really not good
for the ecosystem's health performance and
the risk management. So that's really struck my core.
Last year I was like man, someone needs to do the
index, the pricing, the benchmarking layer of the
GPU compute as human resource
I feel like will be the biggest human resource in the next few years.
Surpass all energy combined, right? So that's why I left Bloomberg right
away. Super passionate. I think we can bring so much transparency to
ecosystem will benefit everybody, right? Not only benefiting other people,
needs compute benefiting like you know, the end consumers. Because think about
the whole funnel, right? You had finance and gpu the
actual clusters cost, right? So
if the banks don't have enough information or hedging for the
banks then they have to charge you high interest, they have no other way. Or
you have to look for alternative capitals which traditionally
they're more expensive, right? Because they're not banks. Banks are cheap as a
cost of capital, right? So then the cost from you
know, stage zero is high. Then think about the second stage, third
stage and then people like you and me using Sora with OpenAI
everything will be more expensive because of that, right? So fix the problem
with transparency from this from Gecko is really really
critical and then their benchmarkings and encourage the
secondary markets and all those flexibility and then
availability will be really incredible to benefit the whole
ecosystem. Interesting. So is it fair to say you've built basically a
futures market for GPU. Compute I building a
benchmark index layer. We are working with future exchange,
right? So I'm not a futures exchange so that would be something we
will think about S and P. Right. So they license the index with a. Right,
right, right, right, right. That's what we do. Right. Well, we will index
to an exchange and they will have futures options on top of that and other
financial products. That's a fascinating concept because like
you're right, we need that because the scarcity
of GPU compute is a real issue. It comes up.
And if, if, if Amazon, the rate. Of volatility, how do
you. With, with. With like 40,
60% fluctuation every daily volatility and then it's
just not a, a very transparent market
which is. Breeds inefficiency. Right.
Absolutely. So for those of. Oh, sorry. Go ahead,
Andy. Okay. I was just going to ask. So are you tracking
features and functionality and all of that? That, that would be the. How you value
the GPU itself and compare that to the price and
you're coming up with some ratio. Exactly. So
compute is not like. Unfortunately it's not as easy as electricity
or even oil have different grid. Right. So even 100
has different configurations. Right. They all, it's not the same. Right.
Different CPUs, different RAMs and geolocation matters.
Right. So a lot of things. So normalization become very critical
component to financially settle index.
Right now we have H100A100 indexes published at Bloomberg and Refinitiv.
So the way we do it is we have a base case and all
the factors normalize to the base case. And the way we normalize
historical data, what factor is actually important to the users,
the CPU matter? How much does it matter? What's the wave, whatever it
contributes. How often do we calibrate? Maybe it matters
today, maybe tomorrow. This, this particular. Whatever
inputs value more. Right. So we do calibration,
period of calibration as well.
Interesting. Yeah, it's fascinating to kind of see because I mean
it always seemed like there's something missing around
the GPU market. Right. Because it's just. And I also think too
it's been a while since we had any kind of compute limitations on what we
wanted to do. Right. Like that CPU is like. Yeah, it's cheap
and you can get what you want and it's not supply demand kind of shifting.
Yeah, I agree. Right. So I didn't really think of like,
you know, kind of this, this market kind of response to
it, which I think is, is an interesting approach and I think, I think,
I think it's fascinating. Yeah. Even if you think about
AI SaaS company. Right. I don't know if you heard the saying that
SAS is 80% margin AI SaaS is 0% Mar.
So I mean it depends on how you run your workflow. If
you are not being thoughtful, right.
You just dump everything, everything you need to do into the most
expensive closed source model. And you're not
optimizing your thinking tokens, your input tokens,
output tokens. It can get very pricey very
quickly, right? Not batching it, you're not doing all the right things.
And even you do all the right things, it's gonna be such a meaningful
percentage of your cost. And then all those companies not ready for it. Right.
Because in before what's the raw material cost?
Electricity. Like really nothing. Right. But now
every company becomes, you know, a which is great company
but then their cost structure is shifting from zero cost
to. To 40%, 60%, any percent to token
or to GPU at the end. Right? Right. So how do you think about hedging
that kind of cost component? Can you control that? Can you optimize for it? Can
you monitor it, can you benchmark it? You know, can you hedging it? So
no, that's a good point. So do you think
there's multiple, I guess, inputs and levers to this? Right. Because it doesn't seem like
this would be a straight thing. So what's, you know, Andy mentioned that you were
tracking certain benchmarks. Like what benchmarks are you tracking? Because I'm very curious about
this. Right? So there's a few things, depends on
your position at least this can change
every single day. Like our ecosystem is so nuts, right? So it
depends your
positioning, the whole workflow, right. So think about if you are new
clouds, you are selling token, right?
The cost for you is a gpu, right. So then your margin
becomes the diff between the margin and the GPU cost. And that's
the way we calculate it, right. Which is different units.
And then your worry is okay, so for
token survey for the tokens, how much money can I get rate
from one particular gpu? The flops, right? How can I optimize for that? And
what if I'm doing even hosting open source models?
And how do I make sure people using that open source model, should I
shifting it? What's the pricing for that? Think about that strategy and GPU said
okay, am I renting GPUs? I'm like outright
purchase those GPUs and put on my books and depreciate it. How
long can I depreciate it for? How do I let's say if
everyone's the latest and greatest, I'm selling the GPU after second, third year,
what's the terminal value for the GPUs? Who should validate that? Which bank should
depreciate the asset classes. So it's a lot of things coming to the new
cloud space. If you think about your inferencing infrastructure,
right? So let's say you're
AI tech company, right? Then your revenue is token,
right? Ideally they're paying you based on token
use cases as well. And then your cost is token which is
easier but same time for you is thinking through okay so
right now open source tokens, the price
they do move up and down. For example
if you look at Deep Seq, even Deep Seq, they host their own servicing but
then the price changes, they have the off peak hours and that change all the
time. Or you can do closed source which the price is pretty
static. The way I think about it is again it's extremely
free market approach, right? Is how can we
make sure especially open source ones, the token prices
is driven by the market demand supply curve,
right? Let's say if everyone, if I have like 100 GPUs
right now and obviously let's say I
choose to host only one llama open source
model and then I know I can produce X amount of tokens,
both input output tokens, right? And I can just auction off
and you guys and you can buy a million token and one day he's like
I'm not going to use it, why do I sell it to Frank? Can this
be some market where right now you are stuck
with it, right? In
my mind, unfortunately I'm very brainwashed to free market. I feel like you have to
give people option. The more option you give people and
any have flexibility, franchise flexibility and people more
willing to participate because they know they can get out. Because right now you're stuck
with hyperscaler GPUs or any tokens, you're stuck with it
and then you're less likely to commit because you know you
can get out or you get fined even worse, right? You know those cases, you
get fined millions of dollars when you back out on cloud deals.
That's one of the things I really think I should encourage people thinking about tokens
and GPUs as a main cost structure. How can we drive
efficiency so people can commit and then get out if
they need to and then swap out and everyone gets more value
and efficiency from those transactions. So is it
more like an exchange or an auction?
What's the mechanism? Right? So from token GPU side
obviously there's Spot exchange already like compute
exchange, where you can actually tell them, hey, I need this
configuration how many nodes? And then they will
say okay, let's do an auction. And then the
best price, best quality, whatever combination wins. Right?
Yeah. You can potentially do other asset class as well. Right. So we're.
Siliconita is a data company. So think about us as the Bloomberg and there's the
Nices, NASDAQqs and everybody, right. This spot, right,
you can actually get GPUs. You, you can actually get stocks from those exchanges.
And the FAST is we collect data from those exchanges like
Bloomberg. Right. And then we'll produce financial products on top of that. Right.
So that's right, there's spot, which is the
nasdaq. Right. You can buy and sell, get actual physical
deliveries, all the compute or token you need. And there's
data side which is making data the Bloomberg. Right. And then FAST
is structurally the financial products layer. Right, data layer. And
then we're agnostic, meaning we look agnostic of chips,
agnostic of spot markets, agnostic of everything. Right.
And it's a future exchange which they license
our indexes to create futures product. Ideally we're
settling to spa. Maybe some of them will sell at spa. Right. So it's pretty
standard practices. So
would the currency or the coin of the realm be tokens
or compute time or compute seconds? Things
change. It's, it's making my life really fun
and you know, also different. Yeah, all the time.
And then you, you mentioned you have this quantum thing, right? Right.
It's a lot. We track all compute. So it doesn't for us
what chips and what, what architecture framework
and you know, we don't really care. We benchmark the performances and the data
inside. And everything we don't know for us is getting
ready for everything. So we want to create product
that's actually going to be helpful to the marketplaces, not just creating
things like gambling table. People bet on binary things. Right. For
us, how can we make it useful for the people who actually
naturally long compute? So the Neo clouds everybody else,
they need product to hedge their revenue fluctuation. Right.
So they issue short futures and whoever naturally short compute.
So you need computer and for you is a cost management.
So I want to make sure my product is usable by them. It
depends on how they pay. Right. If they pay tokens,
nothing to create token products. You're very right now paying people paying
per GP power and you create product for that. If they pay
things all right, then it's different contracts for that.
So it really depends on how people using it today and tomorrow.
And then, you know, we. We hyped to create products that may
not be the S&P 500, which live forever. We probably create financial
products live for next five to 10 years. Because guess what? Chips
what our style, right? The A100 people still using it, but
like L4s, people are using it, but like other chips like the V's,
the, you know, probably not as much. Right. Then similarly, my
financial products associated with that underlying asset
probably will, you know, retire, be retired. Right? Which is fine.
That's cool. I'm sorry, go ahead, Andy.
I was just thinking about it and a couple of ideas popped into
my head as you were describing that, Carmen. One is
capacity. It sounds like you're literally selling
compute capacity, GPU capacity, time, just
whatever. But it kind of falls into that bucket under one hand.
But then on the other hand, it seems like that
it almost creates this utility market.
Is that fair or am I missing something, right? No,
you're right. But two pieces. So one is a compute exchange part, right? This is
where you can actually get either depends on what people,
the mode of people preferences. You can get GPUs or get
tokens, whatever, right. Physically delivered, you do you. You
don't have to touch any financial products, right. It is literally like you going to
a store buying stuff. And then the more option based, right.
You can actually get instances. And the silicon data is. You
cannot actually getting any compute. Right? Like you cannot
get any stocks from Bloomberg. Well, you can get this data.
What asset is trading, what prices? So that informal decision, ideally
in your spot market be like, hey, I think everyone, you know,
the H100 price is a little too high, in my opinion. I'm not going to.
Right. Right now, like, forget about this. And I can totally use a
100. Right. It's fine. So this data is data
layer, which is liquid data, right? So those are those the
sort of two pieces to I guess resolve the
workflow equation. So it's kind of like when you go to the supermarket. I'm
sorry, Andy. When you go to. That's okay, go ahead. When you go to the
supermarket, you buy the beef, you buy the pork, but you don't think about the
pork belly futures and stuff like that. It's kind of abstracted away from you.
Exactly. The farmers will think about this, right? Yeah, farmers think about it.
Yeah. They need to hatch the corn futures, right? But if
you are farmer, you still say you were someone to eat the
corn. You go supermarket, you don't think about, hey,
Right. So you may have covered this already, but how does
or does fungibility come into play?
It's a great question. So I went through so many different iterations about this.
Initially I was like, okay, why don't I just normalize across flops? And I was
like, nope, can't do that because there's
just, there's so many things wrong with this approach. But obviously
we can dig into details, but we're not going to do that. And then secondly
is okay, why don't we do like inferencing
chips? Like just make a pot and then we realize, okay, how can.
So again back to the initial question. I want to make product actually going
to help people hedging. Right. If you
do a combination of different chips, then if you
are and you know we're using of a lot of people, are you going to
really use that to hatch? How would some correlation look like. Right.
Maybe you just rather have different chip types and then just hatch accordingly
because the correlation will be much higher than the combination of indexes.
Maybe the composition of indexes is good for just tracking
general, but not for actually financial products. So we have, we have,
we can have all. Some of them will be tradable. Some of them. Well, right.
For us is if people start, if, if we move to the world
where it's not going to be Nvidia only kind of play
in the like amd. We can eventually,
it'll probably end eventually. Well, we'll see when, right? We'll
see quantum happens first or everyone catching up first. I have no
idea. Right. So if it's like a more vibrant
ecosystems. Right. And then maybe we're thinking about, hey, maybe we can do
like doing some of the chips. Even different firms would normalize it and then we
do something like a inferencing chips, chaining chips. I don't
know. So that's another thing. Or like token, token indexes. Right.
So can we do open source ones? Multimodality? Is
multimodality going to be a thing in a few years? Everything going to go back
to one model only? Because right now with different models. But maybe it's the interim
stage. Right. We. I don't know. So it's one of the things we have to
keep like looking and thinking and just moving things
forward. Yeah, I was thinking too about, you
know, the, the amortization that people
do in their heads at least when they buy a new car. Yeah. So
that's the math is you drive it off the lot, it's worth what, a 75,
80% of what you pay for.
So we need a Carfax for GPUs, right? So that's what we do too
for silicon Mark. So what we do is okay, everything. Well
at least right now or before Last year or T minus 1, everything
is brand new. So okay, we'll take whatever the
number they published and tdbs, the flops, we all know
there's like haircut to that number.
That's funny, right? And then a year later, right, A year later I
say, Andy, you're growing great in great data centers. Your
thermal cooling was doing great. I'm old data
center, I don't have the latest cooling. Obviously my chip
is after year. You can argue they own different curves,
decay curve. And are we treating the same prices even
though same configuration? Probably we shouldn't. Should it be reflection of
the actual quality? So that's something Mark does.
And then we do things even more basic than that. So number one is
when you tell me you have H100 like 100 nodes, each node has
say 8 GPUs, right? Yeah. Is that true? Can I
number one verify the UID of that? And you see, it's all the CPUs
and this operation systems on all
the nodes, they all live connected. Number one, can we just
verify are they connected? What's the latency? So that's very
basic things, right? So we do that piece at least, you know,
are they truly UIDs and CPUs? The machine, is the machine ever
changed? Because we do mesh IDs based on
CPU changes. We know something changes, right? And then the UID of every
chip. So we do the decay curve for the individual chips and also the machine
level and then thermal staggeration, everything. So we do
that and then we do validation. Almost like Bloomberg Validate fixing
compound. Because you have to understand the issuers and it's
a bridge and it's a school and with cash flows and all those stuff. So
we do that for GPUs. The geolocation. If you build data
centering somewhere in North Korea,
it's great, but no one going to use it, right?
We took all those in considerations when we created those data models. So then
we figured out, okay, so based on the setup and
we run a benchmark on specific GPUs, this is our grade and then
this is our validation. Obviously you can do whatever you want. And then you can
say hey, screw that, I believe this is much higher price. You can do that
as well, Right? But this is our valuation. So almost like a scoring system.
That's interesting. So My mind immediately went
to, when, when we started talking about cars, my
mind immediately went to, you know, the used GPR lot
some guy in bib overhauls out here in Farmville, Virginia
kicking the tires. What's it going to take to get you into this
gpu?
Yep. See, there we go. And network them together. Right. Like I think there's also,
you know, maybe, you know, I don't know if
you've been tracking the, the DGX Spark device
that Nvidia has, but apparently they have ports
in them so you can network I think up to four together. I'm not sure
but yeah, I'm sorry I
cut you off but like. No, no, no. Nvidia we
definitely leveraging a lot of. So we do the container within
container and we do integrate with Nvidia DGX
benchmarking. So they have open sourced some of their LM
benchmarking based on GPUs and we do streamline their products so
you can test lms. So Nvidia Digitex testing
through system data. The benefit is if you do it all right yourself,
number one, you can
obviously people want but people can just change up the, the
benchmark results themselves. Right? It's open source but through us it's data Oracle. You
can't really change results. Number two things is more streamlined. It takes a few hours
to run versus take weeks because you've download a bunch of things you may or
may not need. You may or may not need.
Well, I also think too like, you know, how does this, you know, you
mentioned you, you kind of skirted around the location thing with sovereign
AI, right? So like if I'm okay with using Google
Services, right. And I can, I have access to TPUs, right. I have a lot
more access to whatever Amazon's chip. Microsoft I think is
working on something. Custom that's on prices too, right. The Geolocation
they have different prices and different carbon footprint. We haven't even touched that.
Right, right, right. We do track that as well based
on local grid power grid information. We do track the carbon cost associated with
different AI workflows. I think it's important, I think so
for me is let me at least surfacing the number to you and
you decide what to do with it. Right. So I think that's a good idea
or you know, maybe it turns out that you know,
this type of model of GPU is you know, depending on what your
core. I think it's, I think it's great because I think one of the things
that I've heard And I didn't Peter
Drucker. What gets measured gets managed, right? So you're, what you're doing is you're providing
ways to measure GPUs and GPU performance. Right.
So if I don't care. One of the things I heard about and I'm sure
you have some thoughts on this is like cloud providers that are
starting up and they're just doing
GPUs, right. They're just doing kind of training loads. Right.
And they don't need to be located anywhere special. Right. Like they don't
need to be in the northeast corridor. They could be in the middle of
nowhere as long as they have power. Right. And
because you're going to run a load, right, you're going to run a load on
the thing, it's going to take 72 hours say to run. You don't really care
if the latency is, you know, 150 milliseconds versus
3. Right. It doesn't really matter. Yes.
That's why you see a lot of us get built up in like Iceland, Finland,
the users can be in Americas, can be in Asia. Right,
right. For them is can they get the capacity
looking for and you hard deal if you're thermal powered
data centers, cheap electricity. Yeah.
And then it's cleaner supposedly. Right. As
long as you're not on the volcano belt.
Right. As long as it's not going to blow up. Yeah.
But yeah, so we definitely see that trend and a lot of energies, you
know, what do we call it oversupply sometimes can
be in Spain because overbuilt and the grid couldn't handle it. And
then they need to get data center up and running like now to take over
the power. But then
it takes a lot to make the racks start running. Right.
More than just the GPU itself, you need the connectivities and network
and that could be in shortage. So you need to solve a lot of different
pieces to actually deliver the actual computer.
But that's why it's fascinating industry for us because
we see things from dsml, tsmc, side.
So anything supply demand shifting will have
an impact on the whole ecosystem. And then this industry is winner takes off
from LTSMC to a solution level.
You have to be the solution. Your alternative solution just not
going to work. So. So every single piece is so critical to
the whole chain packaging. Right. You have to work,
right. If you don't know how to do it, then you just can't do it.
It's not like you can buy a cheaper pair of socks or whatever
so we do. We're from end to end, right. From the SM of production,
tsmc. So we're official TSMC partners are going to be actually
TSMC conferences to this
November. Very cool. It is really cool. I
kicked out by those stuff very quickly. And all the way to
the model A, the token layer. Right. Agentic layer. So
we sort of see things all the way. Which
I think my brain get overclocked every single.
I know what you mean because I get till the time of like
2:33pm and I'm like, I can't take any more input. Like
and the muscle, my brain muscle just dead. I know. How
do you do that? How do you get a roller in my brain, just like
relax my brain muscles? I. I found going for a walk
is a. Is a good way to do it. Right.
No, like. And a co worker of mine calls it everything turns to
hieroglyphic hieroglyphics when he's like
looking at like stuff. And I was like, yeah, that's a good way to put
it. Because it's just kind of like, yeah, I can't. I don't want to have
time by a daughter. So I usually spend time with my daughters. I feel like
they've been silly. And I would tell them, I'm so stressed out. When my daughter
was like, me too. I was like, what are you stressed about one last donut
than the other guy. I was like, that's very important thing. I agree with that.
That's very stressful. I will be really upset if I get one less
donut. So. Yeah, so definitely put things in
perspective. Yeah, that's cool.
I think one of the best things. Any other questions? No, plenty,
plenty. Like, I'm just fascinated by this. I know
we're kind of short on time, but one of the things that you mentioned was
tcmc. Tsmc.
So for those who don't know who they are and how important they are to
the global economy, could you explain for those folks
and why I was so excited that you're going to one of their conferences? I
didn't know they had conferences, so. I don't think I would do the justice
of explaining how important TSMC is. All right, how about I explain it and
then you tell me where I'm wrong. I'm sure you'll do a better job
than I can. So. Tsmc. Taiwan
Semiconductor Manufacturing Company. That's right.
They are based in Taiwan. And
the reason why. Nvidia. There's a fascinating
story in the book called the Nvidia Way. I Don't know if you've listened to
that or read that book. Really awesome book. But basically
one of the advantages Nvidia had early on and arguably
now was that they off they outsourced their chip
manufacturing to this company tsmc. I'll get it right that
time. They are basically what they call a fab.
And you could, I mean not
now they're so busy like you know, you kind of the you in general. Right.
Like I couldn't call them up and be like hey, I have some prints for
you. I have some chip designs I want you to make for me. Can you
send me. They're not at that scale but
so they're a fab. And so what happens is people like Nvidia, companies like
Nvidia, a few other companies too will go and they will, they
will design their chips and then they'll, they'll basically
not drop ship but effectively kind of print to order
chips. Which frees up a company like Nvidia
from having to build their own fabs. Kind of like intel does. Is that a
good description? 100 so I usually call
on Nvidia and AMD like design houses and then sometimes
confused with people who's like oh, are they like Louis Vuitton was like no,
Right, right. Or like graphic designers? Yeah, yeah. So they're design
houses and then they are Fabless. Right. And intel,
which is interesting because they do both. Right? Yeah, yeah.
Intel like as I was saying that intel doesn't. Yeah, they do both. Yeah.
Right. And then it could be a great strategy. Could work
or. Well, depends on many things. Right then anyways,
so TSMC is like the, as I said before, this
industry, I don't know if it's good or bad but it's a winner takes
all market. Right. So TNC is definitely
the winner for a lot of different
reasons. I think for the leadership, self
and technical team for the whole supply chain ecosystem. The
gravity, all the years, the hard work they've put in.
So it's a position where I don't think anyone
can seriously challenge them
in a meaningful way in the next whatever
years. So they're very critical. And then the good
thing interesting about them, they're the agnostic of design houses,
right. So they have great relationship with Nvidia for sure and I'm sure with
them, with everybody, right. It's their job to
produce those chips and then it's
interesting enough it's aligned with mine. Silicon Data. Because
I'm agnostic of chips, right. So
obviously I want to create products that's most important to the
ecosystem. So right now people care a few chips and
those chips happen to be from one design houses. But let's say
if another design house start picking up a lot of momentum. For me, it's
like, how can I help everybody in ecosystem
compare, contrast hashing, right? Use them benchmarking, normalize
it in a meaningful way. So it's my job to work with all the design
houses. It's their job to produce chips that can be usable for
defunding the houses too. So we're very aligned in that sense. And
anything they do, right? So think about, they are
future looking because they're not thinking about next year or next quarter. They think
about 20 years, 10 years. It takes them five, six
years to build a fab, right? And then they need a fab to
be utilized. And they have a threshold, right? If you're
building a fabric and that's not utilized by year eight,
they plan right now by year a year 10, they are
losing a lot of money. A lot like billions of dollars,
right? Like can you make sure the fab will be utilized, the demand
will be there by year 10. Forecasting from today.
It's very, very, very hard job to do. And it's not
like it's not like a new reim, you know,
like what are minings and all things that you can hedge it, right?
Like there's a way to hatch the future curve. But like it's not like they
can forecast, forecast and do a swap on that because
the market is so concentrated and then very
binary and a huge size. Who's taking the other side?
I don't know. It's very hard over the concentrate to
do so for them is to get clarity supply demand curve in 10
years. I mean they do also edge computing chips as well, not just data
center chips. Right? But how do they think through that? I think that's
really challenging. I think will be really challenging for me
for sure. I'm sure they have way smarter people there to think through those problems.
But yeah, it's an interesting problem to have.
That's why TSMC and I, for example, they sell to
their clients who are in the vds of the world. So they have that kind
of transparency. But what they don't have, which
may be a different indicator for the supply demand curve in
10 years is end users
pricing volatility. Right? And then you know, okay, so if
every single chip, every single chip I produced, right, Data center
quality chips, one dying price, right.
Is the indicator for supply demand shifting. Maybe it
is Maybe it's not right. At least you have some, some data points which your
immediate sales and revenues which is T0
won't give you because then a few degrees removed from
end user experiences you give Nvidia and Nvidia packages it to
AWS and GCP and end users and you and me. Right.
So that's something that for them to think through as well.
Interesting. One of the stories I heard and I
wonder if it's true, was that part of the
reason why there was part of the reason
why Nvidia was able to really capitalize on this. There's a lot of
reasons, but one of them was the fact that in the
crypto craze, the run up to get chips for that Nvidia
had purchased. Now what you said makes a lot more sense now. Nvidia had purchased
the. They basically purchased a certain amount of capacity at TSMC
for like three to four years, something like that. And then that happened to
coincide with the AI boom. Is that, is that true? And
that. I guess that's a market too, right? Like you know, like hey,
so I wasn't. I know so 7 so I'm not following all ASICS
so they have a specific for. For the, for. For the mining
chips. That could be true. So I think
not because I'm straight, I mean and a girl can dream. I'm
strapped to be like, you know, to really
help the industry and then be, you know, like
the company the team hopefully can propel the industry
move forward. Right. I'm strive to point zero over percent people
and then competency is very important. Obviously execution, your
hard work is important. Not a big piece is you have to be
really, really lucky. That is also everyone's control.
And then Nvidia puts so much time effort into everything they do. You can argue
they were really great company even before the AI boom and
everything. But the lock piece and how do you control that? How do
you. How do you know quota gonna be like the piece
that's needed? Right. Well, some.
Someone said that, you know, Jensen Wang is like the epitome of,
you know, the better you, the harder you work, the more luck you have.
True. Like there's a lot to that and I know it's
complicated but like I'm just, I just. It's interesting how the crypto kind
of boom and bust really kind of also
propel us into the AI. Not, not all by
itself but it definitely I think gave. There was some momentum where
no momentum was expected, if that makes sense. Right. Yeah, I agree,
I agree Timing is so interesting, but
we just have to two point like the heart of your world. You have to
do everything you can with the environment. Right? That's
cool. That's cool. All data. So we'll see happens what
I mean. That'S the importance of data. Right. Like, you know, people don't realize that.
And I go calling back to Bloomberg. So I'm referring to Michael Bloomberg,
former mayor of New York. But before he was mayor he
basically started a company called Bloomberg. And
he was not the only factor but like
a big part of, you know, people getting into, you know, his
philosophy. As I understood it, if there's a good, if there's a good biography book
on him, I totally would want to listen to it. But basically getting
the traders access to data gave them an advantage. Right. And it was
really, he was really early on in the idea of that data is
not just something that's created as a byproduct of
transactions, but can actually be, you know, monetized
and arguably weaponized. Right. Like so.
And you know, Bloomberg terminals
before, you know, it was interesting because he basically sold these custom terminals so you'd
not to rely on like local ID who were still struggling with like, you
know, just keeping the network up and running, you know, these separate
devices that became status symbols. And ultimately he, that's become like
this media empire that, you know, I can watch Bloomberg on my
tv, I can listen to it, you know, whether it's a satellite radio or the
app or you know, FM or AM radio
stations. You know, I think it's in San Francisco, New York and
D.C. they have a big office in D.C. they always have an
interesting show called Political Capital. I think that plays
at 5pm every day. I listen to it because it's kind of the
policy side of finance and kind of what's going on in the world around.
And AI has come up a lot digital sovereignty. So it's interesting
how all of these worlds, I like your thoughts on this,
right. The worlds of finance, the worlds of tech and the worlds of policy,
politics and dare I say war. Right. They're all kind of like
crashing together in this giant thing. And
it's kind of cool, kind of scary.
I think it can be. I mean, sometimes I'm scared I was like,
you know, because you see a few things, it's like, whoa.
There's a lot I feel like for people born post Covid,
not born, but grew up post Covid, I would call Jen the second
Gen Z Gen Alpha. Yes. I think Gen
Z's apparently now like I'm all confused. But for
them it's like, of course they should. They should. My AI should be my
boyfriend, girlfriend. Right. Like whatever. And then for me it's like,
this is not comfortable at all. Weird.
Yeah, yeah, yeah. For me it's not. I have no idea what's going on.
Like, I just so creeped out by this. But for lot of people it's like,
of course you do that. Of course you tell AI all your secrets.
Of course they can. My phone can record my conversation. Of course
you can train, you know, your AI model. My
model use my all my Gmail content information.
All edge computing. I have my own AI model. Of course you can wear,
you know, glasses and then record everything you and me talk
about. And how secure is everything
right now? Right.
The hardware level encryption
is only available on a very specific few chips.
TPU can do that. You rely on software encryption.
No, it's true. And software encryption that is vulnerable to a quantum
attack which is not that far away. We are not the
software and use cases moving so quickly. The hardware hasn't been able to cut
up. And it's expensive to do hardware encryption. It takes
longer and it's more expensive. That's why sometimes the hyperscaler charging
higher premium for that reason. Right. Are you willing to spend a
token and time and effort to do so? Some use cases, you can argue.
Yes, yes, absolutely. No edge computing
chips can do that kind of hardware level encryption.
And it's happening like now. Right, Right.
I was talking to a startup called Quantum Knight. Nate claimed to have a solution
that is a low, low compute kind of post
quantum ready thing. So I can send you their
link and information. Yeah, we, we track quantum
computing prices as well. Very different than GPU pricing and like, you know, like
a thousand per second per minute pricing versus hourly. Right. This is like
different cycles you run. And then GPU become like error correction component to the whole
thing. But for us it's like, okay, so
computers compute now, GPU and tpu, whatever, pu. And then
it becomes like quantum. How we think through that? I don't
know. My brain just like, you know. Yeah, I know. At some point it just
becomes like. I'm not smart enough
right now to, to. To. To figure that out. I tell you, like I go
through like quantum stuff and like I always joke with Andy, like I'd be like
15 minutes, I get a migraine, which is basically like my brain's version of
blue screening. And like, just like, okay, I can stop. I can get
to about. I can get to about 45 minutes now, which is, you know, an
improvement. But this is actually a good book.
And he was actually a guest recently on the Quantum Computing podcast.
It's a thick book. It's a thick book. But I'll tell you this.
The, the, the, the first three chapters, introducing the concepts
are probably the single best introduction to the
concept I have ever read. Yeah, I will send you the link. Yeah,
yeah. Dancing with Cubits.
Really interesting book. Super nice author too. He's a, he's a trip.
But it, it. No,
you're right. Like, these are. The thing that really worries me is I kind of
think about this like we built our entire economy and we're, we're
on a house of sand. Can we start on this? That's
another thing. We'll have to have you back on the show for
a second one. But like other countries
where they lay off hundreds and thousands of people, not. Not just by American
companies. Right?
Yeah. Don't even get me s on that. Well, like, and like, you know, we're
all based on. And, and the other thing, the elephant in the room, right, is
the fact that TC the, the T in
TSMC stands for Taiwan. Right. Kind of.
I know, I know it's very dangerous to talk about this, but, but like. It'S
kind of like, shoot. So I won't say much, but I'll just say it's
contested real estate. How about that? Right. That's a pretty safe way to say it.
Right? It's contested. Right. And you know,
the entire world effectively revolves around the kind of modern
civilization revolves around the manufacturing that happens there. And
God forbid, like, you know, whether it's man made or a tsunami or a bad
earthquake, like, I mean, our world, I mean, we, we get sent back
to the 1700s pretty quickly. You know, 1700 is not,
you know, there are still people, they're still human beings in the
hundreds. That could be worse than that. That's true. It could be way worse than
that. That is a good point. I was trying to keep it. I was trying
to end it on a positive. And I know you're traveling there
like no humans. Well, no, I mean, like,
I mean, there's a lot of ways that the, you know, this apocalypse could go,
so to speak. Right. It could be, you know, but like, it's a very. And
like, just from an infrastructure point of view and supply chain point of view, like,
you know, we, we. We've really championed
globalism and kind of all of these extended supply
chains for, you know, there were reasons there's always reasons, but like
at the cost of resilience. Right, right. That's kind of scary.
I assume you've read Taleb, right? The like anti fragile.
I'm so sorry. No, that's fine. That's fine. But I really appreciate you taking the
time. Where can folks find out more about you? Silicon
Data.com Silicon Data.com awesome. And we'd love to have you back on the show.
And you can tell us what these conferences were like. The. The
ts. Let's see how much I can understand
first. Right, right, right, right, right. That wasn't a good question.
That's why you got to be like the kids today and record all your conversations
so you can talk to the transcript later. All right,
nice seeing you guys. All right, thank you. And we'll let our AI finish the
show. And that wraps up another episode of Data Driven, the podcast
where we ponder the future of AI data and occasionally
the fate of humanity if we don't get GPU pricing under control.
Big thanks to Carmen Lee for joining us and blowing our minds with
compute market mechanics, financial innovation, and just a
touch of economic existentialism. Be sure to check out
silicondata.com to learn more. Just don't try to day trade
H1 hundreds after midnight. If you liked what you heard,
subscribe, leave a review, or send us compute credits.
Until next time, stay curious, stay caffeinated,
and remember, in a world of exponential AI, transparency
might just be the killer app.