Welcome back to another riveting episode of Data Driven.
Joining us today, lakeside and positively glowing from his
Appalachian retreat, is Frank. Meanwhile, the
always astute and ever energetic Andy is here to keep us
grounded. But enough about us. Today, we have
a true luminary in the field of AI, someone who's blending the worlds
of academia and enterprise with seamless finesse. He's an
associate professor at the Technion, has published over 100
research papers on automated speech recognition, and is the chief
scientist at Iola. Please welcome doctor Yossi
Keshet or as he's known to his friends, Yossi.
Alright. Hello, and welcome to Data Driven, the podcast where we explore the
emergent fields of artificial intelligence, data science, and,
and, of course, data engineering, without which the whole world would probably stop turning.
And you know, data engineering is important. That's
basically it. Still working on that that that revamped
monologue, for, for season 8, Andy. Were
you on vacation? You're on vacation. I am on vacation. And
for those of you who can't see on camera who are not who are
listening, not watching, I am literally lakeside,
in the foothills. Well, not the foothills. We are actually in the Appalachian Mountains. Or
is it Appalachian? I I never I I've heard of those. I I never
got a clear read on it. Say either. So, you know When I say either.
Yeah. Yeah. Yeah. Yeah. Yeah. So I am in Deep Creek Lake,
Maryland, which is kind of like, Maryland doesn't really have a Panhandle
per se, but if it did, it would be this is what this would be.
I probably think I'm 5 miles from West Virginia and about
20 miles from Pennsylvania. So it's kind of like this quiet
little corner of the state.
And I've been, you know, reading and studying
today. I hit day 600 on Pluralsight Consecutive. Nice.
So recording this June 17th. And, how
things with you, Andy? Things are good. I'm gonna throw out a plug for
data driven media dot tv because Frank mentioned.
If you're listening, he while he was mentioning that, he was
actually panning the camera over to the lake. But if
you're, subscribing to data driven media dot tv, you get
to see us. You get to see the video, and you
can see, for instance, that I am wearing the, my data is the
new oil t shirt, which you can pick up. I'm just full of
sponsor stuff today. I'm just doing Well, it's self out. It's
self sponsored. And, honestly, we really need to get better at that. Right? We have
data channel. Tv. There is a for listeners to the show, I will give
a preview. There is gonna be data driven academy is is launching soon. You have
a course coming up the end of the month. Actually, yeah, it's fabric.
Today. We're recording this on 17th. It's 24th
of of June, but I'm also doing, 2 more, at
near the ends of July August. And in addition
to that, while we're shameless plugging away here,
before we get to our very interesting guest, now I'm also bringing
back my, day of Azure Data Factory as wildly
popular. I delivered it at a couple of, conferences,
international conferences, 22, 23. And,
yeah. Let's see see if people are interested. What do you do Friday this
afternoon Friday afternoons, Andy? Oh, there's this thing, Frank. Thanks for
mentioning that. Totally free. We we gotta we're trying to get better at this. That's
all. We do. Yeah. Data engineering Fridays. And if you go to data engineering
fridays.com, you can learn more about that. Frank, you're doing a lot
of stuff with I noticed with using the, encore
replay feature in Restream. And it's
right you you shared that with me. I started doing that with data engineering
Fridays as well. But great a great way to,
you know, to get your message out there. And, you
know, I I had no idea replays would help. But my gosh.
They really have. It's just a matter of just hitting the echo of I
can't even talk. Algorithm the right way. Yeah. And Yeah. You know,
maybe we can get the so I think it's a good segue, for our
guest. Doctor Yossi, Keshet. He's the chief
scientist at AIOLA, an AI powered tech
company that automates business workflows
by capturing spoken data. Yossi is also
an associate professor at the Faculty of Electrical and Computer
Engineering at the Technion in Israel.
Yossi is an award winning scholar and has published over a 100 research
papers about automated speech recognition and speech
synthesis. Welcome to the show, Yossi. Hi.
Nice for having me. Thank you for having me. Hey. No problem. No
problem. We are very excited to have you. And, you're not just an
academic, but you've also proven yourself in in actual enterprise. So
which sounds really bad as I say that out loud, but I think you knew
there was a compliment.
But, so what is AIOLA?
Can you tell me a little bit about that? Because I'm curious about that and
and and workflows
around spoken data. So
Iola is a company that is aimed to target
the, you know, the very basic and foundational
industries. Maybe if I
may, let's start with the a general scene of the
automatic speech recognition now, and then you will understand where are YOLA stands because we
have now open AI and everything is like we you
can say we solve the AI problem. So it's not like that.
So we are in a in a amazing shape in in
terms of automatic speech recognition. So we we have a paper that shows
that whisper, the model of OpenAI, is as good as humans in
detecting and transcribing language when we speak about
American English with noise, without noise, and
also, l 2 speakers. That is the
speakers of non non native American speakers of the
language. And the the results are so whisper. The
OpenAI model is the same as human listeners. And that is
the main thing. But the thing is that
when you come to industries, usually they have jargon, they have special words.
And and those words are either rare in
their language or they they they are not none
word. It's like I don't know. I when I'm a medical doctor and would like
to make a surgery surgery and I would like to transcribe what I'm saying during
the surgery. I'm there isn't words that which are not
often used or which are none, non English words. And
in that case, those, automatic speech recognizer doesn't
work at all. They don't detect those words. And in Ayala, this
is our target to take those words, which are actually the most important word. Those
are the jargon of the of the industry of the of the facility.
So the goal is to help those industries to come
up with the with the automatic speech recognition for
reporting for transcribing speech.
I have a question. When you say automatic, what what makes it automatic? Is
it just kinda, what exactly does that mean?
So automatic speech recognition today works very similar
very, very similar to the way KJGPT works.
KJGPT works on a model called transformer. It's an, deep
learning architecture, which has, a
history based on previous recurrent architectures.
And it can predict, as as we all know, it can
predict text amazingly. In speech recognition, automatic
speech recognition, it's almost the same thing, but there is another
component, to the to the to the
this transformer, which is which is called encoder.
This this part take the speech and actually transfer it to
a great representation that can be used
with this, with this, let's call it with this with the other side, with
this, GPT together. Together, they can,
transcribe speech in, as I described, in a very good
way, as good as humans in some
cases. I will say, like,
I've been messing around with the app that's on the phone,
for, chat g p chat gbt, and,
I use the the voice interaction feature. It is
amazingly good at getting rid of the umms, the ahs,
the scatterbrain thoughts that I sometimes have when I talk to it.
Like, it it could kinda really distill a lot of
things. Like, I'm impressed with it. It's it's really gotten last time I
did anything serious with speech recognition was probably, like, maybe 4 years
ago, and it's really improved. Like, I mean, orders of magnitude
than I thought. I mean, it's it's it's it's almost at Star Trek level. You
know? I'm not sure
in those it depends on the company if it's Apple or
Google. And I'm not sure which they don't declare
which models they use. I think, personally, they don't use this whisper or
the latest model that we have for automatic speech recognition that
is transcribing speech. And the goal is a little bit different
in the in the phone. You actually want to maybe Right. Make,
make notes, send an email, send a text message,
and maybe the vocabulary the vocabulary is less
less defined. There is another problem with
the phones. Oh, no. Go ahead. I want to call my
friend. His name is xi, and
the last name is CHUNG. How do you pronounce it?
What what do you do with that? I'm gonna say he or chi or
so there is a there is a problem of proper name and how do you
define them. And this is a completely different problem. It's still an open problem, and
the goal is a little bit different. So
it's when we assessing the quality of those models, it's
a little bit different than the assessment of just spoken language
like what we do now. No. I mean, that's a great point. I mean, my
last name has, you know, technically is Lavin.
But, you know, growing up for for reasons many,
big and small, it became Lavinia. And like, so, like,
the phone, depending on if it's Android or an Apple, it will, it
will he gets confused pretty easily.
And that is an interesting point. Some names, Andy is lucky to have an
easy name for the, the system.
But not everybody does. So I understand that. Sure.
I also wanna double click on American
English. You you you said that a bunch of times. Like, is there is there
an inherent bias in these model trainings because these are done by American
companies? Yes. There is. Okay. The
day the data is mostly of American English. The research institutes
are mostly American. So the reason maybe I don't know
if you'd call it you call it inherent or implicit bias, but there is a
bias, definitely.
We are investigating, by the way, the the intelligibility
of speech in some cases And what is the intelligibility of
of American listener versus the inter intelligibility of
myself, which I'm not American listener, but I I know English.
What is the best, what is the best, double quote speaker? What is the best
listener? How can we transform those
to speech recognizer? How can we transform those to assessing the
quality of speech? What does it mean? What does it mean about the pathologies in
speech? And this is ongoing research on
this on this field. Interesting.
I I often wonder, like, you know, what it's not just English.
Right? Like, you know, if you listen to Spanish, like, there's different dialects of
Spanish. Right? Even even German. You know, I'm sure
there's, you know, plenty of dialects of all these languages and,
like, how do you the training of a
model that where it can get to be as good at
understanding x and x versus x and y versus, you know,
the base language, the base standard. I don't know. That's
fascinating. It seems like it seems like it could be an endless loop of, like,
training. It it is. Indeed, it
is. And when we train, there is another so I'm I'm
working on deep learning and AI. And what we found out
that it it may it may be the case that if you train
on 1 language, huge amount of data from 1 language, let's say
American English, but then train on less data on Spanish,
you actually get you get some advantage of training from
from the American English. So, again, in this modern whisper of
OpenAI, most of the data is American English, but,
actually, other languages are really great.
Again, Spanish is amazing. So maybe like
humans maybe like humans as we learn more and more languages, it's easier
for us. This is very interesting, point.
No. That's an interesting idea because I know, like, I never
understood American English grammar, American or otherwise,
until I studied a foreign language. And then when I studied it, it was German.
And, you know, German kept a lot of the archaic things that
are in English and kept them and kept make kept them,
made continue to keep them important. Like in English, you know, who
and whom used to confuse the you know what out of me.
Right? But when I when I learned in German about different cases and things
like that, I was like, oh, that's why it is. Right? So,
like, all these things that just like you said, like, learning another
having more data or data from another point of view, I suppose,
or another way to look at the world help me look at my world
a little better. Maybe maybe that's how
AI will work too. I don't know.
Maybe. We don't know. We we actually have a guess about that
because it those networks actually solve an optimization problem,
mathematical optimization problem. It's a problem that
that is, we define it with equation, and we need to have
a computer running and solve it. The equation is
overtraining set of examples. So it's 1
1 person say that, another person said something else.
And what happened is that when, again, when we have
a large amount of data,
it seems that those those networks get to an amazing place.
So this this, algorithm, this whisper or other
algorithms, it's really from the recent years, like 2, 3 years.
That's it. We it's they they perform amazingly
amazingly, with the with the
same with the same mechanism, not with the same amount of
data. Yeah. That's that's that's the
fascinating aspect of all of this. It's just that some of these things just seem
some problems seem harder than they ought to be,
and then some solutions to problems seem way more effective than they
ought to be. It's an interesting also to say
it's always the case that we so Whisper, OpenAI Whisper, was trained
on 600000 hours of speech. But this is
way, way much more than just a kid learning a language.
Kid language learning a language exposed to way much less hours of
speech, less less accurate, less,
coherent. And this is something,
Nom Chomski raised years ago, like, 50 years ago.
And it's still an open question. Like, if we can make those
system works better, if we know the language,
I guess you learn German faster than any
machine that works today.
That's yeah. It's it's and I'm glad you mentioned Noam
Chomsky because that kinda was like so for those who don't know, Noam
Chomsky is, among other things, a noted linguist scholar.
I highly recommend you do a search on him because that's a that's a
good Wikipedia rabbit hole to fall into. But,
how much does linguistics come up in this? Right? Because I think
what's fascinating about this field for me is a lot
of, my grandfather, my great grandfather
was a a linguistic professor. And, you know, as the
family lore goes, I never met him. He died decade or 2 before I was
born. He spoke, like, 12 languages. He was a professor of, like, 5
or 6. And, you know, a lot of people in my family
seem to have on that side of the family seem to be gifted in language.
And 1 of the fields I was tempted to to study in
university was linguistics. And I just find
it interesting how there's
a now a Venn diagram now is much larger
than it used to be in terms of linguistics and computer science.
So what are your thoughts on? Like, how much does like,
if you're if you have a
company like AIO. Right? Like, how many people are, you know, honest to
goodness, linguists versus computer scientists and and AI engineers?
So there is there is no no linguists there. Oh,
really? Okay. There are no linguists. But I have to tell you, so there was
a professor called Freddie Frederick, Jelinek. He was the
head of language, research at the John Hopkins University
at Baltimore. He was amazing. He was 1 of the smartest,
people on earth. And he said he was
developed many of the speech recognition algorithms. He said,
every time I fire a linguist, the performance of speech recognizer goes
up.
And this is, this is embarrassing. But I've been I
made myself, 1st, really like
linguistics. I really like cognitive sciences, and I really
try to combine it with with my work. But it's really
amazing that we don't have all those AI system
don't have any of that. So you don't train CEGPT
to what is a noun, what is a verb, what is anything. You don't train
speech that this is the
this is the you don't you don't use linguist. You don't use this is
the prominent word. This is the end of the sentence. It just happened
by huge amount of data. And
this is interesting. This is somehow contradict Noam Chomsky who said that
there there is a universal grammar. There is a
we are born innate with language. There is a
maybe some black box in our brain which
is tuned to learn a language. And,
we are not sure about that. There is no direct proof if it's correct or
no. We are born with language. We are as humans, we're
born with language. We this is part of our, human being.
We are not born with written language. So written language was invented.
The spoken language is something like like a zebra
has stripes. This is this is our nature, and this is
interesting. This is not happening not happening in
AI. The best success that didn't have linguist, they don't have any
restriction of what should be say or not.
Maybe maybe AI will be a tool to somehow
make the linguist research more effective and
try to understand what happened in the brain, what happened in the cognition part.
But I would like to tell you about another research we are preparing here, which
is really amazing. 1 of the thing is that we have
so there is this JGPT. It's a language model.
We also have something in the brain. It's also neural network.
And we when we try to compare them, there is a huge
correlation between the the what happened in the artificial neural
network of GPT and the neural
biological neural network in the brain. And, it was
shown, several years ago, and here we
show it again with, with this, with the most modern,
automatic speech recognizers. So this is
a phenomenal post correlation between the artificial and the
neural mechanisms. I was gonna ask about that
because I'm I'm familiar with, you know, at least the abstracts of
the research, from a few years ago and now. And
I was curious if there had been any new correlations
or, you know, or new research, new connections that have been made
between machines learning languages
and the way our brains work. It sounds like
that's true.
So we try to we just initiate, man,
a research here in my lab about that. There was
some French guys from, mainly King
and his colleague at, Meta. And
and I forgot the university in France. So they
show that there is those correlation. They show simple correlation. What we
they show it with LLM, with language model. What we show is a little bit
different. We show correlation with automatic speech
recognition. So we ask people under fMRI, under MRI.
They're we scan their brain at some
resolution, and we try to find correlation with their brain activity
during reading and during speaking aloud,
and ask what is the correlation with the the best model we know for
speech recognition. And then there are correlation.
I have to say that there is a mechanism in the transforming this
architecture of neural network. There is a mechanism called attention. This
mechanism allow those model to to have the connection between
worlds and themselves. So, I'm eating an
apple. It was delicious. So it refers to the apple.
Okay? So there is attention mechanism. This what makes those
model amazing. So there is attention mechanism, I guess, in the
brain. So we try to correlate the this attention mechanism in
the models and compare it to what the activity in the brain. We don't have
results yet, but it seems promising. And we also ask
another question. What if you don't read aloud? What if you read
like silent reading? What if you have dyslexia? What if you have,
other type of, pathology? What
what are the correlation then? So this is fascinating. So and
there is correlation. I don't I don't know still what what's going to happen
with that. But I I know the pathologist, but it's unbelievable, the
correlation. That that is really exciting,
especially when you're examining things like dyslexia,
which is considered, you know, not normal,
or maybe that's not the right term for it, but a
challenge at a minimum. The cool the cool kids call that neurodivergent
now. I think Neurodivergent. Thank you, Frank. So when you're studying, you
know, when you're studying that sort, I'm wondering if there's a place for
that, in in the artificial.
I'm curious. What what do you mean? Can you
So, yeah, is there is is there any benefit
to, I say, transferring the thought processes
of people who are neurodivergent and and automating that
and making that part of the, you know,
the the language model or or speech recognition?
Yeah. I think so. I think so. 1st, it's a it's a tool
to to an to analyze what happened in the
brain. Yeah. What happened
but it's very difficult. So we don't, we don't have any debugger for the build
the brain. We don't see the code of the brain. We don't see that this
function doesn't work. And it's, most of the work
is to design the experiment and
and it's really amazing. In our design, we have the
same so as yet as I told you, I'm asking people to read aloud
and compare it to what automatic speech recognition,
is plan is, supposed to do. But I'm
also asking people to read silently, and then I follow
their eyes. I have a make a make a machine that follows their eyes, and
I know where where is the where like, III
track their eyes and I see which wall they are reading
now. And I can and I can use that to follow
what what they read. But in order to operate that on a speech
recognizer model, I need the speech. So it's during the design of
the experiment, I need artificial speech or I need them to to read aloud
afterwards. It's a it's a big, it's a big question
how to do that properly and how to
make things happen, but definitely walking with
people with, with problems first to help them.
And second, to understand them. And 3rd, to maybe make
understand the brain and make, AI better.
I also think, like, stroke victims, right, could benefit down the line
from a better understanding of lang language models. Right? Like, maybe there would be some
kind of therapy that could be directed to that. I think I think it's
fascinating. I always love those fields where they touch upon more than 1 thing.
Right? This isn't just math. This isn't just computer science. Like, it's linguistics. But,
you know, it's a little bit of everything. It's like a giant, like, pot of
stew that you just throw a bunch of stuff in, and it all kind of
mixes. And, like, it's kind of like, almost like intellectual gumbo,
I guess, would be the word. Right? But,
what what,
what drove you to make, your your your
your company? Like, what what was the driving force to
say, hey. You know, we have
I remember many, many years ago in an office, and you would always see
doctors talking into these little, like, miniature recorders.
Right? In the olden days, they would go off to
some data center somewhere and somebody would not data center, but, like,
some piping center, call center where people would
transcribe that. You know, obviously, that is now an artifact of
the past as these models have gotten better.
What what was the goal in in in, your
company to say we can do this better? What what was the the that breakthrough
moment of, like, here's here's what the industry already does. Here's how we can do
it better. So there is
so we all know Check GPT, and it influence our life. We search now
instead of Google, we search with GPT and it's amazing. It's unbelievable.
So I thought, what about the very fundamental industries? What
about,
like, when you check-in when you, check an airplane, you
use a special jargon. You cannot touch anything. You cannot
leave even a pen there because otherwise the the plane wouldn't be,
valid for flight. What about industries like the food
industries when you need to report, the process? You
have gloves, you cannot touch an iPad, you cannot barely
write. And what about, other industries
like, maybe the cheap technology when you make nanotechnologies and
when you make chips, you make, you know,
silicon chips and silicon
first. So you need you you are cover all.
You are with gloves. You need to report the process. It's a all
those industries has this have special jargons. They use special
terms to describe what they're doing. They don't have access to
to to write something,
and they are very limited in the way they they provide. And on the other
end, we had speech recognition, but speech recognition doesn't work on
those jargon world. Those jargon world are actually the
most important to those industries, and this was the goal for
Iola. So what we do is we operate,
automatic speech recognition, the best automatic speech recognition,
but we also operate something else. We also operate something called keyword spotting.
It's another deep network, which is focused
on detecting only the jargon words. So you can define those jargon
words in advance. You don't need to train them. You you can
define them, and it they all work together. They work like, as a
complimentary, couple to make a
very robust prediction, and we can detect those,
jargon words and make reporting on on on on the
process, without just by speaking. So it
can it can use in any industries,
any, industry that doesn't
have access to the most modern AI system, the speech
recognizer wouldn't walk there. They have problems, like,
writing and formulating their reports.
Yeah. So I'm curious how those work together. You mentioned
that you've got the speech recognizer. You've got the keyword,
engine. Are they 2 separate engines that are just always running
maybe agents, running at the same time or are
they encapsulated, say, is the speech
recognizer does the speech recognizer have a, you know, a
subset or a a function built into it to do the
keyword recognition? So just to
be sure, those keywords in some industries are not are
not are not English words. So it can be a word which nobody
knows about. It was not shown in the in
the, like, in the Internet, like, JGPT strain on the data over the
Internet. There are some walls that are not not there. This is
your, proprietary company. You have invented a wall to
describe what is the this, part of the engine. So
Yeah. So what we so we have this keyword spotting. It was it it
is trained to detect keyword in general. They are defined by,
by text and it operates. We have 2 model for preparation. 1 of them
works on the this encoder part of
the of the automatic speech recognition, and then it guides.
It's still the speech recognition towards the correct
transcription. And there is another mode, which is,
our self, encode our self representation of
speech, and then it also guides the automatic speech
recognition to a better, location and to detect those
words. And, actually, we can show that you can buy combine
any word can be from different languages, and we can
detect them, like, almost 100% correct, those jargon
words. That was that was going sorry. Go ahead.
No. No. No. Sorry. That no. That's okay. That that makes perfect
sense now, what you just said about the languages using
multiple languages, you know, English plus all of the
other languages because sometimes
people will struggle if their English as a second
language speaker. They'll struggle to find the right
English word, and they'll substitute a word from their native language.
And in other cases, they'll be perhaps teaching
on a topic, and they may revert back
to an older language, Greek, Latin, something
like that. That may be part of the, the
lecture or, you know, I could see that in
medicine. I could see it in, you know, all all sorts
of literature studies. I could see a lot of that. And that
that kinda clicked for me as you were saying that that makes sense that you
would have additional languages. Yeah. I also wonder, like, in in
also conversational context. Right? Like, you know, Spanglish is a
thing. Frankel is is the French and
English kinda mashed together, and I know that other language
whenever you have 2 groups of people kinda come together, like, you know, there's always
some kind of weird mix of language that that kinda
just evolves either naturally or forced. I mean, that's Right. That's another
debate. Are you thinking belt or creole? I know we're Belter, you know, I
wasn't going there, but that that's a that's an excellent example.
So, Yosie looks very confused. So so there's a series of
books, called The Expanse. It was an excellent TV show
for about 6 seasons, and it's basically set, 2,
300 years in the future.
And as humans colonize the asteroid belt,
their people from all over the world kinda all end up living
together. So, like, the the Belter Creole language is this is a
creole of, you know, literally dozens of languages. Right?
So, like, it'll switch from, you know, Hindi to Arabic to,
English to French to there's even some German in there. I've heard some of that.
Like, and there are these kind of these weird mixes of things. Right? So they'll
say the the word for the Belter people, like,
people live in the Belk, is Beltaloda. Belt obviously comes from, you
know, the asteroid belt English. Loda, I think is a Hindu term. I
think. Don't hate on me in the comments. Don't hate on me in the comments.
But, I know Walla is a is a is a Hindu term. Right? So
they'll they'll, you know, when they talk to people who live in the Earth or
Mars, they refer to them as well wallahs, gravity well
wallahs. Right? Like so it's like, and I only know wallah because
of dish wallahs, and Wired Magazine did a whole story about dish wallows in
the nineties. Anyway, but I mean, I think, like, you know, I
I suppose that approach could work for something like a creole. Right? Like, we have
multiple languages kinda mixed together. Or is that not really a
massive business case?
It's Creole is really complicated. It's a language. It's like real real a
real language, and it's complicated. This the the more
delicate cases of that, what we call in research, code switching when
I'm Right. When I speak Hebrew, for example, I don't have a
word for the, you know, the Internet router. So I say the router in
in English. Or I said email or I will say
I don't know. There are so many words in English that are used especially
in technology that you use worldwide in other languages, and this
is code switching. There is another case. I think Andy pointed it
out that sometimes when you are stressed
or let's say your l 1 is Spanish, but l 2 is American
English or you're bilingual. And sometimes when you are
stressed, you you just switch the the 1
word and it this is amazing phenomena. This is a research with Tamar Golang
from, University of San Diego and Matt Goldrick from Northwestern
University. And I provide, again, a mechanism to detect
that and to make research of that. And the the key question is,
like, why do you do that? Why do and when do you do that? Is
it stress? What what what is the what is the state of
describing those? Are you gonna describe it in the American
way, the Spanish word, or is it gonna be vice
versa? And this is really interesting.
It's not my field of research. I just know how to detect them
and, and Interesting. To detect them really well,
but I don't know why it happens and what is the mechanism
behind that. I could definitely see,
the opportunity with starting with being
able to detect, you know, these I
don't I don't know the right word for them. I'll I'll call them modes. You
know, a mode of speech where someone is mixing 2
languages. And I'm sure those vary.
So Like when I go Jersey on you. Right? That's we we
can't we can't say any more about that, Frank. We're trying to keep our
clean rating. But yes. Exactly. But,
that's sorry. Inside, Joe. But the,
but, yeah, I could see modes of speaking where someone who is
more familiar with English as a second language.
And and they've still you know, of course, they know their native language. They'll always
know that. But as they I don't I don't wanna use the wrong word
here, but I'm thinking experience is probably the best word is they get more
experience, gain more experience with their second language.
They may switch words less or switch languages
less. And detecting that, I think, is the
is key. I understand now more about what what you're doing, what
you're accomplishing. And that that's the
very first step to then being able to produce speech
in those different modes. And that would be a
fascinating, you know, a fascinating accomplishment.
If you do, the more we can have. Machines
speak to us in the language that we're most familiar with, that,
of course, you know, is is almost there now, mostly
there right now, but have it be able to to speak to us in these
different modes where we where the machine switches where it's
back to our first language, you know, based
on some algorithmic calculation. That sounds
fascinating. Yeah. It is.
I'm not sure we are there yet. It's we have a long way to go
there. But, Sure. Yeah. Makes
sense. Fascinating. Well, this is how it starts, though. Right?
This is fascinating. This is, yeah, this is,
somehow there is an elephant in the room. There we may have to say
something about AI and their regulation and what happens now.
And, if I may, I would like to say something about this because I have
a deep totally different point of view about that.
Please. So everybody is speaking about
regulation and it might be a catastrophic situation
if those, machine are connected
together and they start to train themselves. They try to
build a meta architecture and try to train themselves,
and then they come up with something which is better than human. Some some people
call it the singularity point. So this is frightening. They're smarter
than us. Maybe they they're gonna kill us all. And
people say now people speak about regulation now, and there are
several institutes in Europa, in Europe and in, the US
trying to tackle that. And that
is amazing. That is really important, but I think we missed something here.
And I'll tell you why. So the so there is a book. It's here.
You know, Isaac Asimov, I, Robot. You probably
know that. So he, like, the first page of this book is like the 3
laws of robotic. A robot may not in in injury a
human being or through an interaction, allow human being to come to harm.
A robot must obey others and so on. So we have let's say
we have the regulation. AI cannot hurt humans. Okay?
But that doesn't enough. It's not good enough because if the AI is smart
enough, it will not do the I mean, it will
show us humans that it really obey the law
the laws, but it wouldn't. And this is frightening.
And here I suggest to look a little bit about the human morality
and what why human are have do they have laws? So we need to
think about, if I may, think about the
human psychology. In human psychology, we have a mechanism to obey law.
It's called the superego. It was embedded or defined by
Freud. So we have a mechanism that if we
if we doesn't we if we don't obey a law, we feel either
guilt or fear. And this mechanism was evolutionary.
So do we have a group of monkey? They obey
the the alpha monkey because they're frightened from him. They have some kind of
primitive superego. We obey the law because either we fight them from the
police or either we feel the guilt, we
we it's like the
those experiments that show that, there is, somebody,
left something on the table, and we don't take it because we feel guilt or
we feel something. So this is this mechanism, what
I claim, should be transferred to the
AI machine. This should be the regulation. So what is it superego? Superego
is a infrastructure for to be moral,
and we need a digital version for that for the this is the regulation we
need. We need the infrastructure to be moral in machine. And what it what
does it mean? So superego means that it's a little bit like
self harm, if I may. It's like we feel guilt. We feel something bad if
we do something not okay, if you're not obey the law.
So it's like a self destruction for AI machine. So AI machine,
if it doesn't obey the law, should feel something. It
cannot feel so. Right. It will distract itself. So this is my
claim. This is a book I'm writing, and this is something very fun fundamental.
We we all speak about this regulation, but I think it
it doesn't help just to to do standard
regulation. And if you if I may say another thing, the last thing is that
if you read the I, Robert, carefully, so
he speak there are several short stories there, and he speak about robots that
obey the law. And if you look carefully about those robots that
obey the law, the those robots have super all
all of them have have super ego. They feel guilt.
The the first story is about a robot that play with a girl,
and he feel guilt about winning all the time. So he let her win.
So he feels guilt. It means that it has superhego.
And then he feels frightened from the mother of the girl. And it's
really amazing. So I think, so
this book I'm trying to describe the psychological concept of superego
and then describe why it need to be more and how we can,
find a way to put it in regulation, like the the infrastructure
itself and not just lows.
That is a very interesting problem you're trying to solve.
Very important problem at that. Agreed. And
culturally, we speak, in the US, we have a saying that you
cannot legislate morality, which
legislate, regulate would be, you know,
synonyms. Exactly. Right? So Right. Right. And and legal code
is code. I I
definitely get what you're what you're saying. And I think it's super
important. You mentioned you were writing a book about this. Now
now now you have to tell me more because I wanna read this book.
Same. I'm in the process of looking
for an agent and it's, it's complicated. It's supposed
to be a popular book trying to explain the psychology of fraud.
What is, superego, ego, and the id,
and then describe what is the pathology? So we all have a pathology. So
you have the pathology of, it's called,
the, personalities criminal personality disorder. This
person will not have a super ego, ego ego. It's like Richard the
third from Shakespeare. He didn't have superego. He killed
his family and didn't feel guilt. So this wouldn't what's
going to happen with the with the with those machine. And then I
give some literature examples of,
what is a superego like from the, criminal and
punishment that that the guy killed the the
old lady, but he didn't he nobody,
caught him killing the lady. He murdered her. Nobody caught him, but he
still feel guilt. So he has a very, big
superego. And then we describe I describe, what happened in
other moral theories of human being, all of them connected to the
superego. And then I tried to describe a little bit how machine
learning is trained. Again, solving an optimization problem. And then I try
to describe how can we do superego with, how can we have
a digital superego if we can? No.
It's like you're giving it a conscience of of sorts. Exactly.
Yeah. And I I just wanted to, to add, we
may be able to help you. Maybe not find an
agent, but find a publisher. Both Frank and I are
published. And we, you know, we know Andy has a lot of
Andy's got a lot of connections in the publishing. Well That would be
great. I am I am not, I just wrote a lot of books
for different, publishing houses, and I know some people that if
they can't help you directly, they can probably point you to someone who
can. And, again, I am wholly motivated by wanting to
read this book. Same. Like, I think it's important
because I live in the Washington DC area. Right?
So so, like, there's a lot of people there who they're policy
makers. Right? Like, and they just assume
and I think a lot of humans fall for this. Right? You you see this
when the European Union passed their AI regulation act.
They assume that regulation's gonna solve all their problems.
And I think regulations prove that 1 of the fundamental forces
in the universe is is unintended consequences.
And, you know, when you regulate something, you don't end
the problem. You change the way people will route around it. Right? Like,
and I think a good example of this in AI is the movie Megan, which
I don't know if you've seen, or m threagan. I'm not sure how to pronounce
it, where I think she was about to torture
she was I don't wanna give the plot away, but the the robot
child, Chucky, kinda goes evil, Like, this is the
basic kind of plot line, and the the the person who created her
was like, you can't kill me because it's against your programming. He goes, oh, I
said nothing about killing you. I was gonna put you in a coma, and you'll
live, you know, however many years. Like, it was just like I mean,
that's a great example of, like, she you know, don't kill. Right? Seems like a
pretty reasonable instruction to give a robot, particularly a child's toy.
They'll kill anyone. But, you know, she was realized, like, well, kill
equals death. So if I don't kill you, if I just hospitalize you or
incapacitate you, that doesn't conflict with rule number 1.
Right? Which I think is no. Obviously, as, you
know, humans, we're like, well, it's not really the spirit of the
law, or the rule. But clearly,
the robot or the AI in this case, kind of figured it
out. Like, I don't know. I think you're right. Like and any regulations like that
too. Right? How many loopholes do people discover, whether it's
tax laws or, you know, this. It's like, well, technically, it's
legal. Is it actually, you know,
what the law intended? No. Like, it's Yeah. You need a you need
almost an something like a Nuance engine,
you'll see to Yeah. To get the the
what the machine to interpret
to the laws. And that's I've read Asimov as well,
big fan. And that's what happens down stream of
the 3 laws as they begin to fail as because the
robots are doing exactly what they're programmed to
do. And they're not they're they're
finding ways that in our opinion, human opinion,
circumvents the 3 laws, but really doesn't
break the robot's programming. And it's all about, you know,
how do you define harm? Like, Frank's example is a great, you know,
great example of that. So, yeah,
fascinating stuff. Yeah. We gotta Awesome stuff. We gotta help you write this
book. I wanna read this book. Yeah. I want to raise
another point, but the opposite point that you raised. Like, what happened with
the autonomous car, for example, or people say,
let's let's let's focus on autonomous cars. So so there will be
autonomous car. Who is in charge of a of a car accident?
Accidentally, somebody was killed. You are the
owner you. Somebody is the owner of the car. He sits
there. He bought the car, but the car killed
somebody. So
who who this is an open problem. This is, again,
moral problem. So what I suggest here is
maybe it will take time,
I guess. Maybe the the car, if we can be the
superego and mechanism for morality, you know, the just
the infrastructure for morality can take the
morality of the human. And if somehow he
inherit the the the driver morality, you
can blame the driver. I'll give you another example, which will be much
more maybe concrete. So we say now that there will be change GPT for
every person, for every laptop and iPhone and whatever.
You will have your own GPT with your own life follows
your own history. And the discussion with this GPT will be, And the
discussion with this, GPT will be very personalized and
very helpful. What happened in that case? So in that
case, if this, GPT
will take your responsibilities and morality, somehow we
can copy your morality and be part of it. So if you're moral, it
will be moral. If you're not, you're not, but this is
your responsibility as a human. And I think this
is the way to to go with that. We need just the infrastructure and not
the the law. Anybody can define the low, and anybody
can break the low. We just need the infrastructure to know that
at least the machine to know that it break the broke the low.
And and this is really important. I I think
Oh, I totally agree. Totally agree. Well, we're
gosh. We're coming up on time, Frank. Yeah. This was
awesome. So we'll just any
book recommendations? Obviously, I, Robot, I think, would be good reading
in this space. You also mentioned Shakespeare too,
Richard the 3rd. So Eddie, you can book
which I'm which I'm reading now, which is the band,
Vernon Stuputeux. It's, it's
amazing. It's amazing. It's 3 books, and it's actually
discussed whatever which is not AI. Anything which cannot be solved with
AI. It's speak about a a person who has a vinyl shop,
shop to sell vinyl and then CD runs, and now we cannot sell
anything. So this shop is is closed, and then he
he he try to somehow manage, but he get up at the street. He's, like,
homeless, and he meets many people. And the way like,
every chapter is a different, person or
or a group of pair of people, and it's really
fascinating. It's all those things that you cannot solve with AI. It's all
the human interaction, the very, very basic human interaction. Amazing.
It won the Booker Prize in the, 2018.
Nice. Where can folks find out more about
you? So I have a website
under Joseph Keshet, and, and they
can find me there. Excellent.
Any parting thoughts, Andy? No. Just great great
interview. I appreciate that. 1, I would ask if you repeat the name of
the book you just mentioned about the the different stories.
What's the name of that book? It's not it's a it's a single
story. It's called the the pants,
for non subtext. It's from French. Oh, okay.
Amazing. Amazing. Amazing. Awesome. Excellent. That's it. That's
it for me. But that's great talk. Thank you. Excellent talk. Thank you.
And we'll let Bailey finish the show. Well, folks, that brings us to the end
of another enlightening episode of data driven. We've
navigated the fascinating intricacies of automatic speech
recognition, explored the moral quandaries of AI, and
pondered the future of technology with none other than 1 of the best minds
in the field, doctor Yossi Keshet. Remember, if you
enjoyed today's conversation, don't forget to subscribe to data
driven media TV for exclusive video content.
You can also grab some fantastic merch like the my data is the
new oil t shirt Andy's sporting today. And while Frank is
basking in the Appalachian sunshine, you can bet we're already cooking up the
next episode to keep your data driven minds engaged and entertained.
Until next time, stay curious, stay informed, and
always keep questioning. Cheerio.