Voices in AI – Episode 37: A Conversation with Mike Tamir

[voices_in_ai_byline]
In this episode, Byron and Mike talk about AGI, Turing Test, machine learning, jobs, and Takt.
[podcast_player name=”Episode 37: A Conversation with Mike Tamir” artist=”Byron Reese” album=”Voices in AI” url=”https://voicesinai.s3.amazonaws.com/2018-03-27-(00-55-21)-mike-tamir.mp3″ cover_art_url=”https://voicesinai.com/wp-content/uploads/2018/03/voices-headshot-card-1.jpg”]
[voices_in_ai_byline]
Byron Reese: This is Voices in AI, brought to you by Gigaom. I’m Byron Reese. I’m excited today, our guest is Mike Tamir. He is the Chief Data Science Officer at Takt, and he’s also a lecturer at UC Berkeley. If you look him up online and read what people have to say about him, you notice that some really, really smart people say Mike is the smartest person they know. Which implies one of two things: Either he really is that awesome, or he has dirt on people and is not above using it to get good accolades. Welcome to the show, Mike!
Mark Cuban came to Austin, where we’re based, and gave a talk at South By Southwest where he said the first trillionaires are going to be in artificial intelligence. And he said something very interesting, that if he was going to do it all over again, he’d study philosophy as an undergrad, and then get into artificial intelligence. You studied philosophy at Columbia, is that true?
I did, and also my graduate degree, actually, was a philosophy degree, cross-discipline with mathematical physics.
So how does that work? What was your thinking? Way back in the day, did you know you were going to end up where you were, and this was useful? That’s a pretty fascinating path, so I’m curious, what changed, you know, from 18-year-old Mike to today?
[Laughs] Almost everything. So, yeah, I think I can safely say I had no idea that I was going to be a data scientist when I went to grad school. In fact, I can safely say that the profession of data science didn’t exist when I went to grad school. I did, like a lot of people, who joined the field around when I did, kind of became a data scientist by accident. My degree, while it was philosophy, was fairly technical. It made me more focused on mathematical physics and helped me learn a little bit about machine learning while I was doing that.
Would you say studying philosophy has helped you in your current career at all? I’m curious about that.
Um, well, I hope so. It was very much a focus thing, the philosophy of science. So I think back all the time when we are designing experiments, when we are putting together different tests for different machine learning algorithms. I do think about what is a scientifically-sound way of approaching it. That’s as much the physics background as it is the philosophy background. But it certainly does influence, I’d say daily what we do in our data science work.
Even being a physicist that got into machine learning, how did that come about?
Well, a lot of my graduate research in physics was focused on a little bit of neural activity, but also a good deal of it was focusing in quantum statistical mechanics, which really involved doing simulations and thinking about the world in terms of lots of random variables and unknowns that results in these emergent patterns. And in a lot of ways what we do now, in fact, at Takt is actually writing a lot about group theory and how that can be used as a tool for analyzing the effectiveness of deep learning. Um, there are a lot of, at least at a high level, similarities in trying to find those superpatterns in the signal in machine learning and the way you might think about emergent phenomenon in physical systems.
Would an AGI be emergent? Or is it going to be just nuts and bolts brute force?
[Laughs] That is an important question. The more I find out about successes, at least the partial successes, that can happen with deep learning and with trying to recreate the sorts of sensitivities that humans have, that you would have with object recognition, with speech recognition, with semantics, with general, natural language understanding, the more sobering it is thinking about what humans can do, and what we do with our actual, with our natural intelligence, so to speak.
So do you think it’s emergent?
You know, I’m hesitant to commit. It’s fair to say that there is something like emergence there.
You know this subject, of course, a thousand times better than me, but my understanding of emergence is that there are two kinds: there’s a weak kind and a strong one. A weak one is where something happens that was kind of surprising—like you could study oxygen all your life, and study hydrogen but not be able to realize, “Oh, you put those together and it’s wet.” And then there’s strong emergence which is something that happens that is not deconstructable down to its individual components, it’s something that you can’t actually get to by building up—it’s not reductionist. Do you think strong emergence exists?
Yeah, that’s a very good question and one that I refuse to think about quite a bit. The answer, or my answer I think would be, it’s not as stark as it might seem. Most cases of strong emergence that you might point to, actually, there are stories you can tell where it’s not as much of a category distinction or a non-reducible phenomenon as you might think. And that goes for things as well studied as space transitions, and criticality phenomenon in the physics realm, as it does possibly for what we talk about when we talk about intelligence.
I’ll only ask you one more question on this, and then we’ll launch into AI. Do you have an opinion on whether consciousness is a strong emergent phenomenon? Because that’s going to speak to whether we can build it.
Yeah so, that’s a very good question, again. I think that what we find out when we are able to recreate some of the—we’re really just in the beginning stages in a lot of cases—at least semi-intelligent, or a component of what integrated AI look like. It shows more about the magic that we see when we see consciousness. It brings human consciousness closer to what we see in the machines rather than the other way around.
That is to say, human consciousness is certainly remarkable, and is something that feels very special and very different from what maybe imperatively constructed machine instructions are. There is another way of looking at it though, which is that maybe by seeing how, say, a deep neural net is able to adapt to signals that are very sophisticated and maybe even almost impossible to really boil it down, it’s actually something that we do that we might imagine are brains are doing all the time, just in a far, far larger magnitude of parameters and network connections.
So, it sounds like you’re saying it may not be that machines are somehow ennobled with consciousness, but that we discover that we’re not actually conscious. Is that kind of what you’re saying?
Yeah, or maybe something in the middle.
Okay.
Certainly, our personal experience of consciousness, and what we see when we interact with other humans or other people, more generally; there’s no denying that, and I don’t want to discount how special that is. At the same time, I think that there is a much blurrier line, is the best way to put it, between artificial, or at least the artificial intelligence that we are just now starting to get our arms around, and what we actually see naturally.
So, the shows called Voices in AI, so I guess I need to get over there to that topic. Let’s start with a really simple question: What is artificial intelligence?
Hmm. So, until a couple years ago, I would say that artificial intelligence really is what we maybe now call integrated AI. So a dream of using maybe several integrated techniques of machine learning to create something that we might mistake for, or even accurately describe as, consciousness.
Nowadays, the term “artificial intelligence” has, I’d say, probably been a little bit whitewashed or diluted. You know, artificial intelligence can mean any sort of machine learning or maybe even no machine learning at all. It’s a term that a lot of companies put in their VC deck, and it could be something as simple as just using a logistic regression—hopefully, logistic regression that uses gradient descendants as opposed to closed-form solution. Right now, I think it’s become kind of indistinguishable from generic machine learning.
I, obviously, agree, but, take just the idea that you have in your head that you think is legit: is it artificial in the sense that artificial turf isn’t really grass, it just looks like it? Or is it artificial in the sense we made it. In other words, is it really intelligence, or is it just something that looks like intelligence?
Yeah, I’m sure people bring up the Turing test quite a bit when you broach this subject. You know, the Turing test is very coarsely… You know, how would you even know? How would you know the difference between something that is an artificial intelligence and something that’s a bona fide intelligence, whatever bona fide means. I think Turing’s point, or one way of thinking about Turing’s point, is that there’s really no way of telling what natural intelligence is.
And that again makes my point, that it’s a very blurry line, the difference between true or magic soul-derived consciousness, and what can be constructed maybe with machines, there’s not a bright distinction there. And I think maybe what’s really important is that we probably shouldn’t discount ostensible intelligence that can happen with machines, any more than we should discount intelligence that we observe in humans.
Yeah, Turing actually said, a machine may do it differently but we still have to say that the machine is thinking, it just may be different. He, I think, would definitely say it’s really smart, it’s really intelligent. Now of course the problem is we don’t have a consensus definition even of intelligence, so, it’s almost intractable.
If somebody asks you what’s the state of the art right now, where are we at? Henceforth, we’re just going to use your idea of what actual artificial intelligence is. So, if somebody said “Where are we at?” are we just starting, or are we actually doing some pretty incredible things, and we’re on our way to doing even more incredible things?
[Laughs] My answer is, both. We are just starting. That being said, we are far, we are much, much further along than I would have guessed.
When do you date, kind of, the end of the winter? Was there a watershed event or a technique? Or was it a gradualism based on, “Hey, we got faster processors, better algorithms, more data”? Like, was there a moment when the world shifted? 
Maybe harkening back to the discussion earlier, you know, someone who comes from physics, there’s what we call the “miracle year,” when Einstein published his theory—a really remarkable paper—roughly just over a hundred years ago. You know, there is a miracle year and then there’s also when he finally was able to crack the code in general relativity. I don’t think we can safely say that there been a miracle year until far, far in the future, when it comes to the realm of deep learning and artificial intelligence.
I can say that, in particular, with natural language understanding, the ability to create machines that can capture semantics, the ability of machines to identify objects and to identify sounds and turn them into words, that’s important. The ability for us to create algorithms that are able to solve difficult tasks, that’s also important. But probably at the core of it is the ability for us to train machines to understand concepts, to understand language, and to assign semantics effectively. One of the big pushes that’s happened, I think, in the last several years, when it comes to that, is the ability to represent sequences of terms and sentences and entire paragraphs, in a rich mathematically-representable way that we can then do things with. That’s been a big leap, and we’re seeing a lot of the progress that with neural word embeddings with sentence embeddings. Even as recently as a couple months ago, some of the work with sentence embedding that’s coming out is certainly part of that watershed, and that move from dark ages in trying to represent natural language in a intelligible way, to where we are now. And I think that we’ve only just begun.
There’s been a centuries-old dream in science to represent ideas and words and concepts essentially mathematically, so that they can manipulated just like anything else can be. Is that possible, do you think?
Yeah. So one way of looking at the entire twentieth century is a gross failure in the ability to accurately capture the way humans reason in Boolean logic, and the way we represent first order logic, or more directly in code. That was a failure, and it wasn’t until we started thinking about the way we represent language in terms of the way concepts are actually found in relation to one another, training an algorithm to read all of Wikipedia and to start embedding that with Word2vec—that’s been a big deal.
The fact that by doing that, and now we can start capturing everything. It’s sobering, but we now have algorithms that can, with embed sentences, detect things like logical implications or logical equivalence, or logical non-equivalence. That’s a huge step, and that’s a step that I think we tried quite a bit to do, or many tried to do without experience and failed.
Do you believe that we are on a path to creating an AGI, in the sense that what we need is some advances in algorithms, some faster machines, and more data, and eventually we’re going to get there? Or, is AGI going to come about, if it does, from a presently-unknown approach, a completely different way of thinking about knowledge?
That’s difficult to speculate. Let’s take a step back. Five years ago, less than five years ago, if you wanted to propose a deep learning algorithm for an industry to solve a very practical problem, the response you would get is stop being too academic, let’s focus on something a little simpler, a little bit easier to understand. There’s been a dramatic shift, just in the last couple years, that now, the expectation is if you’re someone in the role that I’m in, or that my colleagues are in, if you’re not considering things like deep learning, then you’re not doing your job. That’s something that seems to have happened overnight, but was really a gradual shift over the past several years.
Does that mean that deep learning is the way? I don’t know. What do you really need in order to create an artificial intelligence? Well, we have a lot of the pieces. You need to be able to observe maybe visually or with sounds. You need to be able to turn those observations into concepts, so you need to be able to do object recognition visually. Deep learning has been very successful in solving those sorts of problems, and doing object recognition, and more recently making that object recognition more stable under adversarial perturbation.
You need to be able to possibly hear and respond, and that’s something that we’ve gotten a lot better at, too. We’ve got a lot of the work done by doing research labs, there’s been some fantastic work in making that more effective. You need to be able to not just identify those words or those concepts, but also put them together, and put them together, not just in isolation but in the context of sentences. So, the work that’s coming out of Stanford and some of the Stanford graduates, Einstein Labs, which is sort of at the forefront there, is doing a very good job in capturing not just semantics—in the sense of, what is represented in this paragraph and how can I pull out the most important terms?—but doing a job of abstractive text summarization, and, you know, being able to boil it down to terms and concepts that weren’t even in the paragraph. And you need to be able to do some sort of reasoning. Just like the example I gave before, you need to be able to use sentence embedding to be able to classify—we’re not there yet, but—that this sentence is related to this sentence, and this sentence might even entail this sentence.
And, of course, if you want to create Cylons, so to speak, you also need to be able to do physical interactions. All of these solutions in many ways have to do with the general genre of what’s now called “deep learning,” of being able to add parameters upon parameters upon parameters to your algorithm, so that you can really capture what’s going on in these very sophisticated, very high dimensional spaces of tasks to solve.
No one’s really gotten to the point where they can integrate all of these together, and I think is that going to be something that is now very generic, that we call deep learning, which is really a host of lots of different techniques that just use high dimensional parameter spaces, or is it going to be something completely new? I wouldn’t be able to guess.
So, there are a few things you left of your list, though, so presumably you don’t think an AGI would need to be conscious. Consciousness isn’t a part of our general intelligence. 
Ah, well, you know, maybe that brings us back to where we started.
Right, right. Well how about creativity? That wasn’t in your list either. Is that just computational from those basic elements you were talking about? Seeing, recognizing, combining?
So, an important part of that is being able to work with language, I’d say, being able to do natural language understanding and do natural language understanding at higher than the word level, but at the sentence level, certainly anything that might be what they call mistaken or “identified as” thinking. Have to have that as a necessary component. And being able to interact, being able to hold conversations, to abstract and to draw conclusions and inferences that aren’t necessarily there.
I’d say that that’s probably the sort of thing that you would expect of a conscious intelligence, whether it’s manifest in a person or manifest in a machine. Maybe I should say manifested in a human, or manifested in a machine.
So, you mentioned the Turing test earlier. And, you know, there are a lot of people who build chatbots and things that, you know, are not there yet, but people are working on it. And I always type in one, first question, it’s always the same, and I’ve never seen a system that even gets the question, let alone can answer it.
The question is, “What’s bigger, a nickel or the sun?” So, two questions, one, why is that so hard for a computer, and, two, how will we solve that problem?
Hmm. I can imagine how would I build a chatbot, and I have worked on this sort of project in the past. One of the things—and I mentioned earlier, this allusion to a miracle year—is the advances that happened, in particular, in 2013 with figuring out ways of doing neural-word embeddings. That’s so important, and one way of looking at why that’s so important is that, when we’re doing machine learning in general—this is what I tell my students, this what drives a lot of our design—you have to manage the shape of your data. You have to make sure that the amount of examples you have, the density of data points you have, is commensurate with the amount of degrees of freedom that you have representing your world, your model.
Until very recently, there have been attempts, but none of them as successful as we’ve seen in the last five years. The baseline has been what’s called the one-hot vector encoding, where you have a different dimension for every word in your language, usually it’s around a million words. You have all zeros and then for the word maybe in the first dimension you take the first word in the dictionary to order them that way, and you have the word ‘a,’ which is spelled with the letter ‘a,’ and that’s then the one and all zeros. And then for the second word you have a zero and a one and the rest zeros. So the point here, and not to get technical, but your dimensions are just too many.
You have millions and millions of dimensions. When we talk with students about this, it’s called the curse of dimensionality, every time you add even one dimension, you need twice as many data points in order to maintain the same density. And maintaining that density is what you need in order to abstract, in order to generalize, in order to come up with an algorithm that can actually find a pattern that works, not just for the data that it sees, but for the data that it will see.
What happens with these neural word embeddings? Well, they solve the problem of the curse of dimensionality, or at least they’ve really gotten their arms a lot further around it than ever before. They’ve enabled us to represent terms, represent concepts, not in these million dimensional vector spaces, where all that rich information is still there, but it’s spread so thinly across so many dimensions that you can’t really find a single entity as easily as you can if it were only representing a smaller number of dimensions, and that’s what these embeddings do.
Now, once you have that dimensionality, once you’re able to compress them into a lower dimension, now you can do all sorts of things that you want to do with language that you just couldn’t do before. And that’s part of why we see this slow operation with chatbots, they probably have something like this technology. What does this have to do with your question? These embeddings, for the most part, happen not by getting instructions—well nickels are this size, and they’re round, and they’re made of this sort of composite, and they have a picture of Jefferson stamped on the top—that’s not how you learn to mathematically represent these words at all.
What you do is you feed the algorithm lots and lots of examples of usage—you let it read all of Wikipedia, you let it read all of Reuters—and slowly but surely what happens is, the algorithm will start to see these patterns of co-usage, and will start to learn how one word follows after another. And what’s really remarkable, and could be profound, at least I know that a lot of people would want to infer that, is that the semantic kind of comes out for free.
You end up seeing the geometry of the way these words are embedded in such a way that you see, a famous example is a king vector minus a man vector plus a woman vector equals a queen vector, and that actually bears out in how the machine can now represent the language, and it did that without knowing anything about men, women, kings, or queens. It did it just by looking at frequencies of occurrence, how those words occur next to each other. So, when you talk about nickels and the sun, my first thought, given that running start, is that well, the machine probably hasn’t seen a nickel and a sun in context too frequently, and one of the dirty secrets about these neural embeddings is that they don’t do as well on very low-frequency terms, and they don’t always do well in being able to embed low frequency co-occurrences.
And maybe it’s just the fact that it hasn’t really learnt about, so to speak, it hasn’t read about, nickels and suns in context together.
So, is it an added wrinkle that, for example, you take a word like set, s-e-t, I think OED has two or three hundred definitions of it, you know—it’s something you do, it’s an object, etcetera. You know there’s a Wikipedia entry on a sentence, an eight word long grammatically correct sentence which is, “Buffalo buffalo buffalo buffalo buffalo buffalo buffalo buffalo,” which contains nouns, verbs, all of that. Is there any hope that if you took all the monkeys in all the universe typing cogent and coherent sentences, would it ever be enough to train it to what a human can do?
There’s a couple things there, and one of the key points that you’re making is that there are homonyms in our language, and so work should be done on disambiguating the homonyms. And it’s a serious problem for any natural language understanding project. And, you know, there are some examples out there of that. There’s one recently which is aimed at not just identifying a word but also disambiguating the usages or the context.
There are also others, not just focused on how to mathematically-represent how to pinpoint a representation of a word, but also how to represent the breadth of the usage. So maybe imagine not a vector, but a distribution or a cloud, that’s maybe a little thicker as a focal point, and all of those I think are a step in the right direction for capturing what is probably more representative of how we use language. And disambiguation, in particular with homonyms, is a part of that.
I only have a couple more questions in this highly theoretical realm, then I want to get down to the nitty gritty. I’m not going to ask you to pick dates or anything, but the nickel and the sun example, if you were just going to throw a number out, how many years is it until I type that question in something, and it answers it? Is that like, oh yeah we could do it if we wanted to, it’s just not a big deal, maybe give it a year? Or, is it like, “Oh, no that’s kind of tricky, wait five years probably.”
I think I remember hearing once never make a prediction.
Right, right. Well, just, is that a hard problem to solve?
The nickel and the sun is something that I’d hesitate to say is solvable in my lifetime, just to give a benchmark there, violating that maxim. I can’t say exactly when, what I can say is that the speed with which we are solving problems that I thought would take a lot longer to solve, is accelerating.
To me, while it’s a difficult problem and there are several challenges, we are still just scratching the surface in natural language understanding and word representation in particular, you know words-in-context representation. I am optimistic.
So, final question in this realm, I’m going to ask you my hard Turing test question, I wouldn’t even give this to a bot. And this one doesn’t play with language at all.
Dr. Smith is eating lunch at his favorite restaurant. He receives a call, takes it and runs out without paying his tab. Is management likely to prosecute? So you have to be able to infer it’s his favorite restaurant, they probably know who he is, he’s a doctor, that call was probably an emergency call. No, they’re not going to prosecute because that’s, you know, an understandable thing. Like, that doesn’t have any words that are ambiguous, and yet it’s an incredibly hard problem, isn’t it?
It is, and in fact, I think that is the, that is one of the true benchmarks—even moreso than comparing a nickel and a sun—of real, genuine natural language understanding. It has all sorts of things—it has object permanence, it has tracking those objects throughout different sentences, it has orienting sequences of events, it has management, which is mentioned in that last sentence, which is how you would be able to infer that management is somehow connected to the management of the restaurant.
That is a super hard one to solve for any Turing machine. It’s also something we’re starting to make progress on. Using LSDMs that do several passes through a sequence of sentences, classic artificial sentence dataset, that natural language understanding finds—the Facebook of AGI dataset, which actually is out there to help use as a benchmark for training these sorts of object permanence in multi-sentence thread. And we’ve made modest gains in that. There are algorithms like the Ask Me Anything algorithm, that have shown that it’s at least possible to start tracking objects over time, and with several passes come up with the right answer to questions about objects in sentences across several different statements.
Pulling back to the here and now, and what’s possible and what’s not. Did you ever expect AI to become part of the daily conversation, just to be part of popular culture the way it is now?
About as much as I expect that in a couple years that AI is going to be a term much like Big Data, which is to say overused.
Right.
I think, with respect to an earlier comments, the sort of AI that you and I have been dancing around, which is fully-integrated AI, is not what we talk about when we talk about what’s in daily conversation now, or for the most part not what we’re talking about in this context. And so it might be a little bit of a false success, or a spurious usage of “AI” in as much frequency as we see it.
That doesn’t mean that we haven’t made remarkable advances. It doesn’t mean that the examples that I’ve mentioned, in particular, in deep learning aren’t important, and aren’t very plausibly an early set of steps on the path. I do think that it’s a little bit of hype, though.
If you were a business person and you’re hearing all of this talk, and you want to do something that’s real, and that’s actionable, and you walk around your business, department to department—you go to HR, and to Marketing and you got to Sales, and Development—how do you spot something that would be a good candidate for the tools we have today, something that is real and actionable and not hype?
Ah, well, I feel like that is the job I do all the time. We’re constantly meeting with new companies, Fortune 500 CEOs and C-Suite execs, and talking about the problems that they want to solve, and thinking about ways of solving them. Like, I think a best practice is to always to keep it simple. There are a host of free deep learning techniques for doing all sorts of things—classification, clustering, user item matching—that are still tried-and-true, and that should probably be done first.
And then there are now, a lot of great paths to using these more sophisticated algorithms that mean that you should be considering them early. How exactly to consider one case from the other, I think that part of that is practice. It’s actually one of the things that when I talk to students about what they’re learning, I find that they’re walking away with not just, “I know what the algorithm is, I know what the objective function is, and how to manage momentum in the right way and optimizing that function,” but also how do you see the similarity between matching users and items in the recommender, or abstracting the latent semantic association of a bit of text or with an image, and there are similarities, and certain algorithms that solve all those problems. And that’s, in a lot of ways, practice.
You know, when the consumer web first came out and it became popularized, people had, you know, a web department, which would be a crazy thought today, right? Everything I’ve read about you, everybody says that you’re practical. So, from a practical standpoint, do you think that companies ought to have an AI taskforce? And have somebody whose job it is to do that? Or, is it more the kind of thing that it’s going to gradually come department by department by department? Or, is it prudent to put all of your thinking in one war room, as it were?
So, yeah, the general question is what’s the best way to do organizational design with machine learning machines, and the first answer is there are several right ways and there are a couple wrong ways. So, one of these wrong ways of the early-days are where you have this data science team that is completely isolated and is only responsible for R&D work, prototyping certain use cases and then they, to use a phrase you hear often, throw it over the wall to engineering to go implement, because I’m done with this project. That’s a wrong way.
There are several right ways, and those right ways usually involve bringing the people who are working on machine learning closer to production, closer to engineering, and also bringing the people involved in engineering and production closer to the machine learning. So, overall blurring those lines. You can do this with vertical integrated small teams, you could do this with peer teams, you can do this with a mandate that some larger companies, like Google, are really focused on making all their engineers machine learning engineers. I think all those strategies can work.
It all sort of depends on the size and the context of your business, and what kind of issues you have. And depending on those variables, then, among the several solutions, there might be one or two that are most optimal.
You’re the Chief Data Science Officer at Takt, spelled T-A-K-T, and is takt.com if anybody wants to go there. What does Takt do?
So we do the backend machine learning for large-scale enterprises. So, you know, many of your listeners might go to Starbucks and use the app to pay for Starbucks coffee. We do all of the machine learning personalization for the offers, for the games, for the recommendors in that app. And the way we approach that is by creating a whole host of different algorithms for different use cases—this goes back to your earlier question of abstracting those same techniques for many different use cases—and then apply that for each individual customer. We find the list completion use case, the recursive neural network approach, where there’s a time series of opportunity, where you can have interactions with an end user, and then learn from that interaction, and follow up with another interaction, doing things like reinforcement learning to do several interactions in a row, which may or may not get a signal back, but we have been trained to work towards that goal over time without that direct feedback signal.
This is the same sort of algorithms, for instance, that were used to train AlphaGo, to win a game. You only get that feedback at the end of the game, when you’ve won or lost. We take all of those different techniques and embed them in different ways for these large enterprise customers.
Are you a product company, a service company, a SaaS company—how does all that manifest?
We are a product company. We do tend to focus on the larger enterprises, which means that there is a little bit of customization involved, but there’s always going to be some customization involved when it comes to machine learning. Unless it’s just a suite of tools, which we are not. And what that means is that you do have to train and apply and suggest the right kinds of use cases for the suite of tools that we have, machine learning tools that we have.
Two more questions, if I may. You mentioned Cylons earlier, a Battlestar Galactica reference to those who don’t necessarily watch it. What science fiction do you think gets the future right? Like, when you watch it or read it, or what have you, you think “Oh yeah, things could happen that way, I see that”?
[Laughs] Well, you know the physicist in me still is both hopeful and skeptical about faster-than-light travel, so I suppose that wouldn’t really be the point of your question, is more with computers and with artificial intelligence.
Right, like Her or Ex Machina or what have you.
You know, it’s tough to say which of these, like, conscious-being robots is the most accurate. I think there are scenes worth observing that already have happened. Star Trek, you know, we create the iPad way before they had them in Star Trek time, so, good for reality. We also have all sorts of devices. I remember, when, in the ’80s—to date myself—the movie Star Trek came out, and Scotty gets up in front of his computer, an ’80s computer, and picks up the mouse and starts speaking into it and saying, “Computer, please do this.”
And my son will not get that joke, because he can say “Hey, Siri” or “Okay, Google” or “Alexa” or whatever the device is, and the computer will respond. And that’s, I like to focus on those smaller wins, that we are dramatically much quicker than forecasts in some cases able to accomplish that. I did see an example the other day about HAL, the Space Odyssey artificial intelligence, where people were mystified that this computer program could beat a human in chess, but didn’t blink an eye that the computer program could not only hold a conversation, but has a very sardonic disposition towards the main character. That, probably, very well captures this dichotomy of the several things are very likely to be captured, and we can get to very quickly, and other things that we thought were easy but take quite a lot longer than expected.
Final question, overall, are you an optimist? People worry about this technology—not just the killer robots scenario, but they worry about jobs and whatnot—but what do you think? Broadly speaking, as this technology unfolds, do you see us going down a dystopian path, or are you optimistic about the future?
I’ve spoken about this before a little bit. I don’t want to say, “I hope,” but I hope that Skynet will not launch a bunch of nuclear missiles. I can’t really speak with confidence to whether that’s a true risk or just an exciting storyline. What I can say is that the displacement of service jobs by automated machines is a very clear and imminent reality.
And that’s something that I’d like to think that politicians and governments and everybody should be thinking about—in particular how we think about education. The most important skill we can give our children is teaching them how to code, how to understand how computer programs work, and that’s something that we really just are not doing enough of yet.
And so will Skynet nuke everybody? I don’t know. Is it the case that I am, at six years old, teaching my son how to code already? Absolutely. And I think that will be make a big difference in the future.
But wouldn’t coding be something relatively easy for an AI? I mean it’s just natural language, tell it what you want it to do.
Computers that program themselves. It’s a good question.
So you’re not going to suggest, I think you mentioned, your son be a philosophy major at Columbia?
[Laughs] You know what, as long as he knows some math and he knows how to code, he can do whatever he wants.
Alright, well we’ll leave it on that note, this was absolutely fascinating, Mike. I want to thank you, thank you so much for taking the time. 
Well thank you, this was fun.
Byron explores issues around artificial intelligence and conscious computers in his upcoming book The Fourth Age, to be published in April by Atria, an imprint of Simon & Schuster. Pre-order a copy here.
[voices_in_ai_link_back]

Voices in AI – Episode 36: A Conversation with Bill Mark

[voices_in_ai_byline]
In this episode Byron and Bill talk about SRI International, aging, human productivity and more.
[podcast_player name=”Episode 36: A Conversation with Bill Mark” artist=”Byron Reese” album=”Voices in AI” url=”https://voicesinai.s3.amazonaws.com/2018-03-22-(00-59-22)-bill-mark.mp3″ cover_art_url=”https://voicesinai.com/wp-content/uploads/2018/03/voices-headshot-card-2.jpg”]
[voices_in_ai_byline]
Byron Reese: This is Voices in AI, brought to you by GigaOm. I’m Byron Reese. Today our guest is Bill Mark. He heads up SRI International’s Information and Computing Sciences Division which consists of two hundred and fifty researchers, in four laboratories, who create new technology and virtual personal assistants, information security, machine learning, speech, natural language, computer visionall the things we talk about on the show. He holds a Ph.D. in computer science from MIT. Welcome to the show, Bill.
Bill Mark:  Good to be here.
So, let’s start off with a little semantics. Why is artificial intelligence, artificial? Is artificial because it’s not really intelligence, or what? 
No, it’s artificial, because it’s created by human beings as opposed to nature. So, in that sense, it’s an artifact, just like any other kind of physical artifact. In this case, it’s usually a software artifact.
But, at its core, it truly is intelligent and its intelligence doesn’t differ in substance, only in degree, from human intelligence?
I don’t think I’d make that statement. The definition of artificial intelligence to me is always a bit of a challenge. The artificial part, I think, is easy, we just covered that. The intelligence part, I’ve looked at different definitions of artificial intelligence, and most of them use the word “intelligence” in the definition. That doesn’t seem to get us much further. I could say something like, “it’s artifacts that can acquire and/or apply knowledge,” but then we’re going to have a conversation about what knowledge is. So, what I get out of it is it’s not very satisfying to talk about intelligence at this level of generality because, yes, in answer to your question, artificial intelligence systems do things which human beings do, in different ways and, as you indicated, not with the same fullness or level that human beings do. That doesn’t mean that they’re not intelligent, they have certain capabilities that we regard as intelligent.
You know it’s really interesting because at its core you’re right, there’s no consensus definition on intelligence. There’s no consensus definition on life or death. And I think that’s really interesting that these big ideas aren’t all that simple. I’ll just ask you one more question along these lines then. Alan Turing posed the question in 1950, Can a machine think? What would you say to that?
I would say yes, but now we have to wonder what “think” might mean, because “think” is one aspect of intelligent behavior, it indicates some kind of reasoning or reflection. I think that there are software systems that do reason and reflect, so I will say yes, they think.
All right, so now let’s get to SRI International. For the listeners who may not be familiar with the company can you give us the whole background and some of the things you’ve done to date, and why you exist, and when it started and all of that?
Great, just a few words about SRI International. SRI International is a non-profit research and development company, and that that’s a pretty rare category. A lot of companies do research and development—a fewer than used to, but still quite a few—and very few have research and development as their business, but that is our business. We’re also non-profit, which really means that we don’t have shareholders. We still have to make money, but all the money we make has to go into the mission of the organization which is to do R&D for the benefit of mankind. That’s the general thing. It started out as part of Stanford, it was formerly the Stanford Research Institute. It’s been independent since 1970 and it’s one of the largest of these R&D companies in the world, about two thousand people.
Now, the information and computing sciences part, as you said, that’s about two hundred and fifty people, and probably the thing that we’re most famous for nowadays is that we created Siri. Siri was a spinoff of one of my labs, the AI Center. It was a spinoff company of SRI, that’s one of the things we do, and it was acquired by Apple, and has now become world famous. But we’ve been in the field of artificial intelligence for decades. Another famous SRI accomplishment would be Shakey the Robot, which was really the first robot that could move around and reason and interact. That was many years ago. We’ve also, in more recent history, been involved in very large government-sponsored AI projects which we’ve led, and we just have lots of things that we’ve done in AI.
Is it just a coincidence that Siri and SRI are just one letter different, or is that deliberate?
It’s a coincidence. When SRI starts companies we bring in entrepreneurs from the outside almost always, because it would be pretty unusual for an SRI employee to be the right person to be the CEO of the startup company. It does happen, but it’s unusual. Anyway, in this case, we brought in a guy named Dag Kittlaus, and he’s of Norwegian extraction, and he chose the name. Siri is a Norwegian women’s name and that became the name of the company. Actually, somewhat to our surprise, Apple retained that name when they launched Siri.
Let’s go through some of the things that your group works on. Could we start with those sorts of technologies? Are there other things in that family of conversational AI that you work on and are you working on the next generation of that?
Yes, indeed, in fact, we’ve been working on the next generation for a while now. I like to think about conversational systems in different categories. Human beings have conversations for all kinds of reasons. We have social conversations, where there’s not particularly any objective but being friendly and socializing. We have task-oriented kinds of conversations—those are the ones that we are focusing on mostly in the next generation—where you’re conversing with someone in order to perform a task or solve some problem, and what’s really going on is it’s a collaboration. You and the other person, or people, are working together to solve a problem.
I’ll use an example from the world of online banking because we have another spinoff called Kasisto that is using the next-generation kind of conversational interaction technology. So, let’s say that you walk into a bank, and you say to the person behind the counter, “I want to deposit $1,000 in checking.” And the person on the other side, the teller says, “From which account?” And you say, “How much do I have in savings?” And the teller says, “You have $1,500, but if you take $1,000 out you’ll stop earning interest.” So, take that little interaction. That’s a conversational interaction. People do this all the time, but it’s actually very sophisticated and requires knowledge.
If you now think of, not a teller, but a software system, a software agent that you’re conversing with—we’ll go through the same little interaction. The person says, “I want to deposit $1,000 in checking.” And the teller said, “From which account?” The software system has to know something about banking. It has to know that a deposit is a money transfer kind of interaction and it requires a from-account and a to-account. And in this case, the to-account has been specified but the from-account hasn’t been specified. In many cases that person would simply ask for that missing information, so that’s the first part of the interaction. So, again, the teller says, “From which account?” And the person says, “How much do I have in savings?” Well, that’s not an answer to the question. In fact, it’s another question being introduced by the person and it’s actually a balance inquiry question. They want to know how much they have in savings. Now, when I go through this the first time, the reason I do this twice is that when I went through it the first time, almost nobody even notices that that wasn’t an answer to the question, but if you try out a lot of the personal assistant systems that are out there, they tend to crater on that kind of interaction, because they don’t have enough conversational knowledge to be able to handle that kind of thing. And then the interaction goes on where the teller is providing information, beyond what the person asked, about potentially losing interest, or it might be that they would get a fee or something like that.
That illustrates the point that we expect our conversational partners to be proactive, not just to simply answer our questions, but to actually help us solve the problem. That’s the kind of interaction that we’re building systems to support. It’s very different than the personal assistants that are out there like Siri, and Cortana, and Google which are meant to be very general. Siri doesn’t really know anything about banking, which isn’t a criticism it’s not supposed to know anything about banking, but if you want to get your banking done over your mobile phone then you’re going to need a system that knows about banking. That’s one example of sort of next-generation conversational interaction.
How much are we going to be able to use transfer learning to generalize from that? You built that bot, that highly verticalized bot that knows everything about banking, does anything it learned make it easier now for it to do real estate, and then for it to do retail, and then all the other things? Or is it the case that like every single vertical, all ten thousand of them are going to need to start over from scratch?
It’s a really good question, and I would say, with some confidence, that it’s not about starting over from scratch because some amount of the knowledge will transfer to different domains. Real estate has transactions, if there’s knowledge about transactions some of that knowledge will carry over, some of it won’t.
You said, “the knowledge that it has learned,” and we need to get pretty specific about that. We do build systems that learn, but not all of their knowledge is picked up by learning. Some of it is built in, to begin with. So, there’s the knowledge that has been explicitly represented, some of which will transfer over. And then there’s knowledge that has been learned in other ways, some of that will transfer over as well, but it’s less clear-cut how that will work. But it’s not starting from scratch every time.
So, eventually though you get to something that could pass the Turing test. You could ask it, “So, if I went into the bank and wanted to move $1,000, what would be the first question you would ask me?” And it would say, “Oh, from what account?” 
My experience with every kind of candidate Turing test system, and nobody purports that we’re there by a long shot, but my first question is always, “What’s bigger, a nickel or the sun?” And I haven’t found a single one that can answer the question. How far away is that?
Well, first just for clarity, we are not building these systems in order to pass the Turing test, and in fact, something that you’ll find in most of these systems is that outside of their domain of expertise, say banking, in this case, they don’t know very much of anything. So, again, the systems that we build wouldn’t know things like what’s bigger, the nickel or the sun.
The whole idea of the Turing test is that it’s meant to be some form of evaluation, or contest for seeing whether you have created something that’s truly intelligent. Because, again, this was one of Turing’s approaches to answering this question of what is intelligence. He didn’t really answer that question but he said if you could develop an artifact that could pass this kind of test, then you would have to say that it was intelligent, or had human-like behavior at the very least. So, in answer to your question, I think we’re very far from that because we aren’t so good at getting the knowledge that, I would say, most people have into a computer system yet.
Let’s talk about that for a minute. Why is it so hard and why is it so, I’ll go out on a limb and say, easy for people? Like, a toddler can tell me what’s bigger the nickel or the sun, so why is it so hard? And what makes humans so able to do it?
Well, I don’t know that anyone knows the answer to that question. I certainly don’t. I will say that human beings spend time experiencing the world, and are also taught. Human beings are not born knowing that the sun is bigger than a nickel, however, over time they experience what the sun is and, at some point, they will experience what a nickel is, and they’ll be able to make that comparison. By the way, they also have to learn how to make comparisons. It would be interesting to ask toddlers that question, because the sun doesn’t look very big when you look up in the sky, so that brings in a whole other class of human knowledge which I’ll just broad-brush call book learning. I certainly would not know that the sun is really huge, unless I had learned that in school. Human beings have different ways of learning, only a very small sample of which have been implemented in artificial intelligence learning systems.
There’s Calvin and Hobbes, where his dad tells Calvin that it’s a myth that the sun is big, that it’s really only the size of a quarter. And he said, “Look, hold it up in the sky. They’re the same.” So, point taken. 
But, let me ask it this way, human DNA is, I don’t know, I’m going to get this a little off, but it’s like 670MB of data. And if you look at how much that’s different than, say, a banana, it’s a small amount that is different. And then you say, well, how much of it is different than, say, a chimp, and it’s a minuscule amount. So, whatever that minuscule difference in code is, just a few MBs, is that, kind of, the secret to intelligence? Is that a proof point that there may be some very basic, simple ways to acquire generalized knowledge, that we just haven’t stumbled across yet that, but there may be something that gives us this generalized learner, we can just plug into the Internet and the next day it knows everything. 
I don’t make that jump. I think the fact that a relatively small amount of genetic material differentiates us from other species doesn’t indicate that there’s something simple out there, because the way those genes or the genetic material impacts the world is very complex, and lead to all kinds of things that could be very hard for us to understand and try to emulate. I also don’t know that there is a generalist learner anyway. I think, as I said, human beings seem to have different ways of learning things, and that doesn’t say to me that there is one general approach.
Back in the Dartmouth days, when they thought they could knock out a lot of AI problems in a summer, it was in the hope that intelligence followed a few simple laws, like how the laws of physics explain so much. It’s been kind of the consensus move to think that we’re kind of a hack of a thousand specialized things that we do that all come together and make generalized intelligence. And it sounds like you’re more in that camp that it’s just a bunch of hard work and we have to tackle these domains one at a time. Is that fair?
I’m actually kind of in between. I think that there are general methods, there are general representations, but there’s also a lot of specific knowledge that’s required to be competent in some activity. I’m into sort of a hybrid.
But you do think that building an AGI, generalized intelligence, that is as versatile as a human is theoretically possible I assume? 
Yes.
You mentioned something when we were chatting earlier that a child explores the world. Do you think embodiment is a pathway to that, that until we give machines away, in essence, to “experience” the world, that that will always limit what we’re able to do? Is that embodiment, that you identified as being important for humans, also important for computers?
Well, I would just differentiate the idea of exploration from embodiment. I think that exploration is a fundamental part of learning. I would say that we, yes indeed, will be missing something unless we design systems that can explore their world. From my point of view, they may or may not be embodied in the usual sense of that word, which means that they can move around and actuate within their environment. If you generalize that to software and say, “Are software agents embodied because they can do things in the world?” then, yeah, I guess I would say embodiment, but it doesn’t have to be physical embodiment.
Earlier when you were talking about digital assistants you said Siri, Cortana and then you said, “Oh, and Google.” And that highlights a really interesting thing that Amazon named theirs, you named yours, Microsoft named theirs, but Google’s is just the Google Assistant. And you’re undoubtedly familiar with the worries that Weizenbaum had with ELIZA. He thought that this was potentially problematic that we name these devices, and we identify with them as if they are human. He said, “When a computer says, ‘I understand,’ it’s just a lie. There’s no ‘I,’ and there’s nothing that understands anything.” How would you respond to Weizenbaum? Do you think that’s an area of concern or you think he was just off?
I think it’s definitely an area of concern, and it’s really important in designing. I’ll go back to conversational systems, systems like that, which human beings interact with, it’s important that you do as much as possible to help the human being create a correct mental model of what it is that they’re conversing with. So, should it be named? I think it’s kind of convenient to name it, as you were just saying, it kind of makes it easier to talk about, but it immediately raises this danger of people over-reading into it: what it is, what it knows, etcetera. I think it’s very much something to be concerned about.
There’s that case in Japan, where there’s a robot that they were teaching how to navigate a mall, and very quickly learned that it got bullied by children who would hit it, curse at it, and all these things. And later when they asked the children did you think it was upset, was it acting upset? Was it acting human-like or mechanical? They overwhelmingly said it was human-like. 
And I still have a bit of an aversion to interrupting the Amazon deviceI can’t say its name because it’s on my desk right next to meand telling it, “Stop!” And so I just wonder where it goes because, you’re right, it’s like the Tom Hanks’ movie Castaway when his only friend was a soccer ball named “Wilson” that he personified. 
I remember there was a case in the ‘40s where they would show students a film of circles and lines moving around, and ask them to construct stories, and they would attribute to these lines and circles personalities, and interactions, and all of that. It is such a tempting thing we do, and you can see it in people’s relationships to their pets that one wonders how that’s all going to sort itself out, or will we look back in forty years and think, “Well, that was just crazy.”
No, I think you’re absolutely right. I think that human beings are extremely good at giving characteristics to objects, systems, etcetera, and I think that will continue. And, as I said, that’s very much a danger in artificial intelligence systems, the danger being that people assume too much knowledge, capability, understanding, given what the system actually is. Part of the job of designing the system is, as I said before, to go as far as we can to give the person the right idea about what it is that they’re dealing with.
Another area that you seem to be focused on, as I was reading about you and your work, is AI and the aging population. Can you talk about what the goal is there and what you are doing, and maybe some successes or failures you’ve had along the way?
Yes, indeed, we are, SRI-wide actually, looking at what we can do to address the problem, the worldwide problem, of higher percentage of aging population, lower percentage of caregivers. We read about this in the headlines all the time. In particular, what we can do to have people experience an optimal life, the best that is possible for them as they age. And there’s lots of things that we’re looking at there. We were just talking about conversational systems. We are looking at the problem of conversational systems that are aimed at the aging population, because interaction tends to be a good thing and sometimes there aren’t caregivers around, or there aren’t enough of them, or they don’t pay attention, so it might actually be interesting to have a conversational system that elderly people can talk to and interact with. We’re also looking at ways to preserve privacy and unobtrusively monitor the health of people, using artificial intelligence techniques. This is indeed a big area for us.
Also, your laboratories work on information security and you mentioned privacy earlier, talk to me, if you would, about the state of the art there. Across all of human history, there’s been this constant battle between the cryptographers and the people who break the codes, and it’s unclear who has the upper hand in that. It’s the same thing with information security. Where are we in that world? And is it easier to use AI to defend against breaches, or to use that technology to do the breach?
Well, I think, the situation is very much as you describe—it’s a constant battle between attackers and defenders. I don’t think it’s any easier to use AI to attack, or defend. It can be used for both. I’m sure it is being used for both. It’s just one of the many sets of techniques that can be used in cybersecurity.
There’s a lot of concern wrapped up in artificial intelligence and its ability to automate a lot of work, and then the effect of that automation on employment. What’s your perspective on how that is going to unfold?
Well, my first perspective is it’s a very complex issue. I think it’s very hard to predict the effect of any technology on jobs in the long-term. As I reflect, I live in the Bay Area, a huge percentage of the jobs that people have in the Bay Area didn’t exist at all a hundred years ago, and I would say a pretty good percentage didn’t exist twenty years ago. I’m certainly not capable of projecting in the long run what the effect of AI and automation will be. You can certainly guess that it will be disruptive, all new technologies are disruptive, and that’s something as a society we need to take aboard and deal with, but how it’s going to work out in the long-term, I really don’t know.
Do you take any comfort that we’ve had transformative technologies aplenty? Right, we had the assembly line which is a kind of artificial intelligence, we had the electrification of industry, we had the replacement of animal power with steam power. I mean each of those was incredibly disruptive. And when you look back across history each one of them happened incredibly fast and yet unemployment never surged from them. Unemployment in the US has always been between four and ten percent, other than the depression. And you can’t the point and say, “Oh, when this technology came out unemployment went briefly to fourteen percent,” or anything like that. Do you take comfort in that or do you say, “Well, this technology is materially different”? 
I take comfort in it in the sense that I have a lot of faith in the creativity and agility of people. I think what that historical data is reflecting is the ability of individuals and communities to adapt to change and I expect that to continue. Now, artificial intelligence technology is different, but I think that we will learn to adapt and thrive with artificial intelligence in the world.
How is it different though, really? Because technology increases human productivity, that’s kind of what it does. That’s what steam did. That’s what electricity did. That’s what the Industrial Revolution did. And that’s what artificial intelligence does. How is it different?
I think in the sense that you’re talking about, it’s not different. It is meant to augment human capability. It’s augmenting now, to some extent, different kinds of human activity, although arguably that’s been going on for a long time, too. Calculators, printing presses, things like that have taken over human activities that were once thought to be core human things. It’s sort of a difference in degree, not a difference in kind.
One interesting thing about technology and how the wealth that it produces is disseminated through culture, is that in one sense technology helps everybodyyou get a better TV, or better brakes in your car, better deodorant, or whateverbut in two other ways, it doesn’t. If you’re somebody who sells your labor by the hour, and your company can produce a labor-saving device, that benefit doesn’t accrue to you it generally would accrue to the shareholders of the company in terms of higher earnings. But if you’re self-employed, or you buy your own time as it were, you get to pocket all of the advances that technology gets you, because it makes your productivity higher and you get all of that. So, do you think that the technology does inherently make worse the income-inequality situation, or am I missing something in that analysis? 
Well, I don’t think that is inherent and I’m not sure that the fault lines will cut that way. We were just talking about the fact that there is disruption and what that tends to mean is that some people will benefit in the short-term, and some of the people will suffer in the short-term. I started by saying this is a complex issue. I think one of the complexities is actually determining what that is. For example, let’s take stuff around us now like Uber and other ride-hailing services. Clearly that has disrupted the world of taxi drivers, but on the other hand has created opportunities for many, many, many other drivers, including taxi drivers. What’s the ultimate cost-benefit there? I don’t know. Who wins and loses? Is it the cab companies, is it the cab drivers? I think it’s hard to say.
I think it was Niels Bohr that said, “Making predictions is hard, especially if they’re about the future.” And he was a Nobel Laureate.
Exactly.
The military, of course, is a multitrillion-dollar industry and it’s always an adopter of technology, and there seems to be a debate about making weapon systems that make autonomous kill decisions. How do you think that’s going to unfold?
Well, again, I think that this is a very difficult problem and is a touchpoint issue. It’s one manifestation of an overall problem of how we trust complex systems of any kind. This is, to me anyway, this goes way beyond artificial intelligence. Any kind of complex system, we don’t really know how it works, what its limitations are, etcetera. How do we put boundaries on its behavior and how do we develop trust in what it’s done? I think that’s one of the critical research problems of the next few decades.
You are somebody who believes we’re going to build a general intelligence, and it seems that when you read the popular media there’s a certain number of people that are afraid of that technology. You know all the names: Elon Musk says it’s like summoning the demon, Professor Hawking says it could be the last thing we do, Bill Gates says he’s in the camp of people who are worried about it and don’t understand why other people aren’t was, Wozniak, the list goes on and on. Then you have another list of people who just almost roll their eyes at those sorts of things, like Andrew Ng who says it’s like worrying about overpopulation on Mars, the roboticist Rodney Brooks says that it’s not helpful, Zuckerberg and so forth. So, two questions: why, among a roomful of incredibly smart people is there such a disagreement over it, and, two, where do you fall in that kind of debate?
Well, I think the reason for disagreements, is that it’s a complex issue and it involves something that you were just talking about with the Niels Bohr quote. You’re making predictions about the future. You’re making predictions about the pace of change, and when certain things will occur, what will happen when they occur, really based on very little information. I’m not at all surprised that there’s dramatic difference of opinion.
But to be clear, it’s not a roomful of people saying, “These are really complex issues,” it’s a roomful of people were half of them are saying, “I know it is a problem,” and half of them saying, “I know it is not a problem.” 
I guess that might be a way of strongly stating a belief. They can’t possibly know.
Right, like everything you’re saying you’re taking measured tones like, “Well, we don’t know. It could happen this way or that way. It’s very complicated.” They are not taking that same tone. 
Well, let me get to your second question, we can come back to the first one. So, my personal view, and here comes this measured response that you just accused me of is, yes, I’m worried about it, but, honestly, I’m worried about other things more. I think that this is something to be concerned about. It’s not an irrational concern, but there are other concerns that I think are more pressing. For example, I’m much more worried about people using technology for untoward purposes than I am about superintelligence taking over the world.
That is an inherent problem with technology’s ability to multiply human effort, if human effort is malicious. Is that an insoluble problem? If you can make an AGI you can, almost by definition, make an evil AGI, correct?
Yes. Just to go back a little bit, you asked me whether I thought AGI was theoretically possible, whether there are any theoretical barriers. I don’t think there are theoretical barriers. We can extrapolate and say, yes, someday that kind of thing will be created. When it is, you’re right, I think any technology, any aspect of human behavior can be done for good or evil, from the point of view of some people.
I have to say, another thing I think about when we talk about super intelligence, I was relating it to complex systems in general. I think of big systems that exist today that we live with, like high-speed automated trading of securities, or weather forecasting, these are complex systems that definitely influence our behavior. I’m going to go out on a limb and say nobody knows what’s really going on with them. And we’ve learned to adapt to them.
It’s interesting, I think part of the difference of opinion boils down to a few technical questions that are very specific that we don’t know the answer to. One of them is, it seems like some people are kind of, I don’t want to say down on humans, but they don’t think human abilities, like creativity and all of that are all that difficult, and machines are going to be able to master that. There’s a group of people who would say the amount of time between one of these systems being able to self-improve is short, not long. I think that some would say intelligence isn’t really that hard, but there’s probably a few breakthroughs. You stack enough of those together and you say, “Okay, it’s really soon.” But if you take the opposite side on thosecreativity is very hard, intelligence is very hardthen you’re, kind of, in the other camp. I don’t doubt the sincerity of any of the parties involved. 
On your comment about the theoretical possibility of a general intelligence, just to explore that for a moment, without any regard for when it will happen—we understand how a computer could, for instance, measure temperature, but we don’t really understand how a computer, or I don’t, could feel pain. For a machine to go from measuring the world to experiencing the world, we don’t really know that, and so is that required to make a general intelligence, to be able to, in essence, experience qualia, to be conscious, or not. 
Well, I think that if we’re truly talking about general intelligence in the sense that I think most people mean it, which is human-like intelligence, then one thing that people do is experience the world and react to it, and it becomes part of the way that we think and reason about the world. So, yes, I think, if we want computers to have that kind of capability, then we have to figure out a way for them to experience it.
The question then becomes—I think this is in the realm of the very difficult—when, to use your example, a human being or any animal experiences pain, there is some physical and then electrochemical reaction going on that is somehow interpreted in the brain. I don’t know how all of that works, but I believe that it’s theoretically possible to figure out how that works and to create artifacts that exhibit that behavior.
Because we can’t really confine it to how humans feel pain, right? But, I guess I’m still struggling over that. What would that even look like, or is your point, “I don’t know what it looks like, but that would be what’s required to do it.” 
I definitely don’t know what it looks like on the inside, but you can also look at the question of, “What is the value of pain, or how does pain influence behavior?” For a lot of things, pain is a warning that we should avoid something, touching a hot object, moving an injured limb, etcetera. There’s a question of whether we can get computer systems to be able to have that kind of warning sensation which, again, isn’t exactly the same thing as creating a system that feels pain in any way like an animal does, but it could get the same value out of the experience.
Your lab does work in robotics as well as artificial intelligence, is that correct?
Right.
Talk a little bit about that work and how those two things come together, artificial intelligence and robots.
Well, I think that, traditionally, artificial intelligence and robotics have been the same area of exploration. One of the features of any maturing discipline, which I think AI is, is that various specializations and specialty groups start forming naturally as the field expands and there’s more and more to know.
The fact that you’re even asking the question shows that there has become a specialization in robotics that is seen as separate from, some people may say, part of, some people may say, completely different from, artificial intelligence. As a matter of fact, although my labs work on aspects of robotics, other labs within SRI, that are not part of the information computing sciences division, also work on robotics.
The thing about robotics is that you’re looking at things like motion, manipulation, actuation, doing things in the world, and that is a very interesting set of problems that has created a discipline around it. Then on top of that, or surrounding it, is the kind of AI reasoning, perception, etcetera, that enables those things to actually work. To me, they are different aspects of the same problem of having, to go back to something you said before, some embodiment of intelligence that can interact with the real world.
The roboticist Rodney Brooks, who I mentioned earlier, says something to the effect that he thinks there’s something about biology, something very profoundly basic that we don’t understand which he calls, “the juice.” And to be clear, he’s 100% convinced that “the juice” is biology, that there’s nothing mystical about it, that it’s just something we don’t understand. And he says it’s the difference between, you put a robot in a box and it tries to get out, it just kind of runs through a protocol and tries to climb. But you put an animal in a box and it frantically wants out of that boxit’s scratching, it’s getting agitated and worked upand that difference between those two systems he calls “the juice.” Do you think there is something like that that we don’t yet know about biology that would be beneficial to have to put in robots? 
I think that there’s a whole lot that we don’t know about biology, and I can assure you there’s a huge amount that I don’t know about biology. Calling it “the juice,” I don’t know what we learn from that. Certainly, the fact that animals have motivations and built-in desires that make them desperately want to get out of the box, is part of this whole issue of what we were talking about before of how and whether to introduce that into artifacts, into artificial systems. Is it a good thing to have in robots? I would say, yes. This gets back to the discussion about pain, because presumably the animal is acting that way out of a desire for self-preservation, that something that it has inherited or learned tells it that being trapped in a box is not good for its long-term survival prospects. Yes, it would be good for robots to be able to protect themselves.
I’ll ask you another either/or question you may not want to answer. The human body uses one hundred watts and we use twenty of that to power our brain, and we use eighty of it to power our body. The biggest supercomputers in the world use twenty million watts and they’re not able to do what the brain does. Which of those is a harder thing to replicate? If you had to build a computer that operated with the capabilities of the human brain that used twenty watts, or you had to build a robot that only used eighty watts that could mimic the mobility of a human. Which of those is a harder problem?
Well, as you suggested when you brought this up, I can’t take that either/or. I think that they’re both really hard. The way you phrased that makes me think of somebody who came to give a talk at SRI a number of years ago, and was somebody who was interested in robotics. He said that, as a student, he had learned about the famous AI programs that had become successful in playing chess. And as he learned more and more about it, he realized that what was really hard was a human being picking up the chess piece and moving it around, not the thinking that was involved in chess. I think he was absolutely right about that because chess is a game that is abstract and has certain rules, so even though it’s very complex, it’s not the same thing as the complexities of actual manipulation of objects. But if you ask the question you did, which is comparing it not to chess, but to the full range of human activity then I would just have to say they’re both hard.
There isn’t a kind of a Moore’s law of robotics is there—the physical motors and materials and power, and all of that? Is that improving at a rate commensurate with our advances in AI, or is that taking longer and slower? 
Well, I think that you have to look at that in more detail. There has been tremendous progress in the ability to build systems that can manipulate objects that use all kinds of interesting techniques. Cost is going down. The accuracy and flexibility is going up. In fact, that’s one of the specialty areas of the robotics part of SRI. That’s absolutely happening. There’s also been tremendous progress on aspects of artificial intelligence. But other parts of artificial intelligence are coming along much more slowly and other parts of robotics are coming along much more slowly.
You’re about the sixtieth guest on the show, and I think that all of them, certainly all of them that I have asked, consume science fiction, sometimes quite a bit of it. Are you a science fiction buff? 
I’m certainly not a science fiction buff. I have read science fiction. I think I used to read a lot more science fiction than I do now. I think science fiction is great. I think it can be very inspiring.
Is there any vision of the future in a movie, TV, or book, or anything that you look at and say, “Yes, that could happen, that’s how the world might unfold? You can say Her, or Westworld, or Ex Machina, or Star Trek, or any of those.
Nope. When I see things like that I think they’re very entertaining, they’re very creative, but they’re works of fiction that follow certain rules or best practices about how to write fiction. There’s always some conflict, there’s resolution, there’s things like that are completely different from what happens in the real world.
All right, well, it has been a fantastically interesting hour. I think we’ve covered a whole lot of ground and I want to thank you for being on the show, Bill. 
It’s been a real pleasure.
Byron explores issues around artificial intelligence and conscious computers in his upcoming book The Fourth Age, to be published in April by Atria, an imprint of Simon & Schuster. Pre-order a copy here.
[voices_in_ai_link_back]

Voices in AI – Episode 35: A Conversation with Lorien Pratt

[voices_in_ai_byline]
In this episode, Byron and Lorien talk about intelligence, AGI, jobs, and the human genome project.
[podcast_player name=”Episode 35: A Conversation with Lorien Pratt” artist=”Byron Reese” album=”Voices in AI” url=”https://voicesinai.s3.amazonaws.com/2018-03-20-(00-45-11)-lorien-pratt.mp3″ cover_art_url=”https://voicesinai.com/wp-content/uploads/2018/03/voices-headshot-card-4.jpg”]
[voices_in_ai_byline]
Byron Reese: This is Voices in AI, brought to you by Gigaom, I’m Byron Reese. Today our guest is Lorien Pratt, the Chief Scientist and Co-founder over at Quantellia. They’re a software consulting company in the AI field. She’s the author of The Decision Intelligence Primer.” She holds an AB in Computer Science from Dartmouth, and an MS and PhD in Computer Science from Rutgers. Welcome to the show, Lorien!
Lorien Pratt: Thank you Byron delighted to be here, very honored thank you.
So, Lorien, let’s start with my favorite question, which is, what is artificial intelligence?
Artificial intelligence has had an awful lot of definitions over the years. These days when most people say AI, ninety percent of the time they mean machine learning, and ninety percent of the time that machine learning is a neural network underneath.
You say that most people say that, but is that what you mean by it?
I try to follow how people tend to communicate and try to track this morphing definition. Certainly back in the day we all had the general AI dream and people were thinking about Hal and the robot apocalypse, but I tend to live in the applied world. I work with enterprises and small businesses and usually when they say AI it’s, How can I make better use of my data and drive some sort of business value?” and they’ve heard of this AI thing and they don’t quite know what it is underneath.
Well, let me ask a different question then, what is intelligence?
What is intelligence, that’s a really nebulous thing isn’t it?
Well it does not have a consensus definition, so, in one sense you cannot possibly answer it incorrectly.
Right, I guess my world, again, is just really practical, what I care about is what drives value for people. Around the world sometimes intelligence is defined very broadly as the thing that humans do, and sometimes people say a bird is much more intelligent than a human at flying and a fish is much more intelligent than a human at swimming. So, to me the best way to talk about intelligence is relative to some task that has some value, and I think it’s kind of dangerous waters when we try to get too far into defining such a nebulous and fluctuating thing.
Let me ask one more definition and then I will move on. In what sense do you interpret the word artificial”? Do you interpret it as, artificial intelligence isn’t real intelligence, it’s just faking it—like artificial turf isn’t real grass—or, No, it’s really intelligence, but we built it, and that’s why we call it artificial”?
I think I have to give you another frustrating answer to that, Byron. The human brain does a lot of things, it perceives sound, it interprets vision, it thinks through, Well if I go to this college, what will be the outcome?” Those are all, arguably, aspects of intelligence—we jump on a trampoline, we do an Olympic dive. There are so many behaviors that we can call intelligence, and the artificial systems are starting to be able to do some of those in useful ways. So that perception task, the ability to look at an image and say, that’s a cat, that’s a dog, that’s a tree etcetera,” yeah, I mean, that’s intelligence for that task, just like a human would be able to do that. Certain aspects of what we like to call intelligence in humans, computers can do, other aspects, absolutely not. So, we’ve got a long path to go, it’s not just a yes or a no, but it’s actually quite a complex space.
What is the state of the art? This has been something we’ve explored since 1955, so where are we in sixty-two year journey?
Sure, I think we had a lot of false starts, people kept trying to, sort of, jump start and kick start general intelligence—this idea that we can build Hal from 2001 and that he’d be like a human child or a human assistant. And unfortunately, between the fifth generation effort of the 1980’s and stuff that happened earlier, we’ve never really made a lot of progress. It’s been kind of like climbing a tree to get to the moon. Over the years there’s been this second thread, not the AGI artificial general intelligence, but a much more practical thread where people have been trying to figure out how do we build an algorithm that does certain tasks that we usually call intelligent.
The state of the art is that we’ve gotten really good at, what I call, one-step machine learning tasks—where you look at something and you classify it. So, here’s a piece of text, is it a happy tweet or a sad tweet? Here’s a job description, and information about somebody’s resume, do they match, do they not? Here’s an image, is there a car in this image or not? So these one-step links we’re getting very, very good at, thanks to the deep learning breakthroughs that Yann LeCun and Geoffrey Hinton and Yoshua and all of those guys have done over the last few years.
So, that’s the state of the art, and there’s really two answers to that, one is, what is the state of the art in terms of things that are bringing value to companies where they’re doing breakthrough things, and the other is the state of the art from a technology point of view, where’s the bleeding edge of the coolest new algorithms, independent of whether they’re actually being useful anywhere. So, we sort of have to ask that question in two different ways.
You know AI makes headlines anytime it beats a human at a new game, right? What do you think will be the next milestone that will make the popular media, AI did _______.”
AI made a better decision about how to address climate change and sea level rise in this city than the humans could have done alone, or AI helped people with precision medicine to figure out the right medicine for them based on their genetics and their history that wasn’t just one size fits all.
But I guess both of those are things that you could say are already being done. I mean, they’re already being done, there’s not a watershed moment, where Aha! Lee Sedol just got beaten by AlphaGo.” We already do some genetic customization, we can certainly test certain medications against certain genomic markers.
We can, but I think what hasn’t happened is the widespread democratization of AI. Bill Gates said, we’re going to have a computer on every desk.” I also think that Granny, who now uses a computer, will also be building little machine learners within a few years from now. And so when I talk about personalized medicine or I talk about a city doing climate change, those are all, kind of, that general umbrella—it’s not going to be just limited to the technologists. It’s a technology that’s going through this democratization cycle, where it becomes available and accessible in a much more widespread way to solve really difficult problems.
I guess that AIs are good at games because they’re a confined set of rules, and there’s an idea of a winner. Is that a useful way to walk around your enterprise and look for things you can apply AI to?
In part, I would say necessary, but not sufficient, right? So, a game, what is that? It’s a situation in which somebody’s taking an action and then based on that some competitor—maybe literally your competitor in a market—is taking some counter action, and then you take an action, and vice versa, right? So, thinking in terms of games, is actually a direction I see coming down the pike in the future, where these single-link AI systems are going to be integrated more and more with game theory. In fact, I’ve been talking to some large telecoms about this recently, where we are trying to, sort of, game out the future, right? Right now in AI, primarily, we’re looking at historical data from the past and trying to induce patterns that might be applicable to the future, but that’s a different view of the future than actually simulating something—I’ll take this action and you’ll take this other action. So, yes, the use of games has been very important in the history of AI, but again it’s not the whole picture. It does, as you say, tend to over-simplify things when we think in terms of games. When I map complex problems, it does kind of look like game moves that my customers take, but it is way more complex than a simple game of chess or checkers, or Go.
Do you find that the people who come to you say, I have this awesome data, what can AI teach me about it?” Or do they say, I have this problem, how do I solve it?” I mean, are they looking for a problem or looking to match the data that they have?
Both. By and large, by the time they make it to me, they have a big massive set of data, somebody on the team has heard about this AI thing, and they’ll come with a set of hypotheses—we think this data might be able to solve problem X or Y or Z. And that’s a great question, Byron, because that is how folks like me get introduced into projects, it’s because people have a vague notion as to how to use it, and it’s our job to crisp that up and to do that matching of the technology to the problem, so that they can get the best value out of this new technology.
And do you find that people are realistic in their expectations of where the technology is, or is it overhyped in the sense that you kind of have to reset some of their expectations?
Usually by the time they get to me, because I’m so practical, I don’t get the folks who have these giant general artificial intelligence goals. I get the folks who are like, I want to build a business and provide a lot of value, and how can I do that?” And from their point of view, often I can exceed their expectations actually because they think, Ah, I got to spend a year cleansing my data because the AI is only as good as the data”—well it turns out that’s not true and I can tell you why if you want to hear about it—they’ll say, you know, I need to have ten million rows of data because AI only works on large data sets,” it turns out that’s not necessarily true. So, actually, the technology, by and large, tends to exceed people’s expectations. Oh, and they think, I’ve been googling AI, and I need to learn all these algorithms, and we can’t have an AI project until I learn everything,” that’s also not true. With this technology, the inside of the box is like a Ferrari engine, right? But the outside of the box is like a steering wheel and two pedals, it’s not hard to use if you don’t get caught up in the details of the algorithms.
And are you referring to the various frameworks that are out there specifically?
Yeah, Theano, Torch, Google stuff like TensorFlow, all of those yes.
And how do you advise people in terms of evaluating those solutions?
It really depends on the problem. If I was to say there’s one piece of advice I almost always give, it’s to recognize that most of those frameworks have been built over the last few years by academics, and so they require a lot of work to get them going. I was getting one going about a year ago, and, you know, I’m a smart computer scientist and it took me six days to try to get it working. And, even then, just to have one deep learning run, it was this giant file and it was really hard to change, and it was hard to find the answers. Whereas, in contrast, I use this H2O package and R frontend to it, and I can run deep learning in one line of code there. So, I guess, my advice is to be discerning about the package, is it built for the PhD audience, or is it built, kind of, more for a business user audience, because there are a lot of differences. There very, very powerful, I mean, don’t get me wrong, TensorFlow, and those systems are hugely powerful, but often it’s power that you don’t need, and flexibility that you don’t need, and there’s just a tremendous amount of value you can get out of the low-hanging fruit of simple-to-use frameworks.
What are some guiding principles? There’s that one piece of advice, but what are some others? I have an enterprise, as you say, I’ve heard of this AI thing, I’m looking around, what should I be looking for?
Well, what you’re looking for is some pattern in your data that would predict something valuable. So, I’ll give you an example, I’m working with some educational institutions, they want to know, what topics that they offer in their courses will help students ultimately be successful in terms of landing a job. In the medical domain, what aspects of someone’s medical history would determine which of these five or six different drug regiments would be the most effective? In stock prices, what data about the securities we might invest in will tell us whether they’re going to go up or down? So, you see that pattern—you’ve always got some set of factors on one side, and then something you’re trying to predict, which if you could predict it well, would be valuable on the other side. That one pattern, if your listeners only listen to one thing, that’s the outside of the box. It’s really simple, it’s not that complicated. You’re just trying to get one set of data that predicts another set of data, and try to figure out if there would be some value there, then we would want to look into implementing an AI system. So that’s, kind of, thing number one I’d recommend, is to just have a look for that pattern in your business, see if you can find a use case or scenario in which that holds.
Switching gears a bit, you say that we had these early dreams of building a general intelligence, do you still think we’re going to build one sometime?
Maybe. I don’t like to get into those conversations because I think they’re really distracting. I think we’ve got so many hard problems, poverty, conflict—
An AGI would sure be helpful with those, wouldn’t it?
No. See that’s the problem, an AGI, it’s not aiming in the right direction, it’s ultimately going to be really distracting. We need to do the work, right? We need to go up the ladder, and the ladder starts with this single-link machine learning that we just talked about, you’ve got a pattern, you predict something. And then the next step is you try linking those up, you say, well if I’m going to have this feature in my new phone, then, let me predict how many people in a particular demographic will buy it, and then the next link is, given how many people will buy it, what price can I charge? And the next link is, how much price can I charge, how much money can I make? So it’s a chain of events that start with some action that you take, and ultimately lead to some outcome.
I’m solidly convinced, from a lot of things I’ve done over the thirty years I’ve been in AI, that we have to go through this phase, where we’re building these multi-linked systems that get from actions to outcomes, and that’ll maybe ultimately get us to what you might call, generalized AI, but we’re not there yet. We’re not even very good at the single-link systems, let alone multi-link and understanding feedback loops and complex dynamics, and unintended consequences and all of the things that start to emerge when you start trying to simulate the future with multi-link systems.
Well, let me ask the question a different way. Do you think that an AGI is an evolutionary result of a path we’re already on? Like, we’re at one percent and then we’ll be at two and then four, and eventually we’ll get there, or is that just a whole different beast, and you don’t just get there gradually, that’s an Aha!” kind of technology.
Yeah, I don’t know, that’s kind of a philosophical question, because even if I got to a full robot, we’d still have this question as to whether it was really conscious or intelligent. What I really think is important, is turn AI on its head, intelligence augmentation. What’s definitely going to happen is that humans are going to be working alongside intelligent systems. What was once a pencil, and once was a calculator, now is a computer is next going to be an AI? And just like computers have really super-powered our ability to write a document or have this podcast, right? They’re going to start also supercharging our ability to think through complex situations, and it’s going to be a side-by-side partnership for the foreseeable future, and perhaps indefinitely.
There’s a fair amount of fear in terms of what AI and automation in general will do to jobs. And, just to set up the question, there are often three different narratives. One is that, we’re about to enter this period where we’re going to have some portion of the population that is not able to add economic value and there’ll be, kind of, a permanent Great Depression. Then another view is that it will be far different than that, that every single thing a person can do, we’re going to build technology to do. And then there’s a third view that this is no different than any other transformative technology, people take it and use it to grow their own productivity, and everybody goes up a notch. What do you think, or a fourth choice, how do you see AI’s impact?
Well, I think multiple things are going to happen, we’re definitely seeing disruption in certain fields that AI is now able to do, but is it a different disruption than the introduction of the cotton gin or the automobile or any other technology disruption? Nah, it’s just got this kind of overlay of the robot apocalypse that makes it a little sexier to talk about. But, to me, it’s the same evolution we’ve always been going through as we build better and better tools to assist us with things. I’m not saying that’s not painful and I’m not saying that we won’t have displacement, but it’s not going to be a qualitatively different sort of shift in employment than we’ve seen before. I mean people have been predicting the end of employment because of automation for decades and decades. Future Shock, right? Alvin Toffler said that in the 60’s, and, AI is no different.
I think the other thing to say is we get into this hype-cycle because the vendors want you, as a journalist, to think it’s all really cool, then the journalists write about it and then there are more and more vendors, and we get really hyped about this, and I think it’s important to realize that we really are just in one-link AI right now—in terms of what’s widespread and what’s implemented and what’s useful, and where the hard implementation problems have been solved—so I would, sort of, tone down that side of things. From a jobs point of view, that means we’re not going to suddenly see this giant shift in jobs and automation, in fact I think AI is going to create many jobs. I wouldn’t say as many as we’ll lose, but I think there is a big opportunity for those fields. I hear about coal miners these days being retrained in IT, turns out that a lot of them seem to be really good, I’d love to train those other populations in how to be data scientists and machine learning people, I think there’s a great opportunity there.
Is there a shortage of talent in the field?
Absolutely, but, it’s not too hard to solve. The shortage of talent only comes when you think everybody has to understand these really complex PhD level frameworks. As the technology gets democratized, the ability to address the shortage of talent will become much easier. So we’re seeing one-click machine learning systems coming out, we’re seeing things like the AI labs that are coming out of places like Microsoft and Amazon. The technology is becoming something that lots of people can learn, as opposed to requiring this very esoteric, like, three computer science degrees like I have. And so, I think we’re going to start to see a decrease in that shortage in the near future.
All of the AI winters that happened in the past were all preceded by hype followed by unmet expectations, do you think we’re going to have another AI winter?
I think we’ll have an AI fall, but it won’t be a winter and here’s why—we’re seeing a level of substantive use cases for AI being deployed, especially in the enterprise, you know, widespread large businesses, at a level that never happened before. I was just talking to a guy earlier about the last AI hype cycle in the 80’s, where VLSI computer design by AI was this giant thing and the fifth generation,” and the Japanese and people were putting tens, hundreds of millions of dollars into these companies, and there was never any substance. There was no there” there, right? Nobody ever had deployed systems. AI and law, same thing, there’s been this AI and law effort for years and years and years, and it really never produced any commercial systems, for like a decade, and now we’re starting to see some commercial solidity there.
So, in terms of that Gartner hype-cycle, we’re entering the mass majority, but we are still seeing some hype, so there’ll be a correction. And we’ll probably get to where we can’t say AI anymore, and we’ll have to come up with some new name that we’re allowed to say, because for years you couldn’t say AI, you had to say data mining, right? And then I had to call myself an analytics consultant, and now it’s kind of cool I can call myself an AI person again. So the language will change, but it’s not going to be the frozen winter we saw before.
I wonder what term we’ll replace it with? I mean I hear people who avoid it are using, cognitive systems” and all of that, but it sounds just, kind of, like synonym substitution.
It is and that’s how it always goes, I’m evangelizing multi-link machine learning right now, I’m also testing decision intelligence. It’s kind of fun to be at the vanguard, where you can, as you’re inventing the new things, you get to name them, right? And you get to try to make everybody use that terminology. It’s in flux right now, there’s a time when we didn’t call e-mail e-mail,” right? It was computer mail.” So, I don’t know it hasn’t started to crystalize yet, it’s still in the twenty different new terminologies.
Eventually it will become just mail,” and the other will be, you know, snail mail.” It happens a lot, like, corn on the cob used to just be corn, and then canned corn came along so now we say corn on the cob, or cloth diapers… Well, anyway, it happens.
Walk me through some of the misconceptions that you come across in your day-to-day?
Sure. I think that the biggest mistake that I see is people get lost in algorithms or lost in data. So lost in algorithms, let’s say you’re listening to this and you say, Oh I’d like to be interested in AI,” and you go out and you google AI. The analogy, I think, is, imagine we’re the auto industry, and for the last thirty years, the only people in the auto industry had been inventing new kinds of engines, right? So you’re going to see the Wankel engine, and the four cylinder, you’re going to read about the carburetors, and it’s all been about the technology, right? And guess what, we don’t need five hundred different kinds of engines, right? So, if you go out and google it you’re going to be totally lost in hundreds of frameworks and engines and stuff. So the big misconception is that you somehow have to master engine building in order to drive the car, right? You don’t have to, but yet all the noise out there, I mean it’s not noise, it’s really great research, but from your point of view, someone who actually wants to use it for something valuable, it is kind of noise. So, I think one of the biggest mistakes people get into is they create a much higher barrier, they think they have to learn all this stuff in order to drive a car, which is not the case, it’s actually fairly simple technology to use. So, you need to talk to people like me who are, kind of, practitioners. Or, as you google, have a really discerning eye for the projects that worked and what the business value was, you know? And that applied side of things as opposed to the algorithm design.
Without naming company names or anything, tell me some projects that you worked on and how you looked at it and how you approached it and what was the outcome like, just walk me through a few use cases.
So I’ll rattle through a few of them and you can tell me which one to talk about, which one you think is the coolest—morphological hair comparison for the Colorado Bureau of Investigation, hazardous buried waste detection for the Department of Energy, DNA pattern recognition for the human genome project, stock price prediction, medical precision medicine prediction… It’s the coolest field, you get to do so much interesting work.
Well let’s start with the hair one.
Sure, so this was actually a few years back, it was during the OJ trials. The question was, you go out to a crime scene and there’s hairs and fibers that you pick up, the CSI guys, right? And then you also have hairs from your suspect. So you’ve got these two hairs, one from the crime scene, one from your suspect and if they match, that’s going to be some evidence that you’re guy was at the scene right? So how do you go about doing that, well, you take a microphotograph of the two of them. The human eye is pretty good at, sort of, looking at the two hairs and seeing if they match, we actually use a microscope that shows us both at the same time. But, AI can take it a step further. So, just like AI is, kind of, the go-to technology for breast cancer prediction and pap smear analysis and all of this micro-photography stuff, this project that I was on used AI to recognize if these two hairs came from the same guy or not? It’s a pretty neat project.
And so that was in the 90’s?
Yeah it was a while back.
And that would have been using techniques we still have today, or using older techniques?
Both, actually, that was a back-propagation neural network, and I’m not allowed to say back propagation, nor am I really allowed to say neural network, but the hidden secret is that all the great AI stuff still use back-propagation-like neural networks. So, it was the foundations of what we do today. Today we still use neural nets, they’re the main machine learning algorithm, but they’re deeper, they have more and more layers of artificial neurons. We still learn, we still change the weights of the simulated synapses on the networks, but we have a more sophisticated algorithm that does that. So, foundationally, it’s really the same thing, it hasn’t changed that much in so many years, we’re still artificial neural network centric in most of AI today.
Now let’s go to hazardous waste.
Sure, so this was for the Department of Energy. Again it was an imaging project, but here, the question was, you’ve got these buried drums of leaking chemical nerve gas, that’ve been dumped into these superfund sites, and it was really carelessly done. I mean, literally, trenches were dug and radioactive stuff was just dumped in them. And after a few years folks realized that wasn’t so smart, and so, then they took those sites and they passed these pretty cool sensors over them, like gravitometers, that detected micro-fluctuations in gravity, and ground-penetrating radar and other techniques that could sense what was underground—this was originally developed for the oil industry, actually, to find buried energy deposits—and you try to characterize where those things are. Where the neural net was good was in combining all those sensors from multiple modalities into a picture that was better than any one of the sensors.
And what technologies did that use?
Neural nets, same thing, back propagation.
At the beginning you made some references to some recent breakthroughs, but would you say that most of our techniques are things we’ve known about since the 60’s, we just didn’t have the computer horsepower to do it? Would that be fair to say or not?
It’s both, it’s the rocket engines plus the rocket fuel, right? I remember as a graduate student, I used to take over all the faculties computers at night when there was no security, I’d run my neural net training on forty different machines and then have them all RPC the data back to my machine. So, I had enough horsepower back then, but what we were missing was the modern deep-learning algorithms that allow us to get better performing systems out of that data, and out of those high-performance computing environments.
And now what about the human genome project, tell me about that project.
That was looking at DNA patterns, and trying to identify something called a ribosomal-binding site. If you saw that Star Trek episode where everybody turns into a lizard, there are these parts of our DNA that we don’t really know what they do between the parts that express themselves. This was a project nicely funded by a couple of funding agencies to detect these locations on a DNA strand.
Was that the one where everybody essentially accelerated their evolution and Picard was some kind of a nervous chimp of some kind, somebody else was a salamander?
Yes that’s right, remember it was Deanna Troi who turned into a salamander, I think. And she was expressing the introns, the stuff that was between the currently expressed genome. This was a project that tried to find the boundaries between the expressed and the unexpressed parts. Pretty neat science project, right?
Exactly. Tell me about the precision medicine one, was that a recent one?
Yeah, so the first three were kind of older. I’m Chief Scientist, also, at ehealthanalytics.net and they’ve taken on this medical trials project. It turns out that if you do a traditional medical trial, it’s very backward facing and you often have very homogenous data. In contrast, we’ve got a lot of medical devices that are spitting out data, like, I’m wearing my Fitbit right now and it’s got data about me, and, you know, we have more DNA information, and with all of that we can actually do better than traditional medical trials. So, that was a project I did for those guys. More recently we’re predicting failure in medical devices. That’s not as much precision medicine as precision analysis of medical devices, so that we can catch them in the field before they fail, and that’s obviously a really important thing to be able to do.
And so you’ve been at this for, you say, three decades.
Three decades, yeah. It was about 1984, when I built my first neural net.
Would you say that your job has changed over that time, or has it, in a way, not—you still look at the data, look at the approach, figure out what question you’re asking, figure out how to get an answer?
From that point of view, it’s really been the same. I think what has changed is, once I built the neural net—before, the accuracies and the false-positives and the false-negatives were kind of, eh, they weren’t really exciting results. Now, we see Microsoft, a couple of years ago, using neural network transfer, which was my big algorithm invention, to beat humans at visual pattern recognition. So, the error rates, just with the new deep learning algorithms, have plummeted, as I’m sure your other interviewee’s have told you about, but the process has been really the same.
And I’ll tell you what’s surprising, you’d think that things would have changed a lot, but there just hasn’t been a lot of people who drive the cars, right? Up until very recently, this field has really been dominated by people who build the engines. So, we’re just on the cusp. I look at SAP is a great example of this. SAP’s coming out with this big new Leonardo launch of its machine learning platform, and, they’re not trying to build new algorithms, right? SAP is partnering with Google and NVIDIA, and what they recognize is that the next big innovation is in the ability of connecting the algorithms to the applied problems, and just churning out one use case after another, that drives value for their customers. I would’ve liked to have seen us progress further along those lines over the last few years, but I guess just the performance wasn’t there and the interest wasn’t there. That’s what I’m excited about with this current period of excitement in AI, that we’ll finally start to have a bunch of people who drive the cars, right? Who use this technology in valuable ways to get from here to there to predict stock prices, to match people to the perfect job—that’s another project that I’m doing, for HR human resources—all these very practical things that have so much value. But yeah, it hasn’t really changed that much, but I hope it does, I hope we get better at software engineering for AI, because that’s really what’s just starting right now.
So, you, maybe, will become more of a car-driver—to use your analogy—in the future. Even somebody as steeped in it as you, it sounds like you would prefer to use higher-level tools that are just that much easier to use.
Yeah, and the reason is, we have plenty of algorithms, we’re totally saturated with new algorithms. The big desperate need that everybody has is, again, to democratize this and to make it useful, and to drive business value. You know, a friend of mine who just finished an AI project said on a ten million dollar project, we just upped our revenue by eighteen percent from this AI thing. That’s typical, and that’s huge, right? But yet everybody was doing it for the very first time, and he’s at a fairly large company, so, that’s where the big excitement is. I mean, I know it’s not as sexy as artificial general intelligence, but it’s really important to the human race, and that’s why I keep coming back to it.
You made a passing reference to image recognition and the leap forward we have there, how do you think it is that people do such a good job, I mean is it just all transferred learning after a while, do we just sort of get used to it, or do you think people do it in a different way than we got machines to do it?
In computer vision, there was a paper that came out last year that Yann LeCun was sending around that said that somebody was looking at the structure of the deep-learning vision networks and had found this really strong analogue to the multiple layers—what is it the lateral geniculate nucleus, I’m not a human vision person, but there’s these structures in the human vision system that are very analogous. So, it’s like this convergent evolution, that computers converge to the same way of recognizing images that it turns out the human brain does things.
Were we totally inspired by the human brain? Yes, to some extent. Back in the day when we’d go to the NIPS conference, half the people there were in neurophysiology, and half of us were computer modelers, more applied people, and so there was a tremendous amount of interplay between those two sides. But more recently, folks have just tried to get computers to see things, for self-driving cars and stuff, and we keep heading back to things that sort of look like the human vision system, I think that’s pretty interesting.
You know, I think the early optimism in AI—like the Dartmouth project where they thought they could do a bunch of stuff if they worked really hard on it for one summer—stemmed from a hope that, just like in Physics you had a few laws that explain everything, in electronics, in magnetism, it’s just a few laws. And the hope was that intelligence would just be three or four simple laws, we’ll figure them out and that’s all it’s going to be. I guess we’ve given up on that, or have we, we’re essentially brute forcing our way to everything, right?
Yeah, it’s sort of the submergent property, right? Like Conway’s Game of Life,” has these very complex emergent epiphenomenon from just a few simple rules. I, actually, haven’t given up on that, I just think we don’t quite have the substrate right yet. And again I keep going back to single-link learning versus multi-link. I think when we start to build multi-link systems that have complex dynamics that end up doing four-at-a-time simulation using piecewise backward machine learning based on historical data, I think we are going to see a bit of an explosion and start to see, kind of, this emergence happen. That’s the optimistic, non-practical side of me. I just think we’ve been focusing so much on certain low-hanging fruit problems, right? We had image recognition—because we had these great successes in medicine, even with the old algorithms, they were just so great at cancer recognition and images—and then Google was so smart with advertising, and then Netflix with the movies. But if you look at those successful use cases, there’s only like a dozen of them that have been super successful, and we’ve been really focused on these use cases that fit our hammer, we’ve been looking at nails, right? Because that’s the technology that we had. But I think multi-link systems will make a big difference going forward, and when we do that I think we might start to see this kind of explosion in what the systems can do, I’m still an optimist there.
There are those who think we really will have an explosion, literally, from it all.
Yeah, like the singularitists, yep.
It’s interesting that there are people, high profile individuals of unquestionable intelligence, who believe we are at the cusp of building something transformative, where do you think they err?
Well, I can really only speak to my own experience, I think there’s this hype thing, right? All the car companies want to show that they’re still relevant, so they hype the self-driving cars, and of course we’re not taking security, and other things into account, and we all kind of wanted to get jumping on that bandwagon. But, my experience is just very plebeian, you just got to do the work, you got to roll up your sleeves you got to condition your data, you got to go around the data science loop and then you need to go forward. I think people are really caught up in this prediction task, like, What can we predict, what will the AI tell us, what can we learn from the AI?” and I think we’re all caught up in the wrong question, that’s not the question. The question is, what can we do? What actions will we take that lead to which outcomes we care about, right? So, what should we do in this country, that’s struggling in conflict, to avoid the unintended consequences? What should we teach these students so that they have a good career? What actions can we take to mitigate against sea-level rise in our city?
Nobody is thinking in terms of actions that lead to outcomes, they’re thinking of data that leads to predictions. And again I think this comes from the very academic history of AI, where it was all about the idea factory and what can we conclude from this. And yeah, it’s great, that’s part of it, being able to say, here’s this image, here’s what we’re looking at, but to really be valuable for something it can’t be just recognizing an image, it has to be take some action that leads to some outcome. I think that’s what’s been missing and that’s what’s coming next.
Well that sounds like a great place to end our conversation.
Excellent.
I want to thank you so much, you’ve just gave us such a good overview of what we can do today, and how to go about doing it, and I thank you for taking the time.
Thank you Byron, I appreciate the time.
Byron explores issues around artificial intelligence and conscious computers in his upcoming book The Fourth Age, to be published in April by Atria, an imprint of Simon & Schuster. Pre-order a copy here.
[voices_in_ai_link_back]

Voices in AI – Episode 31: A Conversation with Tasha Nagamine

[voices_in_ai_byline]
In this episode, Byron and Tasha talk about speech recognition, AGI, consciousness, Droice Lab, healthcare, and science fiction.
[podcast_player name=”Episode 31 – A Conversation with Tasha Nagamine” artist=”Byron Reese” album=”Voices in AI” url=”https://voicesinai.s3.amazonaws.com/2018-01-22-(00-57-02)-tasha-nagamine.mp3″ cover_art_url=”https://voicesinai.com/wp-content/uploads/2017/09/voices-in-ai-cover.png”]
[voices_in_ai_byline]
Byron Reese: This is Voices in AI, brought to you by Gigaom. I’m Byron Reese. Today our guest is Tasha Nagamine. She’s a PhD student at Columbia University, she holds an undergraduate degree from Brown and a Masters in Electrical Engineering from Columbia. Her research is in neural net processing in speech and language, then the potential applications of speech processing systems through, here’s the interesting part, biologically-inspired, deep neural network models. As if that weren’t enough to fill up a day, Tasha is also the CTO of Droice Labs, an AI healthcare company, which I’m sure we will chat about in a few minutes. Welcome to the show, Tasha.
Tasha Nagamine: Hi.
So, your specialty, it looks like, coming all the way up, is electrical engineering. How do you now find yourself in something which is often regarded as a computer science discipline, which is artificial intelligence and speech recognition?
Yeah, so it’s actually a bit of an interesting meandering journey, how I got here. My undergrad specialty was actually in physics, and when I decided to go to grad school, I was very interested, you know, I took a class and found myself very interested in neuroscience.
So, when I joined Columbia, the reason I’m actually in the electrical engineering department is that my advisor is an EE, but what my research and what my lab focuses on is really in neuroscience and computational neuroscience, as well as neural networks and machine learning. So, in that way, I think what we do is very cross-disciplinary, so that’s why the exact department, I guess, may be a bit misleading.
One of my best friends in college was a EE, and he said that every time he went over to like his grandmother’s house, she would try to get him to fix like the ceiling fan or something.  Have you ever had anybody assume you’re proficient with a screwdriver as well?
Yes, that actually happens to me quite frequently. I think I had one of my friends’ landlords one time, when I said I was doing electrical engineering, thought that that actually meant electrician, so was asking me if I knew how to fix light bulbs and things like that.
Well, let’s start now talking about your research, if you would. In your introduction, I stressed biologically-inspired deep neural networks. What do you think, do we study the brain and try to do what it does in machines, or are we inspired by it, or do we figure out what the brain’s doing and do something completely different? Like, why do you emphasize “biologically-inspired” DNNs?
That’s actually a good question, and I think the answer to that is that, you know, researchers and people doing machine learning all over the world actually do all of those things. So, the reason that I was stressing a biologically-inspired—well, you could argue that, first of all, all neural networks are in some way biologically-inspired; now, whether or not they are a good biologically-inspired model, is another question altogether—I think a lot of the big, sort of, advancements that come, like a convolutional neural network was modeled basically directly off of the visual system.
That being said, despite the fact that there are a lot of these biological inspirations, or sources of inspiration, for these models, there’s many ways in which these models actually fail to live up to the way that our brains actually work. So, by saying biologically-inspired, I really just mean a different kind of take on a neural network where we try to, basically, find something wrong with a network that, you know, perhaps a human can do a little bit more intelligently, and try to bring this into the artificial neural network.
Specifically, one issue with current neural networks is that, usually, unless you keep training them, they have no way to really change themselves, or adapt to new situations, but that’s not what happens with humans, right? We continuously take inputs, we learn, and we don’t even need supervised labels to do so. So one of the things that I was trying to do was to try to draw from this inspiration, to find a way to kind of learn in an unsupervised way, to improve your performance in a speech recognition task.
So just a minute ago, when you and I were chatting before we started recording, a siren came by where you are, and the interesting thing is, I could still understand everything you were saying, even though that siren was, arguably, as loud as you were. What’s going on there, am I subtracting out the siren? How do I still understand you? I ask this for the obvious reason that computers seem to really struggle with that, right?
Right, yeah. And actually how this works in the brain is a very open question and people don’t really know how it’s done. This is actually an active research area of some of my colleagues, and there’s a lot of different models that people have for how this works. And you know, it could be that there’s some sort of filter in your brain that, basically, sorts speech from the noise, for example, or a relevant signal from an irrelevant one. But how this happens, and exactly where this happens is pretty unknown.
But you’re right, that’s an interesting point you make, is that machines have a lot of trouble with this. And so that’s one of the inspirations behind these types of research. Because, currently, in machine learning, we don’t really know the best way to do this and so we tend to rely on large amounts of data, and large amounts of labeled data or parallel data, data corrupted with noise intentionally, however this is definitely not how our brain is doing it, but how that’s happening, I don’t think anyone really knows.
Let me ask you a different question along the same lines. I read these stories all the time that say that, “AI has approached human-quality in transcribing speech,” so I see that. And then I call my airline of choice, I will not name them, and it says, “What is your frequent flyer number?” You know, it’s got Caller ID, it should know that, but anyway. Mine, unfortunately, has an A, an H, and an 8 in it, so you can just imagine “AH8H888H”, right?
It never gets it. So, I have to get up, turn the fan off in my office, take my headset off, hold the phone out, and say it over and over again. So, two questions: what’s the disconnect between what I read and my daily experience? Actually, I’ll give you that question and then I have my follow up in a moment.
Oh, sure, so you’re saying, are you asking why it can’t recognize your—
But I still read these stories that say it can do as good of a job as a human.
Well, so usually—and, for example, I think, recently, there was a story published about Microsoft coming up with a system that had reached human parity in speech recognition—well, usually when you’re saying that, you have it on a somewhat artificial task. So, you’ll have a predefined data set, and then test the machine against humans, but that doesn’t necessarily correspond to a real-world setting, they’re not really doing speech recognition out in the wild.
And, I think, you have an even more difficult problem, because although it’s only frequent flyer numbers, you know, there’s no language model there, there’s no context for what your next number should be, so it’s very hard for that kind of system to self-correct, which is a bit problematic.
So I’m hearing two things. The first thing, it sounds like you’re saying, they’re all cooking the books, as it were. The story is saying something that I interpret one way that isn’t real, if you dig down deep, it’s different. But the other thing you seem to be saying is, even though there’s only thirty-six things I could be saying, because there’s no natural flow to that language, it can’t say, “oh, the first word he said was ‘the’ and the third word was ‘ran;’ was that middle word ‘boy’ or ‘toy’?” It could say, “Well, toys don’t run, but boys do, therefore it must be, ‘The boy ran.'” Is that what I’m hearing you saying, that a good AI system’s going to look contextually and get clues from the word usage in a way that a frequent flyer system doesn’t.
Right, yeah, exactly. I think this is actually one of the fundamental limitations of, at least, acoustic modeling, or, you know, the acoustic part of speech recognition, which is that you are completely limited by what the person has said. So, you know, maybe it could be that you’re not pronouncing your “t” at the end of “eight,” very emphatically. And the issue is that, there’s nothing you can really do to fix that without some sort of language-based information to fix it.
And then, to answer your first question, I wouldn’t necessarily call it “cooking the books,” but it is a fact that, you know, really the data that you have to train on and test on and to evaluate your metrics on, often, almost never really matches up with real-world data, and this is a huge problem in the speech domain, it’s a very well-known issue.
You take my 8, H, and A example—which you’re saying that’s a really tricky problem without context—and, let’s say, you have one hundred English speakers, but one is from Scotland, and one could be Australian, and one could be from the east coast, one could be from the south of the United States; is it possible that the range of how 8 is said in all those different places is so wide that it overlaps with how H is said in some places. So, in other words, it’s a literally insoluble problem.
It is, I would say it is possible. One of the issues is then you should have a separate model for different dialects. I don’t want to dive too far into the weeds with this, but at the root of a speech recognition system is often things like the fundamental linguistic or phonetic unit is a phoneme, which is the smallest speech sound, and people even argue about whether or not that these actually exist, what they actually mean, whether or not this is a good unit to use when modeling speech.
That being said, there’s a lot of research underway, for example, sequence to sequence models or other types of models that are actually trying to bypass this sort of issue. You know, instead of having all of these separate components modeling all of the acoustics separately, can we go directly from someone’s speech and from there exactly get text. And maybe through this unsupervised approach it’s possible to learn all these different things about dialects, and to try to inherently learn these things, but that is still a very open question, and currently those systems are not quite tractable yet.
I’m only going to ask one more question on these lines—though I could geek out on this stuff all day long, because I think about it a lot—but really quickly, do you think you’re at the very beginning of this field, or do you feel it’s a pretty advanced field? Just the speech recognition part.
Speech recognition, I think we’re nearing the end of speech recognition to be honest. I think that you could say that speech is fundamentally limited; you are limited by the signal that you are provided, and your job is to transcribe that.
Now, where speech recognition stops, that’s where natural language processing begins. As everyone knows, language is infinite, you can do anything with it, any permutation of words, sequences of words. So, I really think that natural language processing is the future of this field, and I know that a lot of people in speech are starting to try to incorporate more advanced language models into their research.
Yeah, that’s a really interesting question. So, I ran an article on Gigaom, where I had an Amazon Alexa device on my desk and I had a Google Assistant on my desk, and what I noticed right away is that they answer questions differently. These were factual questions, like “How many minutes are in a year?” and “Who designed the American flag?” They had different answers. And you can say it’s because of an ambiguity in the language, but if this is an ambiguity, then all language is naturally ambiguous.
So, the minutes in a year answer difference was that one gave you the minutes in 365.24 days, a solar year, and one gave you the minutes in a calendar year. And with regard to the flag, one said Betsy Ross, and one said the person who designed the fifty-star configuration on the current flag.
And so, we’re a long way away from the machines saying, “Well, wait a second, do you mean the current flag or the original flag?” or, “Are you talking about a solar year or a calendar year?” I mean, we’re really far away from that, aren’t we?
Yeah, I think that’s definitely true. You know, people really don’t understand how even humans process language, how we disambiguate different phrases, how we find out what are the relevant questions to ask to disambiguate these things. Obviously, people are working on that, but I think we are quite far from true natural language understanding, but yeah, I think that’s a really, really interesting question.
There were a lot of them, “Who invented the light bulb?” and “How many countries are there in the world?” I mean the list was endless. I didn’t have to look around to find them. It was almost everything I asked, well, not literally, “What’s 2+2?” is obviously different, but there were plenty of examples.  
To broaden that question, don’t you think if we were to build an AGI, an artificial general intelligence, an AI as versatile as a human, that’s table stakes, like you have to be able to do that much, right?
Oh, of course. I mean, I think that one of the defining things that makes human intelligence unique, is the ability to understand language and an understanding of grammar and all of this. It’s one of the most fundamental things that makes us human and intelligent. So I think, yeah, to have an artificial general intelligence, it would be completely vital and necessary to be able to do this sort of disambiguation.
Well, let me ratchet it up even another one. There’s a famous thought experiment called the Chinese Room problem. For the benefit of the listener, the setup is that there’s a person in a room who doesn’t speak any Chinese, and the room he’s in is full of this huge number of very specialized books; and people slide messages under the door to him that are written in Chinese. And he has this method where he looks up the first character and finds the book with that on the spine, and goes to the second character and the third and works his way through, until he gets to a book that says, “Write this down.” And he copies these symbols, again, he doesn’t know what the symbols are; he slides the message back out, and the person getting it thinks it’s a perfect Chinese answer, it’s brilliant, it rhymes, it’s great.
So, the thought experiment is this, does the man understand Chinese? And the point of the thought experiment is that this is all a computer does—it runs this deterministic program, and it never understands what it’s talking about. It doesn’t know if it’s about cholera or coffee beans or what have you. So, my question is, for an AGI to exist, does it need to understand the question in a way that’s different than how we’ve been using that word up until now?
That’s a good question. I think that, yeah, to have an artificial general intelligence, I think the computer would have to, in a way, understand the question. Now, that being said, what is the nature of understanding the question? How do we even think, is a question that I don’t think even we know the answer to. So, it’s a little bit difficult to say, exactly, what’s the minimum requirement that you would need for some sort of artificial general intelligence, because as it stands now, I don’t know. Maybe someone smarter than me knows the answer, but I don’t even know if I really understand how I understand things, if that makes sense to you.
So what do you do with that? Do you say, “Well, that’s just par for the course. There’s a lot of things in this universe we don’t understand, but we’re going to figure it out, and then we’ll build an AGI”? Is the question of understanding just a very straightforward scientific question, or is it a metaphysical question that we don’t really even know how to pose or answer?
I mean, I think that this question is a good question, and if we’re going about it the right way, it’s something that remains to be seen. But I think one way that we can try to ensure that we’re not straying off the path, is by going back to these biologically-inspired systems. Because we know that, at the end of the day, our brains are made up of neurons, synapses, connections, and there’s nothing very unique about this, it’s physical matter, there’s no theoretical reason why a computer cannot do the same computations.
So, if we can really understand how our brains are working, what the computations it performs are, how we have consciousness; then I think we can start to get at those questions. Now, that being said, in terms of where neuroscience is today, we really have a very limited idea of how our brains actually work. But I think it’s through this avenue that we stand the highest chance of success of trying to emulate, you know—
Let’s talk about that for a minute, I think that’s a fascinating topic. So, the brain has a hundred billion neurons that somehow come together and do what they do. There’s something called a nematode worm—arguably the most successful animal on the planet, ten percent of all animals on the planet are these little worms—they have I think 302 neurons in their brain. And there’s been an effort underway for twenty years to model that brain—302 neurons—in the computer and make a digitally living nematode worm, and even the people who have worked on that project for twenty years, don’t even know if that’s possible.
What I was hearing you say is, once we figure out what a neuron does—this reductionist view of the brain—we can build artificial neurons, and build a general intelligence, but what if every neuron in your brain has the complexity of a supercomputer? What if they are incredibly complicated things that have things going on at the quantum scale, that we are just so far away from understanding? Is that a tenable hypothesis? And doesn’t that suggest, maybe we should think about intelligence a different way because if a neuron’s as complicated as a supercomputer, we’re never going to get there.
That’s true, I am familiar with that research. So, I think that there’s a couple of ways that you can do this type of study because, for example, trying to model a neuron at the scale of its ion channels and individual connections is one thing, but there are many, many scales upon which your brain or any sort of neural system works.
I think to really get this understanding of how the brain works, it’s great to look at this very microscale, but it also helps to go very macro and instead of modeling every single component, try to, for example, take groups of neurons, and say, “How are they communicating together? How are they communicating with different parts of the brain?” Doing this, for example, is usually how human neuroscience works and humans are the ones with the intelligence. If you can really figure out on a larger scale, to the point where you can simplify some of these computations, and instead of understanding every single spike, perhaps understanding the general behavior or the general computation that’s happening inside the brain, then maybe it will serve to simplify this a little bit.
Where do you come down on all of that? Are we five years, fifty years or five hundred years away from cracking that nut, and really understanding how we understand and understanding how we would build a machine that would understand, all of this nuance? Do you think you’re going to live to see us make that machine?
I would be thrilled if I lived to see that machine, I’m not sure that I will. Exactly saying when this will happen is a bit hard for me to predict, but I know that we would need massive improvements; probably, algorithmically, probably in our hardware as well, because true intelligence is massively computational, and I think it’s going to take a lot of research to get there, but it’s hard to say exactly when that would happen.
Do you keep up with the Human Brain Project, the European initiative to do what you were talking about before, which is to be inspired by human brains and learn everything we can from that and build some kind of a computational equivalent?
A little bit, a little bit.
Do you have any thoughts on—if you were the betting sort—whether that will be successful or not?
I’m not sure if that’s really going to work out that well. Like you said before, given our current hardware, algorithms, our abilities to probe the human brain; I think it’s very difficult to make these very sweeping claims about, “Yes, we will have X amount of understanding about how these systems work,” so I’m not sure if it’s going to be successful in all the ways it’s supposed to be. But I think it’s a really valuable thing to do, whether or not you really achieve the stated goal, if that makes sense.
You mentioned consciousness earlier. So, consciousness, for the listeners, is something people often say we don’t know what it is; we know exactly what it is, we just don’t know how it is that it happens. What it is, is that we experience things, we feel things, we experience qualia—we know what pineapple tastes like.
Do you have any theories on consciousness? Where do you think it comes from, and, I’m really interested in, do we need consciousness in order to solve some of these AI problems that we all are so eager to solve? Do we need something that can experience, as opposed to just sense?
Interesting question. I think that there’s a lot of open research on how consciousness works, what it really means, how it helps us do this type of cognition. So, we know what it is, but how it works or how this would manifest itself in an artificial intelligence system, is really sort of beyond our grasp right now.
I don’t know how much true consciousness a machine needs, because, you could say, for example, that having a type of memory may be part of your consciousness, you know, being aware, learning things, but I don’t think we have yet enough really understanding of how this works to really say for sure.
All right fair enough. One more question and I’ll pull the clock back thirty years and we’ll talk about the here and now; but my last question is, do you think that a computer could ever feel something? Could a computer ever feel pain? You could build a sensor that tells the computer it’s on fire, but could a computer ever feel something, could we build such a machine?
I think that it’s possible. So, like I said before, there’s really no reason why—what our brain does is really a very advanced biological computer—you shouldn’t be able to feel pain. It is a sensation, but it’s really just a transfer of information, so I think that it is possible. Now, that being said, how this would manifest, or what a computer’s reaction would be to pain or what would happen, I’m not sure what that would be, but I think it’s definitely possible.
Fair enough. I mentioned in your introduction that you’re the CTO of an AI company Droice Labs, and the only setup I made was that it was a healthcare company. Tell us a little bit more, what challenge that Droice Labs is trying to solve, and what the hope is, and what your present challenges are and kind of the state of where you’re at?
Sure. Droice is a healthcare company that uses artificial intelligence to help provide artificial intelligence solutions to hospitals and healthcare providers. So, one of the main things that we’re focusing on right now is to try to help doctors choose the right treatment for their patients. This means things like, for example, you come in, maybe you’re sick, you have a cough, you have pneumonia, let’s say, and you need an antibiotic. What we try to do is, when you’re given an antibiotic, we try to predict whether or not this treatment will be effective for you, and also whether or not it’ll have any sort of adverse event on you, so both try to get people healthy, and keep them safe.
And so, this is really what we’re focusing on at the moment, trying to make a sort of artificial brain for healthcare that can, shall we say, augment the intelligence of the doctors and try to make sure that people stay healthy. I think that healthcare’s a really interesting sphere in which to use artificial intelligence because currently the technology is not very widespread because of the difficulty in working with hospital and medical data, so I think it’s a really interesting opportunity.
So, let’s talk about that for a minute, AIs are generally only as good as the data we train them with. Because I know that whenever I have some symptom, I type it into the search engine of choice, and it tells me I have a terminal illness; it just happens all the time. And in reality, of course, whatever that terminal illness is, there is a one-in-five-thousand chance that I have that, and then there’s also a ninety-nine percent chance I have whatever much more common, benign thing. How are you thinking about how you can get enough data so that you can build these statistical models and so forth?
We’re a B2B company, so we have partnerships with around ten hospitals right now, and what we do is get big data dumps from them of actual electronic health records. And so, what we try to do is actually use real patient records, like, millions of patient records that we obtain directly from our hospitals, and that’s how we really are able to get enough data to make these types of predictions.
How accurate does that data need to be? Because it doesn’t have to be perfect, obviously. How accurate does it need to be to be good enough to provide meaningful assistance to the doctor?
That is actually one of the big challenges, especially in this type of space. In healthcare, it’s a bit hard to say which data is good enough, because it’s very, very common. I mean, one of the hallmarks of clinical or medical data is that it will, by default, contain many, many missing values, you never have the full story on any given patient.
Additionally, it’s very common to have things like errors, there’s unstructured text in your medical record that very often contains mistakes or just insane sentence fragments that don’t really make sense to anyone but a doctor, and this is one of the things that we work really hard on, where a lot of times traditional AI methods may fail, but we basically spend a lot of time trying to work with this data in different ways, come up with noise-robust pipelines that can really make this work.
I would love to hear more detail about that, because I’m sure it’s full of things like, “Patient says their eyes water whenever they eat potato chips,” and you know, that’s like a data point, and it’s like, what do you do with that. If that is a big problem, can you tell us what some of the ways around it might be?
Sure. I’m sure you’ve seen a lot of crazy stuff in these health records, but what we try to do is—instead of biasing our models by doing anything in a rule-based manner—we use the fact that we have big data, we have a lot of data points, to try to really come up with robust models, so that, essentially, we don’t really have to worry about all that crazy stuff in there about potato chips and eyes watering.
And so, what we actually end up doing is, basically, we take these many, many millions of individual electronic health records, and try to combine that with outside sources of information, and this is one of the ways that we can try to really augment the data on our health record to make sure that we’re getting the correct insights about it.
So, with your example, you said, “My eyes water when I eat potato chips.” What we end up doing is taking that sort of thing, and in an automatic way, searching sources of public information, for example clinical trials information or published medical literature, and we try to find, for example, clinical trials or papers about the side effects of rubbing your eyes while eating potato chips. Now of course, that’s a ridiculous example, but you know what I mean.
And so, by augmenting this public and private data together, we really try to create this setup where we can get the maximum amount of information out of this messy, difficult to work with data.
The kinds of data you have that are solid data points, would be: how old is the patient, what’s their gender, do they have a fever, do they have aches and pains; that’s very coarse-level stuff. But like—I’m regretting using the potato chip example because now I’m kind of stuck with it—but, a potato chip is made of a potato which is a tuber, which is a nightshade and there may be some breakthrough, like, “That may be the answer, it’s an allergic reaction to nightshades. And that answer is so many levels removed.
I guess what I’m saying is, and you said earlier, language is infinite, but health is near that, too, right? There are so many potential things something could be, and yet, so few data points, that we must try to draw from. It would be like, if I said, “I know a person who is 6’ 4” and twenty-seven years old and born in Chicago, what’s their middle name?” It’s like, how do you even narrow it down to a set of middle names?
Right, right. Okay, I think I understand what you’re saying. This is, obviously, a challenge, but one of the ways that we kind of do this is, the first thing is our artificial intelligence is really intended for doctors and not the patients. Although, we were just talking about AGI and when it will happen, but the reality is we’re not there yet, so while our system tries to make these predictions, it’s under the supervision of a doctor. So, they’re really looking at these predictions and trying to pull out relevant things.
Now, you mentioned, the structured data—this is your age, your weight, maybe your sex, your medications; this is structured—but maybe the important thing is in the text, or is in the unstructured data. So, in this case, one of the things that we try to do, and it’s one of the main focuses of what we do, is to try to use natural language processing, NLP, to really make sure that we’re processing this unstructured data, or this text, in a way to really come up with a very robust, numerical representation of the important things.
So, of course, you can mine this information, this text, to try to understand, for example, you have a patient who has some sort of allergy, and it’s only written in this text, right? In that case, you need a system to really go through this text with a fine-tooth comb, and try to really pull out risk factors for this patient, relevant things about their health and their medical history that may be important.
So, is it not the case that diagnosing—if you just said, here is a person who manifests certain symptoms, and I want to diagnose what they have—may be the hardest problem possible. Especially compared to where we’ve seen success, which is, like, here is a chest x-ray, we have a very binary question to ask: does this person have a tumor or do they not? Where the data is: here’s ten thousand scans with the tumor, here’s a hundred thousand without a tumor.
Like, is it the cold or the flu? That would be an AI kind of thing because an expert system could do that. I’m kind of curious, tell me what you think—and then I’d love to ask, what would an ideal world look like, what would we do to collect data in an ideal world—but just with the here and now, aspirationally, what do you think is as much as we can hope for? Is it something, like, the model produces sixty-four things that this patient may have, rank ordered, like a search engine would do from the most likely to the least likely, and the doctor can kind of skim down it and look for something that catches his or her eye. Is that as far as we can go right now? Or, what do you think, in terms of general diagnosing of ailments?
Sure, well, actually, what we focus on currently is really on the treatment, not on the diagnosis. I think the diagnosis is a more difficult problem, and, of course, we really want to get into that in the future, but that is actually somewhat more of a very challenging sort of thing to do.
That being said, what you mentioned, you know, saying, “Here’s a list of things, let’s make some predictions of it,” is actually a thing that we currently do in terms of treatments for patients. So, one example of a thing that we’ve done is built a system that can predict surgical complications for patients. So, imagine, you have a patient that is sixty years old and is mildly septic, and may need some sort of procedure. What we can do is find that there may be a couple alternative procedures that can be given, or a nonsurgical intervention that can help them manage their condition. So, what we can do is predict what will happen with each of these different treatments, what is the likelihood it will be successful, as well as weighing this against their risk options.
And in this way, we can really help the doctor choose what sort of treatment that they should give this person, and it gives them some sort of actionable insight, that can help them get their patients healthy. Of course, in the future, I think it would be amazing to have some sort of end to end system that, you know, a patient comes in, and you can just get all the information and it can diagnose them, treat them, get them better, but we’re definitely nowhere near that yet.
Recently, IBM made news that Watson had prescribed treatment for cancer patients that was largely identical to what the doctors did, but it had the added benefit that in a third of the cases it found additional treatment options, because it had virtue of being trained on a quarter million medical journals. Is that the kind of thing that’s like “real, here, today,” that we will expect to see more things like that?
I see. Yeah, that’s definitely a very exciting thing, and I think that’s great to see. One of the things that’s very interesting, is that IBM primarily works on cancer. It’s lacking in these high prescription volume sorts of conditions, like heart disease or diabetes. So, I think that while this is very exciting, this is definitely a sort of technology, and a space for artificial intelligence, where it really needs to be expanded, and there’s a lot of room to grow.
So, we can sequence a genome for $1,000. How far away are we from having enough of that data that we get really good insights into, for example, a person has this combination of genetic markers, and therefore this is more likely to work or not work. I know that in isolated cases we can do that, but when will we see that become just kind of how we do things on a day-to-day basis?
I would say, probably, twenty-five years from the clinic. I mean, it’s great, this information is really interesting, and we can do it, but it’s not widely used. I think there are too many regulations in place right now that keep this from happening, so, I think it’s going to be, like I said, maybe twenty-five years before we really see this very widely used for a good number of patients.
So are there initiatives underway that you think merit support that will allow this information to be collected and used in ways that promote the greater good, and simultaneously, protect the privacy of the patients? How can we start collecting better data?
Yeah, there are a lot of people that are working on this type of thing. For example, Obama had a precision medicine initiative and these types of things where you’re really trying to, basically, get your health records and your genomic data, and everything consolidated and have a very easy flow of information so that doctors can easily integrate information from many sources, and have very complete patient profiles. So, this is a thing that’s currently underway.
To pull out a little bit and look at the larger world, you’re obviously deeply involved in speech, and language processing, and health care, and all of these areas where we’ve seen lots of advances happening on a regular basis, and it’s very exciting. But then there’s a lot of concern from people who have two big worries. One is the effect that all of this technology is going to have on employment. And there’s two views.
One is that technology increases productivity, which increases wages, and that’s what’s happened for two hundred years, or, this technology is somehow different, it replaces people and anything a person can do eventually the technology will do better. Which of those camps, or a third camp, do you fall into? What is your prognosis for the future of work?
Right. I think that technology is a good thing. I know a lot of people have concerns, for example, that if there’s too much artificial intelligence it will replace my job, there won’t be room for me and for what I do, but I think that what’s actually going to happen, is we’re just going to see, shall we say, a shifting employment landscape.
Maybe if we have some sort of general intelligence, then people can start worrying, but, right now, what we’re really doing through artificial intelligence is augmenting human intelligence. So, although some jobs become obsolete, now to maintain these systems, build these systems, I believe that you actually have, now, more opportunities there.
For example, ten to fifteen years ago, there wasn’t such a demand for people with software engineering skills, and now it’s almost becoming something that you’re expected to know, or, like, the internet thirty years back. So, I really think that this is going to be a good thing for society. It may be hard for people who don’t have any sort of computer skills, but I think going forward, that these are going to be much more important.
Do you consume science fiction? Do you watch movies, or read books, or television, and if so, are there science fiction universes that you look at and think, “That’s kind of how I see the future unfolding”?
Have you ever seen the TV show Black Mirror?
Well, yeah that’s dystopian though, you were just saying things are going to be good. I thought you were just saying jobs are good, we’re all good, technology is good. Black Mirror is like dark, black, mirrorish.
Yeah, no, I’m not saying that’s what’s going to happen, but I think that’s presenting the evil side of what can happen. I don’t think that’s necessarily realistic, but I think that show actually does a very good job of portraying the way that technology could really be integrated into our lives. Without all of the dystopian, depressing stories, I think that the way that it shows the technology being integrated into people’s lives, how it affects the way people live—I think it does a very good job of doing things like that.
I wonder though, science fiction movies and TV are notoriously dystopian, because there’s more drama in that than utopian. So, it’s not conspiratorial or anything, I’m not asserting that, but I do think that what it does, perhaps, is causes people—somebody termed it “generalizing from fictional evidence,” that you see enough views of the future like that, you think, “Oh, that’s how it’s going to happen.” And then that therefore becomes self-fulfilling.
Frank Herbert, I think, it was who said, “Sometimes the purpose of science fiction is to keep a world from happening.” So do you think those kinds of views of the world are good, or do you think that they increase this collective worry about technology and losing our humanity, becoming a world that’s blackish and mirrorish, you know?
Right. No, I understand your point and actually, I agree. I think there is a lot of fear, which is quite unwarranted. There is actually a lot more transparency in AI now, so I think that a lot of those fears are just, well, given the media today, as I’m sure we’re all aware, it’s a lot of fear mongering. I think that these fears are really something that—not to say there will be no negative impact—but, I think, every cloud has its silver lining. I think that this is not something that anyone really needs to be worrying about. One thing that I think is really important is to have more education for a general audience, because I think part of the fear comes from not really understanding what AI is, what it does, how it works.
Right, and so, I was just kind of thinking through what you were saying, there’s an initiative in Europe that, AI engines—kind of like the one you’re talking about that’s suggesting things—need to be transparent, in the sense they need to be able to explain why they’re making that suggestion.
But, I read one of your papers on deep neural nets, and it talks about how the results are hard to understand, if not impossible to understand. Which side of that do you come down on? Should we limit the technology to things that can be explained in bulleted points, or do we say, “No, the data is the data and we’re never going to understand it once it starts combining in these ways, and we just need to be okay with that”?
Right, so, one of the most overused phrases in all of AI is that “neural networks are a black box.” I’m sure we’re all sick of hearing that sentence, but it’s kind of true. I think that’s why I was interested in researching this topic. I think, as you were saying before, the why in AI is very, very important.
So, I think, of course we can benefit from AI without knowing. We can continue to use it like a black box, it’ll still be useful, it’ll still be important. But I think it will be far more impactful if you are able to explain why, and to really demystify what’s happening.
One good example from my own company is that in medicine it’s vital for the doctor to know why you’re saying what you’re saying, at Droice. So, if a patient comes in and you say, “I think this person is going to have a very negative reaction to this medicine,” it’s very vital for us to try to analyze the neural network and explain, “Okay, it’s really this feature of this person’s health record, for example, the fact that they’re quite old and on another medication.” That really makes them trust the system, and really eases the adoption, and allows them to integrate into traditionally less technologically focused fields.
So, I think that there’s a lot of research now that’s going into the why in AI, and it’s one of my focuses of research, and I know the field has really been blooming in the last couple of years, because I think people are realizing that this is extremely important and will help us not only make artificial intelligence more translational, but also help us to make better models.
You know, in The Empire Strikes Back, when Luke is training on Dagobah with Yoda, he asked him, “Why, why…” and Yoda was like, “There is no why.” Do you think there are situations where there is no why? There is no explainable reason why it chose what it did?
Well, I think there is always a reason. For example, you like ice cream; well, maybe it’s a silly reason, but the reason is that it tastes good. It might not be, you know, you like pistachio better than caramel flavor—so, let’s just say the reason may not be logical, but there is a reason, right? It’s because it activates the pleasure center in your brain when you eat it. So, I think that if you’re looking for interpretability, in some cases it could be limited but I think there’s always something that you could answer when asking why.
Alright. Well, this has been fascinating. If people want to follow you, keep up with what you’re doing, keep up with Droice, can you just run through the litany of ways to do that?
Yeah, so we have a Twitter account, it’s “DroiceLabs,” and that’s mostly where we post. And we also have a website: www.droicelabs.com, and that’s where we post most of the updates that we have.
Alright. Well, it has been a wonderful and far ranging hour, and I just want to thank you so much for being on the show.
Thank you so much for having me.

Byron explores issues around artificial intelligence and conscious computers in his upcoming book The Fourth Age, to be published in April by Atria, an imprint of Simon & Schuster. Pre-order a copy here.
[voices_in_ai_link_back]

Voices in AI – Episode 30: A Conversation with Robert Mittendorff and Mudit Garg

[voices_in_ai_byline]
In this episode, Byron, Robert and Mudit talk about Qventus, healthcare, machine learning, AGI, consciousness, and medical AI.
[podcast_player name=”Episode 30 – A Conversation with Robert Mittendorff and Mudit Garg” artist=”Byron Reese” album=”Voices in AI” url=”https://voicesinai.s3.amazonaws.com/2018-01-22-(00-58-58)-garg-and-mittendorf.mp3″ cover_art_url=”https://voicesinai.com/wp-content/uploads/2018/01/voices-headshot-card-3.jpg”]
[voices_in_ai_byline]
Byron Reese: This is Voices in AI, brought to you by Gigaom. I’m Byron Reese. Today is a first for Voices in AI, we have two guests. The first one is from Qventus; his name is Mudit Garg. He’s here with Robert Mittendorff, who’s with Norwest Venture Partners, who also serves on Qventus’ board. Mudit Garg is the co-founder and CEO of Qventus, and they are a company that offers artificial-intelligence-based software designed to simplify hospital operations. He’s founded multiple technology companies before Qventus, including Hive, a group messaging platform. He spent two years as a consultant with Seattle-based McKinsey & Company, focusing, I think, on hospital operations.
Robert, from Norwest Ventures, before he was VP of Marketing and Business Development at Hansen Medical, a publicly traded NASDAQ company. He’s also a board-certified emergency physician who completed his residency training at Stanford. He received his MD from Harvard Medical School, his MBA from Harvard Business School, and he has a BS in Biomedical Engineering from Johns Hopkins University. Welcome to the show, gentlemen.
Mudit Gard: Thank you. Good morning. Thank you for having us.
Robert Mittendorff: Thank you, Byron.
Mudit, I’ll start with you. Tell us about Qventus and its mission. Get us all oriented with why we’re here today.
Mudit: Absolutely. The best way to think of Qventus, our customers often describe us like air traffic control. Much like what air traffic control does for airports, where it allows many flights to land, much more than if they were uncoordinated, and much more safely than if they were uncoordinated. We do the same for healthcare and hospitals.
For me—as, kind of, boring and uncool as a world of operations and processes might be—I had a chance to see that firsthand working in hospitals when I was at McKinsey & Company, and really just felt that we were letting all of our clinicians down. If you think about the US healthcare system, we have the best clinicians in the world, we have great therapies, great equipment, but we fail at providing great medicine. Much of that was being held back by the complex operations that surround the delivery of care.
I got really excited about using data and using AI to help support these frontline clinicians in improving the core delivery of care in the operation. Things like, as a patient sitting in an emergency department, you might wonder what’s going on and why you aren’t being taken care of faster. On the flip side, there’s a set of clinicians who are putting in heroic efforts trying to do that, but they are managing so many different variables and processes simultaneously that it’s almost humanly impossible to do that.
So, our system observes and anticipates problems like, it’s the Monday after Thanksgiving, it’s really cold outside, Dr. Smith is working, he tends to order more labs, our labs are slow—all these factors that would be hard for someone to keep in front of them all the time. When it realizes we might run out of capacity, three or four hours in advance, they will look and find the bottleneck, and create a discussion on how to fix that. We do things like that at about forty to fifty hospitals across the country, and have seen good outcomes through that. That’s what we do, and that’s been my focus in the application of AI.
And Robert how did you get involved with Qventus?
Robert: Well, so Qventus was a company that fit within a theme that we had been looking at for quite some time in artificial intelligence and machine learning, as it applies to healthcare. And within that search we found this amazing company that was founded by a brilliant team of engineers/business leaders who had a particular set of insights from their work with hospitals, at McKinsey, and it identified a problem set that was very tractable for machine learning and narrow AI which we’ll get into. So, within that context in the Bay Area, we found Qventus and we’re just delighted to meet the team and their customers, and really find a way to make a bet in this space.
We’re always interested in case studies. We’re really interested in how people are applying artificial intelligence. Today, in the here and now, put a little flesh on the bones of what are you doing, what’s real and here, how did you build it, what technology you are using, what did you learn? Just give us a little bit of that kind of perspective.
Mudit: Absolutely. I’ll first start with the kinds of things that we are doing, and then we’ll go into how did we build it, and some of the lessons along the way as well. I just gave you one example of running an emergency department. In today’s world, there is a charge nurse that is responsible for managing the flow of patients through that emergency department, constantly trying to stay ahead of it. The example I gave was where, instead the systems are observing it, realizing, learning from it, and then creating a discussion among folks about how to change it.
We have many different things—we call them recipes internally—many different recipes that the system keeps looking for. It looks for, “Hey, here’s a female who is younger, who is waiting and there are four other people waiting around her, and is an acute pain.” She is likely to get up and leave without being seen by a doctor much more than other folks, and you might nudge and greet her, to go up and talk to them. We have many recipes and examples like these, I won’t go into specific examples in each of those, but we do that in different areas of delivery of healthcare.
So, patient flow, just having patients go through the health systems in ways that don’t require them to add resources, but allow them to provide the same care is one big category. You do that in the emergency department, in unison to the hospital and in the operating room. More recently, starting to do that in pharmacy operations, pharmacy costs have started rising. What are the things that today require a human to manually realize, follow up on, escalate and manage, and how can we help the AIs with that process? We’ve seen really good results with that.
I think you’re asking about case studies, in the emergency department side alone, one of our customers treated three thousand more patients in that ED this year than last, without adding resources. They saved almost a million minutes of patient wait time in that single ED alone and that’s been fascinating. What’s been even more amazing is hearing from the nurse manager there how the staff feel like they have the ability to shape the events versus always being behind, and always feeling like they are trying to solve the problem after the fact. They’ve seen some reductions in turnover and that ability of using AI to, in some ways, making health care more human for the people who help us, the caregivers, is what’s extremely exciting in this work for me.
Just to visualize that for a moment, if I looked at it from thirty thousand feet—people come into a hospital, all different ways, and they have all different characteristics of all the things you would normally think, and then there’s a number of routings through the hospital experience, right? Rush them straight into here, or there, or this, so it’s kind of a routing problem. It’s a resource allocation problem, right? What does all of that look like? This is not a rhetorical question, what is all that similar to outside of the hospital? Where is that approach broadly and generally applicable to? It’s not a traffic routing problem, it’s not an inventory management problem, are there any corollaries you can think of?
Mudit: Yeah. In many ways there are similarities to anywhere where there are high fixed asset businesses and there’s a distributed workforce, there’s lots of similarities. I mean, logistics is a good example of it. Thinking about how different deliveries are routed and how they are organized in a way that you meet the SLAs for different folks, but your cost of delivery is not too high. It has similarities to it.
I think hospitals are, in many ways, one of the most complex businesses, and given the variability is much, much higher, traditional methods have failed. In many of the other such logistical and management problems you could use your optimization techniques, and you could do fairly well with them. But given the level of variability is much, much higher in healthcare—because the patients that walk in are different, you might have a ton walk in one day and very few walk in the next, the types of resources they need can vary quite a bit—that makes the traditional methods alone much, much harder to apply. In many ways, the problems are similar, right? How do you place the most product in a warehouse to make sure that deliveries are happening as fast as possible? How do you make sure you route flights and cancel flights in a way that causes minimum disruption but still maximize the benefit of the entirety of the system? How do you manage the delivery of packages across a busy holiday season? Those problems have very similar elements to them and the importance of doing those well is probably similar in some ways, but the techniques needed are different.
Robert, I want to get to you in just a minute, and talk about how you as a physician see this, but I have a couple more technical questions. There’s an emergency room near my house that has a big billboard and it has on there the number of minutes of wait time to get into the ER. And I don’t know, I’ve always wondered is the idea that people drive by and think, “Oh, only a four-minute wait, I’ll go to the ER.” But, in any case, two questions, one, you said that there’s somebody who’s in acute pain and they’ve got four people, and they might get up and leave, and we should send a greeter over… In that example, how is that data acquired about that person? Is that done with cameras, or is that a human entering the information—how is data acquisition happening? And then, second, what was your training set to use AI on this process, how did you get an initial training set?
Mudit: Both great questions. Much of this is part of the first-mile problem for AI in healthcare, that much of that data is actually already generated. About six or seven years ago a mass wave of digitization started in healthcare, and most of the digitization was taking existing paper-based processes and having them run through electronic medical record systems.
So, what happens is when you walk into the emergency department, let’s say, Byron, you walk in, someone would say, “Okay, what’s your name? What are you here for?” They type your name in, and a timestamp is stored alongside that, and we can use that timestamp to realize a person’s walked in. We know that they walked in for this reason. When you got assigned a room or assigned a doctor then I can, again, get a sense of, okay, at this time they got assigned a room, at this time they got assigned a doctor, at this time their blood was drawn. All of that is getting stored in existing systems of record already, and we take the data from the systems of record, learn historically—so before we start we are able to learn historically—and then in the moment, we’re able to intervene when a change needs to take place.
And then the data acquisition part of the acute patient’s pain?
Mudit: The pain in that example is actually coming from the kind of what they have complained about.
I see, perfect.
Mudit: So, we’re looking the types of patients who complain about similar pieces, what’s their likelihood versus this likelihood, that’s what we will be learning on it.
Robert, I have to ask you before we dive into this, I’m just really intensely curious about your personal journey, because I’m guessing you began planning to be a medical practitioner, and then somewhere along the way you decided to get an MBA, and then somewhere along the way you decided to invest in technology companies and be on their boards. How did all of that happen? What was your progressive realization that took you from place to place to place?
Robert: I’ll spend just a couple of minutes on it, but not exactly. I would say in my heart I am an engineer. I started out as an engineer. I did biomedical electrical engineering and then I spent time at MIT when I was a medical student. I was in a very technical program between Harvard and MIT as a medical student. In my heart, I’m an engineer which means I try to reduce reality to systems of practice and methods. And coupled with that is my interest in mission-driven organizations that also make money, so that’s where healthcare and engineering intersect.
Not to go into too much detail on a podcast about myself, I think the next step in my career was to try to figure out how I could deeply understand the needs of healthcare, so that I could help others and myself bring to bear technology to solve and address those needs. The choice to become a practitioner was partially because I do enjoy solving problems in the emergency department, but also because it gave me a broad understanding of opportunities in healthcare at the ground level and above in this way.
I’ll just give you an example, when I first saw what Mudit and his team had done in the most amazing way at Qventus, I really understood the hospital as an airport with fifty percent of the planes landing on schedule. So, to go back to your emergency department example, imagine if you were responsible for safety and efficiency at SFO, San Francisco airport, without a tower and knowing only the schedule landing times for half of the jets, where each jet is patient. Of the volume of patients that spend their night in the hospital, about half come to the ED, and when I show up for a shift that first, second, and third patient can be stroke, heart attack, broken leg, can be shortness of breath, skin rash, etcetera. The level of complexity in health care to operationalize improvements in the way that Mudit has is incredibly high. We’re just at the beginning, they are clearly the leader here, but what I saw in my personal journey in this company is the usage of significant technology to address key throughput needs in healthcare.
When one stack-ranks what we hope artificial intelligence does for the world, on most people’s list, right up there at the very top is impact health. Do you think that’s overly hyped because there’s all kinds of, you know, we have an unending series of wishes that we hope artificial intelligence can do? Do you think it’s possible that it delivers eventually on all of that, that it really is a transformative technology that materially alters human health at a global level?
Robert: Absolutely and wholeheartedly. My background as a researcher in neuroscience was using neural networks to model brain function in various animal models, and I would tell you that the variety of ways that machine learning and AI, which are the terms we use now for these technologies, the variety of ways they will affect human health are massive. I would say within the Gartner hype cycle we are early, we are overhyping in the short term the value of this technology. We are not overhyping the value of this technology in the next ten, twenty, or thirty years. I believe that AI is the driver of our Industrial Revolution. This will be looked back at as an industrial revolution of sorts. I think there’s a huge benefit that are going to be accrued to healthcare providers and patients to the usage of these technologies.
Talk about that a little more, paint a picture of the world in thirty years, assuming all goes well. Assuming all goes well, what would our health experience look like in that world?
Robert: Yeah, well, hopefully your health experience, and I think Mudit’s done a great job describing this, will return to a human experience between a patient and a physician, or provider. I think in the backroom, or when you’re at home interacting with that practice, I think you’re going to see a lot more AI.
Let me give you one example. We have a company that went public, a digital health company, that uses machine learning to read EKG data, so cardiac electrical activity data. A typical human would take eight hours to read a single study on a patient, but by using machine learning they get down to five to tens of minutes. The human is still there, overreading what the machine learned software is producing—this company is called iRhythm—and what that allows us to do is reach a lot more patients at a lower cost than you could achieve with human labor. You’ll see this in radiology. You’ll see this in coaching patients. You’ll see this in where I think Mudit has really innovated, which is he has created a platform that is enabling.
In the case that I gave you with humans being augmented by, what I call, the automation or semi-automation of a human task, that’s one thing, but what Mudit is doing is truly enabling AI. Humans cannot do what he does in the time and scale that he does it. That is what’s really exciting—machines that can do things that humans cannot do. Just to visualize that system, there are some things that are not easily understood today, but I think you will see radiology improve with semi-automation. I think patients will be coached with smart AI to improve their well-being, and that’s already being seen today. Human providers will have leverage because the computer, the machine will help prioritize their day, which patient talk to about, what, when, how, why. So, I think you’ll see a more human experience.
That’s the concern is that we will see a more manufactured experience. I don’t think that’s the case at all. The design that we’ll probably see succeed is one where the human will become front and center again, where physicians will no longer be looking at screens typing in data, they’ll be communicating face to face with a human, with an AI helping out, advising, enabling those tedious tasks that the human shouldn’t be burdened with, to allow the relationship between the patient and physician to return.
So, Mudit, when you think of artificial intelligence and applying artificial intelligence to this particular problem, where do you go from that? Is the plan to take that learning—and, obviously, scale it out to more hospitals—but what is the next level to add depth to it to be able to say, “Okay, we can land all the planes now safely, now we want to refuel them faster, or…”? I don’t know, the analogy breaks down at some point. Where would you go from here?
Mudit: We already as customers are starting to see results of this approach in one area. We’ve started expanding already and have a lot more expansion coming down the line as well. If you think of it, at the end of the day, so much of healthcare delivery is heavily process driven, right? Anywhere from how your bills get generated to when you get calls. I’ve had times when I might get a call from a health system saying I have a ten-dollar bill that they are about to send to collection but I paid all the bills today. There are things like that that are constantly happening that are breakdowns in processes, across delivery, across the board.
We started, as I said, four or five years ago and very specifically focused on the emergency department. Going from there into the surgery area, where operating rooms can cost upwards of hundreds of dollars a minute, so how do you manage that complex an operation, and the logistics setting to deliver the best value? And I’ve seen really good results there, managing the entirety of all the units in the hospital. More recently, as I was saying, we are now starting to work with Sutter Health across twenty-six of their hospital pharmacies, in looking at what are the key pieces around operations in the pharmacy which are, again, manually holding people back from delivering the best care. These are the different pieces across the board that we are already starting to see.
The common thread across all of these I find is that we have amazing, incredible clinicians today, that, if they had all the time and energy in the world to focus on anticipating these problems and delivering the best care, they would do a great job, but we cannot afford to keep having more people solve these problems. There are significant margin pressures across healthcare. The same people who were able to do these things before have to-do lists that are growing faster than they can ever comprehend. The job of AI really is to act as, kind of, their assistant and watch those decisions on their behalf, and help make those really, really easy. To take all of the boring, mundane logistics out of their hands, so they can focus on what they can do best which is deliver care to their patients. So, right now, as I said, we started on the flow side, pharmacies are a new area, outpatient clinics, and imaging centers is another area that we are working with a few select customers on and there’s some really, really exciting stuff there in increasing the access to care—when you might call a physician to get access—while reducing the burden on that physician, that we are working on.
Another really exciting piece for me is, in many ways the US healthcare system is unique, but in this complexity of logistics and operation it is not. So, we are already signed to work with hospitals globally, just started with working with our first international customer recently, and the same problems exist everywhere. There was an article in BBC, I think a week or two weeks ago, where there’s a long surgery waiting lists in the UK, and they are struggling to get those patients seen in that system, due to lack of efficiency in these logistics. So, that’s the other piece that I’m really excited about, it’s not only the breadth of these problems where there’s complexity of processes, but also the global applicability of it.
The exciting thing to me about this episode of Voices is that I have two people who are engineers, who understand AI, and who have a deep knowledge of health. I just have several questions that kind of sit at the intersection of all of that I would love to throw at you.
My first one is this, the human genome is, however many billions of base pairs that works out to something like 762MB of data, but if you look at what makes us different than, say, chimps, it may be one percent of that. So, it’s something like 7MB or 8MB of data is the code you need to build an intelligent brain, a person. Does that imply to you that artificial intelligence might have a breakthrough, there might be a relatively straightforward and simple thing about intelligence that we’re going to learn, that will supercharge it? Or, is your view that, no, unfortunately, something like a general intelligence is going to be, you know, hunks of spaghetti code that kind of work together and pull off this AGI thing. Mudit, I’ll ask you first.
Mudit: Yeah, and boy that’s a tough question. I will do my best in answering that one. Do I believe that we’ll be able to get a general-purpose AI, with, like, 7MB or 8MB of code? There’s a part of me that does believe in that simplicity, and does want to believe in that the answer. If you look at a lot of the machine learning code, it’s not the code itself that’s actually that complex, it’s the first mile and the last mile of that code that ends up taking the vast majority of the code. How to get the training sets in and how do you get the output out—that is what takes the majority of the AI code today.
The fundamental learning code isn’t that big today. I don’t know if we’ll solve general purpose AI anytime soon. I’m certainly not holding my breath for that, but there’s a part of me that feels and hopes that the fundamental concepts of the learning and the intelligence, will not be that complicated at an individual micro scale. Much like ourselves, we’ll be able to understand them, and there will be some beauty and harmony and symphony in how they all come together. And that actually won’t be complex in hindsight, but it will be extremely complex to figure out the first time around. That’s purely speculative but that would be might be my belief and my hunch right now.
Robert, do you want to add anything to that, or let that answer stand?
Robert: I’d be happy to. I think it’s an interesting analogy to make. There are some parts of it that will break down and parts that will parallel between the human genomes complexity, and utility, and the human brain. You know, just I think when we think about the genome you’re right, it’s several billion base pairs where we only have twenty thousand genes, and a small minority percentage that actually code for protein, and a minority of those that we understand affect the human in a diseased way, like a thousand genes to two thousand genes. There’s a lot of base pairs that we don’t understand and could be related to structure of the genome as it needs to do what it does in the human body, in the cell.
On the brain side, though, I think I would go with your latter response which is if you look at the human brain—and I’ve had the privilege of working with animal models and looking at human data—the brain is segmented into various functional units. For example, the auditory cortex is responsible for taking information from the ear and converting it to signals that then are pattern-recognized in to, say, language, and where those symbols of what words we’re speaking are then processed by other parts of the cortex. Similarly, the hippocampus, which sits in, kind of, the oldest part of the brain, is responsible for learning. It is able to look at various inputs from all of these, from the visual and auditory and other courtesies, and then upload them to long-term memory from short-term memory, so that the brain is functionally segmented and physically segmented.
I believe that a general-purpose AI will have the same kind of structure. It’s funny we have this thing called the AI effect where when we solve a problem with code or with machinery, it’s no longer AI. So, for example, natural language processing, some would consider now not part of AI because we’ve somewhat solved it, or speech recognition used to be AI, but now it’s an input to the AI, because the AI is thinking about more understanding than interpretation of audio signals and converting them into words. I would say what we’re going to see, which is similar to the human body encoded by these twenty thousand genes, is you will have functional expertise with, presumably, code that is used for segmenting the problem of creating a general AI.
A second question then. You, Robert, waxed earlier about how big the possibilities are for using artificial intelligence with health. Of course, we know that the number of people who are living to one hundred keeps going up, up, up. The number of people who become supercentenarians is in the dozens, who’ve gotten to one hundred and ten. The number of people who have lived to one hundred and twenty-five is stubbornly fixed at zero. Do you believe—and not even getting aspirational about “curing death”—that what’s most likely to happen is more of us are going to make it to one hundred healthily, or do you think that one hundred and twenty-five is something we’ll break and maybe somebody will live to one hundred and fifty. What do you think about that?
Robert: That’s a really hard question. I would say that if I look at the trajectory of gains that, public health, primarily, with things like treated water to medicine, we’ve seen a dramatic increase in human longevity in the developed world. From taking down the number of children dying during childbirth, which lowers the average obviously, to extending life in the later years, and if you look at the effects there those conclusions have never effects on society. For example, when Social Security was invented a minority of individuals would live to the age in which they would start accruing significant benefits, obviously that’s no longer the case.
So, to answer your question, there is no theoretical reason that I can come up with that I can’t imagine someone making it to one hundred and twenty-five. One hundred and fifty is obviously harder to imagine. But we understand the human cell at a certain level, and the genome, and the machinery of the human body, and we’ve been able to thwart the body’s effort to fatigue and expire, a number of times now. Whether it’s cardiovascular disease or cancer, and we’ve studied longevity—“we” meaning the field, not myself—so, I don’t see any reason why we would say we will not have individuals reach one hundred and twenty-five, or even one hundred and fifty.
Now, what is the time course of that? Do we want that to happen and what are the implications for society? Those are big questions to answer. But science will continue to push the limits of understanding human function at the cellular and the physiologic level to extend the human life. And I don’t see a limit to that currently.
So, there is this worm, called the nematode worm, little bitty fella, he’s as long as a hair is wide, the most successful animal on the planet. Something like seventy percent of all animals are nematode worms. The brain of the nematode worm has 302 neurons, and for twenty years or so, people have been trying to model those 302 neurons in a computer, the OpenWorm project. And even today they don’t know if they can do it. That’s how little we understand. We don’t not understand the human brain because it’s so complex, we don’t understand anything—or I don’t want to say anything—we don’t understand just how neurons themselves work.  
Do you think that, one, we need to understand how our brains work—or how the nematode brain works for that matter—to make strides towards an AGI? And, second, is it possible that a neuron has stuff going on at the Planck level that it’s as complicated as a supercomputer, making intelligence acquired that way incredibly difficult? Do either of you want to comment on that?
Mudit: It’s funny that you mention that, when I was at Stanford doing some work in the engineering, one of the professors used to say that our study of the human brain is sort of like someone just had a supercomputer and two electrodes and they’re poking the electrodes in different places and trying to figure out how it works. And I can’t imagine ever figuring out how a computer works outside-in by just having like two electrodes and seeing the different voltages coming out of it. So, I do see the complexity of it.
Is it necessary for us to understand how the neuron works? I’m not sure it’s necessary for us to understand how the neuron works, but if you were to come up with a way where we can build a system that’s, both resilient, redundant, and simple, that can do that level of intelligence, I think that’s hundreds of thousands of years of evolution that have helped us get to that solution, so it would, I think, be a critical input.
Without that, I see a different approach, which is what we are taking today, which is inspired, likely, but it’s not the same. In our brain when neurons fire, yes, we now have a similar transfer function for many of our neural networks of how the neuron fires, but for any kind of meaningful signal to come out we have a population of neurons firing in our brain that makes the impulsing more continuous and very redundant and very resilient. It wouldn’t fail even if some portion of those neurons stopped working. But that’s not how our models work, that’s not how our math works today. I think in finding the most optimized, probably, elegant and resilient way of doing it, I think it would be remiss not to take inspiration from what has been evolved over a long, long period of time, to, perhaps, being one of the most efficient ways of having general purpose AI. So, at least my belief would be we will have to learn, and I would think that our understanding is still largely simplistic and, at least, I would hope and believe that we’ll learn a lot more and find out that, yeah, each one of those perhaps either communicates more, or does it in a way that brings the system to the optimal solution a lot faster than we would imagine.
Robert: Just to add to that I would say, I agree with everything Mudit said, I would say do we need to study the neuron and neural networks in vivo, in animals? And the answer to that is, as humans, we do. I mean, I believe that we have an innate curiosity to understand ourselves and that we need to do. Whether it’s funded or not, the curiosity to understand who we are, where we came from, how we work, will drive that just like it’s driven fields as diverse as astronomy to aviation.
I think, do we need to understand at the level of detail you’re describing, for example, what exactly happens at the synapse stochastically, where neurotransmitters find the receptors that open ion channels that change the resting potential of a neuron, such that additional axonal effects occur where at the end of that neuron you then release another neurotransmitter. I don’t think so. Because I think we learn a lot, as Mudit said, from understanding how these highly developed and trained systems we call, animals and humans, work, but they were molded over large periods of time for specific survival tasks, to live in the environment that they live in.
The systems we’re building, or Mudit’s building, and others, are designed for other uses, and so we can take, as he said, inspiration from them, but we don’t need to model how a nematode thinks to help the hospital work more effectively. In the same way that, there are two ways, for example, someone could fly from here in San Francisco, where I’m sitting, to, let’s say, Los Angeles. You could be a bird, which is a highly evolved flying creature which has sensors, which has, clearly, neural networks that are able to control wing movement, and effectively the wing surface area to create lift, etcetera. Or, you could build a metal tube with jets on it that gets you there as well. I think they have different use cases and different criteria.
The airplane is inspired by birds. The wing of an airplane, the cross-section of the wing is designed like a bird’s wing is in that the one pathway is longer than the other which changes pressure above and below the wing that allows flight to occur. But clearly, the rest of it is very different. And so, I think the inspiration drove aviation to a solution that has many parts from what birds have, but it’s incredibly different because the solution was to the problem of transporting humans.
Mudit, earlier you said we’re not going to have an AGI anytime soon. I have two questions to follow up on that thought. The first is that among people who are in the tech space there’s a range of something like five to five hundred years as to when we might get a general intelligence. I’m curious, one, why do you think there’s such a range? And, two, I’m curious, with both of you, if you were going to throw a dart at that dartboard, where would you place your bet, to mix a metaphor.
Mudit: I think in the dart metaphor, chances of being right are pretty low, but we’ll give it a shot. I think part of it, at least I ask myself, is the bar we hold for AGI too high? At what point do we start feeling that a collection of special-purpose AIs that are welded together can start feeling like an AGI and is that good enough? I don’t know the answer to that question and I think that’s part of what makes the answer harder. Similar to what Robert was saying where the more problems we solve, the more we see them as algorithmic and less as AI.
But I do think at some point, at least in my mind, if I can see an AI starting to question the constraints of the problem and the goal it’s trying to maximize, that’s where true creativity for humans comes from; when we break rules and when we don’t follow the rules we were given. And that’s also the scary part of AI comes from because it can do that at scale then. I don’t see us close to that today. And if I had to guess I’m going to just say, on this exponential curve, I’m going to probably not pick out the right point, but four to five decades is when we start seeing enough of the framework and maybe essentially, we can see some tangible general-purpose AI come to form.
Robert, do you want to weigh in, or you will take a pass on that one?
Robert: I’ll weigh in quickly. I think we often see this in all of investing, actually—whether it’s augmented reality, virtual reality, whether it’s stenting or robotics in medicine—we as investors have to work hard to not overestimate the effect of technology now, and not underestimate the effect of technology in the long run. This came from, I believe a Stanford professor Roy Amara, who unfortunately passed a while ago, but that idea of saying, “Let’s not overhype it, but it’s going to be much more profound than we can even imagine today,” puts my estimate, probably—and it depends how you define general AI which is probably not worth doing—I would say it’s within fifteen to twenty years.
We have this brain, the only general intelligence that we know of. And then we have the mind and, kind of, a definition of that which I think everybody can agree to that the mind as a set of abilities that don’t seem, at first glance, to be something an organ could do, like creativity, or a sense of humor. And then we have consciousness, we actually experience the world. A computer to measure temperature, but we can burn our finger and feel that. My questions are, we would expect the computer to have a “mind,” we would expect an AGI to be creative, do you think, one, that consciousness is required for general intelligence, and, to follow up on that, do you believe computers can become conscious? That they can experience the world as opposed to just measure it?
Mudit: That’s a really hard one too. I think actually in my mind what’s most important, and there’s kind of a grey line between the two, is creativity is what’s most important, the element of surprise is what’s most important. The more an AI can surprise you, the more you feel like it is truly intelligent. So, that creativity is extremely important. But I think the reason I said there’s kind of a path from one to the other is—and this is very philosophical of how to define consciousness—in many ways it’s when we start taking a specific task that is given to us, but really start asking the larger objective, the larger purpose, that’s when, I feel like, that’s what truly distinguishes a being or a person being conscious.
Until the AIs are able to be creative and break the bounds of the specific rules, or the specific expected behavior that it’s programmed to do, certainly the path to consciousness is very, very hard. So, I feel like creativity and surprising us is probably the first piece, which is also the one that honestly scares us as humans the most, because that’s when we feel a sense of losing control over the AI. I don’t think true consciousness is necessary, but they might go hand in hand. I can’t think of it being necessary, but they might evolve simultaneously and they might go hand in hand.
Robert: I would just add one other thought there which is, so I spent many hours in college having this debate of what is consciousness, you know, where is the sea of consciousness? Anatomists for centuries have dissected and dissected it, you know, is it this gland, or is it that place, or is it an organized effect of the structure and function of all of these parts. I think that’s why we need to study the brain, to be fair.
One of the underlying efforts there is to understand consciousness. What is it that makes a physical entity able to do what you said, to experience what you said? More than just experiencing a location, experiencing things like love. How could a human do that if they were a machine? Can a machine of empathy?
But I think beyond that, as I think practically as an investor and as a physician, I frankly, I don’t know if I care if the machine is conscious or not, I care more about who do I assign responsibility to for the actions and thoughts of that entity. So, for example, if they make a decision that harms someone, or if they make the wrong diagnosis, what recourse do I have? Consciousness in human beings, well, we believe in free will, and that’s where all of our entities around human justice come from. But if the machine is deterministic, then a higher power, may be the human that designed it, is ultimately responsible. For me, it’s a big question about responsibility with effect to these AIs, and less about whether they’re conscious or not. If they’re conscious then we might be able to assign responsibility to the machine, but then how do we penalize it—financially, otherwise? If they’re not conscious, then we probably need to assign responsibility to the owner, or the person that configured the machine.
I started the question earlier about why is there such a range of beliefs about when we might get a general intelligence, but the other interesting thing, which you’re kind of touching on, is there’s a wide range of belief about whether we would want one. You’ve got the Elon Musk camp of summoning the demon, Professor Hawking saying it’s an existential threat, and Bill Gates said, “I don’t understand why more people aren’t worried about it,” and so forth. And on the other end, you have people like Andrew Ng who said, “That’s like worrying about overpopulation of Mars,” and Rodney Brooks the roboticist, and so forth, who dismissed those. It’s almost eye-rolling, that you can see. What are the core assumptions that those two groups have, and why are they so different from each other in their regard to this technology?
Mudit: To me it boils down to the same things that make me excited about large-scale potential, from a general-purpose side, are the things that make me scared. You know how we were talking about what creativity is, if I go back to creativity for a second. Creativity will come from if an AI is told to maximize an objective function and the objective function has constraints, should it be allowed to question the constraints and the problem itself? If it is allowed to do that that’s where true creativity would come from, right? That’s what a human would do. I might give someone a task or a problem, but then they might come back and question it, and that’s where true creativity will come from. But the minute we allow an AI to do that is also when we lose that sense of control. We also don’t have that sense of control in humans today, but what freaks us out about AI is that AI can take that and do that at very, very rapid scale, at a pace at which we may not even as a society catch up to, realize, and be able to control or regulate, which we can in case of humans. I think that’s both the exciting part and the fear, they are really hand in hand.
The pace at which AI can then bring about the change once those constraints are loosened is something we haven’t seen before. And we already see, in today’s environment, our inability to keep pace with how fast technology is changing, from a regulation, from a framework standpoint as a society. And I think once that happens that will be called into question even more. I think that’s probably why many in the camp of Elon Musk, Sam Altman, and others, in many ways, I think, the part of their ask that resonates with me is we probably should start thinking about how we will tackle the problem, what framework should we have in place earlier, so we have time as a society to wrestle with it before it comes and it’s right in our face.
Robert: I would add to that with four things. I would say the four areas that I think kind of define us a bit—and there were a couple of them that were mentioned by Mudit—I think it’s speed, so speed of computation of affecting the world in which the machine would be in; scalability; the fact that it can affect the physical environment; and the fact that machines as we currently believe them do not have morals or ethics, I don’t know how you define it. So, there’s four things. Something that’s super fast, that’s highly scaled, that can affect the physical world with no ethics or morality, that is a scary thing, right? That is a truck on 101 with a robotic driver that is going to go 100 MPH and doesn’t care what it hits. That’s the scary part of it. But there’s a lot of technology that looks like that. If you are able to design it properly and constrain it, it can be incredibly powerful. It’s just that the conflict in those four areas could be very detrimental to us.
So, to pull the conversation back closer to the here and now, I want to ask each of you what’s a breakthrough in artificial intelligence in the medical profession that we may not have heard about, because there are so many of them? And then tell me something—I’ll put both of you on the spot on this—you think we’re going to see in, like, two or three years; something that’s on a time horizon where we can be very confident we’re going to go see that. Mudit, why don’t you start, what is something we may not know about, and what is something that will happen pretty soon do you think, in AI and medicine?
Mudit: I think—and this might go back to what I was saying—the breakthrough is less in the machine learning itself, but the operationalization of it. The ability—if we have the first mile and the last mile solved—to learn exists, but in the real, complex world of high emotions, messy human-generated data, the ability to actually, not only predict, but, in the moment, prescribe and persuade people to take action, is what I’m most excited about and I’m starting to see happen today, that I think is going to be transformative in the ability of existing machine learning prowess to actually impact our health and our healthcare system. So, that’s the part that I’m most excited about. It may not be, Byron, exactly what you’re looking for in terms of what breakthrough, but I think it’s a breakthrough of a different type. It’s not an algorithmic breakthrough, but it’s an operationalization breakthrough which I’m super excited about.
The part you asked about, what do I think in two to three years we could start doing, that we perhaps don’t do as well now… I know one that is very clear is places where there’s high degrees of structured data that we require humans to pore through—and I know Robert spent a lot of time on this, so I’ll leave this one to him—around radiology, around EKG data, around these huge quantities of structured data that are just impossible to monitor. But the number of poor quality outcomes, mortality, and bad events like that that happen which, if it was humanly feasible to monitor all that and realize, I believe we are two to three years away from starting to meaningfully bend that, both kind of process-wise, logistically, and then from a diagnosis standpoint. And it will be basic stuff, it will be stuff that we have known for a long time that we should do. But, you know, as the classic saying goes, it takes seventeen years from knowing something should be done, to doing it at scale in healthcare; I think it will be that kind of stuff where it will start rapidly shortening and reducing that cycle time and seeing vast effects of that in a healthcare system.
Robert: I’ll give you my two, briefly. I think it’s hard to come up with something that you may not have heard about, Byron, with your background, so I’ll think more about the general audience. First of all, I agree with Mudit, I think the two to three year time frame what’s obvious is that any signal processing in healthcare that is being done by human is going to be rapidly moved to a computer. So, iRhythm as an example of a company trading over a billion in a little over a year out of its IPO does that for cardiology data, EKG data, acquired through a patch. There are over forty companies that we have tracked in the radiology space that are prereading, or in some sense providing a pre-diagnostic read of CTs, MRIs, x-rays, for human radiology overreads for diagnosis. That is happening in the next two to five years. That is absolutely going to happen in the next two to five years. Companies like GE and Philips are leading it, there are lots of startups doing work there.
I think the area that might not be so available to the general public is the usage of machine learning on human conversation. Imagine in therapy, for example, therapy is moving to teletherapy, telemedicine; those are digitized conversations, they can be recorded and translated into language symbols, which can then be evaluated. Computational technology is being developed and is available today that can look at those conversations to decipher whether, for example, someone is anxious today, or depressed, needs more attention, may need a cognitive behavioral therapy intervention that is compatible with their state. And that allows, not only the scaling of signal processing, but the scaling of human labor that is providing psychological therapy to these patients. And so, I think, where we start looking at conversations, this is already being done in the management of sales forces with companies using AI to monitor sales calls and coach sales reps as to how to position things in those calls, to more effectively increase the conversion of a sale, we’re seeing that in healthcare as well.
All right, well that is all very promising, that’s all like kind of lifts up our day to know that there’s stuff coming and it’s going to be here relatively soon. I think that’s probably a good place to leave it. As I look at our timer, we are out of time, but I want to thank both of you for taking the time out of, I’m sure, your very busy days, to have this conversation with us and let us in on a little bit of what you’re thinking, what you’re working on, so thank you.
Mudit: Thank you very much, thanks, Byron.
Robert: You’re welcome.
Byron explores issues around artificial intelligence and conscious computers in his upcoming book The Fourth Age, to be published in April by Atria, an imprint of Simon & Schuster. Pre-order a copy here.
[voices_in_ai_link_back]

Voices in AI – Episode 29: A Conversation with Hugo LaRochelle

[voices_in_ai_byline]
In this episode, Byron and Hugo discuss consciousness, machine learning and more.
[podcast_player name=”Episode 29 – A Conversation with Hugo LaRochelle” artist=”Byron Reese” album=”Voices in AI” url=”https://voicesinai.s3.amazonaws.com/2018-01-15-(00-49-50)-hugo-larochelle.mp3″ cover_art_url=”https://voicesinai.com/wp-content/uploads/2018/01/voices-headshot-card-2.jpg”]
[voices_in_ai_byline]
Byron Reese: This is Voices in AI, brought to you by Gigaom. I’m Byron Reese. Today I’m excited; our guest is Hugo Larochelle. He is a research scientist over at Google Brain. That would be enough to say about him to start with, but there’s a whole lot more we can go into. He’s an Associate Professor, on leave presently. He’s an expert on machine learning, and he specializes in deep neural networks in the areas of computer vision and natural language processing. Welcome to the show, Hugo.
Hugo Larochelle: Hi. Thanks for having me.
I’m going to ask you only one, kind of, lead-in question, and then let’s dive in. Would you give people a quick overview, a hierarchical explanation of the various terms that I just used in there? In terms of, what is “machine learning,” and then what are “neural nets” specifically as a subset of that? And what is “deep learning” in relation to that? Can you put all of that into perspective for the listener?
Sure, let me try that. Machine learning is the field in computer science, and in AI, where we are interested in designing algorithms or procedures that allow machines to learn. And this is motivated by the fact that we would like machines to be able to accumulate knowledge in an automatic way, as opposed to another approach which is to just hand-code knowledge into a machine. That’s machine learning, and there are a variety of different approaches for allowing for a machine to learn about the world, to learn about achieving certain tasks.
Within machine learning, there is one approach that is based on artificial neural networks. That approach is more closely inspired from our brains, from real neural networks and real neurons. It is still somewhat vaguely inspired by—in the sense that many of these algorithms probably aren’t close to what real biological neurons are doing—but some of the inspiration for it, I guess, is a lot of people in machine learning, and specifically in deep learning, have this perspective that the brain is really a biological machine. That it is executing some algorithm, and would like to discover what this algorithm is. And so, we try to take inspiration from the way the brain functions in designing our own artificial neural networks, but also take into account how machines work and how they’re different from biological neurons.
There’s the fundamental unit of computation in artificial neural networks, which is this artificial neuron. You can think of it, for instance, that we have neurons that are connected to our retina. And so, on a machine, we’d have a neuron that would be connected to, and take as input, the pixel values of some image on a computer. And in artificial neural networks, for the longest of time, we would have such neural networks with mostly a single layer of these neurons—so multiple neurons trying to detect different patterns in, say, images—and that was the most sophisticated type of artificial neural networks that we could really train with success, say ten years ago or more, with some exceptions. But in the past ten years or so, there’s been development in designing learning algorithms that leverage so called deep neural networks that have many more of these layers of neurons. Much like, in our brain we have a variety of brain regions that are connected with one another. How the light, say, flows in our visual cortex, it flows from the retina to various regions in the visceral cortex. In the past ten years there’s been a lot of success in designing more and more successful learning algorithms that are based on these artificial neural networks with many layers of artificial neurons. And that’s been something I’ve been doing research on for the past ten years now.
You just touched on something interesting, which is this parallel between biology and human intelligence. The human genome is like 725MB, but so much of it we share with plants and other life on this planet. If you look at the part that’s uniquely human, it’s probably 10MB or something. Does that imply to you that you can actually create an AGI, an artificial general intelligence, with as little as 10MB of code if we just knew what that 10MB would look like? Or more precisely, with 10MB of code could you create something that could in turn learn to become an AGI?
Perhaps we can make that parallel. I’m not so much an expert on biology to be able to make a specific statement like that. But I guess in the way I approach research—beyond just looking at the fact that we are intelligent beings and our intelligence is essentially from our brain, and beyond just taking some inspiration from the brain—I mostly drive my research on designing learning algorithms more from math or statistics. Trying to think about what might be a reasonable approach for this or that problem, and how could I potentially implement it with something that looks like an artificial neural network. I’m sure some people have a better-informed opinion as to what extent we can draw a direct inspiration from biology, but beyond just the very high-level inspiration that I just described, what motivates my work and my approach to research is a bit more taking inspiration from math and statistics.
Do you begin with a definition of what you think intelligence is? And if so, how do you define intelligence?
That’s a very good question. There are two schools of thought, at least in terms of thinking of what we want to achieve. There’s one which is we want to somehow reach the closest thing to perfect rationality. And there’s another one which is to just achieve an intelligence that’s comparable to that of human beings, in the sense that, as humans perhaps we wouldn’t really draw a difference between a computer or another person, say, in talking with that machine or in looking at its ability to achieve a specific task.
A lot of machine learning really is based on imitating humans. In the sense that, we collect data, and this data, if it’s labeled, it’s usually produced by another person or committee of persons, like crowd workers. I think those two definitions aren’t incompatible, and it seems the common denominator is essentially a form of computation that isn’t otherwise easily encoded just by writing code yourself.
At the same time, what’s kind of interesting—and perhaps evidence that this notion of intelligence is elusive—is there’s this well-known phenomenon that we call the AI effect, which is that it seems very often whenever we reach a new level of AI achievement, of AI performance for a given task, it doesn’t take a whole lot of time before we start saying that this actually wasn’t AI, but this other new problem that we are now interested in is AI. Chess is a little bit like that. For a long time, people would associate chess playing as a form of intelligence. But once we figured out that we can be pretty good by treating it as, essentially, a tree search procedure, then some people would start saying, “Well that’s not really AI.” There’s now this new separation where chess-playing is not AI anymore, somehow. So, it’s a very tough thing to pin down. Currently, I would say, whenever I’m thinking of AI tasks, a lot of it is essentially matching human performance on some particular task.
Such as the Turing Test. It’s much derided, of course, but do you think there’s any value in it as a benchmark of any kind? Or is it just a glorified party trick when we finally do it? And to your point, that’s not really intelligence either.
No, I think there’s value to that, in the sense that, at the very least, if we define a specific Turing Test for which we currently have no solution, I think it is valuable to try to then succeed in that Turing Test. I think it does have some value.
There are certainly situations where humans can also do other things. So, arguably, you could say that if someone plays against AlphaGo, but wasn’t initially told if it was AlphaGo or not—though, interestingly, some people have argued it’s using strategies that the best Go players aren’t necessarily considering naturally—you could argue that right now if you played against AlphaGo you would have a hard time determining that this isn’t just some Go expert, at least many people wouldn’t be able to say that. But, of course, AlphaGo doesn’t really classify natural images, or it doesn’t dialog with a person. But still, I would certainly argue that trying to tackle that particular milestone is useful in our scientific endeavor towards more and more intelligent machines.
Isn’t it fascinating that Turing said that, assuming the listeners are familiar with it, it’s basically, “Can you tell if this is a machine or a person you’re talking to over a computer?” And Turing said that if it can fool you thirty percent of the time, we have to say it’s smart. And the first thing you say, well why isn’t it fifty percent? Why isn’t it, kind of, indistinguishable? An answer to that would probably be something like, “Well, we’re not saying that it’s as smart as a human, but it’s intelligent. You have to say it’s intelligent if it can fool people regularly.” But the interesting thing is that if it can ever fool people more than fifty percent, the only conclusion you can draw is that it’s better at being human than we are…or seeming human.
Well definitely that’s a good point. I definitely think that intelligence isn’t a black or white phenomenon, in terms of something is intelligent or isn’t, it’s definitely a spectrum. What it means for someone to fool a human more than actual humans into thinking that they’re human is an interesting thing to think about. I guess I’m not sure we’re really quite there yet, and if we were there then this might just be more like a bug in the evaluation itself. In the sense that, presumably, much like we have now adversarial networks or adversarial examples, so we have methods that can fool a particular test. I guess it just might be more a reflection of that. But yeah, intelligence I think is a spectrum, and I wouldn’t be comfortable trying to pin it down to a specific frontier or barrier that we have to reach before we can say we have achieved actual AI.
To say we’re not quite there yet, that is an exercise in understatement, right? Because I can’t find a single one of these systems that are trying to pass the test that can answer the following question, “What’s bigger, a nickel or the sun?” So, I need four seconds to instantly know. Even the best contests restrict the questions enormously. They try to tilt everything in favor of the machine. The machine can’t even put in a showing. What do you infer from that, that we are so far away?
I think that’s a very good point. And it’s interesting, I think, to talk about how quickly are we progressing towards something that would be indistinguishable from human intelligence—or any other—in the very complete Turing Test type of meaning. I think that what you’re getting at is that we’re getting pretty good at a surprising number of individual tasks, but for something to solve all of them at once, and be very flexible and capable in a more general way, essentially your example shows that we’re quite far from that. So, I do find myself thinking, “Okay, how far are we, do we think?” And often, if you talk to someone who isn’t in machine learning or in AI, that’s often the question they ask, “How far away are we from AIs doing pretty much anything we’re able to do?” And it’s a very difficult thing to predict. So usually what I say is that I don’t know because you would need to predict the future for that.
One bit of information that I feel we don’t often go back to is, if you look at some of the quotes of AI researchers when people were, like now, very excited about the prospect of AI, a lot of these quotes are actually similar to some of the things we hear today. So, knowing this, and noticing that it’s not hard to think of a particular reasoning task where we don’t really have anything that would solve it as easily as we might have thought—I think it just suggests that we still have a fairly long way in terms of a real general AI.
Well let’s talk about that for just a second. Just now you talked about the pitfalls of predicting the future, but if I said, “How long will it be before we get to Mars?” that’s a future question, but it’s answerable. You could say, “Well, rocket technology and…blah, blah, blah…2020 to 2040,” or something like that. But if you ask people who are in this field—at least tangentially in the field—you get answers between five and five hundred years. And so that implies to me that not only do we not know when we’re going to do it, we really don’t know how to build an AGI.  
So, I guess my question is twofold. One, why do you think there is that range? And two, do you think that, whether or not you can predict the time, do you think we have all of the tools in our arsenal that we need to build an AGI? Do you believe that with sufficient advances in algorithms, sufficient advances in processors, with data collection, etcetera, do you think we are on a linear path to achieve an AGI? Or is an AGI going to require some hitherto unimaginable breakthrough? And that’s why you get five to five hundred years because that’s the thing that’s kind of the black swan in the room?
That is my suspicion, that there are at least one and probably many technological breakthroughs—that aren’t just computers getting faster or collecting more data—that are required. One example, which I feel is not so much an issue with compute power, but is much more an issue of, “Okay, we don’t have the right procedure, we don’t have the right algorithms,” is being able to match how as humans we’re able to learn certain concepts with very little, quote unquote, data or human experience. An example that’s often given is if you show me a few pictures of an object, I will probably recognize that same object in many more pictures, just from a few—perhaps just one—photographs of that object. If you show me a picture of a family member and you show me other pictures of your family, I will probably identify that person without you having to tell me more than once. And there are many other things that we’re able to learn from very little feedback.
I don’t think that’s just a matter of throwing existing technology, more computers and more data, at it; I suspect that there are algorithmic components that are missing. One of them might be—and it’s something I’m very interested in right now—learning to learn, or meta-learning. So, essentially, producing learning algorithms from examples of tasks, and, more generally, just having a higher-level perspective of what learning is. Acknowledging that it works on various scales, and that there are a lot of different learning procedures happening in parallel and in intricate ways. And so, determining how these learning processes should act at various scales, I think, is probably a question we’ll need to tackle more and actually find a solution for.
There are people who think that we’re not going to build an AGI until we understand consciousness. That consciousness is this unique ability we have to change focus, and to observe the world a certain way and to experience the world a certain way that gives us these insights. So, I would throw that to you. Do you, A), believe that consciousness is somehow key to human intelligence; and, B), do you think we’ll make a conscious computer?
That’s a very interesting question. I haven’t really wrapped my head around what is consciousness relative to the concept of building an artificial intelligence. It’s a very interesting conversation to have, but I really have no clue, no handle on how to think about that.
I would say, however, that clearly notions of attention, for instance, being able to focus attention on various things or adding an ability to seek information, those are clearly components for which there’s, currently—I guess for attention we have some fairly mature solutions which work, thought in somewhat restrictive ways and not in the more general way; information seeking, I think, is still very much related to the notion of exploration and reinforcement learning—still a very big technical challenge that we need to address.
So, some of these aspects of our consciousness, I think, are kind of procedural, and we will need to figure out some algorithm to implement these, or learn to extract these behaviors from experience and from data.
You talked a little bit earlier about learning from just a little bit of data, that we’re really good at that. Is that, do you think, an example of humans being good at unsupervised learning? Because obviously as kids you learn, “This is a dog, and this is a cat,” and that’s supervised learning. But what you were talking about, was, “Now I can recognize it in low light, I can recognize it from behind, I can recognize it at a distance.” Is that humans doing a kind of unsupervised learning? Maybe start off by just explaining the concept and the hope about unsupervised learning, that it takes us, maybe, out of the process. And then, do you think humans are good at that?
I guess, unsupervised learning is, by definition, something that’s not supervised learning. It’s kind of an extreme of not using supervised learning. An example of that would be—and this is something I investigated quite a bit when I did my PhD ten years ago—to have a procedure, a learning algorithm, that can, for instance, look at images of hundreds of characters and be able to understand that each of these pixels in these images of characters are related. That they are higher-level concepts that explain why this is a digit. For instance, there is the concept of pen strokes; a character is really a combination of pen strokes. So, unsupervised learning would try to—just from looking at images, from the fact that there are correlations between these pixels, that they tend to look like something different than just a random image, and that pixels arrange themselves in a very specific way compared to any random combination of pixels—be able to extract these higher-level concepts like pen stroke and handwritten characters. In a more complex, natural scene this would be identifying the different objects without someone having to label each object. Because really what explains what I’m seeing is that there’s a few different objects with a particular light interacting with the scene and so on.
That’s something that I’ve looked at quite a bit, and I do think that humans are doing some form of that. But also, we’re, probably as infants, we’re interacting with our world and we’re exploring it and we’re being curious. And that starts being something a bit further away from just pure unsupervised learning and a bit closer to things like our reinforcement learning. So, this notion that I can actually manipulate my environment, and from this I can learn what are its properties, what are the facts and the variations that characterize this environment?
And there’s an even more supervised type of learning that we see in ourselves as infants that is not really captured by purely supervised learning, which is being able to exchange or to learn from feedback from another person. So, we might imitate someone, and that would be closer to supervised learning, but we might instead get feedback that’s worded. So, if a parent says do this or don’t do that, this isn’t exactly an imitation this is more like a communication of how you should adjust your behavior. And this is a form of weakly supervised learning. So, if I tell my kid to do his or her homework, or if I give instructions on how to solve a particular problem set, this isn’t a demonstration, so this isn’t supervised learning. This is more like a weak form of supervised learning. Which even then I think we don’t use as much in the known systems that work well currently that people are using in object recognition systems or machine translation systems and so on. And so, I believe that these various forms of learning that are much less supervised than the common supervised learning is a direction in research where we still have a lot of progress to make.
So earlier you were talking about meta learning, which is learning how to learn, and I think there’s been a wide range of views about how artificial intelligence and an AGI might work. And on one side was an early hope that, like the physical universe which is governed just by very few laws, and magnetism very few laws, electricity very few laws, we hoped that intelligence was governed by just a very few laws that we could learn. And then on the other extreme you have people like the late Marvin Minsky who really saw the brain as a hack of a couple of hundred narrow AIs, that all come together and give us, if not a general intelligence at least a really good substitute for one. I guess a belief in meta learning is a belief in the former case, or something like it, that there is a way to learn how to learn. There’s a way to build all those hacks. Would you agree? Do you think that?
We can take one example there. I think under a somewhat general definition of what learning to learn or meta learning is, it’s something that we could all agree exists, which is, as humans, we’re the result of years of evolution. And evolution is a form of adaptation, I guess. But then within our lifespan, each individual will also adapt to its specific human experience. So, you can think of evolution as being kind of like the meta learning to the learning that we do as humans in our individual lives every day. But then even in our own lives, I think there are clearly ways in which my brain is adapting as I’m growing older from a baby to an adult, that are not conscious. There are ways in which I’m adapting in a rational way, in conscious ways, which rely on the fact that my brain has adapted to be able to perceive my environment—my visual cortex just maturing. So again, there are multiple layers of learning that rely on each other. And so, I think this is, at a fairly high level, but I think in a meaningful way, a form of meta learning. For that reason, I think that investigating how to have learning of learning systems is that there is a process that’s valuable here in informing how to have more intelligent agents and AIs.
There’s a lot of fear wrapped up in the media coverage of artificial intelligence. And not even getting into killer robots, just the effects that it’s going to have on jobs and employment. Do you share that? And what is your prognosis for the future? Is AI in the end going to increase human productivity like all other technologies have done, or is AI something profoundly different that’s going to harm humans?
That’s a good question. What I can say is that I am motivated by—and what makes me excited about AI—is that I see it as an opportunity of automating parts of my day-to-day life which I would rather be automated so I can spend my life doing more creative things, or the things that I’m more passionate about or more interested in. I think largely because of that, I see AI as a wonderful piece of technology for humanity. I see benefits in terms of better machine translation which will better connect the different parts of the world and allow us to travel and learn about other cultures. Or how I can automate the work of certain health workers so that they can spend more time on the harder cases that probably don’t receive as much attention as they should.
For that reason—and because I’m personally motivated automating these aspects of life which we would want to see automated—I am fairly optimistic about the prospects for our society to have more AI. And, potentially, when it comes to jobs we can even imagine automating our ability to progress professionally. Definitely there’s a lot of opportunities in automating part of the process of learning in a course. We now have many courses online. Even myself when I was teaching, I was putting a lot of material on YouTube to allow for people to learn.
Essentially, I identified that the day-to-day teaching that I was doing in my job was very repetitive. It was something that I could record once and for all and instead focus my attention on spending time with the student and making sure that each individual student solves its own misunderstanding about the topic. Because my mental model of the student in general is that it’s often unpredictable how they will misunderstand a particular aspect of the course. And so, you actually want to spend some time interacting with that student, and you want to do that with as many students as possible. I think that’s an example where we can think of automating particular aspects of education so as to support our ability to have everyone be educated and be able to have a meaningful professional life. So, I’m overall optimistic, largely because of the way I see myself using AI and developing AI in the future.
Anybody who’s listened to many episodes of the show will know I’m very sympathetic to that position. I think it’s easy to point to history and say in the last two hundred and fifty years, other than the depression which wasn’t caused by technology obviously, unemployment has been between five and nine percent without fail. And yet, we’ve had incredibly disruptive technologies, like the mechanization of industry, the replacement of animal power with human power, electrification, and so forth. And in every case, humans have used those technologies to increase their own productivity and therefore their incomes. And that is the entire story of the rising standard of living for everybody, at least in the western world.
But I would be remiss not to make the other case, which is that there might be a point, an escape velocity, where a machine can learn a new job faster than a human. And at that point, at that magic moment, every new job, everything we create, a machine would learn it faster than a human. Such that, literally, everything from Michael Crichton down to…everybody—everybody finds themselves replaced. Is that possible? And if that really happened, would that be a bad thing?
That’s a very good question I think for society in general. Maybe because my day-to-day is about identifying what are the current challenges in making progress in AI, I see—and I guess we’ve touched that a little bit earlier—that there are still many scientific challenges, that it doesn’t seem like it’s just a matter of making computers faster and collecting more data. Because I see these many challenges, and because I’ve seen that the scientific community, in previous years, has been wrong and has been overly optimistic, I tend to err on the side of less gloomy and a bit more conservative in how quickly we’ll get there, if we ever get there.
In terms of what it means for society—if that was to ever happen that we can automate essentially most things—I unfortunately feel ill-equipped as a non-economist to be able to really have a meaningful opinion about this. But I do think it’s good that we have a dialog about it, as long as it’s grounded in facts. Which is why it’s a difficult question to discuss, because we’re talking about a hypothetical future that might not exist before a very long time. But as long as we have, otherwise, a rational discussion about what might happen, I don’t see a reason not to have that discussion.
It’s funny. Probably the truest thing that I’ve learned from doing all of these chats is that there is a direct correlation between how much you code and how far away you think an AGI is.
That’s quite possible.
I could even go further to say that the longer you have coded, the further away you think it is. People who are new at it are like, “Yeah. We’ll knock this out.” And the other people who think it’s going to happen really quickly are more observers. So, I want to throw a thought experiment to you.
Sure.
It’s a thought experiment that I haven’t presented to anybody on the show yet. It’s by a man named Frank Jackson, and it’s the problem of Mary, and the problem goes like this. There’s this hypothetical person, Mary, and Mary knows everything in the world about color. Everything is an understatement. She has a god-like understanding of color, everything down to the basic, most minute detail of light and neurons and everything. And the rub is that she lives in a room that she’s never left, and everything she’s seen is black and white. And one day she goes outside and she sees red for the first time. And the question is, does she learn anything new when that happens that she didn’t know before? Do you have an initial reaction to that?
My initial reaction is that, being colorblind I might be ill-equipped to answer that question. But seriously, so she has a perfect understanding of color but—just restating the situation—she has only seen in black and white?
Correct. And then one day she sees color. Did she learn anything new about color?
By definition of what understanding means, I would think that she wouldn’t learn anything about color. About red specifically.
Right. That is probably the consistent answer, but it’s one that is intuitively unsatisfying to many people. The question it’s trying to get at is, is experiencing something different than knowing something? And if in fact it is different, then we have to build a machine that can experience things for it to truly be intelligent, as opposed to just knowing something. And to experience things means you return to this thorny issue of consciousness. We are not only the most intelligent creature on the planet, but we’re arguably the most conscious. And that those two things somehow are tied together. And I just keep returning to that because it implies, maybe, you can write all the code in the world, and until the machine can experience something… But the way you just answered the question was, no, if you know everything, experiencing adds nothing.
I guess, unless that experience would somehow contradict what you know about the world, I would think that it wouldn’t affect it. And this is partly, I think, one challenge about developing AI as we move forward. A lot of the AIs that we’ve successfully developed that have to do with performing a series of actions, like playing Go for instance, have really been developed in a simulated environment. In this case, for a board game, it’s pretty easy to simulate it on a computer because you can literally write all the rules of the game so you can put them in the computer and simulate it.
But, for an experience such as being in the real world and manipulating objects, as long as that simulated experience isn’t exactly what the experience is in the real world, touching real objects, I think we will face a challenge in transferring any kind of intelligence that we grow in simulations, and transfer it to the real world. And this partly relates to our inability to have algorithms that learn rapidly. Instead, they require millions of repetitions or examples to really be close to what humans can do. Imagine having a robot go through millions of labeled examples from someone manipulating that robot, and showing it exactly how to do everything. That robot might essentially learn too slowly to really learn any meaningful behavior in a reasonable amount of time.
You used the word transfer three or four times there. Do you think that transfer learning, this idea that humans are really good at taking what we know in one domain space and applying it in another—you know, you walk around one big city and go to a different big city and you kind of map things. Is that a useful thing to work on in artificial intelligence?
Absolutely. In fact, we’re seeing that with all the success that has been enabled by the ImageNet data set and the competition. It turns out if you train an object recognition system on this large ImageNet data set, it really is responsible for the revolution of deep neural nets and convolutional neural nets in the field of computer vision. It turns out that these models trained on that source of data could transfer really well to a surprising number of paths. And that has very much enabled a kind of a revolution in computer vision. But it’s a fairly simple type of transfer, and I think there are more subtle ways of transferring, where you need to take what you knew before but slightly adjust it. How to do to that without forgetting what you learned before? So, understanding how these different mechanisms need to work together to be able to perform a form of lifelong learning, of being able to accumulate one task after another, and learning each new task with less and less experience, is something I think currently we’re not doing as well as we need to.
What keeps you up at night? You meet a genie and you rub the bottle and the genie comes out and says, “I will give you perfect understanding of something.” What do you wrestle with that maybe you can phrase in a way that would be useful to the listeners?
Let’s see. That’s a very good question. Definitely, in my daily research, how are we able to accumulate knowledge, and how would a machine accumulate knowledge, in a very long period, and learn the sequence of tasks and abilities in a sequence, cumulatively, is something that I think a whole lot about. And this has led me to think about learning to learn, because I suspect that there are ideas. And effectively once you have to learn one ability after the other after the other, that process of doing this and doing it better, the fact that we do it better is, perhaps, because we are learning how to learn each task also. That there’s this other scale of learning that is going on. How to do this exactly I don’t quite know, and knowing this I think would be a pretty big step in our field.
I have three final questions, if I could. You’re in Canada, correct?
As it turns out, I’m currently still in the US because I have four kids, two of them are in school so I wanted them to finish their school year before we move. But the plan is for me to go to Montreal, yes.
I noticed something. There’s a lot of AI activity in Canada, a lot of leading research. How did that come about? Was that a deliberate decision or just a kind of a coincidence that different universities and businesses decided to go into that?
If I speak for Montreal specifically, very clearly at the source of it is Yoshua Bengio deciding to stay in Montreal, staying in academia, and then continuing to train many students, gathering other researchers that are also in his group, and also training more PhDs in the field that doesn’t have as much talent as is needed. I think this is essentially the source of it.
And then my second to the last question is, what about science fiction? Do you enjoy it in any form, like movies or TV or books or anything like that? And if so, is there any that you look at it and think, “Ah, the future could happen that way”?
I definitely used to be more into science fiction. Now maybe due to having kids I watch many more Disney movies than I watch science fiction. It’s actually a good question. I’m realizing I haven’t watched a sci-fi movie for a bit, but it would be interesting, now that I’ve actually been in this field for a while, to sort of confront my vision of it from how artists potentially see AI. Maybe not seriously. A lot of art is essentially philosophy around what could happen, or at least projecting a potential future and seeing how we feel about it. And for that purpose, I’m now tempted to revisit either some classics or seeing what are recent sci-fi movies.
I said only one more question, so I’ve got to combine two into one to stick with that. What are you working on, and if a listener is going into college or is presently in college and wants to get into artificial intelligence in a way that is really relevant, what would be a leading edge that you would say somebody entering the field now would do well to invest time in? So first, you, and then what would you recommend for the next generation of AI researchers?
As I’ve mentioned, perhaps not so surprisingly, I am very much interested in learning to learn and meta learning. I’ve started publishing on the subject, and I’m still very much thinking around various new ideas for meta learning approaches. And also learning from, yes, weaker signals than in the supervised learning setting. Such as learning from worded feedback from a person is something I haven’t quite started working on specifically, but I’m thinking a whole lot about these days. Perhaps those are directions that I would definitely encourage other young researchers to think about and study and research.
And in terms of advice, well, I’m obviously biased, and being in Montreal studying deep learning and AI, currently, is a very, very rich and great experience. There are a lot of people to talk to, to interact with, not just in academia but now much more in industry, such as ourselves with Google and other places. And also, being very active online. On Twitter, there’s now a very, very rich community of people sharing the work of others and discussing the latest results. The field is moving very fast, and in large part it’s because the deep learning community has been very open about sharing its latest results, and also making the discussion open about what’s going on. So be connected, whether it be on Twitter or other social networks, and read papers and look at what comes up on archives—engage in the global conversation.
Alright. Well that’s a great place to end. I want to thank you so much. This has been a fascinating hour, and I would love to have you come back and talk about your other work in the future if you’d be up for it.
Of course, yeah. Thank you for having me.
Byron explores issues around artificial intelligence and conscious computers in his upcoming book The Fourth Age, to be published in April by Atria, an imprint of Simon & Schuster. Pre-order a copy here.
[voices_in_ai_link_back]

Voices in AI – Episode 27: A Conversation with Adrian McDermott

[voices_in_ai_byline]
In this episode, Byron and Adrian discuss intelligence, consciousness, self-driving cars and more.
[podcast_player name=”Episode 27 – A Conversation with Adrian McDermott” artist=”Byron Reese” album=”Voices in AI” url=”https://voicesinai.s3.amazonaws.com/2018-01-15-(00-58-48)-adrian-mcdermott.mp3″ cover_art_url=”https://voicesinai.com/wp-content/uploads/2018/01/voices-headshot-card.jpg”]
[voices_in_ai_byline]
Byron Reese: This is Voices in AI, brought to you by Gigaom. I’m Byron Reese. Today our guest is Adrian McDermott, he is Zendesk’s President of Products where he works to build software for better customer relationships, including, of course, exploring how AI and machine learning impacts the way customers engage with businesses. Adrian is a Yorkshireman, living in San Francisco, and he holds a Bachelor of Science and Computer Science from De Montfort University. Welcome to the show, Adrian!
Adrian McDermott: Thanks, Byron! Great to be here!
My first question is almost always: What is artificial intelligence?
When I think about artificial intelligence, I think about AI as a system that can interact with and learn from its environment in an independent manner. I think that’s where the intelligence comes from. AI systems have traditionally been optimized for achieving specific tasks. In computer science, we used to write programs using procedural languages and we would tell them exactly what to do at every stage of that language. With AI, it can actually learn and adapt from its environment and, you know, reason to a certain extent and build the capabilities to do that. Narrowly, I think that’s what AI is, but societally I think the term has a series of connotations it takes on, some scary and some super interesting and exciting meanings and consequences when we think about it and when we talk about it.
We’ll get to that in due course, but back to your narrow definition, “It learns for its environment,” that’s a pretty high bar, actually. By that measure, my dog food bowl that automatically refills when it runs out, even though it’s reacting to its environment, it’s not learning from its environment; whereas a Nest thermometer, you would say, is learning from its environment and therefore is AI. Did I call the ball right on both of those, kind of the way you see the world?
I think so. I mean, your dog bowl, perhaps, it learns, over time, how much food your dog needs every day, and it adapts to its environment, I don’t know. You could have an intelligent dog bowl, dog feeding system, hopefully one that understands the nature of most dogs is to keep eating until they choke. That would be an important governor on that system, let’s be honest, but I think in general that characterization is good.
We, as biological computational devices, learn from our environment and take in a series of inputs from those environments and then use those experiences, I think, to pattern match new stimuli and new situations that we encounter so that we know what to do, even though we’ve never seen that exact situation before.
So, and not to put any words in your mouth, but it sounds like you think that humans react to our environment and that is the source of our intelligence, and a computer that reacts to its environment, it’s artificial intelligence, but it really is intelligent. It’s not artificial, it’s not faking it, it really is intelligent. Is that correct?
I think artificial intelligence is this ability to learn from the environment, and come up with new behaviors as a result of this learning. There is a tremendous number of examples of AI systems that have created new ways of doing things and have learned. I think one of the most famous is move thirty-four in Google’s AlphaGo when it’s playing the game Go against Lee Sedol, one of the greatest players in the world. It performed a move that was shocking to the Go community and the Go intelligentsia because it had learned and it had evolved its thinking to a point where it created new ways of doing things that were not natural for us as humans. I think artificial intelligence, really, when it fulfills its promises, is able to create and learn in that way, but currently most systems do that within a very narrow problem domain.
With regard to an artificial general intelligence, do you think that the way we think of AI today eventually evolves into an AGI? In other words, are we on a path to create one? Or do you think a truly generalized intelligence will be built in a completely different way than how we are currently building AI systems today?
I mean, there are a series of characteristics of intelligence that we have, right, that we think about. One of them is the ability to think about a problem, think about a scenario, and run our head through different ways of handling that scenario and imagine different outcomes, and almost to self actualize in those situations. I think that modern deep-learning techniques actually are, you know, the construction is such that they are looking at different scenarios to come up with different outcomes. Ultimately, we don’t necessarily, I believe it’s true to say, understand a great deal about the nature of consciousness and the way that our brains work.
We know a lot about the physiology, not necessarily about the philosophy. It does seem like our brains are sort of neuron-based computation devices that take a whole bunch of inputs and process them based on stored experiences and learnings, and it does seem like that’s the kind of systems that we’re building with artificial-intelligence-based machines and computers.
Given that technology gets better every year, year over year, it seems like a natural conclusion that ultimately technology advancements will be such that we can reach the same point of general intelligence that our cerebral cortex reached hundreds of thousands of years ago. I think we have to assume that we will eventually get there. It seems like we’re building the systems in the same way that our brains function right now.
That’s fascinating because, that description of human’s ability to imagine different scenarios is in fact some people’s theory as to how consciousness emerged. And, not putting you on the spot because, as you said, we don’t really know, but is that plausible to you? That being able to essentially, kind of, carry on that internal dialogue, “I wonder if I should go pull that tiger’s tail,” you know, is that what you think made us conscious or are you indifferent on that question?
I only have a layman’s opinion, but, you know, there’s a test—I don’t know if it’s in evolutionary biology or psychology—the mirror test where if you put a dog in front of a mirror it doesn’t recognize itself, but Asian elephants and dolphins do recognize themselves in the mirror. So, it’s an interesting question of that ability to self-actualize, to understand who you are, and to make plans and go forward. That is the nature of intelligence and from an evolutionary point of view you can imagine a number of ways in which that consciousness of self and that ability to make plans was essential for the species to thrive and move forward. You know we’re not the largest species on the planet, but we’ve become somewhat dominant as a result of our ability to plan and take actions.
I think certain behaviors that we manifest came from the advantageous nature of cooperation between members of our species, and the way that we act together and act independently and dream independently and move together. I think it seems clear that that is probably how consciousness evolved, it was an evolutionary advantage to be conscious, to be able to make plans, to think about oneself, and we seem to be on the path where we’re emulating those structures in artificial intelligence work.
Yeah, the mirror test is fascinating because only one bird passes it and that is the magpie.
The magpie?
Yeah, and there’s recent research, very recent, that suggests that ants pass it, which would be staggering. It looks like they’ve controlled for so many things, but it is unquestionably a fascinating thing. Of course, people disagree on what exactly it means.
Yeah, what does it mean? It’s interesting that ants pass because ants do form a multi-role complex society. So, is it one of the requirements of a multi-role complex society that you need to be able to pass the mirror test, and understand who you are and what your place is in that society?
Yeah, that is fascinating. I actually emailed Gallup and asked him, “Did you know ants passed the test?” And he’s like, “Really, I hadn’t heard that?” You know, because he’s the originator of it.
The argument against the test goes like this: If you put a red dot on a dog’s paw, the dog knows that’s its paw and it might lick it off its own paw, right? The dog has a sense of self, it knows that’s its foot. And so, maybe all the mirror test is doing is testing to see if the dog is smart enough to understand what a mirror is, which is a completely different thing.
Do you think, by extension, and again with your qualification that it’s a layman’s viewpoint, I asked you a question about AGI and you launched into a description of consciousness. Can I infer from your answer that you believe that an AGI will be conscious?
You can infer from my answer that I believe that to have a truly artificial general intelligence, I think that consciousness is a requirement, or some kind of ability to have freedom in thought direction. I think that is part of the nature of consciousness or one way of thinking about it.
I would tend to agree, but let me just… Everybody’s had that sensation where you’re driving and you kind of space, right, and all of a sudden you snap to a minute later and you’re like, “Whoa, I don’t have any memory of driving to this spot,” and, in that moment, you merged traffic, you changed lanes, and all of that. So, you acted intelligently but you were not, in a sense, conscious at that moment. Do you think that saying, “Oh, that’s an example of intelligence without consciousness,” is the problem? Like, “No, no you really were conscious all that time,” or is it like, “No, no, you didn’t have, like, some new idea or anything, you just managed off rote.” Do you have a thought on that?
I think it’s true that so much of what we do as beings is managed off rote, but probably a lot of the reason we’re successful as a species is because we don’t just go off rote. Like, if someone had driven in front of you or the phone had rung, if all these things had happened, that would have created a suitably justifiable, stored in short-term memory because it’s important event while you were driving, then you would have moved into a different mode of consciousness. I think the human brain takes in a massive amount of input in some ways but filters it down to just this, quote unquote, “stream of consciousness” of experiences that are important, or things that are happening. And it’s that filter of consciousness, or the filter of the brain, that puts you in the moment where you’re dealing with the most important thing. That, in some ways, characterizes us.
When we think about artificial intelligence and how machines experience the world, I mean, we have five sensory inputs falling into our brains and our memories, but a machine can have, yes, vision, sound, but GPS, infrared, just some random event stream from another machine. There are all of these inputs that act in some ways as sensors for an artificially-intelligent machine that are much, in some ways, richer and more diverse, or could be. And that governor, that thing that filters that down, figures out what the objective is for the artificial intelligence machine and takes the right inputs and does the right pattern matching and does the right thinking, is going to be incredibly important to achieve, I think, artificial general intelligence. Where, it knows how to direct, if you like, it’s thoughts and how to plan and how to do and how to act, how to think about solving problems.
This is fascinating to me, so I have just a few more questions about AGI, if you’ll just indulge me for another minute. The range of time that people think it’s going to take us to get it, by my reckoning, is five years on the soonest and five-hundred on the longest. Do you have any opinion of when we might develop an AGI?
I think I agree with five years on the soonest, but, you know, honestly one of the things I struggle with as we think about that is, who really knows? We have so little understanding of how the brain actually works to produce intelligence and sentience that it’s hard to know how rapidly we’re approaching that or replicating it. It could be that, as we build smarter and smarter non-general artificial intelligence, eventually we’ll just wander into a greater understanding of consciousness or sentience by accident just because we built a machine that emulates the brain. That’s, in some ways, a plausible outcome, like, we’ll get enough computation that eventually we’ll figure it out or it will become apparent. I think, if you were to ask me, I think that’s ten to fifteen years away.
Do you think we already have computers fast enough to do it, we just don’t know how to do it, or do you think we’re waiting on hardware improvements as well?
I think the primary improvements we’re waiting on are software, but software activities are often constrained by the power and limits of the hardware that we’re running it on. Until you see a more advanced machine, it’s hard to practically imagine or design a system that could run upon it. The two things improve in parallel, I think.
If you believe we’ll, maybe, have an AGI in fifteen years, that if we have one it could very easily be conscious, and if it’s conscious therefore it would have a will, presumably, are you one of the people that worries about that? The super intelligence scenario, that it has different goals and ambitions than we have?
I think that’s one of many scenarios that we need to worry about. In our current society, any great idea, it seems like, is either weaponizable in a very direct way, which is scary. The way that we’re setup, locally and globally, is intensely competitive. Where any advantage one could eek out is then used to dominate, or take advantage of, or gain advantage from our position against our fellow man in this country and other countries, globally, etcetera.
There’s quite a bit of fear-mongering about artificial general intelligence, but, artificial intelligence does give the owner of those technologies, the inventor of those technologies, innate advantages in terms of taking and using those technologies to get great gain. I think there’s many stages along the way where someone can very competitively put those technologies to work without even achieving artificial general intelligence.
So, yes, the moment of singularity, when artificial general intelligence machines can invent machines that are considerably faster in ways that we can’t understand. That’s a scary thought, and technology may be out-thinking our moral and philosophical understanding of the implications of that, but at the same time some of the things that we’re building now—like you said, are just fifty percent better or seventy-seven percent smarter—could actually be, through weaponization or just through extreme mercantile advantage taking, those could have serious effects on the planet, humankind, etcetera. I do believe that we’re in an AI arms race and I do find that a little bit scary.
Vladimir Putin just said that he thinks the future is going to belong to whoever masters AI, and Elon Musk recently said, “World War Three will be fought over AI.” It sounds like you think that’s maybe a more real-world concern than the rogue AGI.
I think it is, because we’ve seen tremendous leaps in the capability of technology just in the last five years, certainly no less than five to ten years. More and more people are working in this problem domain; that number must be doubling every six months, or something ridiculous like that, in terms of the number of people who are starting to think about AI, the number of companies deploying some kind of technology. As a result, there are breakthroughs that are going to begin happening, either in public academia or more likely, in private labs that will be leverageable by the entities that create them in really meaningful ways.
I think by one count there are twenty different nations whose militaries are working on AI weapons. It’s hard to get a firm grip on it because: A, they wouldn’t necessarily say so, and, B, there’s not a lot of agreement on what the term AI means. In terms of machines that can make kill decisions, that’s probably a reasonable guess.
I think one shift that we’ve seen, and, you know, this is just anecdotal and my own opinion, is that so much of base research in computer science or artificial intelligence is done in academia and done basically publicly, publishable, and for the public good, I think, traditionally. And if you look at artificial intelligence where, you know, the greatest minds of our generation are not necessarily working in the public sphere on artificial intelligence; they’re locked up, tied up in private entity companies, generally very, very large companies, or they’re working on the military-industrial complex. I think that’s a shift, I think that’s different from scientific discovery, medical research, all these things in the past.
The closed-door nature of this R&D effort, and the fact that it’s becoming almost a national or nationalistic concern, with very little… You know there are weapons treaties, there are nuclear treaties, there are research weapons treaties, right? I think we’re only just beginning to talk about AI treaties, and AI understanding and we’re a long way from any resolve because the potential gains for whomever goes first, or makes the biggest discovery first, makes a great breakthrough first, are tremendous. It’s a very competitive world, and it’s going on behind closed doors.
The thing about the atomic bomb is that they were hard to build, and so even if you knew how to build it, it was hard. AI won’t be that way. It’ll fit on a flash drive, or at least the core technology would, right?
I think building an AGI, some of these things require web-scale computational power that currently, based on today’s technology, that requires data centers not flash drives. So, there is a barrier to entry to some of these things, but, that said, the great breakthrough more than likely will be an algorithm or some great thinking, and that will, yes, indeed, fit on a modern flash drive without any problem.
What do you think of the open AI initiative which says, “Let’s make this all public and share it all. It’s going to happen, we might as well make sure everybody has access to it and not just one party.”
I work at SaaS company, we build products to sell, and through open-source technologies, through cloud platforms, we get to stand on the shoulders of giants and use amazing stuff and shorten our development cycles and do things that we would never be able to do as a small company founded in Copenhagen. I’m a huge believer in those initiatives. I think that part of the reason that open-source has been so successful in just the problems of computer science and computer infrastructure is that, to a certain extent, there’s been a maturation of thought where not every company believes its ability to store and retrieve its data quickly is a defining characteristic for them. You know, I work at Zendesk and we’re in the business of customer service software, we build software that tries to help our customers have better relationships with their customers. It’s not clear that having the best cloud hosting engine or being able to use NoSQL technology is something that’s of tremendous commercial value to us.
We believe in open-sources, so we contribute back and we contribute because there’s no perceived risk of commercial impairment by doing that. This isn’t our core IP, our core IP is around how we treat customers. I think that, while I’m a huge believer in the open AI initiative, I think that there isn’t necessarily that widespread same belief when the parties are at investment levels in AI research, and at the forefront of thinking. I think that there’s a clear, for some of those entities, there’s a clear notion that they can gain tremendous advantage by keeping anything that they invent inside of the walled garden for as long as possible and using it to their advantage. I would dearly love that initiative to succeed. I don’t know that right now we have the environment in which it will truly succeed.
You’ve made a couple of references to artificial intelligence mirroring the human brain. Do you follow the human brain project in Europe, which is taking that approach? They’re saying, “Why don’t we just try to replicate the thing that we know can think already?”
I don’t really. I’m delighted by the idea, but I haven’t read too much about it. What are they learning?
It’s expensive, and they’re behind schedule. But it’s been funded to the tune of one and a half billion dollars, I mean it’s a really serious effort. The challenge is going to be if it turns out that a neuron is as complicated as a supercomputer, that things go on at the Planck level, that it is this incredible machine. Because I think the hope is that it if you take it at face value, that is something maybe we can duplicate, but if there’s other stuff going on it might be more problematic.
As an AI researcher yourself, do you ever start with the question, “How do humans do that?” Is that how you do it when you’re thinking about how to solve a problem? Or do you not find a lot of corollaries, in your day to day, between how a human does something and how a computer would do it?
When we’re thinking about solving problems with AI, we’re at the basic level of directed AI technology, and what we’re thinking about is, “How can we remove these tasks that humans perform on a regular basis? How can we enrich the lives of, in our case, the person needing customer service or the person providing customer services?” It’s relatively simple, and so the standard approach for that is to, yes, look directly at the activities of a person, look at ways that you can automate and take advantage of the benefits that the AI is going to buy you. In customer service land, you can remember every interaction very easily that every customer has had with a particular brand, and then you can look at the outcomes that those interactions have had, good or bad, through the satisfaction, the success and the timing. And you can start to emulate those things, remove friction, replace the need for people whatsoever, and build out really interesting things to do.
The primal way to approach the problem is really to look at what humans are doing, and try and replace them certainly where it’s not their cognitive ability that is necessarily to the fore or being used, and that’s something that we do a lot. But I think that misses the magic, because one of the things that happens with an AI system can be that it produces results that are, to use Arthur C. Clarke’s phrase, “sufficiently advanced to be indistinguishable from magic.” You can invent new things that were not possible because of the human brains limited bandwidth, because our limited memories or other things. You can basically remember all experiences all at once and then use those to create new things.
In our own work, we realize that it’s incredibly difficult, with any accuracy, given an input from a customer, a question from a customer, to predict the ultimate customer satisfaction score, the CSAT score that you’ll get. But it’s an incredibly important number for customer service departments, and knowing ahead of time that you’re going to have a bad experience with this customer based on signals in the input is incredibly useful. So, one of the things we built was a satisfaction-prediction engine, using various models, that allows us to basically route tickets to experts and do other things. There’s no human who sits there and gives out predictions on how a ticket is going to go, how our experience with the customer is going to go; that’s something that we invented because only a machine can do that.
So, yes, there is an approach to what we do which is, “How can we automate these human tasks?” But there’s also an approach of, “What is it that we can do that is impossible for humans that would be awesome to do?” Is there magic here that we can put in place?
In addition to there being a lot of concern about the things we talked about, about war and about AGI and all of that, in the narrow AI, in the here and now, of course, there’s a big debate about automation, and what these technologies are going to do for jobs. Just to, kind of, set the question up, there are three different narratives people offer. One is that automation is going to take all of the really low-skilled jobs, and they’ll be a group of people who are unable to compete against machines and we’ll have, kind of, permanent unemployment at the level of the Great Depression or something like that. Then there’s a second camp that says, “Oh, no, no, you don’t understand, it’s far worse than that, they’re going to take everybody’s job, everybody, because there’ll be a moment that the machine can learn something faster than a human.” Then there’s a third one that says, “No, with these technologies, people just take the technology and they use it to increase their own productivity, and they don’t actually ever cause unemployment.” Electricity and mechanization and all of that didn’t increase unemployment at all. Do you believe one of those three, or maybe a fourth one? What do you think about the effects of AI on employment?
I think the parallel that’s often drawn is a parallel to the Industrial Revolution. The Industrial Revolution brought us a way to transform energy from one form into another, and allowed us to mechanize manufacturing which altered the nature of society from agrarian to industrial, which created cities which had this big transformation. The Industrial Revolution took a long time. It took a long time for people to move from the farms to the factories, it took a long time to transform the landscape, comparatively. I think that one of the reasons that there’s trepidation and nervousness around artificial intelligence is it doesn’t seem like it will take that long, it’s almost fantastical science fiction to me that I get to see different vendors, self-driving cars mapping San Francisco on a regular basis, and I see people driving around with no hands on the wheel. I mean, that’s extraordinary, I don’t think even five years ago I would believe that we would have self-driving cars on public roads, it didn’t seem like a thing, and now it seems like automated driving machines are not very far away.
If you think about the societal impacts of that, well, according to an NPR study in 2014, I think, truck driving is the number one job in twenty-nine states in America. There are literally millions of driving jobs, and I think it’s one of the fastest growing categories of jobs. Things like that will all disappear, or to a certain extent will disappear, and it will happen rapidly.
It’s really hard for me to subscribe to the… Yes, we’re improving customer service software here at Zendesk in such a way that we’re making agents more efficient and they’re getting to spend more time with customers and they’re upping the CSAT rating, and consequently those businesses have better Net Promoter scores and they’re thriving. I believe that that’s what we’re doing and I believe that that’s what’s going to happen. If we can answer automatically ten percent of a customers’ tickets that means that you need ten percent less agents to answer those tickets, unless they’re going to invest more in customer service. The profit motive says that there needs to be a return on investment analysis between those two things. So, in my own industry I see this, and across society it’s hard not to believe that there won’t be a fairly large-scale disruption.
I don’t know that, as a society, we’re necessarily in a position to absorb that destruction yet. I know in Finland, they’re experimenting with a guaranteed minimum income to take away the stress of having to find work or qualify for unemployment benefit and all these things, so that people have a better quality of life and can hopefully find ways to be productive in society. Not many countries are as progressive as Finland. I would put myself in the “very nervous about the societal effects of large-scale removal of sources of employment,” because it’s not clear what the alternative structures are, that are set up in society to find meaningful work and sustenance for people who were losing those jobs. We’ve been under a trajectory since, I think, the 1970s, of polarization in society, and generating inequality. And I worry that the large-scale creation of an unemployed mass could be a tipping point. I take a very pessimistic view.
Let me give you a different narrative on that, and tell me what what’s wrong with it, how the logic falls down on it. Let’s talk just about truck drivers. That would go like this, it would say, “That concern that you’re going to have in mass all these unemployed truck drivers is beyond ill-founded. To begin with, the technology’s not done, and it will still need to be worked out. Then the legislative hurdles will have to be worked out, and that’ll be done gradually state by state by state. Then, there’ll be a long period of time when law will require that there be a driver, and self-driving technology would kick in when it feels like the driver’s making a mistake, but there’ll be an override; just like we can fly airplanes without pilots now but we insist on having a pilot.
Then, the driving part of the job is actually not the whole job, and so like any other job when you automate part of it, like the driving, that person takes on more things. Then, on top of that, the equipment’s not retrofit to it, so you going to have to figure out how do you retrofit all this stuff. Then, on top of that, having self-driving cars is going to open up all kinds of new employment, and because we talk about this all the time, there are probably fewer people going into truck driving, and there are people who retire in it every year. And that, just like every other thing, it’s just going to gradually work as the economy reallocates resources. Why do you think truck driving is like this big tipping point thing?
I think driving jobs in general are a tipping point thing because, yes, there are challenges to rolling it out, and obviously there’s legislative challenges, but it’s not hard to see, certainly interstate trucking going first and then drivers meeting those trucks and driving through urban areas and various things like that happening. I think people are working on retrofit devices for trucks. What will happen is truck drivers who are not actually driving will be allowed to work more hours, so you’ll need less truck drivers. In general, as a society, we’re shifting from going and getting our stuff to having our stuff delivered to us. And so, the voracious appetite for more drivers, in my opinion, is not going to abate. Yeah, the last mile isn’t driven by trucks, it’s smaller delivery drivers or things that can be done by smarter robots, etcetera.
I think those challenges you communicated are going to be moderating forces of the disruption, but when something reaches the tipping point of acceptance and cost acceptability, change tends to be rapid if driven by the profit motive. I think that is what we’re going to see. The efficiency of Amazon, and the fact that every product is online in that marketplace is driving a tremendous change in the nature of retail. I think the delivery logistics of that need are going to go through a similar turnaround, and companies driving that are going to be very aggressive about it because the economics is so appealing.
Of course, again, the general answer to that is that when technology does lower the price of something dramatically—like you’re talking about the cost of delivery, self-driving cars would lower it—that that in turn increases demand. That lowering of cost means all of a sudden you can afford to deliver all kinds of things, and that ripple effect in turn creates those jobs. Like, people spend all their money, more or less, and if something becomes cheaper they turn around and spend that money on something else which, by definition, therefore creates downstream employment. I’m just having a hard time seeing this idea that somehow costs are going to fall and that money won’t be redeployed in other places that in turn creates employment, which is kind of two hundred and fifty years of history.
I wouldn’t necessarily say that as costs fall in industries all of those profits are generally returned to the consumer, right? Businesses in the logistics retail space, generally, retailers run at a two percent margin, right, and businesses in logistics run with low margins. So, there’s room for those people to kind of optimize their own businesses, and not necessarily pass down all those benefits for the consumer. Obviously, there’s room for disruption where someone will come in, shave back down the margins and pass on those benefits. But, in general, you know, online banking is more efficient because we prefer it, and so there are less people working in banking. Conversely, when banks shifted to ATMs banking became much more of a part of our lives, and more convenient so we ended up with more bank tellers because personal service was a thing.
I think that there just are a lot of driving jobs out there that don’t necessarily need to be done by humans, but we’ll still be spending the same amount on getting driven around, so there’ll be more self-driving cars. Self-driving cars crash less, hopefully, and so there’s less need for auto repair shops. There’s a bunch of knock-on effects of using that technology, and for certain classes of jobs there’s clearly going to be a shift where those jobs disappear. There is a question of how readily the people doing those jobs are able to transfer their skills to other employment, and is there other employment out there for them.
Fair enough. Let’s talk with Zendesk for a moment. You’ve alluded to a couple of ways that you employ artificial intelligence, but can you just kind of give me an idea of, like, what gets you excited in the morning, when you wake up and you think, “I have this great new technology, artificial intelligence, that can do all these wondrous things, I want to use it to make people’s lives better who are in charge of customer relationships”? Entice me with some things that you’re thinking of doing, that you’re working on, that you’ve learned, and just kind of tell me about your day-to-day?
So many customer service inquiries begin with someone who has a thirst for knowledge, right? Seventy-six percent of people try to self-serve when trying to find the answer to a question, and many people who do get on the phone or online at the same time trying to discover the answer to that problem. I think often there’s a challenge in terms of having enough context to know what someone is looking for, having that context available to all of the systems that they’re interacting with. I think technology, not just artificial intelligence technology, but artificial intelligence can help us pinpoint the intention of users because the goal of the software that we provide, and the customer service ethos that we have is that we need to remove friction.
The thing that really generates bad experiences in customer service interactions isn’t that someone said no, or we didn’t get the outcome that we want, or we didn’t get our return processed or something like that, it’s that negative experiences tend to be generated from an excess of friction. It’s that I had to switch from one channel to another, it’s that I had to repeat myself over and over again because everyone I was talking to didn’t have context on my account or my experience as the customer and these things. I think that if you look at that sort of pile of problems, you see real opportunities to give people better experiences just by holding a lot more data at one time about that context, and then being able to process that data and make intelligent predictions and guesses and estimations about what it is they’re looking for and what is going to help them.
We recently launched a service we call “answer bot” which uses deep learning to look at the data we have when an email comes in and figure out, quite simply, which knowledgebase article is going to best serve that customer. It’s not driving a car down to the supermarket, this sounds very simple, but in another way these are millions and millions of experiences that can be optimized over time. Similarly, the people on the other side of that conversation generally don’t know what it is that customers are searching for or asking for, for which there is no answer. And so by using the same analysis of environment queries that we have and knowledge bases we can give them cues as to what content to write, and, sort of, direct them to build a better experience and improve their customer experience in that way.
I think from an enterprise software builder’s point of view, artificial intelligence is a tool that you can use at so many points of interaction between brand and consumer, between the two parties basically on either side of any transaction inside of your knowledge base. It’s something that you can use to shave off little moments of pain, and remove friction, and apply intelligence, and just make the world seem frictionless and a little smarter. Our goal internally is basically to meander through our product in a directed way, finding those experiences and making them better. At the end of the day we want someone who’s deploying our stuff and giving a customer experience with it, and we want the consumers experiencing that brand, the people interacting with that brand, to be like, “I’m not sure why that was good, but I did really enjoy that customer service experience. I got what I wanted, it was quick. I don’t know how they quite did that, but I really enjoyed it.” We all have had those moments in service where someone just totally got what you were after and it was delightful because it was just smooth and efficient, good, and no drama—prescient almost.
I think what we are trying to do, what we would like to do is adapt all of our software and experiences that we have to be able to be that anticipatory and smart and enjoyable. I think the enterprise software world—for all types of software like CRM, ERP, all these kind of things—is filled with sharp edges, friction, and pain, you know, pieces of acquisitions glued together, and you’re using products that represent someone’s broken dreams acquired by someone else and shoehorned into other experiences. I think, generally, the consumer of enterprise software at this point is a little bit tired of the pain of form-filling and repetition and other things. Our approach to smoothing those edges, to grinding the stone and polishing the mirror, is to slowly but surely improve each of those experiences with intelligence.
It sounds like you have a broad charter to look at kind of all levels of the customer interaction and look for opportunity. I’m going to ask you a question that probably doesn’t have an answer but I’m going to try anyway, “Do you prefer to find places where there was an epic fail where it was so bad it was just terrible and the person was angry and it was just awful, or would you rather fix ten of a minor annoyance where somebody had entered data too many times?” I mean, are you working to cut the edges off the bad experiences, or just generally make the system phase shift up a little bit?
I think, to a certain extent, I like to think of that as a false dichotomy, because the person who has a terrible experience and gets angry, chances are there wasn’t a momentary snap, there was a drip feed of annoyances that took them to that point. So, our goal, when we think about it, is to pick out the most impactful rough edges that cumulatively are going to engulf someone into the red mist of homicidal fury on the end of the phone, complaining about their broken widget. I think most people do not flip their anger bit over a tiny infraction or over a larger fraction, it’s over a period, it’s a lifetime of infractions, it’s a lifetime of inconveniences that gets you to that point, or the lifetime of that incident and that inquiry and how you got there. We’re generally, sort of, emotionally-rational beings who’ve been through many customer service experiences, so exhibiting that level of frustration, generally, requires a continued and sustained effort on the part of a brand to get us there.
I assume that you have good data to work off of. I mean, there are good metrics in your field and so you get to wade through a lot of data and say, “Wow, here’s a pattern of annoyances that we can fix.” Is that the case?
Yeah, we have an anonymized data set that encompasses billions of interactions. And the beauty of that data set is they’re rated, right? They’re rated either by the time it took to solve the problem, or they’re rated by an explicit rating, where someone said that was a good interaction, that was a bad interaction. When we did the CSAT prediction we were really leveraging the millions of scores that we have that tell us how customer service interactions went. In general, though, we talk about the data asset that we have available to us, that we can use to train and learn a query and analyze.
Last question, you quoted Arthur C. Clarke, so I have to ask you, is there any science fiction about AI that you enjoy or like or think that could happen? Like Her or Westworld or iRobot or any of that, even books or whatnot?
I did find Westworld to be, probably, the most compelling thing I watched this year, and just truly delightful in its thinking about memory and everything else, although it was, obviously, pure fiction. I think Her was also just a, you know, a disturbing look at the way that we will be able to identify with inanimate machines and build relationships that, you know, it was all too believable. I think you quoted two my favorite things, but Westworld was so awesome.
It, interestingly, had a different theory of consciousness from the bicameral mind, not to give anything away.
Well, let’s stop there. This was a magnificently interesting hour, I think we touched on so many fascinating topics, and I appreciate you taking the time!
Adrian McDermott: Thank you, Byron, it’s wonderful to chat too!
Byron explores issues around artificial intelligence and conscious computers in his upcoming book The Fourth Age, to be published in April by Atria, an imprint of Simon & Schuster. Pre-order a copy here.
[voices_in_ai_link_back]

Voices in AI – Episode 24: A Conversation with Deep Varma

[voices_in_ai_byline]
In this episode, Byron and Deep talk about the nervous system, AGI, the Turing Test, Watson, Alexa, security, and privacy.
[podcast_player name=”Episode 24: A Conversation with Deep Varma” artist=”Byron Reese” album=”Voices in AI” url=”https://voicesinai.s3.amazonaws.com/2017-12-04-(00-55-19)-deep-varma.mp3″ cover_art_url=”https://voicesinai.com/wp-content/uploads/2017/12/voices-headshot-card_preview-1.jpeg”]
[voices_in_ai_link_back]
Byron Reese: This is Voices in AI, brought to you by Gigaom. I’m Byron Reese. Today our guest is Deep Varma, he is the VP of Data Engineering and Science over at Trulia. He holds a Bachelor’s of Science in Computer Science. He has a Master’s degree in Management Information Systems, and he even has an MBA from Berkeley to top all of that off. Welcome to the show, Deep.
Deep Varma: Thank you. Thanks, Byron, for having me here.
I’d like to start with my Rorschach test question, which is, what is artificial intelligence?
Awesome. Yeah, so as I define artificial intelligence, this is an intelligence created by machines based on human wisdom, to augment a human’s lifestyle to help them make the smarter choices. So that’s how I define artificial intelligence in a very simple and the layman terms.
But you just kind of used the word, “smart” and “intelligent” in the definition. What actually is intelligence?
Yeah, I think the intelligence part, what we need to understand is, when you think about human beings, most of the time, they are making decisions, they are making choices. And AI, artificially, is helping us to make smarter choices and decisions.
A very clear-cut example, which sometimes what we don’t see, is, I still remember in the old days I used to have this conventional thermostat at my home, which turns on and off manually. Then, suddenly, here comes artificial intelligence, which gave us Nest. Now as soon as I put the Nest there, it’s an intelligence. It is sensing that someone is there in the home, or not, so there’s motion sensing. Then it is seeing what kind of temperature do I like during summer time, during winter time. And so, artificially, the software, which is the brain that we have put on this device, is doing this intelligence, and saying, “great, this is what I’m going to do.” So, in one way it augmented my lifestyle—rather than me making those decisions, it is helping me make the smart choices. So, that’s what I meant by this intelligence piece here.
Well, let me take a different tack, in what sense is it artificial? Is that Nest thermostat, is it actually intelligent, or is it just mimicking intelligence, or are those the same thing?
What we are doing is, we are putting some sensors there on those devices—think about the central nervous system, what human beings have, it is a small piece of a software which is embedded within that device, which is making decisions for you—so it is trying to mimic, it is trying to make some predictions based on some of the data it is collecting. So, in one way, if you step back, that’s what human beings are doing on a day-to-day basis. There is a piece of it where you can go with a hybrid approach. It is mimicking as well as trying to learn, also.
Do you think we learn a lot about artificial intelligence by studying how humans learn things? Is that the first step when you want to do computer vision or translation, do you start by saying, “Ok, how do I do it?” Or, do you start by saying, “Forget how a human does it, what would be the way a machine would do it?
Yes, I think it is very tough to compare the two entities, because the way human brains, or the central nervous system, the speed that they process the data, machines are still not there at the same pace. So, I think the difference here is, when I grew up my parents started telling me, “Hey, this is Taj Mahal. The sky is blue,” and I started taking this data, and I started inferring and then I started passing this information to others.
It’s the same way with machines, the only difference here is that we are feeding information to machines. We are saying, “Computer vision: here is a photograph of a cat, here is a photograph of a cat, too,” and we keep on feeding this information—the same way we are feeding information to our brains—so the machines get trained. Then, over a period of time, when we show another image of a cat, we don’t need to say, “This is a cat, Machine.” The machine will say, “Oh, I found out that this is a cat.”
So, I think this is the difference between a machine and a human being, where, in the case of machine, we are feeding the information to them, in one form or another, using devices; but in the case of human beings, you have conscious learning, you have the physical aspects around you that affect how you’re learning. So that’s, I think, where we are with artificial intelligence, which is still in the infancy stage.
Humans are really good at transfer learning, right, like I can show you a picture of a miniature version of the Statue of Liberty, and then I can show you a bunch of photos and you can tell when it’s upside down, or half in water, or obscured by light and all that. We do that really well. 
How close are we to being able to feed computers a bunch of photos of cats, and the computer nails the cat thing, but then we only feed it three or four images of mice, and it takes all that stuff it knows about different cats, and it is able to figure out all about different mice?
So, is your question, do we think these machines are going to be at the same level as human beings at doing this?
No, I guess the question is, if we have to teach, “Here’s a cat, here’s a thimble, here’s ten thousand thimbles, here’s a pin cushion, here’s ten thousand more pin cushions…” If we have to do one thing at a time, we’re never going to get there. What we’ve got to do is, like, learn how to abstract up level, and say, “Here’s a manatee,” and it should be able to spot a manatee in any situation.
Yeah, and I think this is where we start moving into the general intelligence area. This is where it is becoming a little interesting and challenging, because human beings falls under more of the general intelligence, and machines are still falling under the artificial intelligence framework.
And the example you were giving, I have two boys, and when my boys were young, I’d tell them, “Hey, this is milk,” and I’d show them milk two times and they knew, “Awesome, this is milk.” And here come the machines, and you keep feeding them the big data with the hope that they will learn and they will say, “This is basically a picture of a mouse or this is a picture of a cat.”
This is where, I think, this artificial general intelligence which is shaping up—that we are going to abstract a level up, and start conditioning—but I feel we haven’t cracked the code for one level down yet. So, I think it’s going to take us time to get to the next level, I believe, at this time.
Believe me, I understand that. It’s funny, when you chat with people who spend their days working on these problems, they’re worried about, “How am I going to solve this problem I have tomorrow?” They’re not as concerned about that. That being said, everybody kind of likes to think about an AGI. 
AI is, what, six decades old and we’ve been making progress, do you believe that that is something that is going to evolve into an AGI? Like, we’re on that path already, and we’re just one percent of the way there? Or, is an AGI is something completely different? It’s not just a better narrow AI, it’s not just a bunch of narrow AI’s bolted together, it’s a completely different thing. What do you say?
Yes, so what I will say, it is like in the software development of computer systems—we call this as an object, and then we do inheritance of a couple of objects, and the encapsulation of the objects. When you think about what is happening in artificial intelligence, there are companies, like Trulia, who are investing in building the computer vision for real estate. There are companies investing in building the computer vision for cars, and all those things. We are in this state where all these dysfunctional, disassociated investments in our system are happening, and there are pieces that are going to come out of that which will go towards AGI.
Where I tend to disagree, I believe AI is complimenting us and AGI is replicating us. And this is where I tend to believe that the day the AGI comes—that means it’s a singularity that they are reaching wisdom or the processing power of human beings—that, to me, seems like doomsday, right? Because that those machines are going to be smarter than us, and they will control us.
And the reason I believe that, and there is a scientific reason for my belief; it’s because we know that in the central nervous system the core tool is the neurons, and we know neurons carry two signals—chemical and electrical. Machines can carry the electrical signals, but the chemical signals are the ones which generate these sensory signals—you touch something, you feel it. And this is where I tend to believe that AGI is not going to happen, I’m close to confident. Thinking machines are going to come—IBM Watson, as an example—so that’s how I’m differentiating it at this time.
So, to be clear, you said you don’t believe we’ll ever make an AGI?
I will be the one on the extreme end, but I will say yes.
That’s fascinating. Why is that? The normal argument is a reductionist argument. It says, you are some number of trillions of cells that come together, and there’s an emergent you” that comes out of that. And, hypothetically, if we made a synthetic copy of every one of those cells, and connected them, and did all that, there would be another Deep Varma. So where do you think the flaw in that logic is?
I think the flaw in that logic is that the general intelligence that humans have is also driven by the emotional side, and the emotional side—basically, I call it a chemical soup—is, I feel, the part of the DNA which is not going to be possible to replicate in these machines. These machines will learn by themselves—we recently saw what happened with Facebook, where Facebook machines were talking to each other and they start inventing their own language, over a period of time—but I believe the chemical mix of humans is what is next to impossible to produce it.
I mean—and I don’t want to take a stand because we have seen proven, over the decades, what people used to believe in the seventies has been proven to be right—I think the day we are able to find the chemical soup, it means we have found the Nirvana; and we have found out how human beings have been born and how they have been built over a period of time, and it took us, we all know, millions and millions of years to come to this stage. So that’s the part which is putting me on the other extreme end, to say, “Is there really going to another Deep Varma,” and if yes, then where is this emotional aspect, where are those things that are going to fit into the bigger picture which drives human beings onto the next level?
Well, I mean there’s a hundred questions rushing for the door right now. I’ll start with the first one. What do you think is the limit of what we’ll be able to do without the chemical part? So, for instance, let me ask a straight forward question—will we be able to build a machine that passes the Turing test?
Can we build that machine? I think, potentially, yes, we can.
So, you can carry on a conversation with it, and not be able to figure out that it’s a machine? So, in that case, it’s artificial intelligence in the sense that it really is artificial. It’s just running a program, saying some words, it’s running a program, saying some words, but there’s nobody home.
Yes, we have IBM Watson, which can go a level up as compared to Alexa. I think we will build machines which, behind the scenes, are trying to understand your intent and trying to have those conversations—like Alexa and Siri. And I believe they are going to eventually start becoming more like your virtual assistants, helping you make decisions, and complimenting you to make your lifestyle better. I think that’s definitely the direction we’re going to keep seeing investments going on.
I read a paper of yours where you made a passing reference to Westworld.
Right.
Putting aside the last several episodes, and what happened in them—I won’t give any spoilerstake just the first episode, do you think that we will be able to build machines that can interact with people like that?
I think, yes, we will.
But they won’t be truly creative and intelligent like we are?
That’s true.
Alright, fascinating. 
So, there seem to be these two very different camps about artificial intelligence. You have Elon Musk who says it’s an existential threat, you have Bill Gates who’s worried about it, you have Stephen Hawking who’s worried about it, and then there’s this other group of people that think that’s distracting
saw that Elon Musk spoke at the governor’s convention and said something and then Pedro Domingos, who wrote The Master Algorithmretweeted that article, and his whole tweet was, “One word: sigh. So, there’s this whole other group of people that think that’s just really distractingreally not going to happen, and they’re really put off by that kind of talk. 
Why do you think there’s such a gap between those two groups of people?
The gap is that there is one camp who is very curious, and they believe that millions of years of how human beings evolved can immediately be taken by AGI, and the other camp is more concerned with controlling that, asking are those machines going to become smarter than us, are they going to control us, are we going to become their slaves?
And I think those two camps are the extremes. There is a fear of losing control, because humans—if you look into the food chain, human beings are the only ones in the food chain, as of now, who control everything—fear that if those machines get to our level of wisdom, or smarter than us, we are going to lose control. And that’s where I think those two camps are basically coming to the extreme ends and taking their stands.
Let’s switch gears a little bit. Aside from the robot uprising, there’s a lot of fear wrapped up in the kind of AI we already know how to build, and it’s related to automation. Just to set up the question for the listener, there’s generally three camps. One camp says we’re going to have all this narrow AI, and it’s going to put a bunch of people out of work, people with less skills, and they’re not going to be able to get new work and we’re going to have, kind of, the GreaDepression going on forever. Then there’s a second group that says, no, no, it’s worse than that, computers can do anything a person can do, we’re all going to be replaced. And then there’s a third camp that says, that’s ridiculous, every time something comes along, like steam or electricity, people just take that technology, and use it to increase their own productivity, and that’s how progress happens. So, which of those three camps, or fourth one, perhaps, do you believe?
I fall into, mostly, the last camp, which is, we are going to increase the productivity of human beings; it means we will be able to deliver more and faster. A few months back, I was in Berkeley and we were having discussions around this same topic, about automation and how jobs are going to go away. The Obama administration even published a paper around this topic. One example which always comes in my mind is, last year I did a remodel of my house. And when I did the remodeling there were electrical wires, there are these water pipelines going inside my house and we had to replace them with copper pipelines, and I was thinking, can machines replace those job? I keep coming back to the answer that, those skill level jobs are going to be tougher and tougher to replace, but there are going to be productivity gains. Machines can help to cut those pipeline pieces much faster and in a much more accurate way. They can measure how much wire you’ll need to replace those things. So, I think those things are going to help us to make the smarter choices. I continue to believe it is going to be mostly the third camp, where machines will keep complementing us, helping to improve our lifestyles and to improve our productivity to make the smarter choices.
So, you would say that there are, in most jobs, there are elements that automation cannot replace, but it can augment, like a plumber, or so forth. What would you say to somebody who’s worried that they’re going to be unemployable in the future? What would you advise them to do?
Yeah, and the example I gave is a physical job, but think about an example of a business consultants, right? Companies hire business consultants to come, collect all the data, then prepare PowerPoints on what you should do, and what you should not do. I think those are the areas where artificial intelligence is going to come, and if you have tons of the data, then you don’t need a hundred consultants. For those people, I say go and start learning about what can be done to scale them to the next level. So, in the example I’ve just given, the business consultants, if they are doing an audit of a company with the financial books, look into the tools to help so that an audit that used to take thirty days now takes ten days. Improve how fast and how accurate you can make those predictions and assumptions using machines, so that those businesses can move on. So, I would tell them to start looking into, and partnering into, those areas early on, so that you are not caught by surprise when one day some industry comes and disrupts you, and you say, “Ouch, I never thought about it, and my job is no longer there.”
It sounds like you’re saying, figure out how to use more technology? That’s your best defense against it, is you just start using it to increase your own productivity.
Yeah.
Yeah, it’s interesting, because machine translation is getting comparable to a human, and yet generally people are bullish that we’re going to need more translators, because this is going to cause people to want to do more deals, and then they’re going to need to have contracts negotiated, and know about customs in other countries and all of that, so that actually being a translator you get more business out of this, not less, so do you think things like that are kind of the road map forward?
Yeah, that’s true.
So, what are some challenges with the technology? In Europe, there’s a movement—I think it’s already adopted in some places, but the EU is considering it—this idea that if an AI makes a decision about you, like do you get the loan, that you have the right to know why it made it. In other words, no black boxes. You have to have transparency and say it was made for this reason. Do you think a) that’s possible, and b) do you think it’s a good policy?
Yes, I definitely believe it’s possible, and it’s a good policy, because this is what consumers wants to know, right? In our real estate industry, if I’m trying to refinance my home, the appraiser is going to come, he will look into it, he will sit with me, then he will send me, “Deep, your house is worth $1.5 million dollar.” He will provide me the data that he used to come to that decision—he used the neighborhood information, he used the recent sold data.
And that, at the end of the day, gives confidence back to the consumer, and also it shows that this is not because this appraiser who came to my home didn’t like me for XYZ reason, and he end up giving me something wrong; so, I completely agree that we need to be transparent. We need to share why a decision has been made, and at the same time we should allow people to come and understand it better, and make those decisions better. So, I think those guidelines need to be put into place, because humans tend to be much more biased in their decision-making process, and the machines take the bias out, and bring more unbiased decision making.
Right, I guess the other side of that coin, though, is that you take a world of information about who defaulted on their loan, and then you take you every bit of information about, who paid their loan off, and you just pour it all in into some gigantic database, and then you mine it and you try to figure out, “How could I have spotted these people who didn’t pay their loan? And then you come up with some conclusion that may or may not make any sense to a human, right? Isn’t that the case that it’s weighing hundreds of factors with various weights and, how do you tease out, “Oh it was this”? Life isn’t quite that simple, is it?
No, it is not, and demystifying this whole black box has never been simple. Trust us, we face those challenges in the real estate industry on a day-to-day basis—we have Trulia’s estimates—and it’s not easy. At the end, we just can’t rely totally on those algorithms to make the decisions for us.
I will give one simple example, of how this can go wrong. When we were training our computer vision system, and, you know, what we were doing was saying, “This is a window, this is a window.” Then the day came when we said, “Wow, our computer vision can say I will look at any image, and known this is a window.” And one fine day we got an image where there is a mirror, and there is a reflection of a window on the mirror, and our computer said, “Oh, Deep, this is a window.” So, this is where big data and small data come into a place, where small data can make all these predictions and goes wrong completely.
This is where—when you’re talking about all this data we are taking in to see who’s on default and who’s not on default—I think we need to abstract, and we need to at least make sure that with this aggregated data, this computational data, we know what the reference points are for them, what the references are that we’re checking, and make sure that we have the right checks and balances so that machines are not ultimately making all the calls for us.
You’re a positive guy. You’re like, “We’re not going to build an AGI, it’s not going to take over the world, people are going to be able to use narrow AI to grow their productivity, we’re not going to have unemployment.” So, what are some of the pitfalls, challenges, or potential problems with the technology?
I agree with you, it’s being positive. Realistically, looking into the data—and I’m not saying that I have the best data in front of me—I think what is the most important is we need to look into history, and we need to see how we evolved, and then the Internet came and what happened.
The challenge for us is going to be that there are businesses and groups who believe that artificial intelligence is something that they don’t have to worry about, and over a period of time artificial intelligence is going to start becoming more and more a part of business, and those who are not able to catch up with this, they’re going to see the unemployment rate increase. They’re going to see company losses increase because some of the decisions they’re not making in the right way.
You’re going to see companies, like Lehman Brothers, who are making all these data decisions for their clients by not using machines but relying on humans, and these big companies fail because of them. So, I think, that’s an area where we are going to see problems, and bankruptcies, and unemployment increases, because of they think that artificial intelligence is not for them or their business, that it’s never going to impact them—this is where I think we are going to get the most trouble.
The second area of trouble is going to be security and privacy, because all this data is now floating around us. We use the Internet. I use my credit card. Every month we hear about a new hack—Target being hacked, Citibank being hacked—all this data physically-stored in the system and it’s getting hacked. And now we’ll have all this data wirelessly transmitting, machines talking to each of their devices, IoT devices talking to each other—how are you we going to make sure that there is not a security threat? How are we going to make sure that no one is storing my data, and trying to make assumptions, and enter into my bank account? Those are the two areas where I feel we are going to see, in coming years, more and more challenges.
So, you said privacy and security are the two areas?
Denial of accepting AI is the one, and security and privacy is the second one—those are the two areas.
So, in the first one, are there any industries that don’t need to worry about it, or are you saying, “No, if you make bubble-gum you had better start using AI?
I will say every industry. I think every industry needs to worry about it. Some industries may adapt the technologies faster, some may go slower, but I’m pretty confident that the shift is going to happen so fast that, those businesses will be blindsided—be it small businesses or mom and pop shops or big corporations, it’s going to touch everything.
Well with regard to security, if the threat is artificial intelligence, I guess it stands to reason that the remedy is AI as well, is that true?
The remedy is there, yes. We are seeing so many companies coming and saying, “Hey, we can help you see the DNS attacks. When you have hackers trying to attack your site, use our technology to predict that this IP address or this user agent is wrong.” And we see that to tackle the remedy, we are building an artificial intelligence.
But, this is where I think the battle between big data and small data is colliding, and companies are still struggling. Like, phishing, which is a big problem. There are so many companies who are trying to solve the phishing problem of the emails, but we have seen technologies not able to solve it. So, I think AI is a remedy, but if we stay just focused on the big data, that’s, I think, completely wrong, because my fear is, a small data set can completely destroy the predictions built by a big data set, and this is where those security threats can bring more of an issue to us.
Explain that last bit again, the small data set can destroy…?
So, I gave the example of computer vision, right? There was research we did in Berkeley where we trained machines to look at pictures of cats, and then suddenly we saw the computer start predicting, “Oh, this is this kind of a cat, this is cat one, cat two, this is a cat with white fur.” Then we took just one image where we put the overlay of a dog on the body of a cat, and the machines ended up predicting, “That’s a dog,” not seeing that it’s the body of a cat. So, all the big data that we used to train our computer vision, just collapsed with one photo of a dog. And this is where I feel that if we are emphasizing so much on using the big data set, big data set, big data set, are there smaller data sets which we also need to worry about to make sure that we are bridging the gap enough to making sure that our securities are not compromised?
Do you think that the system as a whole is brittle? Like, could there be an attack of such magnitude that it impacts the whole digital ecosystem, or are you worried more about, this company gets hacked and then that one gets hacked and they’re nuisances, but at least we can survive them?
No, I’m more worried about the holistic view. We saw recently, how those attacks on the UK hospital systems happened. We saw some attacks—which we are not talking about—on our power stations. I’m more concerned about those. Is there going to be a day when we have built massive infrastructures that are reliant on computers—our generation of power and the supply of power and telecommunications—and suddenly there is a whole outage which can take the world to a standstill, because there is a small hole which we never thought about. That, to me, is the bigger threat than the stand alone individual things which are happening now.
That’s a hard problem to solve, there’s a small hole on the internet that we’ve not thought about that can bring the whole thing down, that would be a tricky thing to find, wouldn’t it?
It is a tricky thing, and I think that’s what I’m trying to say, that most of the time we fail because of those smaller things. If I go back, Byron, and bring the artificial general intelligence back into a picture, as human beings it’s those small, small decisions we make—like, I make a fast decision when an animal is approaching very close to me, so close that my senses and my emotions are telling me I’m going to die—and this is where I think sometimes we tend to ignore those small data sets.
I was in a big debate around those self-driven cars which are shaping up around us, and people were asking me when will we see those self-driven cars on a San Francisco street. And I said, “I see people doing crazy jaywalking every day,” and accidents are happening with human drivers, no doubt, but the scale can increase so fast if those machines fail. If they have one simple sensor which is not working at that moment in time and not able to get one signal, it can kill human beings much faster as compared to what human beings are killing, so that’s the rational which I’m trying to put here.
So, one of my questions that I was going to ask you, is, do you think AI is a mania? Like it’s everywhere but it seems like, you’re a person who says every industry needs to adopt it, so if anything, you would say that we need more focus on it, not less, is that true?
That’s true.
There was a man in the ‘60s named Weizenbaum who made a program called ELIZA, which was a simple program that you would ask a question, say something like, I’m having a bad day,” and then it would say, “Why are you having a bad day?” And then you would say, I’m having a bad day because I had a fight with my spouse,” and then would ask, “Why did you have a fight? And so, it’s really simple, but Weizenbaum got really concerned because he saw people pouring out their heart to it, even though they knew it was a program. It really disturbed him that people developed emotional attachment to ELIZA, and he said that when a computer says, “I understand,” that it’s a lie, that there’s no “I,” there’s nothing that understands anything. 
Do you worry that if we build machines that can imitate human emotions, maybe the care for people or whatever, that we will end up having an emotional attachment to them, or that that is in some way unhealthy?
You know, Byron, it’s a very great question. I think, also pick out a great example. So, I have Alexa at my home, right, and I have two boys, and when we are in a kitchen—because Alexa is in our kitchen—my older son comes home and says, “Alexa, what’s the temperature look like today?” Alexa says, “Temperature is this,” and then he says, “Okay, shut up,” to Alexa. My wife is standing there saying “Hey, don’t be rude, just say, ‘Alexa stop.’” You see that connection? The connection is you’ve already started treating this machine as a respectful device, right?
I think, yes, there is that emotional connection there, and that’s getting you used to seeing it as part of your life in an emotional connection. So, I think, yes, you’re right, that’s a danger.
But, more than Alexa and all those devices, I’m more concerned about the social media sites, which can have much more impact on our society than those devices. Because those devices are still physical in shape, and we know that if the Internet is down, then they’re not talking and all those things. I’m more concerned about these virtual things where people are getting more emotionally attached, “Oh, let me go and check what my friends been doing today, what movie they watched,” and how they’re trying to fill that emotional gap, but not meeting individuals, just seeing the photos to make them happy. But, yes, just to answer your question, I’m concerned about that emotional connection with the devices.
You know, it’s interesting, I know somebody who lives on a farm and he has young children, and, of course, he’s raising animals to slaughter, and he says the rule is you just never name them, because if you name them then that’s it, they become a pet. And, of course, Amazon chose to name Alexa, and give it a human voice; and that had to be a deliberate decision. And you just wonder, kind of, what all went into it. Interestingly, Google did not name theirs, it’s just the Google Assistant. 
How do you think that’s going to shake out? Are we just provincial, and the next generation isn’t going to think anything of it? What do you think will happen?
So, is your question what’s going to happen with all those devices and with all those AI’s and all those things?
Yes, yes.
As of now, those devices are all just operating in their own silo. There are too many silos happening. Like in my home, I have Alexa, I have a Nest, those plug-ins. I love, you know, where Alexa is talking to Nest, “Hey Nest, turn it off, turn it on.” I think what we are going to see over the next five years is that those devices are communicating with each other more, and sending signals, like, “Hey, I just saw that Deep left home, and the garage door is open, close the garage door.”
IoT is popping up pretty fast, and I think people are thinking about it, but they’re not so much worried about that connectivity yet. But I feel that where we are heading is more of the connectivity with those devices, which will help us, again, compliment and make the smart choices, and our reliance on those assistants is going to increase.
Another example here, I get up in the morning and the first thing I do is come to the kitchen and say Alexa, “Put on the music, Alexa, put on the music, Alexa, and what’s the weather going to look like?” With the reply, “Oh, Deep, San Francisco is going to be 75,” then Deep knows Deep is going to wear a t-shirt today. Here comes my coffee machine, my coffee machine has already learned that I want eight ounces of coffee, so it just makes it.
I think all those connections, “Oh, Deep just woke up, it is six in the morning, Deep is going to go to office because it’s a working day, Deep just came to kitchen, play this music, tell Deep that the temperature is this, make coffee for Deep,” this is where we are heading in next few years. All these movies that we used to watch where people were sitting there, and watching everything happen in the real time, that’s what I think the next five years is going to look like for us.
So, talk to me about Trulia, how do you deploy AI at your company? Both customer facing and internally?
That’s such an awesome question, because I’m so excited and passionate because this brings me home. So, I think in artificial intelligence, as you said, there are two aspects to it, one is for a consumer and one is internal, and I think for us AI helps us to better understand what our consumers are looking for in a home. How can we help move them faster in their search—that’s the consumer facing tagline. And an example is, “Byron is looking at two bedroom, two bath houses in a quiet neighborhood, in good school district,” and basically using artificial intelligence, we can surface things in much faster ways so that you don’t have to spend five hours surfing. That’s more consumer facing.
Now when it comes to the internal facing, internal facing is what I call “data-driven decision making.” We launch a product, right? How do we see the usage of our product? How do we predict whether this usage is going to scale? Are consumers going to like this? Should we invest more in this product feature? That’s the internal facing we are using artificial intelligence.
I don’t know if you have read some of my blogs, but I call it data-driven companies—there are two aspects of the data driven, one is the data-driven decision making, this is more of an analyst, and that’s the internal reference to your point, and the external is to the consumer-facing data-driven product company, which focuses on how do we understand the unique criteria and unique intent of you as a buyer—and that’s how we use artificial intelligence in the spectrum of Trulia.
When you say, “Let’s try to solve this problem with data, is it speculative, like do you swing for the fences and miss a lot? Or, do you look for easy incremental wins? Or, are you doing anything that would look like pure science, like, “Let’s just experiment and see what happens with this? Is the science so nascent that you, kind of, just have to get in there and start poking around and see what you can do?
I think it’s both. The science helps you understand those patterns much faster and better and in a much more accurate way, that’s how science helps you. And then, basically, there’s trial and error, or what we call an, “A/B testing” framework, which helps you to validate whether what science is telling you is working or not. I’m happy to share an example with you here if you want.
Yeah, absolutely.
So, the example here is, we have invested in our computer vision which is, we train our machines and our machines basically say, “Hey, this is a photo of a bathroom, this is a photo of a kitchen,” and we even have trained that they can say, “This is a kitchen with a wide granite counter-top.” Now we have built this massive database. When a consumer comes to the Trulia site, what they do is share their intent, they say, “I want two bedrooms in Noe Valley,” and the first thing that they do when those listings show up is click on the images, because they want to see what that house looks like.
What we saw was that there were times when those images were blurred, there were times when those images did not match up with the intent of a consumer. So, what we did with our computer vision, we invested in something called “the most attractive image,” which basically takes the three attributes—it looks into the quality of an image, it looks into the appropriateness of an image, and it looks into the relevancy of an image—and based on these three things we use our conventional neural network models to rank the images and we say, “Great, this is the best image.” So now when a consumer comes and looks at that listing we show the most attractive photo first. And that way, the consumer gets more engaged with that listing. And what we have seen— using the science, which is machine learning, deep learning, CNM models, and doing the A/B testing—is that this project increased our enquiries for the listing by double digits, so that’s one of the examples which I just want to share with you.
That’s fantastic. What is your next challenge? If you could wave a magic wand, what would be the thing you would love to be able to do that, maybe, you don’t have the tools or data to do yet?
I think, what we haven’t talked about here and I will use just a minute to tell you, that what we have done is we’ve built this amazing personalization platform, which is capturing Byron’s unique preferences and search criteria, we have built machine learning systems like computer vision recommender systems and the user engagement prediction model, and I think our next challenge will be to keep optimizing the consumer intent, right? Because the biggest thing that we want to understand is, “What exactly is Byron looking into?” So, if Byron visits a particular neighborhood because he’s travelling to Phoenix, Arizona, does that mean you want to buy a home there, or Byron is in San Francisco and you live here in San Francisco, how do we understand?
So, we need to keep optimizing that personalization platform—I won’t call it a challenge because we have already built this, but it is the optimization—and make sure that our consumers get what they’re searching for, keep surfacing the relevant data to them in a timely manner. I think we are not there yet, but we have made major inroads into our big data and machine learning technologies. One specific example, is Deep, basically, is looking into Noe Valley or San Francisco, and email and push notifications are the two channels, for us, where we know that Deep is going to consume the content. Now, the day we learn that Deep is not interested in Noe Valley, we stop sending those things to Deep that day, because we don’t want our consumers to be overwhelmed in their journey. So, I think that this is where we are going to keep optimizing on our consumer’s intent, and we’ll keep giving them the right content.
Alright, well that is fantastic, you write on these topics so, if people want to keep up with you Deep how can they follow you?
So, when you said “people” it’s other businesses and all those things, right? That’s what you mean?
Well I was just referring to your blog like I was reading some of your posts.
Yeah, so we have our tech blog, http://www.trulia.com/tech, and it’s not only me; I have an amazing team of engineers—those who are way smarter than me to be very candid—my data scientist team, and all those things. So, we write our blogs there, so I definitely ask people to follow us on those blogs. When I go and speak at conferences, we publish that on our tech blog, and I publish things on my LinkedIn profile. So, yeah, those are the channels which people can follow. Trulia, we also host data science meetups here in Trulia, San Francisco on the seventh floor of our building, that’s another way people can come, and join, and learn from us.
Alright, well I want to thank you for a fascinating hour of conversation, Deep.
Thank you, Byron.
Byron explores issues around artificial intelligence and conscious computers in his upcoming book The Fourth Age, to be published in April by Atria, an imprint of Simon & Schuster. Pre-order a copy here.
[voices_in_ai_link_back]

Voices in AI – Episode 23: A Conversation with Pedro Domingos

[voices_in_ai_byline]
In this episode Byron and Pedro Domingos talk about the master algorithm, machine creativity, and the creation of new jobs in the wake of the AI revolution.
[podcast_player name=”Episode 23: A Conversation with Pedro Domingos” artist=”Byron Reese” album=”Voices in AI” url=”https://voicesinai.s3.amazonaws.com/2017-12-04-(00-53-50)-pedro-domingos.mp3″ cover_art_url=”https://voicesinai.com/wp-content/uploads/2017/12/voices-headshot-card_preview.jpeg”]
[voices_in_ai_link_back]
Byron Reese: This is Voices in AI, brought to you by Gigaom. I’m Byron Reese. Todayour guest is Pedro Domingosa computer science professor at the University of Washington, and the author of The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake our World. Welcome to the show, Pedro.
Pedro Domingos: Thanks for having me.
What is artificial intelligence?
Artificial intelligence is getting computers to do things that traditionally require human intelligence, like reasoning, problem solving, common sense knowledge, learning, vision, speechand language understanding, planning, decision making and so on.
And is it artificial in the sense that artificial turf is artificial—in that it isn’t really intelligence, it just looks like intelligence? Or is it actually truly intelligent, and it’s just the “artificial” demarks that we created it?
That’s a fun analogy. I hadn’t heard that before. No, I don’t think AI is like artificial turf. I think it’s real intelligence. It’s just intelligence of a different kind. We’re used to thinking of human intelligence, or maybe animal intelligence, as the only intelligence on the planet.
What happens now is a different kind of intelligence. It’s a little bit like, does a submarine really swim? Or is it faking that it swims? Actually, it doesn’t really swim, but it can still travel underwater using very different ideas. Or, you know, does a plane fly even though it doesn’t flap its wings? Well, it doesn’t flap its wings but it does fly. AI is a little bit like that. In some ways, actually, artificial intelligence is intelligent in ways that human intelligence isn’t.
There are many areas where AI exceeds human intelligence, so I would say that they’re different forms of intelligence, but it is very much a form of intelligence.
And how would you describe the state-of-the-art, right now?
In science and technology progress often happens in spurts. There are long periods of slow progress and then there are periods of very sudden, very rapid progress. And we are definitely in one of those periods of very rapid progress in AI, which was a long time in the making.
AI is a field that’s fifty years old, and we had what was called the “AI spring” in the ‘80s, where it looked like it was going to really take off. But then that didn’t really happen at the end of the day, and the problem was that people back then were trying to do AI using what’s called “knowledge engineering.” If I wanted an AI system to do medical diagnosis, I had to interview doctors and program the doctor’s knowledge of diagnosis in the form of rules into the computer, and that didn’t scale.
The thing that has changed recently is that we have a new way to do AI, which is machine learning. Instead of trying to program the computers to do things, the computers program themselves by learning from data. So now what I do for medical diagnosis is I give the computer a database of patient records, what their symptoms and test results were, and what the diagnosis was—and from just that, in thirty seconds, the computer can learn, typically, to do medical diagnosis better than human doctors.
So, thanks to that, thanks to machine learning, we are now seeing a phase of very rapid progress. Also, because the learning algorithms have gotten better—and very importantly: the beauty of machine learning is that, because the intelligence comes from the data, as the data grows exponentially, the AI systems get more intelligent with essentially no extra work from us. So now AI is becoming very powerful. Just on the back of the weight of data that we have.
The other element, of course, is computing power. We need enough computing power to turn all that data into intelligent systems, but we do have those. So the combination of learning algorithms, a lot of data, and a lot of computing power is what is making the current progress happen.
And, how long do you think we can ride that wave? Do you think that machine learning is the path to an AGI, hypothetically? I mean, do we have ten, twenty, thirty, forty more years of running with, kind of, the machine learning ball? Or, do we need another kind of breakthrough?
I think machine learning is definitely the path to artificial general intelligence. But I think there are a few people in AI who would disagree with that. You know, your computer can be as intelligent as you want. If it can’t learn, you know, thirty minutes later it will be falling behind humans.
So, machine learning really is essential to getting to intelligence. In fact, the whole idea of the singularity—it was I. J. Good, back in the ‘50s, who had this idea of a learning machine that could make a machine that learned better than it did. As a result of which, you would have this succession of better and better, more and more intelligent machines until they left humans in the dust.
Now, how long will it take? That’s very hard to predict, precisely because progress is not linear. I think the current bloom of progress at some point will probably plateau. I don’t think we’re on the verge of having general AI. We’ve come a thousand miles, but there’s a million miles more to go. We’re going to need many more breakthroughs, and who knows where those breakthroughs will come from.
In the most optimistic view, maybe this will all happen in the next decade or two, because things will just happen one after another, and we’ll have it very soon. In the more pessimistic view, it’s just too hard and it’ll never happen. If you poll the AI experts, they never just say it’s going to be several decades. But the truth is nobody really knows for sure.
What is kind of interesting is not that people don’t know, and not that their forecasts are kind of all over the map, but that if you look at the extreme estimates, five years are the most aggressive, and then the furthest out are like five hundred years. And what does that suggest to you? 
You know, if I went to my cleaners and I said, “Hey, when is my shirt going to be ready?” and they said, “Sometime between five and five hundred days” I would be like, “Okay… something’s going on here.” 
Why do you think the opinions are so variant on when we get an AGI?
Well, the cleaners, when they clean your shirt, it’s a very well-known, very repeatable process. They know how long it takes and it’s going to take the same thing this time, right? There are very few unknowns. The problem in AI is that we don’t even know what we don’t know.
We have no idea what we’re missing, so some people think we’re not missing that much. There are the optimists that say, “Oh, we just need more data.” Right? Back in the ‘80s they said, “Oh, we just need more knowledge,” and then, that wasn’t the case. So that’s the optimistic view. The more pessimistic view is that this is a really, really hard problem, and we’ve only scratched the surface. So the uncertainty comes from the fact that we don’t even know what we don’t know.
We certainly don’t know how the brain works, right? We have vague ideas of what different parts of it do, but in terms of how a thought is encoded, we don’t know. Do you think we need to know more about our own intelligence to make an AGI, or is it like, “No, that’s apples and oranges. It doesn’t really matter how the brain works. We’re building an AGI differently”?
Not necessarily. So, there are different schools of thought in AI, and this is part of what I talk about in my book. There is one school of thought in AI, the Connectionists, whose whole agenda is to reverse-engineer the brain. They think that’s the shortest path, you know, “Here’s the competition, go reverse-engineer it, figure out how it works, build it on the computer, and then we’ll have intelligence.” So that is definitely a plausible approach.
I think it’s actually a very difficult approach, precisely because we understand so little about how the brain works. In some ways maybe it’s trying to solve a problem by way of solving the hardest of problems.
And then there are other AI types, namely the Symbolists, whose whole idea is, “No, we don’t need to understand things at that low level. In fact, we’re just going to get lost in the weeds if we try to do that. We have to understand intelligence at a higher-level abstraction, and we’ll get there much sooner that way. So forget how the brain works, that’s really not important.”
Again, the analogy with the brains and airplanes is a good one. What the Symbolists say is, “If we try to make airplanes by building machines that will flap their wings, we’ll never have them. What we need to do is understand the laws of physics and aerodynamics, and then build machines based on that.”
So there are different schools of thought. And I actually think it’s good that there are different schools of thought—and we’ll see who gets there first.
So, you mentioned your book, The Master Algorithm, which is of course required reading in this field. Can you give the listener, who may not be as familiar with it, an overview of what is The Master Algorithm? What are we looking for?
Yeah, sure. So the book is essentially an introduction to machine learning for a general audience. So not just for technical people, but business people, policy makers, just citizens and people who are curious. It talks about the impact that machine learning is already having in the world.
A lot of people think that these things are science fiction, but they are already in their lives and they just don’t know it. It also looks at the future, and what we can expect coming down the line. But mainly, it is an introduction to what I was just describing—that there are five main schools of thought in machine learning.
There are the people who want to reverse-engineer the brain; the ones who want to simulate evolution; the ones who do machine learning by automating the scientific method; the ones who use Bayesian statistics; and the ones who do reasoning by analogy, like people do in everyday life. And then I look at what these different methods can and can’t do.
The name The Master Algorithm comes from this notion that a machine learning algorithm is a master algorithm, in the same sense that a master key opens all doors. A learning algorithm can do all sorts of different things while being the same algorithm.
The is really what’s extraordinary about machine learning… In traditional computer science, if I want the computer to play chess, I have to write a program explaining how to play chess. And if I want the computer to drive a car, I had to write a program explaining how to drive a car. With machine learning, the same learning algorithm can learn to play chess, or drive a car, or do a million different other things—just by learning from the appropriate data.
And each of these tribes of machine learning has its own master algorithm. The more optimistic members of that tribe believe that you can do everything with that master algorithm. My contention in the book is that each of these algorithms is only solving part of the problem. What we need to do is unify them all into a grand theory of machine learning, in the same way that physics has a standard model and biology has a central dogma. And then, that will be the true master algorithm. And I suggest some paths towards that algorithm, and I think we’re actually getting pretty close to it.
One thing I found empowering in the book—and you state it over and over at the beginning—is that the master algorithm is aspirationally accessible for a wide range of people. You basically said, “You, listening to the book, this is still a field where the layman can still have some amount of breakthrough.” Can you speak to that for just a minute?
Absolutely. In fact, that’s part of what got me into machine learning is that—unlike physics or mathematics or biology, which are very mature fields, and you really can only contribute once you have at least a PhD—computer science and AI and machine learning are still very young. So, you could be a kid in a garage and have a great idea that will be transformative. And I hope that that will happen.
I think, even after we find this master algorithm that’s the unification of the five current ones, as we were talking about, we will still be missing some really important, really deep ideas. And I think in some ways, someone coming from outside the field is more likely to find those, than those of us who are professional machine learning researchers, and are already thinking along these tracks of these particular schools of thought.
So, part of my goal in writing the book was to get people who are not machine learning experts thinking about machine learning, and possibly having the next great ideas that will get us closer to AGI.
And, you also point out in the book why you believe that we know that such a thing is possible, and one of your proof points is our intelligence. 
Exactly.
Can you speak to that?
Yeah. So this is, of course, one of those very ambitious goals that people should be at the outset a little suspicious of, right? Is this, like the philosopher’s stone or the perpetual motion machine, is it really possible? And again, some people don’t think it’s possible.
I think there’s a number of reasons why I’m pretty sure it is possible, one of which is that we already have existing proofs. One existing proof is our brain, right? As long as you believe in reductionism, which all scientists do, then the way your brain works can be expressed as an algorithm.
And if I program that algorithm into a computer, then that algorithm can learn everything that your brain can. Therefore, in that sense at least, one version of the master algorithm already exists.
Another one is evolution. Evolution created us and all life on Earth. And it is essentially an algorithm, and we roughly understand how that algorithm works; so there is another existing instance of the master algorithm.
Then there are also—besides these more empirical reasons—theoretical reasons which tell us that a master algorithm exists. One of which is that, for each of the five tribes, for their master algorithm there’s a theorem that says: If you give enough data to this algorithm, it can learn any function.
So, at least at that level, we already know that master algorithms exist. Now the question is, how complicated will it be? How hard will it be to get us there? How broadly good would that algorithm be, in terms of learning from a reasonable amount of data in a reasonable amount of time?
You just said all scientists are reductionist. Is that necessarily the case? Like, can you not be a scientist and believe in something like strong emergence, and say, “Actually, you can’t necessarily take the human mind down to individual atoms and kind of reconstruct I mean you don’t have to appeal to mysticism to
Yeah, yeah, absolutely. So, what I mean… This is a very good point. In fact, in the sense that you’re talking about, we cannot be reductionists in AI. So what I mean by “reductionist” is just the idea that we can decompose a complex system into simpler, smaller parts that interact and that make up the system.
This is how all of the sciences and engineering works. But this does not preclude the existence of emergent properties. So, the system can be more than the sum of its parts, if it’s non-linear. And very much the brain is a non-linear system. And that’s what we have to do to reach AI. You could even say that machine learning is the science of emergent properties.
In fact, one of the names by which it has been known in some quarters is “self-organizing systems.” And in fact, what makes AI hard, the reason we haven’t already solved it, is that the usual divide-and-conquer strategy which scientists and engineers follow—of dividing problems into smaller and smaller sub-problems, and then solving the sub-problems, and putting the solutions together—tends not to work in AI, because the sub-systems are very strongly coupled together. So, there are emergent properties, but that does not mean that you can’t reduce it to these pieces; it’s just a harder thing to do.
Marvin Minsky, I remember, talked about how we kind of got tricked a little bit by the fact that it takes very few fundamental laws of the universe to understand most of physics. The same with electricity. The same with magnetism. There are very few simple laws to explain everything that happens. And so the hope had been that intelligence would be like that. Are we giving up on that notion?
Yes, so again, there are different views within AI on this. I think at one end there are people who hope we will discover a few laws of AI, and those would solve everything. At the other end of the spectrum there are people like Marvin Minsky who just think that intelligence is a big, big pile of hacks.
He even has a book that’s like, one of these tricks per page. And who knows how many more there are. I think, and most people in AI believe, that it’s somewhere in between. If AI is just a big pile of hacks, we’re never going to get there. And it can’t really be just a pile of hacks, because if the hacks were so powerful as to create intelligence, then you can’t really call them hacks.
On the other hand, you know, you can’t reduce it to a few laws, like Newton’s laws. So this idea of the master algorithm is that, at the end of the day, we will find one algorithm that does intelligence, but that algorithm is not going to be a hundred lines of code. It’s not going to be millions of lines of code either. You know, if the algorithm is thousands or maybe tens of thousands of lines of code, that would be great. It’ll still be a complex theory—much more complex than the ones we have in physics—but it’ll be much, much simpler than what people like Marvin Minsky envisioned.
And if we find the master algorithmis that good for humanity?
Well, I think it’s good or bad depending on what we do with it. Like all technology, machine learning just gives us more power. You can think of it as a superpower, right? Telephones let us speak at a distance, airplanes let us fly, and machine learning lets us predict things and lets technology adapts automatically to our needs. All of this is good if we use it for good. If we use it for bad, it will be bad, right? The technology itself doesn’t know how it’s going to be used.
Part of my reason for writing this book is that everybody needs to be aware of what machine learning is, and what it can do, so that they can control it. Because, otherwise, machine learning will just give more control to those few who actually know how to use it.
I think if you look at the history of technology, over time, in the end, the good tends to prevail over the bad, which is why we live in a better world today than we did two hundred or two thousand years ago. But we have to make it happen, right? It just doesn’t fall from the tree like that.
And so, in your view, the master algorithm is essentially synonymous with AGI, in the sense that it can figure anything out—it’s a general artificial intelligence. 
Would it be conscious?
Yeah, so, by the way: I wouldn’t say the master algorithm is synonymous with AGI. I think it’s the enabler of AGI. Once we have the master algorithm, we’re still going to need to apply it to vision, and language, and reasoning, and all these things. And then we’ll have AGI.
So, one way to think about this is that it’s an 80/20 rule. The master algorithm is the twenty percent of the work that gets you eighty percent of the way, but you still need to do the rest, right? So maybe this is a better way to think about it.
Fair enough. So, I’ll just ask the question a little more directly. What do you think consciousness is?
That’s a very good question. The truth is, what makes consciousness simultaneously so fascinating and so hard is that, at the end of the day, if there is one thing that I know it’s that I’m conscious, right? Descartes said, “I think, therefore I am,” but maybe he should’ve said “I’m conscious, therefore I am.”
The laws of physics, who knows, they might even be wrong. But the fact that I’m conscious right now is absolutely unquestionable. So, everybody knows that about themselves. At the same time, because consciousness is a subjective experience, it doesn’t lend itself to the scientific method. What are reproducible experiments when it comes to consciousness? That’s one aspect.
The other one is that consciousness is a very complex, emergent phenomenon. So, nobody really knows what it is, or understands it, even at a fairly shallow level. Now, the reason we believe others have consciousness… You believe that I have consciousness because you’re a human being, and I’m a human being, so since you have consciousness, I probably have consciousness as well. And this is really the extent of it. For all you know, I could be a robot talking to you right now, passing the Turing test, and not be conscious at all.
Now, what happens with machines? How can we tell whether a machine is conscious or not? This has been grist for the mill of a lot of philosophers over the last few decades. I think the bottom line is that once a computer starts to act like it’s consciousness, we will treat it as if it’s conscious, we will grant it consciousness.
In fact, we already do that, even with very simple chatbots and what not. So, as far as everyday life goes, it actually won’t be long. In some ways, it’ll happen that people treat computers as being conscious, sooner than they treat the computers as being truly intelligent. Because that’s all we need, right? We project these human properties onto things that act humanly, even in the slightest way.
Now, at the end of the day, if you gaze down into that hardware and those circuits, is there really consciousness there? I don’t know if we will ever be able to really answer that question. Right now, I actually don’t see a good way. I think there will come a point at which we understand consciousness well enough—because we understand the brain well enough—that we are fairly confident that we can tell whether something is conscious or not.
And then at that point I think we will apply this criteria to these machines; and these machines—at least the ones that have been designed to be conscious—will pass the tests. So, we will believe that machines have consciousness. But, you know, we can never be totally sure.
And do you believe consciousness is required for a general intellect?
I think there are many kinds of AI, and many AI applications which do not require consciousness. So, for example, if I tell a machine learning system to go solve cancer—that’s one of the things we’d like to do, cure cancer, and machine learning is a very big part of the battle to cure cancer—I don’t think it requires consciousness at all. It requires a lot of searching, and understanding molecular biology, and trying different drugs, maybe designing drugs, etc. So, ninety percent of AI will involve no consciousness at all.
There are some applications of AI, and some types of AI, that will require consciousness, or something indistinguishable from of it. For example, housebots. We would like to have a robot that cooks dinner and does the dishes and makes the bed and what not.
In order to do all those things, the robot has to have all the capabilities of a human, has to integrate all of these senses: vision, and touch, and perception, and hearing and what not; and then make decision based on it. I think this is either going to be consciousness or something indistinguishable from it.
Do you think there will be problems that arise if that happens? Let’s say you build Rosie the Robotand you don’t know if the robot is conscious or merely acting as if it is. Do you think at that point we have to have this question of, “Are we fine enslaving what could be a conscious machine to plunge our toilet for us?”
Well, that depends on what you consider enslaving, right? So, one way to look at this—and it’s the way I look at it—is that these are still just machines, right? Just because they have consciousness doesn’t mean that they have human rights. Human rights are for humans. I don’t think there’s such thing as robot rights.
The deeper question here is, what gives something rights? One school of thought is that it’s the ability to suffer that gives you rights, and therefore animals should have rights. But, if you think about it historically, the idea of having animal rights… even fifty years ago would’ve seemed absurd. So, by the same standard, maybe fifty years from now, people will want to have robot rights. In fact, there are some people already talking about it.
I think it’s a very strange idea. And often people talk about, “Oh, well, will the machines be our friends or will they be our slaves? Will they be our equals? Will they be inferior?” Actually, I think this whole way of framing things is mistaken. You know, the robots will be neither our equals nor our slaves. They will be our extensions, right?
Robots are technology, they augment us. I think it’s not so much that the machines will be conscious, but that through machines we will have a bigger consciousness—in the same way that, for example, the Internet already gives us a bigger consciousness than we had when there was no Internet.
So, discussing robots leads us to a topic that’s on the news literally every day, which is the prospect that automation and technological advances will eliminate jobs faster than it can create new ones. Or, it will eliminate jobs and replace them with inaccessible kinds of jobs. What do you think about that? What do you think the future holds?
I think we have to distinguish between the near term, by which I mean the next ten years or so, and the long term. In the near term, I think some jobs will disappear, just like jobs have disappeared to automation in the past. AI is really automation on steroids. So I think what’s going to happen in the near term is not so different from what has happened in the past.
Some jobs will be automated, so some jobs will disappear, but many new jobs will appear as well. It’s always easier to see the jobs which disappear than the ones that appear. Think for example of being an app developer. There’s millions of people today who make a living today being an app developer.
Ten years ago that job didn’t exist. Fifty years ago you couldn’t even imagine that job. Two hundred years ago, ninety-something percent of Americans were farmers, and then farming got automated. Now today only two percent of Americans work in agriculture. That doesn’t mean that the other ninety-eight percent are unemployed. They’re just doing all these jobs that people couldn’t even imagine before.
I think a lot of that is what’s going to happen here. We will see entirely new job categories appear. We will also see, on a more mundane level, more demand for lots of existing jobs. For example, I think truck drivers should be worried about the future of their jobs, because self-driving trucks are coming, so there will be an endpoint.
There are many millions of truck drivers in the US alone. It’s one of the most widespread occupations. But now, what will they do? People say, “Oh, you can’t turn truck drivers into programmers.” Well, you don’t have to turn them into programmers. Think about what’s going to happen…
Because trucks are now self-driving, goods will cost less. Goods will cost less, so people will have more money in their pockets, and they will spend it on other things—like, for example, having bigger, better houses. And therefore, there will be more demand for construction workers, and some of these truck drivers will become construction workers and so on.
You know, having said all that, I think that in the near term the most important thing that’s going to happen to jobs is actually—neither the ones that will disappear, nor the ones that will appear—most jobs will be transformed by AI. The way I do my job will change because some parts will become automated. But now I will be able to do more things better, or more than I could do before, when I didn’t have the automation. So, really the question everybody needs to think about is, what parts of my job can I automate? Really, the best way to protect your job from automation is to automate it yourself, and then ask, “What can I do using these machine learning tools?”
Automation is like having a horse. You don’t try to outrun a horse; you ride the horse. And we have to ride automation, to do our jobs better and in more ways than we can now.
So, it doesn’t sound like you’re all that pessimistic about the future of employment?
I’m optimistic, but I also worry. I think that’s a good combination. I think if we’re pessimistic we’ll never do anything. Again, if you look at the history of technology, the optimists at the end of the day are the ones who made the world a better place, not the pessimists.
But at the same time, naïve optimism is very dangerous, right? We need to worry continuously about all the things that could go wrong, and make sure that they don’t go wrong. So I think that a combination of optimism and worry is the right one to have.
Some people say we’ll find a way to merge, mentally, with the AI. Is that even a valid question? And if so, what do you think of it?
I think that’s what’s going to happen. In fact, it’s already happening. We are going to merge with our machines step-by-step. You know, like a computer is a machine that is closer to us than a television. A smartphone is closer to us than a desktop is, and the laptop is somewhere in between.
And we’re already starting to see these things such as Google Glass and augmented reality, where in essence the computer is extending our senses, and extending our part to do things. Elon Musk has this company that is going to create an interface between neurons and computers, and in fact, in research labs this already exists.
I have colleagues that work on that. They’re called brain-computer interfaces. So, step-by-step, right? The way to think about this is, we are cyborgs, right? Human beings are actually the cyborg species. From day one, we were of one with our technology.
Even our physiology would be different if we couldn’t do things like light fires and throw spears. So this has always been an ongoing process. Part of us is technology, and that will become more and more so in the future. Also with things like the Internet, we are connecting ourselves into a bigger, you know… Humanity itself is an emergent phenomenon, and having the Internet and computers allows a greater level to emerge.
And I think exactly how this happened and when, of course, is up for grabs; but that’s the way things are going.
You mentioned in passing a minute ago the singularity. Do you believe that that is what will happen, as it’s commonly thought? That there is going to be this kind of point, in the reasonably near future, from which we cannot see anything beyond it? Because we don’t have any frame of reference?
I don’t believe that the singularity will happen in those terms. So this idea of exponentially increasing progress that goes on forever… that’s not going to happen, because it’s physically impossible, right? No exponential goes on forever. It always flattens out sooner or later.
All exponentials are really what are called “S curves” in disguise. They go up faster and faster—and this is how all previous technology waves have looked—but then they flatten out, and finally they plateau.
Also, this notion that at some point things will become completely incomprehensible for us… I don’t believe that either, because there will always be parts that we understand, number one; and there are limits to what any intelligence can do, human or non-human.
By that stance, the singularity has already happened. A hundred years ago, the most advanced technology was maybe something like a car, right? And I could understand every part of how a car works, completely. Today we already have technology, like the computer systems that we have today, and nobody understands that whole system. Different people understand different parts.
With machine learning in particular, the thing that’s notable about machine learning algorithms is that they can do very complex things very well, and we have no idea how they’re doing them. And yet, we are comfortable with that, because we don’t necessarily care about the details of how it is accomplished, we just care whether the medical diagnosis was correct, or the patient’s cancer was cured, or the car is driving correctly. So I think this notion of the singularity is a little bit off.
Having said that, we are currently in the middle of one of these S curves. We are seeing very rapid progress, and by the time this has run its course, the world will be a very, very different place from what it is today.
How so?
All these things that we’ve been talking about. We will have intelligent machines surrounding us. Not just humanoid machines but intelligence on tap, right? In the same way that today you can use electricity for whatever you want just by plugging into a socket, you will be able to plug into intelligence.
And indeed, the leading tech companies are already trying to make this happen. So there will be all these things which the greater intelligence enables. Everybody will have a home robot in the same way that they have a car. We will have this whole process that the Internet is enabling, and that the intelligence on top of the Internet is enabling, and the Internet of things, and so on.
There will something like this larger emergent being, if you will, that’s not just individual human beings or just societies. But again, it’s hard to picture exactly what that would be, but this is going to happen.
You know, it always makes the news when an artificial intelligence masters some game, right? We all know the list: you had chess, and then you had Jeopardy, of course, and then you had AlphaGo, and then recently you had poker. And I get that games are kind of a natural place, because I guess it’s a confined universe with very rigid, specific rules, and a lot of training data for teaching it how to function in that. 
Are there types of problems that machine learning isn’t suited to solve? I mean, just kind of philosophically—it doesn’t matter how good your algorithms are, or how much data you have, or how fast a computer is—this is not the way to solve that particular problem.
Well, certainly some problems are much harder than others, and—as you say—games are easier in the sense that they are these very constrained, artificial universes. And that’s why AI can do so well in them. In fact, the summary of what machine learning and AI are good for today, is that they are good for these tasks which are somewhat well-defined and constrained.
What people are much better at are things that require knowledge of the world, they require common sense, they require integrating lots of different information. We’re not there yet. We don’t have the learning algorithms that can do that.
So the learning algorithms that we have today are certainly good for some things, but not others. But again, if we have the master algorithm then we will be able to do all these things, and we are making progress towards that, so, we’ll see.
Any time I see a chatbot or something that’s trying to pass the Turing test, I always type the same first question, which is: “Which is bigger, a nickel or the sun?” And not a single one of them has ever answered it correctly. 
Well, exactly, because they don’t have common sense knowledge. It’s amazing what computers can do in some ways, and it’s amazing what they can’t do in others—like these really simple pieces of common sense logic. In a way, one of the big lessons that we’ve learned in AI is that automating the job of a doctor or a lawyer is actually easy.
What is very hard to do with AI is what a three-year-old can do. If we could have a robot baby that can do what a one-year-old can do, and learn the same way, we would have solved AI. It’s much, much harder to do those things; things that we take for granted, like picking up an object, for example, or like walking around without tripping. We take this for granted because evolution spent five hundred million years developing it. It’s extremely sophisticated, but for us it’s below the conscious level.
The things for us that we are conscious of, and that we have to go to college for, well, we’re not very good at them; we just learned to do them recently. Those, the computers can do much better. So, in some ways in AI, it’s the hard things that are easy and the easy things that are hard.
Does it mean anything if something finally passes the Turing test? And if so, when do you think that might happen? When will it say, “Well, the sun is clearly bigger than a nickel?
Well, with all due respect to Alan Turing—who was a great genius and an AI pioneer—most people in AI, including me, believe that the Turing test is actually a bad idea. The reason the Turing test is a bad idea is that it confuses being intelligent with being human. This idea that you can prove that you’re intelligent by fooling a human into thinking you’re a human is very weird, if you think about it. It’s like saying an airplane doesn’t fly until it can fool birds into thinking it’s a bird. That doesn’t make any sense.
True intelligence can take many forms, not necessarily the human form. So, in some ways we don’t need to pass the Turing test to have AI. And in other ways, the Turing test is too easy to pass, and by some standards has already been passed by systems that no one would call intelligent. Talking with someone for five minutes and fooling them into thinking you’re a human is actually not that hard, because humans are remarkably adept at projecting humanity into anything that acts human.
In fact, even in the ‘60s there was this famous thing called ELIZA, that basically just picked up keywords in what you said and gave back these canned responses. And if you talked to ELIZA for five minutes, you’d actually think that it was a human.
Although Weizenbaum’s observation was, even when people knew ELIZA was just a program, they still formed emotional attachments to it, and that’s what he found so disturbing.
Exactly, so human beings have this uncanny ability to treat things as human, because that’s the only reference point that we have, right? It’s this whole idea of reasoning by analogy. If we have something that behaves even a little bit like a human—because there’s nothing else in the universe to compare it to—we start treating it more like a human and project more human qualities into it.
And, by the way, this is something that, once companies start making bots—this is already happening with chatbots like Siri and Cortana and what not, and it’ll happen even more so with home robots—there’s going to be a race to make the robots more and more humanlike. Because if you form an emotional attachment to my product, that’s what I want, right? I’ll sell more of it, and for a higher price, and so on and so forth. So, we’re going to see uncannily human-like robots and AIs—whether this is a good or bad things is another matter.
What do you think creativity is? And would an AGI, by definition, be creative, right? It could write a sonnet, or…
Yeah, an AGI, by definition, would be creative. One thing that you hear a lot these days, and that unfortunately is incorrect, is that, “Oh, we can automate these menial, routine jobs, but creativity is this deeply human thing that will never be automated.” And, this is kind of like a superficially-plausible notion, but, in fact, there are already examples of, for example, computers that can compose music.
There is this guy, David Cope, a professor at UC Santa Cruz—he has a computer program that will create music in the style of the composer of your choice. And he does this test where he plays a piece by Mozart, a piece by a human composer imitating Mozart, and a piece by his computer—by his system. And he did this at a conference that I was in, and he asked people to vote for which one was the real Amadeus, and the real one won, but the second place was actually the computer. So a computer can already write Mozart better than a professional, highly-educated human composer can.
Computers have made paintings that are actually quite beautiful and striking, many of them. Computers these days write news stories. There’s this company called Narrative Fiction that will write news stories for you. And the likes of Forbes or Fortune—I forget which one it is—actually published some of the things that they write. So it’s not a novel yet, but we will get there.
And also, in other areas, like for example chess and AlphaGo are notable examples… Both Kasparov and Lee Sedol, when they were beaten by the computer, had this remarkable reaction saying, “Wow, the computer was so creative. It came up with these moves that I would never have thought of, that seemed dumb at first but turned out to be absolutely brilliant.”
And computers have done things in mathematics, theorems and proofs and etc., all of which, if done by humans, would be considered highly creative. So, automating creativity is actually not that hard.
It’s funny, when Kasparov first said it seemed creative, what he was implying was that IBM cheated, that people had intervened. And IBM hadn’t cheated. But, that’s a testament to just how—
—There were actually two phases, right? He said that at first, so he was suspicious; because, again, how could something not human actually be doing that? But then later, after the match when he had lost and so on, if you remember, there was this move that Deep Blue made that seemed like a crazy move, and Kasparov said, like, “I could smell a new kind of intelligence playing against me.”
Which is very interesting for us AI-types, because we know exactly what was going on, right? It was these, you know, search algorithms and a whole bunch of technology that we understand fairly well. It’s interesting that from the outside this just seemed like a new kind of intelligence, and maybe it is.
He also said, “At least it didn’t enjoy beating me.” Which I guess someday, though, it may, right?
Oh, yeah, yeah! And you know that could happen depending on how we build them, right? The other very interesting thing that happened in that match—and again, I think it’s symptomatic—is that Kasparov is someone who always won by basically intimidating his opponents into submission. They just got scared of him, and then he beat them.
But the thing that happened with Deep Blue, was that Deep Blue couldn’t be intimidated by him; it was just a machine, right? As a result of which, Kasparov himself—suddenly, for the first time in his life, probably—became insecure. And then, after he lost that game, in the following game, he actually made these mistakes that he would never make, because he had suddenly become insecure.
Foreboding, isn’t it? We talked about emergence a couple of times. There’s the Gaia hypothesis that maybe all of the life on our planet has an emergent property: some kind of an intelligence that we can’t perceive, any more than our cells can perceive us. 
Do you have any thoughts on that? And do you have any thoughts on if, eventually, the Internet could just become emergent—an emergent consciousness?
Right. Like most scientists, I don’t believe in the Gaia hypothesis, in the sense that the Earth, as it is, does not have enough self-regulating ability to achieve the homeostasis that living beings do. In fact, sometimes you get these negative feedback cycles where things actually go very wrong. So, most scientists don’t believe in the Gaia hypothesis for Earth today.
Now, what I think—and a lot of other people think this is the case—is that maybe the Gaia hypothesis will be true in the future. Because as the Internet expands, and the Internet of Things—with sensors all over the place, literally all over the planet—and a lot of actions continue being taken based on those sensors to, among other things, preserve us and presumably other kinds of life on Earth… I think if we fast-forward a hundred years, there’s a very good chance that Earth will look like Gaia, but it will be a Gaia that is technological, as opposed to just biological.
And in fact, I don’t think that there’s an opposition between technology and biology. I think technology will just be the extension of biology by other means. It’s biology that’s made by us. I mean, we’re creatures, and so the things that we make are also biology, in that sense.
So if you look at it that way, maybe what has happened is that since the very beginning, Earth has been evolving towards Gaia, we just haven’t gotten there yet. But technology is very much part of getting there.
What do you think of the OpenAI initiative?
The OpenAI initiative’s goal is to do AI for the common good. Because, you know, people like Elon Musk and Sam Altman were afraid that because the biggest quantity of AI research is being done inside companies—like Google and Facebook and Microsoft and Amazon and what not—it would be owned by them. And AI is very powerful, so it’s dangerous if AI is just owned by these companies.
So, their goal is to do AI research that is going to be open, hence the name, and available to everybody. I think this is a great agenda, so I very much agree with trying to do that. I think there’s nothing wrong with having a lot of AI research in companies, but I think it’s important that there also be AI research that is in the public domain. Universities are one aspect of doing that, something like OpenAI is another example, something like the Allen Institute for AI is another example of doing AI for the public good in this way. So, I think this is a good agenda.
What they’re going to do exactly, and what their chances of succeeding are, and how their style of AI will compare to the styles of AI that are being produced by these other labs, whether industry or academia, is something that remains to be seen. But I’m curious to see what they get out of it.
The worry from some people is that… They make it analogous to a nuclear weapon, in that if you say, “We don’t know how to build one, but we can get 99% of the way there, and we’re going to share that with everybody on the planet.” And then you hope that the last little bit that makes it an AGI isn’t a bad actor of some kind. Does that make sense to you?
Yeah, yeah… I understand the analogy, but you have to remember that AI and nuclear weapons are very different for a couple of reasons. One is that nuclear weapons are essentially destructive things, right? Yeah, you can turn them into nuclear power, but they were invented to blow things up.
Whereas AI is a tool that we use to do all sorts of things, like diagnose diseases and place ads on webpages, and things from big to small. The thing is, the knowledge to build a nuclear bomb is actually not that hard to come by. Fortunately, what is very hard to come by is the enriched uranium, or plutonium, to build the bomb.
That’s actually what keeps any terrorist group from building a bomb. It’s not the lack of knowledge, it’s the lack of the materials. Now, in AI it’s actually very different. You just need computing power, and you can just plug into the cloud and get that computing power. AI is just algorithms. It’s already accessible. Lots of people can use it for whatever they want.
In a way, the safety lies in actually having AI in the hands of everybody, so that it’s not in the hands of a few. If only one person or one company had access to the master algorithm, they would be too powerful. If everybody has access to the master algorithm then there will be competition, there will be collaboration. There will be like a whole ecosystem of things that happen, and we will be safer that way, just as we are with the economy as it is. But, having said that, we will need something like an AI police.
William Gibson in Neuromancer had this thing called the Turing police, right? The Turing police are AIs whose job is to police the other AIs, to make sure that they don’t go bad, or that they get stopped when they go bad. And this is no different from what already happens. We have highways, and bank robbers can use the highways to get away. That’s no reason to not have highways, but of course the police also need to have cars so they can catch the robbers, so I think it’s going to be a similar thing with AI.
When I do these chats with people in AI, science fiction writers always come up. They always reference them, they always have their favorites and what not. Do you have any books, movies, TV shows or anything like that that you watch them and you go, “Yes, that could happen”?
Unfortunately, a lot of the depictions of AI and robots in movies and TV shows is not very realistic, because the computers and robots are really just humans in disguise. This is how you make an interesting story, is by making the robots act like humans. They have evil plans to take over the world, or somebody falls in love with them, and things like that—and that’s how you make an interesting movie.
But real AIs, as we were talking about, are very different than that. A lot of the movies that people associate with AI—like Terminator, for example—are really not stuff that will happen, but with a provision that science fiction is a great source of self-fulfilling prophecies, right? People read those things and then they try to make them happen. So, who knows.
Having said that, what is an example of a movie depicting AI that I think could happen, and is fairly interesting and realistic? Well, one example is the movie Her. The movie Her is basically about a virtual assistant that is very human-like, and ten years ago that would’ve been a very strange movie. These days we already have things like Siri, and Cortana, and Google Now, which are, of course, still a far cry from Her. But I think we’re going to get closer and closer to that.
And final question: What are you working on, and are you going to write another book? What keeps you busy?
Two things: I think we are pretty close to unifying those five master algorithms, and I’m still working on that. That’s what I’ve been working on for the last ten years. And I think we’re almost there. I think once we’re there, the next thing is that, as we’ve been talking about, that’s not going to be enough. So we need something else.
I think we need something beyond the existing five paradigms we have, and I’m working on a new type of learning that I hope will actually take us beyond what those five could do. Some people have jokingly called it the sixth paradigm, and maybe my next book will be called The Sixth Paradigm. That makes it sound like a Dan Brown novel, but that’s definitely something that I’m working on.
When you say you think the master algorithm is almost ready… Will there be a “ta-da” moment, like, here it is? Or, is it kind of a gradualism?
It’s a gradual thing. Look at physics, they’ve unified three of the forces—electromagnetism and the strong and weak forces, but they still haven’t unified gravity with them. There are proposals like string theory to do that.
These “a-ha” moments often only happen in retrospect. People propose a theory, and then maybe it gets tested, and then maybe it gets revised, and then finally when all the pieces are in place people go, “Oh, wow.” And I think it’s going to be like that with the master algorithm as well.
We have candidates, we have ways of putting these pieces together. It still remains to be seen whether they can do all the things that we want, and how well they will scale. Scaling is very important, because if it’s not scalable then it’s not really solving the problem, right? So, we’ll see.
All right, well thank you so much for being on the show. 
Thanks for having me, this was great!
Byron explores issues around artificial intelligence and conscious computers in his upcoming book The Fourth Age, to be published in April by Atria, an imprint of Simon & Schuster. Pre-order a copy here.
Byron explores issues around artificial intelligence and conscious computers in his upcoming book The Fourth Age, to be published in April by Atria, an imprint of Simon & Schuster. Pre-order a copy here.
[voices_in_ai_link_back]

Voices in AI – Episode 21: A Conversation with Nikola Danaylov

[voices_in_ai_byline]
In this episode, Byron and Nikola talk about singularity, consciousness, transhumanism, AGI and more.
[podcast_player name=”Episode 21: A Conversation with Nikola Danaylov” artist=”Byron Reese” album=”Voices in AI” url=”https://voicesinai.s3.amazonaws.com/2017-11-20-(01-05-27)-nikola-danaylov.mp3″ cover_art_url=”https://voicesinai.com/wp-content/uploads/2017/11/voices-headshot-card-3.jpg”]
[voices_in_ai_link_back]
Byron Reese: This is Voices in AI, brought to you by Gigaom. I’m Byron Reese. Today our guest is Nikola Danilov. Nikola started the Singularity Weblog, and hosts the wildly popular singularity.fm podcast. He has been called the “Larry King of the singularity.” He writes under the name Socrates, or to the Bill & Ted fans out there, Socrates. Welcome to the show, Nikola.
Nikola Danaylov: Thanks for having me, Byron, it’s my pleasure.
So let’s begin with, what is the singularity?
Well, there are probably as many definitions and flavors as there are people or experts in the field out there. But for me, personally, the singularity is the moment when machines first catch up and eventually surpass humans in terms of intelligence.
What does that mean exactly, “surpass humans in intelligence”?
Well, what happens to you when your toothbrush is smarter than you?
Well, right now it’s much smarter than me on how long I should brush my teeth.
Yes, and that’s true for most of us—how long you should brush, how much pressure you should exert, and things like that.
It gives very bad relationship advice, though, so I guess you can’t say it’s smarter than me yet, right?
Right, not about relationships, anyway. But about the duration of brush time, it is. And that’s the whole idea of the singularity, that, basically, we’re going to expand the intelligence of most things around us.
So now we have watches, but they’re becoming smart watches. We have cars, but they’re becoming smart cars. And we have smart thermostats, and smart appliances, and smart buildings, and smart everything. And that means that the intelligence of the previously dumb things is going to continue expanding, while unfortunately our own personal intelligence, or our intelligence as a species, is not.
In what sense is it a “singularity”?
Let me talk about the roots of the word. The origin of the word singularity comes from mathematics, where it basically is a problem with an undefined answer, like five divided by zero, for example. Or in physics, where it signifies a black hole. That’s to say a place where there is a rupture in the fabric of time-space, and the laws of the universe don’t hold true as we know them.
In the technological sense, we’re borrowing the term to signify the moment where humanity stops being the smartest species on our planets, and machines surpass us. And therefore, beyond that moment, we’re going to be looking into a black hole of our future. Because our current models fail to be able to provide sufficient predictions as to what happens next.
So everything that we have already is kind of going to have to change, and we don’t know which way things are going to go, which is why we’re calling it a black hole. Because you cannot see beyond the event horizon of a black hole.
Well if you can’t see beyond it, give us some flavor of what you think is going to happen on this side of the singularity. What are we going to see gradually, or rapidly, happen in the world before it happens?
One thing is the “smartification” of everything around us. So right now, we’re still living in a pretty dumb universe. But as things come to have more and more intelligence, including our toothbrushes, our cars—everything around us—our fridges, our TVs, our computers, our tables, everything. Then that’s one thing that’s going to keep happening, until we have the last stage where, according to Ray Kurzweil, quote, “the universe wakes up,” and everything becomes smart, and we end up with different things like smart dust.
Another thing will be the merger between man and machine. So, if you look at the younger generation, for example, they’re already inseparable from their smartphones. It used to be the case that a computer was the size of a building—and by the way, those computers were even weaker in terms of processing power than our smartphones are today. Even the Apollo program used a much less powerful machine to send astronauts to the moon than what we have today in our pockets.
However, that change is not going to stop there. The next step is that those machines are going to actually move inside of our bodies. So they used to be inside of buildings, then they went on our body, in our pockets, and are now becoming what’s called “wearable technology.” But tomorrow it will not be wearable anymore, because it will be embedded.
It will be embedded inside of our gut, for example, to monitor our microbiome and to monitor how our health is progressing; it will be embedded into our brains even. Basically, there may be a point where it becomes inseparable from us. That in turn will change the very meaning of the definition of being human. Not only at the sort of collective level as a species, but also at the personal level, because we are possibly, or very likely, going to have a much bigger diversification of the understanding of what it means to be a human than we have right now.
So when you talk about computers becoming smarter than us, you’re talking about an AGI, artificial general intelligence, right?
Not necessarily. The toothbrush example is artificial narrow intelligence, but as it gets to be smarter and smarter there may be a point where it becomes artificial general intelligence, which is unlikely, but it’s not impossible. And the distinction between the two is that artificial general intelligence is equal or better than human intelligence at everything, not only that one thing.
For example, a calculator today is better than us in calculations. You can have other examples, like, let’s say a smart car may be better than us at driving, but it’s not better than us at Jeopardy, or speaking, or relationship advice, as you pointed out.
We would reach artificial general intelligence at the moment when a single machine will be able to be better at everything than us.
And why do you say that an AGI is unlikely?
Oh no, I was saying that an AGI may be unlikely in a toothbrush format, because the toothbrush requires only so many particular skills or capabilities, only so many kinds of knowledge.
So we would require the AGI for the singularity to occur, is that correct?
Yeah, well that’s a good question, and there’s a debate about it. But basically the idea is that anything you can think of which humans do today, that machine would be equal or better at it. So, it could be Jeopardy, it could be playing Go. It could be playing cards. It could be playing chess. It could be driving a car. It could be giving relationship advice. It could be diagnosing a medical disease. It could be doing accounting for your company. It could be shooting a video. It could be writing a paper. It could be playing music or composing music. It could be painting an impressionistic or other kind of piece of art. It could be taking pictures equal or better than Henri Cartier-Bresson, etc. Everything that we’re proud of, it would be equal or better at.
And when do you believe we will see an AGI, and when would we see the singularity?
That’s a good question. I kind of fluctuate a little bit on that. Depending on whether we have some kind of general sort of global-scale disaster like it could be nuclear war, for example—right now the situation is getting pretty tense with North Korea—or some kind of extreme climate-related event, or a catastrophe caused by an asteroid impact; falling short of any of those huge things that can basically change the face of the Earth, I would say probably 2045 to 2050 would be a good estimate.
So, for an AGI or for the singularity? Or are you, kind of, putting them both in the same bucket?
For the singularity. Now, we can reach human-level intelligence probably by the late 2020’s.
So you think we’ll have an AGI in twelve years?
Probably, yeah. But you know, the timeline, to me, is not particularly crucial. I’m a philosopher, so the timeline is interesting, but the more important issues are always the philosophical ones, and they’re generally related to the question of, “So what?” Right? What are the implications? What happens next?
It doesn’t matter so much whether it’s twelve years or sixteen years or twenty years. I mean, it can matter in the sense that it can help us be more prepared, rather than not, so that’s good. But the question is, so what? What happens next? That’s the important issue.
For example, let me give you another crucial technology that we’re working on, which is life extension technology, trying to make humanity “amortal.” Which is to say we’re not going to be immortal—we can still die if we get ran over by a truck or something like that—but we would not be likely to die from general causes of death that we see today, which are usually old-age related.
As an individual, I’m hoping that I will be there when we develop that technology. I’m not sure I will still be alive when we have it, but as a philosopher what’s more important to me is, “So what? What happens next?” So yeah, I’m hoping I’ll be there, but even if I’m not there it is still a valid and important question to start considering and investigating right now—before we are at that point—so that we are as intellectually and otherwise prepared for events like this as possible.
I think the best guesses are, we would live to about 6,750. That’s how long it would take for some, you know, Wile E Coyote kind of piano-falling-out-the-top-floor-of-a-building-and-landing-on-you thing to happen to you, actuarially-speaking.
So let’s jump into philosophy. You’re, of course, familiar with Searle’s Chinese Room question. Let me set that up for the listeners, and then I’ll ask you to comment on it.
So it goes like this: There’s a man, we’ll call him the librarian. And he’s in this giant room that’s full of all of these very special books. And the important part, the man does not speak any Chinese, absolutely no Chinese. But people slide him questions under the door that are written in Chinese.
He takes their question and he finds the book which has the first symbol on the spine, and he finds that book and he pulls it down and he looks up the second symbol. And when he finds the second symbol and it says go to book 24,601, and so he goes to book 24,601 and looks up the third symbol and the fourth and the fifth—all the way to the end.
And when he gets to the end, the final book says copy this down. He copies these lines, and he doesn’t understand what they are, slides it under the door back to the Chinese speaker posing the question. The Chinese speaker picks it up and reads it and it’s just brilliant. I mean, it’s absolutely over-the-top. You know, it’s a haiku and it rhymes and all this other stuff.
So the philosophical question is, does that man understand Chinese? Now a traditional computer answer might be “yes.” I mean, the room, after all, passes the Turing test. Somebody outside sliding questions under the door would assume that there’s a Chinese speaker on the other end, because the answers are so perfect.
But at a gut level, the idea that this person understands Chinese—when they don’t know whether they’re talking about cholera or coffee beans or what have you—seems a bit of a stretch. And of course, the punchline of the thing is, that’s all a computer can do.
All a computer can do is manipulate ones and zeros and memory. It can just go book to book and look stuff up, but it doesn’t understand anything. And with no understanding, how can you have any AGI?
So, let me ask you this? How do you know that that’s not exactly what’s happening right now in my head? How do you know that me speaking English to you right now is not the exact process you described?
I don’t know, but the point of the setup is: If you are just that, then you don’t actually understand what we’re actually talking about. You’re just cleverly answering things, you know, it is all deterministic, but there’s, quote, “nobody home.” So, if that is the case, it doesn’t invalidate any of your answers, but it certainly limits what you’re able to do.
Well, you see, that’s a question that relates very much with consciousness. It relates to consciousness, and, “Are you aware of what you’re doing,” and things like that. And what is consciousness in the first place?
Let’s divide that up. Strictly speaking, consciousness is subjective experience. “I had an experience of doing X,” which is a completely different thing than “I have an intellectual understanding of X.” So, just the AGI part, the simple part of: does the man in the room understand what’s going on, or not?
Let’s be careful here. Because, what do you mean by “understand”? Because you can say that I’m playing chess against a computer. Do I understand the playing of chess better than a computer? I mean what do you mean by understand? Is it not understanding that the computer can play equal or better chess than me?
The computer does not understand chess in the meaningful sense that we have to get at. You know, one of the things we humans do very well is we generalize from experience, and we do that because we find things are similar to other things. We understand that, “Aha, this is similar to that,” and so forth. A computer doesn’t really understand how to play chess. It’s arguable that the computer is even playing chess, but putting that word aside, the computer does not understand it.
The computer, that program, is never going to figure out baccarat any more than it can figure out how many coffee beans Colombia should export next year. It just doesn’t have any awareness at all. It’s like a clock. You wind a clock, and tick-tock, tick-tock, it tells you the time. We progressively add additional gears to the clockwork again and again. And the thesis of what you seem to be saying is that, eventually, you add enough gears so that when you wind this thing up, it’s smarter than us and it can do absolutely anything we can do. I find that to be, at least, an unproven assumption, let alone perhaps a fantastic one.
I agree with you on the part that it’s unproven. And I agree with you that it may or may not be an issue. But it depends about what you’re going for here, and it depends on the computer you’re referring to, because we have the new software that was invented by AlphaGo to play Go. And that actually learned to play the program exactly based on the previous games—that’s to say, on the previous experience by other players. And then that same kind of approach of learning from the past, and coming up with new creative solutions to the future was then implemented in a bunch of other fields, including bioengineering, including medicine, and so on.
So when you say the computer will never be able to calculate how many beans that country needs for next season, actually it can. That’s why it’s getting more and more generalized intelligence.
Well, let me ask that question a slightly different way. So I have, hypothetically, a cat food dish that measures out cat food for my cat. And it learns, based on the weight of the food in it, the right amount to put out. If the cat eats a lot, it puts more out. If the cat eats less, it puts less out. That is a learning algorithm, that is an artificial intelligence. It’s a learning one, and it’s really no different than AlphaGo, right? So what do you think happens from the cat dish—
—I would take issue with you saying it’s really no different from AlphaGo.
Hold on, let me finish the question; I’m eager to hear what you have to say. What happens, between the cat food AI and AlphaGo and an AGI? At what point does something different happen? Where does that break, and it’s not just a series of similar technologies?
So, let me answer your question this way… When you have a baby born, it’s totally dumb, stupid, blind, and deaf. It lacks complete self-awareness. Its unable to differentiate between itself and its environment, and it lacks complete self-awareness for probably the first, arguably, year-and-a-half to two years. And there’s a number of psychological tests that can be administered as the child develops. Usually girls, by the way, do about three to six months better, or they develop personal awareness faster and earlier than boys, on average. But let’s say the average age is about a year-and-a-half to two years—and that’s a very crude estimation, by the way. The development of AI would not be exactly the same, but there will be parallels.
The question you’re raising is a very good question. I don’t have a good answer because, you know, that can only happen with direct observational data—which we don’t have right now to answer your question, right? So, let’s say tomorrow we develop artificial general intelligence. How would we know that? How can we test for that, right? We don’t know.
We’re not even sure how we can evaluate that, right? Because just as you suggested, it could be just a dumb algorithm, processing just like your algorithm is processing how much cat food to provide to your cat. It can lack complete self-awareness, while claiming that it has self-awareness. So, how do we check for that? The answer is, it’s very hard. Right now, we can’t. You don’t know that I even have self-awareness, right?
But, again, those are two different things, right? Self-awareness is one thing, but an AGI is easy to test for, right? You give a program a list of tasks that a human can do. You say, “Here’s what I want you to do. I want you to figure out the best way to make espresso. I want you to find the Waffle House…” I mean, it’s a series of tasks. There’s nothing subjective about it, it’s completely objective.
Yes.
So what has happened between the cat food example, to the AlphaGo, to the AGI—along that spectrum, what changed? Was there some emergent property? Was there something that happened? Because you said the AlphaGo is different than my cat food dish, but in a philosophical sense, how?
It’s different in the sense that it can learn. That’s the key difference.
So does my cat food thing, it gives the cat more food some days, and if the cat’s eating less, it cuts the cat food back.
Right, but you’re talking just about cat food, but that’s what children do, too. Children know nothing when they come into this world, and slowly they start learning more and more. They start reacting better, and start improving, and eventually start self-identifying, and eventually they become conscious. Eventually they develop awareness of the things not only within themselves, but around themselves, etc. And that’s my point, is that it is a similar process; I don’t have the exact mechanism to break down to you.
I see. So, let me ask you a different question. Nobody knows how the brain works, right? We don’t even know how thoughts are encoded. We just use this ubiquitous term, “brain activity,” but we don’t know how… You know, when I ask you, “What was the color of your first bicycle?” and you can answer that immediately, even though you’ve probably never thought about it, nor do you have some part of your brain where you store first bicycles or something like that.
So, assuming we don’t know that, and therefore we don’t really know how it is that we happen to be intelligent. By what basis do you say, “Oh, we’re going to build a machine that can do something that we don’t even know how we do,” and even put a timeline on it, to say, “And it’s going to happen in twelve years”?
So there are a number of ways to answer your question. One is, we don’t necessarily need to know. We don’t know how we create intelligence when we have babies, too, but we do it. How did it happen? It happened through evolution; so, likewise, we have what are called “evolutionary algorithms,” which are basically algorithms that learn to learn. And the key point, as Dr. Stephen Wolfram proved years ago in his seminal work Mathematica, from very simple things, very complex patterns can emerge. Look at our universe; it emerged from tiny little, very simple things.
Actually I’m interviewing Lawrence Krauss next week, he says it emerged from nothing. So from nothing, you have the universe, which has everything, according to him at least. And we don’t know how we create intelligence in the baby’s case, we just do it. Just like you don’t know how you grow your nails, or you don’t know how you grow your hair, but you do it. So, likewise, just one of the many different paths that we can take to get to that level of intelligence is through evolutionary algorithms.
By the way, this is what’s sometimes referred to as the black box problem, and AlphaGo is a bit of an example of that. There are certain things we know, and there are certain things we don’t know that are happening. Just like when I interviewed David Ferrucci, who was the team leader behind Watson, we were talking about, “How does Watson get this answer right and that answer wrong?” His answer is, “I don’t really know, exactly.” Because there are so many complicated things coming together to produce an answer, that after a certain level of complexity, it becomes very tricky to follow the causal chain of events.
So yes, it is possible to develop intelligence, and the best example for that is us. Unless you believe in that sort of first-mover, God-is-the-creator kind of thing, that somebody created us—you can say that we kind of came out of nothing. We evolved to have both consciousness and intelligence.
So likewise, why not have the same process only at the different stratum? So, right now we’re biologically-based; basically it’s DNA code replicating itself. We have A, C, T, and G. Alternatively, is it inconceivable that we can have this with a binary code? Or even if not binary, some other kind of mathematical code, so you can have intelligence evolve—be it silicone-based, be it photon-based, or even organic processor-based, be it quantum computer-based… what have you. Right?
So are you saying that there could be no other stratum, and no other way that could ever hold intelligence other than us? Then my question to you will be, well what’s the evidence of that claim? Because I would say that we have the evidence that it’s happened once. We could therefore presume that it could not be necessarily limited to only once. We’re not that special, you know. It could possibly happen again, and more than once.
Right, I mean it’s certainly a tenable hypothesis. The Singularians, for the most part, don’t treat it as a hypothesis, they treat it as a matter of faith.
That’s why I’m not such a good Singularitarian.
They say, “We have achieved consciousness and an AGI. We have a general intelligence. Therefore, we must be able to build one.” You don’t generally apply that logic to anything else in life, right? There is a solar system, therefore we must be able to build one. There is a third dimension, we must be able to build one.
With almost nothing else in life do you do it, and yet people who talk about the singularity, and are willing to put a date on it, by the way, to them there’s nothing up for debate. Even though all the things that are required for it are completely unknown, how we achieved them.
Let me give you Daniel Dennett’s take on things, for example. He says that consciousness doesn’t exist. That it is self-delusion. He actually makes a very, very good argument about it, per se. I’ve been trying to get him on my podcast for a while. But he says it’s total self-fabrication, self-delusion. It doesn’t exist. It’s beside the point, right?
But he doesn’t deny that we’re intelligent though. He just says that what we call “consciousness” is just brain activity. But he doesn’t say, “Oh, we don’t really have a general intelligence, either.” Obviously, we’re intelligent.
Exactly. But that’s kind of what you’re trying to imply with the machines, because they will be intelligent in the sense that they will be able to problem-solve anything that we’re able to problem-solve, as we pointed out—whether it’s chess, whether it’s cat food, whether it’s playing or composing the tenth symphony. That’s the point.
Okay, well that’s at least unquestionably the theory.
Sure.
So let’s go from there. Talk to me about Transhumanism. You write a lot about that. What do you think we’ll be able to do? And if you’re willing to say, when do you think we’ll be able to do it? And, I mean, a man with a pacemaker is a Transhuman, right? He can’t live without it.
I would say all of us are already cyborgs, depending on your definition. If you say that the cyborg is an organism consisting of, let’s say, organic and inorganic parts working together in a single unit, then I would answer that if you have been vaccinated, you’re already a cyborg.
If you’re wearing glasses, or contact lenses, you’re already a cyborg. If you’re wearing clothes and you can’t survive without them, or shoes, you’re already a cyborg, right? Because, let’s say for me, I am severely short-sighted with my eyesight. I’m like, -7.25 or something crazy like that. I’m almost kind of blind without my contacts. Almost nobody knows that, unless people listen to these interviews, because I wear contacts, and for all intents and purposes I am as eye-capable as anybody else. But take off my contacts and I’ll be blind. Therefore you have one single unit between me and that inorganic material, which basically I cannot survive without.
I mean, two hundred years ago, or five hundred years ago, I’d probably be dead by now, because I wouldn’t be able to get food. I wouldn’t be able to survive in the world with that kind of severe shortsightedness.
The same with vaccinations, by the way. We know that the vast majority of the population, at least in the developed world, has at least one, and in most cases a number of different vaccines—already by the time you’re two years old. Viruses, basically, are the carriers for the vaccines. And viruses straddle that line, that gray area between living and nonliving things—the hard-to-classify things. They become a part of you, basically. You carry those vaccine antibodies, in most cases, for the rest of your life. So I could say, according to that definition, we are all cyborgs already.
That’s splitting a hair in a very real sense though. It seems from your writing you think we’re going to be doing much more radical things than that; things which, as you said earlier, call into question whether or not we’re even human anymore. What are those things, and why does that affect our definition of “human”?
Let me give you another example. I don’t know if you’ve seen in the news, or if your audience has seen in the news, a couple of months ago the Chinese tried to modify human embryos with CRISPR gene-editing technology. So we are not right now at the stage where, you know… It’s been almost 40 years since we had the first in vitro babies. At the time, basically what in vitro meant was that you do the fertilization outside of the womb, into a petri dish or something like that. And then you watch the division process begin, and then you select—by basically visual inspection—what looks to be the best-fertilized egg, simply by visual examination. And that’s the egg that you would implant.
Today, we don’t just observe; we actually we can preselect. And not only that, we can actually go in and start changing things. So it’s just like when you’re first born, you start learning the alphabet, then you start reading full words; then you start reading full sentences; and then you start writing yourself.
We’re doing, currently, exactly that with genetics. We were starting to just identify the letters of the alphabet thirty, or forty, or fifty years ago. Then we started reading slowly; we read the human genome about fifteen years ago. And now we’re slowly starting to learn to write. And so the implication of that is this: how does the meaning of what it means to be human change, when you can change your sex, color, race, age, and physical attributes?
Because that’s the bottom line. When we can go and make changes at the DNA level of an organism, you can change all those parameters. It’s just like programming. In computer science it’s 0 and 1. In genetics it’s ATCG, four letters, but it’s the same principle. In one case, you’re programming a software program for a computer; in the other case, you’re programming living organisms.
But in that example, though, everybody—no matter what race you are—you’re still a human; no matter what gender you are, you’re still a human.
It depends how you qualify “human,” right? Let’s be more specific. So right now, when you say “humans,” what you mean actually is Homo sapiens, right? But Homo sapiens has a number of very specific physical attributes. When you start changing the DNA structure, you can actually change those attributes to the point where the result doesn’t carry those physical attributes anymore. So are you then Homo sapiens anymore?
From a biological point of view, the answer will most likely depend on how far you’ve gone. There’s no breakpoint, though, and different people will have a different red line to cross. You know for some, just a bit. So let’s say you and your wife or partner want to have a baby. And both of you happen to be carriers of a certain kind of genetic disease that you want to avoid. You want to make sure, before you conceive that baby, the fertilized egg doesn’t carry that genetic material.
And that’s all you care about, that’s fine. But someone else will say, that’s your red line, whereas my red line is that I want to give that baby the good looks of Brad Pitt, I want to give it the brain of Stephen Hawking, and I want to give it the strength of a weightlifter, for example. Each person who is making that choice would go for different things, and would have different attributes that they would choose to accept or not to accept.
Therefore, you would start having that diversification that I talked about in the beginning. And that’s even before you start bringing in things like neural cognitive implants, etc.—which would be basically the merger between men and machine, right? Which basically means that you can have both parallel developments of biotech or genetics. Our biological evolution and development, accelerated, on the other hand; and on the other hand, you can have the merger of that with the acceleration and evolution and improvement of computer technology and neurotech. When you put those two things together, you end up with a final entity which is nothing like what we are today, and it definitely would not fit the definition of being human.
Do you worry, at some level, that it’s taken us five thousand years of human civilization to come up with this idea that there are things called human rights? That there are these things you don’t do to a person no matter what. That you’re born with them, and because you are human, you have these rights.
Do you worry that, for better or worse, what you’re talking about will erode that? That we will lose this sense of human rights, because we lose some demarcation of a human is.
That’s a very complicated question. I would suggest people read Yuval Harari’s book Homo Deus on that topic, and the previous one was called Sapiens. Those two are probably the best two books that I’ve read in the last ten years. But basically, the idea of human rights is an idea that was born just a couple hundred years ago. It came to exist with humanism, and especially liberal humanism. Right now, if you see how it’s playing out, humanism is kind of taking what religion used to do, in the sense of that religion used to put God in the center of everything—and then, since we were his creation, everything else was created for us, to serve us.
For example the animal world, etc., and we used to have the Ptolemaic idea of the universe, where the earth was the center, and all of those things. Now, what humanism is doing is putting the human in the center of the universe, and saying humanity has this primacy above everything else, just because of our very nature. Just because you are human, you have human rights.
I would say that’s an interesting story, but if we care about that story we need to push it even further.
In our present context, how is that working out for everyone else other than humanity? Well the moment we created humanism and invented human rights, we basically made humanity divine. We took the divinity from God, and gave it to humanity, but we downgraded everybody else. So animals which, back in the day—let’s say the hunter-gatherer society—we considered ourselves to be equal and on par with the animals.
Because you see, one day I would kill you and eat you, next day maybe a tiger would eat me. That’s how the world was. But now, we downgraded all the animals to machine—they don’t have consciousness, they don’t have any feelings, they lack self-awareness—and therefore we can enslave and kill them any way we wish and like.
So as a result, we pride ourselves on our human rights and things like that, and yet we enslave and kill seventy to seventy-five billion animals every year, and 1.3 trillion sea organisms like fish, annually. So the question then is, if we care so much about rights, why should they be limited only to human rights? Are we saying that other living organisms are incapable of suffering? I’m a dog owner, I have a seventeen-and-a-half-year-old dog. She’s on her last leg. She actually had a stroke last weekend.
I can tell you that she has taught me that she possesses the full spectrum of happiness and suffering that I do, pretty much. Even things like jealousy, and so on, she demonstrated to me multiple times, right? Yet, we today use that idea of humanism and human rights to defend ourselves and enslave everybody else.
I would suggest it’s time to expand that and say, first, to our fellow animals, that we need to include them, that they have their own rights, first of all. Second of all, that possibly rights should not be limited to organic organisms, and should not be called human or animal rights, but they should be called intelligence rights, or even beyond intelligence—any kind of organism that can exhibit things like suffering and happiness and pleasure and pain.
Because obviously, there is a different level of intelligence between me and my dog—we would hope—but she’s able to suffer as much as I am, and I’ve seen it. And that’s true especially more for whales and great apes and stuff like that, which we have brought to the brink of extinction right now. We want to be special, that’s what religion does to us. That’s what humanism did with human rights.
Religion taught us that we’re special because God created us in his own image. Then humanism said there is no God, we are the God, so we took the place of God—we took his throne and said, “We’re above everybody else.” That’s a good story, but it’s nothing more than a story. It’s a myth.
You’re a vegan, correct?
Yes.
How far down would you extend these rights? I mean, you have consciousness, and then below that you have sentience, which is of course a misused word. People use “sentience” to mean intelligence, but sentience is the ability to feel something. In your world, you would extend rights at some level all the way down to anything that can feel?
Yeah, and look: I’ve been a vegan for just over a year and a couple of months, let’s say fourteen months. So, just like any other human being, I have been, and still am, very imperfect. Now, I don’t know exactly how far we should expand that, but I would say we should stop immediately at the level we can easily observe that we’re causing suffering.
If you go to a butcher shop, especially an industrialized farming butcher shop, where they kill something like ten thousand animals per day—it’s so mechanized, right? If you see that stuff in front of your eyes, it’s impossible not to admit that those animals are suffering, to me. So that’s at least the first step. I don’t know how far we should go, but we should start at the first steps, which are very visible.
What do you think about consciousness? Do you believe consciousness exists, unlike Dan Dennett, and if so where do you think it comes from?
Now you’re putting me on the spot. I have no idea where it comes from, first of all. You know, I am atheist, but if there’s one religion that I have very strong sympathies towards, that would be Buddhism. I particularly value the practice of meditation. So the question is, when I meditate—and it only happens rarely that I can get into some kind of deep meditation—is that consciousness mine, or am I part of it?
I don’t know. So I have no idea where it comes from. I think there is something like consciousness. I don’t know how it works, and I honestly don’t know if we’re part of it, or if it is a part of us.
Is it at least a tenable hypothesis that a machine would need to be conscious, to be an AGI?
I would say yes, of course, but the next step, immediately, is how do we know if that machine has consciousness or not? That’s what I’m struggling with, because one of the implications is that the moment you accept, or commit to that kind of definition, that we’re only going to have AGI if it has consciousness, then the question is, how do we know if and when it has consciousness? An AGI that’s programmed to say, “I have consciousness,” well how do you know if it’s telling the truth, and if it’s really conscious or not? So that’s what I’m struggling with, to be more precise in your answers.
And mind you, I have the luxury of being a philosopher, and that’s also kind of the negative too—I’m not an engineer, or a neuroscientist, so…
But you can say consciousness is required for an AGI, without having to worry about, well how do we measure it, or not.
Yes.
That’s a completely different thing. And if consciousness is required for an AGI, and we don’t know where human consciousness comes from, that at least should give us an enormous amount of pause when we start talking about the month and the day when we’re going to hit the singularity.
Right, and I agree with you entirely, which is why I’m not so crazy about the timelines, and I’m staying away from it. And I’m generally on the skeptical end of things. By the way, for the last seven years of my journey I have been becoming more and more skeptical. Because there are other reasons or ways that the singularity…
First of all, the future never unfolds the way we think it will, in my opinion. There’s always those black swan events that change everything. And there are issues when you extrapolate, which is why I always stay away from extrapolation. Let me give you two examples.
The easy example is when you have positive, or let’s say negative extrapolation. We have people such as Lord Kelvin—he was the president of the British Royal Society, one of the smartest people—who wrote a book in the 1890’s about how heavier-than-air aircraft are impossible to build.
The great H.G. Wells wrote, just in 1902, that heavier-than-air aircraft are totally impossible to build, and he’s a science fiction writer. And yet, a year later the Wright brothers, two bicycle makers, who probably never read Lord Kelvin’s book, and maybe didn’t even read any of H.G. Wells’ science fiction novels, proved them both wrong.
So people were extrapolating negatively from the past. Saying, “Look, we’ve tried to fly since the time of Icarus, and the myth of Icarus is a warning to us all: we’re never going to be able to fly.” But we did fly. So we didn’t fly for thousands of years, until one day we flew. That’s one kind of extrapolation that went wrong, and that’s the easy one to see.
The harder one is the opposite, which is called positive extrapolation. From 1903 to, let’s say, the late 1960s, we went from the Wright brothers, to the moon. People said—amazing people, like Arthur C. Clarke—said, well if we made it from 1903 to the late 1960s to the moon, by 2002 we will be beyond Mars; we will be outside of our solar system.
That’s positive extrapolation. Based on very good data for, let’s say, sixty-five years from 1903 to 1968—very good data—you saw tremendous progress in aerospace technology. We went to the moon several times, in fact, and so on and so on. So it was logical to extrapolate that we would be by Mars and beyond, today. But actually, the opposite happened. Not only did we not reach Mars by today, we are actually unable to get back to the moon, even. As Peter Thiel says in his book, we were promised flying cars and jetpacks, but all we got was 140 characters.
In other words, beware of extrapolations, because they’re true until they’re not true. You don’t know when they are going to stop being true, and that’s the nature of black swan sorts of things. That’s the nature of the future. To me, it’s inherently unknowable. It’s always good to have extrapolations, and to have ideas, and to have a diversity of scenarios, right?
That’s another thing which I agree with you on: Singularians tend to embrace a single view of the future, or a single path to the future. I have a problem with that myself. I think that there’s a cone of possible futures. There are certainly limitations, but there is a cone of possibilities, and we are aware of only a fraction of it. We can extrapolate only in a fraction of it, because we have unknown unknowns, and we have black swan phenomena, which can change everything dramatically. I’ve even listed three disaster scenarios—like asteroids, ecological collapse, or nuclear weapons—which can also change things dramatically. There are many things that we don’t know, that we can’t control, and that we’re not even aware of that can and probably will change the actual future from the future we think will happen today.
Last philosophical question, and then I’d like to chat about what you’re working on. Do you believe humans have free will?
Yes. So I am a philosopher, and again—just like with the future—there are limitations, right? So all the possible futures stem from the cone of future possibilities derived from our present. Likewise, our ability to choose, to make decisions, to take action, have very strict limitations; yet, there is a realm of possibilities that’s entirely up to us. At least that’s what I’m inclined to think. Even though most scientists that I meet and interview on my podcast are actually one level, or one degree or another degree, of determinist.
Would an AGI need to have free will in order to exist?
Yes, of course.
Where do you think human free will comes from? If every effect had a cause, and every decision had a cause—presumably in the brain—whether it’s electrical or chemical or what have you… Where do you think it comes from?
Yeah, it could come from quantum mechanics, for example.
That only gets you randomness. That doesn’t get you somehow escaping the laws of physics, does it?
Yes, but randomness can be sort of a living-cat and dead-cat outcome, at least metaphorically speaking. You don’t know which one it will be until that moment is there. The other thing is, let’s say, you have fluid dynamics, and with the laws of physics, we can predict how a particular system of gas, will behave within the laws of fluid dynamics. But it’s impossible to predict how a single molecule or atom will behave within that system. In other words, if the laws of the universe and the laws of physics set the realm of possibilities, then within that realm, you can still have free will. So, we are such tiny minuscule little parts of the system, as individuals, that we are more akin to atoms, if not smaller particles than that.
Therefore, we can still be unpredictable.
Just like it’s unpredictable, by the way, with quantum mechanics, to say, “Where is the electron located?” and if you try to observe it, then you are already impacting on the outcome. You’re predetermining it, actually, when you try to observe it, because you become a part of the system. But if you’re not observing it, you can create a realm of possibilities where it’s likely to be, but you don’t know exactly where it is. Within that realm, you get your free will.
Final question: Tell us what you’re working on, what’s exciting to you, what you’re reading about… I see you write a lot about movies. Are there any science fiction movies that you think are good ones to inform people on this topic? Just talk about that for a moment.
Right. So, let me answer backwards. In terms of movies—it’s been awhile since I’ve watched it, but I actually even wrote a review on in—one of the movies that I really enjoyed watching, it’s by the Wachowskis, and it’s called “Cloud Atlas.” I don’t think that movie was very successful at all, to be honest with you.
I’m not even sure if they managed to recover the money they invested in it, but in my opinion it was one of the top ten best movies I’ve ever seen in my life. Because it’s a sextet—so it had six plots progressing in a parallel fashion, in six different timelines. So six things happening in six different locations in six different epochs, with six different timelines, with tremendous actors, and it touched on a lot of those future technologies, and even the meaning of being human—what separates us from the others, and so on.
I would suggest people check out “Cloud Atlas.” One of my favorite movies. The previous question you asked was, what am I working on?
Mm-hmm.
Well, to be honest, I just finished my first book three months ago or something. I launched it on January 23rd I think. So I’ve been basically promoting my book, traveling, giving speeches, trying to raise awareness about the issues, and the fact that, in my view, we are very unprepared—as a civilization, as a society, as individuals, as businesses, and as governments.
We are going to witness a tremendous amount of change in the next several decades, and I think we’re grossly unprepared. And I think, depending on how we handle those changes, with genetics, with robotics, with nanotech, with artificial intelligence—even if we never reach the level of artificial general intelligence, by the way, that’s beside the point to me—just the changes we’re going to witness as a result of the biotech revolution can actually put our whole civilization at risk. They’re not just only going to change the meaning of what it is to be human, they would put everything at risk. All of those things converging together, in the narrow span of several decades basically, I think, create this crunch point which could be what some people have called a “pre-singularity future,” which is one possible answer to the Fermi Paradox.
Enrico Fermi was this very famous Italian mathematician who, a few decades ago, basically observed that there are two-hundred billion galaxies just in the observable realm of the universe. And each of those two-hundred billion galaxies has two-hundred billion stars. In other words, there’s almost an endless number of exoplanets like ours—which are located in the Goldilocks area, where it’s not too hot or too cold—which can potentially give birth to life. The question then is, if there are so many planets and so many stars and so many places where we can have life, where is everybody? Where are all the aliens? There’s a diversity of answers to that question. But at least one of those possible scenarios, to explain this paradox, is what’s referred to as the pre-singularity future. Which is to say, in each civilization, there comes a moment where its technological prowess surpasses its capacity to control it. Then, possibly, it self-destructs.
So in other words, what I’m saying is that it may be an occurrence which happens on a regular basis in the universe. It’s one way to explain the Fermi Paradox, and it’s possibly the moment that we’re approaching right now. So it may be a moment where we go extinct like dinosaurs; or, if we actually get it right—which right now, to be honest with you, I’m getting kind of concerned about—then we can actually populate the universe. We can spread throughout the universe, and as Konstantin Tsiolkovsky said, “Earth is the cradle of humanity, but sooner or later, we have to leave the cradle.” So, hopefully, in this century we’ll be able to leave the cradle.
But right now, we are not prepared—neither intellectually, nor technologically, nor philosophically, nor ethically, not in any way possible, I think. That’s why it’s so important to get it right.
The name of your book is?
Conversations with the Future: 21 Visions for the 21st Century.
All right, Nikola, it’s been fascinating. I’ve really enjoyed our conversation, and I thank you so much for taking the time.
My pleasure, Byron.
Byron explores issues around artificial intelligence and conscious computers in his upcoming book The Fourth Age, to be published in April by Atria, an imprint of Simon & Schuster. Pre-order a copy here.
[voices_in_ai_link_back]