Voices in AI – Episode 70: A Conversation with Jakob Uszkoreit

[voices_in_ai_byline]

About this Episode

Episode 70 of Voices in AI features host Byron Reese and Jakob Uszkoreit discuss machine learning, deep learning, AGI, and what this could mean for the future of humanity. Jakob has a masters degree in Computer Science and Mathematics from Technische Universität Berlin. Jakob has also worked at Google for the past 10 years currently in deep learning research with Google Brain.
Visit www.VoicesinAI.com to listen to this one-hour podcast or read the full transcript.

Transcript Excerpt

Byron Reese: This is Voices in AI, brought to you by GigaOm. I’m Byron Reese. Today our guest is Jakob Uszkoreit, he is a researcher at Google Brain, and that’s kind of all you have to say at this point. Welcome to the show, Jakob.
Let’s start with my standard question which is: What is artificial intelligence, and what is intelligence, if you want to start there, and why is it artificial?
Jakob Uszkoreit: Hi, thanks for having me. Let’s start with artificial intelligence specifically. I don’t think I’m necessarily the best person to answer the question what intelligence is in general, but I think for artificial intelligence, there’s possibly two different kind of ideas that we might be referring to with that phrase.
One is kind of the scientific or the group of directions of scientific research, including things like machine learning, but also other related disciplines that people commonly refer to with the term ‘artificial intelligence.’ But I think there’s this other maybe more important use of the phrase that has become much more common in this age of the rise of AI if you want to call it that, and that is what society interprets that term to mean. I think largely what society might think when they hear the term artificial intelligence, is actually automation, in a very general way, and maybe more specifically, automation where the process of automating [something] requires the machine or the machines doing so to make decisions that are highly dynamic in response to their environment and in our ideas or in our conceptualization of those processes, require something like human intelligence.
So, I really think it’s actually something that doesn’t necessarily, in the eyes of the public, have that much to do with intelligence, per se. It’s more the idea of automating things that at least so far, only humans could do, and the hypothesized reason for that is that only humans possess this ephemeral thing of intelligence.
Do you think it’s a problem that a cat food dish that refills itself when it’s empty, you could say has a rudimentary AI, and you can say Westworld is populated with AIs, and those things are so vastly different, and they’re not even really on a continuum, are they? A general intelligence isn’t just a better narrow intelligence, or is it?
So I think that’s a very interesting question. Whether basically improving and slowly generalizing or expanding the capabilities of narrow intelligences, will eventually get us there, and if I had to venture a guess, I would say that’s quite likely actually. That said, I’m definitely not the right person to answer that. I do think that guesses, that aspects of things are today still in the realms of philosophy and extremely hypothetical.
But the one trick that we have gotten good at recently that’s given us things like AlphaZero, is machine learning, right? And it is itself a very narrow thing. It basically has one core assumption, which is the future is like the past. And for many things it is: what a dog looks like in the future, is what a dog looked like yesterday. But, one has to ask the question, “How much of life is actually like that?” Do you have an opinion on that?
Yeah so I think that machine learning is actually evolving rapidly from the initial classic idea of basically trying to predict the future just in the past, and not just the past as a kind of encapsulated version of the past. So it’s basically a snapshot captured in this fixed static data set. You expose machines to that, you allow it to learn from that, train on that, whatever you want to call it, and then you evaluate how the resulting model or machine or network does in the wild or on some evaluation tasks, and tests that you’ve prepared for it.
It’s evolving from that classic definition towards something that is quite a bit more dynamic, that is starting to incorporate learning in situ, learning kind of “on the job,” learning from very different kinds of supervision, where some of it might be encapsulated by data sets, but some might be given to the machine through somewhat more high level interactions, maybe even through language. There is at least a bunch of lines of research attempting that. Also quite importantly, we’re starting slowly but surely to employ machine learning in ways where the machine’s actions actually have an impact on the world, from which the machine then keeps learning. I think that that’s actually something [for which] all of these parts are necessary ingredients, if we ever want to have narrow intelligences, that maybe have a chance of getting more general. Maybe then in the more distant future, might even be bolted together into somewhat more general artificial intelligence.
Listen to this one-hour episode or read the full transcript at www.VoicesinAI.com
[voices_in_ai_link_back]
 
Byron explores issues around artificial intelligence and conscious computers in his new book The Fourth Age: Smart Robots, Conscious Computers, and the Future of Humanity.

Voices in AI – Episode 64: A Conversation with Eli David

[voices_in_ai_byline]

About this Episode

Episode 64 of Voices in AI features host Byron Reese and Dr. Eli David discuss evolutionary computation, deep learning and neural networks, as well as AI’s role in improving cyber-security. Dr. David is the CTO and co-founder of Deep Instinct as well as having published multiple papers on deep learning and genetic algorithms in leading AI journals.
Visit www.VoicesinAI.com to listen to this one-hour podcast or read the full transcript.

Transcript Excerpt

Byron Reese: This is Voices in AI, brought to you by GigaOm. I’m Byron Reese. And today, our guest is Dr. Eli David. He is the CTO and the co-founder of Deep Instinct. He’s an expert in the field of computational intelligence, specializing in deep learning and evolutionary computation. He’s published more than 30 papers in leading AI journals and conferences, mostly focusing on applications of deep learning and genetic algorithms in various real-world domains. Welcome to the show, Eli.
Eli David: Thank you very much. Great to be here.
So bring us up to date, or let everybody know what do we mean by evolutionary computation, and deep learning and neural networks? Because all three of those are things that, let’s just say, they aren’t necessarily crystal clear in everybody’s minds what they are. So let’s begin by defining your terms. Explain those three concepts to us.
Sure, definitely. Now, both neural networks and evolutionary computation take inspiration from intelligence in nature. If instead of trying to come up with smart mathematical ways of creating intelligence, we just look at the nature to see how intelligence works there, we can reach two very obvious conclusions. First, the only algorithm that is in charge of creating intelligence – we started from single-cell organisms billions of years ago, and now we are intelligent organisms – and the main algorithm, or maybe the only algorithm, in charge of that was evolution. So evolutionary computation takes inspiration from the evolutionary process in the nature and trying to evolve computer programs so that, from one generation to other, they will become smarter and smarter, and the smarter they are, the more they breed, the more children they have, and so, hopefully the smart gene improves one generation after the other.
The other thing that we will notice when we observe nature is brains. Nearly all the intelligence in humans or other mammals or the intelligent animals, it is due to a neural network and network of neurons which we refer to as a brain — many small processing units connected to each other via what we call synapses. In our brains, for example, we have many tens of billions of such neurons, each one of them, on average, connected to about ten thousand other neurons, and these small processing units connected to each other, they create the brain; they create all our intelligence. So the two fields of evolutionary computation and artificial neural networks, nowadays referred to as deep learning, and we will shortly dwell on the difference as well, take direct inspiration from nature.
Now, what is the difference between deep learning, deep neural networks, traditional neural networks, etc? So, neural networks is not a new field. Already in the 1980s, we had most of the concepts that we have today. But the main difference is that during the past several years, we had several major breakthroughs, while until then, we could train only shallow neural networks, shallow artificial neural networks, just a few layers of neurons, just a few thousand synapses, connectors. A few years ago, we managed to make these neural networks deep, so instead of a few layers, we have many tens of layers; instead of a few thousand connectors, we have now hundreds of millions, or billions, of connectors. So instead of having shallow neural networks, nowadays we have deep neural networks, also known as deep learning. So deep learning and deep neural networks are synonyms.
Listen to this one-hour episode or read the full transcript at www.VoicesinAI.com
[voices_in_ai_link_back]
 
Byron explores issues around artificial intelligence and conscious computers in his new book The Fourth Age: Smart Robots, Conscious Computers, and the Future of Humanity.

Voices in AI – Episode 63: A Conversation with Hillery Hunter

[voices_in_ai_byline]

About this Episode

Episode 63 of Voices in AI features host Byron Reese and Hillery Hunter discuss AI, deep learning, power efficiency, and understanding the complexity of what AI does with the data it is fed. Hillery Hunter is an IBM Fellow and holds an MS and a PhD in electrical engineering from the University of Illinois Urbana-Champaign.
Visit www.VoicesinAI.com to listen to this one-hour podcast or read the full transcript.

Transcript Excerpt

Byron Reese: This is Voices in AI brought to you by GigaOm, I’m Byron Reese. Today, our guest is Hillery Hunter. She is an IBM Fellow, and she holds an MS and a PhD in electrical engineering from the University of Illinois Urbana-Champaign. Welcome to the show, Hillery.
Thank you it’s such a pleasure to be here today, looking forward to this discussion, Byron.
So, I always like to start off with my Rorschach test question, which is: what is artificial intelligence, and why is it artificial?
You know that’s a great question. My background is in hardware and in systems and in the actual compute substrate for AI. So one of the things I like to do is sort of demystify what AI is. There are certainly a lot of definitions out there, but I like to take people to the math that’s actually happening in the background. So when we talk about AI today, especially in the popular press and such and people talk about the things that AI is doing, be it understanding medical stands or labelling people’s pictures on a social media platform, or understanding speech or translating language, all those things that are considered core functions of AI today are actually deep learning, which means using many layered neural networks to solve a problem.
There’s also other parts of AI though, that are much less discussed in popular press, which include knowledge and reasoning and creativity and all these other aspects. And you know the reality is where we are today with AI, is we’re seeing a lot of productivity from the deep learning space and ultimately those are big math equations that are solved with lots of matrix math, and we’re basically creating a big equation that matches in its parameters to a set of data that it was fed.
So, would you say though that that it is actually intelligent, or that it is emulating intelligence, or would you say there’s no difference between those two things?
Yeah, so I’m really quite pragmatic as you just heard from me saying, “Okay, let’s go talk about what the math is that’s happening,” and right now where we’re at with AI is relatively narrow capabilities. AI is good at doing things like classification or answering yes and no kind of questions on data that it was fed and so in some sense, it’s mimicking intelligence in that it is taking in sort of human sensory data a computer can take in. What I mean by that is it can take in visual data or auditory data, people are even working on sensory data and things like that. But basically a computer can now take in things that we would consider sort of human process data, so visual things and auditory things, and make determinations as to what it thinks it is, but certainly far from something that’s actually thinking and reasoning and showing intelligence.
Well, staying squarely in the practical realm, that approach, which is basically, let’s look at the past and make guesses about the future, what is the limit of what that can do? I mean, for instance, is that approach going to master natural language for instance? Can you just feed a machine enough printed material and have it be able to converse? Like what are some things that model may not actually be able to do?
Yeah, you know it’s interesting because there’s a lot of debate. What are we doing today that’s different from analytics? We had the big data era, and we talked about doing analytics on the data. What’s new and what’s different and why are we calling it AI now? To refer to your question from that direction, one of the things that AI models do, be it anything from a deep learning model to something that’s more in the knowledge reasoning area, is that they’re much better interpolators, they’re much better able to predict on things that they’ve never seen before.
Classical rigid models that people programmed in computers, could answer “Oh, I’ve seen that thing before.” With deep learning and with more modern AI techniques, we are pushing forward into computers and models being able to guess on things that they haven’t exactly seen before. And so in that sense there’s a good amount of interpolation influx, whether or not and how AI pushes into forecasting on things well outside the bounds of what it’s never seen before and moving AI models to be effective at types of data that are very different from what they’ve seen before, is the type of advancement that people are really pushing for at this point.
Listen to this one-hour episode or read the full transcript at www.VoicesinAI.com
[voices_in_ai_link_back]
 
Byron explores issues around artificial intelligence and conscious computers in his new book The Fourth Age: Smart Robots, Conscious Computers, and the Future of Humanity.

Voices in AI – Episode 62: A Conversation with Atif Kureishy

[voices_in_ai_byline]

About this Episode

Episode 62 of Voices in AI features host Byron Reese and Atif Kureishy discussing AI, deep learning, and the practical examples and implications in the business market and beyond. Atif Kureishy is the Global VP of Emerging Practices at Think Big, a Teradata company. He also has a B.S. in physics and math from the University of Maryland as well as an MS in distributive computing from Johns Hopkins University.
Visit www.VoicesinAI.com to listen to this one-hour podcast or read the full transcript.

Transcript Excerpt

Byron Reese: This is Voices in AI, brought to you by GigaOm, I’m Byron Reese. Today my guest is Atif Kureishy. He is the Global VP of Emerging Practices, which is AI and deep learning at Think Big, a Teradata company. He holds a BS in Physics and Math from the University of Maryland, Baltimore County, and an MS in distributive computing from the Johns Hopkins University. Welcome to the show Atif.
Atif Kureishy: Welcome, thank you, appreciate it.
So I always like to start off by just asking you to define artificial intelligence.
Yeah, definitely an important definition, one that unfortunately is overused and stretched in many different ways. Here at Think Big we actually have a very specific definition within the enterprise. But before I give that, for me in particular, when I think of intelligence, that conjures up the ability to understand, the ability to reason, the ability to learn, and we usually equate that to biological systems, or living entities, and now with the rise of probably more appropriate machine intelligence, we’re applying the term ‘artificial’ to it, and the rationale is probably because machines aren’t living and they’re not biological systems.
So with that, the way we’ve defined AI in particular is: leveraging machine and deep learning to drive towards a specific business outcome. And it’s about giving leverage for human workers, to enable higher degrees of assistance and higher degrees of automation. And when we define AI in that way, we actually give it three characteristics. Those three characteristics are: the ability to sense and learn, and so that’s being able to understand massive amounts to data and demonstrate continuous learning, and detecting patterns and signals within the noise, if you will. And the second is being able to reason and infer, and that is driving intuition and inference with increasing accuracy again to maximize a business outcome or a business decision. And then ultimately it’s about deciding and acting, so actioning or automating a decision based on everything that’s understood, to drive towards more informed activities that are based on corporate intelligence. So that’s kind of how we view AI in particular.
Well I applaud you for having given it so much thought, and there’s a lot there to unpack. You talked about intelligence being about understanding and reasoning and learning, and that was even in your three areas. Do you believe machines can reason?
You know, over time, we’re going to start to apply algorithms and specific models to the concept of reasoning, and so the ability to understand, the ability to learn, are things that we’re going to express in mathematical terms no doubt. Does it give it human lifelike characteristics? That’s still something to be determined.
Well I don’t mean to be difficult with the definition because, as you point out, most people aren’t particularly rigorous when it comes to it. But if it’s to drive an outcome, take a cat food dish that refills itself when it’s low, it can sense, it can reason that it should put more food in, and then it can act and release a mechanism that refills the food dish, is that AI, in your understanding, and if not why isn’t that AI?
Yeah, I mean I think in some sense it checks a lot of the boxes, but the reality is, being able to adapt and understand what’s occurring, for instance if that cat is coming out during certain times of the day ensuring that meals are prepared in the right way and that they don’t sit out and become stale or become spoiled in any way, and that is signs of a more intelligent type of capability that is learning the behaviors and anticipating how best to respond given a specific outcome it’s driving towards.
Got you. So now, to take that definition, your company is Think Big. What do you think big about? What is Think Big and what do you do?
So looking back in history a little bit, Think Big was actually an acquisition that Teradata had done several years ago, in the big data space, and particularly around open source and consulting. And over time, Teradata had made several acquisitions and now we’ve unified all of those various acquisitions into a unified group, called Think Big Analytics. And so what we’re particularly focused on is how do we drive business outcomes using advanced analytics and data science. And we do that through a blend of approaches and techniques and technology frankly.
Listen to this one-hour episode or read the full transcript at www.VoicesinAI.com
[voices_in_ai_link_back]
 
Byron explores issues around artificial intelligence and conscious computers in his new book The Fourth Age: Smart Robots, Conscious Computers, and the Future of Humanity.

5 Common Misconceptions about AI

In recent years I have ran into a number of misconceptions regarding AI, and sometimes when discussing AI with people from outside the field, I feel like we are talking about two different topics. This article is an attempt at clarifying what AI practitioners mean by AI, and where it is in its current state.
The first misconception has to do with Artificial General Intelligence, or AGI:

  1. Applied AI systems are just limited versions of AGI

Despite what many think,the state of the art in AI is still far behind human intelligence. Artificial General Intelligence, i.e. AGI, has been the motivating fuel for all AI scientists from Turing to today. Somewhat analogous to Alchemy, the eternal quest for AGI that replicates and exceeds human intelligence has resulted in the creation of many techniques and scientific breakthroughs. AGI has helped us understand facets of human and natural intelligence, and as a result, we’ve built  effective algorithms inspired by our understanding and models of them.
However, when it comes to practical applications of AI, AI practitioners do not necessarily restrict themselves to pure models of human decision making, learning, and problem solving. Rather, in the interest of solving the problem and achieving acceptable performance, AI practitioners often do what it takes to build practical systems. At the heart of the algorithmic breakthroughs that resulted in Deep Learning systems, for instance, is a technique called back-propagation. This technique, however, is not how the brain builds models of the world. This brings us to the next misconception:

  1. There is a one-size-fits-all AI solution.

A common misconception is that AI can be used to solve every problem out there–i.e. the state of the art AI has reached a level such that minor configurations of ‘the AI’ allows us to tackle different problems. I’ve even heard people assume that moving from one problem to the next makes the AI system smarter, as if the same AI system is now solving both problems at the same time. The reality is much different: AI systems need to be engineered, sometimes heavily,  and require specifically trained models in order to be applied to a problem. And while similar tasks, especially those involving sensing the world (e.g., speech recognition, image or video processing) now have a library of available reference models, these models need to be specifically engineered to meet deployment requirements and may not be useful out of the box. Furthermore, AI systems are seldom the only component of AI-based solutions. It often takes many tailor-made classically programed components to come together to augment one or more AI techniques used within a system. And yes, there are a multitude of different AI techniques out there, used alone or in hybrid solutions in conjunction with others, therefore it is incorrect to say:

  1. AI is the same as Deep Learning

Back in the day, we thought the term artificial neural networks (ANNs) was really cool. Until, that is, the initial euphoria around it’s potential backfired due to its lack of scaling and aptitude towards over-fitting. Now that those problems have, for the most part, been resolved, we’ve avoided the stigma of the old name by “rebranding” artificial neural networks as  “Deep Learning”. Deep Learning or Deep Networks are ANNs at scale, and the ‘deep’ refers not to deep thinking, but to the number of hidden layers we can now afford within our ANNs (previously it was a handful at most, and now they can be in the hundreds). Deep Learning is used to generate models off of labeled data sets. The ‘learning’ in Deep Learning methods refers to the generation of the models, not to the models being able to learn real-time as new data becomes available. The ‘learning’ phase of Deep Learning models actually happens offline, needs many iterations, is time and process intensive, and is difficult to parallelize.
Recently, Deep Learning models are being used in online learning applications. The online learning in such systems is achieved using different AI techniques such as Reinforcement Learning, or online Neuro-evolution. A limitation of such systems is the fact that the contribution from the Deep Learning model can only be achieved if the domain of use can be mostly experienced during the off-line learning period. Once the model is generated, it remains static and not entirely robust to changes in the application domain. A good example of this is in ecommerce applications–seasonal changes or short sales periods on ecommerce websites would require a deep learning model to be taken offline and retrained on sale items or new stock. However, now with platforms like Sentient Ascend that use evolutionary algorithms to power website optimization, large amounts of historical data is no longer needed to be effective, rather, it uses neuro-evolution to shift and adjust the website in real time based on the site’s current environment.   
For the most part, though, Deep Learning systems are fueled by large data sets, and so the prospect of new and useful models being generated from large and unique datasets has fueled the misconception that…

  1. It’s all about BIG data

It’s not. It’s actually about good data. Large, imbalanced datasets can be deceptive, especially if they only partially capture the data most relevant to the domain. Furthermore, in many domains, historical data can become irrelevant quickly. In high-frequency trading in the New York Stock Exchange, for instance, recent data is of much more relevance and value than, for example data from before 2001, when they had not yet adopted decimalization.
Finally, a general misconception I run into quite often:

  1. If a system solves a problem that we think requires intelligence, that means it is using AI

This one is a bit philosophical in nature, and it does depend on your definition of intelligence. Indeed, Turing’s definition would not refute this. However, as far as mainstream AI is concerned, a fully engineered system, say to enable self-driving cars, which does not use any AI techniques, is not considered an AI system. If the behavior of the system is not the result of the emergent behavior of AI techniques used under the hood, if programmers write the code from start to finish, in a deterministic and engineered fashion, then the system is not considered an AI-based system, even if it seems so.
AI paves the way for a better future
Despite the common misconceptions around AI, the one correct assumption is that AI is here to stay and is indeed, the window to the future. AI still has a long way to go before it can be used to solve every problem out there and to be industrialized for wide scale use. Deep Learning models, for instance, take many expert PhD-hours to design effectively, often requiring elaborately engineered parameter settings and architectural choices depending on the use case. Currently, AI scientists are hard at work on simplifying this task and are even using other AI techniques such as reinforcement learning and population-based or evolutionary architecture search to reduce this effort. The next big step for AI is to make it be creative and adaptive, while at the same time, powerful enough to exceed human capacity to build models.  
by Babak Hodjat, co-founder & CEO Sentient Technologies

How AI can help detect & filter offensive images and videos

Advances in Artificial Intelligence and Deep Learning have transformed the way computers understand images and videos. Over the last few years, innovative neural network structures and high-end hardware have helped research teams achieve groundbreaking results in object detection and scene description. Those structures have in turned been used to build generalist models aiming to recognize any object in any image.
Those breakthroughs are now being applied to specific use-cases, one of which is Content Moderation. Sightengine, an A.I. company, is making its image and video moderation service available worldwide through a simple API. Built upon specialist Neural Networks, the API analyzes incoming images or videos and detects if they contain offensive material such as nudity, adult content or suggestive scenes. Just like Human moderators would. As opposed to the networks that companies like Google, Facebook or Microsoft use for object detection, these neural networks are specialists, designed and trained to excel at one specific task.
Historically, content moderation has been mostly in demand with Dating and Social Networking websites. They relied either on staff who had to go through user-submitted content manually, or on their community who had to flag and report content. But today, content moderation is no longer restricted to niche markets. As camera-equipped smartphones have become ubiquitous, and as the usage of social networks and self-expression tools have continued to rise, photo generation and sharing have literally exploded over the last few years.
It is estimated that more than 3 Billion images are shared every day online, along with millions of hours of video streams. Which is why more and more app owners, publishers and developers are looking for solutions to make sure their audience and users are not exposed to unwanted content. This is a moral as well as a legal imperative, and is key to building a product users trust and like.
Sightengine’s Image Moderation and Nudity Detection Technology is a ready-to-use SaaS offering, accessible via a simple and fast API.

How PayPal uses deep learning and detective work to fight fraud

Hui Wang has seen the nature of online fraud change a lot in the 11 years she’s been at PayPal. In fact, a continuous evolution of methods is kind of the nature of cybercrime. As the good guys catch onto one approach, the bad guys try to avoid detection by using another.

Today, said Wang, PayPal’s senior director of global risk sciences, “The fraudsters we’re interacting with are… very unique and very innovative. …Our fraud problem is a lot more complex than anyone can think of.”

In deep learning, though, Wang and her team might have found a way to help level the playing field between PayPal and criminals who want exploit the online payment platform.

Deep learning is a somewhat new approach to machine learning and artificial intelligence that has caught fire over the past few years thanks to companies such as [company]Google[/company], [company]Facebook[/company], [company]Microsoft[/company] and Baidu, and a handful of prominent researchers (some of whom now work for those companies). The field draws a lot of comparisons to the workings of the human brain because deep learning systems use artificial neural network algorithms, although “inspired by the brain” might be a more accurate description than “modeled after the brain.”

How DeepFace sees Calista Flockhart. Source: Facebook

A visual diagram of a deep neural network for facial recognition. Source: Facebook

Essentially, the stacks of neural networks that comprise deep learning models are very good at recognizing patterns and features of the data they’re trained on, which has led to some huge advances in computer vision, speech recognition, text analysis, machine listening and even video-game playing in the past few years. You can learn more about the field at our Structure Data conference later this month, which includes deep learning and artificial intelligence experts from Facebook, Microsoft, Yahoo, Enlitic and other companies.

It turns out deep learning models are also good at identifying the complex patterns and characteristics of cybercrime and online fraud. Machine-learning-based pattern recognition has long been a major part of fraud detection practices, but Wang said PayPal has seen a “major leap forward” in its abilities since it began investigating precursor (what she calls “non-linear”) techniques to deep learning several years ago. PayPal has been working with deep learning itself for the past two or three years, she said.

Some of these efforts are already running in production as part of the company’s anti-fraud systems, often in conjunction with human experts in what Wang describes as a “detective-like methodology.” The deep learning algorithms are able to analyze potentially tens of thousands of latent features (time signals, actors and geographic location are some easy examples) that might make up a particular type of fraud, and are even able to detect “sub modus operandi,” or different variants of the same scheme, she said.

Some of PayPal's fraud-management options for developers.

Some of PayPal’s fraud-management options for developers.

The patterns are much more complex than “If someone does X, then the result is Y,” so it takes artificial intelligence to analyze them at a level much deeper than humans can. “Actually,” Wang said, “that’s the beauty of deep learning.”

Once the models detect possible fraud, human “detectives” can get to work assessing what’s real, what’s not and what to do next.

PayPal uses a champions-and-challengers approach to deciding which fraud-detection models to rely on most heavily, and deep learning is very close to becoming the champion. “We’ve seen roughly a 10 percent delta on top of today’s champion,” Wang said, which is very significant.

And as the fraudulent behavior on PayPal’s platform continues to grow more complex, she’s hopeful deep learning will give her team the ability to adapt to these new patterns faster than before. It’s possible, for example, that PayPal might some day be able to deploy models that take live data from its system and become smarter, by retraining themselves, in real time.

“We’re doing that to a certain degree,” Wang said, “but I think there’s still more to be done.”

IBM acquires deep learning startup AlchemyAPI

So much for AlchemyAPI CEO Elliot Turner’s statement that his company is not for sale. IBM has bought the Denver-based deep learning startup that delivers a wide variety of text analysis and image recognition capabilities via API.

IBM plans to integrate AlchemyAPI’s technology into the core Watson cognitive computing platform. IBM will also use AlchemyAPI’s technology to expand its set of Watson cloud APIs that let developers infuse their web and mobile applications with artificial intelligence. Eventually, the AlchemyAPI service will shut down as the capabilities are folded into the IBM Bluemix platform, said IBM Watson Group vice president and CMO Stephen Gold said.

Elliot Turner — CEO, AlchemyAPI; Stephen Gold, Watson Solutions, IBM Software Group. Structure Data 2014

Love at first sight? AlchemyAPI CEO Elliot Turner (left) and IBM Watson vice president Stephen Gold (center) at Structure Data 2014.

Compared with Watson’s primary ability to draw connections and learn from analyzing textual data, AlchemyAPI excels at analyzing text for sentiment, category and keywords, and for recognizing objects and faces in images. Gold called the two platforms “a leather shoe fit” in terms of how well they complement each other. Apart from the APIs, he said AlchemyAPI’s expertise in unsupervised and semi-supervised learning systems (that is, little human oversight over model creation) will be a good addition to the IBM team.

We will discuss the burgeoning field of new artificial intelligence applications at our Structure Data conference later this month in New York, as well as at our inaugural Structure Intelligence event in September.

I have written before that cloud computing will be the key to IBM deriving the types of profits it wants to from Watson, as cloud developers are the new growth area for technology vendors. Cloud developers might not result in multi-million-dollar deals, but they represent a huge user base in aggregate and, more importantly, can demonstrate the capabilities of a platform like Watson probably better than IBM itself can. AlchemyAPI already has more than 40,000 developers on its platform.

Other companies delivering some degree of artificial intelligence and deep learning via the cloud, and sometimes via API, include Microsoft, Google, MetaMind, Clarifai and Expect Labs.

celebrity_chadsmith_willferrell_cropped (1)

AlchemyAPI’s facial recognition API can distinguish between Will Ferrell and Red Hot Chili Peppers drummer Chad Smith.

AlchemyAPI’s Turner said his company decided to join IBM, after spurning numerous acquisition offers and stating it wasn’t for sale, in part because it represents an opportunity to “throw rocket fuel on” the company’s long-term goals. Had the plan been to buy AlchemyAPI, kill its service and fold the team into research roles — like what happens with so many other acquisitions of deep learning talent — it probably would not have happened.

Gold added that IBM is not only keeping the AlchemyAPI services alive (albeit as part of the Bluemix platform) but also plans to use the company’s Denver headquarters as the starting point of an AI and deep learning hub in the city.

[protected-iframe id=”447ad6f774bfa076438dfe73b2d084db-14960843-6578147″ info=”https://www.youtube.com/embed/iHVeoJBtoIM?list=PLZdSb0bQCA7mpVy–2jQBxfjcbNrp9Fu4″ width=”640″ height=”390″ frameborder=”0″ allowfullscreen=””]

Update: This post was updated at 9:10 a.m. to include quotes and information from Elliot Turner and Stephen Gold.

Google, Stanford say big data is key to deep learning for drug discovery

A team of researchers from Stanford University and Google have released a paper highlighting a deep learning approach they say shows promise in the field of drug discovery. What they found, essentially, is that that more data covering more biological processes seems like a good recipe for uncovering new drugs.

Importantly, the paper doesn’t claim a major breakthrough that will revolutionize the pharmaceutical industry today. It simply shows that by analyzing a whole lot of data across a whole lot of different target processes — in this case, 37.8 million data points across 259 tasks — seems to work measurably better for discovering possible drugs than does analyzing smaller datasets and/or building models specifically targeting a single a task. (Read the Google blog post for a higher-level, but still very-condensed explanation.)

But when talking about a process in drug discovery that can take years and cost drug companies billions of dollars that ultimately make their way into the prices of prescription drugs, any small improvement helps.

This graph shows a measure of prediction accuracy (ROC AUC is the area under the receiver operating characteristic curve) for virtual screening on a fixed set of 10 biological processes as more datasets are added.

This graph shows a measure of prediction accuracy (ROC AUC is the area under the receiver operating characteristic curve) for virtual screening on a fixed set of 10 biological processes as more datasets are added.

Here’s how the researchers explain the reality, and the promise, of their work in the paper:

The efficacy of multitask learning is directly related to the availability of relevant data. Hence, obtaining greater amounts of data is of critical importance for improving the state of the art. Major pharmaceutical companies possess vast private stores of experimental measurements; our work provides a strong argument that increased data sharing could result in benefits for all.

More data will maximize the benefits achievable using current architectures, but in order for algorithmic progress to occur, it must be possible to judge the performance of proposed models against previous work. It is disappointing to note that all published applications of deep learning to virtual screening (that we are aware of) use distinct datasets that are not directly comparable. It remains to future research to establish standard datasets and performance metrics for this field.

. . .

Although deep learning offers interesting possibilities for virtual screening, the full drug discovery process remains immensely complicated. Can deep learning—coupled with large amounts of experimental data—trigger a revolution in this field? Considering the transformational effect that these methods have had on other fields, we are optimistic about the future.

If they’re right, we might look back on this research as part of a handful of efforts that helped spur an artificial intelligence revolution in the health care space. Aside from other research in the field, there are multiple startups, including Butterfly Network and Enlitic (which will be presenting at our Structure Data conference later this month in New York) trying to improve doctors’ ability to diagnose diseases using deep learning. Related efforts include the work IBM is doing with its Watson technology to analyze everything from cancer to PTSD, as well as from startups like Ayasdi and Lumiata.

There’s no reason that researchers have to stop here, either. Deep learning has proven remarkably good at tackling machine perception tasks such as computer vision and speech recognition, but the approach can technically excel at more general problems involving pattern recognition and feature selection. Given the right datasets, we could soon see deep learning networks identifying environmental factors and other root causes of disease that would help public health officials address certain issues so doctors don’t have to.

Microsoft is building fast, low-power neural networks with FPGAs

Microsoft on Monday released a white paper explaining a current effort to run convolutional neural networks — the deep learning technique responsible for record-setting computer vision algorithms — on FPGAs rather than GPUs.

Microsoft claims that new FPGA designs provide greatly improved processing speed over earlier versions while consuming a fraction of the power of GPUs. This type of work could represent a big shift in deep learning if it catches on, because for the past few years the field has been largely centered around GPUs as the computing architecture of choice.

If there’s a major caveat to Microsoft’s efforts, it might have to do with performance. While Microsoft’s research shows FPGAs consuming about one-tenth the power of high-end GPUs (25W compared with 235W), GPUs still process images at a much higher rate. Nvidia’s Tesla K40 GPU can do between 500 and 824 images per second on one popular benchmark dataset, the white paper claims, while Microsoft predicts its preferred FPGA chip — the Altera Arria 10 — will be able to process about 233 images per second on the same dataset.

However, the paper’s authors note that performance per processor is relative because a multi-FPGA cluster could match a single GPU while still consuming much less power: “In the future, we anticipate further significant gains when mapping our design to newer FPGAs . . . and when combining a large number of FPGAs together to parallelize both evaluation and training.”

In a Microsoft Research blog post, processor architect Doug Burger wrote, “We expect great performance and efficiency gains from scaling our [convolutional neural network] engine to Arria 10, conservatively estimated at a throughput increase of 70% with comparable energy used.”

fpgacnn

This is not Microsoft’s first rodeo when it comes deploying FPGAs within its data centers, and in fact is a corollary of an earlier project. Last summer, the company detailed a research project called Catapult in which it was able to improve the speed and performance of Bing’s search-ranking algorithms by adding FPGA co-processors to each server in a rack. The company intends to port production Bing workloads onto the Catapult architecture later this year.

There have also been other attempts to port deep learning algorithms onto FPGAs, including one by State University of New York at Stony Brook professors and another by Chinese search giant Baidu. Ironically, Baidu Chief Scientist, and deep learning expert, Andrew Ng is big proponent of GPUs, and the company claims a massive GPU-based deep learning system as well as a GPU-based supercomputer designed for computer vision. But this needn’t be and either/or situation: companies could still use GPUs to maximize performance while training their models, and then port them to FPGAs for production workloads.

Expect to hear more about the future of deep learning architectures and applications at Gigaom’s Structure Data conference March 18 and 19 in New York, which features experts from Facebook, Microsoft and elsewhere. Our Structure Intelligence conference, September 22-23 in San Francisco, will dive even deeper into deep learnings, as well as the broader field of artificial intelligence algorithms and applications.