If you want to navigate data’s deep waters, you need a dashboard

In countless movies that depict our future digital lives, larger-than-life dashboards loom, delivering whatever information you seek at the touch of a screen. Every data point is visually compelling, easily digestible, and able to convert into myriad formats. Microsoft took a stab at such a future years ago with Surface, a product that seems to have evolved into PixelSense but hasn’t seen much traction since. So, much like flying cars, where are the touchscreens encompassing entire walls?

G_F5Kp_j_400x400A startup called Dive, the latest entrant in the data visualization space, is taking a stab at this vision. The company launched at CES, with the aim of equipping Fortune 500 companies with interactive dashboards surfacing insights and trends in visually compelling graphics.

Intending to “change the way media is purchased,” according to Deb Hall, founder and CEO, Dive consolidates data sources for brands and turns them into eye-catching marketing intelligence that is “accessible, fun, and entertaining.”  It’s all real-time and meant to help marketers not only digest trend data but turn it into actionable content marketing about their industry and competitors. This area is something that Buzzfeed Director of Data Science Ky Harlin might have a few things to say about at Structure Data in March in an interview with Gigaom founder Om Malik.

twg-image-b (1)

Say an ad agency based in Austin has a big pitch with Universal Music Group coming up. That agency can display all the area searches around UMG musicians, alongside trending hashtags, Spotify plays, and Facebook likes. Taking data from across the Web ensures trends and insights that aren’t confined to just one channel. Austinites love this particular song because they’re streaming it constantly on YouTube, for instance.

DIVE - JNJ News Hashtags

Dive arrived at CES with some heavy hitters in tow. Its first content partner is Google’s online platform for marketers, Think with Google, with installations at agencies and brands. Pulling data from Google searches and YouTube, the Dive platform will curate content and trends from Think with Google and deliver it in a visually compelling format. Leslee D’Antonio, Managing Editor of Think with Google called Dive “a unique and engaging way to distribute insights in real-time.” And in an onsite demonstration at CES, LinkedIn collaborated with Dive to show off ‘A Year in Review – 2014 Trending Topics and Articles’ alongside some of the world’s top brands, for OMD Worldwide.

Data visualization is an increasingly crowded space. It’s about as hot a market as you can find right now, one we write about often. Gravity is a key competitor to Dive and they’re a formidable one, recently acquired by AOL. Gravity was also ranked first by Gigaom analysts in a 2013 Sector Roadmap report on content personalization. Dive is also competing in the content marketing sector, which has gone from hot to overcrowded to a subject about which everyone professes expertise.

Self-funded, not yet profitable but earning revenue, Dive sees 2015 as a big growth year, adding to its stable of both clients and employees. They also plan to build further on premium products, Dive Desktop, Mobile, and Interactive.

The data scraping and coalescing is the easy part (like I can talk). The winner of the visualization battle will be the company with the best designers. The reason the category exists to begin with is because humans need eye candy to process, especially when it comes to correlating multiple factors. Dive was smart in making a Creative Director one of its first hires and it shows in its product. If it keeps its focus on the visual, it could have a chance against AOL.

Real-time data analytics could change sports forever

It was early in the 2015 NFC championship football game last Sunday when Green Bay Packers coach Mike McCarthy was twice confronted with a tough decision: should the Packers, having put the Seattle Seahawks — owners of the National Football League’s best defense — on its heels in front of its raucous home crowd, go for a touchdown on fourth-and-short from near the goal line? Or should they settle for the easy field goal attempt?

McCarthy chose to take the three points on two separate occasions, despite the agonized screams of Structure Data event lead (and Wisconsin native) Derrick Harris into his television thousands of miles away, urging McCarthy to roll the dice. That decision did not work out for the Packers, who eventually succumbed to Seattle in overtime after the defending Super Bowl champs mounted a furious comeback in the last three minutes of the game.

Criticizing coaches with the benefit of hindsight is a time-honored sports media tradition. But we’re finally getting to the point where gut instinct is starting to look foolish given the reams of data available to coaches of all sports; especially in the NFL, contractually obligated to show off Microsoft’s Surface 3 tablets in as many ways as possible. I’m talking about stuff way beyond the New York Times’ Fourth-Down Bot.

What if McCarthy had access to real-time data that showed how Seattle’s defense was responding (or not) to a ferocious Green Bay drive that got them deep into enemy territory? What if he could seize upon that data to identify a weakness in the secondary within the 40-second play clock and and hurry-up offense (which makes it almost impossible for the defense to substitute) to call a play designed to exploit that weakness, one that he could feel much more confident about employing than the standard generic run up the middle on a goal-line situation?

The proliferation of data and mobile computing has made it quite possible for NFL coaches to start employing such tactics, and that’s something I want to explore on stage at Structure Data when I have the opportunity to interview Krish Dasgupta, vice president of data analytics and technology for sports broadcasting giant ESPN, and Bill Squadron, executive vice president for pro analytics at stats giant Stats Inc., on March 18th in New York.

Coaches have always sought to exploit weaknesses in an opponent in their game planning. And ever since Bill James and Billy Beane woke up the stodgy baseball world to the power of statistics and data analysis, the science of player evaluation and scouting has been changed forever.

But I’m really curious about the impact of in-game data analysis. We explored this a little at Structure last year, when Booz-Allen Hamilton showed off some of the work it had done analyzing Major League Baseball pitchers and their tendencies to throw certain pitches in certain situations. But there is so much potential here, both for teams themselves and broadcasters like ESPN seeking to give its viewers more insight into the game.

Take the Packers-Seahawks game: What if McCarthy (or Fox, broadcasting the game) had access to real-time data about how the Seahawks defense was reacting to the Packers’ drive. Were the linebackers hesitating on a certain snap cadence? Did they tend to blitz in situations in which the offense deployed a certain formation? Were certain members of the defense more fatigued then others, something you could assess by seeing how fast they were running downfield on pass plays?


Football coaches are obsessed with preparation, and go over hours and hours of film in the days leading up to a game. They also review pictures of a given drive within a game once the offense or defense gets back to the sidelines. But imagine a situation in which McCarthy, Green Bay offensive coordinator Tom Clements, and quarterback Aaron Rodgers had access to a real-time heads-up database (projected onto Rodgers’ face mask, perhaps) that provided accurate information about the condition of the defense cross-referenced against the Packers playbook?

That might have changed McCarthy’s thinking in those crucial fourth-down situations, had he been able to ascertain weaknesses in Seattle’s defensive approach. And, of course, Seattle’s defense wouldn’t be standing still: if it could tell its linebackers that the right tackle is getting off the snap a half-second slower than the rest of the line, that could translate into a huge advantage.

I’m also interested in the ability of real-time data analysis to monitor the health of professional football players. Anyone who has grown up with the NFL has been witness to an uneasy transition in which players have gotten so much bigger and faster while playing basically the same game with basically the same equipment. Even Iron Mike Ditka isn’t sure he would allow his kids to play football at this point, knowing what we know now: data (especially real-time data) could give us a much better sense of what is actually happening to these athletes at all levels of the game and how we can best protect them.

As a fan, I’m not entirely sure how real-time data analysis would affect the game. Do we really need computers to make all of our decisions for us? I’d like to think that NFL competitors would use data analysis as a tool, and not a crutch, when analyzing how best to proceed in games that are still subject to a huge degree of randomness.

But as a journalist, these are fascinating topics. I hope that Dasgupta and Squadron are willing to share some interesting insights with the Structure Data audience, and if you’d like to hear what they have to say in person in New York this March, you can buy tickets here.

The promise of big data still looms, but execution lags

When something is hyped as much as the notion of big data, there’s bound to be disappointment when results don’t meet expectations right this second.

That realization — that implementation of big data analytics and related technologies hasn’t matched expectations — is a common thread across a recent spate of research reports. While corporate execs now “get” the possible impact of aggregating and analyzing all the data their companies generate, very few companies have realized that potential.

Most adoption remains at pilot or test stage

A new McKinsey Quarterly report acknowledged that earlier predictions that retailers would parlay big data analytics to boost operating margins by more than 60 percent, and that the healthcare sector could likewise use the technology to slice costs 8 percent, haven’t played out.

While massively scaled companies like [company]Amazon[/company] and [company]Google[/company] use data analytics to wring out significant costs, data analytics success at most legacy companies is limited to a few test projects or narrow pieces of the overall business. Very few of those accounts “have achieved what we would call ‘big impact through big data,’ or impact at scale,” according to McKinsey.

CEOs have an inflated view of their own big data projects

According to a just-released survey of 362 executives worldwide, there is a pronounced disconnect between what the CEO of a company perceives is happening with that company’s big data efforts and what the rest of the organization sees.

From the key findings of the work conducted The Economist Intelligence unit for [company]Teradata[/company],:

While 47 percent of CEOs believe that all employees have access to the data they need, only 27 percent of all respondents agree that they do. Similarly, 43 percent of CEOs think relevant data are captured and made available in real time, compared to 29 percent of all respondents. CEOs are also more likely to think that employees extract relevant insights from data – 38 percent of them hold this belief, as compared to 24 percent of all respondents and only 19 percent of senior vice presidents, vice presidents and directors.

So, while there is indeed a ton of available data, it’s still hard for most companies to parlay it into useful insights. More than half (57 percent) of the total respondents said their companies do a poor job in this respect, although almost everyone agreed that access to data and the ability to wring actionable insights out of it are critical.

Paranoia isn’t driving enough action

A whopping 60 percent of 226 execs surveyed by [company]Capgemini[/company] Consulting, in yet another report, said that big data will “disrupt” their industry in the next three years but only 13 percent of them have big data implementations in production. And, less than a third of those responding (27 percent) described their own big data initiatives as successful; 8 percent described them as very successful.

So while progress has been made — as Derrick Harris and Fast Forward Labs’ Founder Hilary Mason noted on this week’s Structure Show podcast — executives definitely know what Hadoop is now and have a grasp of what big data can do, much heavy lifting still needs to be done to realize the benefits of analyzing all that data.

For more on this topic of the latest and greatest technologies and how they are put to use — and the cognitive dissonance between big data hype and real-world adoption — will be discussed by top names in the field at Gigaom’s Structure Data event  in March.

Big data implementations status


To hear Fast Forward Labs’ founder Hilary Mason on how that company combs academia, open-source and “outsider art” sources for big data applications, check out the second half of the podcast below:

[soundcloud url=”https://api.soundcloud.com/tracks/187259451?secret_token=s-4LM4Z” params=”color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false” width=”100%” height=”166″ iframe=”true” /]

Data privacy isn’t dead with the internet of things, just different

Even as websites, wearable computers and, increasingly, every piece of technology we touch gathers and analyzes our data, there’s still hope that privacy will survive. Making that case, however, might mean working from a different definition of privacy than we’re used to.

One cold, hard fact about data privacy is that the data-collection ship sailed long ago, never to return. With limited exceptions, consumers can’t really stop tech companies from collecting data about them. When we log into web services, make phone calls, play our favorite apps or buy the latest in connected jewelry, we’re giving those companies the right to collect just about whatever information they please about who we are and how we use their products.

The situation isn’t wholly good or bad — data analysis is behind lots of user experience improvements as well as targeted ads, for example —  but understanding it is critical to understanding what the future of data privacy might look like. There’s not much point in debating what companies can or should collect (because doing so is too easy and regulating it is so hard), but there is an opportunity to put some limits on what companies do with data once they have it.

This why the White House, as part of its new consumer privacy push unveiled on Monday morning, is talking about how student data is used and smart grid data is secured rather than what’s collected. It’s why Federal Trade Commission chairperson Edith Ramirez, speaking about the internet of things at last week’s Consumer Electronics Show, spoke about how long companies should store user data and not whether they should collect it.


The internet of things, in fact, is a prime example of why we’ll probably never be able to put a lid on data collection: because many people actually crave it. The whole point of connected devices is that they collect our data and do something with it, presumably something that users view as beneficial. If I love my fitness tracker or my smart thermostat, I can’t really be upset that it’s sucking up my data.

What I can be upset about, however, is when the company does something unethical or negligent with my data, or something I didn’t agree to (at least constructively) in the privacy policy. It seems this is where a lot of regulatory energy is now being spent, and that’s probably a good thing. (We’ll also delve into this topic at our Structure Data conference in March, with FTC Commissioner Julie Brill.)

Even if it’s forced on them, companies selling connected devices need a framework for thinking of user data not just as a valuable resource, but also as something over which they’re the stewards. Collect the data, analyze it, make your money — the whole industry is predicated on these things. But know there will be penalties in place if you do something bad, or even just stupid.

The August lock.

The August lock.

Of course, the devil here will be in the details. What constitutes an acceptable use, security protocol or retention period could vary widely based on industry, company, product, cost or any other of a number of variables. A connected car is not a fitness tracker. A smart door lock is not a connected toothbrush.

But hopefully, the attention the internet of things is getting early on means lawmakers and regulators will be able to come up with some workable, flexible and relatively future-proof rules sooner rather that later. The last thing we want — especially when dealing with data about our physical-world activity — is a repeat of the web, where it’s 25 years later and we still haven’t figured out what privacy means.

Mo’ money, mo’ data, mo’ cloud on the Structure Show

[soundcloud url=”https://api.soundcloud.com/tracks/180964822?secret_token=s-MTCRX” params=”color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false” width=”100%” height=”166″ iframe=”true” /]


If you want an informed opinion on the state of the cloud and the relative merits of the players, Sebastian Stadil’s a good person to ask. Founder and CEO of Scalr, a multi-cloud management company, he keeps his finger on the pulse of all the players and — perhaps more importantly — their customers.

On this week’s Structure Show he handicaps how [company]Google[/company] and [company]Microsoft[/company] are doing in public cloud not just technologically but in terms of their sales strategies which, when it comes to enterprise accounts, may be just as important as technology. And of course the company everyone is measuring by is [company]Amazon[/company] Web Services which leads the pack.  Oh, and he’s got lots to say about the OpenStack ecosystem as well; cloud technologies from [company]Oracle[/company], [company]Joyent[/company] and more.

It was a busy week in the funding arena with Nginx, DataGravity, and Mesopshere all getting substantial VC rounds ($20 million, $50 million and $36 million respectively.) That ain’t chicken feed and we talk that out.

And, of course, this week’s Hortonworks IPO puts the spotlight back on Hadoop and big data in a  big way, which gives us a chance to tout Gigaom’s upcoming Structure Data event which will feature talks from Hortonworks CEO Rob Bearden, Cloudera CEO Tom Reilly and MapR CEO John Schroeder.


Sebastian Stadil, CEO of Scalr

Sebastian Stadil, CEO of Scalr



Hosts: Barbara Darrow and Derrick Harris

Download This Episode

Subscribe in iTunes

The Structure Show RSS Feed


Why CoreOS went its own way on containers

More from Facebook on its new networking architecture 

Do you find OSS hard to deploy? Say hey to ZerotoDocker

All about AWS Re:Invent and oh some Hortonworks and Microsoft news too

Ok, we admit it. Google may be serious about cloud after all


Wise.io raises $2.5M for machine learning, simplified

Machine learning startup Wise.io, whose founders are University of California, Berkeley, astrophysicists, has raised a $2.5 million series A round of of venture capital. The company claims its software, which was built to analyze telescope imagery, can simplify the process of predicting customer behavior.