DataSift introduces VEDO, a means to structure and analyze social data

I got a heads-up from Nick Halstead, the founder and CEO of DataSift, the social data company, about some news that’s going live today.

Nick frames the problem confronting companies that want to dig into the rich possibilities latent in social data (from the company blog):

Social Data is very much a Big Data problem – the data generated each day is beyond the reach of 99.99% of businesses and to store even a fraction of it is a challenge. Second the content itself is in the most part unstructured. If you look at a Tweet – there is almost nothing you can do to it in a purely analytical sense other than count it. For two years I have been wrestling with how we could make both understanding the data simple and to bring it into context for business.

DataSift has created very sophisticated means to pull in, analyze, and quantify social data from sources like Twitter, but providing tools that non-programmers can use has been challenging.

Enter VEDO

So today we are announcing VEDO – an extension of our core platform that brings programmable intelligence to the masses. Building upon our incredibly rich text pre-processing and parsing capabilities, we have added a whole new engine that allows customers to take advantage of advances in machine learning, statistical models, rich taxonomies and much more all through a simple and unified approach. As with the rest of our platform, we want to reduce the cost of developing this kind of functionality for our customers and let them focus on innovation and not on infrastructure.

VEDO brings the power to understand the context and the meaning of the content itself. It can be trained to understand any subject and to contextualize it so that the data can be inherently joined to other structured data within the business. This to me goes to the heart of the value of Social – bringing it together with other business data to set it in context and allow customers to understand why and how Social is impacting them and be able to make decisions off the back of it.

Consider applying machine learning to unstructured enterprise social data as the backbone of a new generation of work management tools: the algorithmic ‘engines of meaning’ might augment or replace the ‘collaboration’ architecture of human-defined project spaces, access controls, and org chart-based sharing schemes, relying instead on intelligent agents that automatically and on-the-fly manipulate our online workspaces and pull information from whatever sources that may be relevant to the task at hand, for everyone.

The third way of work makes new technologies core to the conduct of business: we are moving to a model where — for the first time, really — our tools will no longer emulate pre-computer era ways of working, but will break free of those restrictions.

I hope to interview Nick in the next few weeks, and learn more about how DataSift’s clients are using the tool, and what’s on the horizon for social data analysis.