How telcos are using big data to set prices and maybe make bills better

Ten years ago Anukool Lakhina, CEO and Founder of Guavus, was working at Sprint (s s) collecting network data as part of a research project. The phone company had put sensors in its network and generated so much data every day that it had to be shipped to Lakhina via FedEx truck for analysis. At the time, the network wasn’t able to handle that kind of volume. Today the networks can handle more traffic, but the problem of processing the petabytes of data generated by cell phones still exists.

This is why, in 2006, Lakhina founded Guavus, a company that has built a software platform for real-time analysis of streaming data. The company claims its proprietary platform can handle massive amounts of real-time data, but also that it can match that incoming data to existing warehoused data as well. In its latest product, aimed at the telco market, Guavus lets operators process about half a petabyte of data every day and match that real-time data stream against subscribers.

This means your wireless provider can match your YouTube viewing to you while you are viewing it. Lakhina calls this “DPI-level insights.” Subscribers might not see the value but operators do. Lakhina says that a Tier 1 operators in the U.S. used his company’s software to catch a customer who was violating the terms of its contract with the operator and was subsequently flooding the network in New York.

Lakhina says the carrier noticed that during business hours in New York its network traffic had suddenly jumped by 10 percent for no clear reason. By looking at the traffic patterns at an individual level using Guavus’ software it matched that jump in data to a new customer that had sold an M2M payment processing product to taxi cabs. The cab drivers were supposed to use the cellular network for processing payments, but instead were using it to stream video inside their cabs, a violation of the terms between the payment platform provider and the carrier. The operator renegotiated the terms of the agreement.

This has obvious benefits for the carrier, which can now police contracts and can also use the software for planning where it puts capacity. For example, when AT&T (s t) got the iPhone(S aapl,) it had no idea how it would affect its network, and until things started breaking and calls began dropping it was unable to predict where and when problems would arise. At the time, network engineers dealt with capacity problems in quarterly or monthly reviews, but in today’s world capacity planning isn’t something static enough that a monthly or even weekly adjustment works.

Or at least, that’s the pitch from Guavus. For the end user the use of big data and real-time analysis by operators is less compelling. Lakhina says that operators could use it to help reduce the number of customer care calls. For example, instead of being surprised by an overage charge or being unaware of where such a change came from, an operator might proactively notify a subscriber or could at a minimum explain to them after the fact what behavior caused them to go over their current data cap.

Guavus has two of the top carriers in the U.S. as customers of its NetReflex software, and plans to take its underlying data platform to other industries that could benefit from the mix of real-time data analysis.