Posted on by — Leave a comment

Day 1 of Big Data In Focus: so where is it on the hype cycle?

I came to TMForum’s Big Data In Focus Summit in Amsterdam today expecting one of two things: either that hype would still be the main driver behind this market, or that we’d finally moved forward and fallen into the trough of disillusionment.

For Rob Rich, TMForum’s Managing Director of Research, it’s clearly the latter, which made him all the more proud that attendance was on a par with last year. But I think something else may be happening on the Big Data scene. Last year’s conference was still all about building the infrastructure for collecting and storing all that data. This year, Jessica Rausch, TMForum’s conference producer, set it up to be bullish with as many real world success stories as possible. As Rob pointed out to me nobody really wants to talk about failures, at least not their own. This positive feeling, during day one at least, was furthered by a move away from setting Big Data up, towards Big Data Analytics, which means really using it.

On the down side though, I was disappointed with the actual use cases described on the podium this year. Despite being interesting to listen about and the fact that Big Data made them more accessible, none were really new.

I mean, Big Data for Fraud detection, really? Credit card companies have been using algorithms since the 1980s.

I was also a bit startled by some of the revelations given about data collection with vendors easily spliping into the “I’m just doing what I’m told” roles while their operator clients don’t necessarily take the responsibility either. I couldn’t help thinking that it’s as if the Snowden affair never happened and Big Data is here to give the NSA a new lease of life.

I had a passing feeling of déjà vu with Big Data in the 2010s reminding me of Objet Orientation in the 1990s: it’s probably hugely important and underpins the future of much of IT, but may not ever become a market in its own right.

To counter this, a characteristic of Big Data that came up as much as last year was the strong linkage between process, organization and Big Data projects or as @yifatkafkafi tweeted: “People-driven transformation was key to Big Data success at Skype - @DvirYuval #bigdatainfocus” (you can find other tweets with the #bigdatainfocus overly long twitter hash tag).

Nobody used the term “Machine Data” last year, so Alice Crook of Splunk’s Marketing team put my out of my ignorance telling me that it’s “anything unstructured, created by machine”. I’ll come back to this and dig a bit deeper to get to a clearer conclusion in the next blog, where I’ll also write about the three companies I met: Cvidya, Splunk and Ontology and give TMForum’s view after my interview with Rob Rich.

Posted on by — 3 Comments

The Big Data emperor will need Big Change within companies, that is if he has any clothes on. Summit report Part III

There was a great turnout to TM Forum’s inaugural event on Big Data in January. It was small enough to enable proper networking, but the packed room made it feels like this something more than just hype or buzz is happening around Big Data.

Some of the clear benefits Big Data brings at once

A key benefit EBay has gotten out of Big Data analytics after having started with Hadoop in 2010 is a greater flexibility. An example of what they can do better now is to work out how much to bid on specific keywords like “iPad” because the decision often has to be made in near real-time. Big Data helps eBay manage they key differences in word meaning from market to market.

Bell Canada was one of the more upbeat operators on Big Data. James Gazzola made a case for Advertising Forensics where the operator could use analytics to determine which ads are actually watched. Bell hopes that these insights, once mastered could be monetized. Gazzola went on to point out that as Bell Canada serves 66% of Canadians, analytics could show what's happening in all of Canada. That sent a slight shiver down my back as I wondered if the journey from network planning to user analytics actually terminated at a station called to Big Brother, but oops this is the part on benefits. So back to more down to earth issues, Gazzola told the audience that voice traffic used to be relatively predictable, but that data traffic driven by smartphones is anything but. Big Data is what Bell is looking at to help planning future network capacities.

Google’s presentation was disappointing. I don’t really blame Google speakers because the expectations are always unrealistically high: there’s so much we crave to know about Google. Matt McNeil, from Google’s Enterprise division was asked if they have any big Telco clients for Big Data yet. His wooden answer that "we're talking to several" showed the limits of the company’s transparency policy. But during his sales pitch, Matt got quite excited explaining that “it'll cost you 5M$ to build the capacity Google charges just 500$ per hour, for a Hadoop-powered Big Data analysis platform”. When McNeil showed off with http://bigquery.cloud.google.com, the exciting fact that “Led Zeppelin” is more controversial than “Hitler” got me a bit concerned that maybe all this Big Data stuff really was hype after all. I suppose we need practice finding more telling examples because as Matt did say himself “this year will be a trough of disillusionment for Big Data”.

Big Data is about Big change

Peter Crayfourd who recently left Orange, pointed out that becoming truly customer-centric can be scary. Such an approach may uncover many unhappy customers. But becoming truly customer-centric will take at least 5 years. All speakers at the Big Data conference seemed in agreement that user centric KPIs based on averages are to be shunned because users are so very unique! That sounds fine in theory but CFO’s are going to need to stay up late finding out how to live without the concept of ARPU.

The eye-opening presentation from Belgacom's Emmanuel Van Tomme stayed on the customer-centricity able but made the clearest case so far that change management is key to Big Data implementation. Emmanuel was the first Telco guy I’ve heard talk about a great new metric called VAP or Very Annoyed People. They can now be identified with Big Data analytics.

Many speakers converged on the theme of "change management" as THE key challenge for Big Data going forward. The general message was that if Hadoop is ready to deliver, people and even less their organizations were not yet.

Thinking of the bigger Telcos conjures up an image of oil tankers trying to steer away from network metrics towards usage metrics. Looking solely at the agility dimension I couldn’t help wondering if they could survive the speedboats like Amazon or Google.

As the conference was wrapping up I gleaned an interesting metric: subscribers are 51% more willing to share their data if they have the control of whether or not to share it in the first place! It’s one of those Doh!-I-knew-that statistics you feel you should have come up with, but didn’t.

Earlier it had been pointed out during one of the panel sessions that to make Big Data work for Telcos, subscribers must entrust ALL their data to the operator. For them to agree to this, the outbound sales & marketing link must be cut. It’s probably wiser to have one unhappy head of sales than many unhappy customers.

But things aren’t so simple

The limitations of KPIs

Peter Crayfourd illustrated the danger of KPIs with the voice continuity metric. In his example it was 96% when calculated over 15 minutes, so if that’s what your tracking all is hunky dory.  But in the same network environment, when the same metric is calculated over 45 days the result is usually 0%. Crayfourd went on to explain how averages can be dangerous within an organization: someone with their head in the oven feet in freezer has good average temp! Matt Olson from US operator Century Link pointed out that in the User eXperience (UX) domain simple maths just don't work: UX isn't the sum of the parts but some more complex function thereof.

Listening to the UX focussed presentations one got the feeling that the Big Data story might just be a pretext for some new guys to come steal the carpet from under the feet of the QoE market stakeholders. They’ve been saying this for almost a decade … Big Data is a means not an end.

Cost of Big Data & Hadoop.

For EBay, Hadoop may be cheaper to setup, but it’s so much less efficient to run than structured data that the TCO currently seems the same as with other enterprise solutions.

Google, eBay and even Microsoft made compelling presentations about the nuts and bolts of Big Data and then tried to resell their capabilities to the service providers in the room. TM Forum could have been a bot more ambitious and tried to get more head-on strategic discussions going on how the big pure-play OTT giants are actually eating Telco and other Service provider’s lunch. Maybe a lively debate to setup in Cannes?

Does the Emperor have any clothes on?

UK hosting company MEMSET's Kate Craig-Wood isn’t sure at all. Kate said that Big Data techniques are only needed in a very few cases where many hundreds of terabytes are involved and near real-time results are required. She does concede that the concepts born from Big Data are useful for all.

MEMSET’s co-founder went on to explain how a simple open source SQL based DBMS called SQlite successfully delivered interesting analysis on hundreds of Billions of data points, where MySQL had fallen over. She had to simplify and reorganize data and importing it took 4 days, but once that was done she got her query answered in minutes. Ms Craig-Wood went on to say that the SQL community is working flat out to solve scalability issues going as far as saying "I'd put my money on SQL evolving to solve most of the Big Data problems". There's so much SQL expertise out there!

Perhaps the most controversial part of this refreshing Big Data debunking session from Kate Craig-Wood of MEMSET was when she said that “I don't believe in data scientists, most DevOps will do fine, and Hadoop isn't that complex anyway”. She has a point: we're at the pinnacle of the hype cycle.

Caution

Less extreme but still on the side of caution were the sensible questions from Telefonica that is experimenting with Big Data. The Spanish operator is still cautious about the “high entrance cost” and uncertain final price tag or TCO. So far the Telco has built both a centralized cloud instance of its platform and also separate instances for each of its operating companies in different markets. Telefonica’s Daniel Rodriguez Sierra gave an amusing definition of Big Data as simply those queries we can't handle with current technology.

Verizon wireless also reaffirmed the need for caution pointing out that to implement Big Data and reap any benefit thereof you need an agile trial and error approach. That’s a tall order for any incumbent Telco. The US mobile operator admitted that it was being wooed by the likes of Google, Amazon and EBay that would all love to resell their analytics capability to Verizon. But staunch resistance is the party line as Verizon mobile has the scale (and pockets) to determine that the core data is too strategic to be outsourced. In terms of scale Verizon wireless has 100M subs and 57K towers that generate a petabyte of data or 1,25 trillion objects per day crunched currently with 10K CPUs. Verizon’s Ben Parker was pleasantly open saying that an "army of lawyers is happily supplied with plenty of privacy work now we're capturing info on all data packets".

Governance was too frequently mentioned during several presentations not raise an alarm bell in my mind. It seems that those who’ve actually got their hands dirty with Big Data are finding themselves embarked on projects that are difficult to control.

In the end

I was really impressed by the commitment operators are making to big Data on the one hand while clearly expressing reservations or at least warning that we’re just at the beginning of what’s going to be a long Journey.

For further reading here are three other write-ups of the event that I commend:

There’s a mini video interview of Peter Crayfourd here: http://vimeo.com/58535980

Part I of this report (interview of TM Forum's Strategy Officer) is here.

Part II, a discussion with Guavus and Esri, is here.

Posted on by — 2 Comments

Part I of III: TM Forum’s chief Strategy officer on Big Data

The Big Data Summit organized by TM Forum in Amsterdam was my first.
My first on Big Data - well that’s not so surprising as there haven’t been many yet - but also my first TM Forum event. That’s strange as I’ve been in and out of the Telco industry for over a decade – it was a great event so better late than never.

Before reporting on the event itself in part III (BTW you can get a preview with 50-odd tweets from my time-line @nebul2), this blog is the feedback from my discussion with Nik Willetts, chief strategy officer of the Forum. He started by reminding me who they are. Part II covers the rare demos I saw.
TM-Forum is a 25-year-old non-profit US incorporated organization with about 120 staff. It is Telco based with around a thousand member companies, a quarter of which are service providers. These operators account for a whopping great 90% of the world’s subscribers!

Nik told me this conference was in line with the general Telco movement away from pure network management towards more services. His job, and this conference in particular, are to look for the next wave of digital growth.

He sees Big Data underpinning most successful customer programs in the future. “The whole industry must become data driven, with shorter cycles so as to establish new services that can compete with the eBays or Googles of the world. Without this transformation operators will be loosing very real money. Today’s digital services are built around user experience whereas traditional Telco services are built around technology. It’s going to be about market pull or what customers perceive, where it used to be about what engineering departments pushed. Successful operators will have a deep understanding of User Experience.” Nik pointed to an example given during the conference by Cricket, where analytics were used to determine exactly where user calls were being dropped geographically and feeding that experience data to network engineering teams, thus reducing churn. “But cost reduction will also be a key driver for Big Data within service providers as a better understanding of User Experience helps operators anticipate customer issues and reduce truck rolls.”

I asked Willets what had already struck him during this event. He told me it was “different stake-holders playing multiple roles, attacking the issue from different angles. TM-Forum always tries to get different people round the table together.”

I asked about the general squeeze operators are feeling here in Europe, and whether Nik saw it as a global phenomenon or one restricted to developed markets. “TM Forum covers the whole world, so operators in developing markets have the benefit of seeing what operators in developed markets have or haven’t done successfully to cope with new threats and opportunities. That’s exactly what you can see at this Big Data event. TM Forum is instrumental in this knowledge and experience sharing”.

This is the first dedicated TM Forum event on Big Data although the subject has been covered for about 2 years within other conferences.
Willetts was happy with attendance for a launch event with over 150 delegates and 12 CxOs. The event will be replicated in a year somewhere else.
When I complemented Nik for the absence of the usual sales pitches in presentations he told me that “as with other TM Forum events, there is no pay-to-play here i.e. vendors cannot pay to get to speak. The business model is for delegate fees, sponsorship and exhibitors, with all presentations being vetted. We do allow vendors to speak alongside their customers where it adds value to the presentation, as you may have seen with the Cricket/TEOCO presentation”.

TM Forum logo 1

Part II (the exhibition) is here.

And if you want to skip straight to the conference content itself, that's here.

Stay tuned for the full write up of the event.