The golden age of Big Data bullshit

Without a shadow of doubt, we have entered the golden age of Big Data bullshit.

In these postmodern times of always-on, instant-gratification, and vain superficiality; where crapulence has become the new credible, mean boloney the new benign and trite the new treasure; it is hardly surprising that banality has been elevated to the status of wisdom and knowledge.  

So, in spite of what the good people at the prestigious house of Gartner have been saying, the Big Data hype-cycle is far from over. This is why I believe that we have entered what might be fairly albeit humorously described as the golden age of Big Data bullshit.

Now, I remember the renaissance of Data Warehouse boloney, and some of it was outrageous, and in many cases perpetrated by the very same companies (and in some cases, people) who are now spreading the Big Data doo-doo around, thick and fast. But although it has some parallels with the golden age of Big Data bullshit, the comparison doesn’t really do justice to either.

When I started on my first Data Warehouse projects, there were no Data Warehouse success stories in Europe, it was just all too new. Sure, I had been doing things like managing the design and build Information Centres and MIS solutions in the eighties, and the projects I was involved in and responsible for were largely successful, but they weren’t the full-Inmon DW enchilada, and sometimes these solutions became unstuck and in unpredictable ways. Later, with satisfied DW users and successful deliver of DW projects, came a slew of tangible, coherent and verifiable success stories. But it wasn’t all about success, as more than fifty percent of so called Data Warehouse projects were accidents waiting to happen. Nonetheless, there were enough tangible, coherent and verifiable Data Warehouse success stories around that the task of providing this sort of information to interested parties wasn’t turned into an onerous task of ‘inventiveness and creativity’.

Up until that point, at least on my own Data Warehousing projects, success was based on the understanding that for success to be assured the process had to be absolutely business driven, market focused and technology based.

Sometime around 1995, technology companies realised that Data Warehousing was no longer going to be a simple niche solution. So what did they do?

I’ll tell you.

All of a sudden new and massive marketing campaigns were oriented to ensure that Data Warehousing was seen primarily as technology driven, technology focused and technology based. Even if the supposed outcome was to be a large population of pleasantly surprised DW users and business stakeholders. The key to all things wonderful in DW land was to be technology. Technology, technology licenses and technology services.

So, what happened next?

Well, the massive-shift to Data Warehousing as being mainly a technical solution almost killed the fatted calf, the goose that lay the golden eggs and almost gave away all of the Data Warehousing the family silver, in one fell swoop. From those days, the world of Data Warehousing never truly recovered from that massive cluster**** - a massive and naïve act of Homeric strategic incompetence committed by those in the IT industry who should really have known better.

Big Data is like that, but worse. There is no Inmon of Big Data. There is no coherent development process. Unsurprisingly really, as there is no real equivalence at both the business or market level, and what connections there are between technologies employed in both are almost purely coincidental.

Inmon Data Warehousing was a pragmatic and business oriented solution framework looking for technologies. It was side-lined by corporations looking to maximise their hardware and license sales, and by service providers who based their models on maximising offshoring, maximising hours worked per artefact, minimising quality and by creating and selling seriously dodgy contractual agreements.

Big Data is about niche ‘roman census’ technology looking for a problem. So far, in many domains, where no suitable and obvious challenge actually exists.

So, Big Data is an entirely distinct proposition.

One way or the other, like it or hate it, so bereft is the Big Data world of success stories, beyond the usual triad of Google, Facebook and Amazon, that the leading influential pundits of the day are ‘obliged’ to eke out success stories elsewhere.

How many times have we read Big Data success stories, that…?

1.      Were actually success stories from another area, such as a success story from Data Warehousing or Business Intelligence or Statistics?

2.     Weren’t actually success stories at all, but lazy, misleading and inaccurate notions about how Big Data might be applied.

3.     Were simply taking advantage of some human tragedy or another in order to schlepp Big Data snake-oil medicine around the social media.

Sure, it's all nonsense. But it's nonsense that means that evwerything becomes much more complex, and unnecessarily so. That things are done that should not be done, that projects fail that should not fail, and that any superficial initial savings on offshoring are wiped out because deliverables are basically unusable.

So, when I hear terms such as ‘amazing’, ‘guru’ and ‘influencer’ mentioned in connection with Big Data, well, what more can I do than reach for a nice cup of tea.

That's quite enough about that for now. I hope it makes sense to you and that you can avoid any nasty surprises in the future. Whether in Big Data or Data Warehousing.

Many thanks for reading.

As always, please share your questions, views and criticisms on this piece using the comment box below. I frequently write about communication, data, information, knowledge, strategy, organisational leadership and information technology topics, trends and tendencies. You are more than welcome to keep up with my posts by clicking the ‘Follow’ link and perhaps even send me a LinkedIn invite. Also feel free to connect via Twitter, Facebook and the Cambriano Energy website.

Finally, if you feel up to being a member of the elite Big Data Contrarians community then please send in a request to join from this LinkedIn page:

https://www.linkedin.com/groups/8338976