The golden age of Big Data bullshit

Without a shadow of doubt, we have entered the golden age of Big Data bullshit.

In these postmodern times of always-on, instant-gratification, and vain superficiality; where crapulence has become the new credible, mean boloney the new benign and trite the new treasure; it is hardly surprising that banality has been elevated to the status of wisdom and knowledge.  

So, in spite of what the good people at the prestigious house of Gartner have been saying, the Big Data hype-cycle is far from over. This is why I believe that we have entered what might be fairly albeit humorously described as the golden age of Big Data bullshit.

Now, I remember the renaissance of Data Warehouse boloney, and some of it was outrageous, and in many cases perpetrated by the very same companies (and in some cases, people) who are now spreading the Big Data doo-doo around, thick and fast. But although it has some parallels with the golden age of Big Data bullshit, the comparison doesn’t really do justice to either.

When I started on my first Data Warehouse projects, there were no Data Warehouse success stories in Europe, it was just all too new. Sure, I had been doing things like managing the design and build Information Centres and MIS solutions in the eighties, and the projects I was involved in and responsible for were largely successful, but they weren’t the full-Inmon DW enchilada, and sometimes these solutions became unstuck and in unpredictable ways. Later, with satisfied DW users and successful deliver of DW projects, came a slew of tangible, coherent and verifiable success stories. But it wasn’t all about success, as more than fifty percent of so called Data Warehouse projects were accidents waiting to happen. Nonetheless, there were enough tangible, coherent and verifiable Data Warehouse success stories around that the task of providing this sort of information to interested parties wasn’t turned into an onerous task of ‘inventiveness and creativity’.

Up until that point, at least on my own Data Warehousing projects, success was based on the understanding that for success to be assured the process had to be absolutely business driven, market focused and technology based.

Sometime around 1995, technology companies realised that Data Warehousing was no longer going to be a simple niche solution. So what did they do?

I’ll tell you.

All of a sudden new and massive marketing campaigns were oriented to ensure that Data Warehousing was seen primarily as technology driven, technology focused and technology based. Even if the supposed outcome was to be a large population of pleasantly surprised DW users and business stakeholders. The key to all things wonderful in DW land was to be technology. Technology, technology licenses and technology services.

So, what happened next?

Well, the massive-shift to Data Warehousing as being mainly a technical solution almost killed the fatted calf, the goose that lay the golden eggs and almost gave away all of the Data Warehousing the family silver, in one fell swoop. From those days, the world of Data Warehousing never truly recovered from that massive cluster**** - a massive and naïve act of Homeric strategic incompetence committed by those in the IT industry who should really have known better.

Big Data is like that, but worse. There is no Inmon of Big Data. There is no coherent development process. Unsurprisingly really, as there is no real equivalence at both the business or market level, and what connections there are between technologies employed in both are almost purely coincidental.

Inmon Data Warehousing was a pragmatic and business oriented solution framework looking for technologies. It was side-lined by corporations looking to maximise their hardware and license sales, and by service providers who based their models on maximising offshoring, maximising hours worked per artefact, minimising quality and by creating and selling seriously dodgy contractual agreements.

Big Data is about niche ‘roman census’ technology looking for a problem. So far, in many domains, where no suitable and obvious challenge actually exists.

So, Big Data is an entirely distinct proposition.

One way or the other, like it or hate it, so bereft is the Big Data world of success stories, beyond the usual triad of Google, Facebook and Amazon, that the leading influential pundits of the day are ‘obliged’ to eke out success stories elsewhere.

How many times have we read Big Data success stories, that…?

1.      Were actually success stories from another area, such as a success story from Data Warehousing or Business Intelligence or Statistics?

2.     Weren’t actually success stories at all, but lazy, misleading and inaccurate notions about how Big Data might be applied.

3.     Were simply taking advantage of some human tragedy or another in order to schlepp Big Data snake-oil medicine around the social media.

Sure, it's all nonsense. But it's nonsense that means that evwerything becomes much more complex, and unnecessarily so. That things are done that should not be done, that projects fail that should not fail, and that any superficial initial savings on offshoring are wiped out because deliverables are basically unusable.

So, when I hear terms such as ‘amazing’, ‘guru’ and ‘influencer’ mentioned in connection with Big Data, well, what more can I do than reach for a nice cup of tea.

That's quite enough about that for now. I hope it makes sense to you and that you can avoid any nasty surprises in the future. Whether in Big Data or Data Warehousing.

Many thanks for reading.

As always, please share your questions, views and criticisms on this piece using the comment box below. I frequently write about communication, data, information, knowledge, strategy, organisational leadership and information technology topics, trends and tendencies. You are more than welcome to keep up with my posts by clicking the ‘Follow’ link and perhaps even send me a LinkedIn invite. Also feel free to connect via Twitter, Facebook and the Cambriano Energy website.

Finally, if you feel up to being a member of the elite Big Data Contrarians community then please send in a request to join from this LinkedIn page:

https://www.linkedin.com/groups/8338976

12 Amazing Big Data Success Stories for 2016

12 Amazing Big Data Success Stories for 2016

If this piece tickles your fancy, then please consider joining The Big Data Contrarians on LinkedIn:

https://www.linkedin.com/groups/8338976

Every year I ask myself the same question. Will there be any tangible, coherent and verifiable Big Data success stories in the coming year? Every year I come up with nothing. Nothing at all. "Sorry, no rooms at the Big Data Success Inn, as we are closed for vacations."

However, this year things are different. More positive, more alive and more fantastic.

As you can probably guess, I am well excited to be able to reach out and tell you about the twelve amazingly fab Big-Data stories that will appear during the course of 2016. The year of the incredible, startling and awesome Big Data monkey.

To this end, and as this is a magically special occasion, I have made an extra-special effort to deliver the goods, to do full justice to the task, and to go that extra Big Data kilometer for my demanding readership.

So, I gazed into Madame Frufru's crystal ball, I opened up the kimono with the Ouija spirits of Von Neumann, Babbage and Jobs, and I pushed the envelope in the vast disruptive solution-spaces habited by Ada Augusta, Audrey Tautou and Jennifer Saunders… and, I came back with the best of the best.

I only hope it was all worth the blood, the sweat and the tears.

So, here for your veritable delight and salutary entertainment, I give you the twelve remarkable Big Data success stories of 2016.

Big Data leads to massive government savings – 2016 will be the year in which right-thinking, common sense and pragmatic governments around the world will leverage Big Data to bring about a radical reduction in government expenditure. Unchained from the dogma of professionality, administrations will replace overpaid, over-educated and over-bearing statisticians, with Data Scientists who can produce 'the required numbers', a priori, and at a tenth of the cost. If this works well, as no doubt it will, other professions, such as medicine, teaching and the law enforcement agencies, will also be subjected to the Big Data treatment. Why pay a professional Doctor, Teacher or Police Officer their exorbitant fees and salaries, when a Quack Scientist, Chalky Scientist or Plod Scientist can fill their places, and for a fraction of the cost.

Big Data clamps down on gum chewing in Singapore – Radical Polymer masticating criminals are the bane of the upstanding street-walking citizens of Singapore. However, in 2016, this will change. Why? Because Big Data will be used to identify, track-down and apprehend gum-chewing, sidewalk spoiling and anti-social spearmint-breathed offenders. Yes, capital punishment for such offenses may seem harsh, but remember, if Hadoop says it is a heinous crime, especially if it's backed up by expert social media opinion, then it must be right.

Big Data solves the Climate Change conundrum – Following the amazingly successful climate talks in Paris this year, 2016 will herald in a period of fantastic adjustment in how climate change is seen, measured and addressed. No longer will Climate Change it be seen as a threat or a problem, but as a seriously good opportunity for market capitalism in general, and Big Data in particular. Measurement of temperature changes will no longer be made, but massive Big Data technologies will collect climate change opinions from global social media, and that will be our unique guide to the actual effectiveness of the fight against things 'getting too hot'. Big

Data will lead the way, and factor 10,000 sun blocker and super-mega walk-in fridge-freezers, will follow.

Big Data helps put Real Madrid back in the top tier – The BBC* might not like it, but Big Data will triumph in sport in 2016, thanks in the main to its innate ability to help Real Madrid win the Champions League, the Spanish League, and the Spanish King's Cup. Even though the mighty-whites have already been eliminated from the last of these competitions (for fielding a Big Data player who was under a match ban). Okay, so Big Data can't get it all right, but no one is perfect.

*Bale, Benzema and Cristiano.

Big Data knocks out Data Warehousing – In 2016, Big Data will finally put Data Warehousing to bed. It's been on the cards for a while now, but in 2016 it will be proven beyond any shadow of doubt that the best input into strategic, tactical and operational decision making are massive concatenations of simple word counts, done on a vast array of what people are now describing as commodity hardware. Commodity hardware, to distinguish it from the other hardware that we were using up until now, which was also confusingly termed 'commodity hardware'.

LinkedIn publishes its first ever Big Data success story – Incredible, but true. In 2016, LinkedIn will get its resident Big Data guru, data master and influencer to document a tangible, coherent and verifiable Big Data success story. It will matter not a jot that it is a knock-off plagiarism of a late nineteen-ninety Data Warehousing partial success-story, as it's the thoughts, and not the facts, that count.

Queen Brenda inaugurates the Lady Di Memorial Big Data Lake – During 2016, HRH will inaugurate the former Windermere Lake as the new Lady Di Memorial Big Data Lake. Millions of subjects will hail this as a clear success story for Big Data and for Britain. The inauguration day will be slightly marred (no pun intended) by a gushing Big Data guru being told to beggar 'orf by none other than Phil the Greek.

Big Data housing becomes an issue of significant importance to the EU – Because of the incredible speeds amazingly valuable Big Data is being created at, the EU will move to take measures to capture and more importantly store all of this new Big Data. There will exist an existential realization that none of this life-giving Big Data should be lost or compromised, or both. Chancellor Merkel has already come out strongly and offered to take much of the generated Big Data in 2016, which will be housed in both public and private premises. For example, each German household will be asked to house volumes of Big Data based on the size of the family abode, the internet bandwidth and the number of smart phones im haus. France and Spain will follow suit, but with modestly reduced quotas. The UK will spend most of 2016 trying to opt out, and will even threaten a Big Data Referendum if the onus on them to take so much Big Data is not radically reduced. So, in net, a win-win for Europe and Big Data.

The CIA will be charged with custodianship of all Big Data success stories – During 2016, together with the custodians of Fort Knox, the CIA will be charged with custodianship of all tangible, coherent and verifiable Big Data success stories, and only those who should know, and can handle the power of information, will have access to the files. This will be done to avoid information of global importance from falling into the hands of evil-doers, delinquents and busy-bodies. This is a success story because it will demonstrate once again a truly tangible, coherent and verifiable Big Data success story. The Head of the FBI was unavailable for comment.

Big Data solves world issues – For years we have struggled to see the elephants in the global room. Now with the help of Big Data, not only will we finally be able to see them, but also we will have a key component of the solution within a click of the mouse and a rapid stroke of a smartphone gesture. Yes, hunger, poverty and the refugee crisis can all be identified in 2016, thanks mainly to Big Data. What's more, if we get the political will to do so we can also think of ways of partially, or wholly, fixing those problems. Although admittedly that is a 'big ask' of Big Data, especially in one enormously hectic year, where the focus of attention will be mainly on the UEFA European Championship, the Olympics and the war on terror. Now if that isn't a Big Data success story then I really don't know what is.

Big Data success stories to top a million by the end of 2016 – Thanks to a global and socially responsible market-driven initiative to reclassify Microsoft Access and Microsoft Excel as Big Data repositories, the number of Big Data success stories for 2016 will amazingly exceed a million, and that's just in Milton Keynes.

Democratic Elections replaced by Mega-Democratic Big Data Social Media Mining – Sick and tired of having to turn out to vote every four years? Tense, nervous pre-election headaches over not being able to think, weigh or decide? Worry no more. Thanks to advanced social-media mining techniques, from 2016 the election of politicians will be decided not by you – least not in the legacy way – but by a broad interconnected raft of machine learning, sentiment analysis and other data science gizmos – guaranteed 100% democratic. This is what we have all been waiting for. The end of old fashioned and boringly DIY-democratic elections, and the heralding of a brave new world of online interactive social-media politics. Don't look at it as the trivialization of democracy, the puerility of post-modernity and the throwing away of centuries of fights for civil and human rights, look upon it as being real progress – progress with a capital pee.

On the other hand, what 2016 might really herald might just be The Golden Age of Big Data Bullshit.

Let's wait and see.

Thank you so much for reading.

Also, if you are of a mind, then please join The Big Data Contrarians on LinkedIn:

https://www.linkedin.com/groups/8338976

Big Data: And the hype played on

MARTYN RICHARD JONES

In spite of the best-efforts of Hadoop evangelists, consulting houses, and IT infrastructure and service vendors, Big Data – hailed as the greatest thing since the dawn of greatest things – is failing, and dramatically so, to produce the necessarily corresponding quantity and quality of tangible, detailed and verifiable success-stories.

So, given the dearth of Big Data success stories, why are so many in the industry still banging on with troll-like insistence about the ‘indisputable’ relevance, importance and universal applicability of Big Data?

Seen from where I stand, and I don’t think I am particularly unique in this respect, a lot of arrows are pointing towards a quieter future for Big Data, one where it has far more limited use than was once imagined, a data niche rather than a data driven all-encompassing market-based revolution. But obviously not everyone is sharing that view, at least not ostensibly so.

The fact of the matter is this. That in spite of what the good people at the prestigious house of Gartner have been saying, the Big Data hype-cycle is far from over. In fact, we have entered what might be crudely yet accurately described as the golden age of Big Data boloney.

When I survey the Big Data landscape and see companies, organisations and individuals continue to bang on the Big Data drum like as if it was going out of fashion, it’s quite embarrassing. Acutely so. Even for someone who isn’t participating, either actively or passively, in the Big Data dog and pony bestiality-show.

So, what’s happening with the Big Data in-crowd? Words were coming to my mind, words such as overcompensation, projection, doublethink, denial, delusion, social, awkward, wacky correlation and non-existent causation. But none of these were entirely satisfactory. Lacking, as they do, in conveying any sort of coherent, tangible and credible explanation of the continuing and burgeoning Big Data hype phenomena.

Then it dawned on me. Black tulips.

Industry players and Big Data pundits are inexplicably engaged in what is at best a zero-sum game, and at its worst has clear parallels with the hubris, wilful ignorance and  greed which lead to the last major global financial crisis and also has nasty parallels with the dot-com bubble.

So, the next time you are reading a gushing puff-piece on Big Data take this advice to heart. Instead of thinking about the author as a friendly adviser, think of them as an unqualified, unregistered and unregulated financial adviser who is trying to flog you something that they probably don’t understand and that they have no idea how to value – remember, all they are thinking is in their ‘commission’.

More to the point, don’t go repeating the same talking points as if they were facts you can take to the bank, when you yourself are unsure about how reliable those ‘facts’ might be.

Stated simply. There is very little which could be more suspect than an announced global revolution in Big Data, a revolution that is accompanied by a dearth of tangible, detailed and verifiable success-stories. So take good note of what is really going on.

If someone deceives you once, it’s their fault, if they deceive you twice it’s your fault. If you then go on to ‘do unto others what has been done unto you’, then expect to be rightfully derided and decried.

Be professional, be on the side of the angels, and do the right thing. In the longer run you will thank yourself for your wise choices.

Many thanks for reading.

The Banality of Big Data Hype

Sick and tired of the amazing, incredible and fabulous velocities, varieties and volumes of  Big Data bullshit washing the decks of the SS LinkedIn? Well, be sick and tired no longer. Here is the antidote!

Some interesting Big Data facts to think about this weekend.

I. More Big Data bullshit has been created in the last couple of years, than in the entire history of humankind.
II. Big Data bullshit will grow faster than ever before, in spite of what Gartner say to the contrary.
III. By 2021, if the mega-trending nonsense does not go unabated, there will be 40 megabytes of Big Data bullshit created for every living woman, man and child, every sixty seconds. 
IV. Also, in 2021 the accumulated digital universe of Big Data bullshit will grow from 8 spartabytes to 22 marrsabytes.
V. Every second people are thinking about creating new Big Data bullshit. For example, 20 million search queries alone (per minute) are generated with the sole intent of creating even more Big Data bullshit. This is set to grow to over 100 thousand brazilian bulslhit queries per year by 2020.
VI. Every minute an estimated 280 hours of Big Data oriented porn is uploaded to the next.
VII. By 2017 over 1 trillion Big Data bullshitters will be connected via Facebook.
VIII. Facebook usage by Big Data bullshitters will make the current social media scene look like a walk in the bullring.
IX. In 2015, an astounding 1 million trolleyloads of photos were uploaded to the web every single hour of the day. By 2017, nearly 80% of photos taken will include a cameo by one or more smartass Big Data bullshit artist.
X. This year, over 4 billion smartass Big Data bullshitters will be shipped - all packed with communication devices capable of collecting and communicating all kinds of Big Data bullshit, not to mention the Big Data bullshit the amazing Big Data babblers create themselves.
XI. By 2020, we will have over 8 billion Big Data idiot savants (overtaking sentient and rational human beings).
XII. Within five years there will be over 5 billion Big Data smartasses connected in the world, all developed to collect, analyze and share Big Data bullshit.
XIII. By 2020, at least a third of all Big Data bullshit will pass through the bullshit cloud (a network of Big Data bullshit servers connected over the Big Data bullshit Internet).
XIV. Distributed Big Data bullshitting (performing Big Data bullshitting tasks using a network of computers in the cloud) is very real. Google uses it every day to involve about 10 Big Data bullshitters in answering a single search query, which takes no more that 0.2 weeks to complete.
XV. The Hadoop Bullshit Ecosystem (open bullshit software for distributed bullshitting) market is forecast to grow at a compound annual growth rate 299,258% surpassing $111 billion by 2021.
XVI. Estimates suggest that by better integrating Big Data bullshit, we could save as much as $300Bn a year on smoking, drinking and having a wild time. That’s equal to reducing costs by $1000000 a year for every person on earth.
XVII. The White House, who first recognized Big Data as the bullshit it is, has already invested more than $200 in big data bullshit projects.
XVIII. For an archetypal Fortune 1000 company, just a 10% increase in data accessibility will result in more than $650 billion additional net income.
XIX. Retailers who leverage the full power of big data could increase their operating margins by as much as 36,660%
XX. 173% of organizations have already invested or plan to invest in big data bullshit by 2099.

Many thanks for reading. Think about it. I hope you get the message.

Something funny happened on the way to the Data Lake

When one surveys the vast array of Big Data tomes and the even vaster output of the regularly amazing Big Data bullshit babblers, one thing that strikes you immediately is that Big Data isn’t explained starting with a story about the marvellous useful applications that you might imagine are generating all of this additional Big Data, but instead they almost invariably dwell on all of the data generated by social media applications of dubious business value and even more dubious social value.

Basically we are being led to believe that applications of peripheral interest, that is, peripheral in the grander scheme of things, are capable of simultaneously meeting almost all of our immediate needs and also of generating a voluminous wealth of data detritus that is somehow going to bring about an amazing economic revolution, the awesome galvanising of innovation and the creation of absolutely fabulous riches beyond the dreams of avarice.

What’s more, every day some Big Data proselyting dope has a cunning plan and pops up on the web like a drunken gopher with some extrapolating and theorising boloney about the quite unusual successes of very unusual businesses such as Facebook, Twitter and Google and then hammers square pegs into round holes in order to produce a generalised theory of awesome Big Data irksome-foolishness that will certainly mean, without a doubt, that we will see the raising of the luck and fortunes of all businesses, everywhere, if only they would buy into the Big Data religion, now.

If you don’t believe me then check it out for yourself. For example, take a look at the Big Data channel over on LinkedIn, now that it’s been apparently taken over by the master guru of astro-turfing Big Data bullshit and babble himself, it now comes with wall to wall Big Data hype, and it’s all the same, and it’s all so obviously wrong that it makes you wonder if LinkedIn is not deliberately converting itself into some species of professional laughing academy, for idiots by idiots, with no room for even the occasionally and superficially contrarian.

In fact, looking at some of the latest blunderbuss blasts of blatant bullshit I am reminded of the quote ascribed to Groucho Marx which I will makeover for modern-day consumption: “These are my Big Data principles, and if you don't like them... well, I have others.”

Big Data with BIG SMILES

Having got your attention I would like to introduce you to a pragmatic, real-world and business centric approach to Big Data and Big Data Analytics. When I say that this is the best approach to Big Data you are ever likely to find in the whole universe and in your entire life, I am still significantly understating the magnificent utility, timeliness and the here-and-now facets of the approach.

Now with the introduction done and dusted with, and the virtues of the BIG SMILES approach exalted, it should come as no surprise that this eminently sensible, highly rational and thoroughly reasonable methodical and no-nonsense technique has been applied successfully in more than 500 business-oriented situations.

Best of all, this amazing Big Data approach is free of charge and with no-strings attached – you don’t even have to buy my book. Now, isn’t that amazing?

Let’s start with the basics. The BIG in BIG SMILES refers to Business Insight Gains. This refers to the focus of SMILES. Simples, right? Now what does SMILES refer to:

Fig. The process chain of Big Data SMILES

SMILES is also an acronym, and it refers to the six major components of the SMILES Big Data approach (as illustrated above). Or, more precisely the various phases of the approach. Namely:

1. Start with a significant data-centric business challenge
2. Model high-level options and approaches
3. Implement your chosen option
4. Leverage the products
5. Evaluate performance and value
6. Socialise the outcomes

Let’s take a look at each of those aspects of Big Data SMILES in a little more detail.

1.      Start with a significant data-centric business challenge

It makes sound business sense to start any business initiative with a compelling business reason. If you don’t have one then don’t start, it’s as easy as that.

Fig. Start SMILES

Now, having identified your significant business challenge you should ask the following questions:

• What: What do you want to accomplish with respect to the significant business challenge?
• Why: Why do we want to address the challenge?
• Who: Who should be involved in helping address this challenge?
• When and where: Can you identify the time and place the challenge first comes into effect?
• Windows of opportunity: During what periods can we most effectively address the challenge?
• Which: Can you enumerate the requirements and constraints associated with the challenge and the possible responses?

When compiling your view of the significant business challenge, you could look for example at questions along the lines of:

1. How do I find the Big Data I need?
2. What is the original source of the Big Data?
3. How was this summarization, enrichment or derivation created in the Big Data?
4. What queries and mechanisms are available to access the Big Data?
5. How have Big Data related business definitions and terms changed?
6. How do interpretations of the Big Data/Data vary across organizations?
7. What business assumptions have been made that are related to this Big Data?

Don’t forget, asking great questions about significant business challenges will lead to even more questions, which is where you will want to highlight the new questions that could quite possibly be answered, wholly or partially, using Big Data. Also, don´t confuse great questions with complex questions, the idea is not to impress the audience but to identify and address the challenge.

Another important thing to point out in relation to the SMILES approach is that you should never, ever, in no way shape or form, try and boil the ocean. That is, do not try and implement and leverage Big Data analytics in your organisation in big leaps and bounds. Start out with baby steps, and treat it like a game of tennis. Win points, games, and sets and beat challenges and bad decisions one-step at a time.

Finally, make sure each iteration of SMILES starts with an objective that is big enough to be significant yet small enough to be doable in a reasonably short-time scale, one preferably made up of sprints of no more than 5 to 10 days.

2.      Model high-level options and approaches

Here we are looking at a process of discover and insight; conceptualisation and creativity; deign and innovation; and, prototyping expertise and domain capabilities.

Fig. Model SMILES

This phase has three parts:

1. Defining the problem and developing options
2. Evaluating and selecting the best model
3. Finalising and developing the implementable prototype

Defining the problem and developing options

Using Co-operative Prototype Ideation, Concretisation and Realisation (a unique rapid development feature of SMILES), you may now choose to develop options and models to various stages of maturity and extensiveness.

Evaluating and selecting the best model

Through a cycle of hypothesise and test you may arrive at the model most suited to your needs. If however, no such model is forthcoming or the best model simply is not good enough or promising enough then do not proceed to the next step. Make sure you have a good reason to progress and inertia is definitely not a good a reason, and neither is ‘because we have to do something’.

Developing the implementable prototype

In this phase, you develop the chosen model through to the stage where it is ready ‘productised’ in the following prototype implementation phase.

In addition, as part of this phase, you will be looking at which technology options might provide the best fit with your requirements. Common technological options are typically associated in one way or another with the Hadoop ecosphere, so your problem may be adequately addressed using technology such as Hive, Pig, Bash, Spark or Python, or indeed with products such as MapR, Neo4j or EXASol.

3.      Implement your chosen option

In this phase, you take your well thought out proof of concept prototype and turn it into a business ready and production hardened product.

What is involved in turning a Big Data solution prototype into a product?

Here we are focusing on tools, build competence and teamwork; product piloting and development support; and, the realities of production and support.

Also, as a closing message for this phase description, please note that the implementation phase also includes the active participation of the development sponsors and target user group, as should every phase up to this point, and all of the follow on phases.

4.      Leverage the product

It’s been designed, prototyped, built and productised. Now what?

Well, here comes the moment of truth.

In this phase SMILES can help Big Data teams quickly react to changing business and market needs by capturing and managing new and changing requirements, and by constantly monitoring feedback. The framework can be totally integrated into true agile development processes especially where requirements are extremely dynamic and free-flowing but nonetheless must be managed at a complete Big Data product level.

5.      Evaluate performance and value

An assessment of the value of the Big Data initiative should be made periodically. However, there should be at least four event-pegged must-do valuations carried out during the life-time of the project products.

Fig. Evaluate SMILES

1. Initial acceptance criteria alignment and qualitative valuation.
2. Maturity performance and tangible ROI contribution. Include both the mitigation and avoidance of loss and the enablement of all direct and indirect gains.
3. Life-time-value-to-date to be carried out before all major enhancements.
4. Sunset life-time assessment and valuation. On replacement or withdrawal from service.

6.      Socialise the outcomes

These are the activities that generally put the smiles in SMILES.

Whatever happens with your initiative you must never fail to socialise the outcomes, for as kitsch, cute or painful the exercise may appear to you or anyone else.

Put it this way. You’ve gone to all the effort and trouble of making a Big Data initiative work, and it’s working well, people like it and it’s delivering value. So, what else is there to do? It’s a success, so you shout it from the rooftops, tell all your colleagues and peers, put the news on the intranet and in the company house magazine, and organise a party.

If your project is killed off early or late, or simply fails to deliver value, and sometimes this just happens, then hold a wake, in the Celtic manner, and learn all the lessons that are worth shaking a stick at and make them part of your corporate data management story and knowledge base.

The golden rules of SMILES for beginners

When entering the new dynamic world of Big Data and Big Data Analytics and to ensure that SMILES delivers the kind of success that others enjoy, you must take ensure that the 9 golden rules of SMILES are also followed. These are the golden rules for beginners.

Fig. The 9 golden rules of SMILES

1. Infrastructure – Ensure that the infrastructure is adequate for the needs of the project and ensure that executive management disconnects your prototyping, development and deployment cycles from the necessarily rigorous, time-consuming and constricting requirements imposed on core business operational systems. Remember this is operational, but it is not life threatening, customers will not be lost nor will serious money be burned. So make sure your senior management unchains Big Data from Big IT bureaucracy – and if possible, on a forever basis.
2. Pilot – When you first take on Big Data and Big Data analytics, always start with pilot prototypes and projects. Again, small enough to be doable (avoid overreach at all costs) and large enough to be significant (small and useful doesn’t have to be trivial).
3. Timescale – Aim to deliver initial pilot Big Data prototypes in around the 3 to 6 week mark, and aim to get the first projects into the leverage phase in around 3 to 5 months, tops.
4. Long-term – Aim to deliver fast, simple and elegantly, but also keep a keen eye on the long-term prospects and issues for Big Data and Big Data Analytics.
5. Cash-flow – Control the cash but make sure you have enough to do what you need to do. Focus on value, keep sprints and iterations short, and be intelligent in the management of funding. (see comments on funding later in this piece)
6. Continuous involvement and justification – Justify every decision in terms of business (mandatory) and technological (optional) drivers. Involve business partners, continuously. Make this an absolute mandatory condition for starting the project and continuing. If involvement from business stops, then stop the project until the implication and involvement of the business stakeholders picks up again. Make all of this clear from the outset. Continually seek and reaffirm business justifications for the projects existence – this is a showcase, and people are watching.
7. Sponsors – Ensure that your project has high-level business sponsorship. This cannot come from IT and it cannot come from the CIO or the CDO, unless your Big Data project is to measure and report the performance of aspects of IT and Data Governance.
8. Clean – Make sure that the data that you use is to the quality levels required. The data that you analyse must be at least as good at that stage as when it was sourced. In many cases you will need to scrub and clean data, especially when it’s coming from badly designed, tragically engineered and shoddily built web applications where the designers and developers have only had a passing acquaintance with sound database engineering principles, if at all.
9. Tenacity – finally, never give up until it’s time to do so. If you believe that success is achievable then go for it. If you see that the project is on a suicide mission then kill it quickly, don’t wait until you’ve burned all your hours and cash.

When using SMILES keep these 9 golden nuggets of rules in mind, and you won’t go far wrong.

A note on funding

Try to make sure that your funding aligns with the phases of SMILES.

However, split your funding requests into three parts.

• Start with a significant data-centric business challenge. 2. Start with a significant data-centric business challenge. 3. Model high-level options and approaches.
• Implement your chosen option.
• Leverage the products. 6. Evaluate performance and value. 7. Socialise the outcomes.

This ensures an optimum allocation of resources and provides additional executive and project management safeguards and options, and helps to ensure alignment of contractual assurances and obligations with committed and planned budgets.

A note on testing

Testing is an integral element of every phase of the SMILES approach. For brevity the approach has not been detailed in this document, but the philosophy is essentially one of test early and test often. Under normal circumstances the SMILES approach does not require a User Acceptance Testing phase, and if we are in shops that do require this testing then the UAT becomes a mere bureaucratic formality.

Things to remember

Some people may be surprised that the Big Data SMILES approach does not start with a strategy. In my view, the idea that strategy can begin without the need for identifying a significant challenge is to get things wrong on two counts: i. It’s not the way to go about strategy, and ii. It’s not the way to go about Big Data.

The BIG SMILES approach is not in itself a strategy, it is a guide, reference and framework for those who wish to develop a specific Big Data oriented strategy for addressing a significant business challenge. This is where it is powerful, useful and relevant. It’s a roadmap, a cookbook and a method to understand issues, formulate questions, and provide adequate, appropriate and timely responses, whilst separating the wheat from the chaff, the core from the periphery, and the important from the inconsequential.

Lastly, if you or your Big Data analysis, design development team haven’t done so already, I suggest that you take a look at the Cambriano Information Supply Framework, which provides a solid-basis upon which to architect and design Big Data oriented solutions.

That’s all from me for now. Have fun and enjoy the Big Data journey.

Thank you for reading.

If you would like to know more about SMILES, the Information Supply Framework, Core Statistics, Core Data Sourcing, Data Governors, the Analytics Data Store or 4^th generation Enterprise Data Warehousing then please drop me an email or visit:

http://www.cambriano.es

On a lighter note, readers may also be interested in joining The Big Data Contrarians, the friendliest, most relevant and massively irreverent Big Data community on the whole of the entire world wide web. So, if you think you are up for it then we can be found on LinkedIn at this address:

https://www.linkedin.com/groups/8338976