Big Data with BIG SMILES

Having got your attention I would like to introduce you to a pragmatic, real-world and business centric approach to Big Data and Big Data Analytics. When I say that this is the best approach to Big Data you are ever likely to find in the whole universe and in your entire life, I am still significantly understating the magnificent utility, timeliness and the here-and-now facets of the approach.

Now with the introduction done and dusted with, and the virtues of the BIG SMILES approach exalted, it should come as no surprise that this eminently sensible, highly rational and thoroughly reasonable methodical and no-nonsense technique has been applied successfully in more than 500 business-oriented situations.

Best of all, this amazing Big Data approach is free of charge and with no-strings attached – you don’t even have to buy my book. Now, isn’t that amazing?

Let’s start with the basics. The BIG in BIG SMILES refers to Business Insight Gains. This refers to the focus of SMILES. Simples, right? Now what does SMILES refer to:

Fig. The process chain of Big Data SMILES

SMILES is also an acronym, and it refers to the six major components of the SMILES Big Data approach (as illustrated above). Or, more precisely the various phases of the approach. Namely:

1. Start with a significant data-centric business challenge
2. Model high-level options and approaches
3. Implement your chosen option
4. Leverage the products
5. Evaluate performance and value
6. Socialise the outcomes

Let’s take a look at each of those aspects of Big Data SMILES in a little more detail.

1.      Start with a significant data-centric business challenge

It makes sound business sense to start any business initiative with a compelling business reason. If you don’t have one then don’t start, it’s as easy as that.

Fig. Start SMILES

Now, having identified your significant business challenge you should ask the following questions:

• What: What do you want to accomplish with respect to the significant business challenge?
• Why: Why do we want to address the challenge?
• Who: Who should be involved in helping address this challenge?
• When and where: Can you identify the time and place the challenge first comes into effect?
• Windows of opportunity: During what periods can we most effectively address the challenge?
• Which: Can you enumerate the requirements and constraints associated with the challenge and the possible responses?

When compiling your view of the significant business challenge, you could look for example at questions along the lines of:

1. How do I find the Big Data I need?
2. What is the original source of the Big Data?
3. How was this summarization, enrichment or derivation created in the Big Data?
4. What queries and mechanisms are available to access the Big Data?
5. How have Big Data related business definitions and terms changed?
6. How do interpretations of the Big Data/Data vary across organizations?
7. What business assumptions have been made that are related to this Big Data?

Don’t forget, asking great questions about significant business challenges will lead to even more questions, which is where you will want to highlight the new questions that could quite possibly be answered, wholly or partially, using Big Data. Also, don´t confuse great questions with complex questions, the idea is not to impress the audience but to identify and address the challenge.

Another important thing to point out in relation to the SMILES approach is that you should never, ever, in no way shape or form, try and boil the ocean. That is, do not try and implement and leverage Big Data analytics in your organisation in big leaps and bounds. Start out with baby steps, and treat it like a game of tennis. Win points, games, and sets and beat challenges and bad decisions one-step at a time.

Finally, make sure each iteration of SMILES starts with an objective that is big enough to be significant yet small enough to be doable in a reasonably short-time scale, one preferably made up of sprints of no more than 5 to 10 days.

2.      Model high-level options and approaches

Here we are looking at a process of discover and insight; conceptualisation and creativity; deign and innovation; and, prototyping expertise and domain capabilities.

Fig. Model SMILES

This phase has three parts:

1. Defining the problem and developing options
2. Evaluating and selecting the best model
3. Finalising and developing the implementable prototype

Defining the problem and developing options

Using Co-operative Prototype Ideation, Concretisation and Realisation (a unique rapid development feature of SMILES), you may now choose to develop options and models to various stages of maturity and extensiveness.

Evaluating and selecting the best model

Through a cycle of hypothesise and test you may arrive at the model most suited to your needs. If however, no such model is forthcoming or the best model simply is not good enough or promising enough then do not proceed to the next step. Make sure you have a good reason to progress and inertia is definitely not a good a reason, and neither is ‘because we have to do something’.

Developing the implementable prototype

In this phase, you develop the chosen model through to the stage where it is ready ‘productised’ in the following prototype implementation phase.

In addition, as part of this phase, you will be looking at which technology options might provide the best fit with your requirements. Common technological options are typically associated in one way or another with the Hadoop ecosphere, so your problem may be adequately addressed using technology such as Hive, Pig, Bash, Spark or Python, or indeed with products such as MapR, Neo4j or EXASol.

3.      Implement your chosen option

In this phase, you take your well thought out proof of concept prototype and turn it into a business ready and production hardened product.

What is involved in turning a Big Data solution prototype into a product?

Here we are focusing on tools, build competence and teamwork; product piloting and development support; and, the realities of production and support.

Also, as a closing message for this phase description, please note that the implementation phase also includes the active participation of the development sponsors and target user group, as should every phase up to this point, and all of the follow on phases.

4.      Leverage the product

It’s been designed, prototyped, built and productised. Now what?

Well, here comes the moment of truth.

In this phase SMILES can help Big Data teams quickly react to changing business and market needs by capturing and managing new and changing requirements, and by constantly monitoring feedback. The framework can be totally integrated into true agile development processes especially where requirements are extremely dynamic and free-flowing but nonetheless must be managed at a complete Big Data product level.

5.      Evaluate performance and value

An assessment of the value of the Big Data initiative should be made periodically. However, there should be at least four event-pegged must-do valuations carried out during the life-time of the project products.

Fig. Evaluate SMILES

1. Initial acceptance criteria alignment and qualitative valuation.
2. Maturity performance and tangible ROI contribution. Include both the mitigation and avoidance of loss and the enablement of all direct and indirect gains.
3. Life-time-value-to-date to be carried out before all major enhancements.
4. Sunset life-time assessment and valuation. On replacement or withdrawal from service.

6.      Socialise the outcomes

These are the activities that generally put the smiles in SMILES.

Whatever happens with your initiative you must never fail to socialise the outcomes, for as kitsch, cute or painful the exercise may appear to you or anyone else.

Put it this way. You’ve gone to all the effort and trouble of making a Big Data initiative work, and it’s working well, people like it and it’s delivering value. So, what else is there to do? It’s a success, so you shout it from the rooftops, tell all your colleagues and peers, put the news on the intranet and in the company house magazine, and organise a party.

If your project is killed off early or late, or simply fails to deliver value, and sometimes this just happens, then hold a wake, in the Celtic manner, and learn all the lessons that are worth shaking a stick at and make them part of your corporate data management story and knowledge base.

The golden rules of SMILES for beginners

When entering the new dynamic world of Big Data and Big Data Analytics and to ensure that SMILES delivers the kind of success that others enjoy, you must take ensure that the 9 golden rules of SMILES are also followed. These are the golden rules for beginners.

Fig. The 9 golden rules of SMILES

1. Infrastructure – Ensure that the infrastructure is adequate for the needs of the project and ensure that executive management disconnects your prototyping, development and deployment cycles from the necessarily rigorous, time-consuming and constricting requirements imposed on core business operational systems. Remember this is operational, but it is not life threatening, customers will not be lost nor will serious money be burned. So make sure your senior management unchains Big Data from Big IT bureaucracy – and if possible, on a forever basis.
2. Pilot – When you first take on Big Data and Big Data analytics, always start with pilot prototypes and projects. Again, small enough to be doable (avoid overreach at all costs) and large enough to be significant (small and useful doesn’t have to be trivial).
3. Timescale – Aim to deliver initial pilot Big Data prototypes in around the 3 to 6 week mark, and aim to get the first projects into the leverage phase in around 3 to 5 months, tops.
4. Long-term – Aim to deliver fast, simple and elegantly, but also keep a keen eye on the long-term prospects and issues for Big Data and Big Data Analytics.
5. Cash-flow – Control the cash but make sure you have enough to do what you need to do. Focus on value, keep sprints and iterations short, and be intelligent in the management of funding. (see comments on funding later in this piece)
6. Continuous involvement and justification – Justify every decision in terms of business (mandatory) and technological (optional) drivers. Involve business partners, continuously. Make this an absolute mandatory condition for starting the project and continuing. If involvement from business stops, then stop the project until the implication and involvement of the business stakeholders picks up again. Make all of this clear from the outset. Continually seek and reaffirm business justifications for the projects existence – this is a showcase, and people are watching.
7. Sponsors – Ensure that your project has high-level business sponsorship. This cannot come from IT and it cannot come from the CIO or the CDO, unless your Big Data project is to measure and report the performance of aspects of IT and Data Governance.
8. Clean – Make sure that the data that you use is to the quality levels required. The data that you analyse must be at least as good at that stage as when it was sourced. In many cases you will need to scrub and clean data, especially when it’s coming from badly designed, tragically engineered and shoddily built web applications where the designers and developers have only had a passing acquaintance with sound database engineering principles, if at all.
9. Tenacity – finally, never give up until it’s time to do so. If you believe that success is achievable then go for it. If you see that the project is on a suicide mission then kill it quickly, don’t wait until you’ve burned all your hours and cash.

When using SMILES keep these 9 golden nuggets of rules in mind, and you won’t go far wrong.

A note on funding

Try to make sure that your funding aligns with the phases of SMILES.

However, split your funding requests into three parts.

• Start with a significant data-centric business challenge. 2. Start with a significant data-centric business challenge. 3. Model high-level options and approaches.
• Implement your chosen option.
• Leverage the products. 6. Evaluate performance and value. 7. Socialise the outcomes.

This ensures an optimum allocation of resources and provides additional executive and project management safeguards and options, and helps to ensure alignment of contractual assurances and obligations with committed and planned budgets.

A note on testing

Testing is an integral element of every phase of the SMILES approach. For brevity the approach has not been detailed in this document, but the philosophy is essentially one of test early and test often. Under normal circumstances the SMILES approach does not require a User Acceptance Testing phase, and if we are in shops that do require this testing then the UAT becomes a mere bureaucratic formality.

Things to remember

Some people may be surprised that the Big Data SMILES approach does not start with a strategy. In my view, the idea that strategy can begin without the need for identifying a significant challenge is to get things wrong on two counts: i. It’s not the way to go about strategy, and ii. It’s not the way to go about Big Data.

The BIG SMILES approach is not in itself a strategy, it is a guide, reference and framework for those who wish to develop a specific Big Data oriented strategy for addressing a significant business challenge. This is where it is powerful, useful and relevant. It’s a roadmap, a cookbook and a method to understand issues, formulate questions, and provide adequate, appropriate and timely responses, whilst separating the wheat from the chaff, the core from the periphery, and the important from the inconsequential.

Lastly, if you or your Big Data analysis, design development team haven’t done so already, I suggest that you take a look at the Cambriano Information Supply Framework, which provides a solid-basis upon which to architect and design Big Data oriented solutions.

That’s all from me for now. Have fun and enjoy the Big Data journey.

Thank you for reading.

If you would like to know more about SMILES, the Information Supply Framework, Core Statistics, Core Data Sourcing, Data Governors, the Analytics Data Store or 4^th generation Enterprise Data Warehousing then please drop me an email or visit:

http://www.cambriano.es

On a lighter note, readers may also be interested in joining The Big Data Contrarians, the friendliest, most relevant and massively irreverent Big Data community on the whole of the entire world wide web. So, if you think you are up for it then we can be found on LinkedIn at this address:

https://www.linkedin.com/groups/8338976