Artificial Intelligence and Our 5 Senses

Artificial Intelligence (AI) is very much the thing of the moment. A lot of new enterprise models appear to be built on a combination of AI, IoT and Big Data to collect and process massive amounts of data and to apply human like skills of recognition and decision making with a level of consistency, speed and volume that was previously unattainable with conventional processes or systems. This leads to greater insight, improved performance, better use of scarce skills and consequently other benefits such as improved customer experience, better security and higher cost efficiency. It also means that some tasks which would never have been practicable before are readily delivered via automation.

However when we look at what a human can do, AI capabilities are still lacking. The human capability to use our 5 senses, think abstractly, innovate and test hypotheses, to develop new learning are still a challenge to AI.

Just focusing on the 5 senses: Sight, Hearing, Touch, Smell and Temperature, AI's progress is quite patchy.

In the past 30 years or so, AI's ability to deal with vision and speech has progressed from a miserable score of  about 1 to 2 out of 5, to a 3 to 3 and a half out of 5. Handwriting recognition works well with typed characters, but still has a significant error rate with hand writing. Face recognition systems work quite well with caucasian faces but apparently struggle to deal with black african faces. It is not clear to me  whether the latter is to do with poor training (i.e. not been given enough sample data for adequate deep learning), fundamental flaws in the assumptions made when designing the systems to recognise features or some problems with technical measurement of contrasts and camera sensitivity across the visual spectrum. Whatever the problems, error rates are still quite high.

Temperature is quite easy with thermo-couples and can be engineered to deal with temperatures that people could not with stand. So high marks of 5 out of 5 are to be expected.

Touch has been a major area of research ever since robotics became a hot topic in the late seventies. Most of the work so far has focused on sensing around grip, so that a robot could pick up an egg or some other delicate object without crushing it. But I have not seen anywhere the type of sensing that could differentiate between the touch of skin, rubber, silk, cotton and wool. So this definitely languishing in 2 out of 5 territory.

Smell has barely been touched. However some work is suggesting that techniques for smell processing, which mimic natural biological processes could be invaluable in extending AI capabilities to deal with situations where learning data is limited, ambiguous or masked with other "noise". Smell has potentially many applications in medecine, agriculture and food processing. The techniques however, may be useful for Autonomous Unmanned Vehicles which often have to deal with unforeseen situations or noisy environments.

So there still a long way to go for AI to reach science fiction like capabilities. Conquering the 5 senses represents the first step. The question is, how long will this take and will all humans be machine augmented, by the time we get there?

Has Big Data Lost its Mojo?

A number of surveys were published during 2017 which suggested that Big Data has lost the CxO mind share that it had.

The overall impression gained was that business leaders are putting their emphasis on Artificial Intelligence (in particular Machine Learning) and IoT, whilst CIOs and Technical Leaders are worrying about Security and Lean (Agile and DevOps). 

At the same time the consumer and gadget end of things are focusing on wearables, voice and VR/AR based devices (implying other types of AI are getting important). 

Somewhere in the mix, people are worrying about culture, product management and marketing, and organisations designed around empowered product teams.

There almost appears to be an assumption that big data has been cracked and apps are easy. Also, data governance is not really getting the attention that it should do and vendors are pushing APIs for integration.

This suggests not just a gap between business vision and technical capability to deliver, but also that the consumer led boom in exploitation is going places which don't necessarily fit well with classical business environments. Open plan offices are not the place where people should all be talking to their digital assistants.

The lessons I draw from this are than business management teams need to start thinking holistically about what is needed to deliver their new business (i.e. digitally augmented) strategies and models, but also about what the future workplace looks like. Is the office finally dying?

They also need to get real about data. It needs to be managed using lean data principles. Integration and security are vital to get right. IoT and other exploitation approaches are going to emphasise Big Data's importance further. So it's vitally important to develop data specialists who understand your business and are networked with the right people. A recent article in the Harvard Business Review suggests that most organisations have focused on technicalities and need to look at a more immersive approach and organisational issues too.

The Value of Data

Today I realised that the evidence for Digital being the New Normal was compelling when I read about the way that the agricultural industry is embracing IoT, Big Data and Machine Learning (see: FoodIT: Fork to Farm ) which provides comprehensive overview of the 4th annual conference examining how to join the ecosystem from consumer to farmer up, so that farms respond to consumer tastes, improve yields, avoid waste and provide the provenance etc. that consumers now want. There are great opportunities to provide the next Deliveroo for farm products and by-pass the super markets entirely. This would greatly improve profitability for farmers and the experience for customers as they get fresh, seasonal, sustainable, quality products directly. It could also bypass the scandal of the modern monopoly of slaughter houses, if the right investments were made.

I especially enjoyed the redefining IoT by the conference as the Internet of Tomatoes, spelling out how much the industry has got into it.

There were also some interesting predictions for Data in another couple of articles on the data makes possible  site by Kirk Borne of Booz Allen Hamilton about trends for this year. Personally I found his comments on graph analytics the most compelling as this is an area of growing maturity in which it is now possible to easily uncover insights into linkages of cause and affect and networks of people and activity. He also had interesting things to say about hyper personalisation and realigning AI with a people centric model of mutual assistance to do more (similar to some of my previous observations). Finally he coined a new phrase DataOps to describe lean development approaches as applied to big data.

Digital Adoption Framework

A lot get's talked about Digital, but there are few comprehensive approaches to adoption available for reference. 

This is why I was interested when I came across Vadrim's DAF diagram, reproduced below. Enjoy!

Can Machine Learning Save Big Data?

Business Intelligence (BI) and Big Data (BD) are supposed to help enterprises improve their overall business control by identifying trends, anomalies and insight into what is really happening, so that they can take appropriate action to correct problems and address opportunities, as well as predict what might happen with a degree of informed confidence. Yet most businesses fail to either exploit BI and BD solutions or even if the do just don't get the expected returns. 

One of the biggest problems facing anyone trying to do Business Intelligence or Big Data is quality of data. In some cases, someone has just chosen the wrong source of data (a Master Data Management issue). An example of this that I fell over many years ago was that an organisation tried to build a customer database for Sales and Marketing, using financial sales ledger data. This caused immense problems because Finance is not interested in the same sort of data as Sales and Marketing and don't care who the key contacts are or what the organisdation structure is, as long as someone pays the invoices.

However, an overwelming problem is that the data is dirty. Legacy systems usually were not designed with BI data in mind and external sources used for BD usually involve different terminology and definitions when discussing or dealing with the same thing. Alternatively, the same terminology may be used to use completely different things by different people or organisations. Furthr issues arise when users are asked to input data which is of no utility to them into a system. The act of inputting the data just complicates their day job without helping them do it. So they often provide default information. Additionaly, other forms of context may inform what was meant by an iitem of data or a vague description.

So this often reduces organisations to using a Subject Matter Expert (SME) to manually interpret data and stick everything onto a spreadsheet to conduct a particular analysis. This is costly, labourious and slow. It may even be innacuarate as SME fatigue (or conflicts in opinion between different SMEs) leads to inconsistency and errors in interpretting data. So many analyses just don't get carried out or produce information too late,  lack much detail beyond generalistic interpretations or may even be suspect in accuracy.

In some cases, if the analysis is likely to be needed frequently with new or updated data sets, it may be possible to write programs with interpretation rules to deal with many of the problems mentioned above. However this is painstaking and laborious and often requires continuous maintenance, constricting its viability for many analyses as new costs and delays are built in.

Machine Learning however may offer an answer. If you can get an SME to provide some examples and use them to train a neural network, you can quickly develop the facility to improve data quality. The machine learning tool should clean up data where there are clear rules and identify areas where there is uncerainty or conflict in interpretation. This requires a relatively small amount of effort to put right and then improves consistency. The more data and examples you feed at a machine learning system, the better it becomes, so you get ever improving interpretation of dirty data. This is quick too. 

I found this out from experience, when in a previous company I worked in, we used machine learning to interpret data readings and categorisation of test results for thousands of readings taken of ship hull  plate thickness around different parts of the vessels being inspected, so that they could be analysed by structural analysis experts. A manual process which used to take a highly skilled engineer several days to complete (without creating any value) was reduced to less than 20 minutes and improved accuracy. This meant improved job satisfaction, higher productivity, better quality of output and a more responsive (i.e. quicker) service for the client.

So there you have it. Machine learning can bring some very quick gains with quite simple applications. You don't need complicated Deep Learning solutions to deliver value and it can vastly improve your BI and BD efforts where you have to live with dirty data.

At The Edge of The Enterprise and the New Lean

One of the Truisms for anyone practising IT Strategy and Architecture, is that All the Competitive Opportunities Arise at the Edge of the Enterprise. Everything else is about making business more fit to exploit them through improving the agility, effectiveness, efficiency and protection of  business capabilities.

This is why Digital has been so important. Digital blurs the edge of the enterprise so that a business may operate globally to reach more customers or provide a more comprehensive service to existing customers. Digital also make it easier to partner with other organisation to deliver new products and services or a better sales and delivery experience. 

Digital also makes it easier to reach out and find out what is going on in the market place and find out what is going on and to affect delivery of service in customers homes, premises and assets through combinations of IoT and Big Data.

However, as companies all adopt digital models and Digital becomes the new normal other things are starting to happen. Customer expectations have risen and improving the "Customer Journey" or the life cycle of customer experience has now become essential. This is now encouraging greater examination of internal processes and capabilities.

Whereas before, internal capabilities were improved for scalability, predictability and efficiency. Now, internal capability has to be optimised to address everything that is essential for delivering service to customers. Digital has become part of the New Lean Organisation. Not only does this change the nature of investment, it requires continuous discipline, development of Enterprise Architecture capability, partnering with Product Managers (especially in Marketing) and investment in flexible productivity technologies such as BPM and Machine Learning to reduce lead times for customer fulfilment, improve consistency and enable employees to spend their time doing meaningful work (as opposed to the drudgery of many repetitive clerical tasks).

This will not only make employees more productive, but it should enable organisations to deliver a wider range of products and services, tailored more specifically to individual customer needs and with enhanced economies. For some people this means more fulfilling jobs. For others this represents a threat to low skill jobs. The biggest challenge now is going to be the (re)training of low skill employees.