Will We Ever Get Smart Data?

Recently IDC released its analysis on trends with Smart City investment. Apparently global investment in Smart City Infrastructure and Services is growing at an annual rate in excess of 25% per year and 2018 expenditure is expected to top $80Bn.

IDC's analysis suggests that one of the leading areas of spend is on surveillance technology in China, with authorities emphasising their ability to use facial recognition to rapidly recognise and locate wanted criminals.

This news comes fast on the heels of china's recent cyber security legislation which as introduced last June. The text of China's Cyber Security Law has not been made available in English, but basically it prohibits the storage of sensitive personal data on Chinese citizens outside China. China also requires all websites operating within China to be licensed (with a Bei'An licence).

This effectively provides the authorities with strong control over what happens with data and web sites within China as access to external websites is filtered and controlled via "The Great Firewall of China", which also acts as a choke point on performance.

In Europe, GDPR is about to go live in about a month's time with its own increased controls over the management of personal data. It is probably safe to say that a lot of large corporate organisations will be struggling to be fully in control of compliance. Any company which is still recovering from legacy problems caused by cost cutting following the 2008 recession, or which has grown rapidly via Merger and Acquisition activity or has lost control of "Shadow IT" as a result of DIY acquisition of SaaS services, is likely to be struggling to get to grips with where all its personal data is, let alone to assess whether it complies with the requirements for (a) data subject consent, (b) only holding the amount of personal data that is necessary, (c) ensuring that the data is properly protected and (d) being able to deliver the right to be forgotten.

This is probably the time where we need to look for Machine Learning based approaches to help identify personal data, where it is, who accesses it and whether it meets the test of not  exceeding the minimum amount legally required to fulfil the tasks for which it is collected stored and managed. It would also help in identifying all the data associated with an individual, requesting the right to be forgotten. However, if we ever are going to trully have control, then its probably time that the concept of smart data (i.e. self managing according to policy rules) technology was introduced, so that data understands its own importance, sensitivity, context and environments and can self certify compliance (or not) when policy rules are subject to change.