Human 2.0 - The State of the Art

What's Needed for HCI

Human Computer Interface based technology is already here in many simple forms. However, in a previous article, I mentioned that lots is needed before a mature and safe implementations can be accepted with minimal risks of Sci-Fi type Terminators or the ability to hack implanted technology (see Stealing Light by Gary Gibson for a reference).

In a previous article I mentioned some pre-requisites which are summarised in the diagram below. 

Basically, we need to crack not just the ability to interact with technology, but some quite meaty problems around dealing with information overload, shifting contexts of use, human psychology, individual preferences and behaviour, security, future proofing and dealing with the inevitable SNAFUs that technology incurs.

One of the key issues will be how much is the technology wired into us and how much will be accessed in a wearable manner.

In the meant time there are a whole host of areas where universities, the defence establishment and consumer led technology companies are developing one-off single application devices and demonstrators.

Some of these are mentioned below.

Health Monitoring

There's lots of activity around the sports and wellness area of wearables. Well established capabilities in clothing, watches, rings, smart insoles etc. include capabilities for tracking and analysing: footsteps (pedometers), heart rate/pulse, temperature, tread patterns/gait. There are devices for looking at blood flow, but these can be easily fooled.

The latest announcements extending the commercially available repetoire to include:

• Omron's forthcoming Heartguide which promises to deliver accurate blood pressure monitoring (countering reports that many smart watches are innacurate);

• Fitbit's hint that it will be delivering blood sugar monitoring technology.

There's also a lot around relaxation and sleep improvement, which I have covered in a previous article.

Treatment
Increasingly, medecine is beginning to adopt devices which are not just wearable but implantable. These devices not only monitor conditions but can deliver therapy, usually via drugs. However the most dramatic is probably the system used for epilepsy, Resposive Neuro Stimulation or RNS. This is placed within the skull and mnitors EEG signals, an electrode is placed on or in the brain at the area where seizures normally occur. This recognises patterns leading to an attack and delivers electrical neuro stimulation signals to dampen down any attack.

Mental Capabilities

There are a few interesting niche applications such as Brainco's  application for monitoring student's focus during lessons and study (to provide feedback for self improvement) and Foc.us's brain stimulation device for improving computer gamers' reaction times.

Thought Reading

It's well known that Elon Musk and Facebook rivals are looking at technologies for reading thoughts. Though at present the most accessible device is AlterEgo - a wearable device which can read words that users are thinking about produce by MIT as a research demonstrator.

An interesting article on brain controlled gadgets can also be read at Hongkiat's site.

Cyborg Extensions

Additionally, whilst the military experiment with power assisted exoskeletons for combat soldiers, there are any number of devices now being used as prostheses to replace limbs or to by-pass problems such as severed spinal cords to convey nerve impulses along broken nerve pathways.

Lockheed Human Universal Load Carrier (HULC) exoskeleton

Call To Arms

The logical conclusion is that point applications will be replaced by suites of applications, so a common architecture and interoperability is needed. Additionally, there are potential risks with some of the technologies proposed, especially if they are used by operators of high capital plant, equipment or transport. Safeguards are also needed to avoid unintended self harm from adoption. 

So before we blunder too far down this path, we need to do something to address the ethical framework for deployment and an industry certification scheme for security and personal risk mitigation.

2018 - The Year of The Cyborg

Whilst most pundits are focusing on AI, VR, AR and Machine Learning, it is clear that the Digital Frontier has moved on to Human Integrated Computing (HIC). Though not everyone uses the same terminology, so you may see Human Computer Interface, Brain Computer Integration and a number of similar terms to describe basically the same thing.

So this year has seen the IEEE's 6th International Conference on Brain Computer Interfaces (BCIs) and the Hackaday Competition for the Greatest Human Computer Integration (with a prize of US$50,000). Clear themes from both events were the avoidance of direct implants and using any number of techniques to communicate between people and computer devices. 

A lot of these are based upon reading brain waves or nerve impulses and there is a strong theme of applications around things which help people recover from debilitating accidents or deal with disability.  However it is clear that recent advances in the ease of applying machine learning to solutions has been a clear game changer in interpreting brain waves and nerve impulse signals. The ability of machine learning to recognise how to filter out extraneous signals and focus on what matters to the application is key to this.

Additionally, in a wider social context there is a fringe trend towards people choosing to "self adapt" with home grown technology implants into their bodies. This trend, sometimes called Trans-humanism, involves implanting anything from magnets, through chips to specific devices into their bodies to provide a variety of single function applications. Although Lukas Zpira refers to it as "Body Hacktivism" and espouses a creed of "taking control of our destinies by continuously reinventing the self".

So what does this mean in terms of realisation in practical day-to-day life? Well there are a surprising number of products which are either ready for market or close to release, as well as any number of technical concept demonstrators. These include:

Prosthetic Limbs,

Exoskeleton Devices,

Turning Thought into Speech, see Nuros's Nuos software

Eyeball Tracking,

Remote Controlled Limbs, e.g. CTRL Labs Wrist Band,

Additional Limbs, e.g. a second pair of arms on a backpack - Keio University's Fusion 

Accessing Human Memory,

Improved Physiological Measurement of things such as Blood Pressure,

Detection of Emotions.

Though, to my mind one of the more interesting things is the research being conducted by 

researchers at Drexel and ISAE-SUPAERO into aircraft pilot'scognition during extreme incidents and how they deal with the sensory overload of multiple sound alarms, flashing indicators and situational awareness when an accident occurs during flight. Their research involves attempting to monitor how they deal with such incidents, using functional Near Infra Red Spectroscopy (fNIRS) to quantify brain activity response in the Anterior Prefrontal Cortex. So far they have demonstrated its feasibility and the fact that in real life, the overload is higher than in a simulator and pilots make more mistakes. In the future it should be possible to use this to assist in optimising instrumentation design, reduce cognitive overload and the likelihood of errors.

All this represents practicable and achievable goals in the evolution of the path towards the dreams of Elon Musk and Mark Zuckerberg who are pursuing full embedding of computers into the brain with their Neuralink and Building 8 programmes. But as I mentioned in a previous posting there are immense issues around not just technical practicalities, but ethics, security, phsychology, dealing with potential information overload and long term upgrade capability to be addressed before these goals become safe, let alone desirable.

Human AI Integration Gets Closer

In another mind-blowing breakthrough TNW (the next web) reported on an AI which reads and interprets human brainwaves to understand what you are seeing.

Although this is early days, as a proof of concept it is very powerful, showing that the ability to read and interpret human thoughts and senses reliably is not as far away as you would think.

Obviously there is still a lot to do to refine this particular example, as it focuses on sight and recall of images only, but it definitely puts Elon Musk's vision of direct HCI integration into the next twenty years of achievability (allowing for the need to deal with volumes of data and presentation, psychological and ergonomic issues, as well as security and miniturisation.

But it's definitely coming. So there's an ethical debate to be had too around privacy and safeguards.

Also, recently there have been reports on brain hacks to use AI to control electrical stimulation of the brain and improve its performance.

More prosaically, adoption of implantable chips in people is growing; at present, see Brian Jonhson's video, the chips are for single function usage, e.g. access control or train tickets, but the potential is there for multi-functional  implantable chips with flexible reprogramming. So the seeds are being being sown for full human computer integration, and the potential for embedded AI enhancement.

This leads to the question of how obsolescence and technical upgrades will be managed as current approaches to patching are likely to be inadequate.

Updated 9th Feb 2018

Post Digital Dawn: The New Human Computer Interface

As an expression, La Interfaccia Uomo Macchina sounds a lot more sexy than HCI (or human computer interface), but then english speakers have often been suckers for Romantic Languages. Recently, however, the whole topic has become much more interesting as Elon Musk has started to hint about his interest in Neural Laces or direct brain to AI links.

He has now come into the open and announced his investment and interest in Neuralink, a startup specialising in brain to compute interfaces (see: http://bit.ly/2naV4sB ). This builds on progress in a number of areas around understanding the brain and development of a significant number of devices which interface with the central nervous system to enable things such as control of artificial limbs and to help manage the impacts of neurological conditions such as Parkinson's Disease. 

Basically we are seeing the start of the next wave of Information Science and the next big thing after Digital. Although Digital is powerful and will continue to change the way the world operates for quite some time to come, it is limited by the siloed nature of current technologies. True they operate together, with a little integration effort, but they are constrained by the limits of the internet and existing graphical interfaces.

Others have also come into the open with their plans for human computer interfaces, e.g. Brian Johnson of Braintree discussed this at the Recode code conference and Steve Hoffman at TED.

Musk has now lit the fuse for next revolution. Although to be honest this will take a long time to mature in terms of capability, safety and comfort in using the technologies involved. There are a lot of issues to resolve before Neural Laces become something that people are comfortable to adopt.

Firstly, the operating systems of current technologies are just not robust enough. Anything that is intended to operate closely with the human brain should be as least as reliable as the human brain. Secondly, no only must it be safe to attach and integrate Neural Laces with the human brain, there needs to be a robust path for maintenance, upgrade and removal. As it is highly probable that some people's bodies or brains would reject such technology, no one would want to be locked into an inevitable obsolescence as the technology improves, and there are bound to be faults which need rectifying. So "graceful failure" must also be built in. Thirdly, there needs to be a high level of security and privacy built into the Neural Laces, otherwise the risk of being hacked, swamped with unwanted information or interrogated by thought police is too high. Fourthly, the Neural Lace needs to be able to take power from biological processes within the body and to operate at a temperature which will not damage the human brain. Finally, the Internet will need to be completely re-engineered to work safely and effectively with people. There already is a known challenge for the IoT, but this represents a complete step up for networking capability. This all assumes that the basic interface technology can be developed in the first place.

However, the biggest challenges are likely to lie in the human and societal arena. Putting aside the fact that some people would always oppose such technology as being against their religious beliefs, there is a need to address questions around ethics of use, legal issues around IPR, liability and things such as acceptability of evidence sourced via a Neural Lace. There may even be issues of disability if someone is not capable of adopting such technology. Finally, there is a need to develop psychological understanding of how to deal with the vast amounts of data (as opposed to information) associated with using a Neural Lace. If appropriate filtering of data and information feeds is not designed into a Neural Lace, the danger is that it will cripple its host or drive the person insane.

So it may be that the real beneficiaries of developing this technology will be the psychologists and lawyers needed to make it fit for people and society.