Algorithms and managing machines that make decisions for you

Financial services sector head John Salmon and the Pinsent Masons financial services sector team bring you insight and analysis on what really matters in the world of financial services.

The ongoing investigation into the 'flash crash trader' Navinder Sarao raises questions about how regulators can better deal with the risks to financial markets that result from complex technology.

Sarao, for those who have not been following it too closely, has been accused of breaching US anti-market abuse laws. These laws have loosely been described as prohibitions against wire fraud, commodities fraud, commodities manipulation and spoofing.   

At the heart of the charges is an accusation that Sarao used technology with the intent to deceive the market. By means of customised algorithmic trading, he is accused of creating an impression that there was a lot of interest in selling at a time when that impression may not have otherwise been justified, resulting in rapid and significant market distortion.

The questions that arise centre around what are some pretty basic legal concepts – intent, causation and capturing evidence of deception. But wider considerations about how businesses can control systems that make decisions for them also arise. How does a business implement controls that effectively manage processes dependent on deep learning, artificial intelligence and other forms of automation that result in technology-led decision making?   

This is not an area that has escaped the attention of the regulators. The Market in Financial Instruments Directive II (MiFID II), which came into force on 2 July 2014 and must be implemented within 30 months, directly addresses the issue. It requires investment firms that engage in algorithmic trading to put in place systems and controls that ensure the resilience of their trading systems, that limit the sending of erroneous orders and that in other ways prevent their systems from contributing to a disorderly market.      

These regulatory initiatives help control operators of technology from intentionally, recklessly or negligently disturbing markets. They do not however, deal with the risk of 'innocent' algorithms being misled by other algorithms into causing significant market disruption.

If the allegations against Sarao are true, the most significant damage that occurred was caused not by his software but by other algorithms that acted in response to his actions backed up by his software. The suggestion is that his software did not simply influence the minds of people – investors and potential investors; it led other algorithms into error.

While MiFID II goes someway towards addressing the risk of organisations failing to control algorithms that are making decisions on their behalf, all of the accountability is placed on the shoulders of the technology operators – the algorithmic traders. The new rules which MiFID II introduce do not mitigate the risk of algorithms misleading other algorithms.

But how to mitigate this risk? Should responsibility for misinformed algorithms also fall on the shoulders of those who operate the systems that have been misinformed? It is easy to argue that it is, after all, these systems that cause the most damage.

If regulators took this approach, they could do so by imposing a strict liability standard as opposed to a fault-based one over technologies susceptible to distorting markets. A business would then be held accountable for damage caused by its algorithms regardless of whether it had been diligent in maintaining appropriate governance processes over them or not.

But such an approach would mean that firms would need to focus their risk assessments more on whether or not to use new and innovative technologies rather than on how to effectively govern and monitor their use. Such an approach seems to be in conflict with the trend of regulators forming rules and guidance in ways that support or at least do not unduly restrict innovation. This would make it a backward step.

It is saying nothing new to point out that regulation is failing to keep pace with technology. To balance a regulator's interest in supporting innovation with its mandate to maintain legal frameworks that are effective in reducing investor and market uncertainty, now would be a good time for further investigation and consultation into this issue to begin.