By
JOHN MARKOFF
OCT. 23, 2016
Imagine receiving a phone call from
your aging mother seeking your help because she has forgotten her banking
password.
Except it’s not your mother. The voice
on the other end of the phone call just sounds deceptively like her.
It is actually a computer-synthesized
voice, a tour-de-force of artificial intelligence technology that has been
crafted to make it possible for someone to masquerade via the telephone.
Such a situation is still science
fiction — but just barely. It is also the future of crime.
The software components necessary to
make such masking technology widely accessible are advancing rapidly. Recently,
for example, DeepMind, the Alphabet subsidiary known for a program that has
bested some of the top human players in the board game Go, announced that it had designed a program that
“mimics any human voice and which sounds more natural than the best existing
text-to-speech systems, reducing the gap with human performance by over 50
percent.”
The irony, of course, is that this year
the computer security industry, with $75 billion in annual revenue, has started
to talk about how machine learning and pattern recognition techniques will
improve the woeful state of computer security.
But there is a downside.
“The thing people don’t get is that
cybercrime is becoming automated and it is scaling exponentially,” said Marc
Goodman, a law enforcement agency adviser and the author of “Future Crimes.” He
added, “This is not about Matthew Broderick hacking from his basement,” a
reference to the 1983 movie “War Games.”
The alarm about the malevolent use of advanced artificial
intelligence technologies was founded
earlier this year by James R. Clapper, the director of National Intelligence.
In his annual review of security, Mr. Clapper underscored the point that while
A.I. systems would make some things easier, they would also expand the
vulnerabilities of the online world.
The growing sophistication of computer
criminals can be seen in the evolution of attack tools like the widely used
malicious program known as Blackshades, according to Mr. Goodman. The author of
the program, a Swedish national, was convicted last year in the United States.
The system, which was sold widely in
the computer underground, functioned as a “criminal franchise in a box,” Mr.
Goodman said. It allowed users without technical skills to deploy computer
ransomware or perform video or audio eavesdropping with a mouse click.
The next generation of these tools will
add machine learning capabilities that have been pioneered by artificial
intelligence researchers to improve the quality of machine vision, speech
understanding, speech synthesis and natural language understanding. Some
computer security researchers believe that digital criminals have been
experimenting with the use of A.I. technologies for more than half a decade.
That can be seen in efforts to subvert
the internet’s omnipresent Captcha — Completely Automated Public Turing test to
tell Computers and Humans Apart — the challenge-and-response puzzle invented in
2003 by Carnegie Mellon University researchers to block automated programs from
stealing online accounts.
Both “white hat” artificial
intelligence researchers and “black hat” criminals have been deploying machine
vision software to subvert Captchas for more than half a decade, said Stefan
Savage, a computer security researcher at the University of California, San
Diego.
“If you don’t change your Captcha for
two years, you will be owned by some machine vision algorithm,” he said.
Surprisingly, one thing that has slowed
the development of malicious A.I. has been the ready availability of either
low-cost or free human labor. For example, some cybercriminals have farmed out
Captcha-breaking schemes to electronic sweatshops where humans are used to decoding the puzzles for a tiny fee.
Even more inventive computer crooks
have used online pornography as a reward for human web surfers who break the
Captcha, Mr. Goodman said. Free labor is a commodity that A.I. software won’t
be able to compete with anytime soon.
So what’s next?
Criminals, for starters, can piggyback
on new tech developments. Voice-recognition technology like Apple’s Siri and
Microsoft’s Cortana are now used extensively to interact with computers. And
Amazon’s Echo voice-controlled speaker and Facebook’s Messenger chatbot
platform are rapidly becoming conduits for online commerce and customer
support. As is often the case, whenever a communication advancement like voice
recognition starts to go mainstream, criminals looking to take advantage of it
aren’t far behind.
“I would argue that companies that
offer customer support via chatbots are unwittingly making themselves liable to
social engineering,” said Brian Krebs, an investigative reporter who publishes
at krebsonsecurity.com.
Social engineering, which refers to the
practice of manipulating people into performing actions or divulging
information, is widely seen as the weakest link in the computer security chain.
Cybercriminals already exploit the best qualities in humans — trust and
willingness to help others — to steal and spy. The ability to create artificial
intelligence avatars that can fool people online will only make the problem
worse.
This can already be seen in efforts by
state governments and political campaigns who are using chatbot technology
widely for political propaganda.
Researchers have coined the term
“computational propaganda” to describe the explosion of deceptive social media
campaigns on services like Facebook and Twitter.
In a recent research paper,
Philip N. Howard, a sociologist at the Oxford Internet Institute, and Bence
Kollanyi, a researcher at the Corvinus
University of Budapest, described how political chatbots had a “small but
strategic role” in shaping the online conversation during the run-up to the
“Brexit” referendum.
It is only a matter of time before such
software is put to criminal use.
“There’s a lot of cleverness in
designing social engineering attacks, but as far as I know, nobody has yet
started using machine learning to find the highest quality suckers,” said Mark
Seiden, an independent computer security specialist. He paused and added, “I
should have replied: ‘I’m sorry, Dave, I can’t answer that question right
now.’”
A version of this article appears in
print on October 24, 2016, on page B3 of the New York edition with the
headline: As Artificial Intelligence Evolves, So Does Its Criminal Potential.
Artificially intelligent
‘judge’ developed which can predict court verdicts with 79 percent accuracy
A statue representing the scales of
justice at the Old Bailey, Central Criminal Court in London
Sarah Knapton,
Science Editor
24 October 2016 • 12:05am
A computer ‘judge’ has been developed
which can correctly predict verdicts of the European Court of Human Rights with
79 percent accuracy.
Computer scientists at University College
London and the University of Sheffield developed an algorithm
which can not only weigh up legal evidence, but also moral considerations.
As early as the 1960s experts predicted
that computers would one day be able to predict the outcomes of judicial
decisions.
But the new method is the first to
predict the outcomes of court cases by automatically analyzing case text using a machine learning algorithm.
“We don’t see AI replacing judges or
lawyers, but we think they’d find it useful for rapidly identifying patterns in
cases that lead to certain outcomes,” said Dr. Nikolaos Aletras, who led the study at UCL Computer Science.
“It could also be a valuable tool for
highlighting which cases are most likely to be violations of the European
Convention on Human Rights.”
To develop the algorithm, the team
allowed an artificially intelligent computer to scan the
published judgments from 584 cases
relating to torture and degrading treatment, fair trials, and privacy.
They computer learned that certain
phrases, facts, or circumstances occurred more frequently when there was a
violation of the human rights act. After analyzing
hundreds of cases the computer was able to predict a verdict with 79 per cent accuracy.
“Previous studies have predicted
outcomes based on the nature of the crime, or the policy position of each
judge, so this is the first time judgments
have been predicted using analysis of text prepared by the court,” said
co-author, Dr. Vasileios Lampos, UCL
Computer Science.
“We expect this sort of tool would
improve efficiencies of high level, in demand courts, but to become a reality,
we need to test it against more articles and the case data submitted to the
court.
“Ideally, we’d test and refine our
algorithm using the applications made to the court rather than the published judgments, but without access to that data we rely on the court-published summaries
of these submissions.
The team found that judgments by the European Court of Human Rights
are often based on non-legal facts rather than directly legal arguments,
suggesting that judges are often swayed by moral considerations father than simply sticking strictly to the
legal framework.
Co-author Dr Dimitrios Tsarapatsanis, a law lecturer at the University of Sheffield, said: "The study, which is the
first of its kind, corroborates the findings of other empirical work on the
determinants of reasoning performed by high-level
courts.
"It should be further pursued and
refined, through the systematic examination of more data."
The research was published in the
journal Computer Science.