Brains on Silicon: AI Expert Discusses the Opportunities and Limits of Surveillance in Dresden
Dresden. Can artificial intelligence at a train station detect an attack before it happens? At the Brains on Silicon AI conference at the International Congress Center, Ganen Sethupathy demonstrated how far the technology has come. He heads the public safety division at the IT and consulting firm Sopra Steria. His message: The systems are capable of a lot, but you shouldn’t trust them blindly.
The focus was on image recognition, known as computer vision. Modern systems no longer merely describe individual objects but entire situations. For example, they could distinguish whether two people are fighting or just scuffling, and trigger an alert. His team tested this in India, among other places, using inexpensive cameras and local computers, in collaboration with universities and the police.
Why 97 Percent in the Lab Isn’t Enough
Sethupathy also warned against placing too much trust in the technology. A 97 percent accuracy rate in the lab does not mean that a system will perform just as reliably on the street. As an example, he cited a case from the U.S. in which a woman was wrongfully arrested due to faulty facial recognition. And even good systems can be fooled, for instance with specially printed sweaters or glasses. Systems like those currently being tested in Berlin are also facing criticism for this reason.
Warning Against the Misuse of Technology
The second part addressed the downside. The same technology that helps the police is also available to criminals. Common AI tools can be tricked using simple methods into bypassing their security safeguards and revealing dangerous information, Sethupathy warned. The protective mechanisms of many models are too weak.
In conclusion, he offered three pieces of advice. First, authorities should not build every system from scratch themselves, but rather build upon existing, freely available AI models. Such pre-trained models save time and money because the time-consuming groundwork has already been done, and only the specific use case needs to be added. Second, these systems must be hardened against manipulation - that is, against tricks such as specially rigged sweaters or glasses that can fool cameras. Third, an AI should make it clear why it considers a situation dangerous, for example by showing which part of the image it analyzed. The final decision must be made by a human, because humans tend to trust AI too much.
