In many cities, the police use CCTV cameras to monitor and pursue offenders. In Arnhem TNO tested ‘smart software’ that allows the cameras themselves to recognize street violence. This could be a breakthrough in the maintenance of public order.
Arnhem’s city centre is crowded and convivial at the weekends. Unfortunately, the police then also often have to deal with fights and drunkenness. TNO’s Gertjan Burghouts recently conducted an experiment with the analysis of CCTV camera images. Arnhem has about 70 such cameras in the city centre, running permanently. On Friday and Saturday evening their images are watched ‘live’ by a police officer in a video room, supporting his colleagues on the street. “The police wanted to know whether I could develop software that would enable these cameras to independently recognize street violence, to support their work in maintaining public order,” explains Burghouts.
TNO delivered a computer programme that analyses patterns in images, including those that indicate fights and disturbances. The software was then tested on a database of camera images of Arnhem’s city centre. The results exceeded expectations: the programme successfully recognized 80% of such incidents.
King’s Day 2017
The Arnhem police had approached TNO with a very concrete problem. “It’s simply impossible for a single police officer to watch 70 cameras at the same time,” explains IJsbrand ter Haar, whose police responsibilities include camera surveillance. “So the officer decides which camera images to check, and has digital surveillance of the whole city centre. If the officer sees something suspicious happening, they can send a report to police officers on the street. Unfortunately, it sometimes happens that camera images are not picked up – for example, because the incident occurred at exactly the moment that the officer happened to be checking images from other cameras on the video wall.”
Ter Haar: “We think that the automatic detection of street violence could make tracing the culprits a hundred times more effective”
This problem was seen on King’s Day 2017, when a boy was beaten up in a shopping street by a group of youths. A camera had recorded the incident, but the officer in the video room had missed the images. It took a great deal of time and trouble – including an appeal on the television programme Opsporing Verzocht – to identify and apprehend those responsible. Ter Haar: “It showed the need for ‘smart’ cameras that can tell when something is a fight, and can report it so that we can act on it. We think that the automatic detection of street violence could make tracing the culprits a hundred times more effective.”
Artifical intelligence and the analysis of camera images is a rapidly-growing technology. The cameras in a Tesla car ‘read’ traffic signs and assess traffic situations. But police cameras that can independently recognize street violence is a technology that has never been used before, anywhere. The Arnhem test has shown that it works, although there will have to be a human eye to assess the images and to decide how the police should react. “We’re working towards a test with cameras that don’t just register that something suspicious is happening, but that also report it to the officer at the video wall, who can then decide whether an officer on the street needs to go to the incident,” says Ter Haar.
Veenvliet: “I want to get the earliest possible report of a suspicious situation. It would be great if smart camera technology could help”
Joyce Veenvliet, a police officer on the beat in Arnhem’s food and drink district, hopes that these smart cameras can be tested soon. Every weekend she and her colleagues are confronted with about 20 violent incidents. Veenvliet: “I want to get the earliest possible report of a suspicious situation. It would be great if smart camera technology could help. The faster we get there, the better the chances we can calm things down – or pick up the suspects.”
Meanwhile, Gertjan Burghouts is working to improve the technology still further. “Computers can recognize violence in camera images,” he says, “but there are tricky grey-area situations where it’s not immediately clear whether violence is involved. The technology will never be absolutely faultless. You don’t want automatic reports of situations in which nothing much turns out to be happening, so the assessment of camera images will always be human work.” Nevertheless, according to Joyce Veenvliet the technology could give the police some valuable support: “Cameras can also tell when someone pushes over a scooter, for example. It’s not violence against another person, but it is a disturbance of the peace, and it would be good if we could respond immediately if necessary.”
Burghouts: “The software can learn. We are using input from the Arnhem police to make the cameras smarter all the time”
Burghouts claims the software can learn. He is using input from the Arnhem police – for instance, what does and does not constitute street violence, and how to recognize the vandalizing of street furniture – to make the cameras smarter. Ter Haar: “In the future this system could help the police respond to trouble more quickly. Our investigation work would be more effective, and cheaper, too, because we’d have caught more of the culprits red-handed. When an offender is caught in the act, the police work takes an average of two days; if the culprits have fled the scene, then – with analysing all the camera images, and maybe involving TV programmes – the case can easily take two months or more.”
Scientist on the job
As a TNO ‘scientist on the job’, Gertjan Burghouts is working closely with the Arnhem police. Their camera images are helping him to improve his AI software, and conversely the police are benefiting from the application that TNO is developing ‘on the job’. Burghouts: “My doctorate was in artificial intelligence. In Arnhem I can continue developing my thesis research, which was on camera images. I’m putting my knowledge into actual practice. The people I work with in the police are very enthusiastic about the software I’ve developed, because they can see its potential for keeping the peace. For my part I can also take what I learn here and apply it in other areas, such as smart cameras in care homes that notice when a resident falls over, or when something else happens that needs immediate attention.”
Advancing digitization, the increasing availability of information and new forms of crime are constantly challenging the police organization to innovate. To respond to these changes, the police cooperate with knowledge-intensive organizations that are familiar with these technological and social issues. TNO is an important partner for the police in this matter.
Want to know more? Contact us.
Would you like to know more?
Could your organization also benefit from smart camera technology? Then get in touch with Gertjan Burghouts of TNO.