The Cerberus project introduces a method for real-time crowd monitoring and privacy preservation using edge computing devices with face detection technology. It is designed for urban informatics and smart building management. The following sections provide an overview of the system architecture, performance comparison of machine learning models, and analysis of occupancy patterns, emphasising privacy considerations.

System Overview

The Cerberus setup includes a Raspberry Pi 4 and a high-resolution Pi Camera, designed to leverage compact yet powerful hardware for edge-based computations. This configuration allows for real-time face detection and occupancy analysis without relying on cloud services. An optional hardware accelerator further boosts the system's ability to handle complex machine learning models, ensuring swift and accurate data analysis. This setup combines high-performance computing with privacy-preserving technologies in smart building applications.

The privacy-preserving cameras track human occupancy in real-time. The video below shows a lecture taking place:

Sensor Placement and Coverage

Strategic sensor placement is crucial for maximising coverage and enhancing the accuracy of occupancy detection and localisation. By carefully positioning cameras and sensors, the system ensures no significant blind spots exist, improving data reliability. This method captures a complete picture of space utilisation, allowing for precise monitoring of crowd dynamics and movements. This strategy is essential for optimising space management and ensuring safety in real-time.

System Architecture

The integration of edge devices and high-resolution cameras forms the backbone of the Cerberus system, enabling local data processing directly on the devices. This architecture maintains individual privacy by eliminating the need for image storage or network transmission. By processing data locally, the system minimises privacy risks associated with cloud computing and external data handling. This approach ensures real-time performance and aligns with strict privacy regulations, making it ideal for sensitive environments.

Model Performance

The system employs various machine learning models for face detection, each with trade-offs between inference speed and accuracy. This comparative analysis allows for selecting the optimal model based on specific operational requirements. High-accuracy models may be preferred where precise detection is critical, at the cost of slower processing times. Conversely, faster models may be chosen for environments where speed is paramount. This flexibility ensures that the Cerberus system can adapt to a wide range of settings, maintaining a balance between operational efficiency and surveillance quality.

User Interface and Data Visualisation

The system features a user interface designed for intuitive visualisation of occupancy and crowd data. This interface offers administrators and facility managers a comprehensive view of space utilisation and occupant distribution. Through dynamic charts, graphs, and heatmaps, users can interact with real-time data, gaining insights into movement patterns, peak occupancy times, and congestion areas. This data interpretation aids in effective space management, enabling data-driven decisions for building layout adjustments, emergency response planning, and enhancing overall occupant comfort and safety.

Detecting Events in the Lecture Theatre

The model presented above analyses data from a crowd-counting sensor. Through statistical analysis and the differential rate of change, it detects the timing of lectures, including their start, end, and any in-lecture breaks. Lectures often run long enough to require breaks, referred to as transitions. Each transition comes with a specific summary of events since the beginning of the lecture or the previous transition—a synopsis of the readings leading up to that event. In crowd-counting, synopses enhance lecture data. Initially, they show entry rates, and finally, seating occupancy, CO₂, temperature, and attentiveness metrics. Events are detected in real-time as data is processed in a single pass.

Capturing Attentiveness

Since the cameras only detect occupied seats by detecting faces, we can also examine the percentage of time the faces were directed upwards. The image summarises a lecture using crowd count data for attendance detection, seating arrangements from crowd localisation, and high-frequency data for calculating the percentage of time faces are directed upwards. Each slice corresponds to approximately 15 seconds. Each temporal pie chart lasts 1 hour, or roughly the duration of a single lecture. More information can be found on the Chronogaze page.

Long-term Occupancy Patterns

Kernel Density Estimation (KDE) plots reveal intricate occupancy and movement patterns within the monitored environment. These plots provide a visual representation of space usage over time, highlighting areas of high and low occupancy and identifying movement patterns. By analysing these patterns, facility managers can make informed decisions about space planning, layout modifications, and occupancy limits to enhance efficiency and safety. KDE plots underscore the system's analytical capabilities, enabling a deeper understanding of space functionality and improvements.

Human-Embedded Digital Twins

The Cerberus project is pivotal in the broader deployment of Digital Twins within the Computer Lab, forming the basis of my doctoral research. The GIF above illustrates this by showing how privacy-preserving sensors, along with standard environmental sensors (represented as spheres), can be used to create a comprehensive digital twin of a lecture theatre that visualises exhaled CO₂ particles in real-time. More information on the digital twin project can be found here.

Results and Future Directions

Over its year-long deployment, Cerberus has proven effective in privacy-preserving crowd monitoring, with potential applications in various public and private sectors for improved space management and emergency planning.

Future enhancements will focus on exploring more advanced machine learning models for face detection, improving system scalability, and further enhancing privacy protections through advanced encryption methodologies.