Mini Security SIEM
Course Instructor
Pramod Gupta
Abstract
This project presents the design and implementation of a mini Security Information and Event Management (SIEM) system built from scratch to demonstrate the core functions of modern security monitoring. The purpose of the project was to create a lightweight platform capable of ingesting log data, processing security events, generating alerts, and displaying results through an interactive dashboard. Traditional SIEM systems are widely used in enterprise environments to centralize security data and support threat detection, but they are often complex and costly. This project was motivated by the need to better understand the architecture and workflow of a SIEM by developing a simplified but functional version using accessible open-source tools and custom code.
The system was developed with a Python Flask backend, a React frontend, SQLite for storage, and supporting libraries such as SQLAlchemy and pandas for database interaction and log processing. The backend was responsible for handling file uploads, parsing CSV log files, normalizing event fields, storing processed data, and generating alerts based on event severity. The frontend provided a dashboard interface that displayed key performance indicators, event charts, top source analysis, and an active alerts table. Uploaded logs were analyzed and converted into structured events that could be queried and visualized in near real time, allowing the user to observe how raw data is transformed into actionable security information.
A major objective of the project was to allow direct log uploads through the web interface instead of relying on hardcoded sample data. This feature improved the realism and usability of the system by enabling custom CSV files to be parsed and analyzed dynamically. To support this workflow, the system normalized fields such as timestamps, source IP addresses, destination IP addresses, protocols, and threat labels into a consistent internal format. Events were classified with a simplified severity scale from 1 to 3, representing low, medium, and high levels of concern. Detection logic was then applied to identify suspicious and malicious activity and present these events in the active alerts table.
Throughout development, several technical challenges were encountered, including dependency issues, backend route failures, frontend and backend schema mismatches, and SQLite database resets that affected data availability. Resolving these problems required aligning model definitions, API responses, alert structures, and dashboard components so that the entire pipeline functioned consistently from ingestion to visualization. These debugging efforts became an important part of the project because they highlighted the complexity of integrating multiple system components in a cybersecurity application.
The final result is a functioning mini SIEM that demonstrates the essential pipeline of security event management: data ingestion, normalization, storage, alert generation, and dashboard-based monitoring. Although simplified compared to enterprise-grade solutions, the project successfully captures the main ideas behind SIEM design and provides a strong foundation for future improvements such as real-time streaming, more advanced correlation rules, additional log formats, and enhanced analyst workflows. Overall, the project shows that a practical and educational SIEM prototype can be built from scratch while still reflecting the architecture and purpose of real-world security monitoring systems.
Mini Security SIEM
This project presents the design and implementation of a mini Security Information and Event Management (SIEM) system built from scratch to demonstrate the core functions of modern security monitoring. The purpose of the project was to create a lightweight platform capable of ingesting log data, processing security events, generating alerts, and displaying results through an interactive dashboard. Traditional SIEM systems are widely used in enterprise environments to centralize security data and support threat detection, but they are often complex and costly. This project was motivated by the need to better understand the architecture and workflow of a SIEM by developing a simplified but functional version using accessible open-source tools and custom code.
The system was developed with a Python Flask backend, a React frontend, SQLite for storage, and supporting libraries such as SQLAlchemy and pandas for database interaction and log processing. The backend was responsible for handling file uploads, parsing CSV log files, normalizing event fields, storing processed data, and generating alerts based on event severity. The frontend provided a dashboard interface that displayed key performance indicators, event charts, top source analysis, and an active alerts table. Uploaded logs were analyzed and converted into structured events that could be queried and visualized in near real time, allowing the user to observe how raw data is transformed into actionable security information.
A major objective of the project was to allow direct log uploads through the web interface instead of relying on hardcoded sample data. This feature improved the realism and usability of the system by enabling custom CSV files to be parsed and analyzed dynamically. To support this workflow, the system normalized fields such as timestamps, source IP addresses, destination IP addresses, protocols, and threat labels into a consistent internal format. Events were classified with a simplified severity scale from 1 to 3, representing low, medium, and high levels of concern. Detection logic was then applied to identify suspicious and malicious activity and present these events in the active alerts table.
Throughout development, several technical challenges were encountered, including dependency issues, backend route failures, frontend and backend schema mismatches, and SQLite database resets that affected data availability. Resolving these problems required aligning model definitions, API responses, alert structures, and dashboard components so that the entire pipeline functioned consistently from ingestion to visualization. These debugging efforts became an important part of the project because they highlighted the complexity of integrating multiple system components in a cybersecurity application.
The final result is a functioning mini SIEM that demonstrates the essential pipeline of security event management: data ingestion, normalization, storage, alert generation, and dashboard-based monitoring. Although simplified compared to enterprise-grade solutions, the project successfully captures the main ideas behind SIEM design and provides a strong foundation for future improvements such as real-time streaming, more advanced correlation rules, additional log formats, and enhanced analyst workflows. Overall, the project shows that a practical and educational SIEM prototype can be built from scratch while still reflecting the architecture and purpose of real-world security monitoring systems.
