Project Zero: Cloud Lab (Part 3)

case study

Monitoring & Logging

last edited 2026-08-14

-projectzero-prod-vpc has been created with subnets configured.

-projectzero-prod-jumpbox EC2 instance exists and is accessible.

-AWS CLI configured with appropriate credentials.

I created a centralized S3 security data lake to provide secure storage for AWS security telemetry. The data lake centralizes CloudTrail logs, VPC Flow Logs, and GuardDuty findings, with telemetry integrated into Wazuh for centralized monitoring, analysis, and detection.

#S3 Security Data Lake

1.Create Security Data Lake

I created the projectzero-prod-datalake-msegrt S3 bucket in eu-north-1 to serve as the centralized storage location for security logs and findings.

Created the Project Zero security data lake in the same AWS region as the production environment.
Created the Project Zero security data lake in the same AWS region as the production environment.

2.Secure the Data Lake

I configured the S3 bucket with ACLs disabled and Block Public Access enabled to prevent security telemetry from being exposed publicly. Default server-side encryption using SSE-S3 was enabled to protect stored logs at rest.

Restricted public access to the security data lake to ensure stored security telemetry remains private.
Restricted public access to the security data lake to ensure stored security telemetry remains private.

3.Organize Security Telemetry

I created dedicated S3 prefixes to separate security telemetry by source, keeping the data lake organized for downstream ingestion and analysis.

-cloudtrail/ — CloudTrail activity logs

-guardduty/ — GuardDuty security findings

-vpc-flow/ — VPC Flow Logs

Organized the security data lake by telemetry source to support structured log collection and downstream analysis.
Organized the security data lake by telemetry source to support structured log collection and downstream analysis.

#Configure Bucket Policy for GuardDuty & CloudTrail

I configured the S3 data lake bucket policy to securely accept telemetry from AWS security services. Access was scoped to the required S3 actions and designated prefixes for GuardDuty and CloudTrail.

1.Configure Security Service Access

I configured an S3 bucket policy allowing GuardDuty and CloudTrail to write security telemetry to their designated prefixes while keeping the data lake private.

-GuardDutyguardduty/

-CloudTrailcloudtrail/

-Access is limited to the required S3 actions and resources.

Configured the data lake bucket policy to allow authorized AWS security services to deliver telemetry to their designated S3 prefixes.
Configured the data lake bucket policy to allow authorized AWS security services to deliver telemetry to their designated S3 prefixes.

CloudTrail delivery was configured to preserve bucket-owner control over stored log objects.


#Create S3 IAM Service Account User

I created a dedicated IAM service account to provide Wazuh with controlled access to the centralized S3 security data lake. A custom S3 policy was used to limit the account to the permissions required for security telemetry ingestion.

1.Configure Wazuh Data Lake Access

I created a dedicated IAM user, projectzero-wazuh-s3-user, to provide Wazuh with controlled access to the S3 security data lake. I then created and attached the projectzero-wazuh-s3-read-policy, granting only the S3 permissions required to retrieve and process security telemetry.

-s3:ListBucket

-s3:GetObject

-s3:DeleteObject

-Access scoped to projectzero-prod-datalake-msegrt

Attached the dedicated S3 access policy to  projectzero-wazuh-s3-user , establishing controlled access between Wazuh and the security data lake.
Attached the dedicated S3 access policy to projectzero-wazuh-s3-user, establishing controlled access between Wazuh and the security data lake.

#Install and Configure AWS CLI

-projectzero-prod-vpc has been created with subnets configured.

-projectzero-prod-jumpbox EC2 instance exists and is accessible.

-AWS CLI configured with appropriate credentials.

-Your AWS username for the bucket naming convention.

-projectzero-sec-box configured with Wazuh.

-S3 security datalake bucket has been created and is receiving logs from CloudTrail and VPC Flow Logs.

I configured AWS authentication on projectzero-sec-box to connect Wazuh with the S3 security data lake. The dedicated projectzero-wazuh-s3-user credentials allow the Wazuh AWS integration to retrieve security telemetry for centralized monitoring.

1.Configure AWS CLI for Wazuh

I configured the AWS CLI on projectzero-sec-box, the security monitoring server hosting Wazuh, using the credentials for the dedicated projectzero-wazuh-s3-user IAM account. The credentials were configured under the root user so the Wazuh AWS integration can authenticate to AWS and retrieve security telemetry from the S3 data lake.

Configured AWS authentication on  projectzero-sec-box  to enable Wazuh access to the S3 security data lake.
Configured AWS authentication on projectzero-sec-box to enable Wazuh access to the S3 security data lake.

2.Configure Wazuh Index Retention

On projectzero-sec-box, I configured a Wazuh Index State Management (ISM) policy to automatically delete wazuh-alerts-* indices after 14 days. This provides controlled log retention while limiting storage consumption within the lab environment.

Implemented a 14-day retention policy for Wazuh alert indices to automatically manage log storage.
Implemented a 14-day retention policy for Wazuh alert indices to automatically manage log storage.

#CloudTrail & Integration

-projectzero-prod-vpc has been created with subnets configured.

-projectzero-prod-jumpbox EC2 instance exists and is accessible.

-AWS CLI configured with appropriate credentials.

-Your AWS username for the bucket naming convention.

-projectzero-sec-box configured with Wazuh.

-S3 security datalake bucket has been created with the cloudtrail/ folder structure.

-S3 bucket policy configured to allow CloudTrail to write logs.

I configured AWS CloudTrail to capture account activity and deliver security telemetry to the centralized S3 data lake. The logging pipeline provides centralized collection of AWS API activity, identity actions, and resource changes, with controlled log retention and integration into Wazuh on projectzero-sec-box for security monitoring.

1.Configure CloudTrail Logging

I created the projectzero-prod-management-trail multi-region trail and configured it to deliver logs to the cloudtrail/ prefix of the existing projectzero-prod-datalake-msegrt S3 security data lake. I enabled management read/write events and Lambda write data events to capture security-relevant AWS activity.

Configured the multi-region CloudTrail trail to centralize AWS activity logs within the existing S3 security data lake.
Configured the multi-region CloudTrail trail to centralize AWS activity logs within the existing S3 security data lake.
Verified that  projectzero-prod-management-trail  was active and successfully logging to the S3 security data lake.
Verified that projectzero-prod-management-trail was active and successfully logging to the S3 security data lake.

#Configure S3 Lifecycle Policy for CloudTrail Logs

I configured lifecycle management for CloudTrail logs stored in the projectzero-prod-datalake-msegrt security data lake. This applies a 14-day retention policy to the cloudtrail/ prefix and verifies that CloudTrail telemetry is successfully being delivered to S3.

1.Configure CloudTrail Retention

I configured an S3 lifecycle rule for the cloudtrail/ prefix within the projectzero-prod-datalake-msegrt security data lake to manage CloudTrail log retention. The rule removes noncurrent object versions after 14 days, providing controlled retention while limiting unnecessary storage consumption within the lab environment.

Scoped the lifecycle rule to the  cloudtrail/  prefix to apply retention controls specifically to CloudTrail telemetry.
Scoped the lifecycle rule to the cloudtrail/ prefix to apply retention controls specifically to CloudTrail telemetry.
Configured noncurrent CloudTrail object versions to be permanently removed after 14 days.
Configured noncurrent CloudTrail object versions to be permanently removed after 14 days.

2.Verify CloudTrail Log Delivery

I verified that CloudTrail logs were being delivered to the cloudtrail/ prefix within the projectzero-prod-datalake-msegrt security data lake. The generated AWSLogs/ structure confirms that CloudTrail is successfully writing telemetry to the configured bucket.

Verified CloudTrail log delivery to the  cloudtrail/  prefix within the  projectzero-prod-datalake-msegrt  security data lake.
Verified CloudTrail log delivery to the cloudtrail/ prefix within the projectzero-prod-datalake-msegrt security data lake.
Confirmed the generated  AWSLogs/  structure containing CloudTrail log data within the security data lake.
Confirmed the generated AWSLogs/ structure containing CloudTrail log data within the security data lake.

#Wazuh Integration (CloudTrail)

I integrated CloudTrail with Wazuh on projectzero-sec-box to ingest AWS account activity from the centralized projectzero-prod-datalake-msegrt S3 security data lake. This extends the logging pipeline from collection and storage into centralized security monitoring and analysis.

1.Configure CloudTrail Ingestion

I configured the Wazuh AWS S3 integration to retrieve CloudTrail telemetry from the cloudtrail/ prefix using the previously configured AWS credentials. The integration runs every 10 minutes and processes logs stored in the centralized S3 security data lake.

Configured Wazuh to ingest CloudTrail telemetry from the centralized S3 security data lake.
Configured Wazuh to ingest CloudTrail telemetry from the centralized S3 security data lake.

2.Verify Wazuh Log Retrieval

After restarting the Wazuh Manager, I verified that the AWS S3 integration was successfully connecting to the data lake and fetching logs for processing.

Verified successful log retrieval through the Wazuh AWS S3 integration.
Verified successful log retrieval through the Wazuh AWS S3 integration.

3.Validate CloudTrail Ingestion

I created the trancejedi IAM user to generate test AWS activity and validate the complete CloudTrail monitoring pipeline. I confirmed that the IAM activity was captured by CloudTrail, delivered through the S3 data lake, and successfully ingested into Wazuh.

Confirmed CloudTrail events were successfully ingested and available for analysis in Wazuh.
Confirmed CloudTrail events were successfully ingested and available for analysis in Wazuh.
Confirmed IAM activity generated by  trancejedi  was captured by CloudTrail and ingested into Wazuh.
Confirmed IAM activity generated by trancejedi was captured by CloudTrail and ingested into Wazuh.

#VPC Flow Logs & Integration

I configured VPC Flow Logs to capture network traffic across projectzero-prod-vpc and centralize the resulting telemetry in the projectzero-prod-datalake-msegrt S3 security data lake. I extended the Wazuh IAM permissions, configured VPC-level flow logging and controlled retention, verified log delivery to S3, and integrated the telemetry into Wazuh on projectzero-sec-box for centralized network security monitoring.

Documenatation: https://documentation.wazuh.com/current/cloud-security/amazon/services/supported-services/vpc.html

1.Configure Wazuh IAM Permissions

I updated the existing projectzero-wazuh-s3-read-policy to include the ec2:DescribeFlowLogs permission, allowing Wazuh to query VPC Flow Log metadata while retaining its existing access to the centralized projectzero-prod-datalake-msegrt S3 security data lake.

Extended the Wazuh IAM policy with the permissions required to query VPC Flow Log metadata.
Extended the Wazuh IAM policy with the permissions required to query VPC Flow Log metadata.

2.Configure VPC Flow Logs

I enabled VPC Flow Logs on projectzero-prod-vpc to capture both accepted and rejected network traffic. The flow log was configured to deliver telemetry to the vpc-flow/ prefix within the existing projectzero-prod-datalake-msegrt S3 security data lake.

Configured VPC Flow Logs to capture all VPC traffic and deliver the resulting telemetry to the centralized S3 security data lake.
Configured VPC Flow Logs to capture all VPC traffic and deliver the resulting telemetry to the centralized S3 security data lake.
Verified that  projectzero-prod-vpc-flow-logs  was successfully created and configured for S3 delivery.
Verified that projectzero-prod-vpc-flow-logs was successfully created and configured for S3 delivery.

3.Configure VPC Flow Log Retention

I configured a dedicated S3 lifecycle rule for the vpc-flow/ prefix within projectzero-prod-datalake-msegrt to permanently remove noncurrent object versions after 14 days, helping control storage consumption within the lab environment.

Configured noncurrent VPC Flow Log object versions to be permanently removed after 14 days.
Configured noncurrent VPC Flow Log object versions to be permanently removed after 14 days.

4.Verify VPC Flow Log Delivery

I verified that VPC Flow Log telemetry was being delivered to the vpc-flow/ prefix within the projectzero-prod-datalake-msegrt security data lake, confirming that network flow logging was successfully writing data to the configured S3 destination.

Verified VPC Flow Log delivery within the  projectzero-prod-datalake-msegrt  security data lake.
Verified VPC Flow Log delivery within the projectzero-prod-datalake-msegrt security data lake.

#Wazuh Integration (VPC Flow Logs)

I integrated VPC Flow Log telemetry into Wazuh on projectzero-sec-box, configuring the AWS S3 module to retrieve network flow data from the vpc-flow/ prefix within projectzero-prod-datalake-msegrt. I then validated successful log retrieval and confirmed that VPC network events were being ingested and available for analysis in Wazuh.

1.Configure Wazuh VPC Flow Log Integration

I configured the Wazuh AWS S3 integration to retrieve VPC Flow Logs from the vpc-flow/ prefix using the previously configured AWS credentials. The integration polls the centralized S3 security data lake every 10 minutes for new VPC Flow Log telemetry.

Configured the Wazuh AWS S3 integration to retrieve VPC Flow Logs from  projectzero-prod-datalake-msegrt/vpc-flow/ .
Configured the Wazuh AWS S3 integration to retrieve VPC Flow Logs from projectzero-prod-datalake-msegrt/vpc-flow/.

2.Verify VPC Flow Log Retrieval

After saving the configuration and restarting the Wazuh Manager, I verified that the AWS S3 module initialized the vpcflow integration and began retrieving telemetry from the configured S3 bucket.

Verified that the Wazuh AWS S3 module initialized the  vpcflow  integration and began retrieving telemetry from the S3 data lake.
Verified that the Wazuh AWS S3 module initialized the vpcflow integration and began retrieving telemetry from the S3 data lake.

3.Validate VPC Flow Events in Wazuh

I confirmed that VPC Flow Log events were successfully ingested into Wazuh and available for analysis. The resulting events provided visibility into accepted and rejected network flows across projectzero-prod-vpc.

Confirmed VPC Flow Log events were successfully ingested and available for network security analysis in Wazuh.
Confirmed VPC Flow Log events were successfully ingested and available for network security analysis in Wazuh.

#Why This Matters

This implementation creates a centralized security telemetry pipeline for the AWS environment. CloudTrail provides visibility into API, identity, and resource activity, while VPC Flow Logs provide visibility into network communications. Both telemetry sources are centralized within the S3 security data lake and integrated with Wazuh on projectzero-sec-box for analysis and monitoring.

The architecture also applies scoped IAM permissions, dedicated S3 prefixes, and lifecycle controls to separate data sources, restrict access, and manage log retention. Together, these components provide a foundation for centralized cloud monitoring, investigation, and detection.