Serverless (Lambda)

I configured the Lambda execution role with scoped S3 permissions allowing the threat-intelligence functions to read and write objects in the dedicated threat-intelligence feed bucket.

I created a shared Lambda layer containing the Python dependencies required by the threat-intelligence functions. Packaging the dependencies as a reusable layer keeps the individual Lambda deployment packages smaller and allows multiple functions to use the same libraries.

I created a dedicated S3 bucket to act as the storage interface between the threat-intelligence Lambda functions.

I configured an S3 Gateway VPC endpoint to provide private S3 connectivity from the ProjectZero VPC.

I configured AWS Lambda as the trusted service, allowing Lambda functions to assume the execution role.

I created the projectzero-lambda-feed-exec-role and attached the AWSLambdaBasicExecutionRole managed policy to provide basic Lambda execution and CloudWatch logging permissions.

I created a custom IAM policy granting the Lambda functions s3:GetObject and s3:PutObject permissions for the threat-intelligence S3 bucket, then attached the policy to the Lambda execution role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ThreatIntelFeedBucketReadWriteObjects",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::threat-intelligence-feed-log-bucket/*"
    }
  ]
}

I configured a dedicated security group for the Lambda functions and allowed PostgreSQL traffic from the Lambda security group to the web server on TCP port 5432.

I updated PostgreSQL to listen for network connections so the database could accept traffic from Lambda within the VPC.

sudo nano /etc/postgresql/17/main/postgresql.conf

I configured PostgreSQL host-based authentication to allow the webapp_rw account to connect to projectzerodb from the VPC network using TLS and SCRAM-SHA-256 authentication.

sudo nano /etc/postgresql/17/main/pg_hba.conf

I deployed the updated threat-intel-app to the web server and configured the application with the PostgreSQL connection parameters required to access projectzerodb.

Add .env File to threat-intel-app

I created a .env file to define the application's PostgreSQL connection settings.

DB_HOST='Private IP Address'
DB_PORT=5432
DB_NAME=projectzerodb
DB_USER='webapp_rw'
DB_PASSWORD='PASSWORD'
DB_SSLMODE=require

I created the public-threat-intelligence-feed-parser Lambda function using Python 3.12 and assigned the existing projectzero-lambda-feed-exec-role execution role.

I configured the function with 256 MB of memory and a one minute timeout to support external threat-intelligence retrieval and processing.

I configured the environment variables used by the Lambda function to define the S3 output destination and threat-intelligence feed settings.

Database Ingest Configuration Example

The database-ingest Lambda uses the following environment variable structure to establish an authenticated PostgreSQL connection from the Lambda function.

EC2_ENDPOINT=10.0.0.240
EC2_DB_NAME=projectzerodb
EC2_USERNAME=webapp_rw
EC2_PASSWORD=your-secret
EC2_PORT=5432
EC2_SSLMODE=require

I uploaded the deployment package containing the Python modules used to retrieve, process, and store public threat-intelligence feeds.

I verified that the deployment package was successfully extracted and that the Lambda function contained the handler and individual threat-intelligence feed modules required by the parser.

I attached the threat-intel-lambda-layer to provide the third party Python dependencies required by the feed processing modules.

I created a test event using the security_news_rss panel key with a maximum of five results to validate the feed-processing workflow.

I sccessfully invoked the Lambda function and verified that the processed threat-intelligence data was written to the configured S3 bucket. The function created JSON objects for the requested feeds, confirming that the parser could retrieve the source data, process it, and store the resulting output in S3.

I created the private-db-threat-intelligence-feed-pull Lambda function using Python 3.12 and assigned the existing projectzero-lambda-feed-exec-role. This function serves as the private ingestion component of the pipeline, processing JSON objects generated by the public threat-intelligence parser and writing the data to PostgreSQL.

I configured the function with 256 MB of memory and a one-minute timeout to provide sufficient resources for retrieving JSON objects from S3 and inserting the processed data into PostgreSQL.

I configured environment variables containing the PostgreSQL connection parameters required by the Lambda function. Separating these values from the application code allows the database endpoint, database name, credentials, port, and TLS settings to be managed independently from the deployment package.

Database Connection Configuration

EC2_ENDPOINT=10.0.0.240
EC2_DB_NAME=projectzerodb
EC2_USERNAME=webapp_rw
EC2_PASSWORD=your-secret
EC2_PORT=5432
EC2_SSLMODE=require

I uploaded the deployment package containing the Python modules responsible for retrieving threat-intelligence JSON objects from S3 and ingesting the data into PostgreSQL.

I attached the custom threat-intel-lambda-layer, providing the Python dependencies required by the function, including the PostgreSQL client library used to connect to the database.

I added AWSLambdaVPCAccessExecutionRole to the Lambda execution role, allowing the function to create and manage the network interfaces required to operate inside the VPC.

I configured the threat-intelligence S3 bucket as an event source, allowing newly created threat-intelligence objects to automatically invoke the private ingestion Lambda. This creates an event-driven pipeline between S3 and the private Lambda function, automatically processing new threat-intelligence data as it becomes available.

I tested the complete ingestion workflow and received a statusCode: 200, confirming the Lambda function executed successfully. The execution logs verified that the function retrieved the threat-intelligence JSON object from S3, connected to PostgreSQL over the private VPC network, and inserted the processed data into dashboard.panel_feed.

I built the updated threat-intel-app and launched the Astro application on the web server, exposing the application through the server's network interface.

npm run build

I invoked the public threat-intelligence parser using the security_news_rss feed to generate data for the threat-intelligence application. The invocation returned a statusCode: 200, confirming that the RSS feed was successfully retrieved, processed, and written as a JSON object to S3 for downstream ingestion into the application.

After the public parser wrote the security_news_rss JSON object to S3, the configured S3 event notification automatically triggered the private-db-threat-intelligence-feed-pull Lambda function. The private Lambda then retrieved the JSON object and ingested the threat-intelligence data into PostgreSQL for use by the dashboard.

I retrieved the public IPv4 address of the EC2 web server to access the deployed threat-intelligence application from my browser on port 4321.

I accessed the deployed Threat Intelligence Dashboard and confirmed that the Security News RSS panel was populated with the feed data processed by the Lambda pipeline. This validated the complete flow from the public feed parser, through S3 and the private ingestion Lambda, into PostgreSQL and finally the web application.

After validating the deployment, I removed the temporary inbound rule for TCP port 4321 from the web server security group to prevent unnecessary public access to the application.

The port was only required to validate the application directly from the browser. Removing the rule after testing reduces the exposed attack surface of the EC2 instance.