This guide is the second part about how to get useful information of CloudTrail to alert your system when some configuration has changed. All configurations will be done using Terraform and Go and following the PCI DSS of AWS.
Objective
In the first parte of the guide, we learned about how to create our Trail and contect it with CloudWatch and S3. Now, we are going to process all that information using Lambda, written in Go, to alert our team each time a high severity event happen in our infrastructure. In this second part we are going to focus on whats in the red box. Here’s a picture of the complete architecture:
Now, let’s begin.
Implementing our Lambda function
From AWS, Lambda is a Function as a Service that allows you to run your code without a major configuration of it. You cloud trigger your function by many different places, once of them: CloudWatch. Since our Trail is recording all events in CloudTrail, we could trigger our function for processing them.
We will first write the code and later add it to our Terraform project for deploy it and integrate it with the rest of resources. Our code layout will be like this:
function/
main.go
types.go
severities.go
logHandler.go
Main.go
Here we capture all logs events comming from the CloudWatch Log Group of our Trail and process them using the logHandler. Finally, we log the event using a logger:
package main
import (
"context"
"errors"
"fmt"
"os"
"regexp"
"github.com/your-project-here/logger"
"github.com/your-project-here/cloudtrail/severities"
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
)
var (
defaultLogger = logger.New("cloudtrail-processor")
)
func logsEventHandler(ctx context.Context, logsEvent *events.CloudwatchLogsEvent) error {
cloudwatchLogsData, err := logsEvent.AWSLogs.Parse()
if err != nil {
defaultLogger.Warning(ctx, "invalid_event_type", []logger.Object{
logger.ErrObject(errors.New("error parsing cloudwatch event")),
})
return nil
}
return processLogMessage(ctx, &cloudwatchLogsData)
}
func processLogMessage(ctx context.Context, cloudwatchLogsData *events.CloudwatchLogsData) error {
handler, err := logGroupIdentifier(ctx, cloudwatchLogsData)
if err != nil {
defaultLogger.Warning(ctx, "log_group_not_found", []logger.Object{
logger.ErrObject(err),
})
return nil
}
for _, logEvent := range cloudwatchLogsData.LogEvents {
eventInfo, err := logHandler.LoadEvents(logEvent.Message)
if err != nil {
defaultLogger.Warning(ctx, "event_not_found", []logger.Object{
logger.ErrObject(errors.New("error identifying cloudtrail event")),
})
return err
}
if eventInfo.Event == severities.WhiteListedEvent { // this to avoid logging whitelisted events
return nil
}
properties := map[string]interface{}{}
properties["lambda"] = lambdaInfo
properties["events"] = eventInfo.LoggerMap
defaultLogger.Log(ctx, eventInfo.Event, eventInfo.LogLevel, properties)
}
return nil
}
func main() {
lambda.Start(logsEventHandler)
}
types.go
Before we implement the logHandler, we must define the types and structures for CloudTrail since the SDK for Go does not have it.
package main
// CloudTrailEvent represents a CloudTrail log event
type CloudTrailEvent struct {
AdditionalEventData interface{} `json:"additionalEventData,omitempty"`
AwsRegion string `json:"awsRegion"`
ErrorCode string `json:"errorCode,omitempty"`
ErrorMessage string `json:"errorMessage,omitempty"`
EventCategory string `json:"eventCategory"`
EventID string `json:"eventID"`
EventName string `json:"eventName"`
EventSource AWSService `json:"eventSource"`
EventTime string `json:"eventTime"`
EventType string `json:"eventType"`
EventVersion string `json:"eventVersion"`
ReadOnly bool `json:"readOnly,omitempty"`
RecipientAccountID string `json:"recipientAccountId"`
RequestParameters map[string]interface{} `json:"requestParameters"`
Resources []AWSResource `json:"resources,omitempty"`
ResponseElements map[string]interface{} `json:"responseElements,omitempty"`
ServiceEventDetails ServiceEventDetails `json:"serviceEventDetails,omitempty"`
SourceIPAddress string `json:"sourceIPAddress"`
UserAgent string `json:"userAgent"`
UserIdentity UserIdentity `json:"userIdentity"`
}
// AWSResource describes the ARN, account ID and type of the resource
type AWSResource struct {
Arn string `json:"ARN"`
AccountID string `json:"accountId"`
Type string `json:"type"`
}
// ServiceEventDetails identifies the service event, including what triggered the event and the result
type ServiceEventDetails struct {
KeyID string `json:"keyId"`
}
// UserIdentity identifies the service event, including what triggered the event and the result
type UserIdentity struct {
Kind string `json:"type,omitempty"` // because type is a reserved word
UserName string `json:"userName,omitempty"`
PrincipalID string `json:"principalId,omitempty"`
Arn string `json:"arn,omitempty"`
AccountID string `json:"accountId,omitempty"`
AccessKeyID string `json:"accessKeyId,omitempty"`
InvokedBy string `json:"invokedBy,omitempty"`
SessionContext interface{} `json:"sessionContext,omitempty"`
}
// CloudTrailHandler lambda transform logs.
type CloudTrailHandler struct {
}
severities.go
CloudTrail records events from multiple services, but the deal is there’s no an official list of all of them. But here I found one that has a pretty is good. Based on that list, and also searching in each service which events are recorded on with CloudTrail, I defined my own list and severities for each one.
package main
import "github.com/your-project-here/logger"
// AWSService is the enum for all AWS Services accepted
type AWSService string
const (
serviceApigateway AWSService = "apigateway.amazonaws.com"
serviceCloudtrail AWSService = "cloudtrail.amazonaws.com"
serviceConfig AWSService = "config.amazonaws.com"
serviceDynamoDB AWSService = "dynamodb.amazonaws.com"
serviceEC2 AWSService = "ec2.amazonaws.com"
serviceElasticFileSystem AWSService = "elasticfilesystem.amazonaws.com"
serviceIAM AWSService = "iam.amazonaws.com"
serviceKinesis AWSService = "kinesis.amazonaws.com"
serviceKMS AWSService = "kms.amazonaws.com"
serviceLambda AWSService = "lambda.amazonaws.com"
serviceS3 AWSService = "s3.amazonaws.com"
serviceSTS AWSService = "sts.amazonaws.com"
)
var (
// SeverityLow severity low level
SeverityLow logger.LogLevel = logger.Info
// SeverityMedium medium severity level
SeverityMedium logger.LogLevel = logger.Warning
// SeverityHigh high severity level
SeverityHigh logger.LogLevel = logger.Fatal
// KnownServices a map with the known or most used services of AWS
KnownServices = map[AWSService]bool{
serviceApigateway: true,
serviceCloudtrail: true,
serviceConfig: true,
serviceDynamoDB: true,
serviceEC2: true,
serviceElasticFileSystem: true,
serviceIAM: true,
serviceKinesis: true,
serviceKMS: true,
serviceLambda: true,
serviceS3: true,
serviceSTS: true,
}
// WhiteListedServices a map with the services that are not relevant or do not require any log actions
WhiteListedServices = map[AWSService]bool{
serviceSTS: true,
serviceKMS: true,
}
// WhiteListedEvents a map with the events that are not relevant or do not require any log actions
WhiteListedEvents = map[string]bool{
"DeleteNetworkInterface": true,
"DeletePolicyVersion": true,
}
// EventSeverities a map to associate a severity for each known event name
EventSeverities = map[string]logger.LogLevel{
// ServiceApigateway events
"CreateApi": logger.Info,
"CreateApiKey": logger.Info,
"CreateApiMapping": logger.Info,
"CreateDeployment": logger.Info,
"CreateGraphqlApi": logger.Info,
"CreateResource": logger.Info,
"CreateRestApi": logger.Info,
"DeleteApi": logger.Fatal,
"DeleteApiKey": logger.Fatal,
"DeleteApiMapping": logger.Fatal,
"DeleteGraphqlApi": logger.Fatal,
"DeleteIntegration": logger.Fatal,
"DeleteIntegrationResponse": logger.Fatal,
"DeleteMethod": logger.Fatal,
"DeleteMethodResponse": logger.Fatal,
"DeleteResource": logger.Fatal,
"DeleteRestApi": logger.Fatal,
"ExportApiKeys": logger.Info,
"ImportApiKeys": logger.Info,
"ImportRestApi": logger.Info,
"PutRestApi": logger.Warning,
"UpdateApi": logger.Warning,
"UpdateApiKey": logger.Warning,
"UpdateApiMapping": logger.Warning,
"UpdateGraphqlApi": logger.Warning,
"UpdateRestApi": logger.Fatal,
"UpdateStage": logger.Fatal,
// ServiceCloudtrail events
"StopLogging": logger.Fatal,
// ServiceConfig events
"DeleteConfigRule": logger.Fatal,
"DeleteConfigurationRecorder": logger.Fatal,
"DeleteDeliveryChannel": logger.Fatal,
"DeleteEvaluationResults": logger.Fatal,
"PutConfigurationRecorder": logger.Info,
"PutConfigRule": logger.Info,
"PutDeliveryChannel": logger.Info,
"PutEvaluations": logger.Info,
"StartConfigRulesEvaluationn": logger.Info,
"StartConfigurationRecorder": logger.Info,
"StopConfigurationRecorder": logger.Fatal,
// ServiceDynamoDB events
"CreateGlobalTable": logger.Info,
"CreateTable": logger.Info,
"DeleteBackup": logger.Fatal,
"DeleteTable": logger.Fatal,
"DescribeBackup": logger.Info,
"DescribeContinuousBackups": logger.Info,
"DescribeGlobalTable": logger.Info,
"DescribeLimits": logger.Info,
"DescribeReservedCapacity": logger.Info,
"DescribeReservedCapacityOfferings": logger.Info,
"DescribeScalableTargets": logger.Info,
"DescribeTable": logger.Info,
"DescribeTimeToLive": logger.Info,
"ListBackups": logger.Info,
"ListGlobalTables": logger.Info,
"ListTables": logger.Info,
"ListTagsOfResource": logger.Info,
"PurchaseReservedCapacityOfferings": logger.Fatal,
"RegisterScalableTarget": logger.Info,
"RestoreTableFromBackup": logger.Fatal,
"RestoreTableToPointInTime": logger.Fatal,
"TagResource": logger.Info,
"UntagResource": logger.Warning,
"UpdateGlobalTable": logger.Warning,
"UpdateTable": logger.Warning,
"UpdateTimeToLive": logger.Warning,
// ServiceEC2 events
"AllocateAddress": logger.Info,
"AssignPrivateIpAddresses": logger.Info,
"AssociateAddress": logger.Info,
"AssociateIamInstanceProfile": logger.Info,
"AssociateRouteTable": logger.Info,
"AssociateSubnetCidrBlock": logger.Info,
"AssociateVpcCidrBlock": logger.Info,
"AttachClassicLinkVpc": logger.Info,
"AttachInternetGateway": logger.Info,
"AttachNetworkInterface": logger.Info,
"AttachVpnGateway": logger.Info,
"AuthorizeSecurityGroupEgress": logger.Info,
"AuthorizeSecurityGroupIngress": logger.Info,
"CreateKeyPair": logger.Info,
"CreateNatGateway": logger.Info,
"CreateNetworkAcl": logger.Info,
"CreateNetworkAclEntry": logger.Info,
"CreateNetworkInterface": logger.Info,
"CreateRoute": logger.Info,
"CreateRouteTable": logger.Info,
"CreateSecurityGroup": logger.Info,
"CreateVpc": logger.Info,
"CreateVpcEndpoint": logger.Info,
"CreateVpcPeeringConnection": logger.Info,
"CreateVpnConnection": logger.Info,
"CreateVpnConnectionRoute": logger.Info,
"CreateVpnGateway": logger.Info,
"DeleteCustomerGateway": logger.Fatal,
"DeleteDhcpOptions": logger.Fatal,
"DeleteEgressOnlyInternetGateway": logger.Fatal,
"DeleteInternetGateway": logger.Fatal,
"DeleteKeyPair": logger.Fatal,
"DeleteNatGateway": logger.Fatal,
"DeleteNetworkAcl": logger.Fatal,
"DeleteNetworkAclEntry": logger.Fatal,
"DeleteNetworkInterface": logger.Warning,
"DeleteRoute": logger.Warning,
"DeleteRouteTable": logger.Fatal,
"DeleteSecurityGroup": logger.Fatal,
"DeleteVpcEndpoints": logger.Fatal,
"DeleteVpcPeeringConnection": logger.Fatal,
"DeleteVpnConnection": logger.Fatal,
"DeleteVpnConnectionRoute": logger.Fatal,
"DeleteVpnGateway": logger.Fatal,
"DetachClassicLinkVpc": logger.Warning,
"DetachInternetGateway": logger.Warning,
"DetachNetworkInterface": logger.Warning,
"DetachVolume": logger.Warning,
"DetachVpnGateway": logger.Warning,
"DisableVgwRoutePropagation": logger.Warning,
"DisableVpcClassicLink": logger.Warning,
"DisassociateAddress": logger.Info,
"DisassociateIamInstanceProfile": logger.Info,
"DisassociateRouteTable": logger.Info,
"DisassociateSubnetCidrBlock": logger.Info,
"DisassociateVpcCidrBlock": logger.Info,
"EnableVgwRoutePropagation": logger.Info,
"EnableVolumeIO": logger.Info,
"EnableVpcClassicLink": logger.Info,
"RevokeSecurityGroupEgress": logger.Fatal,
"RevokeSecurityGroupIngress": logger.Fatal,
"RunInstances": logger.Fatal,
"StartInstances": logger.Fatal,
"StopInstances": logger.Fatal,
"TerminateInstances": logger.Fatal,
// ServiceElasticFileSystem events
"CreateFileSystem": logger.Info,
"CreateMountTarget": logger.Info,
"DeleteFileSystem": logger.Fatal,
"DeleteMountTarget": logger.Fatal,
"ModifyMountTargetSecurityGroups": logger.Info,
// ServiceIAM events
"AddClientIDToOpenIDConnectProvider": logger.Info,
"AddRoleToInstanceProfile": logger.Info,
"AddUserToGroup": logger.Info,
"AttachGroupPolicy": logger.Info,
"AttachRolePolicy": logger.Info,
"AttachUserPolicy": logger.Info,
"ChangePassword": logger.Info,
"ConsoleLogin": logger.Info,
"CreateAccessKey": logger.Fatal,
"CreateAccountAlias": logger.Fatal,
"CreateGroup": logger.Info,
"CreateInstanceProfile": logger.Info,
"CreateLoginProfile": logger.Info,
"CreateOpenIDConnectProvider": logger.Info,
"CreatePolicy": logger.Info,
"CreatePolicyVersion": logger.Info,
"CreateRole": logger.Info,
"CreateSAMLProvider": logger.Info,
"CreateUser": logger.Info,
"CreateVirtualMFADevice": logger.Info,
"DeactivateMFADevice": logger.Fatal,
"DeleteAccessKey": logger.Fatal,
"DeleteAccountAlias": logger.Fatal,
"DeleteAccountPasswordPolicy": logger.Fatal,
"DeleteGroup": logger.Fatal,
"DeleteGroupPolicy": logger.Fatal,
"DeleteInstanceProfile": logger.Fatal,
"DeleteLoginProfile": logger.Fatal,
"DeleteOpenIDConnectProvider": logger.Fatal,
"DeletePolicy": logger.Fatal,
"DeletePolicyVersion": logger.Fatal,
"DeleteRole": logger.Fatal,
"DeleteRolePolicy": logger.Fatal,
"DeleteSAMLProvider": logger.Fatal,
"DeleteServerCertificate": logger.Fatal,
"DeleteSigningCertificate": logger.Fatal,
"DeleteSSHPublicKey": logger.Fatal,
"DeleteUser": logger.Fatal,
"DeleteUserPolicy": logger.Fatal,
"DeleteVirtualMFADevice": logger.Fatal,
"DetachGroupPolicy": logger.Warning,
"DetachRolePolicy": logger.Warning,
"DetachUserPolicy": logger.Warning,
"EnableMFADevice": logger.Info,
"PutGroupPolicy": logger.Info,
"PutRolePolicy": logger.Info,
"PutUserPolicy": logger.Info,
"RemoveClientIDFromOpenIDConnectProvider": logger.Warning,
"RemoveRoleFromInstanceProfile": logger.Warning,
"RemoveUserFromGroup": logger.Warning,
"ResyncMFADevice": logger.Info,
"SetDefaultPolicyVersion": logger.Info,
"UpdateAccessKey": logger.Fatal,
"UpdateAccountPasswordPolicy": logger.Fatal,
"UpdateAssumeRolePolicy": logger.Info,
"UpdateGroup": logger.Warning,
"UpdateLoginProfile": logger.Info,
"UpdateOpenIDConnectProviderThumbprint": logger.Warning,
"UpdateSAMLProvider": logger.Warning,
"UpdateServerCertificate": logger.Warning,
"UpdateSigningCertificate": logger.Warning,
"UpdateSSHPublicKey": logger.Fatal,
"UpdateUser": logger.Info,
"UploadServerCertificate": logger.Fatal,
"UploadSigningCertificate": logger.Fatal,
"UploadSSHPublicKey": logger.Fatal,
// ServiceKinesis events
"AddTagsToStream": logger.Info,
"CreateStream": logger.Info,
"DecreaseStreamRetentionPeriod": logger.Warning,
"DeleteStream": logger.Fatal,
"DeregisterStreamConsumer": logger.Warning,
"DescribeStream": logger.Info,
"DescribeStreamConsumer": logger.Info,
"DisableEnhancedMonitoring": logger.Warning,
"EnableEnhancedMonitoring": logger.Info,
"IncreaseStreamRetentionPeriod": logger.Warning,
"ListStreamConsumers": logger.Info,
"ListStreams": logger.Info,
"ListTagsForStream": logger.Info,
"MergeShards": logger.Info,
"RegisterStreamConsumer": logger.Info,
"RemoveTagsFromStream": logger.Warning,
"SplitShard": logger.Info,
"StartStreamEncryption": logger.Info,
"StopStreamEncryption": logger.Fatal,
"UpdateShardCount": logger.Fatal,
// ServiceKMS events
"CancelKeyDeletion": logger.Warning,
"CreateAlias": logger.Info,
"CreateGrant": logger.Info,
"CreateKey": logger.Info,
"Decrypt": logger.Info,
"DeleteAlias": logger.Fatal,
"DeleteExpiredKeyMaterial": logger.Fatal,
"DeleteKey": logger.Fatal,
"DescribeKey": logger.Info,
"DisableKey": logger.Fatal,
"EnableKey": logger.Info,
"EnableKeyRotation": logger.Info,
"Encrypt": logger.Info,
"GenerateDataKey": logger.Info,
"GenerateDataKeyPair": logger.Info,
"GenerateDataKeyPairWithoutPlaintext": logger.Info,
"GenerateDataKeyWithoutPlaintext": logger.Info,
"GenerateRandom": logger.Info,
"GetKeyPolicy": logger.Info,
"GetParametersForImport": logger.Info,
"ImportKeyMaterial": logger.Info,
"ListAliases": logger.Info,
"ListGrants": logger.Info,
"ReEncrypt": logger.Info,
"RotateKey": logger.Warning,
"ScheduleKeyDeletion": logger.Fatal,
// ServiceLambda events
"AddPermission": logger.Info,
"CreateEventSourceMapping": logger.Info,
"CreateFunction": logger.Info,
"DeleteEventSourceMapping": logger.Fatal,
"DeleteFunction": logger.Fatal,
"GetEventSourceMapping": logger.Info,
"GetFunction": logger.Info,
"GetFunctionConfiguration": logger.Info,
"GetPolicy": logger.Info,
"ListEventSourceMappings": logger.Info,
"ListFunctions": logger.Info,
"RemovePermission": logger.Fatal,
"UpdateEventSourceMapping": logger.Warning,
"UpdateFunctionCode": logger.Info,
"UpdateFunctionConfiguration": logger.Info,
// ServiceS3 events
"AbortMultipartUpload": logger.Info,
"CompleteMultipartUpload": logger.Info,
"CopyObject": logger.Info,
"CreateBucket": logger.Info,
"CreateMultipartUpload": logger.Info,
"DeleteBucket": logger.Fatal,
"DeleteBucketCors": logger.Fatal,
"DeleteBucketEncryption": logger.Fatal,
"DeleteBucketLifecycle": logger.Fatal,
"DeleteBucketPolicy": logger.Fatal,
"DeleteBucketReplication": logger.Fatal,
"DeleteBucketTagging": logger.Fatal,
"DeleteBucketWebsite": logger.Fatal,
"DeleteObject": logger.Fatal,
"DeleteObjects": logger.Fatal,
"DeletePublicAccessBlock": logger.Fatal,
"GetBucketCors": logger.Info,
"GetBucketEncryption": logger.Info,
"GetBucketLifecycle": logger.Info,
"GetBucketLocation": logger.Info,
"GetBucketLogging": logger.Info,
"GetBucketNotification": logger.Info,
"GetBucketPolicy": logger.Info,
"GetBucketReplication": logger.Info,
"GetBucketRequestPayment": logger.Info,
"GetBucketTagging": logger.Info,
"GetBucketVersioning": logger.Info,
"GetBucketWebsite": logger.Info,
"GetObject": logger.Info,
"GetObjectAcl": logger.Info,
"GetObjectTagging": logger.Info,
"GetObjectTorrent": logger.Info,
"GetPublicAccessBlock": logger.Info,
"HeadObject": logger.Info,
"ListBuckets": logger.Info,
"ListParts": logger.Info,
"PostObject": logger.Info,
"PutBucketAcl": logger.Warning,
"PutBucketCors": logger.Warning,
"PutBucketEncryption": logger.Warning,
"PutBucketLifecycle": logger.Warning,
"PutBucketLogging": logger.Warning,
"PutBucketNotification": logger.Warning,
"PutBucketPolicy": logger.Warning,
"PutBucketReplication": logger.Warning,
"PutBucketRequestPayment": logger.Warning,
"PutBucketTagging": logger.Warning,
"PutBucketVersioning": logger.Warning,
"PutBucketWebsite": logger.Warning,
"PutObject": logger.Info,
"PutObjectAcl": logger.Info,
"PutObjectTagging": logger.Info,
"PutPublicAccessBlock": logger.Info,
"RestoreObject": logger.Info,
"UploadPart": logger.Info,
"UploadPartCopy": logger.Info,
}
)
Now we have everything we need to process the CloudTrail events in the logHandler.
logHandler.go
The logHandler is going to parse the events from CloudWatch to the CloudTrail structure that we defined before. Later, it will process the logs using the processCloudTrailEvent method. It performs the following conditions:
- This method, check first if the event is in our whitelist, so no action is required and the event is skipped.
- It also check if the event name is related with the deletion of any resource in our infrastructure and assing a high severity for that event. If the source of the event is unknown, the we also log it to have visibility of it and add it later to our known services.
- Finally, we process the event checking our maps of events and severities associated to it and get all possible information calling the
getResourceInformationmethod. This method, get all info defined in ourcloudTrailEventstructure to give us insights about the event.
package main
import (
"encoding/json"
"regexp"
)
const (
// WhiteListedEvent log that describes a whitelisted event that requires no action to be performed
WhiteListedEvent = "aws_resource_white_listed"
resourceDeleted = "an_aws_resource_was_deleted"
resourceNotListed = "aws_resource_not_listed"
lowSeverityEvent = "low_severity_aws_event_happened"
mediumSeverityEvent = "medium_severity_aws_event_happened"
highSeverityEvent = "high_severity_aws_event_happened"
)
var (
deleteResource = regexp.MustCompile(`^([D,d]elete).*|([R,r]evoke).*|([R,r]emove).*`)
)
// LoadEvents determine the type of event
func (log CloudTrailHandler) LoadEvents(eventsData string) (*EventInfo, error) {
cloudTrailEvent := CloudTrailEvent{}
err := json.Unmarshal([]byte(eventsData), &cloudTrailEvent)
if err != nil {
return nil, err
}
return processCloudTrailEvent(cloudTrailEvent), nil
}
func processCloudTrailEvent(cloudTrailEvent CloudTrailEvent) *EventInfo {
if WhiteListedServices[cloudTrailEvent.EventSource] || WhiteListedEvents[cloudTrailEvent.EventName] {
return &EventInfo{
LogLevel: logger.Info,
Event: WhiteListedEvent,
}
}
deleteEvent := deleteResource.FindString(cloudTrailEvent.EventName)
if deleteEvent != "" {
return &EventInfo{
LogLevel: logger.Fatal,
Event: resourceDeleted,
LoggerMap: getResourceInformation(cloudTrailEvent),
}
}
if !KnownServices[cloudTrailEvent.EventSource] {
return &EventInfo{
LogLevel: logger.Info,
Event: resourceNotListed,
LoggerMap: getResourceInformation(cloudTrailEvent),
}
}
return processDefaultKnownEvent(cloudTrailEvent)
}
func processDefaultKnownEvent(cloudTrailEvent CloudTrailEvent) *EventInfo {
eventInfo := &EventInfo{
LogLevel: EventSeverities[cloudTrailEvent.EventName],
LoggerMap: getResourceInformation(cloudTrailEvent),
}
switch eventInfo.LogLevel {
case logger.Info:
eventInfo.Event = lowSeverityEvent
case logger.Warning:
eventInfo.Event = mediumSeverityEvent
case logger.Fatal:
eventInfo.Event = highSeverityEvent
}
return eventInfo
}
func getResourceInformation(cloudTrailEvent CloudTrailEvent) map[string]interface{} {
infoMessage := map[string]interface{}{
"s_event_category": cloudTrailEvent.EventCategory,
"s_event_name": cloudTrailEvent.EventName,
"s_event_source": cloudTrailEvent.EventSource,
"s_event_time": cloudTrailEvent.EventTime,
"s_event_type": cloudTrailEvent.EventType,
"s_source_ip_address": cloudTrailEvent.SourceIPAddress,
"s_user_agent": cloudTrailEvent.UserAgent,
"s_user_identity_account_id": cloudTrailEvent.UserIdentity.AccountID,
"s_user_identity_arn": cloudTrailEvent.UserIdentity.Arn,
"s_user_identity_name": cloudTrailEvent.UserIdentity.UserName,
"s_user_identity_type": cloudTrailEvent.UserIdentity.Kind,
"s_error_code": cloudTrailEvent.ErrorCode,
"s_error_message": cloudTrailEvent.ErrorMessage,
}
if len(cloudTrailEvent.Resources) != 0 {
infoMessage["s_resource_arn"] = cloudTrailEvent.Resources[0].Arn
infoMessage["s_resource_type"] = cloudTrailEvent.Resources[0].Type
infoMessage["s_resource_account_id"] = cloudTrailEvent.Resources[0].AccountID
}
return infoMessage
}
Let’s go back to Terraform
With that we concluded all code we need to process CloudTrail logs from CloudWatch. It’s time to return to our Terraform project and define the Lambda function that will be triggered from CloudWatch…
To our project, we will add the Lambda terraform file like this:
cloudtrail/
cloudwatch/
iam/
kms/
*lambda/ # !NEW
cloudtrail_handler.tf
var.tf
s3/
trails/
backend.tf
main.tf
project.auto.tfvars
vars.tf
cloudtrail_handler.tf
Our Lambda in terraform could be like this:
# Our code is stored in S3
data "aws_s3_bucket_object" "s3_key_lambda" {
bucket = var.s3_bucket
key = var.s3_key
}
# Lambda function
resource "aws_lambda_function" "lambda" {
# Function properties
s3_bucket = local.s3_bucket
s3_key = local.s3_key
s3_object_version = data.aws_s3_bucket_object.s3_key_lambda[0].version_id
function_name = "cloudtrail-logs-handler"
role = var.iam_module.cloudtrail_handler_role
handler = "main"
runtime = "provided.al2"
timeout = var.timeout
memory_size = var.memory_size
reserved_concurrent_executions = 15
publish = true
depends_on = [aws_iam_role_policy_attachment.lambda_attach]
tags = var.tags
}
### CloudWatch Logs subscription
resource "aws_cloudwatch_log_subscription_filter" "logs_subscription" {
name = "logs-subscription-filer"
log_group_name = "CloudTrailLogs"
filter_pattern = ""
distribution = "ByLogStream"
destination_arn = aws_lambda_alias.lambda.arn
}
resource "aws_lambda_permission" "allow_cloudwatch_logs_trigger" {
action = "lambda:InvokeFunction"
principal = "logs.amazonaws.com"
function_name = aws_lambda_function.lambda.arn
source_arn = "arn:aws:logs:${var.region}:${data.aws_caller_identity.current.account_id}:log-group:CloudTrailLogs:*"
}
Later define a role with a policy that will be used by the function (I won’t specify that in this article).
And the last thing we do is add the lambda module in the main.tf of our project like this:
module "lambda" {
source = "./lambda"
region = var.region
account_id = var.account_id
s3_bucket = var.s3_bucket # this bucket is where we store the function's code, not the CloudTrail logs
iam_module = module.iam.data
cloudwatch_module = module.cloudwatch.cloudtrail_log_group
tags = local.tags
}
And we are done!
Conclusion
We’ve created a CloudTrail Trail that records all things that happens in our infrastructure and now we have insights about all the events procesing them using Lambda. We’ve done all this using the PCI best practices.