AWS Lambda Incident Response

Organisations – both large and small – are increasingly leveraging Lambda serverless functions. From a business agility perspective, serverless has significant benefits. Lambda also brings security benefits – the managed runtime environment reduces the attack surface compared to a more traditional server environment. However, short runtime durations, the sheer volume of executions, and the dynamic and ephemeral nature of Lambda functions can make it difficult to detect, investigate and respond to a potential compromise. Under the AWS Shared Responsibility model, AWS secures the underlying Lambda execution environment, but it is up to the customer to secure functions themselves.

The main challenge security teams generally face when investigating a potentially compromised serverless system is determining exactly what data should be collected. Whilst you can’t typically access the underlying container on which Lambda runs (without hooking into libraries at deploy time to collect the writable /tmp folder at run time), you can grab:

• The code of the Lambda function, and previous versions
• Any environment variables it is set to use
• Any logs in CloudWatch and CloudTrail

While the serverless nature of Lambda functions makes investigating potential malicious use difficult, there are a number of free tools and resources available to the security community that can help:

Community Resources

The best practical advice we’ve seen that focuses on how to search through Lambda logs, is this post on StackOverflow. It’s important to note, however, that this is more focused on investigating logs for errors instead of malicious activity.

Cado Security Resources

At Cado, we've published a number of materials on Lambda incident response including:

• Ultimate Guide to Incident Response in AWS: covers best practices and tips for investigating and recovering from incidents in various AWS services including Lambda
• Cado Discovers Denonia: The First Malware Specifically Targeting Lambda: covers a full technical analysis of the first publicly-known case of malware specifically designed to execute in an AWS Lambda environment
• Cado Security Extends Support To Serverless Environments: covers key features of the Cado platform that enables security teams to investigate AWS Lambda serverless functions

Official AWS Resources

AWS has also published a number of guides that may be useful when investigating a suspect Lambda function including:

• Accessing Amazon CloudWatch logs for AWS Lambda
• Security in AWS Lambda