Blog

AWS Lambda Incident Response

Organisations – both large and small – are increasingly leveraging Lambda serverless functions. From a business agility perspective, serverless has significant benefits. Lambda also brings security benefits – the managed runtime environment reduces the attack surface compared to a more traditional server environment. However, short runtime durations, the sheer volume of executions, and the dynamic and ephemeral nature of Lambda functions can make it difficult to detect, investigate and respond to a potential compromise. Under the AWS Shared Responsibility model, AWS secures the underlying Lambda execution environment, but it is up to the customer to secure functions themselves.

The main challenge security teams generally face when investigating a potentially compromised serverless system is determining exactly what data should be collected. Whilst you can’t typically access the underlying container on which Lambda runs (without hooking into libraries at deploy time to collect the writable /tmp folder at run time), you can grab:

  • The code of the Lambda function, and previous versions
  • Any environment variables it is set to use
  • Any logs in CloudWatch and CloudTrail

While the serverless nature of Lambda functions makes investigating potential malicious use difficult, there are a number of free tools and resources available to the security community that can help:

Community Resources

The best practical advice we’ve seen that focuses on how to search through Lambda logs, is this post on StackOverflow. It’s important to note, however, that this is more focused on investigating logs for errors instead of malicious activity.

Cado Security Resources

At Cado, we’ve published a number of materials on Lambda incident response including:

Example analysis of a compromised AWS Lambda function in the Cado platform

Example of the data captured by Cado’s open source varc tool running in a Lambda function

Official AWS Resources

AWS has also published a number of guides that may be useful when investigating a suspect Lambda function including:

About Cado Security

Cado Security is the cloud investigation and response automation company. The Cado platform leverages the scale, speed and automation of the cloud to effortlessly deliver forensic-level detail into cloud, container and serverless environments. Only Cado empowers security teams to investigate and respond at cloud speed.