Cloud Threat Hunting: Investigating Lateral Movement

By Maya Levine, Technical Marketing Engineer, and Lior Sonntag, Security Analyst

A sign of a truly sophisticated attack in the cloud is the ability to move laterally undetected. Doing so successfully requires knowledge of many techniques. In this latest installation of the Cloud Threat Hunting: Attack and Investigation Series, we present the most involved attack flow yet. We will break down all of the steps a threat actor took to successfully exfiltrate data out of an AWS account.

This attack began with a compromised pair of AWS access keys. The first thing the threat actor does is obtain temporary credentials using the

get-session-token 

command. This is a form of defense evasion – if the permanent credentials the threat actor started with are deleted or disabled, they will still have access to the environment.

Next, the threat actor gets the user name associated with the credentials: Low-Priv.

In order to understand what the permissions of this user will allow them to do, they run the

list-user-policies

command. Turns out, they do not have permissions to run that command. This is true for the two other commands they attempt: 

list-attached-user-policies

and 

list groups for users

 which shows them managed policies for the user.

At this point, they could resort to a brute force attack to obtain the permissions. This will require a lot of time and effort. So instead, they check for access to the account’s event history. Using the credentials they already obtained, they run the 

lookup-events 

command to check for access to CloudTrail logs. The output shows them that the user has read-only permissions to CloudTrail logs.

This will give the threat actor the ability to understand what they can accomplish with this compromised user. They begin by searching CloudTrail events for that username. The results show that the user ran the

AssumeRole

command to the

 LambdaCreator

role. They also learn the default region of the user, the role session name, and most importantly – the role ARN. They copy the ARN to later assume the role.

The threat actor will check what events the

LambdaCreator

role did in that session. Once they assume the role and move laterally, they will be able to execute those same commands. The first event was 

UpdateFunctionConfiguration

 on a function called

Automation-UpdateSSMParam

. The second event was 

UpdateFunctionCode

 on that same function.

Finishing up the reconnaissance from the CloudTrail logs, they will now run the

sts assume role

command to escalate their privileges to this

LambdaCreator

role.

Once in the new role – they try the

get-function

command but get an Access Denied error. Meaning, they cannot access the function’s code or environmental variables. So, they will turn to a form of social engineering: googling the function name. They find an AWS walkthrough which tells them that the role attached to this lambda function has permissions for

AmazonSSMFullAccess

. The name implies that if the threat actor is able to gain access to this role, they can easily abuse it, move laterally, and escalate their privileges. Additionally, the actor learns that the function uses boto3 library.

The walkthrough also tells them that the AWS name for this function (

Automation-UpdateSSMParam

) is identical to the name in the victim’s account. This is a common mistake users make—defining resource names exactly as AWS defines them in their examples. It makes the jobs of threat actors infinitely easier. A simple Google search gave them all of the information they needed to continue in the attack.

Remember there were two permissions the LambdaCreator role used in the CloudTrail logs. Abusing 

UpdateFunctionCode

 involves changing the function’s code to be malicious and retrieving environmental variables through a reverse shell. This could break the functionality of the function and alert the victim of an attacker’s presence. So, the better alternative is using the 

UpdateFunctionConfiguration

 permission to inject a malicious lambda layer to the function. This does not affect the functionality of the function whatsoever.

The code of Lambda layers is stored separately from the rest of the function. These layers can be shared cross-account. First, the actor creates the malicious layer in their own AWS account.  They insert malicious code into a boto3 library. When the lambda function is invoked, it will use this altered boto3 library instead of the default one. This malicious code will result in the lambda sending all it’s environments variables to the attackers IP, including the STS token of the function’s role.

The actor makes this malicious layer publically available. Then, switching to the victim’s account, they run the

UpdateFunctionConfiguration

command to insert the malicious backdoor layer.

They wait for the function to be invoked and set up a listener on port 4433. Once invoked, the function sends over all of it’s environmental parameters including: access key ID, secret access key, and session token. Together, these comprise the STS token of the function’s role.

Once again, the threat actor can impersonate this, move laterally, and escalate their privileges. In this new role, they have SSMFullAccess permission. They will now repeat their previous technique of looking at CloudTrail to list the history events – this time related to the SSM service.

One of the events is

SSM start session

API call. This will start a connection to an EC2 instance. The actor copies the instance target ID. Using the STS token from before, they start a session to that target ID and login to that EC2 instance.

From there, they search for metadata service which by default is found under the IP 169.254.169.254. They now extract the EC2 instance’s role name: DynamoDBFullAccess. Users often name roles based on their permissions, which indicates to threat actors the scope of that role. The name clearly states that the role has full access to DynamoDB.

The role name is added to the query, and the metadata service will output it’s STS token.

From within the EC2 instance, they list the DynamoDB tables. The actor knows that the EC2 has DynamoDB full access, so they can scan the table. This reveals sensitive customer information like name, phone number, email, and last 4 digits of credit card numbers.

Now how will the attacker exfiltrate this information? One option is to use the EC2 role STS token outside of the instance environment – but this can be easily detected as suspicious behavior. A more covert option is NTP or DNS tunneling. The attacker uses NTP, which AWS frequently uses to sync the clocks of instances over the network. This allows the attacker to exfiltrate the data relatively silently and under the radar.

Investigation

Even with an attack as sophisticated as this, CloudGuard will generate multiple relevant alerts for an investigation within the time frame of the attack.

The first alert is Permissions scan attempt executed by CLI.

This will show all the commands executed by the attacker from the CLI…

ListUserPolicies

, 

ListGroupsForUser

, 

ListAttachedUserPolicies

, 

GetSessionToken

, and 

AssumeRole

. The 

AssumeRole 

is indicating when the threat actor moved from the original LowPriv role to the LambdaCreator role.

The 

GetSessionToken 

command also prompts this alert: Temporary Creds created from permanent user creds.

The next logical step in an investigation is to take all the tokens from this event and create a new query that searches for CloudTrail logs with these tokens.

Note that the 

AssumeRole 

command’s response has another token that was added to this query.

This reveals more actions taken by the attacker once they moved laterally to a new role. Specifically, the

UpdateFunctionConfiguration 

call which was run on the Automation-UpdateSSMParam function.

Back in the alerts list, we see that there are 2 alerts related to the Automation-UpdateSSMParam function. The first is Lambda configuration update.

The second is Lambda Layer was added from an external account.

The logs show that the 

UpdateFunctionConfiguration

 event has no known identity. They also show that a backdoor layer was inserted from a completely different AWS account.

From this same view we also see an 

AssumeRole

 command…the attacker escalated their privileges to the LambdaSSM role. As part of the investigative process of “following the token”, we copy the access key ID of the role and insert it into the same query that has been built out slowly.

There were several alerts generated for the LambdaSSM role as well…the first is Abuse of access token generated by STS dedicated for lambda.

The logs show a 

StartSession

 call from an external source with a Kali Linux user agent, therefore, this was not initiated from the lambda function itself. Most importantly, the 

StartSession

 call requests an ID.

Searching for this ID reveals two alerts. First, is an alert called Suspicious NTP packets volume per session. This shows the malformed NTP traffic that was the attacker exfiltrating the data.

The Statistics tab clearly shows the uptick in the traffic.

The other alert is Anomaly Detection – Anomalous Network Traffic. As the name suggests, it utilizes machine learning to do anomaly detection of network traffic.

The Statistics tab shows a pretty active traffic trend with ups and downs. However, the 

StartSession

 action caused a significant enough uptick to be deemed an anomaly.

Altogether, all of these events indicate an incident. Throughout the course of the investigation, we followed the token. This allowed us to see the entire attack from beginning to end. From the low-priv-user, the move to automation-updateSsmParam, to 

StartSession

 with the SSM role.

Also Featured In