For SAP Landscape hosted in cloud Infrastructure, keeping Linux patching up to date is a very key requirement to address security hardening, vulnerability assessment and for other mandatory compliance requirement on a regular basis. Applying Linux patching for a large SAP Landscape is a very time-consuming efforts, as it does require a clean stop and start of all SAP applications in the landscape during this exercise.

This document particularly describes the procedure that how OS patching can be fully automated using AWS System Manager for a SAP landscape hosted on AWS EC2 running on Redhat Linux with multiple SAP applications running on HANA DB.

AWS System Manager is an easy and sophisticated tool to achieve such automation.

https://docs.aws.amazon.com/systems-manager/latest/userguide/what-is-systems-manager.html

Pre-requisite

1. To perform the OS patching for EC2 using AWS SSM document ‘AWS-RunPatchBaseline’, need to define a patch baseline as per your patching classification and associate the baseline as Default with all targets EC2 instances using Patch groups.

Please refer the AWS document https://docs.aws.amazon.com/systems-manager/latest/userguide/about-patch-baselines.html for more detail.

Patch baseline and associated Patch Groups

1. Define necessary tagging for all EC2 instances for stop and start automation using SSM. I defined my custom SSM document for SAP Start, Stop by using reference defined in this blog, to cover all ABAP, JAVA, Dual Stack and other non-netweaver based systems like BODS, OpenText etc.

https://aws.amazon.com/blogs/awsforsap/automate-start-or-stop-of-distributed-sap-hana-systems-using-aws-systems-manager/

AWS SSM Document

To achieve the full patching automation, AWS SSM document is created with following parameters.

1. AutomationAssumeRole : Required to perform action on EC2 instances by SSM Automation.
2. PatchGroup : Value of EC2 tag ‘Patch Group’ to target EC2 instances for patching.

More parameters can be introduced as per overall design approach and requirements.

Parameter for Patching SSM Automation

Step 1: Query all EC2 instances for the given Patch Group using EC2 DescribeInstances API. This output is not being used in subsequent steps, but still used to validate all EC2 instances in given Patch Group for troubleshoot purpose.

Step 1

StepName.OutputName work as a variable for subsequent automation steps. e.g  {{ QUERY_INSTANCES_PATCHGROUP.PatchInstances }} is used for EC2 InstanceId as string list.

Step 2: Query all EC2 instances for the given Patch Group that are in ‘Stopped’ status. This is to put all such EC2 instances back in status ‘Stopped’ after patching. Many unused or on-demand purpose instances can be in ‘Stopped’ status, but good to keep them patch as per patching baseline. This is very common for Non-Prod systems in cloud hosting.

Step 2

This is achieved using the AWS SSM action ‘aws:executeScript’ by writing a small Python code to collect the EC2 instances as string list and Number of Stopped EC2 Instances, to use for further decision steps.

Step 3: Query all EC2 instances with status ‘Running’ for the given Patch Group using EC2 DescribeInstances API.

Step 3

Step 4: Query all SAP SID for the EC2 instances with status ‘Running’. List of SID is required to call SAP Start/Stop AWS SSM Automation document, as SAP Start/Stop SSM document are defined based on SID as input parameter.

This is achieved using the AWS SSM action ‘aws:executeScript’ by writing a small Python code to collect the SID string list array.

Step 4

Step 5: Use action ‘aws:branch’ to check if there are any SAP applications that should be stopped before patching.

Step 5

Step 6: In case, there are running SAP applications, automation execution will jump to this step. In this step, another custom SSM automation is called using SSM action ‘aws:executeAutomation’, to stop all SAP applications in parallel. In the ‘DocumetName’ input, provide the full arn name for the SSM document used to stop the applications.

Step 6

Step 7: Branch Step to check if any ‘Stopped’ instances should be bring to status ‘Running’ to apply patch baseline.

Step 7

Step 8: In this step, Start all EC2 instances in given Patch Group that are in ‘stopped’ status to ‘running’ for Patching.

Step 8

Step 9: In this step, execute action  ‘aws:runCommand’ by calling AWS owned document ‘AWS-RunPatchBaseline’ by specifying Targets based on given Patch Group input parameter. This will initiate patching for all EC2 instances in parallel with tag key ‘Patch Group’ value as input parameter.

below is an edit mode to showcase inputs for APPLY_PATCH step.

Step 9

Step 10: Branch Step to check if automation should call step to start back SAP application based on output NUM_OF_RUNNINGSID > 0 from step #4.

Step 10

Step 11: In case, there are running SAP applications, automation execution will jump to this step to start back the SAP applications. In this step, another custom SSM automation is called using SSM action ‘aws:executeAutomation’, to start all SAP applications in parallel. In the ‘DocumetName’ input, provide the full arn name for the SSM document used to start the applications.

Step 11

Step 12: Branch Step to check if any instances that were in status ‘Stopped’ before patching, should be stopped back.

Step 12

Step 13: Automation execution will move to this step based on variable NO_OF_STOPPED_INSTANCES value more than 0 from Step #2.

Step 13

Step 14: Last step to mark the end of automation.

Step 14

Once SSM document is finalized, it can be run manually or also can be scheduled using AWS SSM ‘State Manager’ to automatically apply patch on a defined maintenance schedule.

On successful execution, automation document output will appear as below.

Patch automation output

Detail look at step ‘STOP_SAP_APPS_DB’ will show all the steps for stopping of individual SID. All SIDs are stopped in parallel.

SAP application stop steps

Detail look at step ‘APPLY_PATCH’ will display the output of ‘AWS-RunPatchBaseline’ command document for EC2 targets for the given Patch Group.

AWS-RunPatchBaseline command output

Detail look at step ‘START_BACK_RUNNING_SAP_APPS’ will show all the steps for starting of individual SID. All SIDs are started in parallel.

SAP Start steps

With this SSM document, patching automation for large number of EC2 instances can be performed with zero-touch with a clean, soft stop and start of SAP applications hosted on EC2.

Based on multiple run with monthly patching baseline, overall execution is observed of less than 30 minutes including stopping of all applications, applying patching across more than 100 instances in parallel and starting back of all application after patching for the given Patch Group.

References:

https://docs.aws.amazon.com/systems-manager/latest/userguide/what-is-systems-manager.html

https://aws.amazon.com/blogs/awsforsap/automate-start-or-stop-of-distributed-sap-hana-systems-using-aws-systems-manager/