5

DevSecOps with AWS – ChatOps with AWS and AWS Developer Tools – Part 1

 1 year ago
source link: https://dev.to/aws-builders/devsecops-with-aws-chatops-with-aws-and-aws-developer-tools-part-1-26n2
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client
Cover image for DevSecOps with AWS – ChatOps with AWS and AWS Developer Tools – Part 1

13 1 1 1 1

 

DevSecOps with AWS – ChatOps with AWS and AWS Developer Tools – Part 1

Level 200

Table of Contents

The “boom” of AI is transforming the industry sure thing you are listening to in all social media comments and many applications based on ChatGPT API, an application programming interface that allows users to experience an AI-based chatbot. It is one of the most innovative AI models ever created. Imagine if you could create and respond to all IT operations task through chat channels centralizing the management of infrastructure and applications, as well as to automate and streamline your workflows (ChatOps) just move single pane of glass to your familiar chat interface to get the job done, for example, Microsoft teams or slack.
Today, base on previous post, you can learn how use AWS Chatbot for integrate Microsoft Teams and take actions like:

  • Approve pipelines.
  • Start, Stop pipelines.
  • Get alarms.
  • Describe resources.
  • Receive predefined CloudWatch dashboards interactively.
  • Retrieve Logs Insights logs to troubleshoot issues directly from the chat thread. ...

There aren't limits!

Hands On

Requirements

  • cdk >= 2.73.0
  • AWS CLI >= 2.7.0
  • Python >= 3.10.4
  • Pytest >= 7.1.3
  • cdk-nag >=2.18.44
  • checkov >= 2.1.229

AWS Services

  • AWS Cloud Development Kit (CDK): is an open-source software development framework to define your cloud application resources using familiar programming languages.
  • AWS Identity and Access Management (IAM): Securely manage identities and access to AWS services and resources.
  • AWS IAM Identity Center (Successor to AWS Single Sign-On): helps you securely create or connect your workforce identities and manage their access centrally across AWS accounts and applications.
  • AWS CodeCommit: secure, highly scalable, managed source control service that hosts private Git repositories.
  • AWS CodeBuild: fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy.
  • AWS CodeCommit: secure, highly scalable, managed source control service that hosts private Git repositories.
  • AWS CodePipeline: fully managed continuous delivery service that helps you automate your release pipelines for fast and reliable application and infrastructure updates.
  • AWS Key Management Service (AWS KMS): lets you create, manage, and control cryptographic keys across your applications and more than 100 AWS services.
  • AWS CloudFormation: Speed up cloud provisioning with infrastructure as code as code
  • AWS CloudFormation: Speed up cloud provisioning with infrastructure as code
  • AWS Lambda: A serverless compute service that lets you run code without provisioning or managing servers, build workload-based cluster scaling logic, maintain event integrations, or manage runtimes.
  • AWS Chatbot: Monitor, operate, and troubleshoot your AWS resources with interactive ChatOps

Solution Overview

ChatOps and AWS chatbot


Figure 1- Solution Overview

The Figure 1 depicts the general architecture for this demonstration:

  1. The pipeline start or change.
  2. The pipeline events are captured and trigger a lambda service
  3. Lambda service load the secret webhook url from secrets manager and send the notification card to Microsoft teams Channel.
  4. There is an alternative for sending notifications easy and simple, using codestar notifications or SNS with email subscription.
  5. Finally the manual steps as approve transition is invoked from chat using AWS Chatbot Service.

Step by Step

  1. Create webhook URL in Microsoft Teams -> Create an Incoming Webhook - Teams | Microsoft Learn

  2. Create a custom lambda for sending messages to Microsoft Teams.

The following is a fragment code, according to Monitoring CodePipeline events
Code example- Here the library pymsteams is used to construct a message card with the events information.

import os
import pymsteams
from get_secret import get_secret


def set_state(message, state, connection_card, activity_text):
    if message['detail']['state'] == 'STARTED':
        state.text(message['detail']['state'])
        state.activityText(activity_text)
        state.activityImage(
            "https://support.content.office.net/en-us/media/6b8c0bff-7ddc-4bff-9101-8360f8c8a727.png")
    elif message['detail']['state'] == 'SUCCEEDED':
        state.activityText(activity_text)
        state.activityImage(
            "https://support.content.office.net/en-us/media/773afccb-4687-4b9f-8a89-8b32f640b27d.png")
    elif message['detail']['state'] == 'FAILED':
        state.activityText(activity_text)
        state.activityImage(
            "https://support.content.office.net/en-us/media/c9ed80c9-a24c-40b0-ba59-a6af25dc56fb.png")

    else:
        state.activityImage(
            "https://support.content.office.net/en-us/media/47588200-0bf0-46e9-977e-e668978f459c.png")

        state.text(message['detail']['state'])
    connection_card.addSection(state)


def lambda_handler(event, context):
    print(event)
    #
    # Get event source
    #

    region = os.environ['AWS_REGION']
    # message = event['Records'][0]['Sns']['Message']
    message = event

    webhook_secret_url = get_secret()
    # message = json.loads(message)

    print(message)
    if message['source'] == "aws.codepipeline":
        team_message = pymsteams.connectorcard(webhook_secret_url)

        # create the section action
        state = pymsteams.cardsection()
        pipeline = message['detail']['pipeline']
        # Section Title
        state.title("State")

        if message["detail-type"] == "CodePipeline Action Execution State Change":

            team_message.title(
                f"Action Execution State Change - {message['detail-type']} in {message['detail']['pipeline']} Pipeline")
            team_message.color("7b9683")

            # Add text to the message.
            team_message.text(
                f"Action Execution State Change in pipeline {message['detail']['pipeline']} in account {message['account']} in region {message['region']} \n "

            )

            # create the section stage

            stage = pymsteams.cardsection()
            # create the section action
            action = pymsteams.cardsection()
            # Section Title
            action.title("Action")
            action.text(message['detail']['action'])

            team_message.summary("Approval deployment")
            team_message.addSection(stage)

            set_state(message=message, state=state, connection_card=team_message,
                      activity_text='Action Execution State Change! -Review if you need take action')
            team_message.addSection(action)

            if 'execution-result' in message['detail']:
                # create the section result
                result = pymsteams.cardsection()
                # Section Title
                result.title("Result")
                summary = message['detail']['execution-result'].get('external-execution-summary', '')
                result_summ = message['detail']['execution-result'].get('external-execution-state', '')
                result_url = message['detail']['execution-result'].get('external-execution-url', '')
                result.text(f"{summary} - {result_summ}")
                if result_url != '':
                    team_message.addLinkButton("Go to the console",
                                               result_url)

                team_message.addSection(result)

        elif message["detail-type"] == "CodePipeline Pipeline Execution State Change":
            print("State change ... \n")
            team_message.title(
                f"State Change in {message['detail']['pipeline']} Pipeline")
            team_message.color("7b9683")
            # Add text to the message.
            team_message.text(
                f"Stage Change in pipeline {message['detail']['pipeline']} in account {message['account']} in region {message['region']} \n "

            )

            set_state(message=message, state=state, connection_card=team_message, activity_text='')

        elif message["detail-type"] == "CodePipeline Stage Execution State Change":

            print("Stage change ... \n")
            team_message.title(
                f"Stage Change in {message['detail']['pipeline']} Pipeline")
            team_message.color("7b9683")

            # Add text to the message.
            team_message.text(
                f"Stage Change in pipeline {message['detail']['pipeline']} in account {message['account']} in region {message['region']} \n "

            )
            pipeline = message['detail']['pipeline']
            # create the section stage

            stage = pymsteams.cardsection()
            # create the section action
            # Section Title
            stage.title(f"Stage {message['detail']['stage']}")
            set_state(message=message, state=state, connection_card=team_message,
                      activity_text="")

            team_message.summary("Stage Change")
            team_message.addSection(stage)

            team_message.addLinkButton("Go to the console",
                                       f"https://{region}.console.aws.amazon.com/codesuite/codepipeline/pipelines/{pipeline}/view?region={region}")

        team_message.send()
...

  1. Create event rules for any codepipeline event.

Finally, in the construct L3 the lambda and events are created:

from aws_cdk import (
    SecretValue,
    aws_secretsmanager as secretsmanager,
    aws_lambda as _lambda,
    aws_codedeploy as codedeploy,
    aws_codepipeline as pipeline,
    aws_events as events,
    aws_events_targets as targets,
    Duration
)
import os
from constructs import Construct


class LambdaNotification(Construct):

    def __init__(self, scope: Construct, construct_id: str,
                 props: dict = None,
                 pipeline: pipeline.IPipeline = None,
                 emails: list = None,

                 **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        dirname = os.path.dirname(__file__)

        # lambda definitions for events
        lambda_notification = _lambda.Function(self, f"lambdanotification_{props['project_name']}",
                                               runtime= _lambda.Runtime.PYTHON_3_9,
                                               code=_lambda.Code.from_asset(
                                                   os.path.join(dirname, "lambda_notifications/lambda_function")),
                                               handler='app.lambda_handler',
                                               function_name=f"lambdanotification_{props['project_name']}",
                                               timeout=Duration.seconds(70),
                                               environment={
                                                   "webhook_secret_name": f"webhook_{props['project_name']}_channel"
                                               }
                                               )

        secret = secretsmanager.Secret(self, f"webhook_{props['project_name']}_channel",
                                       description="WebHook Url for notifications channel in microsoft teams",
                                       secret_name=f"webhook_{props['project_name']}_channel",
                                       secret_string_value=SecretValue.unsafe_plain_text(props["webhook_ops_channel"])
                                       )
        secret.grant_read(lambda_notification)
        alias = _lambda.Alias(self, f"{props['project_name']}-LambdaAlias",
                              alias_name="Prod_2", version=lambda_notification.current_version)

        codedeploy.LambdaDeploymentGroup(self, f"{props['project_name']}-lambda-DeploymentGroup",
                                         alias=alias,
                                         deployment_config=codedeploy.LambdaDeploymentConfig.ALL_AT_ONCE
                                         )
        pipe_states = pipeline.on_event(id=f"{props['project_name']}-pipelinevent",
                                        description="Event for activate lambda when app_pipeline",
                                        event_pattern=events.EventPattern(
                                            detail={
                                                "state": [
                                                    "SUCCEEDED",
                                                    "INPROGRESS",
                                                    "FAILED",
                                                    "SUPERSEDED",
                                                    "STOPPED",
                                                    "STOPPING",
                                                    "CANCELED",
                                                    "STARTED",
                                                    "RESUMED",
                                                    "ABANDONED"

                                                ]
                                            }
                                        )
                                        )

        # Add subscription event lambda
         pipe_states.add_target(targets.LambdaFunction(lambda_notification))

Here, the event bridge service has one rule that listen the events from CodePipeline.

Secrets Manager saves the webhook URL for Microsoft teams channel.

  1. Add notifications object into the pipeline.

Finally, the notifications are added to the main pipeline construct:

...
# Enable notifications
        if enable_notifications == "true":
            LambdaNotification(self, "Notifications", props= props,pipeline= pipeline.pipeline)
        # Define Outputs
...

In the environment options file include the enable_notifications option:

# Multi Environment setup
devsecops_account: "123456789012"
devsecops_region: "us-east-2"
enable_notifications: "true"

Push your changes or redeploy the pipeline stack.

Now, you can get the notifications on teams channel.

Teams Notifications

Figure 2- Teams Notifications.

you can watch the notifications into your channel.

In the next blog you can learn how configure and integrate AWS chatbot.

You can find the code here ⬇️ ⬇️

GitHub logo velez94 / cdkv2_pipeline_multienvironment

Repository for cdk pipelines for multiaccount environment

CDK Pipelines Multi Environment Devployment

This is a project for CDK development with Python for creating multi AWS account deployment.

Solution Overview

Solution Overview - Enhanced CDK Pipeline
Solution Overview – Enhanced CDK Pipeline

The Figure shows the steps to accomplish this task. Also, it shows a cross account pipeline using AWS CodePipeline, AWS CodeCommit, AWS Codebuild and AWS CloudFormation. But, how can you construct this pipeline secure and with the minimum effort? The answer [CDK Pipelines](https://docs.aws.amazon.com/cdk/v2/guide/cdk_pipeline.html).

This pipeline is a default pipeline composed by next steps 1.The changes are detected and activate de pipeline. For this demo the branch master is the default branch. 2.The CDK project is synthesized if is aligned with AWS Security Best practices. 3.The pipeline run self-update action. 4.The unit test runs, and its report is published in codebuild reports group. 5.The SAST…

In the next blog you can learn how configure and integrate AWS Chatbot.
Thanks for reading and sharing 😃


Recommend

  • 61

  • 37
    • www.tuicool.com 5 years ago
    • Cache

    Introducing AWS Chatbot: ChatOps for AWS

    DevOps teams widely use chat rooms as communications hubs where team members interact—both with one another and with the systems that they operate. Bots help facilitate these interactions, delivering important notificatio...

  • 83

    在搭建 GitLab 系列 ChatOps 时需要掌握一些基础的技能知识,整个 ChatOps 涉及到很多方面,我们至少需要掌握以下技能: Linux 示例中我们主要以 CentOS 7 为主,但是容器内的系统会多种多样 Docker 所有的环境...

  • 6

    This post was co-authored by Teddy Ferdinand. Who is working as Cloud Security Architect 🐻 Introduction In this series, we will talk about the emergence of the D...

  • 7
    • blog.jetbrains.com 3 years ago
    • Cache

    Webinar: ChatOps the Easy Way with Abbot

    .NET Tools Webinars Webinar: ChatOps the Easy Way with Abbot...

  • 6

    V2EX  ›  程序员 小公司用有什么推荐的 ChatOps 工具?   zealinux · 14 小时 30 分钟前 ·...

  • 5
    • www.servicemesher.com 3 years ago
    • Cache

    使用Jenkins X实现ChatOps

    使用Jenkins X实现ChatOps · Service Mesh|服务网格中文社区 本文介绍了使用Jenkins X实现ChatOps。很好的阐述了如何使用Jenkins X来实践ChatOps,文中手把手带我们从零开始完成了一次Kubernetes Nat...

  • 5
    • blog.thinkeridea.com 3 years ago
    • Cache

    【ChatOps系列】ChatOps简介

    要说 ChatOps 就不得不说 DevOps,DevOps 是来源于 Development 和 Operations 的一个组合词,顾名思义,是一系列过程、方法与系统的统称,旨在促进开发、测试和运维人员之间的沟通与协作。简单来说,是通过引入一系列的「工具」,通过三种不同角色的开发成员间...

  • 3
    • www.servicemesher.com 3 years ago
    • Cache

    GitOps与ChatOps的落地实践

    说到 GitOps 和 ChatOps ,那就不得不谈到 DevOps 。 DevOps 作为一种文化,旨在促进开发、测试和运维人员之间的沟通与协作。而促进合作的方式,往往是使用一系列工具,完成这三个角色的相互协作。这带来的好处也是显而易见的:更快的交付速度和更低的人力成本...

  • 7

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK