Pass Amazon AWS DevOps Engineer Professional Exam in First Attempt Easily

SDLC Automation (Domain 1)

1. CICD Overview

So welcome to this section on domain one, SDLC automation. I'm just going to introduce you to the concept of CICD. But if you already know this, you can just skip ahead. So what is CICD? CI/CD stands for continuous integration and then continuous delivery or deployment. So let's start with continuous integration. So the way it works is that developers frequently push code to a code repository. It could be GitHub or a big bucket of code committers, whatever you want. And so, as we can see, we push the code as often as possible, and then as soon as the code is pushed, there will be a testing server or a build server that checks the code as soon as it's pushed. And so the reason we do this is that we want to make sure that the code can be built and is tested correctly. Why do we do this? Because then the developer will get feedback about the tests and checks that have passed and failed. And so, therefore, we can improve our code, find bugs early, and fix them very early as well. So this way we can deliver code faster, as it is tested continuously and then deployed often. The result of this when you have continuous integration is that you have happier developers, as they're unblocked. They don't have to run their own tests on their builds; they just push the code to a code repository, and then the build server will do the heavy lifting for them of testing the code, building it, and maybe even notifying the developer of how it went. So continuous integration is the first step in CI CD. Now we have continuous delivery. So we want to make sure that whenever software is built, it can be released reliably whenever we need it. And we want to make sure that the deployments happen very often and are quick because we want to move away from doing one release every three months to doing five releases a day. and that you can only do five releases a day if you have something called continuous delivery. So that means that the deployment will be fully automated. So we could use technology such as code deploy, Jenkins, spinnaker, et cetera, et cetera. So what does the continuous delivery pipeline look like? Well, we have our happy developer, and he pushes code often. And then the build server will get the code, build it, and test it. And then the deployment server, after the build server is done, we'll get the build output and say, "Okay, I want to deploy this new version to my application server so that they go from version one to version two at a high level." So this is the idea behind continuous delivery. So I said that there was CD, which meant continuous delivery or continuous deployments. You need to know the difference between these two things. Continuous delivery implies that you deploy frequently using automation. It may involve a manual step. For example, if you need to approve a deployment, usually when you have a production deployment, maybe you want to manually approve that production deployment, and therefore you would use a manual step, and then you would be in the setting of continuous delivery. Even though there is a manual step here, the deployment itself is still automated, can be repeated, and is safe to do. So continuous delivery has a lot of advantages to it. On the other hand, continuous deployment is complete automation. That means that every code change you will do in your source repository will be deployed all the way to production. That means there's no manual intervention or approval. This is something that is usually done by doing a lot of monitoring. And then if there's a bug, you just push a new code fix in your repository, and then it makes it all the way through to the end of the pipeline. So this is important to understand the difference. Delivery and deployments are very similar, except delivery may involve a manual step, whereas continuous deployment is automated all the way to pushing to production. Okay, so now let's go back to AWS. What is the CI CD Stack looking for? AWS. These are the steps you can do in your CICD: code build, test, deploy, and provision. So, for code, you'd use AWS code commit, as we'll see in the section, or GitHub or another third-party repository. Then for building and testing, you could use AWS CodeBuild, Jenkins CI, or any third-party CI servers. and for deploying and provisioning beanstalk or provisioning infrastructure with cloud formation, for example. And for deployment, especially, we would use code deploy. Finally, to orchestrate everything, you would use an AWS code pipeline. So in this section we'll see couldcommit, could build, could deploy code pipeline. Jenkins, and we'll even see CodeStar as well as an overview. This will allow you to understand how to do CICD and SD DLC automation on AWS. So get ready, and I will see you in the next lecture.

2. CodeCommit – Overview

So let's start with a code commit. Now, you should already know what code commit is, and if you do, please move on to the next lecture. But I'll just give you a quick background recap if you need it. So version control is the ability to understand the various changes that happen to the code over time. And thanks to your version controls, you can roll back through commits. And so all of these can be enabled using a version control system. and a very popular one is called GitGit. A git repository can live on one's machine, but it can also live in a central online repository. And so these repositories can be GitHub, for example, or AWS code commits. The benefits of using a central online repository for your git are that you can collaborate with other developers, you can make sure that the code is backed up somewhere, and you can make sure it's fully viewable and auditable. And you can roll back at any point in time. So this is where code commitment comes in. Now, the code commit looks like this. The developers will push the code up to our code repository, and they can be expensive. So the industry has several players. They have GitHub, where GitHub offers a bunch of free public repositories and also prepaid ones or paid private ones based on if you're a company or not, a big bucket, etc. And then we have AWS code commits. So it's a private git repository for your AWS account, and there's no size limit on the repository, so you can scale seamlessly. And it's fully managed and highly available, and the code lives on your AWS cloud accounts. That means that from a security perspective, all the code is on your account. So you have increased security and maybe even increased compliance. It's also secure. That means that it's fully encrypted, you have access control, and it's integrated with other services in AWS, such as Jenkins for building or other CI tools. So we'll see in this section all there is to know about committing. Let's get started.

3. CodeCommit - First Repo & HTTPS config

Okay, so let's get started. And we'll use the service name "code commit." Now, what is "code commit?" Code commit is a way for us to create git repositories on AWS. And the advantage of using code commit, for example, over another service like GitHub is being able to have your code privately held within your VPC, whereas on GitHub it is on someone else's cloud, even though there are some private offerings as well. So here we'll go ahead and create our first repository, and I'll call it my Web page. So, Web page, here we go. We could add a description to a repository, and we could also add tags, but for now we'll just go ahead and create it. Now with this repository, we have two ways of connecting to it. And for the rest of the tutorial, we will use HTTP. But you have the option to use HTTPS or SSH to connect to your git repository if you know how to set up SSH. Now, because we are using the root account, we cannot configure SSH connections, and HTTPS connections are not recommended. It says that you should consider signing in as an IM user and then setting up your connection. So that's true. We don't want to compromise our credentials. So, why don't we go to im and create an agent that can connect to our code commit repository? So let me go in here. I will go to users and I will add a user. For the username, I will put Stefan, and I will give myself programmatic and AWS management console access with an auto-generated password that is fine; that will be required to be reset when I first connect. Click on Permissions; all looks good. I need to add users to a group. Why don't we go ahead and create a group of administrators? So create a group called admin, and we'll add administrator access to this group. Click on "create group," then click on "tags," and we'll just click on "review" and create that user. So now I have created my user, and I will download my CSV right now, which contains my access key ID, my secret access key, and my password. And now I'm in a bit more secure space because I'm a user. So if you click on this im user and we go to security credentials, we see that we have access keys. We know we just downloaded them, but we have two options. At the bottom, we have SSH keys for AWS code commit, and we also have HTTP get credentials for AWS code commit. So these are the two ways to connect to your code commit repository. And so, as I said, we are going to setup HTTPS, but you might as well set up SSH keys on your own if you wanted to. It's a bit more cumbersome and complicated. And because I want everyone to be able to follow along, I will choose HTTP, so I'll click on "generate HTTPS keys," and here is my Get username that has been generated, and I also have a password. So I'm going to save my username in a text file on the screen you cannot see, and I will also go ahead and save the password. Okay, then I'll click on close, and I will not see the credentials ever again. Excellent. So now that we have set up our git credentials for the im user that I've just created, why don't we go and commit and get our repository set up? So the goal is to take our code from the CICD demo folder that you have downloaded and upload it into our code commit repository. So for this, what we need to do is make a clone, and cloning a repository means taking whatever is inside, and currently, well, there is nothing; it's an empty repository. But we'll take whatever is inside and put it on our computer to clone it. For this, I'll use the clone URL drop-down on the right side of the top right, and I'll select clone Https. This will copy this whole URL right here, which represents the HTTPS URL for my code commit repository. So I will go ahead and go to my command line and type git clone, and the URL we just obtained from before is now asking me for a username. So for this, I will enter the username that I just downloaded from before, and it's asking me for the password for my HTTP credentials. So I'll just go ahead and paste that and press Enter. Checking connectivity is done. So that means that everything worked. And the warning says you appear to have cloned an empty repository. Well, that's perfectly fine because we've just created that repository. So now if I do LS to list the files, I have my web page as a new directory that has appeared right here. And if I look into this directory, well, there is nothing. So this is fine. We're able to initialise our git repository directly from here and test our HTTPS credentials. Now, if the git command doesn't work for you, this is definitely something that you should have on your computer as an AWS DevOps. So please make sure to go online and install git. There's tonnes of tutorials to install git. It's very simple. Okay, so now we've established our connectivity into our repository and are using HTTP. But we could have done SS well, which isn't complicated. And now what we want to do is push code into it. So why don't you follow me into the next lecture? Well, we'll just do that.

4. CodeCommit - clone, add, commit, push

So next, let's push some code into our repository. And for this, I'm going to go ahead and copy all these files right here that I have from my CI CDdemo folder, and I'm going to paste them into the MyWeb page repository that we had from before. So that's it. Now I'm using VS Code as my code editor. And as you can see, VS Code is integrated with git. So as soon as I copied these files, they became green because they're waiting to be added by git and pushed into my repository. But we will not use Vs. code for this. We'll use the command line because you need to know how that works from the command line. So back in our command line, we're going to go to my web page, and my command line is also integrated with git. As such, it shows that I am a master. And this is something that comes from my Mac laptop, which is the Sh command line, if you're curious about installing it. As a result, I'm on Master. And so if I do LS, now I see all my files, and if I do get status, it shows that currently all these files are untracked. So we want to track them because then we want to push them into a repository. So we do get add to track them. And you could specify the name of the file, for example, at spec yamo, or you could specify a dot, and this will add all the files in my repository into my future commit. So I'll be added, and there's a small bonus. And if I do get status again, now we can see that all these changes are ready to be committed. So all these files have been added to my Git repository. So first and foremost, I must commit them. So to commit, you do get committed, minus the need to add a little message and say "first commit." And then you close the quotes. And now we have created our first commit. However, if you return to our repository, for example, if you refresh this, we can see that it is still empty. It is very empty. So the reason why it is empty is because even though we have committed, it's been committed locally. And so we need to push the changes into our upstream repository. And for this, I do get pushed. Now it is asking me again for the username and the password. So I'll go ahead and reenter those. And this is something you can avoid if you're using SSH. It won't ask you for a username and password all the time, but for now we'll just deal with this. And here we go. Our branch has been pushed into our remote repository. So let's go back to the code commit and refresh this page. And now we will see that, yes, all my files have been pushed into my repository, into this branch called Master, and this is the default branch. Now, we'll be talking about branches very shortly in the next lecture. But for now, I want you to realise that we have pushed our code to our repository. So, to summarize, if click on commit, then we cansee here that our first commit we just did right now, first commit has been done 1 minute ago. And if we go to branches, we can see that we are on Master and that is the default branch. And the last commit, again, was one minute ago. So, perfect. We have our code into code commit. That's a great start. Now let's go ahead and learn a little bit more about branching code and pull requests.

5. CodeCommit - Branches and Pull Requests

So let's get a bit more experience with Git and understand the concept of branching and why it is super important to use it. So, let's return to our repository's master branch. If you look at the commit, we had a first commit right here, but let's go ahead and add a second commit. So I'll go into my editor and click on "index HTML," and I'll call it "simple congratulations" at the bottom, congratulations v. two. And so I've changed this file. As you can see, this file has been modified. So vs. code indicates that it has been altered. Let's return to our CLI and perform get status. Yes, this file has been modified. Then we'll add it. So we'll do git add index HTML, git commit message modified index to version 2, and then git push to push our file into the code commit. Now, as you can see, this commit has been pushed, so if we go back to our code and then click on index HTML and scroll down, as we can see now, it shows congratulations v. 2. And if you go to commit, as we can see, there's a new commit that appeared, and it's done right now. and the author is me. And if you click on the commitID, we can see what has changed. And here we can see that congratulations is now congratulations v. two. And so, this is really helpful here. I've been able to add my commits into codecommits, and we can see what has changed over time, and that's the whole power of git. But something you have to realise is that I'm working on this thing called Master, and Master is the default branch. When you create a repository, it is basically the main branch of your code. But for example, if there's me and some other developers working on the same repository, we may both want to contribute features to it, and as such, maybe I will do something and they will do something else, and these two things will conflict, and therefore we will not be able to resolve our conflicts. And we'll have issues because we both want to push for mastery. And so this is where the concept of branches comes in really, really handy. So what is a branch? I really like these diagrams on the Atlantic website, so I'll just use them. A branch is a way to deviate from the master branch and create something like a little feature branch or a big feature branch. And these branches are like a tree, and you can push commits to these branches over time and they will not impact master. And then when you're ready, you will merge them back into the master. So we'll see how that works. So let's go and see how we can create a branch right now. So as we remember here, there's only one branch, right? So one thing I can do is type get checkout minusB to create a branch, and I'll call it my feature. And here we are in a new branch called My Feature. I will go ahead and change this to "congratulations" v. three. And so if we do get status again, then we have a modified HTML index HTML. So do get add, then commit modified index two v three, and then push, which, as you can see, gets push and does not work this time. Why? Well, because we're not a master anymore and it says that to push this new branch called "my feature" into the origin, we need to have this little bit longer command just to run once, but to say, okay, we want to push our branch, but on top of it, create the "my feature" branch on the remote repository. So here we go. Now this is running and we have a new branch called My Feature. So let's go into a code commit and see what happened. I'm going to refresh this page, and if we are in the master, click on index HTML and scroll down. Well, we're still on congratulations v. two. But if I click on here and switch to my feature, there's a new branch that appears. Now we can see that for this branch, for the index HTML file, it says "Congratulations!" So now we have two different branches in our repository. So let me show you. We have the master branch and the my feature branch, and they both have different kinds of codes. And so this comes in really handy because now I can work on this feature on my own, my feature. And when I'm ready, I will merge this back into the master branch. Now, this can be complicated, but you have to understand the concept of branching going into the DevOps exam because they will ask you a lot of questions about different environments and different features, and the answer is to use branches. Okay, so if you click on the left-hand side branches, Now we have the master branch and the feature branch. As we can see, the last commit onto the master branch has been modified from index to version 2. And the most recent commit on my feature branch was titled "modify index to version 3." So what can we do here? Because we want to merge now that my feature is complete, we want to merge the changes from my feature into the master. Well, for this, we'll go back to the dock to show you visually what happens. We'll do a merge. And so for this, for example, we have our master branch and our feature branch. And now we want to merge the feature branch into the master branch by creating something called a merge commit. And so this is addressed in code commit doing pull request. So here we'll go and create a pull request and we're saying, okay, the destination of this pull request is Master and the source is going to be my feature branch. And I'll click on Compare, and it says okay, there's no merge conflict between my feature and Master. And so I can create a feature called feature V3, and that's the name of my pull request. And I'll go ahead and click on "Create a pull request." So this pull request is a way for me to tell my other developers, maybe my boss, to say, "Hey, I've done some changes, and here are all the changes that I've done." And if you're happy with those changes, then merge them; if not, don't merge them. So as of now, I've created a pull request. It's right here, but it hasn't been merged. And so if we go back to our code currently on Master, on index HTML, it just says "Congratulations v Back to our full request, If I'm happy with this full request, I'll click on it and look at it. It's merging. And the changes are something that I'm kind of happy with. We could obviously go ahead and have some activity around it. For example, see the commit and so on, and when we're ready, click on merge. And here we go. Our full request is going to be merged. There are three ways to do a merge; don't worry; we'll just choose the first one and then delete the source branch, my feature after merging, effectively ensuring that this feature branch out here is no longer active. So this is fine. Yes, we want to delete the source branch, my feature, and click on merge build requests. And here we go. My feature has been merged into Master, and thankfully now if we go to my webpage and look at the branches, there is only one branch: the Master branch. And if we go to index HTML, it says "Congratulations!" So through this process of creating a branch, developing the feature on the branch, and then merging back into Master, we are following the best practises when using a git repository. And by using the pull request, we allow people to review our code and make sure that everything looks fine before the code gets merged. And so going into the DevOps exam, it's really important to understand these things because if you don't use feature branches, you will have trouble versioning your code. So the exam will ask you, for example, "Hey, we have a new development branch or we have a new feature; should we create a new repository or should we create a new branch?" Well, the answer is to create a new branch because this is how git works and it's meant to be used this way. Alright, that's it for this lecture. I will see you at the next lecture.

6. CodeCommit - Securing the Repository and Branches

Okay, so now we've seen the concept of branches and how we should use them to develop features and then merge them into Master. Now there's something called a really bad practice, which is to directly push to Master. Master is considered to be the working version of your code, and as such, you definitely do not want most of the time to push to Master right away because that would be quite tricky. You would not be able to get positive feedback, and the code might break during a push. So it's best practise to always use branches. But how do we enforce best practices? Well, for this, we can limit pushes and merges to branches in AWS code commits. So this is an article that I'll obviously link to obviously. And there is an example of a policy we should put in place: separate our junior developers from our senior developers and say, "Okay, only the senior developers have the right to merge into Master and to push into Master, and the junior developers will not have that right." So let's see how that can be accomplished in So I have my user, and let's say that I want to create a group of junior developers. So I'll create a new group, and I'll call it Junior Developers. And these are the junior developers, and they should not be able to push code to master. So for now, I will not add any kind of policy. Click on "next step" and I'll just create the group. Now in this group we're going to attach an inline policy. So we'll create one, and we'll click here, and we can use a policy generator. But here we'll use a custom policy because it will be a little bit more simple. So I'll say I cannot push to master in a code commit. And if we go back to this article right here from the documentation, we'll just copy this entire thing and change what's needed and look at this policy because it's a bit more complicated than the types of policies you've seen before, but you definitely need to understand what's happening. So here we create a policy, and it is a denial. So why do we create a deny policy instead of an allow one? I'm not sure if you remember, but any explicit denial takes precedence over anything else in I am. So if there is an explicit allow and an explicit deny, then the deny is what goes into effect. So we deny the following in code commit: push, delete, branch, putfile, merge branches, merge, pull requests, all of these things. So we need to say what isthe resource we want to deny on? So maybe I'll just use a star right here, a star for account, and a star for repository, which means any repository commit that we will ever create. And if the reference is Master or prod, the condition is good. So we actually don't have prod, so we'll just remove this, but we could have a prod branch; that is definitely possible. So here we're using a condition block, and condition is something that again is a bit more advanced in imit's way of saying, "Okay, you'll deny everything here on all these resources that match this pattern, but it has to also validate a condition, which is that it should be the code commit reference Master, and Master represents a branch." So excellent, now we're saying here, "Deny everything on Master." We'll apply this policy, and okay, it's done. So now we should test it. Well, how do we test it? For example, let's go to the user me, and I'm going to attach myself to this group of junior developers. I'll add your group. So now, even though I am an administrator because I have administrator access from the group administrator, I also am in the junior developers group, which has this policy, which is an explicit deny, and so the explicit deny should definitely take precedence over the administrator access. But let's go ahead and test this. So, let's get back to code commit. I'll go to my webpage. I'll do git checkout master because I want to move into the master branch, and I'll do git pull to pull the changes from master because, remember? Master has changed thanks to the merger request, and so now I'm going to go ahead and change my file and say congratulations. V-4, and since we've just incremented that file, we'll doget status to see if it's changed, and it has. It has changed, so git add index HTML and git commit, and here we'll say modify the index to v4, and it's been committed locally, so everything worked, but now we want to push that change into a code commit. So we get pushed, and as we can see, this is an error. We need to push some refs to this. Why? We don't have the permission to push changes to this branch, so this is really cool. This cannot push to Master's encode commit policy even though we are admins because it is explicitly preventing us from pushing into Master. So Master definitely does not work, and what can I do? Well, I can maybe check out a new branch for my new feature, and maybe here I'll push my commit into my new feature. So I'll copy this, and yes, I was able to push into my new feature. So now if I refresh this page, well, I do have a new branch called my new feature, so I'm able to work in my new feature for a while. When I'm ready, I'm going to create a pull request. So I'll say, "Hey, please merge this senior developer because I want a senior developer to merge this at some point," and I'll say, "Okay, create the pull request, and now a senior developer can go ahead, review this pull request, and then when they're ready, merge it into Master." We can just keep on working like this, but it's a bit more secure. So again, this is something that can come up at the exam: how to protect master branches from M's unwanted commits. So now you know it's by using an IP policy that has a "deny" option in it. So I hope you like this, and I will see you in the next lecture.

Hide