6. EBS Operation: Volume Resizing
Okay. So now let’s talk about EBS volume and resizing. So you can increase the size of your EBS volumes, and you can increase the size of the IUPs if it’s relevant. And after resizing an EBS volume, what you need to do is repartition your drive. That means that after you increase the size of your volume, there’s going to be more size available. But your EC2 instance will not know about it until you repartition your drive and tell it to use that new space. So when you increase the size, it’s possible for the volume to go into a long optimization phase. This is to reorder the blocks, but during that time, the volume is still usable and working as expected. You cannot decrease the size of your EBS volume. If you wanted to do so, you would need to create a smaller volume, copy the data from one volume to another, and then attach it properly.
So if we have a look at what happens, let’s say we have a 250-gigabyte volume and then we modify it to make it 1 TB. So it goes into the “modifying space” phase, then it goes into the “optimising phase,” and then you will be into the “completed phase” with a 1 TB EBS volume. But as I said, from the perspective of your EC2 instance, there will be 250 GB of partitions already being used and mounted properly. But you need to repartition your EBS volume to add on that extra missing 750 GB so that your EC2 instance will be able to use that extra size. So let’s have a look at how we can do it. So let me launch an instance, and I’m going to go quickly, but I’m going to create an Amazon Linux 2 T 2 Micro next. And since we’re starting with 8 GB of storage, I’ll add a security group, and then I can select the security group that I’m already very familiar with. And then Democrats take over, and we’re good to go. Okay, so my instance is now starting, and let me wait for it to be done so I can connect to it using EC2 instance connect, and then if I do Lsvlk, I can see that I have one device of type XVDA with 8GB of type 8GB.This is my main volume on my EC2 instance. Okay, if I do DS minus H, as we can see, the size is 8 GB, of which 5 GB is used and 6.6 GB is available.
Okay? So now what I’m going to do is change the size of that main EBS volume. So for this, I’m going to go into my volumes, and in here, I’m going to take this volume and right-click, and I’m going to modify it. Now there are multiple ways I can modify the volume. I can modify the type itself. So, okay, from GP-2 to GP-3 to IO-1 to IO-2 to magnetic. And then within each category, you can also change some attributes. So say I want to go from 8GB to 10GB, okay? And then it will click on “modify.” So it takes some time for the performance change to take full effect. Okay? And we need to extend the OS file system on the volume to use any newly allocated space. So this is what we’re going to see right here. And there are commands to do it on Linux and Windows. So I click on “yes,” and now the “modify volume request” is being done, so I’m going to have to wait for it to be done. So if we have a look at the status on the right hand side, it says “in use” and then “modifying 0%.” So as the volume is being modified, this is going to revolve. And after the modifying phase is done, it’s going to go into an optimising phase that you can see right now.
So this was quick because I just added 2GB, so that was pretty quick. So now if we run the same commands right here, so I’ll do lsblk, and as we can see now, xvda is ten gigabytes, but xvda One is eight gigabytes. That means that, yes, as you can see, the OS has not been extended to use the full size of the disk. And we can also see it by doing GF minus H. And, as we can see, there are still 8 GB on this mounted disk. Okay? So as you can see, we’re still not using the 10 GB in here. And so to do so, we need to extend a Linux file system after resizing a volume, and there are commands on how to do that. So as you can see, we’re running this command right now, and we’ve run this command as well. But there is a way for you to do it. So you need to use the grow part command, and we need to change the name of the device. So, as we can see right here, we need to dopseudo growpart and then the partition number, which is 1. So let’s have a look and see if that works. So we’ll start with pseudogroparts, then devxvdaone 1, devxvdaone 2, and, of course, VDA one. Okay, so here we go. So sudoku parts dev xvda one changed my partition, and now there are two sizes.
And so now if I do DF minus H, as you can see, or LSB, okay, as you can see, now my XVDA drive is using 10GB. And so for my DF H to return ten gigs, I believe I have to reboot my instance. So to do a reboot, I’m going to reboot it directly from the EC instance page. So let’s go ahead and reboot that instance, and then click on Connect using easy-to-instance Connect. And as we can see, Now, if I do lsblk, it shows ten gigs, and if I do GF minus H, it shows ten gigs as well. So I have extended my partition, and I believe you could do it without a reboot. So if you go into the documentation, once we’re done with the pseudo-growth art, you can extend the file system directly using the XFS growth. If you’re using the XFS file system or reusing the XFS file system, you can use pseudo-resize 2 FS as well to resize it. Okay? And then the DfMany search should work as well. So it was just an example, but at least you can see how you can resize an EBS volume. And that’s it. I hope you like it, and I will see you in the next lecture.
7. EBS Operation: Snapshots
Okay? So now let’s do a deep dive into EBS snapshots so we can know that we can make a snapshot at any point in time of your EBS volume, okay? And it’s not necessary to detach your volume to do a snapshot, but it’s recommended because if the volume is attached to an instance, that means that you may have some kind of consistency issue. Now, you can copy a snapshot across an Az or region. That means that you can take a US-East flight in your SB. So you have an EBS volume here. Then you have an instance in your S1B. And so you take an EBS volume, you make a snapshot from it, and then you restore it in another AZ, which is the action of migrating snapshots. Okay? Next, there is a service with an EBS called Amazon Data Lifecycle Manager, which is used to automate the creation, retention, and deletion of EBS snapshots and EBS-backed AMIs. So it’s a way for you to schedule backups, do cross-border international copies automatically, or delete outdated backups. Using policy, you use resource tags to identify the resources you want to backup.
For example, EC2 instances or EBS volumes And so here’s an example: We can tag an EBS volume with EnvironmentProd to make sure that it automatically gets backed up by Amazon Data Lifecycle Manager. Alternatively, we can tag this Environment Product directly in EasyToInstance. The instance would then be backed up, as well as CBS volumes, so that it could be imported into Data Lifecycle Manager, where we could create snapshots and AMIS. It cannot be used to manage snapshots or AMIS created outside of DLM. So everything has to be contained within the Data Cycle Manager, and it cannot be used to manage instant store-backed AMIS. Next, one little feature that could be really, really helpful but really, really expensive is called Fast Snapshots Restore (FSR).So the idea is that your EBS snapshots are internally stored in Amazon S3, so you don’t see them, but this is how they’re stored internally in AWS.
And so when you restore a snapshot into an EBS volume and you access a specific block of data, okay, there’s a latency on this I operation, the first one, because the block is going to be pulled from Amazon’s asteroid every time. And so there’s a solution to force the initialization of the entire volume by reading the entire volume from the EC instance using the DD or the FIO command. And this is the previous solution. And so the idea is that you start a simple instance. You attach an EBS volume and read the entire volume. Then, therefore, all the blocks will be accessed and initiated. Or you can do something called a “fast snapshot restore” of FSR. So the idea with FSR is that you can enable it on any snapshot at any time. And what FSR will do is that it will do this initialization of the entire volume for you, okay? And then, once a volume is fully initialized, you can restore it into an EBS volume, on which there will be no I/O latency.
So, to show you, an EBS snapshot being restored into an EBS volume may only have a few blocks initialized. And as you read the blocks, they will be pulled from Amazon Kindle. But if you enable FSR on the snapshot and FSR is complete, what will happen is that the entire EBS volume is going to be ready and available to use at best performance right away. So you enable this for a particular snapshot in particular. It’s built per minute, and it’s honestly very, very expensive. So leaving FSR on all the time can be very costly, around $500 per month. But the use case in the real world is to take a snapshot, enable FSR so that AWS will initialise the snapshot for you, and then you restore it into an EBS volume, and then you disable FSR, okay? Finally, you can use data lifecycle managers (DLM) to enable this directly on snapshots, though this is not recommended and is extremely expensive. So I’ll see you in the next lecture for some hands-on practice.
8. EBS Operation: Snapshots Hands On
Okay? So if we take one volume, we can easily create a snapshot by right clicking and doing “Create Snapshots” and then creating the snapshots. So it’s quite easy on the left side, and then you will see this snapshot. Now, if we take a look at these snapshots, I’ll right-click on them. As you can see, it is not encrypted, and there is no fast snapshot restore. So if I click on Manage Fast Snapshot Restore, you can enable it for a specific AZ. Just be very careful. And I wouldn’t enable it because Fast Snapshot Restore is currently built per minute, with a 1 hour minimum. And when you enable it, at least in my region, this costs zero points per hour per snapshot per AC. So this can be quite expensive. But the idea is that if I enable it from this snapshot, it will be fully optimized.
Now, I don’t think this makes a lot of sense for snapshots of 2 GB. You’re not going to see any performance impact. But if you had a snapshot of several terabytes, it could make sense to use FSR just for the snapshot initialization. Okay, what you can do from the snapshots is create a volume, and the volume could be assigned to any specific AC that you want. One EU Central, one B, and one C. And you can also go into a bigger EBS volume while doing so as well as encrypt the volume as part of the operation. Okay? That is quite handy. And finally, lifecycle manager So Lifecycle Manager has lifecycle policies, and these policies allow you to automate the creation, retention, copying, and deletion of EBS snapshots. So for example, we’re saying, “Hey, there’s an EBS Snapshot policy.” We’ll select volumes, and I’ll just call it the demo policy. Then you can have a tag. So it will be applied to any of the following tags: So you can say, for example, “Environment” as your tag, okay? And you need to actually tag your volumes first.
So let’s go take this volume right here, and I will do Environment Production and click on Save. So now if I go back to my Lifecycle Manager, I can create a lifecycle policy, okay? So I’ll say “democracy policy” here. And I can use my environment tag for value production. So this is a policy for these tags, and then we can specify an iRole to do all these actions of backing up my EBS snapshots and my EBS volumes. So they will use a default role, which will be created automatically along with the schedule name. For example, if you want to run a marathon, you should run a marathon every day. Retention type counts, and we can say, okay, you can retain, for example, the last ten snapshots, okay? But you can also enter an age, and you can say how many days you want to retain each snapshot. So it’s up to you to provide some tagging information. If you want to add some tags generated by this data cycle policy, you can do so, but you can also copy the tags from the source if you want to keep the same tags, which is very useful, and then enable fast snapshot restore.
So honestly, I would not enable it because this could be really, really expensive, especially if you start creating a lot of snapshots. But just so you know, the option is available in case you want cross-region copies. So you want to copy these snapshots automatically to additional regions, which is really nice. So you can copy this to any specific region within AWS automatically, which is good for disaster recovery. You can create different policy schedules if you want to do cross account sharing optionally so that you can automate it. So two, three, four, et cetera, and whether or not you want to enable or disable the policy. So I will just disable it, okay, and create this policy just to show you it’s being created. I mean, it will not be active, but at least it will be created. So the policy has been created. As you can see, it is active; it is disabled, but at least we have configured it currently and we can see all the schedules, all the sharing, and so on. Okay, so that’s it for this section on EBS snapshots. I hope you liked it, and I will see you in the next lecture.
9. EBS Operation: Volume Migration
EBS migration is something that system administrators must be aware of. So, as I said, EBS volumes are only locked to a specific AZ. And so if you wanted to migrate it to a different AZ or a different region, you needed to snapshot the volume, which you just did. Then we need to optionally copy the volume to a different region, and then we could create a volume from the snapshot in the AZ of our choice. It’s very simple, but it’s good to see it once. So let’s practice.
So, as a reminder, if I go to my volumes, both of my volumes are in EU West One B. But say there was an instance in EU OneA that I wanted to attach this volume to. What I wanted to do is go to snapshots, right-click, and create a volume. Okay, I’ll say it’s a 5 GB GP 2-volume drive. Great. And now the AZ I want to put it in is EU West One A? But I can choose whatever I want. So now I’ll just say “create volume.” And here we go. Now we go back to our volumes, and as we can see, we’ll get a new available volume right away. Here 5GB have been restored from a snapshot, as we can see. So it’s a snapshot, and it just points from my snapshot, and it’s available in the UAE, West One A. And so that’s perfect. We have very quickly, using snapshot, migrated a volume from EU West One B to EU West One. To be honest, it’s a very simple thing to do, but it must be seen as a system up once. So I hope you enjoyed it, and I will see you in the next lecture.
10. [SAA] EBS Operation: Volume Encryption
Finally, let’s talk about the last operation, which is: how do you encrypt an EBS volume? So, when you create an encrypted EVs volume, you get the following: You get data at rest, encrypted inside your volume. All the data in flight between the instance and the volume is encrypted. All the snapshots will be encrypted. And all the volumes created from the snapshots are encrypted. So there’s encryption all around the place.
And the old encryption and decryption mechanisms are handled transparently for you. So you have nothing to do. It’s all handled by EC2 and EBS behind the scenes. So, encryption overall is something you should use because it has a very, very minimal impact on latency—almost nothing. And it leverages keys from KMS. So AES 256 is something that you should know. And so when you copy an unencrypted snapshot, you enable encryption. So let’s talk about a very important thing: how do you encrypt unencrypted EBS volumes? So, to encrypt an unencrypted EBS volume, which is a very tough thing to say, you create an EBS snapshot of the volume. Then you encrypt the EBS snapshot using the copy function. Then we create a new EBS volume from the snapshot.
And that volume will also be encrypted. And now we can attach the encrypted volume to the original instance. So let’s go take a look at how we do this in the console. So let’s practise EBS encryption. And for this, I’m going to create a volume that is not EBS encrypted. As a result, I’ll select one gigabyte of abilities for us east. Two A. And I will choose not to encrypt this volume. So I will create my volume. And my volume has now been created. So we want to go through the operation of encrypting this volume. And the way to do so is by creating a snapshot. So we take the volume and create a snapshot. And as you can see, the snapshot of an encrypted volume is of an unencrypted volume that is also not encrypted. So I will create my snapshot right here. And this will create a snapshot that is also unencrypted. So, what we can do now is go under snapshots, and we can find the snapshot right here. And as you can see, it is not encrypted, but I can do action copy. And we can copy and encrypt this snapshot as we copy it. So we can copy it and choose to encrypt it. So this is one way of doing it. so I will copy it. And now there is a news snapshot right here being created. And this one is encrypted using KMS. And from the snapshots that are being encrypted, I can create a volume.
And this volume will be encrypted because it is created from an encrypted snapshot. So, in this case, yes, we’re good to go. Let’s create this volume. And now if we go into our EBS volumes, we have one that was created from the snapshot. And if we look at the encryption, yes, it is encrypted. With Kms, there is also a little shortcut. You can even take an unencrypted snapshot. So let’s have a look at our snapshots right here. Let’s take this one, which is the unencrypted snapshot. and you can create a volume from it. And if you wanted to create an encrypted volume from it, you could just take the box here, encrypt this volume, and choose your KMS key. And this will function just as well. So I wanted to show you both options, including the long route and the short route. But that’s it for this lecture. I hope you liked it. And to be done, just make sure to delete your snapshots and delete your EBS volumes. I will see you at the next lecture.
11. [SAA/DVA] EFS Overview
Okay, so now let’s talk about EFS. EFS is a service you need to know at a high level going into the exam. But it is very interesting from an architectural standpoint and has a really amazing set of features. So what is EFS? It stands for Elastic File System, and it is a managed NFS, or network file system, that can be mounted on many different instances across many different availability zones. As I previously stated, it works with many multi-AZ, which is a significant difference between EFS and EBS.
EBS was locked into a single availability zone, whereas EFS is going to be mountable across multiple availability zones. And as such, it’s highly available, scalable, but also extremely expensive. It’s about three times the cost of a GP 2-drive. But you only pay for what you use. So if you don’t store that much data, it makes sense to use EFS rather than EBS based on how well you manage your data sets and the size of your EFS drive. So here’s your EFS, and this is a network file system, and you attach a security group to it to manage incoming connections. And so you have different EC2 instances across multiple AZs. So USD one A, USD one B, and USD one C will all be mounting the same NFS and the same EFS onto their file systems, and they will all access the same ill all accost EBS was something that was linked to one instance at a time. And so the data was not shared between multiple E2 instances.
But in this case with the EFS, it’s a network file system. And as such, all the EC2 instances have access to the same files on your EFS drive. So use cases for this. Well, content management, web serving, data sharing, or a WordPress website. You should now be aware that it employs the industry-standard NFSv 4.1 protocol. So it is a standard way to mount a network drive. And to access the EFS file system, you need to use security groups. So this is network security. EFS is only going to work for Linux-based AMI, not Windows. So this is something that’s extremely important. Windows instances cannot mount an EFS on their file system. To encrypt the EFS, you can use KMS keys at rest. And so again, as I said, EFS is going to be used only for POSIX file systems. So basically, Linux has a standard file API, and the file system will scale automatically. It’s pay per use; there’s no capacity planning. As a result, it is a very simple offering to use. So what do we need to know about EFS as well as its options?
So first of all, the scale of EFS supports thousands of concurrent NFS clients and throughput that can go up to 10 GB plus per second. So it’s really, really high performance, and the fastest team itself can grow two petabytes in scale. And so it can be huge, and you don’t need to manage it at full capacity. This is done automatically for you. Now, you need to know a few options going into the exam. So the first one is around the performance mode. And we have two kinds of performance modes. If you’re running a web server or an application that has latency-sensitive files, then you want to use general purpose. So if you run, for example, WordPress, you would use an EFS file system for general purposes. And this is going to be great because there are going to be a lot of small files, and you can access them very quickly. So EFS is built for that. But if you’re trying to do a big data workload on EFS, then you may want to enable max IO. So the latency is going to be higher, but the throughput is going to be better as well. And this is going to be even more parallel. So this is great for big data or media processing.
Next is the throughput mode. So by default, we are in burst throughput mode. That means you’ll get 50 megabytes per second for 1 storage, plus a burst of up to 100 megabytes per second. And if you want to have a higher throughput with a smaller EFS file system because the throughput grows with the file system size, usually you can go into provision throughput mode, which allows you to set your throughput regardless of the storage size. For example, you can request one gigabyte per second of throughput even if you have only one storage device. So this could also be something that came up in the exam.
If you have a very small file system but you need a very high throughput, then you need to move into provision throughput mode for EFS. And finally, one last thing that comes up in the exam is the storage tiers. So this is a lifecycle management feature for your files. to move a file to a new tier after maybe 30 days. So the standard is for frequently accessing files, and then there is a lower cost tier called “infrequent access.” So efsia is available, and there is going to be a lower price to store these files. Okay? But anytime you need to retrieve these files, there’s going to be a cost associated with it. So that’s it for EFS. I hope you like this lecture, and I will see you in the next lecture.
