AWS devops. How to deploy Elasticsearch cluster at AWS using Opensearch, ECS, EC2 with terrafrom and ansible
The course consists of 5 main modules.
What you’ll learn
- How to deploy HA Elasticsearch cluster using AWS Opensearch.
- How to deploy HA Elasticsearch cluster using AWS ECS.
- How to deploy HA Elasticsearch cluster using AWS EC2.
- Elasticsearch autoscaling.
- Elasticsearch cluster conceptions and performance.
- AWS IAM, VPC, ECS, Opensearch, security groups.
- Docker and Docker swarm.
Course Content
- Introduction –> 2 lectures • 10min.
- Initial terraform configuration aspects –> 2 lectures • 11min.
- AWS skeleton configuration –> 5 lectures • 27min.
- Deploying an Elasticsearch cluster using AWS Openseach –> 8 lectures • 42min.
- Deploying an ES cluster using AWS ECS –> 7 lectures • 50min.
- Deploying an ES cluster using AWS EC2 and docker swarm –> 6 lectures • 30min.
- Autoscaling and Elasticsearch –> 1 lecture • 12min.
- Course summary – which deployment scenarios is better? –> 1 lecture • 6min.
- Elasticsearch theory –> 3 lectures • 25min.
Requirements
The course consists of 5 main modules.
Teh first one is about how to deploy ES as an AWS service, which is called OpenSearch. Here I will tell you about interesting OpenSearch pitfalls that I discovered during my commercial practice. And believe me – you will not find that information at AWS documentation. You will get a ready terraform module for deploying the Opensearch cluster and we will analyze it step by step together.
In the 2d part I will show how to deploy an ES cluster at ECS (Elastic Container Service). And again we will use the terraform modules for that. I will show you why you can’t use Fargate in that case and how many problems should be resolved to deploy ES at ECS.
Next course section is about how to deploy ES at EC2 instances using docker swarm. I will prove to you that it is the easiest and at the same time the most cheap way to deploy ES at almost any cloud for most small and middle companies with a steady growth pattern. You will get ready terraform modules that will allow you to prepare AWS infrastructure and ansible modules for deploying HA ES cluster upon it.
The 4th part is devoted to autoscaling possibilities of ES cluster using different deployment approaches. We will talk about what we really can autoscale, when we can use it and what big limitations we have in case of working with ES.
5th module is devoted to the theory. If you are already familiar with ES cluster conceptions and everything that is related to shards and ES performance – then you can skip it. But if not – that is the place where you have to start from.