docs: add week 5 and 6 update docs and update README
This commit is contained in:
@@ -12,7 +12,7 @@ Built in public, one week at a time.
|
|||||||
- Containerization with Docker
|
- Containerization with Docker
|
||||||
- CI/CD automation with GitHub Actions, Amazon ECR, and Docker Hub
|
- CI/CD automation with GitHub Actions, Amazon ECR, and Docker Hub
|
||||||
- Infrastructure as Code with Terraform
|
- Infrastructure as Code with Terraform
|
||||||
- Cloud deployment on AWS
|
- Cloud deployment on AWS EC2
|
||||||
- Container orchestration with Kubernetes
|
- Container orchestration with Kubernetes
|
||||||
- Monitoring and observability with Prometheus and Grafana
|
- Monitoring and observability with Prometheus and Grafana
|
||||||
|
|
||||||
@@ -41,8 +41,8 @@ Built in public, one week at a time.
|
|||||||
| 2 | Dockerize the backend | [Done](docs/week-2-update.md) |
|
| 2 | Dockerize the backend | [Done](docs/week-2-update.md) |
|
||||||
| 3 | React frontend and Docker Compose | [Done](docs/week-3-update.md) |
|
| 3 | React frontend and Docker Compose | [Done](docs/week-3-update.md) |
|
||||||
| 4 | CI/CD pipeline with GitHub Actions, ECR and Docker Hub | [Done](docs/week-4-update.md) |
|
| 4 | CI/CD pipeline with GitHub Actions, ECR and Docker Hub | [Done](docs/week-4-update.md) |
|
||||||
| 5 | Terraform infrastructure setup | Upcoming |
|
| 5 | Terraform infrastructure and CI/CD pipeline extension | [Done](docs/week-5-update.md) |
|
||||||
| 6 | Cloud deployment on AWS EC2 | Upcoming |
|
| 6 | Cloud deployment on AWS EC2 | [Done](docs/week-6-update.md) |
|
||||||
| 7 | Database integration | Upcoming |
|
| 7 | Database integration | Upcoming |
|
||||||
| 8 | Kubernetes deployment | Upcoming |
|
| 8 | Kubernetes deployment | Upcoming |
|
||||||
| 9 | Monitoring with Prometheus and Grafana | Upcoming |
|
| 9 | Monitoring with Prometheus and Grafana | Upcoming |
|
||||||
@@ -97,15 +97,34 @@ Frontend runs on http://localhost:80
|
|||||||
|
|
||||||
## CI/CD Pipeline
|
## CI/CD Pipeline
|
||||||
|
|
||||||
Every pull request and push to `main` triggers a four-stage pipeline:
|
Every pull request and push to `main` triggers a five-stage pipeline:
|
||||||
|
|
||||||
```
|
```
|
||||||
lint -> unit-test -> build-and-integration-test -> publish
|
lint -> unit-test -> build-and-integration-test -> publish -> terraform-apply
|
||||||
```
|
```
|
||||||
|
|
||||||
The publish stage runs on merge to `main` only. Images are pushed to both Amazon ECR and Docker Hub, tagged with `latest` and the short Git SHA for traceability.
|
- `terraform-plan` runs on every pull request — shows exactly what infrastructure will change before merging
|
||||||
|
- `terraform-apply` runs on merge to `main` — infrastructure is provisioned by the pipeline, never from a local machine
|
||||||
|
- The `publish` stage pushes images to both Amazon ECR and Docker Hub, tagged with `latest` and the short Git SHA
|
||||||
|
|
||||||
AWS authentication uses OIDC - no static credentials stored in GitHub secrets.
|
AWS authentication uses OIDC — no static credentials stored in GitHub secrets.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Infrastructure
|
||||||
|
|
||||||
|
All infrastructure is provisioned with Terraform and lives in `infra/terraform/`.
|
||||||
|
|
||||||
|
| Resource | Details |
|
||||||
|
|---|---|
|
||||||
|
| VPC | Custom VPC with public subnet |
|
||||||
|
| Networking | Internet gateway, route tables |
|
||||||
|
| Security | Security groups with least-privilege rules |
|
||||||
|
| Compute | EC2 t3.micro running Amazon Linux 2023 |
|
||||||
|
| IAM | Instance profile with SSM and ECR permissions |
|
||||||
|
| State | S3 remote backend with native state locking |
|
||||||
|
|
||||||
|
EC2 instances configure themselves on first boot via a user data script — Docker is installed, images are pulled from ECR, and containers start automatically. No SSH required. All access via AWS SSM Session Manager.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,119 @@
|
|||||||
|
# Week 5 — Terraform Infrastructure and CI/CD Pipeline Extension
|
||||||
|
|
||||||
|
## What I Built
|
||||||
|
|
||||||
|
Week 5 added two major pieces: full AWS infrastructure provisioned with Terraform, and an extended CI/CD pipeline that owns infrastructure changes end to end.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Terraform Bootstrap
|
||||||
|
|
||||||
|
Before the main infrastructure, I created a bootstrap layer in `infra/terraform/bootstrap/` to set up remote state storage.
|
||||||
|
|
||||||
|
- S3 bucket with versioning, AES256 encryption, and public access blocked
|
||||||
|
- Native state locking using `use_lockfile = true` — available from Terraform 1.10 onwards
|
||||||
|
- No DynamoDB table required — Terraform writes a `.tflock` file directly to S3
|
||||||
|
|
||||||
|
This is the modern approach. One less AWS resource to manage, same production-grade protection.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Main Infrastructure
|
||||||
|
|
||||||
|
All resources live in `infra/terraform/` and are split across focused files:
|
||||||
|
|
||||||
|
**`vpc.tf`**
|
||||||
|
- Custom VPC (`10.0.0.0/16`)
|
||||||
|
- Public subnet (`10.0.1.0/24`) in `us-east-1a`
|
||||||
|
- Internet gateway attached to the VPC
|
||||||
|
- Route table with a default route to the internet gateway
|
||||||
|
- Route table association to the public subnet
|
||||||
|
|
||||||
|
**`security-group.tf`**
|
||||||
|
- Security group allowing inbound HTTP (port 80) from anywhere
|
||||||
|
- All outbound traffic allowed
|
||||||
|
- Port 22 (SSH) intentionally closed — access via SSM only
|
||||||
|
|
||||||
|
**`iam.tf`**
|
||||||
|
- IAM role for EC2 with two managed policies attached:
|
||||||
|
- `AmazonSSMManagedInstanceCore` — enables SSM Session Manager access
|
||||||
|
- `AmazonEC2ContainerRegistryReadOnly` — allows pulling images from ECR
|
||||||
|
- Instance profile wrapping the role
|
||||||
|
|
||||||
|
**`ec2.tf`**
|
||||||
|
- Data source querying AWS at plan time for the latest Amazon Linux 2023 AMI — no hardcoded AMI ID
|
||||||
|
- EC2 `t3.micro` instance in the public subnet
|
||||||
|
- 20GB encrypted `gp3` root volume
|
||||||
|
- User data script that runs on first boot:
|
||||||
|
- Installs Docker and starts the service
|
||||||
|
- Authenticates to ECR
|
||||||
|
- Pulls backend and frontend images
|
||||||
|
- Starts both containers with `--restart unless-stopped`
|
||||||
|
|
||||||
|
**`outputs.tf`**
|
||||||
|
- Public IP of the EC2 instance — printed after every apply
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## CI/CD Pipeline Extension
|
||||||
|
|
||||||
|
Added two new jobs to `.github/workflows/ci-cd.yml`:
|
||||||
|
|
||||||
|
**`terraform-plan`**
|
||||||
|
- Triggers on every pull request
|
||||||
|
- Runs `terraform init` and `terraform plan`
|
||||||
|
- Shows exactly what will change before any merge
|
||||||
|
- Depends on `build-and-integration-test` passing first
|
||||||
|
|
||||||
|
**`terraform-apply`**
|
||||||
|
- Triggers on merge to `main` only
|
||||||
|
- Runs `terraform init` and `terraform apply -auto-approve`
|
||||||
|
- Infrastructure is created by the pipeline — never from a local machine
|
||||||
|
|
||||||
|
Both jobs authenticate to AWS using the existing OIDC role. No static credentials.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Production Discipline
|
||||||
|
|
||||||
|
The local machine provisions once to validate everything works. Then `terraform destroy` hands control to the pipeline permanently. From that point forward, infrastructure changes only happen through Git — a PR shows the plan, a merge triggers the apply.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture After Week 5
|
||||||
|
|
||||||
|
```
|
||||||
|
GitHub
|
||||||
|
│
|
||||||
|
├── Pull Request → terraform-plan (shows what will change)
|
||||||
|
│
|
||||||
|
└── Merge to main → lint → unit-test → build → publish → terraform-apply
|
||||||
|
│
|
||||||
|
AWS Infrastructure
|
||||||
|
VPC / Subnet / SG
|
||||||
|
IAM / EC2
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Files Added
|
||||||
|
|
||||||
|
```
|
||||||
|
infra/terraform/
|
||||||
|
├── vpc.tf
|
||||||
|
├── security-group.tf
|
||||||
|
├── iam.tf
|
||||||
|
├── ec2.tf
|
||||||
|
└── outputs.tf
|
||||||
|
|
||||||
|
.github/workflows/ci-cd.yml (terraform-plan and terraform-apply jobs added)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Screenshots
|
||||||
|
|
||||||
|
See `docs/week-05/` for:
|
||||||
|
- Terraform plan output
|
||||||
|
- Terraform apply output showing 13 resources created
|
||||||
|
- GitHub Actions pipeline with all jobs passing
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
# Week 6 — Cloud Deployment on AWS EC2
|
||||||
|
|
||||||
|
## What I Built
|
||||||
|
|
||||||
|
Week 6 took everything built in Week 5 and made it live. TaskFlow — the full-stack Node.js and React application — is now running on AWS EC2, deployed automatically by the CI/CD pipeline with zero manual server setup.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## How Deployment Works
|
||||||
|
|
||||||
|
When a PR merges to `main`, the pipeline:
|
||||||
|
|
||||||
|
1. Runs lint, unit tests, and integration tests
|
||||||
|
2. Builds Docker images and pushes them to Amazon ECR
|
||||||
|
3. Runs `terraform apply` — creates or updates infrastructure
|
||||||
|
4. The EC2 instance boots and runs a user data script automatically
|
||||||
|
|
||||||
|
The user data script handles everything on first boot:
|
||||||
|
|
||||||
|
```
|
||||||
|
1. Update the system packages
|
||||||
|
2. Install Docker and start the service
|
||||||
|
3. Wait for Docker to be fully ready
|
||||||
|
4. Add ec2-user to the docker group
|
||||||
|
5. Install AWS CLI
|
||||||
|
6. Authenticate to ECR using the IAM instance profile
|
||||||
|
7. Pull task-backend:latest from ECR
|
||||||
|
8. Pull task-frontend:latest from ECR
|
||||||
|
9. Start backend container on port 3000 (restart unless stopped)
|
||||||
|
10. Wait for backend to be ready
|
||||||
|
11. Start frontend container on port 80 (restart unless stopped)
|
||||||
|
```
|
||||||
|
|
||||||
|
No SSH. No manual commands. The instance configures itself.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Access Method
|
||||||
|
|
||||||
|
All server access is through **AWS SSM Session Manager** — not SSH.
|
||||||
|
|
||||||
|
- Port 22 is closed on the security group
|
||||||
|
- The EC2 IAM role has `AmazonSSMManagedInstanceCore` attached
|
||||||
|
- Sessions are opened via the AWS console or AWS CLI
|
||||||
|
|
||||||
|
This is the production-standard approach. No key pairs to manage, no inbound SSH exposure, full audit trail of every session in AWS CloudTrail.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What Runs on EC2
|
||||||
|
|
||||||
|
| Container | Image Source | Port | Restart Policy |
|
||||||
|
|---|---|---|---|
|
||||||
|
| task-backend | Amazon ECR | 3000 (internal) | unless-stopped |
|
||||||
|
| task-frontend | Amazon ECR | 80 (public) | unless-stopped |
|
||||||
|
|
||||||
|
The frontend container runs Nginx, which serves the React build and proxies `/api` requests to the backend container.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture After Week 6
|
||||||
|
|
||||||
|
```
|
||||||
|
Internet
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
EC2 Public IP :80
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
task-frontend (Nginx)
|
||||||
|
│
|
||||||
|
├── Serves React frontend (/)
|
||||||
|
│
|
||||||
|
└── Proxies /api requests → task-backend :3000
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Express API
|
||||||
|
(in-memory store)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Key Decisions
|
||||||
|
|
||||||
|
**Why no Docker Compose on EC2?**
|
||||||
|
The containers are run directly with `docker run` in the user data script. Docker Compose adds a dependency that needs to be installed separately on Amazon Linux 2023. Direct `docker run` with `--link` keeps the bootstrap simple and reliable.
|
||||||
|
|
||||||
|
**Why `--restart unless-stopped`?**
|
||||||
|
If the EC2 instance reboots, Docker restarts the containers automatically. The application comes back online without any intervention.
|
||||||
|
|
||||||
|
**Why ECR and not Docker Hub?**
|
||||||
|
ECR is private, in the same AWS region as the EC2 instance, and authenticated via IAM — no credentials to manage. Pulls are fast and free within the same region.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Files Changed
|
||||||
|
|
||||||
|
```
|
||||||
|
infra/terraform/ec2.tf (user data script updated — sleep added for Docker readiness)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Screenshots
|
||||||
|
|
||||||
|
See `docs/week-06/` for:
|
||||||
|
- TaskFlow running live in the browser at the EC2 public IP
|
||||||
|
- SSM Session Manager connection to the EC2 instance
|
||||||
|
- Docker containers running on the instance (`docker ps`)
|
||||||
|
- GitHub Actions pipeline showing full run with terraform-apply
|
||||||
Reference in New Issue
Block a user