Managing Terraform State: Best Practices

A Terraform state file is Terraform’s mechanism for keeping track of the resources it manages. It serves as a map between your configuration files and the actual resources deployed in your cloud environment.

Terraform relies solely on the state file as its source of truth, ensuring it understands what is in your environment. Without it, Terraform would be unable to decide whether to create, update, or delete resources, resulting in unforeseen outcomes.

Key Functions of the State File:

Resource Mapping: Associates your Terraform configurations with real-world resources.
Change Tracking: Determines what changes need to be applied to achieve the desired state.
Dependency Management: Ensures resources are created, updated, or destroyed in the correct order.
Performance Optimization: Speeds up operations by caching resource information.

Managing Terraform State

Managing the state file is just as important as defining your infrastructure.

Best practices:

Use Remote Backends: To improve cooperation and prevent loss, store state data in a safe, shared place such as S3.
Enable Locking: Using DynamoDB or comparable technologies, enable state locking to avoid simultaneous updates.
Secure State Files: Encrypt important information stored in state files.
Never Modify Manually: To avoid inconsistencies, let Terraform manage the state file.

Why you shouldn't change the state file manually:
- Inconsistencies: Terraform may no longer correctly detect resources, resulting in unintentional modifications or destruction on subsequent runs.
- Errors: Syntax or structural errors in the JSON-formatted state file might disrupt Terraform's ability to plan or implement modifications.
- Lost Changes: Manual modifications may be overwritten by subsequent Terraform runs.

Configuring Remote State Storage

It becomes dangerous to store state locally when teams work together. AWS S3 and other remote backends offer a safe, common location for state data, guaranteeing uniformity and minimizing conflicts.

These are some hard rules:

Never Declare Remote Backend in Terraform Code
Backend configurations should only exist in a backend block in the terraform configuration section or via CLI flags during initialization. Avoid managing the backend resources (such as the DynamoDB table for state locking or the S3 bucket) in the same Terraform project that requires them.
Separate Infrastructure for State Management
The S3 bucket and DynamoDB table can be provisioned manually or with a different Terraform configuration. By doing this, the backend infrastructure is protected against unintentional destruction.

Steps to Set Up Remote State with AWS S3:

Create an S3 Bucket

Use a separate process or Terraform configuration to create the bucket and table. Here's an example of manual CLI commands:

 aws s3api create-bucket --bucket my-terraform-state-bucket --region us-east-1  
 aws s3api put-bucket-versioning --bucket my-terraform-state-bucket --versioning-configuration Status=Enabled 

 aws dynamodb create-table \
    --table-name terraform-state-lock \
    --attribute-definitions AttributeName=LockID,AttributeType=S \
    --key-schema AttributeName=LockID,KeyType=HASH \
    --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1

Configure the Backend in Your Terraform Project

 terraform {
   backend "s3" {
     bucket         = "my-terraform-state-bucket"
     key            = "terraform/state.tfstate"
     region         = "us-east-1"
     dynamodb_table = "terraform-state-lock"
   }
 }

Migrate State to Remote Backend

Use the command terraform init to configure the backend and migrate the local state to the S3 bucket. Terraform will prompt you to confirm the migration.
```
 terraform init
```
Key Takeaways
1. Terraform State is Crucial:
  - It maintains the link between your configurations and the actual infrastructure.
  - Proper management ensures consistency, reliability, and scalability.
2. Remote State Storage Enhances Collaboration:
  - Centralized state files allow multiple team members to work seamlessly.
  - Secure and versioned storage prevents data loss and unauthorized access.
3. Adhere to Best Practices:
  - Avoid manual state file edits.
  - Implement security measures and access controls.
  - Separate backend management to protect state integrity.

Managing Terraform State: Best Practices

Managing Terraform State

Configuring Remote State Storage

Steps to Set Up Remote State with AWS S3:

Key Takeaways