infra-provisioner/scripts/add-resource-and-update-tfstate

NOTE: Teraform init and workspace selection is taken care of

Stages:

1. preTerraformSetup()
   1. This function is used to execute custom scripts before terraform operations(if required)
   2. User can add custom scripts in user-scripts-before-terraform-setup.txt
   3. Function can be moved as per convenience and is optional.
   4. migration_required_for.json is where the output goes in this stage in the format {"a": true, "b": true} where a and b are the resource for which migration is required
2. Manifests are fetched from deployment portal using xAuthToken and portal url
3. postTerraformSetup()
   1. This function is used to execute custom scripts after terraform operations(if required)
   2. User can add custom scripts in user-scripts-after-terraform-setup.txt
   3. User can specify actions such as terraform import.

Caution:

1. Currently tfStateUpdate() is written to work on database related operations in a more optimised manner. Modify and update the code as per requirement
2. User can search for "# NOTE" in the code and find all the place where logic can be changed as per resource type and type of operation
3. It is user's responsibility to update shell commands in user-scripts-before-terraform-setup.txt and user-scripts-after-terraform-setup.txt and provide the list of resources as specified in 1.4 format in migration_required_for.json file.

Usage:

python driver.py -h
Usage: python3 driver.py <vertical> <xAuthToken> <operateOnEnv> <resourceType>
vertical(optional): default(lending), navi-pay, sa, gi, navi-ppl
resourceType: database, elasticCache, docdb, aws_access, s3_buckets, dynamodb, auroradb
operateOnEnv: all, dev, prod, perf

$ python driver.py kjf-adfkjf-12e dev dynamodb
 - vertical -> lending
 - kjf-adfkjf-12e -> aAuthToken
 - dev -> environment for which the script will run on
 - dynamodb -> resource which the script will get and setup terraform for
$ python driver.py navi-pay kjf-adfkjf-12e all database
 - the script will run for all the databases(rds-postgres) in all the environments for navi-pay vertical

Example use case:

preTerraformSetup()/user-scripts-before-terraform-setup.txt

• was used to create cloudwatch alarms for the required RDS instances and update list of DBs to json file getListofAWSDBsRequiredForMigration() read the list of DBs for which migration is required from a json file

driver.py

• gets list of manifests from deployment-portal-backend and iterates over them as per resource type and environment.
• for each manifest, the environment variables are updated in the required keys. Ex: ${DATASOURCE_USERNAME} is updated with the username required to access DB
• some of the variables like TEAM_NAME, DB_NAME, DB_REPLICA are set in env.json file and used in user-scripts-after-terraform-setup.txt
• tfStateUpdate() is used to execute terraform init and terraform import for the required RDS instances

postTerraformSetup()/user-scripts-after-terraform-setup

• was used to execute terraform import for the required RDS instances