Ashvin S 1009012a4d INFRA-3988 | Ashvin | Add autoscaling to Flink (#1251)
* INFRA-3988 | Ashvin | Add autoscaling to Flink

Autoscaling will only be supported by Flink versions 1.18 and later.
Migration is done using the following command
```shell
find . -name "manifest*flink*" | xargs -I{} sh -c "jq -r '.flink.flinkDeployment.flinkConfiguration += {\"autoscaler\": { \"enabled\": false }}' {} | sponge {}"
```
Adds validation in JSON schema
Updates kutegen with autoscaler changes

* INFRA-3988 | Ashvin | Remove promgateway configuration from Flink

This field is not required now that we use servicemonitor to fetch the
metrics. Earlier pushgateway was used.

Command used to do the migration is
```shell
find . -name "kub*flink*" | xargs -I {} sh -c "sed -i 's/promgate//' {}"
```

* INFRA-3988 | Ashvin | Add max-parallelism in test fixtures

Command used
```shell
find . -name "kube*flink*json" | xargs -I{} sh -c "jq -r '. | .kubeObject.items |= map(if .kind == \"FlinkDeployment\" then .spec.flinkConfiguration += { \"pipeline.max-parallelism\": \"100\" } else . end)' {} | sponge {}"
```

* Revert "INFRA-3988 | Ashvin | Remove promgateway configuration from Flink"

This reverts commit 3ef63ad31a587814275d5bad6f6d467cf638a0b9.

* INFRA-3988 | Ashvin | Remove promgateway configuration from Flink

This field is not required now that we use servicemonitor to fetch the
metrics. Earlier pushgateway was used.

Command used to do the migration is
```shell
find . -name "*flink*json" | xargs -I {} sh -c "sed -i '/promgate/d' {}"
```

* INFRA-3988 | Add Flink autoscaler field to all manifests

Only Flink manifests will be effected by this migration

* INFRA-3988 | Ashvin | Update kutegen
2024-11-05 18:57:37 +05:30

Deployment Portal Backend

  • Run git submodule update --init --recursive to pull the kutegen submodule.
  • Use export ENVIRONMENT=test to avoid applying changes to the cluster.
  • Docker Setup
    • To run the application just do docker-compose up
    • It is assumed that deployment-portal-frontend repository is present in parallel to this folder.

Initial Setup

  • Create your user in the database
INSERT INTO users (created_at, updated_at, email, name)
VALUES (now(), now(), 'ashvin.s@navi.com', 'Ashvin S');

INSERT INTO users_roles(user_id, role_id)
    (SELECT u.id, r.id
     from users u,
          role r
     where u.email like 'ashvin.s@navi.com'
       AND r.name IN ('PORTAL_MANAGER', 'Infra_ALL_MANAGER'));

INSERT INTO users_teams(user_id, team_id)
        (SELECT u.id, t.id
         from users u,
              team t
         where u.email like 'ashvin.s@navi.com'
           AND t.name IN ('Infra'));
  • This creates token for GOCD user
-- insert in tokens table id, created_at, updated_at, token, user_id
-- token should be bcrypt of a string
-- user_id should be user_id for gocd@ci.com
-- token is token
INSERT INTO tokens (created_at, updated_at, token, user_id)
VALUES (now(), now(), '$2a$12$LWmbUqPYFPqessz7z9OpvO6k0g/QEp6mBQxXMtIGkqPu4ih9exmbC', 1);
curl -X POST "http://127.0.0.1:8080/api/user/mapping" \
    -H "X_AUTH_TOKEN: token" \
    -H "Content-Type: multipart/form-data; boundary=boundaryValue" \
    -F "file=@<path-to-user-mapping-file>;filename=user-mapping.yml;type=application/x-yaml"
Description
No description provided
Readme 5.8 MiB
Languages
Java 94%
Python 5.6%
Dockerfile 0.2%
Shell 0.2%