Integrating WSO2 IAM-CTL Tool into GitHub CI/CD Pipeline

5 min readSep 10, 2024

Introduction

Managing configurations for WSO2 Identity Server (WSO2 IS) across multiple environments can become complex as the system scales. The WSO2 IAM-CTL command-line tool simplifies this by allowing administrators to manage WSO2 IS configurations programmatically. Written in Go, IAM-CTL interacts with WSO2 IS through its management REST APIs, enabling bulk imports, exports, and promoting configurations between environments.

This article walks through the process of integrating the IAM-CTL tool into a GitHub CI/CD pipeline, ensuring efficient configuration management across various environments (e.g., development, staging, production).

Prerequisites

WSO2 Identity Server 7.0.0 Setup: Ensure that WSO2 IS 7.0.0 is running and configured.
GitHub Repository: Your repository should be structured to manage configurations for each environment, with separate branches for environments like dev, stage, and prod.
IAM-CTL Tool: Download and configure the latest version of IAM-CTL for your operating system.

IAM-CTL Installation

Download the Latest Release:

Download the binary for your platform from the official releases.
Extract the tar or zip file.

2. Create an Alias: Define an alias for easier usage.

On Linux/macOS:

alias iamctl="<IAM-CTL-PATH>/bin/iamctl"

On Windows:

doskey iamctl=<IAM-CTL-PATH>\bin\iamctl.exe $*

3. Verify Installation: Run the following command to verify the installation:

iamctl -h

WSO2 IS Configuration

Before integrating IAM-CTL into your CI/CD pipeline, you need to configure it to interact with WSO2 IS by registering a Machine-to-Machine (M2M) application.

Start WSO2 IS.
Register an M2M Application with the following scopes:

Application Management API: Create, update, delete, and view applications.
Claim Management API: Create, update, delete, and view claims.
Identity Provider Management API: Create, update, delete, and view identity providers.
Userstore Management API: Create, update, delete, and view user stores.

Take note of the Client ID and Client Secret from this application.

CLI Mode for Managing Environments

To facilitate managing WSO2 IS configurations across different environments (e.g., dev, stage, prod), IAM-CTL provides a CLI mode with exportAll and importAll commands. These allow you to extract or promote configurations easily between environments.

Setup Configuration

Create a GitHub Actions workflow file in .github/workflows/config.yml:

name: Initial Setup
on:
  workflow_dispatch:

jobs:
  deploy:
    runs-on: ubuntu-latest
    env:
      RELEASE_VERSION: v1.0.7
      FULL_FILE_NAME: iamctl-1.0.7-linux-x64.tar.gz
      FILE_NAME: iamctl-1.0.7
    steps:
      - name: Download files from the current repository
        uses: actions/checkout@v2
        with:
          ref: ${{ github.ref }}

      - name: Download and unzip latest identity-tools-cli release
        run: |
          # Download the release file
          curl -L -o "${FULL_FILE_NAME}" "https://github.com/wso2-extensions/identity-tools-cli/releases/download/${RELEASE_VERSION}/${FULL_FILE_NAME}"
          tar -xzf "${FULL_FILE_NAME}"
          
      - name: Extract release file
        run: tar -xzf "${FULL_FILE_NAME}"
  
      - name: Run setup command
        run: |
          cd "${FILE_NAME}/bin"
          ./iamctl setupCLI -d "../../"

      - name: Fetch All Remote Branches
        run: git fetch --all

      - name: Get Branch Names
        id: branch-names
        run: |
          branches=($(git branch -a --format="%(refname:short)" | grep -v "HEAD" | sed 's/origin\///' | sort -u))
          echo "::set-output name=branches::${branches[*]}"
      
      - name: Set Branch Names
        id: set-branch-names
        run: |
          branches=$(git branch -r --format="%(refname:short)" | sed 's/origin\///' | tr '\n' ' ')
          echo "::set-output name=branches::$branches"

      - name: Create Branch Folders and Copy Env Content
        run: |
          for branch in ${{ steps.set-branch-names.outputs.branches }}; do
            branch_folder="configs/$branch"
            mkdir -p "$branch_folder"
            cp -R configs/env/. "$branch_folder"
          done

      - name: Delete Env Folder
        run: |
          rm -rf configs/env
      
      - name: Change to Configs Directory and List Contents
        run: |
          cd configs
          ls

      - name: Delete release files
        run: |
          rm "${{ env.FULL_FILE_NAME }}"
          rm -rf "${{ env.FILE_NAME }}"

      - name: Create pull request
        uses: peter-evans/create-pull-request@v3
        with:
          commit-message: "Exported configurations"
          title: "Exported configurations"
          body: |
            Changes have been exported from the Identity Server.
            Please review and merge if applicable.
          branch: export-configurations
          branch-suffix: timestamp
          delete-branch: true
          base: ${{ github.head_ref }}
          labels: export, automated pr

This command generates a configs directory with environment-specific folders like dev, stage, and prod, containing the following files:
serverConfig.json: WSO2 IS server details.
toolConfig.json: Tool-specific settings.
keywordConfig.json: Optional keyword mapping.

Update the serverConfig.json file with your WSO2 IS instance details:

{   
  "SERVER_URL" : "https://localhost:9443",   
  "CLIENT-ID" : "your-client-id",   
  "CLIENT-SECRET" : "your-client-secret",   
  "TENANT-DOMAIN" : "carbon.super" 
}

Export and Import Configuration

To manage configurations across environments, you will need to maintain branches corresponding to each environment (dev, stage, prod). The following workflows can help automate exporting and importing configurations using GitHub Actions.

Workflow for Exporting Configurations

Create a GitHub Actions workflow file in .github/workflows/export-config.yml:

name: Export Configurations
on:
  workflow_dispatch:

jobs:
  deploy:
    runs-on: ubuntu-latest
    env:
      RELEASE_VERSION: v1.0.7
      FULL_FILE_NAME: iamctl-1.0.7-linux-x64.tar.gz
      FILE_NAME: iamctl-1.0.7
      PROD_CLIENT_ID: ${{ secrets.PROD_CLIENT_ID }}
      PROD_CLIENT_SECRET: ${{ secrets.PROD_CLIENT_SECRET }}
      PROD_SERVER_URL: ${{ secrets.PROD_SERVER_URL }}
      PROD_TENANT_DOMAIN: ${{ secrets.PROD_TENANT_DOMAIN }}
      MY_SECRET: ${{ secrets.MY_SECRET }}
    steps:
      - name: Download files from the current repository
        uses: actions/checkout@v2
        with:
          ref: ${{ github.ref }}
        
      - name: Download and unzip latest identity-tools-cli release
        run: |
          # Download the release file
          curl -L -o "${FULL_FILE_NAME}" "https://github.com/wso2-extensions/identity-tools-cli/releases/download/${RELEASE_VERSION}/${FULL_FILE_NAME}"
          tar -xzf "${FULL_FILE_NAME}"
          
      - name: Extract release file
        run: tar -xzf "${FULL_FILE_NAME}"
      
      - name: Run export command
        run: |
          branch_name=$(basename "${{ github.ref }}")
          cd "${FILE_NAME}/bin"
          ./iamctl exportAll -c "../../configs/$branch_name"
          
      - name: Delete release files
        run: |
          rm "${{ env.FULL_FILE_NAME }}"
          rm -rf "${{ env.FILE_NAME }}"

      - name: Create pull request
        uses: peter-evans/create-pull-request@v3
        with:
          commit-message: "Exported configurations"
          title: "Exported configurations"
          body: |
            Changes have been exported from the Identity Server.
            Please review and merge if applicable.
          branch: export-configurations
          branch-suffix: timestamp
          delete-branch: true
          base: ${{ github.head_ref }}
          labels: export, automated pr

Workflow for Importing Configurations

Similarly, create a workflow for importing configurations from a GitHub repository branch into WSO2 IS.

Create .github/workflows/import-config.yml:

name: Import Configurations
on:
  pull_request:
    types:
      - closed
    branches:
      - prod
  workflow_dispatch:

jobs:
  deploy:
    runs-on: ubuntu-latest
    env:
      RELEASE_VERSION: v1.0.7
      FULL_FILE_NAME: iamctl-1.0.7-linux-x64.tar.gz
      FILE_NAME: iamctl-1.0.7
      PROD_CLIENT_ID: ${{ secrets.PROD_CLIENT_ID }}
      PROD_CLIENT_SECRET: ${{ secrets.PROD_CLIENT_SECRET }}
      PROD_SERVER_URL: ${{ secrets.PROD_SERVER_URL }}
      PROD_TENANT_DOMAIN: ${{ secrets.PROD_TENANT_DOMAIN }}
      MY_SECRET: ${{ secrets.MY_SECRET }}
    steps:
      - name: Download files from the current repository
        uses: actions/checkout@v2
        with:
          ref: ${{ github.ref }}
          
      - name: Download and unzip latest identity-tools-cli release
        run: |
          # Download the release file
          curl -L -o "${FULL_FILE_NAME}" "https://github.com/wso2-extensions/identity-tools-cli/releases/download/${RELEASE_VERSION}/${FULL_FILE_NAME}"
          tar -xzf "${FULL_FILE_NAME}"
          
      - name: Extract release file
        run: tar -xzf "${FULL_FILE_NAME}"
      
      - name: Run import command
        run: |
          branch_name=$(basename "${{ github.ref }}")
          cd "${FILE_NAME}/bin"
          ./iamctl importAll -c "../../configs/$branch_name" -i "../../"

In this setup, the workflow will import the configurations from the respective environment branch (dev, stage, or prod) whenever a change is pushed to that branch.

Conclusion

By integrating WSO2 IAM-CTL into your GitHub CI/CD pipeline, you can automate the management of WSO2 Identity Server configurations across multiple environments. This approach ensures that configuration changes are seamlessly exported and imported as part of your deployment process, reducing manual errors and enhancing consistency across environments.

Repo: https://github.com/HasiniSama/iam-cli-tool-demo