[github actions] shell parameter expansion: replace character ( ${parameter/%pattern/string} )

[kwladyka]

Select Topic Area

Question

Body

I am trying to use ${parameter/%pattern/string}

In google cloud simple ${TAG_NAME//./-} do the job.

But I can't make it work in github actions.

How to 0.0.1 to 0-0-1 without using bash commands, so with shell parameter expansion or github expressions?

https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html
https://docs.github.com/en/actions/learn-github-actions/expressions

[jge162]

@kwladyka You can use the sed command to replace a pattern with a string in shell parameter expansion.

TAG_NAME="0.0.1"
TAG_NAME="$(echo $TAG_NAME | sed 's/./-/g')"
echo $TAG_NAME # Output: 0-0-1

TAG_NAME="0.0.1"
TAG_NAME="$(echo $TAG_NAME | tr . -)"
echo $TAG_NAME # Output: 0-0-1

[markjreed]

You should actually run things before stating what their output is. Your sed 's/./-/g' will replace every character in the string with a minus sign, because . in a regular expression matches any character.

$ TAG_NAME="0.0.1"
$ TAG_NAME="$(echo $TAG_NAME | sed 's/./-/g')"
$ echo $TAG_NAME
-----

You could fix that by changing it to 's/\./-/g', but do you really need to call out to sed or tr here? If you can run shell commands, you should be able to use the special expansions like ${TAG_NAME//./-}. Unless GHA is using a shell that doesn't have those? Obviously they don't work in places where the GHA parser is the one doing the substitution, i.e. ${{...}}, but in the context of shell commands...

[kwladyka]

Thank you @jge162 .

Still would like to find a way to set it up on the beginning of the file in env as a right value. At least this is how it was in cloud build. Unless it is not possible in github actions, but I was assuming this is just hard to find how to do it.

[airtower-luna]

There is only very limited support for string formatting in the Actions syntax itself, see Expressions. For anything more complex you'll need to use steps and create outputs (or environment variables) to set the values you want (e.g. using shell syntax).

[kwladyka]

Is a way to do something like below? BUT with tr and setting env as global for all jobs to inject by automate?

jobs:
  set-envs:
    runs-on: ubuntu-latest
    steps:
      - name: set values
        run: |
          echo "VERSION=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV
      - name: Use the value
        run: |
          echo "${{ env.VERSION }}"
  other-job:
    needs: [set-envs]
    runs-on: ubuntu-latest
    steps:
      - name: Use the value
        run: |
          echo "${{ env.VERSION }}"

[airtower-luna]

There's no way to dynamically set an environment variable for all jobs. But you could use outputs to transfer the values between jobs (mind that this requires a needs relationship between the jobs):

jobs:
  set-vars:
    runs-on: ubuntu-latest
    outputs:
      version: ${{ steps.gen_version.outputs.VERSION }}
    steps:
      - name: set values
        id: gen_version
        run: |
          echo "VERSION=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT
      - name: Use the value
        # this sets environment variables for this step
        env:
          VERSION: ${{ steps.gen_version.outputs.VERSION }}
        run: |
          echo "${VERSION}"

  other-job:
    needs: [set-vars]
    runs-on: ubuntu-latest
    # this sets environment variables for the whole job
    env:
      VERSION: "${{ needs.set-vars.outputs.version }}"
    steps:
      - name: Use the value
        run: |
          echo "${VERSION}"

A run step runs in the shell (on Ubuntu it's Bash by default), so you have all the usual syntax available.

Note that I've changed the run steps that use the value to use Bash syntax to access the environment variable instead of Actions syntax. This is highly recommended in general to avoid weird bugs or even security problems: Actions syntax in the script is literally replaced before the script runs, so for example a " in the value will be interpreted by the shell. Accessing environment variables with shell syntax avoids those problems.

[Aryayantox73]

Help

[kwladyka]

jobs:
  set-vars:
    runs-on: ubuntu-latest
    outputs:
      version_dashed: ${{ steps.vars.outputs.version_dashed }}
    steps:
      - name: set values
        id: vars
        run: |
          echo "version_dashed=${GITHUB_REF#refs/tags/}" | tr . - >> $GITHUB_OUTPUT

...

  production-deploy:
    needs: [set-vars]
    permissions:
      contents: 'read'
      id-token: 'write'
    runs-on: ubuntu-latest
    env:
      version_dashed: "${{ needs.set-vars.outputs.version_dashed }}"
    steps:
      - name: checkout
        uses: actions/checkout@v3

  ...

      - name: Deploy to Cloud Run
        id: deploy
        run: >
          gcloud run deploy ${{ env.CLOUD_RUN_SERVICE }}
          --image ${{ env.IMAGE }}:${{ env.VERSION }}
          --revision-suffix ${ version_dashed }
          --set-secrets JANICE_KEY=***
          --set-env-vars LOG_LEVEL=info
          --region ${{ env.REGION }}
          --max-instances 1
          --concurrency 100
          --timeout 300s
          --cpu 1
          --memory 1024Mi
          --port 80
          --platform managed
          --allow-unauthenticated

line 1: ${ version_dashed }: bad substitution

Why?

I was trying with version_dashed and version-dashed.

[jge162]

in the command on line 17, you are trying to reference the variable version_dashed without using the needs keyword. Instead you're trying to access it as if it is an environment variable, but it is not defined as such.

You should change the command on line 17 to

gcloud run deploy ${{ env.CLOUD_RUN_SERVICE }}
--image ${{ env.IMAGE }}:${{ env.VERSION }}
--revision-suffix ${{ needs.set-vars.outputs.version_dashed }}
--set-secrets JANICE_KEY=***
--set-env-vars LOG_LEVEL=info
--region ${{ env.REGION }}
--max-instances 1
--concurrency 100
--timeout 300s
--cpu 1
--memory 1024Mi
--port 80
--platform managed
--allow-unauthenticated

[airtower-luna]

That'd work too, but using the environment variable defined in the job-level env (instead of directly substituting the output) is better style.

[jge162]

You got the experience on me. Thank you for that Luna.

[airtower-luna]

The rationale is that with an environment variable and shell syntax the shell is aware of the variable, and escaping works. With Actions syntax it's literal substitution and can get messy.

And thinking about it I just realized that depending on how the workflow gets triggered the GITHUB_REF could be supplied by an attacker. In this case it might actually be a security issue, and @kwladyka, you need to make sure to actually escape the variable substitution: Make sure to use "${version_dashed}" not ${version_dashed}. I could put certain shell control characters (e.g. $()) into a branch name, attackers could get creative with that. 😉

[airtower-luna]

This is invalid shell syntax, because of the spaces between the curly braces and the variable name:

          --revision-suffix ${ version_dashed }

The following should work:

          --revision-suffix ${version_dashed}

Mind that ${{ ... }} is Actions syntax and replaced before the shell ever sees your command. ${VAR} is Unix shell syntax.

[kwladyka]

Thank you. I wish to have a way to set it as global environment like env on top level do. But there I can't use bash command (at least I think so).

[airtower-luna]

Job-level env is as close as you can get to global for something that needs a shell to build. Anything at the global level must work with Actions syntax only.

[Aryayantox73]

Okey