Shared volumes in freestyle composition

   run_integration_tests:
     title: Test pipeline
     stage: test-build
     working_directory: IMAGE_WORK_DIR
     image: '${{build_scheduler}}'
     commands:
       - ls
       - ./integration-test.sh
     services:
       volumes:
         ${{CF_VOLUME}}:
           external: true
       composition:
         postgres:
           image: postgres:9.5
           environment:
             - POSTGRES_DB=airflow
             - POSTGRES_PASSWORD=Password123
             - POSTGRES_USER=postgres
             - TZ=Asia/Singapore
         pipeline_app_1:
           image: '${{build_mock_app}}'
           volumes:
             - ${{CF_VOLUME}}:/app/output
           command: bash -c "pwd && ls && echo 'Finished tests' > test_result.txt"

gives this error:

ERROR: Named volume "pipeline_5d6e1fa1a6c4390f0b463fcc_trigger_5d6e200a2287a2849ea5f4fc:/app/output:rw" is used in service "pipeline_app_1" but no declaration was found in the volumes section.

try:
volumes:
- ${{CF_VOLUME}}:/codefresh/volume/

Cannot start composition with warnings [“Volume mapping is not supported, try use: /app/output”]

Haven’t tried it myself but this page shows using ${{CF_VOUME_NAME}} instead https://codefresh.io/docs/docs/yaml-examples/examples/shared-volumes-of-service-from-composition-step-for-other-yml-steps/

CF_VOLUME_NAME also yields the same error.

I believe you are hitting a similar issue as we did at https://support.codefresh.io/hc/en-us/requests/1356

This was the response from Codefresh:

We have certain volume declarations in compositions that we support and some others that we don’t.

We DO NOT support absolute path mappings:

  • /opt/data:/var/lib/mysql

But we do support relative paths:

  • ./cache:/tmp/cache

Which are relative to the location of the compose file.

This is why the variable referenced used is {{CF_VOLUME_NAME}} in the form of: ${{CF_VOLUME_NAME}}:${{CF_VOLUME_PATH}}'

I wasn’t able to see the linked issue. In any case, i wasn’t able to use relative paths either. Not sure if i did it wrong, but i checked with a Codefresh staff member and engineering is adding the enhancement for a freestyle step to be able to access volumes in service containers.

Our engineering team added automounting on service containers in the setup block.

version: '1.0'
steps:
  main_clone:
    title: Cloning main repository...
    type: git-clone
    repo: 'codefreshdemo/cf-example-unit-test'
    revision: 'master'
    git: github
  hello_world_step:
    title: free style step
    image: node:11
    commands:
      - ls -l /codefresh/volume/
      - cat /codefresh/volume/test.txt
    services:
      composition:
        my_custom_service:
          image: 'alpine:latest'  
      setup:
        image: 'alpine:latest'
        commands:
          - "ls -l /codefresh/volume"
          - "echo 'from service' > /codefresh/volume/test.txt"

You can also mount the volume yourself in the service container itself, but I don’t recommend this. I think the setup block is better

version: '1.0'
steps:
  main_clone:
    title: Cloning main repository...
    type: git-clone
    repo: 'codefreshdemo/cf-example-unit-test'
    revision: 'master'
    git: github
  hello_world_step:
    title: free style step
    image: node:11
    commands:
      - ls -l /codefresh/volume/
      - cat /codefresh/volume/test.txt
    services:
      composition:
        my_custom_service:
          image: 'alpine:latest'
          volumes:
            - ${{CF_VOLUME_NAME}}:/codefresh/volume
          command: 'ls -l /codefresh/volume'
      setup:
        image: 'alpine:latest'
        commands:
          - "echo 'from service' > /codefresh/volume/test.txt"
1 Like

Basically, this portion is always included then, from now on?

Yes, but only in the setup block (see first example I posted)

If you want it at the composition itself (not recommended) you still have to include it (second example I posted)

The idea is that in the setup block you preload the DB you use for testing or do some other init action, so you grab files from the shared Codefresh volume. And the volume is already there for you with no extra conf :slight_smile:

1 Like