# Best practices for config reusability?

**URL:** <https://community.codefresh.io/t/best-practices-for-config-reusability/643>\
**Category:** Pipeline Workflows\
**Created:** [October 1, 2021, 4:26pm UTC](https://community.codefresh.io/t/best-practices-for-config-reusability/643 "2021-10-01T16:26:38Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![renaud](https://sea1.discourse-cdn.com/flex019/user_avatar/community.codefresh.io/renaud/32/358_2.png) [@renaud](https://community.codefresh.io/u/renaud)\
**Post date:** [October 1, 2021, 4:26pm UTC](https://community.codefresh.io/t/best-practices-for-config-reusability/643/1 "2021-10-01T16:26:38Z")

</div>

Hi there,

We have a multi-repo microservices setup with a codefresh.yml config in each repo.  
We build a docker image for testing, run tests, build a production image, then commit the new tag to the gitops repo.

Most of the steps are heavily parameterized and identical between services, except for the testing steps in the middle, which vary a lot. We use native Codefresh “service containers” YAML for testing, not an external docker-compose, so we can’t even encapsulate testing steps into a shell script with the same name in each repo.

Of course with this setup, repetition is a huge problem and any change needs to be copy/pasted into every service (which also means n commits because we don’t use a monorepo)

Our problem would be solved instantly if Codefresh YAML allowed importing config snippets from a different path, and merged them before execution. I could put the common stuff in a shared repo somewhere (or at worst in a identical file in each repo, still a small win over a slightly different one). Then import it from the repo’s codefresh.yml which would have only the specific testing steps.

Or the other way round : all pipelines using the same shared codefresh config that only loads the different testing steps from each repo.

This isn’t currently possible is it ?  
Is there a chance this could be implemented ?  
If not, what do you suggest as an alternative ?

Thanks

---

<div class="post-metadata">

**Author:** ![Kostis](https://sea1.discourse-cdn.com/flex019/user_avatar/community.codefresh.io/kostis/32/391_2.png) [@Kostis](https://community.codefresh.io/u/Kostis)\
**Post date:** [October 4, 2021, 10:51am UTC](https://community.codefresh.io/t/best-practices-for-config-reusability/643/2 "2021-10-04T10:51:30Z")

</div>

Great question

Our recommended approach is documented here [Building Microservices · Codefresh | Docs](https://codefresh.io/docs/docs/ci-cd-guides/microservices/) or in blog form here [CI/CD Pipelines for Microservices | Codefresh](https://codefresh.io/continuous-deployment/ci-cd-pipelines-microservices/). Have you already gone through these?

Basically the plan is to reuse the whole pipeline, instead of importing different parts of it. This is very easy if you have a single programming language in your company. Otherwise you need one pipeline for each programming language.

The blog post also explains how we at Codefresh manage more than 30 applications using just 3 pipelines [GitHub - kostis-codefresh/codefresh-on-codefresh-pipelines: Pipelines use to manage Codefresh with Codefresh](https://github.com/kostis-codefresh/codefresh-on-codefresh-pipelines). All are in a single repo, so every time we change something globally we only need a single commit. Again the article has more details (see the Codefresh-on-Codefresh section).

Other alternatives we have seen people use, is creating reusable typed steps [Steps · Codefresh | Docs](https://codefresh.io/docs/docs/codefresh-yaml/steps/#creating-your-own-step) but I personally don’t think it is necessary.

And of course people have used shared child pipelines from top-level parent pipelines as way to group common functionality.

Would any of these 3 approaches work for you?

---

<div class="post-metadata">

**Author:** ![Miguel\_Medina](https://sea1.discourse-cdn.com/flex019/user_avatar/community.codefresh.io/miguel_medina/32/428_2.png) [@Miguel\_Medina](https://community.codefresh.io/u/Miguel_Medina)\
**Post date:** [November 16, 2021, 5:27pm UTC](https://community.codefresh.io/t/best-practices-for-config-reusability/643/3 "2021-11-16T17:27:30Z")

</div>

Hi @Kostis,

First of all thanks for your response. Just so you know, I’m a colleague of @renaud and I’ve been trying to follow your suggestion to define a unique pipeline to be used by our different microservices, but I’ve just reached a blocker when it comes to defining a step for running integration tests, as this would be the step that differs the most between services since we need to link different dependencies (messaging emulator, DB and so on).

Basically, the problem comes when I try to define the services for this step using composition with a local `docker-compose.yaml` file as one of the examples of Codefresh’s documentation mentions ([Service Containers - Docs](https://codefresh.io/docs/docs/codefresh-yaml/service-containers/)). My step is defined as follows:

```auto
  test:
    title: Running tests
    image: ${{build_test_image}}
    working_directory: ${{clone}}
    services:
      composition: 'docker-compose.yaml'
    stage: test`

```

and my local `docker-compose.yaml` looks like this:

````auto
version: "3"
accounts-server:
  image: accounts:latest
  environment:
    - CONSOLE=true
    - FIRESTORE_EMULATOR_HOST=firestore:8282
    - FIRESTORE_PROJECT_ID=test
  restart: on-failure
  depends_on:
    - firestore

firestore:
  image: google/cloud-sdk:345.0.0-emulators
  command: gcloud beta emulators firestore start --host-port=0.0.0.0:8282```

````

For me everything looks normal unless I’m missing something obvious but then when running that step on my pipeline I get the following error

```auto
[SYSTEM] .                                                                                                                                                 
 Message Failed to prepare composition: services                                                                                               
 Caused by Failed to read file /codefresh/volume/docker-compose.yaml                                                                             
 Documentation Link https://codefresh.io/docs/docs/codefresh-yaml/steps/freestyle/ 

```

I tried multiple combinations of this definition already and none of them worked. Could you give me a hand here?

Thanks in advance

---

<div class="post-metadata">

**Author:** ![Kostis](https://sea1.discourse-cdn.com/flex019/user_avatar/community.codefresh.io/kostis/32/391_2.png) [@Kostis](https://community.codefresh.io/u/Kostis)\
**Post date:** [November 17, 2021, 8:27am UTC](https://community.codefresh.io/t/best-practices-for-config-reusability/643/4 "2021-11-17T08:27:23Z")

</div>

Hello

The path `/codefresh/volume/docker-compose.yaml` looks wrong to me. Isn’t docker-compose.yaml part of your project? I would expect it to be

`/codefresh/volume/<your project name>/docker-compose.yaml`

As a quick check can you manually modify your step and do this?

```auto
 test:
    title: Running tests
    image: ${{build_test_image}}
    working_directory: ${{clone}}/my-project
    services:
      composition: 'docker-compose.yaml'
    stage: test`

```

or

```auto
 test:
    title: Running tests
    image: ${{build_test_image}}
    working_directory: ${{clone}}
    services:
      composition: '/codefresh/volume/my-project/docker-compose.yaml'
    stage: test`

```

There might be an issue on our side where the “composition” property is not honoring the working\_directory property.

You should also open an issue with our support team so that they can test these scenarios instead of you.

---

<div class="post-metadata">

**Author:** ![Miguel\_Medina](https://sea1.discourse-cdn.com/flex019/user_avatar/community.codefresh.io/miguel_medina/32/428_2.png) [@Miguel\_Medina](https://community.codefresh.io/u/Miguel_Medina)\
**Post date:** [November 17, 2021, 11:40am UTC](https://community.codefresh.io/t/best-practices-for-config-reusability/643/5 "2021-11-17T11:40:49Z")

</div>

Hi @Kostis,

I just tried both suggestions and the second one worked like a charm! As you said, it seems that `composition` property doesn’t point to the correct directory I will raise an issue about this.

Thanks again!

---

<div class="post-metadata">

**Author:** ![Ryan\_Collingham](https://sea1.discourse-cdn.com/flex019/user_avatar/community.codefresh.io/ryan_collingham/32/373_2.png) [@Ryan\_Collingham](https://community.codefresh.io/u/Ryan_Collingham)\
**Post date:** [December 1, 2021, 10:59am UTC](https://community.codefresh.io/t/best-practices-for-config-reusability/643/6 "2021-12-01T10:59:38Z")

</div>

Hello again @Kostis,

FYI I work with both Renaud and Miguel above. The solution you recommended mostly works but occasionally we are seeing an error in the tests:

```auto
Failed to read file /codefresh/volume/<project name>/docker-compose.yaml

```

However it works fine after restarting the pipeline. Any ideas why this might be happening?

---

<div class="post-metadata">

**Author:** ![Kostis](https://sea1.discourse-cdn.com/flex019/user_avatar/community.codefresh.io/kostis/32/391_2.png) [@Kostis](https://community.codefresh.io/u/Kostis)\
**Post date:** [December 1, 2021, 12:09pm UTC](https://community.codefresh.io/t/best-practices-for-config-reusability/643/7 "2021-12-01T12:09:34Z")

</div>

Hello

If you have an approval step in your pipeline make sure that you have enabled this option [Approval · Codefresh | Docs](https://codefresh.io/docs/docs/codefresh-yaml/steps/approval/#keeping-the-shared-volume-after-an-approval)

If you don’t have an approval step, then I don’t see why the file should not be there. There are some rare conditions with parallel steps that lead to file issues.

It is best to open a support ticket with the build URL and let our team investigate.

---

<div class="post-metadata">

**Author:** ![axilis-marko](https://sea1.discourse-cdn.com/flex019/user_avatar/community.codefresh.io/axilis-marko/32/602_2.png) [@axilis-marko](https://community.codefresh.io/u/axilis-marko)\
**Post date:** [June 13, 2023, 10:53am UTC](https://community.codefresh.io/t/best-practices-for-config-reusability/643/8 "2023-06-13T10:53:05Z")

</div>

We are getting a similar error:

```auto
[SYSTEM] .                                                                                                                                          
 Message Failed to prepare composition: services                                                                                        
 Caused by Failed to read file /codefresh/volume/<my-project>/docker-compose.yml                                            
 Documentation Link https://codefresh.io/docs/docs/codefresh-yaml/steps/freestyle/                                                                 

```

I’ve checked that the file exists (by debugging the previous step), and is readable by everyone.

---

<div class="post-metadata">

**Author:** ![Kostis](https://sea1.discourse-cdn.com/flex019/user_avatar/community.codefresh.io/kostis/32/391_2.png) [@Kostis](https://community.codefresh.io/u/Kostis)\
**Post date:** [June 13, 2023, 2:16pm UTC](https://community.codefresh.io/t/best-practices-for-config-reusability/643/9 "2023-06-13T14:16:22Z")

</div>

If the file exists already please open a support ticket with the build URL and let our team investigate.