Best practices for config reusability?

Hi there,

We have a multi-repo microservices setup with a codefresh.yml config in each repo.
We build a docker image for testing, run tests, build a production image, then commit the new tag to the gitops repo.

Most of the steps are heavily parameterized and identical between services, except for the testing steps in the middle, which vary a lot. We use native Codefresh “service containers” YAML for testing, not an external docker-compose, so we can’t even encapsulate testing steps into a shell script with the same name in each repo.

Of course with this setup, repetition is a huge problem and any change needs to be copy/pasted into every service (which also means n commits because we don’t use a monorepo)

Our problem would be solved instantly if Codefresh YAML allowed importing config snippets from a different path, and merged them before execution. I could put the common stuff in a shared repo somewhere (or at worst in a identical file in each repo, still a small win over a slightly different one). Then import it from the repo’s codefresh.yml which would have only the specific testing steps.

Or the other way round : all pipelines using the same shared codefresh config that only loads the different testing steps from each repo.

This isn’t currently possible is it ?
Is there a chance this could be implemented ?
If not, what do you suggest as an alternative ?

Thanks

Great question

Our recommended approach is documented here Building Microservices · Codefresh | Docs or in blog form here https://codefresh.io/continuous-deployment/ci-cd-pipelines-microservices/. Have you already gone through these?

Basically the plan is to reuse the whole pipeline, instead of importing different parts of it. This is very easy if you have a single programming language in your company. Otherwise you need one pipeline for each programming language.

The blog post also explains how we at Codefresh manage more than 30 applications using just 3 pipelines GitHub - kostis-codefresh/codefresh-on-codefresh-pipelines: Pipelines use to manage Codefresh with Codefresh. All are in a single repo, so every time we change something globally we only need a single commit. Again the article has more details (see the Codefresh-on-Codefresh section).

Other alternatives we have seen people use, is creating reusable typed steps Steps · Codefresh | Docs but I personally don’t think it is necessary.

And of course people have used shared child pipelines from top-level parent pipelines as way to group common functionality.

Would any of these 3 approaches work for you?