Hello, I am trying to do something that surely must be a very common requirement, yet I’m finding it very difficult to achieve with codefresh.
I am developing a service, which has lots of dependencies on other services and cloud infrastructure (DB, messaging etc.). For the integration test of this service, I have a docker-compose.yml file which describes that service and all of its dependencies, their configuration (via environment variables) and dependence order (via depends_on). It also contains an “inttest” service which runs the tests. Locally I can run the tests with
docker-compose run inttest and it all works as intended.
However I am having problems translating this into codefresh. The naive approach is to add a freestyle stage with the docker/compose image and to run
docker-compose run inttest. However, it seems that docker-in-docker in not allowed in codefresh when using this “SaaS runtime” which apparently we have.
The second option is to define a “composition” step. This immediately has two problems: first of all, I must split my single docker-compose.yaml file into two parts, one which defines the “composition services” and one which defines the “composition candidate” i.e. the inttest script. But even if I do this, it is not possible to use my docker-compose.yaml file in the
composition: field: since I need to pass in the dynamically generated image tag, but codefresh expects a different syntax for including environment variables than docker-compose itself uses.
The only solution I see is to completely copy the contents of my docker-compose.yml file inline in my codefresh.yml file and modify it to use the codefresh environment variable syntax. This is not a good workaround as it means I now have two independent definitions of the test environment (CI and local) which must be manually kept up to date with each other!
I feel like this must be a very common requirement - to run integration tests locally as well as in CI using docker-compose - so it surprises me that it feels so difficult to achieve! I feel like either I am missing something obvious, or codefresh is missing a feature that would make this common requirement much easier to achieve. Maybe there needs to be some option for codefresh to read valid docker-compose.yml files and convert to its own non-standard syntax? Or a way to use docker-in-docker in the SaaS runtime? Any help would be appreciated!