Hello, I am trying to do something that surely must be a very common requirement, yet I’m finding it very difficult to achieve with codefresh.
I am developing a service, which has lots of dependencies on other services and cloud infrastructure (DB, messaging etc.). For the integration test of this service, I have a docker-compose.yml file which describes that service and all of its dependencies, their configuration (via environment variables) and dependence order (via depends_on). It also contains an “inttest” service which runs the tests. Locally I can run the tests with docker-compose run inttest and it all works as intended.
However I am having problems translating this into codefresh. The naive approach is to add a freestyle stage with the docker/compose image and to run docker-compose run inttest. However, it seems that docker-in-docker in not allowed in codefresh when using this “SaaS runtime” which apparently we have.
The second option is to define a “composition” step. This immediately has two problems: first of all, I must split my single docker-compose.yaml file into two parts, one which defines the “composition services” and one which defines the “composition candidate” i.e. the inttest script. But even if I do this, it is not possible to use my docker-compose.yaml file in the composition: field: since I need to pass in the dynamically generated image tag, but codefresh expects a different syntax for including environment variables than docker-compose itself uses.
The only solution I see is to completely copy the contents of my docker-compose.yml file inline in my codefresh.yml file and modify it to use the codefresh environment variable syntax. This is not a good workaround as it means I now have two independent definitions of the test environment (CI and local) which must be manually kept up to date with each other!
I feel like this must be a very common requirement - to run integration tests locally as well as in CI using docker-compose - so it surprises me that it feels so difficult to achieve! I feel like either I am missing something obvious, or codefresh is missing a feature that would make this common requirement much easier to achieve. Maybe there needs to be some option for codefresh to read valid docker-compose.yml files and convert to its own non-standard syntax? Or a way to use docker-in-docker in the SaaS runtime? Any help would be appreciated!
This is an excellent question but it has several parts, so let’s take things one by one.
First of all, while docker-compose seems like a perfect candidate for local testing, in reality it has several issues. The most important one is that “depends_on” just controls startup order. It doesn’t actually enforce service dependencies (i.e. waiting until a service is ready before starting the next). This limitation is clearly mentioned in the official documentation at Control startup and shutdown order in Compose | Docker Documentation and several workarounds are offered (wait-for-it.sh and friends).
So the setup that you have right now is flaky, and while it seems to work ok, it is prone to racing conditions. Depending on the load of your laptop maybe your tests will fail at some point because there was an error with the dependencies.
So trying to transfer a flaky mechanism of running tests in Codefresh is not ideal (and this is why we haven’t invested that much in that part, per your suggestion of converting an existing docker compose file to Codefresh)
Even though we still support compositions the recommended way to run integration tests in Codefresh is with service containers Service Containers · Codefresh | Docs. This syntax is similar to docker compose but we offer additional features for actually checking the readiness of a service, preloading test data to dbs and so on.
You CAN reuse your docker-compose file. There is an example in the middle of the page (search the text “You can also use a docker-compose.yml file that you might have in your git repository”) but even that comes with some limitations as you already have found out.
I know this is not the answer you want to hear but our recommendation for production applications is to look at specialized tools (tilt, garden, telepresence, okteto etc) for local development and abandon docker-compose.
Hi Kostis, thanks for your reply. It’s disappointing to hear that codefresh does not have a good story with regards to docker-compose.
I have to say I completely disagree with your assertion that tests using docker-compose are inherently “flaky”. As explained in the article you linked about startup order, cloud services should be built with resilience in mind and should not break if one of their dependencies is temporarily unavailable, whether that be due to startup ordering or due to temporary network issues. I would flip it around and say that if your tests require special control of startup ordering/timing, that’s a red flag on how resillient your services will be in production!
The issue I face with re-using my docker-compose file, is that every build will have a dynamically generated tag (via CF_BRANCH_TAG_NORMALIZED), so therefore I cannot hardcode a particular image name/tag in my docker-compose file for the services I am testing - it needs to be passed in somehow. I tried this already with environment variables but codefresh complains it cannot read the docker-compose.yaml file - I presume this is down to the syntax differences in environment variables? Is there any other solution that I am not seeing?
I think we should support parameters in the docker compose file.
My suggestion is to open a ticket with us (from the top right menu in the Codefresh UI) and our support team will investigate further your scenario.
Oh yes I agree with you! If you have managed to make your services resilient then you are already one step ahead. However, the majority of companies I have seen have not reached this state.
I also strongly suggest to look at the local dev tools I mentioned. They are all much more configurable and flexible then docker compose.