Readiness of database server and web server

Pipeline workflows
1

I have two docker images: SQL Server and Web Server.
Each takes 30 seconds to start.
Web Server start-up should wait for SQL Server readiness.
Selenium tests should wait for Web Server readiness.

How do I do that?

2

You can follow the documentation for readiness here Service Containers · Codefresh | Docs

3

The documentation does not have an example for my use case:
Web Server start-up should wait for SQL Server readiness.
Selenium tests should wait for Web Server readiness.

4

Hello

It is impossible to cover all the possible combinations of databases and external services that people use in their applications.

What have you tried so far? Can you paste your pipeline yaml so that we can offer advice?

5

Simplified version of my YAML:

version: "1.0"
steps:
  RunSeleniumTests:
    title: "Running Selenium tests"
    image: 'my-chrome-gradle-jdk8'
    commands:
      - gradle test 
    services:
      composition:
        sqlserver:
          image: 'my-sqlserver'
          ports:
          - 1433 
        webapp:
          image: 'my-webapp'
          ports:
          - 8080           
      readiness:
        image: mcr.microsoft.com/mssql/server:2019-latest
        timeoutSeconds: 30
        commands:
        - /opt/mssql-tools/bin/sqlcmd -S sqlserver,1433 -U my_username -P my_password -Q "SELECT COUNT(*) FROM my_table"

Of course it fails because sqlserver and webapp containers start at the same time.

6

The “commands” block can take multiple checks. Right now you only have one for checking the sql server. Can you add another command that checks your webserver?

This way, the readiness check will finish only when BOTH services are up.

Also does your webapp, retry until the DB is up? Or it only tries to connect once and then it stops?

7

No.

Yes.

That will not help.
The webapp will start and fail. .

8

I see.

Let’s forget Codefresh for a moment. How do you deploy this application in production right now?

Since Kubernetes doesn’t enforce the order of pods, how you achieve this requirement in your production cluster?

9

We do not deploy to Kubernetes now.

10

Well where do you deploy your application right now?

I am asking this, because even if we solve this problem in Codefresh, you will have the exact same issue when going to production.

So instead of solving the problem twice, would it be easier to just modify your application to retry the database connection? This way it will work in Codefresh pipelines right away, in Kubernetes, in Nomad or in any other container orchestration that you choose. Would that work in your case?

For Codefresh a quick solution would be to actually use two service container segments.
One at the top level with the database (Service Containers · Codefresh | Docs) and a second one with just the web server in your specific test for selenium. I haven’t tested it to verify it.

However, a much better solution is to simply modify your application. This a general good advice which is valid even outside of CI/CD. If your app is running in production and for some reason your SQL server loses connectivity (even for a small amount of time) then your app will crash (since it will never reconnect to the db)

If you have a retry/reconnect mechanism in your app then a network error will be a simple hiccup and your app will quickly recover.

11

It is a legacy application.

Could you provide an example?

12

Sure. It should be something like this

version: "1.0"
services:
  name: my_database
  composition:
    sqlserver::
      image: my-sqlserver
      ports:
        - 1433 
  readiness:
    image: mcr.microsoft.com/mssql/server:2019-latest
    timeoutSeconds: 30
    commands:
    - /opt/mssql-tools/bin/sqlcmd -S sqlserver,1433 -U my_username -P my_password -Q "SELECT COUNT(*) FROM my_table"
steps:
  RunSeleniumTests:
    title: "Running Selenium tests"
    image: 'my-chrome-gradle-jdk8'
    commands:
      - gradle test 
    services:
      composition:
        webapp:
          image: 'my-webapp'
          ports:
          - 8080           
      readiness:
        image: 'byrnedo/alpine-curl'
        timeoutSeconds: 30
        commands:
        - curl webapp::8080

If that doesn’t work and you don’t want to change your legacy app, the only other solution I can think of is to use wait-for-it in your Docker image

13

Your example does not work:
webapp does not see sqlserver.

14

I see.

Ok in this case you need to edit your dockerfiles and include the wait-for-it script. GitHub - vishnubob/wait-for-it: Pure bash script to test and wait on the availability of a TCP host and port