How to create custom pipelines from defaults for install/promote etc. helm releases?

Pipeline workflows
#1

Hello,

There are default pipelines provided by codefresh that handles installation / promotion of helm charts.

First, it looks not possible to see those default pipeline running in the “builds” tab, and that would be handy to understand better what’s happening.

Second, I’d really like to create my own promotion pipelines, that could automatically change values, set some metadata, eventually more, but I can’t find a “template” starting point. How can I “clone” the default pipelines ?

Thanks !

#2

Hello

I can check with engineering but I believe that the default pipelines are simply the respective helm commands

install -> helm install, test -> helm test and so on.

We have documentation on what variables the default pipelines use

Let me know if this answers your question.

#3

Hello,

It makes sense, don’t know how I missed this part.

However, I’m not sure how the “promote” pipeline work, and if it is also a simple “helm install” with a different target, what are the available variables there? Can’t find a reference to this case in the page.

I’m planning to do a rather simple thing as “promotion” : just target another namespace (and/or cluster) and configure the chart with different values.

I have no idea neither on best practices for values.yaml storage when they contain sensitive data (for example for production or near-production environment, where it can contain api keys, database access, etc…). Any hints ?

Thanks !

#4

Hello

The variables for promotion are mentioned here https://codefresh.io/docs/docs/new-helm/helm-environment-promotion/#promoting-helm-releases-with-the-gui

If I recall correctly if a release doesn’t exist in the namespace the default pipeline is helm install. If a release exists, the default pipeline is helm upgrade.

Regarding secrets there are many options.

  1. Using Kubernetes secrets
  2. Using Sops (so they are encrypted)
  3. Using Hashicorp vault (might be overkill in your case)

You can search for “Helm + secrets” and see many suggested solutions. All of them should work with Codefresh.

Let me know if that helps. I am happy to clarify more.

#5

Indeed, I don’t understand why but although I went like 10 times on this page but somehow did not get what it was saying … I think I understood the “builtin” part of “Codefresh will use a built-in install/upgrade pipeline for performing the promotion” as “it will use the default pipeline that you can’t inspect”. Anyway, thanks for the pointer.

Although since I asked, I found a solution and the simple builtin pipeline was not sufficient to mimic the default behaviour, I had to add a bunch of environment stuff around. Documented values are indeed provided, but they’re not picked up by the default install/upgrade step.

I was not asking about secrets, but suggestions on where to store helm values files. As codefresh provides opinions on where to store charts (aka chartmuseum), and have opinions about “stages” (the helm kanban) I thought maybe there are ideas about where to store environment value files that depends on kanban columns (values-prod.yaml, values-staging.yaml …). I guess I’ll use a versioned S3 bucket or even a git repo, then.

#6

Ah! If the documentation is not clear it is our fault. I will check that page again and clarify the concepts better.

Regarding values, there are usually 2 choices

  1. Store the values in the same git repo as the chart itself. This is best for small apps and small teams
  2. Use a completely different git repository that has only the values and nothing else (best for large companies).

Up to you. You can also choose to keep the chart with the source code or on its own.

So a very simple scenario is a git repo that has source code + chart + values, while in a big company you might see 3 different git repos for each component.

We are actually preparing a Helm guide for our documentation, and your questions show me that I should move it up in the backlog.

#7

Thanks for your answer.

I just discovered that there is this concept of “shared configurations” within codefresh that may suits me for the smallest / less critical parts. I think I’ll favor something as a S3 versionned bucket for later stages.

Best regards,

Romain

#8

Hello again.

I just published our new Helm guide. It uses different value configurations stored in Git.
I think it is exactly what you are looking.

Any feedback welcome.

The next guide will talk specifically about the Helm promotion board.

#9

Thanks ! I’ll use it very soon to improve my pipelines and will let you know if there are comments while doing it.