We have the feature of running steps in parallel (Advanced Workflows with Parallel steps · Codefresh | Docs). In this case, the number of steps we want to run is known in advance.
I have a scenario where the number of steps I want to run in parallel is dynamic, calculated via an API call to be precise, something like:
version: '1.0'
steps:
GetInfo:
title: Get list of files
image: alpine
commands:
- some logic to get the list_of_files; list might have 1 or many files
- export list_of_files to be usable in subsequent steps
WritingInParallel:
type: parallel
steps:
writing_file_n:
title: Step_n
image: alpine
commands:
- echo "Step_n" > list_of_files[n]
Somehow, I wish to run n (number of file in list_of_files) parallel steps here. Is there a way of doing it using parallel or scale method or any other method?
I have a similar issue where we have an Nx monorepo and I’m not sure how many “apps” I need to build until a step runs. What I do is setup a new pipeline that gets triggered N number of times based on the output from Nx.