Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partition batch jobs launched from SCDF don't set deployer properties for worker task pods it launches #462

Open
vrajkuma opened this issue Oct 24, 2021 · 0 comments
Assignees
Labels
type/enhancement Is an enhancement request
Milestone

Comments

@vrajkuma
Copy link

Discussed the issue in this stackoverflow post : https://stackoverflow.com/questions/69683040/scdf-partition-batch-job-using-spring-cloud-kubernetes-deployer-deployer-pro

Basically in running the partitioned-batch-job sample provided by SCDF in a kubernetes cluster, I am unable to set deployer properties for the worker tasks spawned by the master step. The deployer.<app-name>.kubernetes.<prop-name> properties only apply to the master pod that is created, but the worker pods it spawns using kubernetes deployer don't get them. For example, in my case setting properties for volumes/volumeMounts during the task launch don't get enforced on the worker pods/containers where the volumeMounts are actually needed to process files. These are mounted on the master pod though.

Also, didn't find any documentation on where and how we could set spring.cloud.deployer.properties for worker pods given those are launched by the partitioned task framework

@github-actions github-actions bot added the status/need-triage Team needs to triage and take a first look label Oct 24, 2021
@vrajkuma vrajkuma changed the title Partition batch jobs launched from SCDF can't set deployer properties for worker task pods Partition batch jobs launched from SCDF don set deployer properties for worker task pods Oct 24, 2021
@vrajkuma vrajkuma changed the title Partition batch jobs launched from SCDF don set deployer properties for worker task pods Partition batch jobs launched from SCDF don't set deployer properties for worker task pods it launches Oct 24, 2021
@cppwfs cppwfs added type/enhancement Is an enhancement request and removed status/need-triage Team needs to triage and take a first look labels Nov 3, 2021
@markpollack markpollack modified the milestones: 2.8.0-RC1, 2.9.0-M1 Oct 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement Is an enhancement request
Projects
None yet
Development

No branches or pull requests

3 participants