Skip to content

Latest commit

 

History

History
91 lines (68 loc) · 4.58 KB

README.md

File metadata and controls

91 lines (68 loc) · 4.58 KB
Raw

PingCAP CI/CD public configurations

Servers

New

Only one site in our roadmap.

Notices: when you develop new pipeplines or jobs, you should put them in /staging folder and create a PR. When the PR merged, seed job will deploy it in staging CI server. When you tested with successful results, please create a PR with contents moved in top level folders and comment the PR with your tested job links in staging CI server.

Old ones

Directory structure

  • /docs documents about CI/CD jobs, tools, etc.
  • /prow-jobs store the top level prow jobs.
  • /jobs store Jenkins CI job DSL files. some jobs will be called by prow jobs defined in /prow-jobs.
  • /pipelines store Jenkins CI pipeline scripts.
  • /libraries store Jenkins CI shared libraries.
  • /staging store staging jobs and pipelines before deploying to production.
    • /staging/jobs like /jobs but only deployed to staging env.
    • ......
  • [Deprecated] /jenkins/jobs store migrated or migrating Jenkins CI job DSL files for server ci.pingcap.net, ci2.pingcap.net and cd.pingcap.net.
  • [Deprecated] /jenkins/pipelines store Jenkins CI pipeline scripts for server ci.pingcap.net, ci2.pingcap.net and cd.pingcap.net.

File structure for jobs and pipelines

For prow jobs:

using /prow-jobs/<org>/<repo>/<branch-special>-<job-type>.yaml path to store prow jobs.

  • the <branch-special> part is used to distinguish different branches.
    • the latest stores prow jobs for trunk branch and feature branches.
    • the release-x.y stores prow jobs for specific release branches.
    • if all the branches have the same prow jobs, you can ignore the <branch-special> part.
  • the <job-type> part is used to distinguish different types of jobs, there are some common types:
    • presubmits for jobs running on pull requests.
    • postsubmits for jobs running on pull request merges.
    • periodics for jobs running on a schedule.

Also you should run the command to update the /prow-jobs/kustomization.yaml file for CI GitOps deployment:

# run in top folder of the repo:
.ci/update-prow-job-kustomization.sh

For Jenkins jobs:

We use folder structure to organize Jenkins jobs.

using /jobs/<org>/<repo>/<branch-special>/<job-type>_<job-name>.groovy structure. Examples:

  • /jobs/pingcap/tiflash/latest/merged_build.groovy

  • /jobs/pingcap/tiflash/release-8.5/pull_unit_test.groovy

  • the <branch-special> part is used to distinguish different branches.

    • the latest stores Jenkins jobs for trunk branch and feature branches.
    • the release-x.y stores Jenkins jobs for specific release branches (like release-8.5, release-8.4, etc)
    • if all the branches have the same Jenkins jobs, you can ignore the <branch-special> layer.
    • maybe you will can define some special branches like release-x.y.z for released path version branches to support customized jobs for hotfix branches.

  • the <job-type> part is used to distinguish different types of jobs, there are some common types:

    • pull for jobs running on pull requests, it will work with the presubmits type prow jobs.
    • merged for jobs running on pull request merges, it will work with the postsubmits type prow jobs.
    • periodics for jobs running on a schedule, it will work with the periodics type prow jobs.

  • the <job-name> part is used to define the job name, the naming format is [a-z][a-z0-9_]*[a-z0-9].

  • the aa_folder.groovy file will be used to define the folder name in every folder recursively, do not change the name of the file.

For Jenkins pipelines:

  • using /pipelines/<org>/<repo>/<branch-special>/*.groovy file to store the Jenkins pipeline script which will be used by the Jenkins job.
    • pipelines/pingcap/tiflash/latest/merged_build.groovy
    • pipelines/pingcap/tiflash/release-8.5/pull_unit_test.groovy
  • using /pipelines/<org>/<repo>/<branch-special>/pod-*.yaml file to store the pod template definition which will be used by the Jenkins pipeline. Examples:
    • pipelines/pingcap/tiflash/latest/pod-pull_build.yaml
    • pipelines/pingcap/tiflash/release-8.5/pod-merged_build.yaml
  • the <branch-special> part is used to distinguish different branches, we keep it the same as the Jenkins job definition above.
  • the pod-*.yaml part is used to distinguish different pod templates.

More

Please refer to docs for details.