Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce time required for cloning remote bases #857

Merged
merged 1 commit into from
Mar 7, 2019

Conversation

pst
Copy link
Contributor

@pst pst commented Mar 6, 2019

This commit changes git/cloner.go from cloning the whole history
and then checking out the desired ref to a implementation that
only downloads the history for the desired ref.

It does so by first initializing an empty repository, setting the
source repository as a remote, fetching just the desired ref and
then hard resetting the empty local repo to that ref.

This reduces the time it takes to build the multibases example
as a remote base at ref v2.0.3 from an avg of 8s with the
current implementation to an avg of 2s out of 10 runs each, by
drastically decreasing the data transferred.

The improvement should increase as repositories grow.

This commit changes git/cloner.go from cloning the whole history
and then checking out the desired ref to a implementation that
only downloads the history for the desired ref.

It does so by first initializing an empty repository, setting the
source repository as a remote, fetching just the desired ref and
then hard resetting the empty local repo to that ref.

This reduces the time it takes to build the multibases example
as a remote base at ref v2.0.3 from an avg of 8s with the
current implementation to an avg of 2s out of 10 runs each, by
drastically decreasing the data transferred.

The improvement should increase as repositories grow.
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Mar 6, 2019
@k8s-ci-robot k8s-ci-robot requested review from droot and mengqiy March 6, 2019 18:32
@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Mar 6, 2019
@pst
Copy link
Contributor Author

pst commented Mar 6, 2019

/assign @pwittrock

@Liujingfang1 Liujingfang1 requested review from monopole and Liujingfang1 and removed request for droot and mengqiy March 6, 2019 19:45
Copy link
Contributor

@monopole monopole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the git-foo!
Looks like this passes examples/remoteBuild.md

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 7, 2019
@monopole
Copy link
Contributor

monopole commented Mar 7, 2019

PS, feel free to expand / improve https://github.com/kubernetes-sigs/kustomize/blob/master/examples/remoteBuild.md

It's both an example and a test that must exit with $?==0

@monopole
Copy link
Contributor

monopole commented Mar 7, 2019

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: monopole, pst

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 7, 2019
@k8s-ci-robot k8s-ci-robot merged commit a59577c into kubernetes-sigs:master Mar 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants