Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci]: Download artifact instead of using nfs storage #6570

Merged
merged 4 commits into from
Jan 28, 2021
Merged

[ci]: Download artifact instead of using nfs storage #6570

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
68a1564
Download artifact instead of using nfs storage
qiluo-msft Jan 27, 2021
234d02b
Add missing DownloadPipelineArtifact task in vstest
qiluo-msft Jan 27, 2021
b8bc10a
Add displayName to DownloadPipelineArtifact task and add `set -x`
qiluo-msft Jan 27, 2021
b4efc2a
Change the path for the downloaded artifact
qiluo-msft Jan 27, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 16 additions & 10 deletions azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,7 @@ stages:
ENABLE_DOCKER_BASE_PULL=y make configure PLATFORM=vs
trap "sudo rm -rf fsroot" EXIT
make USERNAME=admin SONIC_BUILD_JOBS=$(nproc) $CACHE_OPTIONS \
target/docker-sonic-vs.gz target/sonic-vs.img.gz target/docker-ptf.gz && \
sudo cp target/sonic-vs.img.gz /nfs/azpl/kvmimage/sonic-vs.$(Build.BuildNumber).img.gz && \
sudo cp target/docker-sonic-vs.gz /nfs/azpl/kvmimage/docker-sonic-vs.$(Build.BuildNumber).gz
target/docker-sonic-vs.gz target/sonic-vs.img.gz target/docker-ptf.gz
displayName: 'Build sonic image'
- publish: $(System.DefaultWorkingDirectory)/
artifact: sonic-buildimage.kvm
Expand Down Expand Up @@ -122,22 +120,26 @@ stages:
runBranch: 'refs/heads/master'
displayName: "Download sonic swss common deb packages"

- task: DownloadPipelineArtifact@2
inputs:
artifact: sonic-buildimage.kvm
displayName: "Download sonic-buildimage.kvm artifact"

- script: |
set -x
sudo dpkg -i --force-confask,confnew ../sonic-swss-common.amd64.ubuntu20_04/libswsscommon_1.0.0_amd64.deb
sudo dpkg -i ../sonic-swss-common.amd64.ubuntu20_04/python3-swsscommon_1.0.0_amd64.deb
sudo docker load -i /nfs/azpl/kvmimage/docker-sonic-vs.$(Build.BuildNumber).gz
sudo docker load -i ../target/docker-sonic-vs.gz
docker tag docker-sonic-vs:latest docker-sonic-vs:$(Build.BuildNumber)
username=$(id -un)
set -x

trap "docker ps; docker images; ip netns list; \
docker rmi docker-sonic-vs:$(Build.BuildNumber); \
ip netns list | grep -E [-]srv[0-9]+ | awk '{print $1}' | xargs -I {} sudo ip netns delete {}; \
sudo chown -R ${username}.${username} .; \
sudo chown -R ${username}.${username} $(System.DefaultWorkingDirectory)" EXIT
pushd platform/vs/tests
sudo py.test -v --junitxml=tr.xml --imgname=docker-sonic-vs:$(Build.BuildNumber) && \
sudo rm /nfs/azpl/kvmimage/docker-sonic-vs.$(Build.BuildNumber).gz
sudo py.test -v --junitxml=tr.xml --imgname=docker-sonic-vs:$(Build.BuildNumber)
displayName: "Run vs tests"

- task: PublishTestResults@2
Expand All @@ -149,9 +151,15 @@ stages:
displayName: "kvmtest"
timeoutInMinutes: 240
steps:
- task: DownloadPipelineArtifact@2
inputs:
artifact: sonic-buildimage.kvm
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am not sure it will download the artifact from the current job?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I learned on https://docs.microsoft.com/en-us/azure/devops/pipelines/artifacts/pipeline-artifacts?view=azure-devops&tabs=yaml

Pipeline artifacts provide a way to share files between stages in a pipeline or between different pipelines.

In this case, we share files between stages.

displayName: "Download sonic-buildimage.kvm artifact"

- script: |
set -x
sudo mkdir -p /data/sonic-vm/images
sudo cp -v /nfs/azpl/kvmimage/sonic-vs.$(Build.BuildNumber).img.gz /data/sonic-vm/images/sonic-vs.img.gz
sudo cp -v ../target/sonic-vs.img.gz /data/sonic-vm/images/sonic-vs.img.gz
sudo gzip -fd /data/sonic-vm/images/sonic-vs.img.gz
username=$(id -un)
sudo chown -R $username.$username /data/sonic-vm
Expand Down Expand Up @@ -193,8 +201,6 @@ stages:

exit 2
else
sudo rm /nfs/azpl/kvmimage/sonic-vs.$(Build.BuildNumber).img.gz

cp -r /data/sonic-mgmt/tests/logs $(Build.ArtifactStagingDirectory)/
fi
displayName: "Run T0 tests"
Expand Down