Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modernize Windows Modern and Legacy Jenkins CI #4767

Closed
wants to merge 7 commits into from
Closed

Modernize Windows Modern and Legacy Jenkins CI #4767

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8cbb049
Revert "Remove Dev12 CI Builds"
dilijev Mar 1, 2018
bb059ae
Update script logic for forcing detection of a specific msbuild version.
dilijev Mar 1, 2018
b35e066
Update Windows build matrix to Server 2016 + Dev15 / Legacy = Server …
dilijev Mar 1, 2018
484f177
Fix error in groovy script.
dilijev Mar 1, 2018
6a529fb
Remove msbuild version 12 detection logic.
dilijev Mar 27, 2018
d382ecc
Propagate defaultMachineTag throughout netci.groovy
dilijev Mar 27, 2018
22cc5d3
Refactor and update platform flag for testExtra param
dilijev Mar 27, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 11 additions & 26 deletions Build/scripts/add_msbuild_path.cmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,12 @@
:: add_msbuild_path.cmd
::
:: Locate msbuild.exe and add it to the PATH
set USE_MSBUILD_12=%1

if "%USE_MSBUILD_12%" == "True" (
echo Skipping Dev15 and trying Dev12...
goto :LABEL_USE_MSBUILD_12
set FORCE_MSBUILD_VERSION=%1

if "%FORCE_MSBUILD_VERSION%" == "msbuild14" (
echo Skipping Dev15 and trying Dev14...
goto :LABEL_USE_MSBUILD_14
)

where /q msbuild.exe
Expand All @@ -20,8 +21,10 @@ if "%ERRORLEVEL%" == "0" (

REM Try Dev15 first

set MSBUILD_VERSION=15.0
echo Trying to locate Dev15...

:LABEL_USE_MSBUILD_15
set MSBUILD_VERSION=15.0
set "MSBUILD_PATH=%ProgramFiles(x86)%\Microsoft Visual Studio\Preview\Enterprise\MSBuild\15.0\Bin"

if not exist "%MSBUILD_PATH%\msbuild.exe" (
Expand All @@ -40,8 +43,9 @@ if exist "%MSBUILD_PATH%\msbuild.exe" (
goto :MSBuildFound
)

echo Dev15 not found, trying Dev14...
echo Dev15 not found, trying to locate Dev14...

:LABEL_USE_MSBUILD_14
set MSBUILD_VERSION=14.0
set "MSBUILD_PATH=%ProgramFiles%\msbuild\%MSBUILD_VERSION%\Bin\x86"

Expand All @@ -53,25 +57,6 @@ if not exist "%MSBUILD_PATH%\msbuild.exe" (
set "MSBUILD_PATH=%ProgramFiles(x86)%\msbuild\%MSBUILD_VERSION%\Bin\amd64"
)

if exist "%MSBUILD_PATH%\msbuild.exe" (
goto :MSBuildFound
)

echo Dev14 not found, trying Dev12...

:LABEL_USE_MSBUILD_12
set MSBUILD_VERSION=12.0
set "MSBUILD_PATH=%ProgramFiles%\msbuild\%MSBUILD_VERSION%\Bin\x86"
echo Dev14 not found, trying Dev %MSBUILD_VERSION%

if not exist "%MSBUILD_PATH%\msbuild.exe" (
set "MSBUILD_PATH=%ProgramFiles(x86)%\msbuild\%MSBUILD_VERSION%\Bin"
)

if not exist "%MSBUILD_PATH%\msbuild.exe" (
set "MSBUILD_PATH=%ProgramFiles(x86)%\msbuild\%MSBUILD_VERSION%\Bin\amd64"
)

if not exist "%MSBUILD_PATH%\msbuild.exe" (
echo Can't find msbuild.exe in "%MSBUILD_PATH%"
goto :SkipMsBuildSetup
Expand All @@ -81,7 +66,7 @@ if not exist "%MSBUILD_PATH%\msbuild.exe" (
echo MSBuild located at "%MSBUILD_PATH%"

set "PATH=%MSBUILD_PATH%;%PATH%"
set USE_MSBUILD_12=
set FORCE_MSBUILD_VERSION=
set MSBUILD_PATH=

:SkipMsBuildSetup
8 changes: 0 additions & 8 deletions Build/scripts/locate_msbuild.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,14 +68,6 @@ function Locate-MSBuild() {
return $msbuildExe
}

$_ = WriteMessage "Dev14 not found, trying Dev12..."

$msbuildExe = Locate-MSBuild-Version("12.0")
if ($msbuildExe -and (Test-Path $msbuildExe)) {
$_ = WriteMessage "Found `"$msbuildExe`""
return $msbuildExe
}

WriteErrorMessage "Can't find msbuild.exe."
return "" # return empty string
}
55 changes: 40 additions & 15 deletions netci.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,18 +18,32 @@ def msbuildTypeMap = [

// convert `machine` parameter to OS component of PR task name
def machineTypeToOSTagMap = [
'Windows_NT': 'Windows', // Windows Server 2012 R2, equivalent to Windows 8.1 (aka Blue)
'Windows 7': 'Windows 7', // Windows Server 2008 R2 ~= Windows 7
'Windows_NT': 'Windows', // 'latest-or-auto' -> Windows Server 2012 R2 ~= Windows 8.1 aka Blue
// 'latest-dev15-5' -> Windows Server 2016 ~= Windows 10 RS2/RS3, with Dev15.5
'Ubuntu16.04': 'Ubuntu',
'OSX': 'OSX'
]

def defaultMachineTag = 'latest-or-auto'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: defaultWindowsMachineTag

def defaultWindowsMachineTag = defaultMachineTag
def defaultXPlatMachineTag = defaultMachineTag

def legacyWindowsMachine = 'Windows 7'
def legacyWindowsMachineTag = defaultMachineTag
def legacyWindowsTestExtra = '-win7'

def latestWindowsMachine = 'Windows_NT'
def latestWindowsMachineTag = 'latest-dev15-5'
def latestWindowsTestExtra = '-win10'

def dailyRegex = 'dailies'

// ---------------
// HELPER CLOSURES
// ---------------

def CreateBuildTask = { isPR, buildArch, buildType, machine, configTag, buildExtra, testExtra, runCodeAnalysis, excludeConfigIf, nonDefaultTaskSetup ->
def CreateBuildTask = { isPR, buildArch, buildType, machine, machineTag, configTag, buildExtra, testExtra, runCodeAnalysis, excludeConfigIf, nonDefaultTaskSetup ->
if (excludeConfigIf && excludeConfigIf(isPR, buildArch, buildType)) {
return // early exit: we don't want to create a job for this configuration
}
Expand Down Expand Up @@ -75,7 +89,7 @@ def CreateBuildTask = { isPR, buildArch, buildType, machine, configTag, buildExt
false, // doNotFailIfNothingArchived=false ~= failIfNothingArchived
false) // archiveOnlyIfSuccessful=false ~= archiveAlways

Utilities.setMachineAffinity(newJob, machine, 'latest-or-auto')
Utilities.setMachineAffinity(newJob, machine, machineTag ?: defaultWindowsMachineTag)
Utilities.standardJobSetup(newJob, project, isPR, "*/${branch}")

if (nonDefaultTaskSetup == null) {
Expand All @@ -95,11 +109,11 @@ def CreateBuildTask = { isPR, buildArch, buildType, machine, configTag, buildExt
}
}

def CreateBuildTasks = { machine, configTag, buildExtra, testExtra, runCodeAnalysis, excludeConfigIf, nonDefaultTaskSetup ->
def CreateBuildTasks = { machine, machineTag, configTag, buildExtra, testExtra, runCodeAnalysis, excludeConfigIf, nonDefaultTaskSetup ->
[true, false].each { isPR ->
['x86', 'x64', 'arm'].each { buildArch ->
['debug', 'test', 'release'].each { buildType ->
CreateBuildTask(isPR, buildArch, buildType, machine, configTag, buildExtra, testExtra, runCodeAnalysis, excludeConfigIf, nonDefaultTaskSetup)
CreateBuildTask(isPR, buildArch, buildType, machine, machineTag, configTag, buildExtra, testExtra, runCodeAnalysis, excludeConfigIf, nonDefaultTaskSetup)
}
}
}
Expand Down Expand Up @@ -142,7 +156,7 @@ def CreateXPlatBuildTask = { isPR, buildType, staticBuild, machine, platform, co
true, // doNotFailIfNothingArchived=false ~= failIfNothingArchived (true ~= doNotFail)
false) // archiveOnlyIfSuccessful=false ~= archiveAlways

Utilities.setMachineAffinity(newJob, machine, 'latest-or-auto')
Utilities.setMachineAffinity(newJob, machine, defaultXPlatMachineTag)
Utilities.standardJobSetup(newJob, project, isPR, "*/${branch}")

if (nonDefaultTaskSetup == null) {
Expand Down Expand Up @@ -213,42 +227,53 @@ def CreateStyleCheckTasks = { taskString, taskName, checkName ->
Utilities.addGithubPushTrigger(newJob)
}

Utilities.setMachineAffinity(newJob, 'Ubuntu16.04', 'latest-or-auto')
Utilities.setMachineAffinity(newJob, 'Ubuntu16.04', defaultXPlatMachineTag)
}
}

// ----------------
// INNER LOOP TASKS
// ----------------

CreateBuildTasks('Windows_NT', null, null, "-winBlue", true, null, null)
CreateBuildTasks(latestWindowsMachine, latestWindowsMachineTag, null, null, latestWindowsTestExtra, true, null, null)

// Add some additional daily configs to trigger per-PR as a quality gate:
// x64_debug Slow Tests
CreateBuildTask(true, 'x64', 'debug',
'Windows_NT', 'ci_slow', null, '-winBlue -includeSlow', false, null, null)
latestWindowsMachine, latestWindowsMachineTag, 'ci_slow', null, "${latestWindowsTestExtra} -includeSlow", false, null, null)
// x64_debug DisableJIT
CreateBuildTask(true, 'x64', 'debug',
'Windows_NT', 'ci_disablejit', '"/p:BuildJIT=false"', '-winBlue -disablejit', false, null, null)
latestWindowsMachine, latestWindowsMachineTag, 'ci_disablejit', '"/p:BuildJIT=false"', "${latestWindowsTestExtra} -disablejit", false, null, null)
// x64_debug Lite
CreateBuildTask(true, 'x64', 'debug',
'Windows_NT', 'ci_lite', '"/p:BuildLite=true"', '-winBlue -lite', false, null, null)
latestWindowsMachine, latestWindowsMachineTag, 'ci_lite', '"/p:BuildLite=true"', "${latestWindowsTestExtra} -lite", false, null, null)
// x64_debug Legacy
CreateBuildTask(true, 'x64', 'debug',
legacyWindowsMachine, legacyWindowsMachineTag, 'ci_legacy', 'msbuild14', "${legacyWindowsTestExtra} -includeSlow", false, null, null)

// -----------------
// DAILY BUILD TASKS
// -----------------

if (!branch.endsWith('-ci')) {
// build and test on the usual configuration (VS 2015) with -includeSlow
CreateBuildTasks('Windows_NT', 'daily_slow', null, '-winBlue -includeSlow', false,
// build and test on the legacy configuration (Windows 7 + VS 2015 (Dev14))
CreateBuildTasks(legacyWindowsMachine, legacyWindowsMachineTag, 'daily_legacy', 'msbuild14', "${legacyWindowsTestExtra} -includeSlow", false,
/* excludeConfigIf */ { isPR, buildArch, buildType -> (buildArch == 'arm') },
/* nonDefaultTaskSetup */ { newJob, isPR, config ->
DailyBuildTaskSetup(newJob, isPR,
"Windows 7 ${config}",
'legacy\\s+tests')})

// build and test on the latest configuration (Server 2016 + VS 2017 (Dev15)) with -includeSlow
CreateBuildTasks(latestWindowsMachine, latestWindowsMachineTag, 'daily_slow', null, "${latestWindowsTestExtra} -includeSlow", false,
/* excludeConfigIf */ null,
/* nonDefaultTaskSetup */ { newJob, isPR, config ->
DailyBuildTaskSetup(newJob, isPR,
"Windows ${config}",
'slow\\s+tests')})

// build and test on the usual configuration (VS 2015) with JIT disabled
CreateBuildTasks('Windows_NT', 'daily_disablejit', '"/p:BuildJIT=false"', '-winBlue -disablejit', true,
// build and test on the latest configuration (Server 2016 + VS 2017 (Dev15)) with JIT disabled
CreateBuildTasks(latestWindowsMachine, latestWindowsMachineTag, 'daily_disablejit', '"/p:BuildJIT=false"', "${latestWindowsTestExtra} -disablejit", true,
/* excludeConfigIf */ null,
/* nonDefaultTaskSetup */ { newJob, isPR, config ->
DailyBuildTaskSetup(newJob, isPR,
Expand Down
8 changes: 4 additions & 4 deletions test/jenkins.build.init.cmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ if not "%JENKINS_BUILD%" == "True" (
set REPO_ROOT=%~dp0\..

set JENKINS_BUILD_ARGS=
set JENKINS_USE_MSBUILD_12=
set JENKINS_FORCE_MSBUILD_VERSION=
:ContinueArgParse
if not [%1]==[] (
if [%1]==[msbuild12] (
set JENKINS_USE_MSBUILD_12=True
if [%1]==[msbuild14] (
set JENKINS_FORCE_MSBUILD_VERSION=%1
goto :ContinueArgParseEnd
)

Expand All @@ -36,4 +36,4 @@ if not [%1]==[] (
:: Set up msbuild.exe
:: ========================================

call %REPO_ROOT%\Build\scripts\add_msbuild_path.cmd %JENKINS_USE_MSBUILD_12%
call %REPO_ROOT%\Build\scripts\add_msbuild_path.cmd %JENKINS_FORCE_MSBUILD_VERSION%