Short-term CI infra plans (discussion) #19
Comments
|
+1 on using a bot if the goal is to replace Jenkins rather than get attached to it. What's the plan for the GH status updates? Jenkins doesn't seem to expose enough parameters to get multiple statuses to work in a useful way, and all the github modules on npmjs.org are missing useful things like using numeric repo ids (who renames or moves repos these days anyway, right?), so I've found Ruby to generally be the most productive way to use GitHub's API. |
|
@rmg I have a collection of packages in npm already for doing stuff with github, I'm not so concerned with that part, it's Jenkins that's the headache because it's API is kind of sloppy. Already I have a webhook to a test server collecting data with github-webhook and I've been playing with some of the Jenkins libraries in npm but they all inherit Jenkin's crappy API problems but since it's fairly simple JSON it might be more productive to roll our own focused on just what we need. The flow looks will look like this, and this is purely for iojs/io.js at the moment:
Additional step in there is to post a comment to the PR with a link that the TC can use (authenticated against GitHub) to trigger a "multi" build when only a "containers" build has been done. For joyent/node there is a Chrome plugin that gives them the ability to do this I believe but I'd rather not have to go that route. Currently I have these libs for working with GitHub:
And I'm always looking for an excuse to expand the functionality and add more libs! |
|
btw, the webhook server is very early at the moment but I'll open a repo here with the code as soon as I can so others can contribute! |
|
The github/janky docs mention a notifications plugin for Jenkins. I imagine it could post to the webhook server. Or worst case, add a curl one liner to the Jenkins jobs themselves to post their $BUILD_NUMBER to a PR specific URL given as a build parameter by the webhook server when it triggers the job. |
|
Check this out: https://jenkins-node-forward.nodesource.com/job/iojs+any-pr+containers/3/api/json "parameters": [
{
"name": "user",
"value": "rvagg"
},
{
"name": "project",
"value": "io.js"
},
{
"name": "branch",
"value": "v0.12"
}
]Additionally I could add a "pr" parameter that's just informational for this purpose so it's even easier to cross reference. The frustrating bit is that when you ask to create a build you don't get an ID straight back (as far as I can tell so far), without that piece of the puzzle we need to do some polling and cataloging of what Jenkins is doing so we can link everything together. |
|
What I meant is create some id (UUID? git sha1?) before the build is submitted and include it as a build parameter. Then have Jenkins call you back, using that provided id (possibly something as lame as |
|
@rmg yes, except the list of hosts is quite diverse, the UUID would be public via the Jenkins API and web site and therefore filtering post-backs from build slaves would be a major security hassle, I think I'd rather poll than have to deal with this! |
|
@rvagg understood. I hadn't considered which parts would be public and which parts would not. |
|
@rvagg I'm interested in contributing to this, when do you think you'll be ready for contributors/help? I tried the gitter room, but it seems to be dead/in-accessible. |
|
For PR and builds on Jenkins this plugin should do the trick AFAIK. |
|
I'm aiming on not investing too much in Jenkins and opt for building our own tooling. Within the next week or so I should be able to make a new repo here with the beginnings of a tool that does what we need and we can all contribute to awesomeifying it. |
|
Awesome, because I would hate to install java on my machine only for testing jenkins stuff out. :D |
|
So I'm thinking we might want to do docker webhooks as well, so we keep the docker containers on the Jenkins server in sync with the docker registry (and in turn with the build-containers repo). Thoughts? If worth while, I can take a stab at this. |
|
@wblankenship intriguing, how would this work? Can we just ping the local webhook to make sure it has the latest image from hub? |
|
Simply point https://registry.hub.docker.com/u/iojs/build/settings/webhooks/ to the server and trigger a |
|
neat, we'll need to do this from >1 server |
|
Certainly doable. Want me to open a ticket for this and self assign? |
|
yes pls |
|
@rvagg Are there still actionable items left? |
|
yeah, plenty of actionable items that are not properly documented elsewhere, should probably leave this open for now |
|
Almost a year! I think we've covered most things on this list, with a gap in containers/auto-testing. At the same time I think our strategy in that department has changed. Inclined to close. |
The current state of the build infrastructure can be seen here: http://jenkins.node-forward.nodesource.com/
Summary:
Currently we also have the ability to trigger builds on either the full "multi" or "containers" from any repo on GitHub but they must be triggered manually by someone who has access to Jenkins. So far that's only myself, @ryanstevens and @indutny but I can expand that list to the full TC and other trusted helpers.
Short-term goals
Pre-first-release goals
Mid-term goals
Miscellaneous goals
Beyond that, we want to fully make our own CI tooling and deprecate Jenkins, but that's a lower priority than just moving io.js forward.
The text was updated successfully, but these errors were encountered: