Grunt-like plugin listing

[vesln]

http://gruntjs.com/plugins

I think it's pretty neat and we'll stop forcing the people to submit a pull requests but fetch plugins automatically. I'll be doing the same for hydro.

@logicalparadox

[keithamus]

This idea has a lot of merit, jQuery recently moved to this approach, and customelements.io.

To do this we'd need to make it a requirement that every plugin has 'chai-plugin' (and/or ecosystem:chai) as a tag in their package.json. It would be worth spending time evangalising those that don't - but are on our current list - before we implement this on the site, to avoid missing any plugins.

To get a list of plugins on the website, we could either simply link to https://www.npmjs.com/browse/keyword/chai-plugin and allow users to browse npm for plugins.

If we want to keep the plugins page, I cannot find any docs about an available npm registry api for retrieving JSON of packages by tag name - which could be a blocker for this - this may mean we have to manually download a cache of the registry and parse through it regularly (just like npm search does).

[keithamus]

I managed to dig up an endpoint here which shows a list of npm packages that have the 'chai-plugin' keyword. The keys array is an index of [0] being the keyword, [1] being the package name and [2] being the description.

It doesn't fit our exact use case - if we are to maintain feature parity we'd ideally need to have the following:

• npm package name
• description
• url (calculable from the name I suppose)
• a list of all of the keywords the package contains
• author

For each plugin page (e.g. http://chaijs.com/plugins/chai-spies) we could probably get away with making a request to the individual package page (e.g. http://registry.npmjs.org/chai-spies) which has all of the necessary details.

The other problem is that the URL does not support CORS - meaning any XHRs we make to it will fail, without a proxy layer.

Maybe someone at @npm could help us with how to tackle this.

[keithamus]

Another option would be to have a script which fetches the npm search cache (https://registry.npmjs.org/-/all/since?stale=update_after&startkey=TIMESTAMP), and builds a plugins.json file from that - by having the search cache we'd have access to all metadata about a package.

The problem with this process would be the regularity of updates from which the plugins.json file is populated. We could use something like the travis ci cron trigger to run a nightly build which updates the plugins.json so at a minimum the list gets updated nightly.

[logicalparadox]

Alternatively we could separate the plugin listing from the main website and host it on a subdomain (http://plugins.chaijs.com). That way we can have the main site as a gh-pages thing and the plugins run separately. If we dockerize the plugins app (or alternatively the whole thing), I can migrate it to my more reliable infrastructure, and automate the release of said assets on github webhooks.

[keithamus]

@logicalparadox while I'd certainly enjoy putting together a dockerized app, the beauty of having a fully static site and using gh-pages is that it's pretty much fire-and-forget, auto-releasing with virtually zero configuration, and maintainable by anyone of our contributors, regardless of server side experience. It also has the added benefit of being free at every scale, while using Heroku/Amazon/<insert host here> comes at some cost, which also means billing & bus factor concerns. I guess its a case of "spend a lot of time once to get it to work nicely as a static site" vs "spend a little bit of time regularly to maintain upkeep of server, billing, upgrades etc".

[keithamus]

As a task for doing this, we need all chai plugins to update their package.json#keywords to include "chai-plugin". A list of the current plugins we know about is here: https://github.com/chaijs/chai-docs/blob/master/plugins.js. Someone needs to go through each one, and raise a PR that adds the keyword. As an example commit, see chai-bignumber#a742467

[keithamus]

Made some significant process with this. All plugins have their contents fetched, and the script we have pulls in plugins from npm with the keyword chai-plugin.

_legacy_plugins exists which is just an enumeration of all of the plugins that /don't/ have the chai-plugin keyword, but we still want to display on the website.

As per my previous comment - we need to go through each plugin listed in _legacy_plugins, and raise a PR to add the keyword to the package.json.

[astorije]

@keithamus, added the keyword to my plugin \o/ (astorije/chai-immutable@39540ff).
Congrats and thanks on this, it sounds cool!

Comparing to the current solution, I have a few questions though:

• I'm guessing you are retrieving the description from package.json, right?
• The name given in package.json and the actual name of the plugin will often be slightly different (for example, mine is "chai-immutable" vs "Chai Immutable". What's your recommendation/solution on this?
• For the tags, will you now use the keywords listed in the package.json minus chai and chai-plugin? What happens to keywords like test, assertions, ... that are useful in the npm page but probably not on Chai's docs?
• How are you planning to display that plugin X does/does not have Node.js/browser support, and the file to use for browser?
• I can see how easy it can be to overload Chai plugins page by adding the chai-plugin to npm projects. How are you planning to blacklist/remove these?

Of course, most of these questions assume you are keeping a 1:1 functionality with the current look of the plugins, so maybe it's irrelevant :-)

Thanks again and good luck on shipping this!!

[keithamus]

I'm guessing you are retrieving the description from package.json, right?

Yup this is correct. Seems like most (or even all) of the descriptions are fine for this purpose.

The name given in package.json and the actual name of the plugin will often be slightly different (for example, mine is "chai-immutable" vs "Chai Immutable". What's your recommendation/solution on this?

Personally, I think this is an acceptable compromise. chai-immutable is your canonical name in Github, npm and possibly bower. Using a friendlier name might look nicer from a design/typography perspective, but as a developer I really want to see the name I use when I run npm i .... If this is unacceptable to plugin developers, we could perhaps get plugin developers to add a friendlyName field to their package.json.

For the tags, will you now use the keywords listed in the package.json minus chai and chai-plugin? What happens to keywords like test, assertions, ... that are useful in the npm page but probably not on Chai's docs?

My plan is to filter out tags that are not relevant to us. So far this includes chai and chai-plugin, but I will certainly add test,assertions, probably also tests, testing, plugin, assert, unit, acceptance, and maybe spend some time talking with plugin developers about curating a nicer list of tags - for example some plugins have jquery while others have jQuery (capital Q).

How are you planning to display that plugin X does/does not have Node.js/browser support, and the file to use for browser?

Good question. I have been mulling this over and I'm not quite sure. Here are the options I've thought about:

• Browser plugins need to add a chai-plugin-browser keyword in addition to chai-plugin. This will probably be the interim solution we use.
• We could hook off of the browser key in the package.json. The package.json browser field links to the appropriate build file, e.g. { "browser": "chai-immutable.min.js" }. This helps bundlers like jspm bundle browser deps, and we could use it as a signal that the plugin supports browsers - so it's likely to already be needed.
• We could abuse the package.json's engine field - to see if package.json#engine.browser exists, we could also use it to get more detailed metrics - e.g. which user agents/browsers are supported, what versions, etc. However this is not a standard inside of the package.json.

P.S. support for browser and/or engines have been discussed as a codified way to handle browser packages within npm: npm/npm#4321.

I can see how easy it can be to overload Chai plugins page by adding the chai-plugin to npm projects. How are you planning to blacklist/remove these?

npm-plugin-fetcher supports blacklists - so if we need to we can blacklist modules from appearing on the plugins page. Its a low-key solution and hopefully we wont have to use it. I'd rather just have more stringent checks on a package.json - for example exclude packages that don't have a repository key, or packages with a README of less than some number of words, etc.

Of course, most of these questions assume you are keeping a 1:1 functionality with the current look of the plugins, so maybe it's irrelevant :-)

While I'd like to keep a reasonable degree of parity, I think we'll definitely not end up being 1:1. I mean, we're already not - considering plugin authors are having to change their package.json to accommodate the new system, but this, to me, is an acceptable tradeoff towards getting a simple, automated plugin system.

If you have any feedback on the above, I'd love to hear it.

[keithamus]

Going to settle on a package having the keywords of browser if they support browsers, and browser-only if they do not support node, only browsers.

---

The following plugins need the following keywords added to their package.json files. Any one who reads this who wants to make PRs to these projects, I'd ask them to do so.

As and when these are done, please let me know and I will tick them off of this list. If they are not done by the time the new website goes live, I'll try to get round to making PRs for the remainder, but it is important to know that the site will go live even if not every plugin has the requisite tags!

Plugins

• Chai-Spies (in chaijs org), (Update package.json keywords chai-spies#40)
  • chai-plugin
  • browser
  • mocks-and-spies
• Chai-HTTP (in chaijs org), (Update package.json keywords chai-http#47)
  • chai-plugin
  • http
• Sinon-Chai (by @domenic), (Update package.json keywords sinon-chai#72)
  • chai-plugin
  • browser
  • vendor
  • mocks-and-spies
• Chai as Promised (by @domenic), (Update package.json keywords chai-as-promised#129)
  • chai-plugin
  • browser
  • async
• Chai jQuery (in chaijs org), (Update package.json keywords chai-jquery#74)
  • chai-plugin
  • browser-only
  • vendor
  • dom
• Chai jQ (by @FormidableLabs / @ryan-roemer)
  • chai-plugin
  • browser
  • vendor
  • dom
• Chai DOM (by @nathanboktae)
  • chai-plugin
  • browser
  • vendor
• Chai Timers (in chaijs org), (Update package.json keywords chai-timers#2)
  • chai-plugin
  • browser
  • math
  • time
• Chai Stats (in chaijs org), (Update package.json keywords chai-stats#9)
  • chai-plugin
  • browser
  • math
• Chai Null (in chaijs org), (Update package.json keywords chai-null#1)
  • chai-plugin
  • browser
  • object-constructors
• Chai Factories (in chaijs org), (Update package.json keywords chai-factories#4)
  • chai-plugin
  • browser
  • object-constructors
• Chai JS Factories (by @solatis), (Update package.json keywords solatis/chai-js-factories#4)
  • chai-plugin
  • browser
  • object-constructors
• Chai Changes (by @matthijsgroen)
  • chai-plugin
  • browser
  • objects
• Chai Backbone (by @matthijsgroen)
  • chai-plugin
  • browser
  • vendor
  • dom
• Chai Fuzzy (by @elliotf), (Update package.json keywords elliotf/chai-fuzzy#15)
  • chai-plugin
  • browser
  • array
• Chai Things (in chaijs org)
  • chai-plugin
  • browser
  • array
• Chai Interface (by @jden), (Update package.json keywords junosuarez/chai-interface#2)
  • chai-plugin
  • browser
  • `objects'
• Chai Json-Schema (in chaijs org), (Update package.json keywords chai-json-schema#20)
  • chai-plugin
  • browser
  • `objects',
  • array
• Chai Json Equal (by @hurrymaplelad)
  • chai-plugin
  • browser
  • objects
  • array
  • equality
• Chai Date Time (by @gaslight), (Update package.json keywords mguterl/chai-datetime#27)
  • chai-plugin
  • objects
  • date
  • time
  • browser
• Chai FS (in chaijs org), (Update package.json keywords chai-fs#11)
  • chai-plugin
  • vendor
  • fs
• chai-webdriver (by @goodeggs)
  • chai-plugin
  • webdriver
  • integration
  • vendor
  • `dom'
• Chai Bookshelf (by @elliotf), (Update package.json keywords elliotf/chai-bookshelf#1)
  • chai-plugin
  • array
• Chai Change (in chaijs org)
  • chai-plugin
  • browser
  • change
• Chai Shallow Deep Equal (by @michelsalib), (Update package.json keywords michelsalib/chai-shallow-deep-equal#12)
  • chai-plugin
  • browser
  • `objects'
• Chai Subset (by @e-conomic), (Update package.json keywords debitoor/chai-subset#24)
  • chai-plugin
  • objects
• Chai String (by @onechiporenko)
  • chai-plugin
  • browser
  • string
• Chai Param (by @theblacksmith / @svallory)
  • chai-plugin
  • guard
  • parameter
• Chai Colors (by @hurrymaplelad)
  • chai-plugin
  • colors
• Chai XML (by @krampstudio)
  • chai-plugin
  • xml
• ChaID (by @hurrymaplelad)
  • chai-plugin
  • id
  • objectid
  • equality
• Chai Immutable (by @astorije)
  • chai-plugin
  • browser
  • immutable
  • equality
• Chai Signals (by @StephenDavidson), (Update package.json keywords StephenDavidson/chai-signals#1)
  • chai-plugin
  • browser
  • js-signals
  • signals
• Chai Match (by @johngeorgewright)
  • chai-plugin
  • browser
  • string
• Chai BigNumber (by @asmarques)
  • chai-plugin
  • browser
  • math

---

Note To Plugin Authors

You've been @mentioned because you have a chai plugin. Firstly thank you for being awesome! Secondly, sorry for the potentially unsolicited emails/notifications! Thirdly - in the above list - you'll find your plugin(s) and some keywords that I would like you to add to your package.json. You may also expect a PR adding these in from one of our contributors, or me if I get round to doing it before you. If you're wondering why I want you to add these keywords, here is the reasoning for each one:

• chai-plugin will let us index your plugin on the next version of the website. If you don't add this to your package.json, you wont be listed on our plugins page.
• browser will let us know that your plugin supports the browser. Please don't add this keyword if you don't support the browser. If you don't add it we'll assume the plugin does not support browsers.
• browser-only will let us know that you don't support Node/iojs. Please don't add this keyword if you do actually support Node. If you don't add it we'll assume you support node.
• Any other keyword listed below has been listed there, because it was in our plugins.js file which is soon to be removed. If you don't want these keywords in your package.json, that's fine, but seeing as they're in our plugins.js I thought it'd be good measure to give you a chance to add them into your package.json. We'll use them to help filter our plugins page, so it is advisable to add them.

If you edit your package.json, please drop me a note here so I can cross your plugin off of the list. Thank you for your time with this, it is much appreciated, and sorry if this caused you any inconvenience.

[krampstudio]

Here we go https://www.npmjs.com/package/chai-xml

[matthijsgroen]

👍

[keithamus]

@krampstudio thanks for updating chai-xml. Feel free to hit the unsubscribe button on this thread - as it'll probably get pretty messy in here. If you have any free time and fancy making PRs for the other repos it'd be huuugely appreciated 😄

[johngeorgewright]

@keithamus keywords updates and published

[keithamus]

@johngeorgewright hi-hive ✋! Feel free to hit the unsubscribe button on this thread - as it'll probably get pretty messy in here. If you have any free time and fancy making PRs for the other repos it'd be huuugely appreciated 😄

[matthijsgroen]

@keithamus Chai-changes is updated, chai-backbone will be done later

[keithamus]

Thanks @matthijsgroen 😄. Let me know when you've done chai-backbone please.

[nathanboktae]

Awesome work here. 👍 chai-dom has been updated and published.

[keithamus]

Sweet, thanks @nathanboktae. FYI you might want to publish with the new keywords - chai-docs will read from the npm registry, not github. Actually same goes for chai-xml (@krampstudio), and chai-changes (@matthijsgroen)

[ryan-roemer]

@keithamus -- Added keywords and published as [email protected]. Thanks!

[keithamus]

@electricmonk I want to get the new site will be live before the end of the year, 3 weeks at most. Putting effort into updating the old site right now is not really worth it - part of the reason why we're rewriting the thing is because the old site is too much of a manual process, so to answer your question: unlikely.

[electricmonk]

@keithamus fair enough :)

[svallory]

chai-param updated!

[keithamus]

Hey y'all who are still following this thread; we have a working plugin page on the github pages site: https://chaijs.github.io/chai-docs/plugins/. The keywords are clickable and filterable, and the tiles go to individual plugin pages - the README data is fetched from your README.mds, the keywords from your package.jsons.

NOTICE

There are still a few outstanding packages - packages which do not include the chai-plugin keyword. If you are an author of one of those plugins, please note that you will need to merge the PR I have sent you, and release a new version to continue to have a place on Chai's plugin listing. Gradually, we will be removing plugins which do not have these keywords, so please add them to your packages!

[astorije]

Great work @keithamus! 👏

[keithamus]

I'm going to close this issue. The only outstanding plugins are the ones in chai core - which we will get around to as part of other refactoring and standardising work, and Chai Bookshelf (by @elliotf) which has an outstanding PR: elliotf/chai-bookshelf#1.

I'd say we can consider this pretty well done.