-
Notifications
You must be signed in to change notification settings - Fork 13
2021 10 22 Bruno's leaving notes
Hi,
This is a page with a list the activities I was working when I left NIWA on 2021/10/22. (N.B. “left NIWA”, I will definitely still bother you guys in the Cylc project, but via Open Source contributions.)
This page is being updated from the day it was first created (2021 10 05) until my last day.
I prefer to use an IDE. You might be able to use a text editor (e.g. vim), or a more robust solution like VSCode or Atom. You can get pretty close to what an IDE offers with vim or VSCode by using plug-ins.
The problem is that, at least for JavaScript and TypeScript development, you might find articles with fixes for issues (e.g. wrong code highlight, or debugger showing wrong lines) that only work for an IDE. The most famous web IDE is, IMHO, WebStorm. Followed by VScode + plug-ins and some custom configuration for debugging.
Also, even though I prefer Eclipse, I tried to use whatever IDE the team was using. So if others are familiar with VSCode, it might be better to use that, rather than using WebStorm. Eclipse works too, with the WDT extension, but I found it slower than WebStorm, and the Webpack integration was really lacking. Although debugging worked fine.
From my past experience with Cylc UI:
- 1 big build problem every 6 months (or twice a year), due to libraries being updated like Vuetify that broke our build, or due to a new version of Yarn, etc.
- 1 UIS issue every 2 months or so; I feel like I would be bothering David Sutherland and others every other month, due to some functionality of the UIS that was not thoroughly tested yet, or that was new and needed some further work. This is normal and expected, but the issue is that the UI developer might have to debug and troubleshoot the UI first, making sure the issue is not in our code base or in Vue or with some library. These issues normally take between 1 and 7 days to get sorted out, and there is normally some pair-work with the UIS devs (a great opportunity to learn!)
- 1 library/issue every 6 months due to a missing update of a transitive dependency; e.g. a new version of
sassis out, but break because it needsvue-clito update too.
I kept track of Vue 3 since it was first announced (when they threatened to drop the HTML-style templates). From what I gathered so far, we should be able to keep our single-page-components (i.e. a .vue file with <template>, <script>, <style>). There are, however, some changes that we will need to make regarding things like v-model and other code that is deprecated or that is not supported now.
I kept updating the issue with notes for things that we need to take care when updating, as well as references like blog posts and reddit threads from people that moved from Vue 2 to Vue 3: https://github.com/cylc/cylc-ui/issues/411
At the time of writing, the main blocker for Cylc UI to be upgraded to Vue 3 is Vuetify. They initially planned to release the new version supporting Vue 3 much earlier, then kept pushing it for later as the work changed (either due to Vue 3 changes and bugs, or due to normal work increase to upgrade Vuetify). The last change was from November 2021 to Q1 2022. One possibility would be to drop Vuetify, but we would either have to use another supported UI framework like Bootstrap or Tailwind, or write our own UI styles and components.
I created the Vuepress pull request (now closed) to create a developer-oriented documentation. Setting it up was a bit challenging. Then, after it was working, adding the first components like Job and Task showed that we had other issues, like having to define what is transpiled for Vuepress (even though we are already doing that for Vue Cli and Babel), and having to further configure Vuepress to mimic what we already have in Cylc UI configuration.
Apparently it's a known issue, and with no workarounds. You basically duplicate your configuration. If you have a very small application, that shouldn't be a problem. But if you have a more complex application, or if you use a complex library (e.g. Vuetify) it can get tricky to stay up to date to all the changes that you need to keep in sync in your application and in Vuepress.
Maybe there is some way to tell Vuepress “use the existing configuration or else…”, but I haven't found such solution. I closed the pull request, and the changes are stashed in this branch, should another developer would like to restart the work: https://github.com/cylc/cylc-ui/tree/vuepress
This is a short note, but that may help others trying to debug unit tests. In WebStorm I couldn't get the debugger to work for a long time. Then after searching over StackOverflow and Vue.js forums, I found someone saying s/he fixed it by changing how sourcemaps were generated.
And that indeed fixed my issue, however, I had to modify a setting that could affect other parts of the build (namely coverage). I left a note where you need to change it, but if someone ever finds a definitive solution, that doesn't require us to manually edit and save that file before running the unit tests with the debugger, please let me know: https://github.com/cylc/cylc-ui/blob/dfb7d291f8379358644f6d84d5de88696cf339ce/vue.config.js#L78-L80
The GScan component is an entry of a Vuetify VList. This list is a child of the Drawer component that came with the Creative Tim design we used for bootstrapping the UI.
If you look at the Design issue, https://github.com/cylc/cylc-ui/issues/87, you will see that there are still some parts missing in GScan, like the location of filters, and the toolbar for GScan.
Also, initially we had a very simple flat GScan component. But with workflows supporting hierarchical names such as research/lab/workflowA/run1, we started moving hierarchy into GScan. At the moment of writing, GScan is using a computed property to create the hierarchy, which is not really efficient.
In this pull request, https://github.com/cylc/cylc-ui/pull/736, we have two features being added:
- use an approach similar to the Tree component, building a lookup for easy access, and a hierarchical tree;
- propagate states in the tree (latest task states)
The first feature, the hierarchical tree and updates from deltas is done and working. The second, however, is not performing well enough. This is where I will focus for the next two weeks.
The Tree component and view were the one that received more attention, as they were included in the initial versions of the UI.
However, there are still things pending, that are logged in issues, but will include them here anyway:
- If you look at the Design issue, https://github.com/cylc/cylc-ui/issues/87, there are still a few parts of layout & functionality missing.
- The filters work well, but it could be improved how you pass filters to certain components.
GScanandTreeboth have to specify computed variables for filtering. It would be nicer if we could pass props (like functions?) instead, reducing the code in the caller code. - It would be great to create a Tree component not only from a GraphQL initial data burst, but also from the existing data in the Vuex store. See the section below about the Table.
- In the past we tried flattening the tree, and using a virtual scroller. That almost worked. At the moment we have recursion, which doesn't play well with Vue's reactivity and DOM/VDOM (reflow, and also preventing some elements from being updated). However, in theory, we should be able to construct a flat tree, and then just add elements in the sorted position. That would be a huge performance improvement.
The Table View is inspired by the Cylc 7 view with the same name. I recommend taking a look at the Cylc 7 Table View before jumping into the issue, if not already familiar with the view.
In Cylc 8, Giuliano Serrao contributed the Table View HTML layout in his pull request: https://github.com/cylc/cylc-ui/pull/672
The pull request is still under review not due to layout issues. When Giuliano finished his work, it was going to be the second view added to the Workflow view (where we have tabs).
After that view was added, it highlighted some structural problems we had in how we handled views and GraphQL subscriptions. We fixed most of these issues, and starter reviewing it again.
This time, the review failed due to missing features from the Cylc 7, like sorting, or choosing which fields are displayed in the table. As well as a missing Views & GraphQL subscription issue: when you add a second view, that re-uses an existing GraphQL subscription, the view doesn't receive an initial data burst.
For that, we can either restart the GraphQL subscription or change the Tree View and Table View to be able to be created using the existing data from the Vuex store.
There are definitely a lot that needs to be done (or fixed) in Cylc UI. If you find anything broken, it will be 99.99% likely that it is directly my fault (and another 0.01% that it is indirectly my fault). Feel free to modify the source code, undo things that I started or decided. Don't be afraid to change or experiment new things, no hard feelings 🙂
I used WebStorm for development, as it has good integration with WebPack and Vue. I did not need to configure anything besides the normal things like location of webpack configuration (in node_modules, search for vue-cli-services…) and even supports debugging. If you want to debug Cylc UI, you will find more about it here.
While there are certain layers in the project, it is really easy to accidentally bypass these layers, mixing things like a GraphQL subscription with Vue template, or a Cylc UI View with a Vuex store action. To avoid mistakes like this, that lead to maintenance issues in the future, I thought about making the project a monorepo with multiple packages/modules, https://github.com/cylc/cylc-ui/issues/412. Feel free to explore this idea if you would like, or perhaps try something different. My idea was to have a module for the Tree View, one for the Table View, one for GraphQL Workflow Service, maybe some common module, and so it goes.
Finally, avoid adding too many dependencies. Really think well before adding a dependency, no matter how small it is. Check if Vue, Lodash, Vuetify, etc, already not provide it. At the moment of writing, we have 88 libraries (dev and build):
kinow@ranma:~/Development/python/workspace/cylc-ui$ yarn info --name-only | wc -l
88And 1890 in total including transitive dependencies:
kinow@ranma:~/Development/python/workspace/cylc-ui$ yarn info --name-only --recursive | wc -l
1890If you compare that with Cylc UIS (135) and with Cylc Flow (83) that might show you why dependencies are a big deal in the UI. There have been multiple issues (performance, regressions, browser compatibility, security, etc) in the past due to dependency hell in JS, so take care 😵 💫
And if you ever need anything, ping me on Twitter or GitHub (kinow).
Thanks for all the 🐟 🖖