Jinja2 v3

[oliver-sanders]

Sibling to an upcoming cylc-flow PR switching from Jinja2 v2->v3.

Jinja2v2 used to support zero-prefixed integers e.g. 01, 02, 03, ...

Jinja2v3 dropped support for this, fair enough, however, our users haven't been getting warnings for this so haven't had time to adapt.

This PR patches the Jinja2 parser we use in cylc-rose to shave off the leading zero before we pass the template variable back through to Jinja2 itself. It's not a particularly nice solution, it is intended as a stopgap to help users to migrate with the intention that we would retire it ASAP, probably with the move to Jinja2 3.1.

Requirements check-list

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Applied any dependency changes to setup.cfg.
• Appropriate tests are included (unit and/or functional).
• Appropriate change log entry included.
• No documentation update required.
• Made Discourse post to raise awareness of this change - https://cylc.discourse.group/t/jinja2-upgrade-to-version-3/489
  (note post not yet public, waiting for the rc4 release first)

[oliver-sanders]

(note the tests will fail until cylc-flow moves to Jinja2 v3)

[oliver-sanders]

ATM this PR supports a single leading zero e.g. 01, 02, 03, ...

Looks like there is some usage of multiple leading zeros e.g. 001, 0002, 00003, ...

This diff should expand support to the latter cases:

diff --git a/cylc/rose/jinja2_parser.py b/cylc/rose/jinja2_parser.py
index 96cebae..9f95567 100644
--- a/cylc/rose/jinja2_parser.py
+++ b/cylc/rose/jinja2_parser.py
@@ -50,8 +50,11 @@ def _lexer_wrap(fcn):
                 and len(value_str) > 1
                 and value_str[0] == '0'
             ):
+                for ind, char in enumerate(value_str):
+                    if char != '0':
+                        break
                 instances.add(value_str)
-                value_str = value_str[1:]  # strip the leading zero
+                value_str = value_str[ind:]  # strip the leading zero
             yield (lineno, token, value_str)
 
     def _inner(

[MetRonnie]

That diff does the trick 👍

However, noticed the warning you get should be

 WARNING - Support for integers with leading zeros was dropped in Jinja2 v3. Rose will extend support until a future version.
     Please amend your Rose configuration files e.g:
      * 0500 => 500
      * 050 => 50
      * 0900 => 900
-     * 0001 => 001
+     * 0001 => 1
      * 0100 => 100

[oliver-sanders]

FYI tested zero patted integers with Jinja2 v2 under Python 2.7 and 3.7 to make sure they behaved the same which they did:

$ cat suite.rc
#!Jinja2

{% set number = 007 %}

# {{ number }}
# {{ number + 2 }}
# {{ number + 002 }}

{% set number = 0099 %}

# {{ number }}
# {{ number + 2 }}
# {{ number + 002 }}

$ cylc view . --stdout --process
# 7
# 9
# 9
# 99
# 101
# 101

(no octal wierdness)

Jinja2 v3 octals take the same format as Python 3 octals.

Adding the patch (with a fix for the warnings)

[MetRonnie]

Also this sentence is a bit difficult to parse, I would suggest something like

Suggested change

	Support integers with leading zeros (e.g `001`) to back support Rose
	configurations for use with cylc-flow>=8.0rc4 which uses Jinja2 v3 which
	no longer supports this.
	(For use with cylc-flow>=8.0rc4 which uses Jinja2 v3)
	Maintain support for integers with leading zeros (e.g `001`)
	in Rose configurations. Note this support is deprecated and you
	should remove leading zeros.

[wxtim]

I think that 4 lines is too many for CHANGES.md