Add the Kedro RichHandler #2592

noklam · 2023-05-19T16:42:38Z

NOTE: Kedro datasets are moving from kedro.extras.datasets to a separate kedro-datasets package in
kedro-plugins repository. Any changes to the dataset implementations
should be done by opening a pull request in that repository.

Description

Fix #2277

Introduce our own rich console logging handler #2277

It should also open up the solutions for these logging issues

This will enable users to customise rich logging and optionally disable the rich's traceback if needed (Some platform doesn't play very well with it, e.g. Databricks and we have some logic to disable it by default)

Bonus: It now also supports more arguments that is compatible with RichHandler and rich.traceback.install

Questions for Reviewer

Where should we put the logger? We have kedro.extras.logging but we plan to remove this namespace, it is already removed in dev. Do we need to put it in kedro.extras.logging and move it to kedro.logging in 0.19.0 or just do it now since it's not breaking?
If some error happen before the log was set up it won't be rich-formatted properly, i.e. invalid RichHandler. I think it's fine but worth to mention.
I did some extra work to map arguments between RichHandler to rich.traceback.install to maximize the flexibility (Tests are not written yet, but if we agree this is useful then I will go on to write them). For example tracebacks_show_locals is mapped as show_locals for rich.pretty.install

Note: I haven't created unit tests since there are many decisions about what should be enable/disable, I will add the test after the 1st round of review.

Development notes

Add some unit tests (test rich is enable by default, test the option to disable rich, and test all known available arguments of RichHandler)
Check if we can simplify the test, we manually delete sys.modules for testing _ProjectLogging, it may be easier now because this is configurable we don't have to reload module.
Add changes to all logging.yml (template)
Add changes to all starters logging.yml (starters)
Add documentation about the changes
Implementation of RichLogger

Checklist

Manual Testing

Run kedro new -s pandas-iris
Edit conf/base/logging.yml

  rich:
    class: kedro.extras.logging.RichHandler
    # class: rich.logging.RichHandler
    rich_tracebacks: True
   # tracebacks_show_locals: True

kedro run

Read the contributing guidelines
Opened this PR as a 'Draft Pull Request' if it is work-in-progress
Updated the documentation to reflect the code changes
Added a description of this change in the RELEASE.md file
Added tests to cover my changes

Note:

Currently user need to set KEDRO_LOGGING_CONFIG to disable rich's traceback, because the default logging installed it and there is no way to uninstall it.

We need to solve this by either implementing it ourselves or raise a PR to rich

Signed-off-by: Nok Chan <[email protected]>

…handler

Signed-off-by: Nok Chan <[email protected]>

…RichHandler Signed-off-by: Nok Chan <[email protected]>

Signed-off-by: Nok Chan <[email protected]>

noklam · 2023-05-23T15:30:15Z

features/steps/test_starter/{{ cookiecutter.repo_name }}/conf/base/logging.yml

-    class: rich.logging.RichHandler
+    class: kedro.extras.logging.RichHandler
+    rich_tracebacks: True
+    # tracebacks_show_locals: False


Like settings.py, we could have some commented default here to help user discover possible options. I put show_locals because initially when we introduce rich, we want to enable it as True. It turns out to be a bit noisy for some users, and there was no good way to turn it off by the user.

I like this idea!

I also like the idea, but remember this file won't exist in the default template in 0.19 and it will move to docs instead - see #2426 (this was a relatively recent decision in tech design: #2281 (comment)).

Also note you changed the test_starter here, not the actual kedro template starter.

Signed-off-by: Nok Chan <[email protected]>

noklam · 2023-05-23T15:39:14Z

kedro/extras/logging/rich_logger.py

+        # Mapping Arguments from RichHandler's Constructor to rich.traceback.install
+        for key, value in kwargs.items():
+            prefix = "tracebacks_"
+            if key.startswith(prefix):
+                key_prefix_removed = key[len(prefix) :]
+                if key_prefix_removed == "suppress":
+                    mapped_kwargs[key_prefix_removed] += value
+                else:
+                    mapped_kwargs[key_prefix_removed] = value
+            elif key in ("locals_max_length", "locals_max_string"):
+                mapped_kwargs[key] = value
+


The original issue propose to support self.rich_tracebacks only, I take it further because it is possible to support most of the arguments as long as they both exists in RichHandler and rich.traceback.install, the signature are slightly different so we need to map it.

noklam · 2023-05-23T15:41:00Z

kedro/extras/logging/rich_logger.py

+        if self.rich_tracebacks:
+            if "DATABRICKS_RUNTIME_VERSION" not in os.environ:
+                # https://rich.readthedocs.io/en/stable/reference/logging.html?highlight=rich%20handler#rich.logging.RichHandler
+                # Support compatible arguments between RichHandler and rich.traceback.install
+                rich.traceback.install(**mapped_kwargs)


Alternatively, we can also remove the special logic about Databricks but just change the default to False in the starter because it is now customisable.

I didn't do it because it's still a breaking change, and there's no point to use Rich's traceback on Databricks.

I would rather keep it as you have it, so the default is True and we include the special logic that disables it on Databricks.

noklam · 2023-05-23T15:42:12Z

kedro/framework/project/__init__.py

-        logging.captureWarnings(True)
-
-        # We suppress click here to hide tracebacks related to it conversely,
-        # kedro is not suppressed to show its tracebacks for easier debugging.
-        # sys.executable is used to get the kedro executable path to hide the
-        # top level traceback.
-        # Rich traceback handling does not work on databricks. Hopefully this will be
-        # fixed on their side at some point, but until then we disable it.
-        # See https://github.com/Textualize/rich/issues/2455
-        if "DATABRICKS_RUNTIME_VERSION" not in os.environ:
-            rich.traceback.install(suppress=[click, str(Path(sys.executable).parent)])
-        rich.pretty.install()


The rich's traceback won't be installed when kedro.framework.project is imported, it's now just a logging handler. This is an improvement because importing a module no longer has side-effect.

docs/source/logging/logging.md

Signed-off-by: Nok Chan <[email protected]>

merelcht

To answer your questions:

Where should we put the logger? We have kedro.extras.logging but we plan to remove this namespace, it is already removed in dev. Do we need to put it in kedro.extras.logging and move it to kedro.logging in 0.19.0 or just do it now since it's not breaking?

I would just put it in kedro.logging straightaway, so we don't need to move it again for the 0.19.0 release.

If some error happen before the log was set up it won't be rich-formatted properly, i.e. invalid RichHandler. I think it's fine but worth to mention.

That makes sense to me and I don't think there's a way around that.

I did some extra work to map arguments between RichHandler to rich.traceback.install to maximize the flexibility (Tests are not written yet, but if we agree this is useful then I will go on to write them). For example tracebacks_show_locals is mapped as show_locals for rich.pretty.install

I don't know that much about Rich so I'm not entirely sure yet what this means, but it sounds good 😄 But it would be good to get @antonymilne 's view on this.

merelcht · 2023-05-24T08:52:11Z

features/steps/test_starter/{{ cookiecutter.repo_name }}/conf/base/logging.yml

-    class: rich.logging.RichHandler
+    class: kedro.extras.logging.RichHandler
+    rich_tracebacks: True
+    # tracebacks_show_locals: False


I like this idea!

antonymilne · 2023-05-24T09:55:01Z

Let me start by answering the questions and will then review the code.

Where should we put the logger? We have kedro.extras.logging but we plan to remove this namespace, it is already removed in dev. Do we need to put it in kedro.extras.logging and move it to kedro.logging in 0.19.0 or just do it now since it's not breaking?

Agree with @merelcht that we should just put it in kedro.logging to begin with. Note there's no need to create a subpackage for it - just a single file logging.py should be good.

If some error happen before the log was set up it won't be rich-formatted properly, i.e. invalid RichHandler. I think it's fine but worth to mention.

This is unavoidable really and should be fine. Users will be able to fix it using the KEDRO_LOGGING_CONFIG environment variable to point to a different configuration anyway.

I did some extra work to map arguments between RichHandler to rich.traceback.install to maximize the flexibility (Tests are not written yet, but if we agree this is useful then I will go on to write them). For example tracebacks_show_locals is mapped as show_locals for rich.pretty.install

I think this is a good idea 👍

Add some unit tests (test rich is enable by default, test the option to disable rich, and test all known available arguments of RichHandler)

Probably you don't need to test all the known available arguments - just one of them should be fine.

antonymilne

Looks good in general! I will try it out but basically I think you're on the right track here and should go ahead and write tests. The only big thing is to put the code in kedro/logging.py instead of in extras.

Edit: tried it out and all seems to work as expected 👍 Note you need to set KEDRO_LOGGING_CONFIG=conf/base/logging.yml to really test this properly at the moment. Again, things will be waaaaay simpler soon when we've done #2426.

antonymilne · 2023-05-24T09:59:35Z

features/steps/test_starter/{{ cookiecutter.repo_name }}/conf/base/logging.yml

-    class: rich.logging.RichHandler
+    class: kedro.extras.logging.RichHandler
+    rich_tracebacks: True
+    # tracebacks_show_locals: False


I also like the idea, but remember this file won't exist in the default template in 0.19 and it will move to docs instead - see #2426 (this was a relatively recent decision in tech design: #2281 (comment)).

antonymilne · 2023-05-24T10:01:54Z

features/steps/test_starter/{{ cookiecutter.repo_name }}/conf/base/logging.yml

-    class: rich.logging.RichHandler
+    class: kedro.extras.logging.RichHandler
+    rich_tracebacks: True
+    # tracebacks_show_locals: False


Also note you changed the test_starter here, not the actual kedro template starter.

antonymilne · 2023-05-24T10:03:25Z

kedro/extras/logging/rich_logger.py

+    """Identical to rich's logging handler but with a few extra behaviours:
+    * warnings issued by the `warnings` module are redirected to logging
+    * pretty printing is enabled on the Python REPL (including IPython and Jupyter)
+    * all tracebacks are handled by rich when rich_tracebacks=True


We should have anoter point here to describe the extra rich.install behaviour you added.

antonymilne · 2023-05-24T10:04:29Z

kedro/extras/logging/rich_logger.py

+        if self.rich_tracebacks:
+            if "DATABRICKS_RUNTIME_VERSION" not in os.environ:
+                # https://rich.readthedocs.io/en/stable/reference/logging.html?highlight=rich%20handler#rich.logging.RichHandler
+                # Support compatible arguments between RichHandler and rich.traceback.install
+                rich.traceback.install(**mapped_kwargs)


I would rather keep it as you have it, so the default is True and we include the special logic that disables it on Databricks.

kedro/extras/logging/rich_logger.py

antonymilne · 2023-05-24T10:05:30Z

kedro/extras/logging/rich_logger.py

+        # Rich traceback handling does not work on databricks. Hopefully this will be
+        # fixed on their side at some point, but until then we disable it.
+        # See https://github.com/Textualize/rich/issues/2455


This comment should be moved down to where it belongs.

kedro/extras/logging/rich_logger.py

antonymilne · 2023-05-24T10:09:02Z

kedro/extras/logging/rich_logger.py

+                    mapped_kwargs[key_prefix_removed] += value
+                else:
+                    mapped_kwargs[key_prefix_removed] = value
+            elif key in ("locals_max_length", "locals_max_string"):


Is the set of arguments that are the same for rich.logging.RichHandler and rich.tracebacks.install? Might be worth a comment explaining why they're here,

kedro/extras/logging/rich_logger.py

antonymilne · 2023-05-24T10:12:33Z

docs/source/logging/logging.md

+
+Kedro's `kedro.extras.logging.RichHandler` is a subclass of [`rich.logging.RichHandler`](https://rich.readthedocs.io/en/stable/reference/logging.html#rich.logging.RichHandler) and supports the same set of arguments. By default, `rich_tracebacks` is set to `True` to use `rich` to render exceptions. However, you can disable it by setting `rich_tracebacks: False`.
+
+When `rich_tracebacks` is set to `True`, the configuration is propagated to [`rich.traceback.install`](https://rich.readthedocs.io/en/stable/reference/traceback.html#rich.traceback.install). If an argument is compatible with `rich.pretty.install`, it will be passed to the traceback's settings.


Suggested change

When `rich_tracebacks` is set to `True`, the configuration is propagated to [`rich.traceback.install`](https://rich.readthedocs.io/en/stable/reference/traceback.html#rich.traceback.install). If an argument is compatible with `rich.pretty.install`, it will be passed to the traceback's settings.

When `rich_tracebacks` is set to `True`, the configuration is propagated to [`rich.traceback.install`](https://rich.readthedocs.io/en/stable/reference/traceback.html#rich.traceback.install). If an argument is compatible with `rich.traceback.install`, it will be passed to the traceback's settings.

Unless I misunderstood something?

My bad! I notice this yesterday but the changes are still in my local branch, good catch!

Co-authored-by: Antony Milne <[email protected]>

…handler

Signed-off-by: Nok Chan <[email protected]>

…handler

Signed-off-by: Nok Chan <[email protected]>

astrojuanlu · 2023-05-30T14:15:23Z

docs/source/kedro.logging.RichHandler.rst

+.. autoclass:: RichHandler
+   :members:
+   :undoc-members:
+   :inherited-members:


Maybe we'll have to remove the :undoc-members: and :inherited-members:

Thank you! This seems to work. Sphinx is a puzzle, I cannot reproduce the same error in my local environment, but I am glad it's fixed now.

I can't believe it worked 😂 I guess I'm a Sphinx black belt now 🥷

Signed-off-by: Nok Chan <[email protected]>

* Remove unnecessary files and subheaders in documentation (#2563) Tidying up. * Leverage PEP-585 (#2540) Signed-off-by: Merel Theisen <[email protected]> Co-authored-by: Jannic <[email protected]> * [READY] Bring deployment docs up-to-date and add new pages for additional targets (#2557) Ready at last. Still some conversation to be had about how we reflect "confidence" in the plugins we describe (what version they were last tested against) but the text is ready and better than what's there currently, so let's go! * Remove mentions to `.egg` and stop producing them from `kedro package` (#2568) * Stop producing `.egg` files from `kedro package` Fix gh-2273. Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> * Upgrade import-linter Fix gh-2570. Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> * Partially remove reliance on setup.py for micropackaging Incomplete effort, see gh-2414. Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> * Revert "Partially remove reliance on setup.py for micropackaging" This reverts commit fd3efa8. Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> --------- Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> * OmegaConfigLoader returns `Dict` instead of `DictConfig`, resolves runtime_params properly (#2467) * Fix typehint Signed-off-by: Nok Chan <[email protected]> * test push Signed-off-by: Nok Chan <[email protected]> * POC of fix to solve the runtime param resolution problem Signed-off-by: Nok Chan <[email protected]> * Fix OmegaConfigLoadaer - resolve runtime_params early Signed-off-by: Nok Chan <[email protected]> * Delegate the intialization of runtime_params to AbstractConfigLoader Signed-off-by: Nok Chan <[email protected]> * Add test for interpolated value and globals Signed-off-by: Nok Chan <[email protected]> * add more test and linting Signed-off-by: Nok Chan <[email protected]> * refactor Signed-off-by: Nok Chan <[email protected]> * update release note Signed-off-by: Nok Chan <[email protected]> * Apply comments and refactor the test Signed-off-by: Nok Chan <[email protected]> * Update RELEASE.md Co-authored-by: Antony Milne <[email protected]> --------- Signed-off-by: Nok Chan <[email protected]> Co-authored-by: Antony Milne <[email protected]> * Revert special logic handling DictConfig (#2582) * Partially revert #2378 as OCL no longer returning DictConfig Signed-off-by: Nok Chan <[email protected]> * Revert #2176 with special logging handling Signed-off-by: Nok Chan <[email protected]> * remove unused import Signed-off-by: Nok Chan <[email protected]> --------- Signed-off-by: Nok Chan <[email protected]> * Re-title and modify introduction to databricks testing guide in contribution section (#2579) * modify databricks testing guide for contributors Signed-off-by: Jo Stichbury <[email protected]> * Update docs/source/contribution/development_for_databricks.md Co-authored-by: Jannic <[email protected]> * Update development_for_databricks.md --------- Signed-off-by: Jo Stichbury <[email protected]> Co-authored-by: Jannic <[email protected]> * Quick fix .gitpod.yml (#2585) * `OmegaConfigLoader` should not show `MissingConfigException` when the file is empty (#2584) * Allow empty config files Signed-off-by: Ankita Katiyar <[email protected]> * Remove condition for empty config Signed-off-by: Ankita Katiyar <[email protected]> * Revert "Remove condition for empty config" This reverts commit 7961155. * Update release notes Signed-off-by: Ankita Katiyar <[email protected]> * Add unit test Signed-off-by: Ankita Katiyar <[email protected]> --------- Signed-off-by: Ankita Katiyar <[email protected]> * Proposal of updated settings.py (#2587) * Proposal of updated settings.py Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> * Update settings after comment Signed-off-by: Nok <[email protected]> --------- Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> Signed-off-by: Nok <[email protected]> * Collaborative Experiment Tracking Docs (#2589) * Add metadata attribute to `kedro.io` datasets (#2537) * Add metadata attribute to kedro.io datasets Signed-off-by: Ahdra Merali <[email protected]> * Update specs in pipeline hooks docstrings (#2598) * Update pipeline hooks docstrings Signed-off-by: Tomas Van Pottelbergh <[email protected]> * Acknowledge community contribution --------- Signed-off-by: Tomas Van Pottelbergh <[email protected]> Co-authored-by: Jo Stichbury <[email protected]> Co-authored-by: Juan Luis Cano Rodríguez <[email protected]> * Miscellaneous hooks improvements (#2596) * Add debug level logging for hook execution order Fix gh-1942. Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> * Clarify role of setuptools in hooks Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> * Remove mentions to outdated and unused `pkg_resources` Fix gh-2391. Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> * Adapt example hook to spec signature Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> * Grammar and style fixes in docs Co-authored-by: Jo Stichbury <[email protected]> Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> * Revert "Adapt example hook to spec signature" This reverts commit ac1b495. Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> --------- Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> Co-authored-by: Jo Stichbury <[email protected]> * Minor modification to titles of deployment docs to replace "deprecated" with "legacy" (#2606) * FIX: Typo in markdown for kedro notebooks docs (#2610) * FIX: Typo in markdown for kedro notebooks docs Signed-off-by: debugger24 <[email protected]> * Acknowledge community contribution --------- Signed-off-by: debugger24 <[email protected]> Co-authored-by: Juan Luis Cano Rodríguez <[email protected]> * Add the Kedro RichHandler (#2592) * Add the Kedro RichHandler Signed-off-by: Nok Chan <[email protected]> * Clean up imports and comments Signed-off-by: Nok Chan <[email protected]> * Map arguments to add flexibility to customize arguments available in RichHandler Signed-off-by: Nok Chan <[email protected]> * Clean up rich_logger.py and unused imports Signed-off-by: Nok Chan <[email protected]> * Add docs - first iteration Signed-off-by: Nok Chan <[email protected]> * Fix imports and small refactoring Signed-off-by: Nok Chan <[email protected]> * Improve the doc Signed-off-by: Nok Chan <[email protected]> * Update the project's template Signed-off-by: Nok Chan <[email protected]> * format docstring Signed-off-by: Nok Chan <[email protected]> * Apply Jo's comment on the doc Signed-off-by: Nok Chan <[email protected]> * Fix the link with RichHandler Signed-off-by: Nok Chan <[email protected]> * Apply suggestions from code review Co-authored-by: Antony Milne <[email protected]> * Fix broken doc Signed-off-by: Nok Chan <[email protected]> * update template defaults Signed-off-by: Nok Chan <[email protected]> * refactoring RichHandler and improve doc Signed-off-by: Nok Chan <[email protected]> * Fix broken test due to new default RichHandler Signed-off-by: Nok Chan <[email protected]> * Fix doc Signed-off-by: Nok Chan <[email protected]> * Move kedro.extras.logging.RichHandler -> kedro.logging.RichHandler Signed-off-by: Nok Chan <[email protected]> * add kedro/logging.py and remove the unnecessary import that cause namespace collision Signed-off-by: Nok Chan <[email protected]> * Refactor test Signed-off-by: Nok Chan <[email protected]> * Add more tests Signed-off-by: Nok Chan <[email protected]> * Fix lots of import, docstring Signed-off-by: Nok Chan <[email protected]> * Reorder docstring Signed-off-by: Nok Chan <[email protected]> * Fix tests Signed-off-by: Nok Chan <[email protected]> * Link options to doc Signed-off-by: Nok Chan <[email protected]> * Clean up doc and unused logger Signed-off-by: Nok Chan <[email protected]> * Fix spelling `customise` Signed-off-by: Nok Chan <[email protected]> * Refactor test with fixture Signed-off-by: Nok Chan <[email protected]> * Remove unnecessary copy() because it is now a fixture Signed-off-by: Nok Chan <[email protected]> * Split up test_rich_traceback_configuration() to smaller tests Signed-off-by: Nok Chan <[email protected]> * Add RELEASE.md Signed-off-by: Nok Chan <[email protected]> * Update doc Signed-off-by: Nok Chan <[email protected]> * Fix template logging * Mention awesome-kedro in docs (#2620) * Mention awesome-kedro in docs See kedro-org/kedro-devrel#40 * Update CONTRIBUTING.md * Spelling and style fixes Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> Co-authored-by: Jo Stichbury <[email protected]> --------- Co-authored-by: Jo Stichbury <[email protected]> * Update SnowparkTableDataSet docs (#2485) * Update SnowparkTableDataSet docs * Add missing snowpark dependencies for docs building Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> --------- Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> Co-authored-by: Juan Luis Cano Rodríguez <[email protected]> * Release 0.18.9 (#2619) * Update __init__.py * Update release note Signed-off-by: Nok <[email protected]> * Update Version Numbers across docs Signed-off-by: Nok <[email protected]> * Update RELEASE.md * Update Release Note Signed-off-by: Nok <[email protected]> --------- Signed-off-by: Nok <[email protected]> Co-authored-by: Jo Stichbury <[email protected]> * Add documentation for deploying packaged Kedro projects on Databricks (#2595) * Add deployment workflow page Signed-off-by: Jannic Holzer <[email protected]> * Add table of contents, entry point guide, data and conf upload guide Signed-off-by: Jannic Holzer <[email protected]> * Add detailed instructions for creating a job on Databricks Signed-off-by: Jannic Holzer <[email protected]> * Add images and automated deployment resources Signed-off-by: Jannic Holzer <[email protected]> * Remove use of 'allows', add summary Signed-off-by: Jannic Holzer <[email protected]> * Remove link to missing image Signed-off-by: Jannic Holzer <[email protected]> * Add deployment workflow to toctree Signed-off-by: Jannic Holzer <[email protected]> * Lint and fix missing link Signed-off-by: Jannic Holzer <[email protected]> * Minor style, syntax and grammar improvements Signed-off-by: Jannic Holzer <[email protected]> * Fixes for correctness during validation Signed-off-by: Jannic Holzer <[email protected]> * Add instructions for creating log output location Signed-off-by: Jannic Holzer <[email protected]> * Lint Signed-off-by: Jannic Holzer <[email protected]> * Lint databricks_run Signed-off-by: Jannic Holzer <[email protected]> * Minor wording change in reference to logs Signed-off-by: Jannic Holzer <[email protected]> * Modify reference to Pyspark-Iris Signed-off-by: Jannic Holzer <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Fix linter errors to enable docs build for inspection Signed-off-by: Jo Stichbury <[email protected]> * Update build-docs.sh * Fix broken link Signed-off-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Remove spurious word Signed-off-by: Jannic Holzer <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Add advantages subheading Signed-off-by: Jannic Holzer <[email protected]> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <[email protected]> * Add alternative ways to upload data to DBFS Signed-off-by: Jannic Holzer <[email protected]> * Move note on unpackaged config and data Signed-off-by: Jannic Holzer <[email protected]> * Fix broken links Signed-off-by: Jannic Holzer <[email protected]> * Move databricks back into deployment section Signed-off-by: Jo Stichbury <[email protected]> * Remove references to PySpark Iris (pyspark-iris) starter Signed-off-by: Jannic Holzer <[email protected]> * Graphics links fixes, revise titles Signed-off-by: Jo Stichbury <[email protected]> * Fix broken internal link Signed-off-by: Jo Stichbury <[email protected]> * Fix links broken by new folder Signed-off-by: Jo Stichbury <[email protected]> * Remove logs directory Signed-off-by: Jannic Holzer <[email protected]> * Update image of final job configuration Signed-off-by: Jannic Holzer <[email protected]> * Add full stops in list. Co-authored-by: Juan Luis Cano Rodríguez <[email protected]> * Fix conda environment name. Co-authored-by: Juan Luis Cano Rodríguez <[email protected]> * Modify wording and image for creating a new job cluster Signed-off-by: Jannic Holzer <[email protected]> * Modify wording in guide to create new job cluster Signed-off-by: Jannic Holzer <[email protected]> * Remove --upgrade option Co-authored-by: Juan Luis Cano Rodríguez <[email protected]> * Add both ways of creating a new job Signed-off-by: Jannic Holzer <[email protected]> --------- Signed-off-by: Jannic Holzer <[email protected]> Signed-off-by: Jo Stichbury <[email protected]> Co-authored-by: Jo Stichbury <[email protected]> Co-authored-by: Juan Luis Cano Rodríguez <[email protected]> * Update docs font family to Inter and swap the logo (#2611) * Update docs font family to Inter Signed-off-by: Tynan DeBold <[email protected]> * Match weights to previous Google font Signed-off-by: Tynan DeBold <[email protected]> * Update logo images; revert css formatting for easier PR review Signed-off-by: Tynan DeBold <[email protected]> * Update text styles Signed-off-by: Tynan DeBold <[email protected]> * Add trailing whitespace (?) Signed-off-by: Tynan DeBold <[email protected]> * Update Kedro image path Signed-off-by: Tynan DeBold <[email protected]> * Update font size Signed-off-by: Tynan DeBold <[email protected]> * update font to font-family: 'Inter', sans-serif; Signed-off-by: huongg <[email protected]> * testing with important for font-family Signed-off-by: huongg <[email protected]> * Make project name lowercase in docs Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> * revert back to kedro/develop Signed-off-by: huongg <[email protected]> --------- Signed-off-by: Tynan DeBold <[email protected]> Signed-off-by: huongg <[email protected]> Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> Co-authored-by: Huong Nguyen <[email protected]> Co-authored-by: huongg <[email protected]> Co-authored-by: Juan Luis Cano Rodríguez <[email protected]> * Enable variable interpolation for the catalog with `OmegaConfigLoader` (#2621) * Enable variable interpolation in catalog by escaping _ Signed-off-by: Merel Theisen <[email protected]> * Add archive link on docs home page (#2636) Signed-off-by: Jo Stichbury <[email protected]> * add ravi-kumar-pilla to tsc (#2629) Signed-off-by: ravi-kumar-pilla <[email protected]> Co-authored-by: Jo Stichbury <[email protected]> * Add file for search console verification (#2631) * add verification file and change to conf Signed-off-by: Jo Stichbury <[email protected]> * Fix conf.py so google file is moved to root of docs build Signed-off-by: Jo Stichbury <[email protected]> --------- Signed-off-by: Jo Stichbury <[email protected]> * Chore/update assets for docs (#2625) * Update docs font family to Inter Signed-off-by: Tynan DeBold <[email protected]> * Match weights to previous Google font Signed-off-by: Tynan DeBold <[email protected]> * Update logo images; revert css formatting for easier PR review Signed-off-by: Tynan DeBold <[email protected]> * Update text styles Signed-off-by: Tynan DeBold <[email protected]> * Add trailing whitespace (?) Signed-off-by: Tynan DeBold <[email protected]> * Update Kedro image path Signed-off-by: Tynan DeBold <[email protected]> * Update font size Signed-off-by: Tynan DeBold <[email protected]> * update font to font-family: 'Inter', sans-serif; Signed-off-by: huongg <[email protected]> * testing with important for font-family Signed-off-by: huongg <[email protected]> * update assets for experiment tracking, docs, pipeline, focus mode, matplotlib and plotly * update modular pipelines section Signed-off-by: huongg <[email protected]> * autoreload Signed-off-by: huongg <[email protected]> * reupload autoreload gif Signed-off-by: huongg <[email protected]> * revert back to develop branch Signed-off-by: huongg <[email protected]> * gif for ET comparison Signed-off-by: huongg <[email protected]> * remove jupyter notebook screenshot since its not used anywhere Signed-off-by: huongg <[email protected]> * Update databricks viz demo Signed-off-by: Jannic Holzer <[email protected]> * pipeline_show_metrics gif Signed-off-by: huongg <[email protected]> * update different gif format Signed-off-by: huongg <[email protected]> * point to main instead of develop for kedro-banner Signed-off-by: huongg <[email protected]> --------- Signed-off-by: Tynan DeBold <[email protected]> Signed-off-by: huongg <[email protected]> Signed-off-by: Jannic Holzer <[email protected]> Co-authored-by: Tynan DeBold <[email protected]> Co-authored-by: ravi-kumar-pilla <[email protected]> Co-authored-by: Jo Stichbury <[email protected]> Co-authored-by: Jannic Holzer <[email protected]> * Fix the level heading for collab exp tracking so it shows in sidebar (#2647) Signed-off-by: Jo Stichbury <[email protected]> * Update Kedro icons in docs with for rebrand (#2652) Signed-off-by: Tynan DeBold <[email protected]> * Modify logos (#2651) Signed-off-by: Jannic Holzer <[email protected]> * Fix lint Signed-off-by: Merel Theisen <[email protected]> * remove the subsection of kedro.extras.extension Signed-off-by: Nok <[email protected]> * Fix lint Signed-off-by: Merel Theisen <[email protected]> Signed-off-by: Nok <[email protected]> * remove the subsection of kedro.extras.extension Signed-off-by: Nok <[email protected]> --------- Signed-off-by: Merel Theisen <[email protected]> Signed-off-by: Juan Luis Cano Rodríguez <[email protected]> Signed-off-by: Nok Chan <[email protected]> Signed-off-by: Jo Stichbury <[email protected]> Signed-off-by: Ankita Katiyar <[email protected]> Signed-off-by: Nok <[email protected]> Signed-off-by: Ahdra Merali <[email protected]> Signed-off-by: Tomas Van Pottelbergh <[email protected]> Signed-off-by: debugger24 <[email protected]> Signed-off-by: Jannic Holzer <[email protected]> Signed-off-by: Tynan DeBold <[email protected]> Signed-off-by: huongg <[email protected]> Signed-off-by: ravi-kumar-pilla <[email protected]> Co-authored-by: Jo Stichbury <[email protected]> Co-authored-by: Merel Theisen <[email protected]> Co-authored-by: Jannic <[email protected]> Co-authored-by: Juan Luis Cano Rodríguez <[email protected]> Co-authored-by: Nok Lam Chan <[email protected]> Co-authored-by: Antony Milne <[email protected]> Co-authored-by: Ankita Katiyar <[email protected]> Co-authored-by: rashidakanchwala <[email protected]> Co-authored-by: Ahdra Merali <[email protected]> Co-authored-by: Tomas Van Pottelbergh <[email protected]> Co-authored-by: Rahul Kumar <[email protected]> Co-authored-by: Tynan DeBold <[email protected]> Co-authored-by: Huong Nguyen <[email protected]> Co-authored-by: huongg <[email protected]> Co-authored-by: Ravi Kumar Pilla <[email protected]> Co-authored-by: Jannic Holzer <[email protected]> Co-authored-by: Merel Theisen <[email protected]>

Add the Kedro RichHandler

db214c2

Signed-off-by: Nok Chan <[email protected]>

noklam linked an issue May 19, 2023 that may be closed by this pull request

Introduce our own rich console logging handler #2277

Closed

noklam added 8 commits May 19, 2023 17:44

Merge branch 'main' into 2277-introduce-our-own-rich-console-logging-…

d817420

…handler

Clean up imports and comments

6a31eff

Signed-off-by: Nok Chan <[email protected]>

Map arguments to add flexibility to customize arguments available in …

2b02951

…RichHandler Signed-off-by: Nok Chan <[email protected]>

Clean up rich_logger.py and unused imports

df7308e

Signed-off-by: Nok Chan <[email protected]>

Add docs - first iteration

c507984

Signed-off-by: Nok Chan <[email protected]>

Fix imports and small refactoring

3ba89c9

Signed-off-by: Nok Chan <[email protected]>

Improve the doc

0fb70c3

Signed-off-by: Nok Chan <[email protected]>

Update the project's template

2dbc0c0

Signed-off-by: Nok Chan <[email protected]>

noklam requested review from antonymilne, merelcht and stichbury May 23, 2023 15:23

noklam commented May 23, 2023

View reviewed changes

format docstring

05eaa91

Signed-off-by: Nok Chan <[email protected]>

noklam commented May 23, 2023

View reviewed changes

stichbury reviewed May 23, 2023

View reviewed changes

docs/source/logging/logging.md Outdated Show resolved Hide resolved

stichbury reviewed May 23, 2023

View reviewed changes

docs/source/logging/logging.md Outdated Show resolved Hide resolved

noklam added 2 commits May 23, 2023 17:02

Apply Jo's comment on the doc

7b9a989

Signed-off-by: Nok Chan <[email protected]>

Fix the link with RichHandler

8e531f0

Signed-off-by: Nok Chan <[email protected]>

merelcht reviewed May 24, 2023

View reviewed changes

antonymilne reviewed May 24, 2023

View reviewed changes

noklam and others added 4 commits May 24, 2023 11:43

Apply suggestions from code review

810d25b

Co-authored-by: Antony Milne <[email protected]>

Merge branch 'main' into 2277-introduce-our-own-rich-console-logging-…

fc0d519

…handler

Fix broken doc

1e5d14a

Signed-off-by: Nok Chan <[email protected]>

update template defaults

85ff28d

Signed-off-by: Nok Chan <[email protected]>

noklam added 8 commits May 25, 2023 13:29

Link options to doc

8b4f63a

Signed-off-by: Nok Chan <[email protected]>

Clean up doc and unused logger

0c166ad

Signed-off-by: Nok Chan <[email protected]>

Fix spelling customise

b81a5ea

Signed-off-by: Nok Chan <[email protected]>

Refactor test with fixture

8f2788a

Signed-off-by: Nok Chan <[email protected]>

Remove unnecessary copy() because it is now a fixture

f263931

Signed-off-by: Nok Chan <[email protected]>

Split up test_rich_traceback_configuration() to smaller tests

5071f5a

Signed-off-by: Nok Chan <[email protected]>

Add RELEASE.md

f9d5769

Signed-off-by: Nok Chan <[email protected]>

Update doc

df58139

Signed-off-by: Nok Chan <[email protected]>

noklam mentioned this pull request May 25, 2023

Implement our own uninstall() method for rich traceback or raise a PR upstream #2609

Closed

noklam added 4 commits May 25, 2023 17:54

Fix template logging

af4a9ce

Signed-off-by: Nok Chan <[email protected]>

Fix the. template

d123a3e

Signed-off-by: Nok Chan <[email protected]>

Lint

6ad4463

Signed-off-by: Nok Chan <[email protected]>

Merge branch 'main' into 2277-introduce-our-own-rich-console-logging-…

632d29e

…handler

noklam mentioned this pull request May 26, 2023

Update logging to use kedro.logging.RichHandler kedro-org/kedro-starters#130

Merged

3 tasks

noklam removed request for astrojuanlu, idanov and yetudada May 30, 2023 11:22

noklam added 4 commits May 30, 2023 12:40

Add doc to fix doc issues

b883906

Signed-off-by: Nok Chan <[email protected]>

fix doc

e749dca

Signed-off-by: Nok Chan <[email protected]>

Fix doc rst so it don't reference to the upstream class

c726d6e

Signed-off-by: Nok Chan <[email protected]>

Remvoe whitespace

af3e3d0

Signed-off-by: Nok Chan <[email protected]>

astrojuanlu reviewed May 30, 2023

View reviewed changes

noklam added 2 commits May 30, 2023 15:19

remove undoc member and inherited-member for RichHandler

40e9156

Signed-off-by: Nok Chan <[email protected]>

Please the linter once again

b7d94e6

Signed-off-by: Nok Chan <[email protected]>

noklam enabled auto-merge (squash) May 30, 2023 14:58

noklam disabled auto-merge May 30, 2023 14:58

noklam enabled auto-merge (squash) May 30, 2023 14:59

noklam merged commit 4a4bd73 into main May 30, 2023

noklam deleted the 2277-introduce-our-own-rich-console-logging-handler branch May 30, 2023 15:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add the Kedro RichHandler #2592

Add the Kedro RichHandler #2592

noklam commented May 19, 2023 •

edited

Loading

noklam May 23, 2023

merelcht May 24, 2023

antonymilne May 24, 2023

antonymilne May 24, 2023

noklam May 23, 2023

noklam May 23, 2023

antonymilne May 24, 2023

noklam May 23, 2023

merelcht left a comment

merelcht May 24, 2023

antonymilne commented May 24, 2023

antonymilne left a comment •

edited

Loading

antonymilne May 24, 2023

antonymilne May 24, 2023

antonymilne May 24, 2023

antonymilne May 24, 2023

antonymilne May 24, 2023

antonymilne May 24, 2023

antonymilne May 24, 2023

noklam May 24, 2023

astrojuanlu May 30, 2023

noklam May 30, 2023

astrojuanlu May 30, 2023


		Kedro's `kedro.extras.logging.RichHandler` is a subclass of [`rich.logging.RichHandler`](https://rich.readthedocs.io/en/stable/reference/logging.html#rich.logging.RichHandler) and supports the same set of arguments. By default, `rich_tracebacks` is set to `True` to use `rich` to render exceptions. However, you can disable it by setting `rich_tracebacks: False`.

		When `rich_tracebacks` is set to `True`, the configuration is propagated to [`rich.traceback.install`](https://rich.readthedocs.io/en/stable/reference/traceback.html#rich.traceback.install). If an argument is compatible with `rich.pretty.install`, it will be passed to the traceback's settings.

Add the Kedro RichHandler #2592

Add the Kedro RichHandler #2592

Conversation

noklam commented May 19, 2023 • edited Loading

Description

Questions for Reviewer

Development notes

Checklist

Manual Testing

Note:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

merelcht left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antonymilne commented May 24, 2023

antonymilne left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

noklam commented May 19, 2023 •

edited

Loading

antonymilne left a comment •

edited

Loading