trulens-legacy #1345

sfc-gh-pmardziel · 2024-08-16T00:00:48Z

Description

Creates a new version of the trulens_eval package for backwards compatibility during a deprecation period.

Preserved trulens_eval.* names defined in __init__.py files for backwards compatibility to be deprecated later. Until then, a warning is issued of an old module is imported or a class or function imported via the old name is invoked. This is tested by test_deprecation.py which was recently added.

Example warning when importing from trulens_eval

from trulens_eval import Tru

DeprecationWarning: The `trulens_eval` module is deprecated. See https://trulens.org/docs/migration-guide for instructions on migrating to `trulens.*` modules.

Example warning when using something imported from trulens_eval:

Tru(...)

DeprecationWarning: Class `Tru` has moved:
    New import: `from trulens.core.tru import Tru`
 See https://trulens.org/docs/migration-guide for instructions on migrating to `trulens` modules.

Brought back some utilities for optional imports which will not be needed after the package is removed in the future. Optional imports are used to preserve how trulens_eval behaved with respect to optional imports in the past. However, in many cases, the packages that need to be installed have been changed to the optional trulens-* packages instead of what was is now these packages' dependencies.

Not covered

This does not cover all of the old trulens_eval names, only the exported ones. Will cover the rest once another PR that has the tests for the other names gets merged: API dump and deeper trulens_eval compatibility test #1309
Documentation changes. These will be in another PR.

Other details good to know for developers

Please include any other details of this change useful for TruLens developers.

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to
not work as expected)
New Tests
This change includes re-generated golden test results
This change requires a documentation update

src/core/trulens/core/utils/deprecation.py

sfc-gh-pdharmana · 2024-08-16T02:38:10Z

src/core/trulens/core/utils/deprecation.py

+    return _MovedClass
+
+
+def moved(


If users have both trulens_eval installed already, and also installed new version trulens without uninstalling trulens_eval what would happen?

Will try to enforce this with package constraints with this idea:

Rename the trulens-legacy package to trulens_eval, starting with version 1.0.0 which is greater than existing trulens_eval version.

In trulens_eval will now have requirements to install the required trulens-* packages.

With new trulens releases, the constraints in trulens_eval will have to be updated to keep track (or at least occasionally). Note that from discussions with @sfc-gh-chu , the plan is to have trulens package track the version of trulens-core.

This should cover the cases where people are still using and installing trulens_eval. To help with the migration to trulens we can suggest a two stage migration process:

We ask users to first replace their install trulens_eval with install trulens or merely upgrade their trulens_eval to this legacy version. As we added a constraint to trulens to install the legacy trulens_eval we can make sure the backwards compatibility trulens_eval replaces their old trulens_eval installation either way (whether they install trulens or install --upgrade trulens_eval.

The user can then observe deprecation warnings from their uses of trulens_eval to migrate the rest of their code so it does not refer to trulens_eval anywhere. Hopefully they do this before we remove the legacy package.

@sfc-gh-chu can you review this idea and check that the implementation of it in the PR is correct?

src/legacy/trulens_eval/__init__.py

sfc-gh-pdharmana · 2024-08-16T02:44:40Z

src/legacy/trulens_eval/__init__.py

+    "AzureOpenAI",
+    "OpenAI",
+    "Langchain",
+    "LiteLLM",


What would happen to this if those imports like Bedrock is not present, would it give a silent error? or do we create this list based on the conditional imports?

Same as before, the optional imports mechanism will make Bedrock a dummy class.

Makefile

src/core/trulens/core/utils/deprecation.py

src/trulens_eval/pyproject.toml

src/core/trulens/core/utils/requirements.optional.txt

src/core/trulens/core/utils/requirements.txt

sfc-gh-dkurokawa · 2024-08-16T05:39:03Z

src/trulens_eval/trulens_eval/database/__init__.py

+deprecation_utils.packages_dep_warn("trulens_eval.database")
+
+from trulens.core.database import *
+from trulens.core.database import base


are lines 13-19 necessary given 12?

Yes but it looks like line 12 does nothing. import * only does anything if __all__ is defined in the module we are importing from. Removed line 12.

Also adding a note explaining this.

sfc-gh-dkurokawa · 2024-08-16T05:40:04Z

src/trulens_eval/trulens_eval/database/migrations/__init__.py

+# NOTE: This file had contents in trulens_eval none were public or aliases.
+# Because of that, this backwards compatibility module is empty.
+
+from trulens.core.database.migrations import *


I'm confused what's going on here, in lines 12-13 you said it's empty, so what are you importing here?

I think it was just a copy and paste to capture __all__ exports but didn't remove it after realizing there was no __all__. However, I'm leaving this file in place because I want to also make the non __all__ names also backwards compatible in the next PR.

I'm deleting the import *, but leaving the file in place is what I meant.

Also adding a note explaining this.

Makefile

src/core/trulens/core/utils/requirements.optional.txt

src/trulens_eval/pyproject.toml

src/trulens_eval/trulens_eval/database/migrations/__init__.py

Merge remote-tracking branch 'origin/main' into piotrm/tru-legacy

* split from larger pr * fix duplicates in makefile * remove unneeded * nits * notes * rename trulens-legacy to trulens_eval * addressing prs * addressing pr comments * another one * relock * relock * fix pr notes * renamed

split from larger pr

628a54b

dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. dependencies Pull requests that update a dependency file labels Aug 16, 2024

sfc-gh-pmardziel added 2 commits August 15, 2024 17:03

fix duplicates in makefile

e2663e8

remove unneeded

291b774

dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels Aug 16, 2024

sfc-gh-pmardziel added 2 commits August 15, 2024 17:13

nits

cc22826

notes

2d1c09c

sfc-gh-pmardziel requested a review from sfc-gh-dkurokawa August 16, 2024 00:21

sfc-gh-pdharmana reviewed Aug 16, 2024

View reviewed changes

rename trulens-legacy to trulens_eval

1f540a5

sfc-gh-dkurokawa reviewed Aug 16, 2024

View reviewed changes

sfc-gh-pmardziel added 5 commits August 16, 2024 15:08

addressing prs

12ff1a3

addressing pr comments

a03118b

another one

00e1baf

relock

3d3890e

Merge remote-tracking branch 'origin/main' into piotrm/tru-legacy

f2a16a4

sfc-gh-pmardziel requested review from sfc-gh-dkurokawa and sfc-gh-pdharmana August 16, 2024 22:36

relock

383950f

sfc-gh-pmardziel mentioned this pull request Aug 19, 2024

[DRAFT] trulens-legacy and trulens-api #1307

Draft

4 tasks

sfc-gh-dkurokawa approved these changes Aug 20, 2024

View reviewed changes

Makefile Outdated Show resolved Hide resolved

src/core/trulens/core/utils/requirements.optional.txt Show resolved Hide resolved

src/trulens_eval/pyproject.toml Show resolved Hide resolved

src/trulens_eval/trulens_eval/database/migrations/__init__.py Outdated Show resolved Hide resolved

dosubot bot added the lgtm This PR has been approved by a maintainer label Aug 20, 2024

sfc-gh-pmardziel added 3 commits August 20, 2024 12:16

fix pr notes

18d8285

Merge.

1251e8c

Merge remote-tracking branch 'origin/main' into piotrm/tru-legacy

renamed

39cea46

sfc-gh-pmardziel merged commit 827b3ab into main Aug 20, 2024
7 checks passed

sfc-gh-pmardziel deleted the piotrm/tru-legacy branch August 20, 2024 22:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

trulens-legacy #1345

trulens-legacy #1345

sfc-gh-pmardziel commented Aug 16, 2024 •

edited

Loading

sfc-gh-pdharmana Aug 16, 2024

sfc-gh-pmardziel Aug 16, 2024 •

edited

Loading

sfc-gh-pmardziel Aug 16, 2024

sfc-gh-pdharmana Aug 16, 2024

sfc-gh-pmardziel Aug 16, 2024

sfc-gh-dkurokawa Aug 16, 2024

sfc-gh-pmardziel Aug 16, 2024

sfc-gh-pmardziel Aug 16, 2024

sfc-gh-dkurokawa Aug 16, 2024

sfc-gh-pmardziel Aug 16, 2024

sfc-gh-pmardziel Aug 16, 2024

sfc-gh-pmardziel Aug 16, 2024

trulens-legacy #1345

trulens-legacy #1345

Conversation

sfc-gh-pmardziel commented Aug 16, 2024 • edited Loading

Description

Not covered

Other details good to know for developers

Type of change

Choose a reason for hiding this comment

sfc-gh-pmardziel Aug 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfc-gh-pmardziel commented Aug 16, 2024 •

edited

Loading

sfc-gh-pmardziel Aug 16, 2024 •

edited

Loading