Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

span database #1608

Open
wants to merge 89 commits into
base: main
Choose a base branch
from
Open

span database #1608

wants to merge 89 commits into from

Conversation

sfc-gh-pmardziel
Copy link
Contributor

@sfc-gh-pmardziel sfc-gh-pmardziel commented Oct 29, 2024

Description

  • Adds trulens-semconv package to hold tracing attribute names and related. This package intentionally has no dependencies other than otel semconv packages.

  • Adds span schema as orm classes and span to/of orm methods.

  • Adds span "semantic categories" for known components like retrievers.

  • Adds span extensions to db base abstract class and the sqlalchemy implementation for inserting, selecting, deleting spans.

  • Adds some prototype abstractions to base DB class for pagination and queries.

  • Extended trulens.core.schema.types with classes organizing types for values we use across python, otel, and sql.

- Recognizing spans into the semantic categories not included (but the categories exist). Will bring this in from an old PR next.

- Feedback evaluation not included.

- Collector not included (though started in this PR).

Other details good to know for developers

Please include any other details of this change useful for TruLens developers.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to
    not work as expected)
  • New Tests
  • This change includes re-generated golden test results
  • This change requires a documentation update

Important

Introduces trulens-semconv for tracing attributes, adds span schema and database methods, and organizes types across Python, OTEL, and SQL.

  • Semantic Conventions:
    • Introduces trulens-semconv package for tracing attribute names.
    • Defines SpanAttributes and SpanType in trace.py for span categorization.
  • Span Management:
    • Adds span schema as ORM classes and methods for span operations in the database.
    • Extends base database class with pagination and query abstractions.
  • Type Organization:
    • Adds classes in trulens.core.schema.types for organizing types across Python, OTEL, and SQL.
  • Miscellaneous:
    • Updates pyproject.toml to include trulens-semconv package.
    • Does not include span recognition into semantic categories, feedback evaluation, or a collector.

This description was created by Ellipsis for 812dfc2. It will automatically update as commits are pushed.

class DB(serial_utils.SerialModel, abc.ABC, text_utils.WithIdentString):
"""Abstract definition of databases used by trulens.

[SQLAlchemyDB][trulens.core.database.sqlalchemy.SQLAlchemyDB] is the main
and default implementation of this interface.
"""

T: ClassVar[Type] = Any
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using more specific types instead of Type for T, Q, and W to improve type safety and clarity. This comment applies to lines 106, 109, and 112.

Suggested change
T: ClassVar[Type] = Any
T: ClassVar[SpecificType] = Any

before_created_timestamp: builtins.NoneType
before_index: builtins.NoneType
before_updated_timestamp: builtins.NoneType
limit: builtins.NoneType
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider initializing limit and offset with integer values instead of NoneType to avoid potential issues in calculations or queries.

Suggested change
limit: builtins.NoneType
limit: builtins.int

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file size:XXL This PR changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants