You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
First off, very much appreciate the package--there are a ton of interesting methods contained here! I was particularly interested in the Hierarchical Shrinkage trees, and had hoped they would be applicable to some of the other gradient boosting methods like LightGBM.
I realize now these are not supported. However, if a user tries to wrap a LGBM classifier with HSTree, they can do so. The fit and predict methods will work without error, but actually nothing changes to the underlying estimator. Similar points were made in #137 and #171. A reproducing example:
from sklearn import datasets
from sklearn.model_selection import train_test_split
from lightgbm import LGBMClassifier
from imodels import HSTreeClassifier
import numpy as np
np.random.seed(15)
X, y = datasets.load_breast_cancer(return_X_y=True) # binary classification
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=10)
#Train LGBM
myLGBM = LGBMClassifier()
myLGBM.fit(X_train, y_train)
#LGBM 'wrapped' by HSTree
myHST = HSTreeClassifier(LGBMClassifier(), reg_param=1000)
myHST.fit(X_train, y_train)
#Show they predict same probas
np.array_equal(myLGBM.predict_proba(X_test), myHST.predict_proba(X_test)) #true
I think this behavior is a bit misleading, and instead the user should be prompted with a warning/error saying the model is unsupported. From my understanding of the code, a model is unsupported if it contains neither a 'tree_' nor an 'estimators_' attribute, so a check for these attributes could perhaps be included as part of the initialization code. Please let me know if I've misunderstood anything!
The text was updated successfully, but these errors were encountered:
First off, very much appreciate the package--there are a ton of interesting methods contained here! I was particularly interested in the Hierarchical Shrinkage trees, and had hoped they would be applicable to some of the other gradient boosting methods like LightGBM.
I realize now these are not supported. However, if a user tries to wrap a LGBM classifier with HSTree, they can do so. The fit and predict methods will work without error, but actually nothing changes to the underlying estimator. Similar points were made in #137 and #171. A reproducing example:
I think this behavior is a bit misleading, and instead the user should be prompted with a warning/error saying the model is unsupported. From my understanding of the code, a model is unsupported if it contains neither a 'tree_' nor an 'estimators_' attribute, so a check for these attributes could perhaps be included as part of the initialization code. Please let me know if I've misunderstood anything!
The text was updated successfully, but these errors were encountered: