Adjust threshold cros_val_score sklearn

2024/11/17 14:27:19

There is a way to set the threshold cross_val_score sklearn?

I've trained a model, then I adjust the threshold to 0.22. The model in the following below :

# Try with Threshold
pred_proba = LGBM_Model.predict_proba(X_test)# Adjust threshold for predictions proba
prediction_with_threshold = []
for item in pred_proba[:,0]:if item > 0.22 :prediction_with_threshold.append(0)else:prediction_with_threshold.append(1)print(classification_report(y_test,prediction_with_threshold))

then I want to validate this model using cross_val_score. I've searched but can't find the method to set threshold for cross_val_score. The cross_val_score that I've used like the following below :

F1Scores = cross_val_score(LGBMClassifier(random_state=101,learning_rate=0.01,max_depth=-1,min_data_in_leaf=60,num_iterations=200,num_leaves=70),X,y,cv=5,scoring='f1')
F1Scores### how to adjust threshold to 0.22 ??

Or there is other method to validate this model using threshold?

Answer

Assuming that you are working with a two-class classification problem you could override the predict method of LGBMClassifier object with your thresholding approach as shown below:

import numpy as np
from lightgbm import LGBMClassifier
from sklearn.datasets import make_classificationX, y = make_classification(n_features=10, random_state=0, n_classes=2, n_samples=1000, n_informative=8)class MyLGBClassifier(LGBMClassifier):def predict(self,X, threshold=0.22,raw_score=False, num_iteration=None,pred_leaf=False, pred_contrib=False, **kwargs):result = super(MyLGBClassifier, self).predict_proba(X, raw_score, num_iteration,pred_leaf, pred_contrib, **kwargs)predictions = [1 if p>threshold else 0 for p in result[:,0]]return predictionsclf = MyLGBClassifier()
clf.fit(X,y)
clf.predict(X,threshold=2)  # just testing the implementation
# [0,0,0,0,..,0,0,0]        # we get all zeros since we have set threshold as 2F1Scores = cross_val_score(MyLGBClassifier(random_state=101,learning_rate=0.01,max_depth=-1,min_data_in_leaf=60,num_iterations=2,num_leaves=5),X,y,cv=5,scoring='f1')
F1Scores
#array([0.84263959, 0.83333333, 0.8       , 0.78787879, 0.87684729])
https://en.xdnf.cn/q/71174.html

Related Q&A

Efficiently insert multiple elements in a list (or another data structure) keeping their order

I have a list of items that should be inserted in a list-like data structure one after the other, and I have the indexes at which each item should be inserted. For example: items = [itemX, itemY, itemZ…

matplotlib versions =3 does not include a find()

I am running a very simple Python script: from tftb.generators import amgauss, fmlinI get this error: C:\Users\Anaconda3\envs\tf_gpu\lib\site-packages\tftb-0.0.1-py3.6.egg\tftb\processing\affine.py in …

How to fix Field defines a relation with the model auth.User, which has been swapped out

I am trying to change my user model to a custom one. I do not mind dropping my database and just using a new one, but when I try it to run makemigrations i get this errorbookings.Session.session_client…

Generate and parse Python code from C# application

I need to generate Python code to be more specific IronPyton. I also need to be able to parse the code and to load it into AST. I just started looking at some tools. I played with "Oslo" and …

An efficient way to calculate the mean of each column or row of non-zero elements

I have a numpy array for ratings given by users on movies. The rating is between 1 and 5, while 0 means that a user does not rate on a movie. I want to calculate the average rating of each movie, and t…

Selecting unique observations in a pandas data frame

I have a pandas data frame with a column uniqueid. I would like to remove all duplicates from the data frame based on this column, such that all remaining observations are unique.

GEdit/Python execution plugin?

Im just starting out learning python with GEdit plus various plugins as my IDE.Visual Studio/F# has a feature which permits the highlighting on a piece of text in the code window which then, on a keyp…

autoclass and instance attributes

According to the sphinx documentation, the .. autoattribute directive should be able to document instance attributes. However, if I do::.. currentmodule:: xml.etree.ElementTree.. autoclass:: ElementTre…

python sqlite3 update not updating

Question: Why is this sqlite3 statement not updating the record?Info:cur.execute(UPDATE workunits SET Completed=1 AND Returns=(?) WHERE PID=(?) AND Args=(?),(pickle.dumps(Ret),PID,Args))Im using py…

Unable to reinstall PyTables for Python 2.7

I am installing Python 2.7 in addition to 2.7. When installing PyTables again for 2.7, I get this error -Found numpy 1.5.1 package installed. .. ERROR:: Could not find a local HDF5 installation. You ma…