i am implementing svm using best parameter of grid search on 10fold cross validation and i need to understand prediction results why are different i got two accuracy results testing on training set notice that i need predictio results of the best parameters on the training set for further analysis the code and results are described below. Any explanation

from __future__ import print_functionfrom sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import classification_report
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score
from time import *
from sklearn import metrics
X=datascaled.iloc[:,0:13]
y=datascaled['num']np.random.seed(1)
# Split the dataset in two equal parts
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)# Set the parameters by cross-validation
tuned_parameters =  [{'kernel': ['rbf'], 'gamma': [1e-2, 1e-3, 1e-4, 1e-5],'C': [0.001, 0.10, 0.1, 10, 25, 50, 100, 1000]},{'kernel': ['sigmoid'], 'gamma': [1e-2, 1e-3, 1e-4, 1e-5],'C': [0.001, 0.10, 0.1, 10, 25, 50, 100, 1000] },{'kernel': ['linear'], 'C': [0.001, 0.10, 0.1, 10, 25, 50, 100, 1000]}]              print()clf = GridSearchCV(SVC(), tuned_parameters, cv=10,scoring='accuracy')
t0 = time()clf.fit(X_train, y_train)
t = time() - t0
print("Best parameters set found on development set:")
print()
print(clf.best_params_)
print()
print('Training accuracy')
print(clf.best_score_)
print(clf.best_estimator_)
print()
print()
print('****Results****')
svm_pred=clf.predict(X_train)
#print("\t\taccuracytrainkfold: {}".format(metrics.accuracy_score(y_train, svm_pred)))
print("=" * 52)
print("time cost: {}".format(t))
print()
print("confusion matrix\n", metrics.confusion_matrix(y_train, svm_pred))
print()
print("\t\taccuracy: {}".format(metrics.accuracy_score(y_train, svm_pred)))
print("\t\troc_auc_score: {}".format(metrics.roc_auc_score(y_train, svm_pred)))
print("\t\tcohen_kappa_score: {}".format(metrics.cohen_kappa_score(y_train, svm_pred)))
print()
print("\t\tclassification report")
print("-" * 52)
print(metrics.classification_report(y_train, svm_pred)) Best parameters set found on development set:{'C': 1000, 'gamma': 0.01, 'kernel': 'rbf'}Training accuracy
0.9254658385093167****Results****
====================================================
time cost: 7.728448867797852confusion matrix[[77  2][ 4 78]]accuracy: 0.9627329192546584roc_auc_score: 0.9629515282494597cohen_kappa_score: 0.9254744638173121classification report
----------------------------------------------------precision    recall  f1-score   support0       0.95      0.97      0.96        791       0.97      0.95      0.96        82avg / total       0.96      0.96      0.96       161

You are using 10-fold cross-validation for training and asking to calculate the prediction accuracy after each fold. I suggest doing the following.

Split the data into 10-folds using sklearn.model_selection.KFold and create a loop that passes through each fold as follows:

for train_index, test_index in kf.split(X):print("TRAIN:", train_index, "TEST:", test_index)X_train, X_test = X[train_index], X[test_index]y_train, y_test = y[train_index], y[test_index]

Inside that loop, build and train the model using the previously used lines repeated below. But use cv=1 rather than cv=10 inside GridSearchCV()

    clf = GridSearchCV(SVC(), tuned_parameters, cv=1, scoring='accuracy')clf.fit(X_train, y_train)

After training the model using data from one fold, then predict its accuracy using the data of the same fold according to the below lines used in your code.

    svm_pred=clf.predict(X_train)print("\t\taccuracy: {}".format(metrics.accuracy_score(y_train, svm_pred)))

The complete code is given below:

for train_index, test_index in kf.split(X):print("TRAIN:", train_index, "TEST:", test_index)X_train, X_test = X[train_index], X[test_index]y_train, y_test = y[train_index], y[test_index]clf = GridSearchCV(SVC(), tuned_parameters, cv=1, scoring='accuracy')clf.fit(X_train, y_train)svm_pred=clf.predict(X_train)print("\t\taccuracy: {}".format(metrics.accuracy_score(y_train, svm_pred)))

Wish that helps :)