Keras 2, TypeError: cant pickle _thread.lock objects

2024/10/1 17:18:29

I am using Keras to create an ANN and do a grid search on the network. I have encountered the following error while running the code below:

model = KerasClassifier(build_fn=create_model(input_dim), verbose=0)
# define the grid search parameters
batch_size = [10, 20]
epochs = [50, 100]
dropout = [0.3, 0.5, 0.7]
param_grid = dict(dropout_rate=dropout, batch_size=batch_size, nb_epoch=epochs)
pipe.append(('classify', model))
params.append(param_grid)
pipeline=Pipeline(pipe)
#the pipeline also contains feature selector, but for convenience I do not include code here
piped_classifier =  GridSearchCV(estimator=pipeline, param_grid=params, n_jobs=-1,cv=nfold)
piped_classifier.fit(X_train, y_train) #this is line 246 of classifier_gridsearch.py causing error, see below,def create_model(input_dim,dropout_rate=0.0):# create modelmodel = Sequential()model.add(Dense(80,input_dim=input_dim,kernel_initializer='uniform', activation='relu'))model.add(Dropout(dropout_rate))model.add(Dense(1, kernel_initializer='uniform', activation='sigmoid'))# Compile modelmodel.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])return model

The error of line 246 throws an error as below, with a long stacktrace:

Traceback (most recent call last):File "/home/zqz/Work/chase/python/src/exp/classifier_gridsearch_main.py", line 176, in <module>classifier.gridsearch()File "/home/zqz/Work/chase/python/src/exp/classifier_gridsearch_main.py", line 155, in gridsearchself.fs_option,self.fs_gridsearch)File "/home/zqz/Work/chase/python/src/ml/classifier_gridsearch.py", line 246, in learn_dnnpiped_classifier.fit(X_train, y_train)File "/home/zqz/Programs/anaconda3/lib/python3.6/site-packages/sklearn/model_selection/_search.py", line 945, in fitreturn self._fit(X, y, groups, ParameterGrid(self.param_grid))File "/home/zqz/Programs/anaconda3/lib/python3.6/site-packages/sklearn/model_selection/_search.py", line 550, in _fitbase_estimator = clone(self.estimator)File "/home/zqz/Programs/anaconda3/lib/python3.6/site-packages/sklearn/base.py", line 69, in clonenew_object_params[name] = clone(param, safe=False)File "/home/zqz/Programs/anaconda3/lib/python3.6/site-packages/sklearn/base.py", line 57, in clonereturn estimator_type([clone(e, safe=safe) for e in estimator])File "/home/zqz/Programs/anaconda3/lib/python3.6/site-packages/sklearn/base.py", line 57, in <listcomp>return estimator_type([clone(e, safe=safe) for e in estimator])File "/home/zqz/Programs/anaconda3/lib/python3.6/site-packages/sklearn/base.py", line 57, in clonereturn estimator_type([clone(e, safe=safe) for e in estimator])File "/home/zqz/Programs/anaconda3/lib/python3.6/site-packages/sklearn/base.py", line 57, in <listcomp>return estimator_type([clone(e, safe=safe) for e in estimator])File "/home/zqz/Programs/anaconda3/lib/python3.6/site-packages/sklearn/base.py", line 69, in clonenew_object_params[name] = clone(param, safe=False)File "/home/zqz/Programs/anaconda3/lib/python3.6/site-packages/sklearn/base.py", line 60, in clonereturn copy.deepcopy(estimator)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 180, in deepcopyy = _reconstruct(x, memo, *rv)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 280, in _reconstructstate = deepcopy(state, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 150, in deepcopyy = copier(x, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 240, in _deepcopy_dicty[deepcopy(key, memo)] = deepcopy(value, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 150, in deepcopyy = copier(x, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 215, in _deepcopy_listappend(deepcopy(a, memo))File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 180, in deepcopyy = _reconstruct(x, memo, *rv)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 280, in _reconstructstate = deepcopy(state, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 150, in deepcopyy = copier(x, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 240, in _deepcopy_dicty[deepcopy(key, memo)] = deepcopy(value, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 150, in deepcopyy = copier(x, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 215, in _deepcopy_listappend(deepcopy(a, memo))File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 180, in deepcopyy = _reconstruct(x, memo, *rv)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 280, in _reconstructstate = deepcopy(state, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 150, in deepcopyy = copier(x, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 240, in _deepcopy_dicty[deepcopy(key, memo)] = deepcopy(value, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 180, in deepcopyy = _reconstruct(x, memo, *rv)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 280, in _reconstructstate = deepcopy(state, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 150, in deepcopyy = copier(x, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 240, in _deepcopy_dicty[deepcopy(key, memo)] = deepcopy(value, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 180, in deepcopyy = _reconstruct(x, memo, *rv)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 280, in _reconstructstate = deepcopy(state, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 150, in deepcopyy = copier(x, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 240, in _deepcopy_dicty[deepcopy(key, memo)] = deepcopy(value, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 180, in deepcopyy = _reconstruct(x, memo, *rv)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 280, in _reconstructstate = deepcopy(state, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 150, in deepcopyy = copier(x, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 240, in _deepcopy_dicty[deepcopy(key, memo)] = deepcopy(value, memo)File "/home/zqz/Programs/anaconda3/lib/python3.6/copy.py", line 169, in deepcopyrv = reductor(4)
TypeError: can't pickle _thread.lock objects

Any suggestions how to fix this please, thanks

Answer

OK the problem seems to be passing method as an argument, in this line:

model = KerasClassifier(build_fn=create_model(input_dim), verbose=0)

create_model is the argument to be passed to build_fn, but I wanted to pass another argument to create_model. But this is not the right way of doing it and hence causing the error.

The error message is not informative in this case, unfortunately.

https://en.xdnf.cn/q/70941.html

Related Q&A

How to remove minimize/maximize buttons while preserving the icon?

Is it possible to display the icon for my toplevel and root window after removing the minimize and maximize buttons? I tried using -toolwindow but the icon cant be displayed afterwards. Is there anoth…
READ MORE...

Undefined reference to `PyString_FromString

I have this C code:... [SNIP] ... for(Node = Plugin.Head; Node != NULL; Node = Node->Next) {//Create new python sub-interpreterNode->Interpreter = Py_NewInterpreter();if(Node->Interpreter == N…
READ MORE...

How to call a method from a different blueprint in Flask?

I have an application with multiple blueprinted modules.I would like to call a method (a route) that would normally return a view or render a template, from within a different blueprints route.How can …
READ MORE...

Dynamically define functions with varying signature

What I want to accomplish:dct = {foo:0, bar:1, baz:2} def func(**dct):pass #function signature is now func(foo=0, bar=1, baz=2)However, the ** syntax is obviously clashing here between expanding a dict…
READ MORE...

python pandas plot with uneven timeseries index (with count evenly distributed)

My dataframe has uneven time index.how could I find a way to plot the data, and local the index automatically? I searched here, and I know I can plot something like e.plot()but the time index (x axis)…
READ MORE...

python OpenAI gym monitor creates json files in the recording directory

I am implementing value iteration on the gym CartPole-v0 environment and would like to record the video of the agents actions in a video file. I have been trying to implement this using the Monitor wra…
READ MORE...

Install xgboost under python with 32-bit msys failing

Trying to install xgboost is failing..? The version is Anaconda 2.1.0 (64-bit) on Windows & enterprise. How do I proceed? I have been using R it seems its quite easy to install new package in R …
READ MORE...

Pythons _winapi module

I was trying to write some python code that requires calls to native WINAPI functions. At first I came across the pypiwin32 package. Then, somewhere on the internet I saw someone using the _winapi modu…
READ MORE...

list comprehension with numpy arrays - bad practice?

I am wondering if the below approach would be considered bad practice, and if so, if someone could give some guidance towards another approach. Here is the code in question:a = np.array([[1,2,3],[4,5,6…
READ MORE...

pandas: write dataframe to excel file *object* (not file)?

I have a dataframe that I want to convert to excel file, and return it using HTTP. Dataframes to_excel method accepts either a path, or an ExcelWriter, which, in turn, refers to a path.Is there any way…
READ MORE...
Latest Q&A

Copyright @ 2022~2023