Scikit-lean GridSearchCV n_jobs! = 1 freeze
I run a grid search in random forests and try to use n_jobs other than one, but the kernel hangs, no CPU usage. With n_jobs = 1 it works great. I can't even stop the ctl-C command and restart the kernel. I am working on Windows 7. I have seen a similar problem occur with OS X, but the solution does not apply to windows 7.
from sklearn.ensemble import RandomForestClassifier
rf_tfdidf = Pipeline([('vect',tfidf),
('clf', RandomForestClassifier(n_estimators=50,
class_weight='balanced_subsample'))])
param_grid = [{'vect__ngram_range':[(1,1)],
'vect__stop_words': [stop],
'vect__tokenizer':[tokenizer]
}]
if __name__ == '__main__':
gs_rf_tfidf = GridSearchCV(rf_tfdidf, param_grid, scoring='accuracy', cv=5,
verbose=10,
n_jobs=2)
gs_rf_tfidf.fit(X_train_part, y_train_part)
thank.
source to share
The indentation after is if __name__ == '__main__':
incorrect. If it is not and it is a copying error, then you can try something like:
if __name__ =='main':
# your code indented !
So the first line of your script is equal if __name__ == '__main__':
and then the remainder code is indented appropriately.
New code
from sklearn.ensemble import RandomForestClassifier
from sklearn.pipeline import Pipeline
if __name__ == '__main__':
rf_tfdidf = Pipeline([('vect',tfidf),('clf', RandomForestClassifier(n_estimators=50,class_weight='balanced_subsample'))])
param_grid = [{'vect__ngram_range':[(1,1)],'vect__stop_words': [stop],'vect__tokenizer':[tokenizer]}]
gs_rf_tfidf = GridSearchCV(rf_tfdidf, param_grid, scoring='accuracy', cv=5,verbose=10, n_jobs=-1)
gs_rf_tfidf.fit(X_train_part, y_train_part)
This works fine for me (windows 8.1)
EDIT
The following works fine with PyCharm. I haven't used spyder, but it should work for spyder as well:
code
Class Test(object):
def __init__(self):
###code here
###code here
if __name__ == '__main__':
Test()
source to share