XGB via Scikit learn API doesn't seem to work on GPU, although compiled to work on GPU
As far as I can tell, Scikit learn API does not currently support GPUs. You need to use the training API (e.g. xgboost.train (...)). It also requires you to convert your data to xgboost DMatrix first.
Example:
params = {"updater":"grow_gpu"}
train = xgboost.DMatrix(x_train, label=y_train)
clf = xgboost.train(params, train, num_boost_round=10)
UPDATE:
Scikit Learn API now supports GPUs via ** kwargs argument: http://xgboost.readthedocs.io/en/latest/python/python_api.html#id1
source to share
I was not able to get this working from the installed XGBoost package, but I pulled the latest XGBoost from GitHub ( git clone --recursive https://github.com/dmlc/xgboost
) and compiled it with a flag PLUGIN_UPDATER_GPU
that allowed me to use the GPU with the sklearn API.This required me to change some NVCC flags to work with mine GTX960 which was causing some build errors and then some runtime errors due to architecture mismatch. After creating it, I installed with pip install -e python-package/
to the repo directory. To use Scikit Learn API (using grow_gpu
either grow_hist_gpu
):
import xgboost as xgb
model = xgb.XGBClassifier(
max_depth=5,
objective='binary:logistic',
**{"updater": "grow_gpu"}
)
model.fit(train_x, train_y)
If anyone is interested in the process of fixing a GPU flag build, here is the process I went through on Ubuntu 14.04.
i) git clone git clone --recursive https://github.com/dmlc/xgboost
ii) cd insto xgboost and make -j4
to create multithreading if no GPU needed
iii) to make a GPU edit make / config.mk to use PLUGIN_UPDATER_GPU
iv) Edit the Makefile Makefile in the NVCC section to use the flag --gpu-architecture=sm_xx
for the GPU version (5.2 for GTX 960) on line 101
#CODE = $(foreach ver,$(COMPUTE),-gencode arch=compute_$(ver),code=sm_$(ver)) TO
CODE = --gpu-architecture=sm_52
v) Run ./build.sh
, it should say it is multithreaded or the NVCC build probably failed (or other error, look above for error)
vi) In virtualenv (if desired) in the same directory run pip install -e python-package/
These are some of the things that caused some nvcc errors for me:
i) Install / update the Cuda Toolkit by downloading the cuda toolkit .deb
from Nvidia (version 8.0 worked for me and is required in some cases?).
ii) Install / update cuda
sudo apt-get update
sudo apt-get install cuda
iii) Add nvcc
to your path. Mine was in/usr/local/cuda/bin/
iv) Restart may be required if startup nvidia-smi
does not work due to some updates to cuda / driver / toolkit.
source to share