RBF-SVM prediction in python from sklearn learning

I trained a classifier with sklearn SVC.fit () and now I have a trained model. I can use the pred () method to predict classes, but now I want to implement it in python. (I'm working on where I need to compute the SVM gradients. The first step is a working python implementation.)

For example, using the following code, I can train my SVM models:

import sklearn.svm
import numpy as np

svm = sklearn.svm.SVC(probability=True, gamma=1)
f = svm.fit([[-3, -2], [0, 1], [.5,.5], [2, 1]], [0, 1, 0, 1])

      

And then compute a bunch of predictions by line:

ar = np.arange(-5,6,.01)
r = [f.predict_proba([[i, i]])[0,0] for i in ar]

      

Now I am trying to implement RBF-SVM method in python and compute predictions like this:

A = 1; B = 0; C = 1;
def pred(vectors, coefs, x):
    dists = np.sum((vectors-x)**2,axis=1)
    probs = (np.exp(-dists*A+B))+C
    probs *= np.reshape(coefs,[-1])
    probs /= np.sum(np.abs(probs))
    return np.sum(probs)+.5

s = [pred(svm.support_vectors_, svm.dual_coef_, [i, i]) for i in ar]

      

And when I build it I get the following plot

sklearn plot svm.predict vs python pred

It looks mostly correct, but not perfect. Why not? What's wrong with my implementation? (Also: why in some flat areas randomly? Sklearn rounds to 0.5 randomly?) I tried gradient descent over possible values ​​(A, B, C) and found that it works best when (A = 1.13; B = 0.31; C = .88;), but that's not all that enlightens.

I've tried looking at the sklearn source, but that just calls right in the libsvm binary, so I don't know what's going on there.

+3


source to share





All Articles