OpenCV - an ensemble of exemplary SVMs

I am trying to implement a sample SVM ensemble using OpenCV. The general idea outside of it is that given K training examples (e.g. car images) it is possible to train K SVMs where each SVM is trained using one positive pattern and negative negative K-1 results. During testing, the ensemble will shoot K times, with the best result being the best forecast.

To implement this, I am using the OpenCV SVM implementation (using the current GIT master - 3.0 at the time of writing) and a C ++ interface. I use HOG as functions, and the response size is ~ 7000. Therefore, each image has a 7000D function vector.

The problem I am facing is that the various SVMs are not being trained properly. In fact, they don't exercise at all! Training is actually very fast and always returns 1 support vector per SVM, with alpha = 1.0. I'm not sure if this is because I have one positive versus many (> 900) negatives, or if this is just a bug in my code. However, after looking at it a few times, I don't see any obvious errors.

This is how I set up my problem (assuming we got HOG responses for the whole dataset and put them in std :: vector> trainingData). Note that EnsambleSVMElement is the Struct that holds the SVM, and a lot of other information.

In short: I am setting up a training matrix where each row contains a HOG response for a specific sample. Then I start training each SVM separately. For each iteration of training, I create a vector of labels, where each entry is -1 (negative sample), except that the entry is associated with the current SVM that I am training, which is set to 1 (so if I participate in the entry 100 , only the positive mark will be on labels [100]).

Learning code

int ensambles = trainingData.size();

if(ensambles>1)
{
    //get params to normalise the data in [0-1]
    std::vector<float> mins(trainingData.size());
    std::vector<float> maxes(trainingData.size());

    for(int i=0; i<trainingData.size(); ++i)
    {
        mins[i] =   *std::min_element(trainingData[i].begin(), trainingData[i].end());
        maxes[i] =  *std::max_element(trainingData[i].begin(), trainingData[i].end());
    }

    float min_val = *std::min_element(mins.begin(), mins.end());
    float max_val = *std::min_element(maxes.begin(), maxes.end());
    int featurevector_size = trainingData[0].size();

    if(featurevector_size>0)
    {
        //set-up training data. i-th row contains HOG response for sample i
        cv::Mat trainingDataMat(ensambles, featurevector_size, CV_32FC1); 
        for(int i=0; i<trainingDataMat.rows; ++i)
            for(int j=0; j<trainingDataMat.cols; ++j)
                trainingDataMat.at<float>(i, j) = (trainingData.at(i).at(j)-min_val)/(max_val-min_val); //make sure data are normalised in [0-1] - libSVM constraint

        for(int i=0; i<ensambles; ++i)
        {
            std::vector<int> labels(ensambles, -1);
            labels[i] = 1; //one positive only, and is the current sample
            cv::Mat labelsMat(ensambles, 1, CV_32SC1, &labels[0]);
            cv::Ptr<cv::ml::SVM> this_svm = cv::ml::StatModel::train<SVM>(trainingDataMat, ROW_SAMPLE, labelsMat, svmparams);
            ensamble_svm.push_back(EnsambleSVMElement(this_svm));   
            Mat sv = ensamble_svm[i].svm->getSupportVectors();
            std::cout << "SVM_" << i << " has " << ensamble_svm[i].svm->getSupportVectors().rows << " support vectors." << std::endl;
        }
    }
    else
        std::cout <<"You passed empty feature vectors!" << std::endl;
}
else
    std::cout <<"I need at least 2  SVMs to create an ensamble!" << std::endl;

      

        
        
        
      

    

Cout always prints "SVM_i has 1 support vectors".

For completeness, these are my SVM options:

cv::ml::SVM::Params params;
params.svmType    = cv::ml::SVM::C_SVC;
params.C           = 0.1;
params.kernelType = cv::ml::SVM::LINEAR;
params.termCrit   = cv::TermCriteria(cv::TermCriteria::MAX_ITER, (int)1e4, 1e-6);

      

        
        
        
      

    

Changing the C value between 0.1 and 1.0 does not affect the results. There is also no setting for sample weights, as read here . Just for reference, this is how I adjust the weight (big penalty for negatives):

cv::Mat1f class_weights(1,2); 
class_weights(0,0) = 0.01; 
class_weights(0,1) = 0.99; 
params.classWeights = class_weights;

      

        
        
        
      

    

There is clearly something wrong in my code, or in my problem statement. Can anyone spot this?

Thank!