Core ML iOS 11 Doesn't Recognize Images Properly
I tried Core ML, which was recently introduced to iOS for image recognition and classification. The problem is that it doesn't give correct results for the submitted images.
If I post an image of the earth (Globe) it gives me the class as a bubble. Below is the code I used,
let model = Resnet50()
let pixelBuffer: CVPixelBuffer = (modelImg.pixelBuffer())!
if let prediction = try? model.prediction(image: pixelBuffer) {
print( "Found it!! It is a/an \(prediction.classLabel)")
}
Is it possible to provision a model with custom use cases on top of an existing Core ML model (e.g. Resnet50)
source to share
Core ML can only be used for inference with already prepared models. You will have to use some other tool to prepare your model first and then convert it to Core ML using Core ML Tools. You can find supported third-party machine learning tools and formats in the documentation:
https://developer.apple.com/documentation/coreml/converting_trained_models_to_core_ml
source to share
To expand on Matusalem's answer, when using an image classification network, you can only return results to one of the classes it was trained with. In this case, you appear to be using the pre-designed ResNet50 Core ML model that Apple has provided for download .
This Image Classification Network, like many others you will find there, has been trained on the ImageNet Large Scale Visual Recognition Challenge 2012 dataset, commonly referred to as ImageNet or ILSVRC2012. This dataset has become the benchmark for convolutional neural network constructs since the 2012 competition, so you will find many pre-trained networks using it. It has 1000 categories of things that it can identify in images, with a full list of labels here . You will notice that the "globe" is not among those, so the network will provide the closest comparable category of those it was trained with.
If you've never seen a globe before, or even been exposed to the word "globe", could you identify it? These networks can be generalized in amazing ways, but if they don't even have categories for what you want to categorize, they won't be able to provide the results you want.
The ImageNet dataset is an interesting starting point, but probably not suitable for real world use. It has a weird class distribution, with something like 200 dog breeds, but no humans. This can be a great starting point for pre-training the network and then using translation training to refine that network on your particular dataset, but you will most likely want to train your own application network. This is a whole different topic.
source to share