Clever Geek Handbook

How to evaluate the accuracy and loss of the trained model as good or not in coffee?

I have been training my data with Caffe. Following are the parameters in the solver.prototxt file.

test_iter: 100
test_interval: 1000
base_lr: 0.00001
lr_policy: "step"
gamma: 0.1
stepsize: 20000
display: 20
max_iter: 13000
momentum: 0.9
weight_decay: 0.0005
snapshot: 10000
solver_mode: GPU

The testing accuracy is about 0.0185, and in the first test, the test loss is about 2.85. On iterator 13000, the test accuracy is 0.92 and the test loss is 0.52. How can I assess if the accuracy is good or is it related to data fit? Is there a way that I can use to evaluate? Below are the accuracy and loss with each test.

I1031 11:36:47.063542 18188 solver.cpp:404]     Test net output #0: accuracy = 0.0184
I1031 11:36:47.063623 18188 solver.cpp:404]     Test net output #1: loss = 2.85812 (* 1 = 2.85812 loss)   

I1031 11:38:49.832749 18188 solver.cpp:337] Iteration 1000, Testing net (#0)
I1031 11:38:55.510437 18188 solver.cpp:404]     Test net output #0: accuracy = 0.764
I1031 11:38:55.510509 18188 solver.cpp:404]     Test net output #1: loss = 1.1725 (* 1 = 1.1725 loss)  

I1031 11:40:59.547670 18188 solver.cpp:337] Iteration 2000, Testing net (#0)
I1031 11:41:04.451655 18188 solver.cpp:404]     Test net output #0: accuracy = 0.6372
I1031 11:41:04.451717 18188 solver.cpp:404]     Test net output #1: loss = 1.44847 (* 1 = 1.44847 loss)  

I1031 11:43:05.340741 18188 solver.cpp:337] Iteration 3000, Testing net (#0)
I1031 11:43:09.262504 18188 solver.cpp:404]     Test net output #0: accuracy = 0.8844
I1031 11:43:09.262568 18188 solver.cpp:404]     Test net output #1: loss = 0.576498 (* 1 = 0.576498 loss)

I1031 11:45:10.821233 18188 solver.cpp:337] Iteration 4000, Testing net (#0)
I1031 11:45:14.686005 18188 solver.cpp:404]     Test net output #0: accuracy = 0.5484
I1031 11:45:14.686064 18188 solver.cpp:404]     Test net output #1: loss = 1.91799 (* 1 = 1.91799 loss)

I1031 11:47:17.107151 18188 solver.cpp:337] Iteration 5000, Testing net (#0)
I1031 11:47:21.160307 18188 solver.cpp:404]     Test net output #0: accuracy = 0.8908
I1031 11:47:21.160372 18188 solver.cpp:404]     Test net output #1: loss = 0.54212 (* 1 = 0.54212 loss)

I1031 11:49:23.325654 18188 solver.cpp:337] Iteration 6000, Testing net (#0)
I1031 11:49:27.229637 18188 solver.cpp:404]     Test net output #0: accuracy = 0.7384
I1031 11:49:27.229677 18188 solver.cpp:404]     Test net output #1: loss = 1.1355 (* 1 = 1.1355 loss)

I1031 11:51:29.619175 18188 solver.cpp:337] Iteration 7000, Testing net (#0)
I1031 11:51:33.568794 18188 solver.cpp:404]     Test net output #0: accuracy = 0.6264
I1031 11:51:33.568837 18188 solver.cpp:404]     Test net output #1: loss = 1.10095 (* 1 = 1.10095 loss)

I1031 11:53:36.075034 18188 solver.cpp:337] Iteration 8000, Testing net (#0)
I1031 11:53:39.969431 18188 solver.cpp:404]     Test net output #0: accuracy = 0.9156
I1031 11:53:39.969481 18188 solver.cpp:404]     Test net output #1: loss = 0.522906 (* 1 = 0.522906 loss)

I1031 11:55:42.594107 18188 solver.cpp:337] Iteration 9000, Testing net (#0)
I1031 11:55:46.473902 18188 solver.cpp:404]     Test net output #0: accuracy = 0.5228
I1031 11:55:46.473961 18188 solver.cpp:404]     Test net output #1: loss = 1.63102 (* 1 = 1.63102 loss)

I1031 11:57:55.669351 18188 solver.cpp:337] Iteration 10000, Testing net (#0)
I1031 11:57:59.571413 18188 solver.cpp:404]     Test net output #0: accuracy = 0.8472
I1031 11:57:59.571485 18188 solver.cpp:404]     Test net output #1: loss = 0.638568 (* 1 = 0.638568 loss

I1031 12:00:01.984112 18188 solver.cpp:337] Iteration 11000, Testing net (#0)
I1031 12:00:05.870985 18188 solver.cpp:404]     Test net output #0: accuracy = 0.7476
I1031 12:00:05.871040 18188 solver.cpp:404]     Test net output #1: loss = 1.19568 (* 1 = 1.19568 loss)

I1031 12:02:08.464495 18188 solver.cpp:337] Iteration 12000, Testing net (#0)
I1031 12:02:12.397044 18188 solver.cpp:404]     Test net output #0: accuracy = 0.596
I1031 12:02:12.397104 18188 solver.cpp:404]     Test net output #1: loss = 1.32557 (* 1 = 1.32557 loss)

I1031 12:04:24.876971 18188 solver.cpp:337] Iteration 13000, Testing net (#0)
I1031 12:04:28.693732 18188 solver.cpp:404]     Test net output #0: accuracy = 0.9284
I1031 12:04:28.693771 18188 solver.cpp:404]     Test net output #1: loss = 0.521483 (* 1 = 0.521483 loss)

Please let me know if you need more information.

0


source to share


1 answer


Overfitting refers to a situation where increasing training accuracy does not increase testing accuracy. Therefore, I suggest that you continue learning until this happens, or you are satisfied with the accuracy of the testing.



0


source






More articles:


All Articles