Anyone have any metrics on the usefulness of formal unit testing?
Does anyone have any metrics for the usefulness of formal unit testing? I see there is a lot of focus on unit testing tools and I was curious why?
I stopped official unit testing more than 5 or 6 years ago and the net performance gain seems to be quite high. I stopped unit testing because I noticed that it never caught anything - let alone anything useful. The type of bugs that device wrapping detects seems to be preventable by not drinking more than 2 glasses of wine / beer per hour (or 2 joints per hour). Also, unit testing seems to pose a risk by letting the developer think there is some protection to catch their bugs.
I am testing to get the code to work as it should, but I am not using any tools. I am testing based on the changes made. My error rate when generating new code is zero. My error rate when changing code is around 2-3 errors per quarter. The above measures are based on 4 production apps I develop / maintain.
source to share
I acknowledge your excellence as a person and a coder.
I am, however, a simple moron, and without the Python unittest I would be lost.
I can't refactor without unit tests, it just takes too much thought.
I can barely code without unit tests, it is too complicated to be absolutely sure that I absolutely understand every nuance.
I unit test because I'm an idiot. Since you're not wrong, you obviously don't need a unit test. I greet you.
Edit. For me, unit tests are not about metrics or costs. I don't need randomized controlled experiments to show me value. I cannot work without them. Indeed, I refuse to work without them. In the same vein, I won't work without a compiler, text editor, or source control; I will not work without demands; I refuse to program without doing the design first.
source to share
I don't see unit testing as a replacement for traditional testing, but rather as an extra step to ensure code is correct. Some specific areas where I find unit testing useful:
- When refactoring / modifying existing code. Unit tests will confirm that at least these cases still work as expected. The more tests you have, the more confidence you have that code changes do not break existing code.
- When submitting bug reports. Having a unit test that throws the error is a great way to demonstrate the error AND know when it's fixed.
- Interface design tool. You have some test code to test the interfaces.
Perhaps a few others I forgot about: -P
PS: How do you know you don't make mistakes? I don't think I was introducing errors into the code I am working on, but this of course is not the case. IMHO, it is naive to think that your code is error free.
(As far as unit testing is concerned, if you know your code might contain bugs - and I would say most code - don't you want to use every possible way to catch them?)
source to share
Below is a white paper on Unit Test that may help you:
But, Martin Fowler put it, the anecdotal evidence in support of unit tests and TDD is overwhelming, but you cannot measure performance.
Unit testing is good because you can change a part and find out if elsewhere it changed something. Some people are "in love" with unit testing and need to calm themselves down. I believe in Unit Testing, but people who try to hide everything are dangerous to people who don't unit test.
source to share
Test Driven Development (TDD) ++ .
, , , .
In this case, I have a huge performance gain.
- To build and run my test: 10 seconds.
- To create, start and manually test a complete server: min. 5 minutes.
This means that I can iterate over the code region very quickly and fully check when I need it.
In addition to this, I've also used integration tests, which take longer to build and run, but only implement a specific piece of functionality that I'm interested in.
source to share
I have some sympathy for what you are saying. It is my experience that my new bugs rarely end up in unit tests. If at all, the unit tests are modified after the bug is discovered to make sure it doesn't reappear.
Where unit tests helped me in designing my (Java) classes. I often refactor the classes to make them testable (like removing singletones) which I think improved the overall design. For me this reason is sufficient for using them.
source to share
In my experience Unit Testing helped me with the following:
- Now I can completely focus on the code block / function / class I am writing without worrying about anything else because I know. If I am doing something stupid or causing a side effect, my tests will tell me
- I can reorganize stuff knowing that I am not breaking things.
- I am confident that my application is working as expected
- Before release, I do not manually check every single feature to confirm that the application is still running.
- I am confident that my application is always stable at some level
However, this has a little to do with me, as I have a really bad attitude about "managing multiple things at once" and "focusing". Therefore Unit Testing works for me and literally saves my day so many times where I introduced a new feature and broke some old features.
If it isn't, you just don't use it. If you still have the same output, performance and quality with the same error rate, then that means you don't need unit testing. Or you need to revisit your unit testing methodologies.
PS Based on your error rate and what you said, you are a genius anyway (assuming they are medium to large projects), so I would say don't use Unit Tests, it looks like you are fine. But for the rest of the world who are not geniuses like me, I highly recommend Unit Test because it worked for me.
source to share
Don't know about you, but I just checked in two patches for two source codes created by changes to existing code that my unit tests caught. And I just had a major performance issue that the unit test would catch if I trusted its results more (our unittests are a little more functional than I'd like to admit).
source to share
It seems to me that formal unit testing with popular testing tools is very much like the security of a US airport.
- It provides the illusion of security
- It makes people feel good.
- It is very uncomfortable (extremely uncomfortable if you have the wrong color skin)
- People will angrily wave their fists at them if you criticize him ...
- Those same people will remain drugged when their process fails and they ride the next cart ...
I think people have different views on software. In my opinion, software is a means of making money (hopefully by increasing income or saving money). I've seen posts for TDD, which is the closest I see as a scientific way to answer the question, but the methodology has no scientific rigor. None of these articles had a basic or fairly contrasting alternative method.
I think fans of formal unit testing will continue to feel safe. I will continue to write my specifications on a piece of paper and put them in a bowl at the feet of my statue of St. Anthony and say a prayer. Who can say which way is more effective, but my way is sure that it feels good ... Maybe I'll write a white paper there.
source to share
Remember the rise in popularity of 70s and 80s haircuts and clothes ... it didn't work out that well for those of us who lived in those decades.
Formal unit testing requires significant effort and maintenance. I guess it takes 20-50% of the time it takes to develop software. What I'm asking for is the known cost of adding 20-50% overhead for each development job, a payoff worth considering and / or provable.
By not doing formal unit testing, you force the developer to think about the appropriate things for testing. The developer gets more ownership of the product.
Formal unit testing sounds like snake oil juice ... Everyone and his brother say it's good, healthy, cool, etc., but there hasn't been a randomized controlled trial to prove it actually saves time or money. All answers so far are subjective evidence.
I would like to know if there is a software manager that can demonstrate better performance (or even better quality) after implementing unit testing.
lol - grand bombast S.Lott is the highest ranked answer ... Given that this forum is anonymous (to me anyway) your respect is not what I'm looking for. I would consider myself a little more mediocre. I've worked with exceptional developers ... these guys don't even do basic tests of their code at all - they just know it will work.
source to share