I don't want to hear the "we-need-to-separate-tests-from-code" excuse

Many programmers tend to place their unit tests in a dedicated source folder. I do it differently. In my projects, the unit-test of class SomeClass is called SomeClass_Tests and it is located in the very same folder as SomeClass.java.




This has all sort of benefits: I can instantly see if a class has a test. I can instantly jump from the test to the testee and vice-versa. I don't have to create two parallel hierarchies of folders. It is very unlikely that I will rename the production class but not its test. Renaming of a package affects both tests and production classes. etc. There is one word that summarizes these benefits: Locality. Things that change together ought to be placed as close as possible.

There are IDE plugins out there that provide similar capabilities over a project structure that has separate source folders. However, these plugins will not help you when you access your code not through your IDE (for instance, I often explore my code repository via a browser).

But even more importantly, as I got more and more (test) infected I realized that I don't want to have this test-production separation. The tests are not something external that needs to be put away. Tests are a central piece of knowledge about my code. I want them to be near by. Think about it: Would you separate your Javadoc text from the class/method it is describing? Will you find it productive to write code in a language which dictates that fields are defined in one file and methods are defined in another file? (Actually, this is pretty much what happens in C++ ...).

Of course not.

The main difficulty is the packaging/deployment phase. I often heard the argument that "we need to have two separate folders because otherwise our deliverable (.jar/.war/...) will include both production code and testing code, and this is bad".

Is it really bad? First, In many situations (web-apps anyone?) the size of the binary does not matter much. Second, in situations where the deliverable includes the source code, the tests can be quite handy as usage samples.

If you're in a situation where these arguments do not hold, and you absolutely cannot place test code in your deliverable, then the remainder of this post is for you.

I often said that it is not hard to write a program that will delete .class files of test classes. It requires some knowledge of bytecode inspection techniques which apparently is not very common. So, as a service to the Java community and for the common good, I cleared a few hours and wrote it. The result is class-wiper (sources) a command line utility that will recursively scan the given directories and will delete all .class files therein that are related to JUnit.

Specifically, it will delete a class if either: (a) it mentions the @Test annotation; or (b) It uses directly or indirectly a class that mentions @Test. Your production code will never meet neither of these conditions. If you don't want test classes to reach your client you just need to invoke it from your build script, as follows:

 java -jar class-wiper <binary-dir1>  <binary-dir2> ...


This software is provided absolutely for free. Use it anyway you like. Change it. Tweak it. Hack it. Sell it. Whatever. I only ask one simple thing: please stop saying that you need to place your tests in a separate directory because you don't have a way to prevent your tests from reaching the deliverable.

12 comments :: I don't want to hear the "we-need-to-separate-tests-from-code" excuse

  1. Sounds good. I've done so many times out of pure laziness. One extra benefit is that now the JUnit can access fields and methods defined with package visibility.
    Thanks for giving birth to the class-wiper :)

  2. I'd argue that you may run into trouble when using all sorts of test helpers, and test resources. How do you separate these from production code?

  3. Or you could just put them all in a separate source folder or even a separate test project (since that would let you control what you export more easily).

    Since you're already using Eclipse, based on the screenshot, then you can just use the moreUnit plugin to annotate which classes have tests and use Ctrl+J to jump between them (if you specify a convention, as you have done). That has the added advantage of letting you see more classes in the same vertical space (twice as many, in fact).

    Class-wiper has its uses, but I'd rather just organise in a way that doesn't need it.

  4. As you wrote, moreUnit does the easy jump back and forth (I use CTRL+j all the time) and does class renaming for you.
    We have lots of tests as well, often more test classes then production classes. Some of the test classes are actually bootstrappers, boilerplate killers, test utils, smart mocks etc.
    It will be awfully hard to distinguish between production and test classes. Some utility classes may never be use in production and they don't have test annotations like factories to entities which augment them with test data, interfaces to io api (more then simple mock), so its critical to verify it. We can verify one way dependency using the compiler, but hard to get all the edge cases with other tools.
    You might also want to do dependency checks (e.g. production code may not depend on mock lib since you may accidentally do bad stuff), don't know how to do it in this case.
    Another thing: static analysis. We do lots of those and the rules for test code are not the same for prod code. Hard to find only test code for instrumentation.
    Sorry if I sound too controversial, the class-wiper does sound cool. Its good to add it to the tool-chest.

  5. Seriously, why don't you use ant or any other automated build tool?
    This way YOU decide what ends up where, regardless of directory structure.
    Also I have no problem in jumping between "production" and test code even if they are on separate folders; and please don't tell me it's ok to deploy test code in production, it is useless bloat and it may cause problems like previous posters noted.
    The only advantage I see in your setup is package renaming, although I don't see it as a serious issue anyway.

  6. I place the tests in a different folder because I like to group code by meaning and usage. I can see a lot of reasons for separating the test code in a different hierarchy :
    - in large projects with lots of classes I don't want to clutter the hierarchy with the test classes ; it basically doubles the number of classes I have to manage in a single hierarchy and I don't want that. I want to keep the hierarchy as simple as possible so that everything remains clear to me at all times
    - test code is client code ; it is a client of your application code. Do you really want to keep client code in the same logic structure as your application code ? No
    - derived from the point above, test code is not like Javadoc. While it describes the application code, it does so for your internal use, not for public use
    - you can still access fields and methods with package visibility even if you put the test code in a different hierarchy, as long as the package name is the same (which should be the case)
    - I want to be able to make some assertions on my production code which will probably fail on test code, meaning I want to be able to apply and enforce some rules on production code that will not be mandatory for test code. Having the test code in the same location as the production code means I have to over-complicate my life and implement some kind of filters when using those rules. I don't want to complicate my life. I'm lazy

    Ionut

  7. Several quicks thoughts:
    - Hope you enjoy and learn writing the tool. Thanks for sharing. I'll keep using ant, if you don't mind, but thank you for your effort.

    - I don't understand your need of having a test class for each production class. Are you still writing a test for each method, or something like that? It's the relationship between your objects, their behaviour what you have to test, no the answer of each method... Otherwise, my experience tell me you end with more test code than production code, and it's easier for maintaining your test code to distribute test methods according to the feature you're testing rather than production method you test.

    -I'd like to know how are you going to distribute your integration test classes? Are you going to label them *_IntegrationTests?

    My 2 cents,
    Abel

  8. With IntelliJ IDEA, you can select the package view so you see both production and test code (I suppose Eclipse has the same feature).

    In my opinion, Maven 2 project standard makes the structure clearer and non need to customize any filter hack stuff when you build your application.

  9. First, if your project grows enough, you easily end up with 40-60 classes in one package (I means top level classes, not to mention the nested/inner ones) .

    2nd, package size does matter ! Moreover, it also makes your app looks Bloated and Resource-hogging (which is true for many Java apps)

    Also, the mentioned "benefits" like to see a class has test class, or package visibility, is also achieved if you follow good practises: name the test class similar to prod class, and use the same package name.
    (All experienced developers I know use Eclipse's Ctrl+Shift+T or Ctrl+Shift+R to open files, which easily shows the test classes. I guess the IDEA developers use similar shortcut-key, too)

  10. @Gili: It's not their location on the drive that allows package access, but which package they're in.

  11. I think this is most idiotic thing I ever read about doing project structuring...

  12. Definitely agree. This is the preferred test structure in GoLang, is very common in C/C++ codebases, and is becoming common in Javascript / TypeScript / CoffeeScript. The compiler does not mandate a location for tests, and so it falls under the domain of codebase style. It comes down to the preference of the programmer or company that starts and maintains a codebase, and is part of the style guide for maintaining a codebase.

    Style can quickly become the subject of holy-wars, and so you need to make the decision, and then move on with the codebase, to focus on the value and results that your app produces. There will always be another app that you write, so if you find you don't prefer a particular style of test location in one project, just try another on your next project.

Post a Comment