June 14, 2012

Hands-on Video: Test-Driven Development - Java Application Skeleton


Watch me code! I will use 30 minutes (split this into two videos) to create a simple Java Swing MVC application skeleton with TDD. The video is quite fast-paced so you'll need some prior knowledge about TDD (Test-Driven Development), mocks and MVC (Model-View-Controller) to get most of it.

This is my first video upload ever, so please forgive the bad quality of the recording. When I have time I'll try to find out what went wrong in conversions/upload and fix the video.

I'd love to have some comments on this. Do you want to see more videos like this? If yes, what type of application would you like me to create? I will create more only if people like this, so please comment.


Tools & Frameworks: Contents:
  • Eclipse
  • m2eclipse (Eclipse Maven plugin)
  • eclEmma (Eclipse coverage plugin)
  • JUnit 
  • Mockito

  • Create a new Maven 2 project
  • Use Maven Central Repository to pick up testing frameworks
  • Create simple Controller, View and Model classes
  • Use mocks for testing 
  • Test that exceptions are thrown
  • Use JUnit's assertThat (Hamcrest)
  • Test a method that launches a new thread
  • Switch execution to EventDispatchThread
  • Run test coverage with EclEmma


Vimeo does not offer HD video embedding, so go to the Vimeo to see the videos:

Hands-on TDD: Java (MVC) Application Skeleton Part I


Hands-on TDD: Java (MVC) Application Skeleton Part II (Youtube)
OR
Hands-on TDD: Java (MVC) Application Skeleton Part II (Vimeo, not HD)




If you like my videos, you may also like James Shore's Let's Play: TDD series. I found that series while I was uploading my videos :)

June 03, 2012

Change of Focus

Hi there, I recently made a decision that is going to change the focus of this blog. I resigned. I have been working for the same employer for 13 years and I will surely miss my old colleagues, but this was something I wanted to do. 

Now I am in a lucky position where I can, for a while, do the projects that I want. Bwahahahaa, I am the master of my universe!

As you read this, I'm building a nice little web application with Vaadin, working against Cassandra DB. My main focus is to update my skills to the web and do that as fast as possible. 

So, from now on you will no longer be hearing stories of the challenges with large code-bases. What you will get, I have no idea. And I am definitely not going to change my habits, so my updates will be as non-regular as before.  

May 05, 2012

An Affair Between ObjectMother And TestDataBuilder

I've read a handful of blog posts discussing whether to choose ObjectMother or TestDataBuilder for test data creation. Most of them goes like this: "I've been using ObjectMother, but now I have found the TestDataBuilder and I'm ditching the ObjectMother for it". Others warn you that "ObjectMother is just another name for GodClass antipattern and you should never use it - use TestDataBuilder". Reading those one could easily think that ObjectMothers are not good for anything.

But wait a moment! If your ObjectMother gets bloated and filled with methods like createAFilthyRichCustomerWhoDrivesAPorsche() then it is not the ObjectMother to blame, but you and your fellow programmers. Any class can become a GodClass if one decides to make it such. You can always build a horrendous test data setup and hide it somewhere, too.

ObjectMother should not be a dump for miscellaneous methods that give name to a multipage test data creation vomit, but a helper that offers centralized methods to create test data. Instead of createAFilthyRichCustomerWhoDrivesAPorsche() it could have methods like Customer newCustomer(String name) and addCars(Customer customer, Car... cars). Often you will find that the methods in ObjectMother are useful for the production code and you end up moving some of them there.

I think it is a good approach to start with ObjectMother and introduce the TestDataBuilder when there is a need to create more complex objects or to avoid dependency injections. At start the ObjectMother could create the customer like this:

public Customer newCustomer(String name){
    Customer customer = new Customer(name);
    customer.setFunds(1000);  
    return customer;



Later on  when customer gets more complicated you can easily let the ObjectMother use TestDataBuilder:

public Customer newCustomer(String name){
    Customer customer = testCustomer().name(name).build();
    return customer;




public CustomerBuilder testCustomer(){
return new CustomerBuilder().defaultData();
}

Now ObjectMother works as a dictionary for existing TestDataBuilders and it has methods that ease up their use. So it really works as a helper for building test data.

My TestDataBuilders don't add default data to the created test objects unless it is asked from them. We started with builders that did, but soon found out we wanted to use the empty versions too. We have data with deep hierarchies and for some test cases we just wanted an empty data with all child elements at place and empty too. So I added a separate method for adding a default data and a method verifyDefaultData() to verify that data.

You also might have noticed how I said you can use TestDataBuilder to avoid dependency injections. I don't advocate dependency injections. Most often they are used to inject critical dependencies and adding public or protected methods to access them just breaks encapsulation. I rather give dependencies in constructor and raise an exception if they are missing. TestDataBuilder lets you construct the test data in phases just like dependency injection would.

@Test
public void testCustomerIsAddedToDb(){

    Db dbMock = mock(Db.class);
    Register register =  mother.testRegister().db(dbMock).build();
    Customer john = mother.newCustomer("John");
    register.addCustomer(john);
    verifyCustomerAddedToDb(john, dbMock);
}

What I also don't advocate is wrapping TestDataBuilders inside each other. You might be tempted to do that to make code look cleaner, but please don't. There are several reasons against. Every now and then you'll want to feed the TestsDataBuilder a real item instead of a builder. You'll end up having to implement both customer(CustomerBuilder builder) and customer(Customer customer). Also, when you give your builder away you no longer see when the build is actually called. Look at this:

public void createAProblem(){
    CustomerBuilder builder = mother.testCustomer();
    AddressBuilder aBuilder = mother.testAddress();
    builder.address(aBuilder);
    aBuilder.street("Abbey Road");
    Customer customer = builder.build();
}

Can you tell me what street the customer lives in? You'll have plenty of work dealing with real problems so keep your builders clean and simple.

If you want to read more about managing test data then Jay Fields' Creating Objects in Java Unit Tests is a nice read. Jay is using something he calls DomainObjectBuilder in place of the ObjectMother. I think the idea is the same though - single access point to TestDataBuilders.

February 18, 2012

Test Driven Development Is The Heart Of Agile


I am working for a large company that has selected agile as our working ideology. In practice it means that we all follow the Agile Manifesto principles, but inside the teams we are quite free to select our ways of working. While some teams absolutely love agile some teams deeply dislike it. I enjoy being agile, but I during this sprint I have learned in the hard way why some teams don't. I would like to share my experience with you.


Let's Try Pair Programming

Some time ago our little Java team was merged with another scrum team that works on the same functionality area but mainly uses Bash and Perl. In this sprint all the tasks landed to the code unfamiliar to the previous Java team members. As we have good experiences on pair programming we suggested it would be a good way to get familiar to the code and get the tasks done. We were wrong.

The Pair Programming Just Did Not Work

We tried pair programming with different tasks and different people. To our surprise it was not fun anymore and it was not effective. It seemed to slow down the work, confuse people, mix up responsibilities and decrease the quality. Some people felt that they did not get enough keyboard-time while others felt that they would have been faster on their own.

At first I was dumbfounded, but then I paired with a colleague from my previous team. We had previous experience of pair programming together, but we were suddenly having similar problems too. What had gone into us? After a while I realized what we did differently.

Test Driven Development (TDD) Was Not Used

What has TDD or any other XDD-method to do with pair programming? A lot actually. After this experience I believe that test/behavior driven methods enable pair programming. However I looked at the problems that we had, I realized that we would not have had them if we were using TDD. Here are some examples.


How Do I Know What He Is Thinking?

"Doesn't that regular expression leave out the thing X that we want now?", I asked from my pair. "Oh no, I'm not parsing that thing right now. I'm parsing Y first then we do X", he replied.

I could see the method names and knew what the code did when he got it ready, but I could not see my pair's intentions and reasons while he was doing it. Of course I could always stop him and ask, but then he would need to stop and cut the flow to explain me. And when I did stop him, we ended having long discussions about how the task should be implemented.

TDD - "Let The Red Light Guide You"

If we had been doing TDD, we would have started by running the test cases. There would be a red one, the failed case he would be fixing. I would read the title of the case and know immediately what we wanted the new code to do. He would open the test case and I would see what aspect of the code is failing. Then he would just move to the code and start fixing it. No lengthy discussions. And if I would have been there from the start, I would have seen how he constructs the test case and that would tell me even more about his intentions.


Is It My Turn Already?

I had been watching him code quite a while. I had already asked for my turn five minutes ago, but he just did not get it to stop. I could see that he had something more in his mind that he was afraid he would forget if he would not do it right away and he mumbled something along. So well... I could wait...

And when it was my turn, I noticed I did the same thing - we all did. We suddenly did not want to let go of the keyboard.

TDD - Each Test Is a Mini Task

That says it, doesn't it. Natural stopping places and a reminder to tell you what you were about to do.


What Did You Do Last Evening?

We have different working time preferences and we have meetings that cut the day every now and then. That means we either need to pause the task or program on our own. With our schedules pausing is not a real option. So whenever either one of us jumped in, the other had to explain what s/he had done in the meantime. Whenever one of us was leaving, we had to make deals again. At the end of the task we were really confused about what got done and what did not.

TDD - "Let The Red Light Guide You"... again

We were programming when my pair said he needed to go somewhere. "I'll be back in an hour", he said. "Well, I want to leave right after four today, so I won't be here when you will come back", I replied. "No problem, you can just continue here on my PC. Lock it when you leave", he said. And so we did. He came back later on that evening and picked up from where I left. Next morning I came to work early and continued where he left. That is something you just cannot do without tests that you can easily run to see what all has been done and where.

It is a good practice to leave a failing case behind when you have to stop working (but cannot commit to version control yet). That way your pair (or yourself) will know where to continue.


How Do We Know Everything Is Done?

Well, we are supposed to try it out, of course. But we are being agile now so there is no test plan. For some reason these tasks are not of small size either, like they used to be. We tried out the code and it seems to work, but I'm not really sure it if does all that is needed. I'm not feeling confident about this code. Maybe we just forgot to implement something.

TDD - Test Is The Specification

How we used to solve this was to implement two levels of test cases to guide us.

While I would be implementing the functionality our architect-tester would implement the higher level test cases for it. Higher level test cases describe the wanted behavior. Sometimes we planned the tests together in the start of the sprint and the test steps became implementation tasks that we then put to our backlog. In best sprints he worked a bit ahead and the test was ready before we started the actual programming.

Did you notice how I said "Architect-Tester"? As the tests are the specification, tester and architect roles merge nicely in TDD. Esko Luontola has written a blog article about this way of thinking: TDD is not test-first. TDD is specify-first and test-last

The lower level test cases would be the module test cases that we did while programming. They provide us a list of mini tasks that got done.

Then we could just look at the module test case list and see that everything is done, and run the higher level test cases to see that the functionality as a whole does what was wanted.


Why Don't We Use TDD Then?

When we started working with Bash and Perl code we asked the rest of the team why TDD was not used already. We were told that it had been tried and found unsuitable. We were stubborn enough to try again and quickly found out we agree.

What you need for TDD:
  1. Time to learn the method
  2. Good code refactoring tool
  3. A language that supports encapsulation
  4. Continuous integration that runs your tests every time code is changed
  5. Testing tool/framework

What, you thought you only need the testing tools and the right attitude? That is true for your home projects where you are the only one touching the code and it does not need to be maintained for years by several different people. But for the large scale commercial projects the situation is different.

First and the most important showstopper is often time. Learning to use TDD fluently takes about a year and during that time you will be slower than normally. If we had not known the method already, we could not have even tried it - the schedules are just too tight!

The biggest showstopper in our case was the missing code refactoring tool. Refactoring in TDD is not an optional step and it is needed especially in the test part of the code. You will find that you often need to pull common parts away, combine or split methods, rename and move stuff. One of the best things in TDD is that it helps you to create code that is very easy to refactor. If you cannot refactor, you will end up with  fragile and unmaintainable code base.

If you are working with old code that was not done with TDD, you need to make the code testable before you can work with it using TDD. That is done by carefully refactoring parts of the code so that you are able to test just the part that you need to change. If you don't have tools that let you refactor code safely and effectively, you just cannot use TDD.

TDD also makes the code highly modular and it quickly becomes unmanageable if you cannot hide the parts that are not meant for public use. I can guarantee you'll end up with "ravioli-code" if you don't have encapsulation.

If you have more than one person working with the code then you need your test cases to be run automatically every time you change the code. Otherwise you'll find that someone has made changes to the code, broke the tests and now you'll have to fix them.

Previously I thought test framework a must too, but most programming languages have features that enable testing them. But that must be mentioned anyway and a good testing tool makes the process more fun.


Conclusion

The common understanding seems to be that agile offers us a box of tools and methods from which we can just cherry-pick the best for our purposes. Now I understand that agile tools have dependencies to each other. What tools become unusable if we don't select certain others?

I think TDD is an enabler for lots of things in addition to pair programming. Are your scrum meetings a chore and feel unnecessary? Maybe your tasks don't connect to what you are actually doing?

I think that there is a real risk of just playing an "agile show" if TDD is not used. You look agile outside, your builds automated and green, but what happens inside is not agile at all - it is just a show with no go. It will feel stupid and unmotivating to the people who are forced to play the show.

January 10, 2012

Observer: Part IV - Generic Observer

This is continuation from Observer: Part III - Java EventListenerList.

Sorry for the delay. I ditched the original ending for this series and did not have time to write a new one until now. I was planning to finish by presenting you generic versions of the original Java Observer/Observable and teaching how to use them. But I changed my mind, since the interesting part about generic observer is not how to use it, but how to write it. And that is the main issue today.


Warning for the beginners! 

If you have been reading through the previous parts and are here to learn how to use the observer pattern, the generic observer is not really the best way. You are better off by sticking to the stuff that you can find in the previous parts. While making the pattern implementation generic does solve some of it's problems, it does not change the fact that the Observer is a concrete class that you have to inherit. Also, you would need to take these classes into your own code base, to keep and take care for the rest of your application's life.

Experience is about knowing when it's best to keep things simple, and when it's worth to drive for a perfect solution that is often more complex. We are dealing with a simple pattern here, why not keep the solution simple too?


You have been warned. The perverts who want The Perfect Observer Pattern may continue...

If you really, really want to bring in some complex stuff to your code-base, there is even MORE complicated solution you can use for the observer pattern. In his blog, Steven Jeuris's has solved the same problem by using dynamic proxies. The solution is beautiful and uses both generics and reflection. Just a type of thing your average maintenance person is going to loooove! Just don't use it in performance-critical places as doing things through reflection is quite slow.

Steven also promotes PerfectJPattern that has generic implementations for several design patterns. Each pattern  is nicely explained with code examples, see the Observer, for example. I just stumbled into that, but it might be interesting to check how it does things and if it's good, maybe run it through the legality checks to see if it's published under a licence that the company I work with would allow. Most big companies cannot just allow you to bring in new third-party libraries without legality-check. The reason is that some open-source licences make their user applications open-source too.


Prerequisites

While the previous parts of this series have been beginner-level, this one is not. It's not that this would be much harder to understand, just that you need some background information. So if you are a beginner, I suggest that you go through this Java lesson on Generics before continuing.

If you are an expert in generics, you are going to have some good laughs on my behalf.

If you just stumbled in without reading the previous parts, you might want to check the Java implementation of the Observer interface and an Observable class to go with the interface. Those are the classes that we are rewriting here.

Are you ready? Let's start.


Defining the signature

When you start designing generic classes, it's always best to begin by thinking about their signatures. In this case the question is - how many generic arguments do we need here?

Fortunately, the answer can be found in the book Java Generics and Collections, page 137. It gives us the following signatures:

public class Observable<S extends Observable<S, O, A>, O extends Observer<S, O, A>, A>

public interface Observer<S extends Observable<S, O, A>, O extends Observer<S, O, A>, A>

That is nice.... huh? S stands for Subject, O for Observer and A as Argument.

It does come with a warning: "So you might wonder: could the declarations be simplified by dropping the type parameter S from Observable and the type parameter O from Observer? But this won't work, since you need S to be in scope in Observable so that it can be passed as a parameter to Observer, and you need O to be in scope in Observer so that it can be passed as a parameter to Observable".

I guess I should have read the first 137 pages also, but I was just googling around and I did not understand the warning. There was no further explanation and I did not see any obvious reason why the extended classes should be in scope. So my little caveman brain told me: "Me more smart, me does this better, more simple".

What IF we try to do this with one generic argument only? That would give us the following signatures:

public class Observable<A>

public interface Observer<S extends Observable<A>, A>

Yeah, yeah, I know it won't work, but let's just play we don't know it yet.


Trying out with one generic argument

Here is the signature for Observer's update method:

void update(S observable, A arg);

At the same time Observable would manage its Observers like this:

public void addObserver(final Observer<? extends Observable<A>, A> observer)

As we implement our Observable we will stumble into a problem in notifyObservers(A arg) method. We need to call the update(S observable, A arg) method of the Observer, but it does not accept our Observable as an argument. Why not, the definition for S was <S extends Observable<A>>? Typing is very strict in generics and the Observer wants something that extends the Observable. Observable extends Object, so based on the generics rules it is not valid argument for update method. I have not yet found syntax in generics for saying: "<? extends Something, but Something will do too>", and that problem actually does not go away by adding more generic arguments. So I just ended up using plain, non-typed Observer in notifyObservers(A arg) method in order to be able to pass Observable.this as an argument.

But this little incident draws our attention to typing. Our Observer accepts items that are of type <? extends Observable<A>>. So is it possible that we end up passing incompatible argument to update method? Actually, in this one generic argument solution, it is. Take a look at this:

    public class Koala extends Observable<Boolean> {

        public void setHungry(final boolean isHungry) {
            setChanged();
            notifyObservers(isHungry);
        }

        @Override
        public String toString() {
            return "koala";
        }
    }

    public class Tiger extends Observable<Boolean> {

        public void setAngry(final boolean isAngry) {
            setChanged();
            notifyObservers(isAngry);
        }

        @Override
        public String toString() {
            return "tiger";
        }
    }

    public interface KoalaObserver extends Observer<Koala, Boolean> {

    }

And here is our test case that shows the problem:

    public void testMixedObservers() {
        final Tiger tiger = new Tiger();
        final KoalaObserver koalaObserver = new ZooKeeper();

        tiger.addObserver(koalaObserver);
        try {
            tiger.setAngry(true);
        }
        catch (final ClassCastException ex) {
            assertEquals(
                    "org.beyondhc.lessons.observer.genericwithonearg.GenericObserverTest$Tiger"
                            + " cannot be cast to "
                            + "org.beyondhc.lessons.observer.genericwithonearg.GenericObserverTest$Koala",
                    ex.getLocalizedMessage());
        }

    }

As you can see, we can add KoalaObserver to Tiger. Tiger will happily accept any Observer that extends Observable<Boolean>. As we cannot check for the correct type, we get an ugly ClassCastException. This is definitely to the worse from the original solution. It is clear now what the warning in the book meant - unless the Observer knows the Observable and vice versa, we cannot be sure that the user does not make similar mistakes than with the non-typed variants. Although I was not able to repeat this particular problem with a version that uses two generic arguments, I am pretty sure someone smarter than me can get it broken too. Thus, let's forget trying this with less arguments and return to the three-argument version.


Observable and Observer with 3 generic arguments

When the signatures are set, the rest of the implementation is not that hard. But the signature does look complex, so you might be wondering about the usage. And doesn't it make this thingy really non-flexible if Observable is so tightly coupled with its Observer?

It turns out that this tight coupling actually forces you to better design. First, it makes it more tempting to implement a separate interface for the Observer:

public interface TigerObserver extends
            Observer<Tiger, TigerObserver, Boolean> {

    }

And secondly, as the Observer can implement only one interface, you are forced to composition if you want to observe more than one type of items:

    public class ImprovedZooKeeper {

        private final TigerObserver tigerObserver;

        public ImprovedZooKeeper() {
            this.tigerObserver = new TigerObserver() {

                @Override
                public void update(final Tiger observable, final Boolean arg) {
                    // Do your actions here
                }
            };
        }

        public void observerTiger(final Tiger tiger) {
            tiger.addObserver(tigerObserver);
        }

    }

And the composition effectively hides the update method so that only Tiger may call it.

The Observer and Observable discussed in this article can be found in the BeyondHc util package.

The broken ones with 1 and 2 generic arguments are available too. They can be found on the test side of the BeyondHc, in packages
observer.genericwithonearg
and
observer.genericwithtwoargs.