Psychology of TDD

I joined the software development team of a mid-size company in 2002. It was, and still is, the market leader in producing a special type of measurement devices. The team has a modest size consisting of some sophisticated engineers who have abundance of knowledge in their application domain along with the software development area. I was recruited as a component expert and started with refactoring and partly reengineering a mammoth code base which was 10 years of continuous development. The software was initiated with ad-hoc methods and features were added gradually over the years. Though this was an object oriented software programmed in C++ with MFC/COM/ActiveX, the principles of Object Orientation were rarely followed. But fortunately, like many other systems in the world, it was working and was the main cash cow of the company. The uniqueness of the product was the main ingredient of its extreme success in the measurement domain. It was the principal contributor to the substantial year-to-year growth of the company.

The software had a release cycle of eight months. The amazing fact was that it was never delivered in planned time. The system testing phase before the release was highly time consuming and alarmingly increasing with new releases. The development cost increased with the time. I started to investigate the root cause of the bottlenecks. I have compiled my findings in this post.

My colleagues were undoubtedly brilliant. But most of the programmers, they were very reluctant to testing. The unit tests were done only to check logic correctness. The integration was done early and often, but the integration tests were performed only as late as just before system test. There was no automated testing process. Most awkwardly, there was no full-time software quality assurance (SQA) position. It took a lot of time to convince the management that "Developers are unsuited for testing own code ()".

At that time I was using xUnit for my C++ code and JUnit for the unit testing. My high interest for test driven development (TDD) naturally drove me to quest for an automated test process. One of my major responsibilities was to design, code, maintain and extend the data access subsystem of the software. So I generated unit tests for the new features and wrote the code. The tests were automatically run at night in the build system. The attempt was an immediate success. My next step was to propagate my knowledge to the coworkers and influence them for TDD. It came out much harder than I thought initially.

It is all about human Psychology

TDD is to generate the test cases first before starting your first line of code. The test will fail as long as there exists no code. This is in direct conflict over classical approach of writing code and then testing the correctness of the code. Developers need a different mindset to adapt to this new paradigm. Once they get used to the system, programming is fun.

Why should a company change their usual process if it is working? Why should the engineers learn a new paradigm if they are experts in the respective application domain and their future is almost secured? This is obviously different from when a company unequivocally decides to apply some processes in its business strategy. I needed some diplomatic approach based on performance proofs. I took one simple product and instrumented some metrics to measure the work hours spent in a release. The same measurements were repeated in the next release now with test-driven approach. The outcomes of the two measurements were compared against the planned work hours. I was able to show that TDD saved 30% of our test phase. Our test process was consuming 25% of our release cycle. So 30% reduction in testing amounts to 7.5% savings of complete release. It a very convincing results for this simple application. We have later found when combined with frequent integration tests, the unit tests bring must better result.

TDD needs a different mindset. The epic book "The Psychology of Computer Programming" by Gerald M. Weinberg has encouraged me to closely investigate how my coworkers react and adapt themselves to this new paradigm. At the end, I found very interesting results.

1. You Know Unit Testing:
   Implicitly or explicitly every programmer needs to do unit tests. It can be done running the program and manually checking it is doing what you wanted it to do. This should be done for every single unit of code you are writing.
   Check what you write is right
2. You Know Test is Recurring:
   Even for a simple program, it is extremely difficult to exhaustively remember the dependency of a statement over others. Changing a single statement at some code point may break your program at some other point. If you believe you are God, you don't need extra care for dependencies. In that case you don't need to program either. The process of continuous testing of program for bugs is know as regression testing.
   You know regression testing
3. You Understand Automatic Test Execution:
   So you are convinced of regression testing. It is a recurring process. If your company is ready to give out infinite amount of money only for testing and you enjoy testing more than than programming, you are welcome to do so manually. In that case you better ask your company to move you to the quality assurance team and concentrate on system testing.
   
   Your can't certainly afford ever increasing number of the tests manually. You have to find some automated mechanism. The classical approach is to use some interpreted language like batch, vbscript, bash etc. to write test scripts. If you love more control over test process continue reading.
   Automate your tests
4. You Accept the Importance of Systematic Approach:
   For a simple program, script may serve you enough. But for complex piece of software you need some engineering approach. Otherwise, you might need unit testing again for your test scripts and this cyclic requirement cause you to fall into an infinite loop.
   
   There are various frameworks available for systematic unit testing in all major programming languages. The xUnit framework is the most widely used among them. The name is derived from widely successful JUnit for Java. There are currently many similar frameworks on other languages as well. I use CppUnit for C++ for quite a long time. NUnit is the first to come for the unit testing of .Net programs including C#. But Visual Studio 2008 integrated testing framework is also very good. A comprehensive list of unit testing frameworks can be found at http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks
   Know appropriate unit testing frameworks
5. Mocking is not mocking:
   As a component engineer, I have always problems to find the perfect granularity to fire up my tests. When a new component is designed, we define the interfaces first, then the abstract classes, finally we can create concrete classes. The interfaces and abstract classes are non creatable, so we can't instantiate them. We can't test them until we implement the concrete classes. The "Test Early, Test Often" principle of TDD can't be applied to non existed components. Let me give another scenario. You are designing some facade objects which are supposed to connect to some other systems. They are good candidates for mocking so that you don't always need a live connection to test your components. My most favorite candidates for mocking are graphical (GUI) elements. They are usually difficult to test in automatic unit testing process and need manual intervention. Mocking them enables unit testing without human interaction.
   Mock early, Mock often

In a separate post, I will explain how mock based unit testing can be easily set up.