The Art of Unit Testing and TDD 101 in C#.NET - Roy Osherove - Key Notes

Roy Osherove, the author of The Art Of Unit Testing, Notes to a software team leader and Beautiful Builds, has online training courses available at http://courses.osherove.com/. I had a chance to watch the first video, Unit Testing and TDD 101, which is available as a free preview of his course The Art of Unit Testing & TDD Master Class in .NET.

Roy spent the first ~30 minutes explaining about unit testing and spent the next hour to demo on how to write unit test in C#.NET as well as the good and bad practices. He hasn'r started demo TDD until the last 30 minutes which I think it is worthwhile to know how to write good unit tests before jump into TDD.

Below are some key notes I did captured from the video (hightlighted in Blue) which I think it may be good to share to eveyone:

Putting TDD in Context

• Engineering Methodologies (e.g. waterfall)
• Agile (flexible & lightweight) Methodologies
• eXtreme Programming
• Test-Driven Development

• Key Notes:
• Unit Testing and Test-Driven Development. Are two different things but related to each other
• Test-Driven Development is about "when" to write those tests.
• If you don't know how to write good tests and do TDD, you may get crappy tests which hurts your project.
• Even TDD was started from XP but you take these activities as it is and implement it wherever you want regardless of methodologies your organization is using.
• TDD is like debugging. It is a personal activity which you can choose to do whenever you want. You always have a choice to do or not to it.
• There were people in some organizations failed the companies when people force them to do TDD and they didn't want to.

A Unit Test is a test of small function piece of code

• Key Notes:
• It is  only pieces of code that contain the logics inside.
• We wouldn't want to write direct unit tests for the code that does not contain logics at all e.g. property setter/getter methods.

Unit Testing makes your developer lives easier

• Easier to find bugs
• Key Notes:
• How do you know your tests have bugs? Are you gonna write test for your tests? How do you know your tests of your tests do not have bugs? This is an unresolvable problem.
• TDD is the solution as your write the test which fail first and then you make the test pass then you actually test the test manually. You know it fails when it should and it passes when it should. This makes sure the tests don't have bug.
• Easier to maintain
• Key Notes:
• Some may think that you have more code to maintain. For example, suppose you have 20 tests for a method. What if you add a parameter to the method? You have to update all 20 tests which take too much time and slow down your project
• This will not be the problem if you know the practices that make the test "maintainable" such as refactoring in the tests themselves.
• This is usually one of the biggest problem of unit testing in TDD that organization tries to implement and gets into maintenance problems.
• Easier to understand
• Key Notes:
• Test is live executable documentation of your code. When you want to get someone fixing a bug in the code, he/she can see the tests to understand everything from that.
• But in reality, tests usually poorly written e.g. bad names or something like test1, test2, test2.1,… that no one can understand. This could happen when people do not pay attention to make the test readable.
• Easier to Develop
• Key Notes:
• If all of above 3 things are true then it is actually when unit testing helps you. But if one of them is false then it doubts the other 2 and unit testing becomes burden.
• Many organizations tried to implement unit testing with TDD and failing as they try to do too many things at the same time and they don't aware these problems so they don't learn how to maintain more maintainable, more readable code. Eventually, they don't trust their tests.
• If you run the tests and get them all in green but you still do debugging to verify then that is when you don't your tests. Then something was wrong. Maybe you've experienced that some tests may be false positive. TDD will help making yourself trust the test you run.
• To wrap up, TDD could fail for some reasons. One of them might be because they get slow down by these problems and another one might be it takes a lot of time to get enough tests.

You have already done Unit Testing

• Not structured
• Key Notes:
• Everybody is already doing unit test nowsaday but each single developer have their own way, their own framework to do unit test. They may create some button or forms which no one else can use. And developer him/herself may not even remember how to test 2 months later.
• Not Repeatable
• Key Notes:
• Most of develops do unit test their code but not in a repeatable way. And that makes developers feel hesitate to add more tests as it takes time to reinvent the wheel by creating those forms or console in their test code.
• Not on all your code
• Key Notes:
• When you write the code, your objective is to prove that it works so you expect it not to fail. This is why QA comes into the process as different person who actually looking for your code to fail.
• With TDD, we have several practices which ensure we don't become too subjective and then we get more tests and we get better code coverage and quality.
• Not easy to do as it should be
• Key Notes:
• However, this is not easy to do. You need practices to learn on how to do that.
• A framework is missing
• Key Notes:
• But the most fundamental shift when the people start testing in automated way is the framework. There are a bunch of frameworks to do the unit testing. Most are open-sourced and free. Some of them (just 1 or 2) are not.

The xUnit Frameworks

• Original was for SmallTalk
• Kent Beck and Erich Gamma
• Ported to Various languages and platform
• Junit, CppUnit Dunit, VBUnit, Runit, PyUnit, Sunit, HtmlUnit, …
• Good list at www.xprogramming.com (Not valid anymore)
• Standard test architecture
• Introducing NUnit

• Key Notes:
• All these frameworks have the same standard architecture. If you know how to write unit test in one language, if you know another language, you should be able to learn easily i.e. how to write unit test, how to structure them, and how to execute them.
• They are all called "xUnit" framework. "x" represent any languages adopting this framework (including ABAP so it should be called ABAPUnit).
• NUnit is for .NET and will be used in this course but you can also apply what you learn here with other programming languages.
• JavaScript have many testing frameworks. The big one is QUnit but it is consider old these days. Now the people talks about "Jasmine" for JavaScript unit test.

How we use the framework

• Write Tests
• Make it easy to create and organize tests
• Reference an assembly, spread some attributes, you're done
• Key Notes:
• The framework gives us a structured way to write the tests, usually methods inside classes. They have special attribute in the methods and the classes.
• Run Tests
• Allow running all of our tests, a group, or just one
• From command line and GUI
• Key Notes:
• The framework gives us a structured way to run the tests. The Test Runner looks for the classes and methods that have such special attribute and execute them all.
• Review Results
• Immediate Pass/Fail feedback
• Details on each failure
• Key Notes:
• If it fails, it tells you which line it failed. What actually failed (expected vs. actual). So you get feedback quickly if something work or not.
• These are working same way across the frameworks.
• Here you can write unit tests which everyone else can run the tests and see the same feedback. So anyone can take action based on it. So we no longer creating/reinventing the wheel.

The 3A Pattern

• Arrange
• Act
• Assert

• Key Notes:
• Normally in .NET, you create a class for testing in a separated project and include required testing libraries.
• Then you put an attribute [TestFixture] so test class get recognized.
• "TestFixture" is the name inherited from JUnit as Java was the first one having xUnit framework. "TestFixture" is not quite meaningful. Roy thinks that it should be "Testing Class" which is more readable.
• The you create a method with no parameter and put an attribute [Test]
• Every tests contains 3 parts: Arrange, Act, and Assert (triple A)
• Arrange is the part where you arrange objects to test
• Act part is actually acting on those objects you arranged
• Assert part is the most important to decide whether the test is pass or failed
• People usually get confused of parameter of expected value and the actual value. Do not mix up!
• You should never ever use the message parameter! If the name of the test is good enough, you don't need to write the message.
• Test names are very important. When you have thousands of test and something was failed, the first thing you see is the test name. The message is harder to read.

NUnit Demo (26:50 onwards)

Key Notes:

Test Project

• You should create tests in a separated project with the same name + suffix "UnitTest". As you may have another test project e.g. "IntegrationTest" which they should be separated for some good reasons.
• We normally don't test the method with only mathematical formula as it does not contain any logic.

Naming Your Test

• Here are how you name your test: It consists of 3 parts of knowledge separated by underscore (_)
1. GIVEN: The thing that you are testing or unit (a.k.a. UUT (Unit Under Test), CUT (Code/Class Under Test))
• It could be a method, multiple methods, or multiple classes.
• It is not a unit in terms amount of code but a unit of work. It is a scenario in the system.
• In this case, it is just one method "Add"
2. WHEN: Scenario = it is explanation of the implementation, not the implementation = "Given"
• In this case, it is "TwoNumbers"
3. THEN: Expected behavior
• In this case, it may be "SumsThemUp" or "ReturnsTheSum".
• If you miss any of these 3 part from the name, the reader would not understand. For example:
• "Add_TwoNumbers" - the reader would think that it added two numbers and then what? Then they have to read the code to understand.
• "Add_ReturnsTheSum" - What should I send in, what is the point?
• "TwoUnmbers_ReturnsTheSum" - What being tested?
• "Test Lint" can help checking what missing in your test code. This add-on is free to use.
• If the name is good enough, it would help you writing the test. For example, when you do a lot of thing in the test like mock up, etc. then you tend to forget what you are trying to do.
• So it is worth to spend amount of time on the name. Sometime it could take 5 - 10 minutes.
• Sometime you don't know how to test something
• Sometime you don't know what you are trying to test
• Coming up with the name is basically coming up of the design of the test in your head and then you can start writing your test. So the name is one of the most important thing.

Test Data

• Sending the same values into two different parameters is not usually a good idea. e.g. Add(1, 1) unless this is a special/specific scenario.
• It is worth to spend time to come up with a good test data to make the test is more readable.
• Add(1, 5) is NOT good as it will make the reader think "Why 5? Does 5 has any special meaning?"
• Add(1, 0) is NOT good as zero has usually special meaning. Don't use zero, if you don't have to.
• Add(1, 2) is good/better because it is the first number after zero and the next consecutive number makes the reader see the pattern. From experience, 1 and 2 is one of the most test signature you will see.
• In this case, the value does not matter so 1 and 2 is just okay. If values do matter, then consider to include "boundary cases".
• There are two problems for this : Assert.AreEqual(1+2, result);
• You should not put any logic in the test
• You should not dynamically create the expected value by repeating the same logic in production code in your test. As it will be always equal and you wouldn't find bug in your code if it have had.
• In this case, Assert.AreEqual(3, result); is better.

Test Readability

• The test readability is more important than the test maintainability.
• Roy prefers to use "TestDriven.NET" (free for personal use) to run the test instead of the default one (ReSharper).
• One benefit from this runner is you can choose what scope of tests you want to run
• It is very fast and you can assign a shortcut key to run. The only result you need to know is shown in the output console. You don't care about what are run successfully. It is a waste to show the detail of everything.
• You can double click in the output console to go to the failed line
• You can also debug your test with TestDriven.NET
• Another benefit is you don't need to write test to test your code. With TestDriven.NET, you can even write a method in the same production class as an entry point to test your method and start debugger to run the test. This is helpful for temporary quick test.
• It is suggested to read the test code from the bottom up (Assert first). But after you read the test name.

Test Maintainability

• To make the test more maintainable:
• You may put the Arrange part in the setup method. Put the object variable as the class property.
• But with this, the reader has to jump between the test method and the setup method to read the whole test which we don't want to.
• This can be solved by declaring object right in the test and use the factory method to instantiate the object.
• Now, you have both readability and maintainability in your test.
• Meaningful name of the variable is too risky as everyone has different way to name variable so it is not consistent then you cannot trust the name of variable.

Q&A

• By Roy's experience, the setup/teardown method will be used only when you have singleton object that you need to reset before or after running each test.
• Testing database connection is not a unit test. You may create a separated project called "Integration Test"
• Unit Test has F.I.R.S.T characteristics. No dependency. Don't touch the file or database or etc. So the test is repeatable and consistent and never fail, unlink the Integration Test that you may expect some failures due to dependency.

Test for Exception

• There are 3 ways to write:
1. Use try…catch… This is not a good way
2. Use attribute [ExpectedException…] for the test method - This seems to be a better way. But there is one big issue…you might get a false positive from the test! What if the test fails before your expected line is executed? Any line in the test can fail. So don't use this way.
3. Use Assert.Throws… (delegate { } ); so you have only one line to expect for an exception

Teardown

• If you read someone's test and you see teardown method then most of the time it is not a cleanup of static object but tend to be something that is Integration Test.

Test-Driven Development (01:27:00 onwards)

• Make it Fail
• No code without failing test - Don't write production code if you don't have a failing test. Renaming method is okay.
• Make it Work
• As simply as possible - Make the test pass without touching your test as simple as possible. This leads you to better test coverage
• Make it Better
• Refactor - Changing existing code without changing its functionality e.g. renaming

• Key Notes:
• TDD is the way we write test first
• We might have some design upfront although not a lot but just enough to start TDD as design changes over time
• Roy never sees any real project starts TDD with NO design upfront
• There are 3 types/levels when you learn something new:
1. Copy someone else, almost blind, even you feel stupid
2. To understand why you do the thing you're doing. Why they make sense.
3. To improvise, To actually do thing differently but with the same concept. e.g. naming convention, someone might prefer BDD but those 3 parts are still existed.

See video for demo in this part…

Related Articles

• F.I.R.S.T Principles of Unit Testing