Visual Studio and TDD: Better late than never

After hearing that UML is touted as a “next big thing” in Visual Studio 2010, I must admit I was less than elated. Since I am hardly a “new kid on the block”, I freely admit that I remember that quirky diagramming tool called Visual Modeler that shipped with Visual Studio 6.0. (“Ten years after” already?). Had it been 1999, I guess UML might even sound, well… intriguing.
Fortunately, I recently came across this video:
TDD with Visual Studio
Believe it or not, but 2010 version of Visual Studio should finally provide a lot less friction for TDD developers. It can generate class and member stubs based on client code. More surprisingly, there is an integrated Test Runner that does not “fall apart” (just pick your song!) if you have tests written is some 3rd party unit testing frameworks. (On the video Karen shows executing MbUnit tests with VS Test runner.)
TDD is hardly a news these days, but hardly feels as passé as UML!

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • Reddit
  • Digg
  • StumbleUpon
  • Bloglines
  • Google Bookmarks
  • Y!GG

YAGNI is not SLACKING!

YAGNI is a clever rule. It says that by doing less your are doing yourself and everyone else a favor. In some schools of Agile Thought like Lean it has become a whole philosophy (although they are more keen on some strange sounding Japanese words, mind you).

Only a decade or so ago, general approach was quite different. You’d try to make your system as encompassing, flexible and configurable as possible. You’d design your system for tomorrow’s needs.

I will give you a quick example. Imagine you need to program a feature on your website where you export some data to an excel file. In the old days, you’d analyze the problem, design your classes let’s say in a form of hierarchy where ExcelExporter and PdfExporter inherit BaseExporter class. You do not need to export data to pdf just yet, but you think you might need it some day, so you better be ready.
These days, however, you live in the present. You just program ExcelExporter. If one day you need pdf export feature, you will reorganize your classes so that both ExcelExporter and PdfExporter inherit BaseExporter class that contains some features common to both child Exporter classes so you can avoid duplication. If you never come to need this feature, you leave your ExcelExporter alone. You will implement new features as they are needed: just-in-time.

But there is a catch. In order to be able to move just-in-time, you should have a well designed, mercilessly refactored code covered with tests. Without refactoring or automated testing you are probably better of doing things the old fashioned way.
I think YAGNI is great and I try to follow YAGNI mercilessly. However, sometimes I hear YAGNI principle invoked in a way that it is clearly misinterpreted. For example “Maybe you do not need to refactor this code just yet” or “Maybe you do not need all those unit tests”. Thing is, to be able to do YAGNI, you need to have your code refactored and covered with tests. You need continuous integration and automated builds. Without these practices, once you need to implement a new feature in JIT fashion, things will inevitably start to break. One way to avoid “Maybe you do not need good quality software” kind of dilemmas is to make practices like TDD, refactoring and continuous integration integral part of your development process. Then, there is no need to think that you might leave out unit tests; since you are doing TDD these are in place already; since you refactor all the time, then there is no way to leave it out.

Remember, YAGNI applies to features, not to quality! One way to use YAGNI properly is to think about Technical Debt. Is the decision NOT to do something resulting in Technical Debt? Technical Debt has to be payed off with interest and with software rates are extremely steep. If you are getting into debt, you are not YAGNI, you are plain’ SLACKING!

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • Reddit
  • Digg
  • StumbleUpon
  • Bloglines
  • Google Bookmarks
  • Y!GG

Another Free C# Refactoring Tool from Devexpress

I just realized I have failed to mention this on time, but I think the news still deserves a post. Developer Express have released another free C# tool that includes “a fresh selection of hand-picked features taken from CodeRush and Refactor! Pro.” You can download the tool from this URL and read more about the features it offers here.

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • Reddit
  • Digg
  • StumbleUpon
  • Bloglines
  • Google Bookmarks
  • Y!GG

Map two classes with one-to-one relationship to a single table in LINQ to SQL.

Some time ago I came across the following problem. After performing a refactoring on a class and extracting associated class, I had to make my classes map to a single table in database. For that purpose I used the usual  Association attribute and ended up with a code akin to this:

[Table()]
    class Customer
    {
        private Address address;

        [Column(IsPrimaryKey=true, IsDbGenerated=false)]
        public String SSN
        {
            get;
            set;
        }

        [Column]
        public String FirstName
        {
            get;
            set;
        }

        [Column]
        public String LastName
        {
            get;
            set;
        }

        [Association(ThisKey="SSN", Storage="address", OtherKey="SSN")]
        public Address Address
        {
            get
            {
                return address;
            }
            set
            {
                address = value;
            }
        }

    [Table(Name="Customer")]
    class Address
    {
        [Column(IsPrimaryKey = true, IsDbGenerated = false)]
        public String SSN
        {
            get;
            set;
        }

        [Column]
        public string Street
        {
            get;
            set;
        }
    }

        static void Main(string[] args)
        {
            DataContext context = new DataContext("Data Source=XYZ;Initial Catalog=TEST;Integrated Security=SSPI");

            Customer customer = new Customer{
                SSN = "123",
                FirstName = "First",
                LastName = "Last",
                Address = new Address{
                    Street = "My Street",
                    SSN = "123"
                }
            };

            Table customers = context.GetTable();
            customers.InsertOnSubmit(customer);
            context.SubmitChanges();
        }

//DB DDL
CREATE TABLE [DBO].[CUSTOMER](
        [SSN] [NVARCHAR](50) NOT NULL,
        [FIRSTNAME] [NVARCHAR](50) NULL,
        [LASTNAME] [NVARCHAR](50) NULL,
        [STREET] [NVARCHAR](50) NULL,
 CONSTRAINT [PK_CUSTOMER] PRIMARY KEY CLUSTERED

As you can see, a single Customer has a single Address. Customer with corresponding Address maps to a table Customer. Everything works well, until you try save new instances. You get the following error: Violation of PRIMARY KEY constraint ‘PK_Customer’. Cannot insert duplicate key in object ‘dbo.Customer’. If you try to execute the code and you should be able to reproduce the error.

Solution

In the end, it is all just about method call order.
The solution is the commit the root object first (Customer) in our example, without related object (Address). This way, the instance of related object is created also, with all properties empty.
Second step is to set the properties of related object. Here the important detail is not to create the instance of related object, since it has been already created by linq2sql. Then, you commit changes again and this time the related object is also updated and saved.
It would probably not work if you had to enforce not null constraint on some of the related object properties.

Here is the same example fixed to work without error.

class Program
    {
        static void Main(string[] args)
        {
            DataContext context = new DataContext(
                "Data Source=XYZ;Initial Catalog=TEST;Integrated Security=SSPI");
            Table customers = context.GetTable();
 

            Customer customer = new Customer
            {
               SSN = "SomeSSN",
               FirstName = "First",
               LastName = "Last"
            };
            customers.InsertOnSubmit(customer);
            //Save customer first, this will also create the Address
            context.SubmitChanges();   

            //Now Address is not null anymore, so it will be updated
            //Linq2SQL will not try to insert new row
            customer.Address.Street = "Updated";
            context.SubmitChanges();

            //get one instance for demo purposes
            Customer saved = (from all in customers
                                 select all).First();
            Console.WriteLine(saved.Address.Street);

        }
    }

    [Table()]
    class Customer
    {
        private EntityRef
address;   [Column(IsPrimaryKey=true, IsDbGenerated=false)] public String SSN { get; set; } [Column] public String FirstName { get; set; } [Column] public String LastName { get; set; } [Association(ThisKey="SSN", Storage="address")] public Address Address { get { return address.Entity; } set { address.Entity = value; } } } [Table(Name="Customer")] class Address { [Column(IsPrimaryKey = true, IsDbGenerated = false)] public String SSN { get; set; } [Column] public string Street { get; set; } } //DB DDL CREATE TABLE [dbo].[Customer]( [SSN] [nvarchar](50) NOT NULL, [FirstName] [nvarchar](50) NULL, [LastName] [nvarchar](50) NULL, [Street] [nvarchar](50) NULL, CONSTRAINT [PK_Customer] PRIMARY KEY CLUSTERED
Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • Reddit
  • Digg
  • StumbleUpon
  • Bloglines
  • Google Bookmarks
  • Y!GG

Code, aesthetics and symmetry

It might sound funny to mention aesthetic qualities like symmetry when speaking about source code. It is exactly the principle that Kent Beck mentions in his book Implementation Patterns. It has to do with inner workings of our brain. These qualities are the way to communicate with fellow programmer on another level. I’d say symmetry might be put under more general “Consistency” principle that Abrams and Cwalina are so keen about. And it is definetly in the spirit of Principle of least surprise.

What got me thinking about it is a bug I investigated recently that had to do with two base java classes: java.net.URL and java.net.URLConnection. The symptom was a generation of leaking file handles, resulting in server hanging after few hours of operation.

Take a look at this java code:

           URL url = configuration.getURL();
           URLConnection connection = url.openConnection();
           long date = connection.getLastModified();

Now, I’d typically put method open on the connection object. If you take a look at URLConnection, you will see that the class does not have the close method. That’s all right, following the symmetry principle, it would be then consistent to place closeConnection method inside the URL class. Alas, neither the URL has the method that closes the connection!

This was the source of our bug. The programmer had to open the connection in order to read getLastModified. But, since neither URL nor URLConnection had the close method, he “forgot” to close the connection. The solution is to do the following after reading the date:

InputStream stream = connection.getInputStream();
stream.close();

No wonder the programmer forgot to close the connection; there was no obvious method he could call to do so. I haven’t bothered to take a look at the code of these classes, but it seems unlikely that there was no other option when designing them. Had the authors thought about symmetry when creating URL and URLConnection classes, a nasty bug could have been averted. 

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • Reddit
  • Digg
  • StumbleUpon
  • Bloglines
  • Google Bookmarks
  • Y!GG

Dealing with legacy code

Recently someone on a group on internet asked for some ideas and suggestions on refactoring legacy code. Not your usual refactoring, but the project consisting of refactoring legacy code.  Now, I guess the first thing that you might ask at that point is “what is legacy code anyway”?
According to some TDD purist, legacy code is any code that is not covered by comprehensive set of unit tests. Generally, legacy code is some code in advanced lifecycle phase, burdened by heavy debt, sprinkled with smells like long methods and large classes and often implemented in some older technology, language or language version.
Here is what I replied:

Hi,

I’d say you are in for a ride.
Here are a few thoughts based on my experience.

First of all, do not discard rewriting application from scratch without giving it a really good consideration. In my experience, it is often much easier to develop an application from zero when compared with refactoring legacy version; it is also much more difficult to refactor the legacy code than it might look at the first site. If your team doesn’t have prior experience with refactorng of legacy code, it’s all too easy to underestimate the task. Do an experiment. Choose a component or a class and see how long does it take to refactor it. On one occasion, I performed one such experiment. I was able to reduce a 4000 LOC component to a 2000 LOC component in a period of two weeks (eliminating dead code, duplication etc). 2000 LOC that were left over were still very far from being in any decent shape. We decided to go for developing the project from zero and to discard old code altogether.

On another occasion, I had to deal with some really old java code. Project contained proprietary implementation of XML parser. Not too much open source in the days when they created the application. Honestly, not a kind of code you wish to look at when maintaining something that is a typical enterprise application. Solution to this proved to be relatively easy; while XML used was not 100% well-formed, it was possible to replace it with 3rd party XML parser.

This leads us to my next thought. Start by refactoring on architectural level. Replace whole layers of application with 3rd party solutions when possible. For example, ORM tools and frameworks are only recently hitting the mainstream. It is not very likely that your application is using it. If possible, replace the persistence layer of coded SQL with some ORM tool.

The third suggestion. Know your problem domain but don’t use your legacy code base for the purpose of understating it. Reverse-engineering legacy codebase into domain knowledge is futile task; legacy code is often littered with duplication, dead code, poorly named elements etc. Try to understand the domain by speaking to domain expert/customer. Without domain knowlege you will find yourself in complete darkness. Maybe it is a feature, maybe a bug or a dead code altogether. Take a look at this story:
Refactoring Finds Dead Code

Finally, build up some reliable testing harness. Start by creating functional/integration tests and then add unit tests wherever possible.

You might wish to take a look at this article of mine, it talk about legacy VB apps, but many points are valid for any legacy code:
Moving up the technology stack: VB6 migration reality check

Good luck!

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • Reddit
  • Digg
  • StumbleUpon
  • Bloglines
  • Google Bookmarks
  • Y!GG

Collaboration, reward and “wisdom of crowds”

Recently I finished reading Wisdom of crowds. The basic tenant of the book is that groups are capable of excellent decision making. People also seem to be endowed with capacity to coordinate and cooperate. Ability of groups of informed laymen to make correct decisions can surpass even experts or small groups of experts. This, of course, provided some conditions are met: groups have to be diverse and decentralized and some mechanism for them to make collective decisions have to be put in place.
This is well illustrated by the following opening story from the book summed up at Wikipedia in the next sentence:

“The opening anecdote relates Francis Galton’s surprise that the crowd at a county fair accurately guessed the weight of an ox when their individual guesses were averaged (the average was closer to the ox’s true butchered weight than the estimates of most crowd members, and also closer than any of the separate estimates made by cattle experts).”

A good case for direct democracy? Might as well be. Think only about this fact: today, in most democracies in the world people choose representatives and governors for periods of 4 years or more. When people are elected, they are generally elected in a certain “context” (historical, social, economical etc.). Take a look at Madrid bombings for example. There is no doubt that they had a decisive effect on elections that followed. Had the bombings occurred only 4 days later, the result of election would be radically different according to all polls. Would George W. Bush win the elections had they been taken after 9/11? I am well aware his approval ratings went berserk after the attack, but at that point he was already elected. Enough about politics…
Internet is play-ground par excellence for groups and collaboration on a large scale. Today we take Internet for granted. Still there is a number of interesting phenomena that occur on the Internet and that are closely related to human nature and society in general. Take for an example newsgroups. One person helps another, a complete stranger, and in most cases without any reward involved in the process. Quite different to how the companies in the free market behave and compete. Imagine companies sharing patents or company secrets. Well, I guess this is not so easy to imagine.
Just look at impromptu collaboration that happens on open source projects. They can involve a huge number of geographically and culturally disperse collaborators; still they can be surprisingly successful. All of this without any complex organizational structure, only slight or merit-based hierarchy etc. With Agile, such practices are getting their foot into the commercial enterprises (Self-organizing Teams).

Experts Exchange vs. Stack Overflow

What made me think about the book again is my experience with a site called Experts Exchange. The site lets users ask the questions and then the experts provide the answers. The one that asks the question assigns question points, based on the question difficulty and based on urgency he has in getting the question answered. These points are then awarded to or split between experts that provide the answer. Decision on to whom the points are awarded is up to questioner. In the case of conflict, a moderator can be asked to mediate.
When I started using Experts Exchange, points were earned by providing answers to questions made by others. You also earned some points for becoming a member, to start you up and then more point with time. They used to lock questions once someone posted the answer. The question would be locked until the person that asked the question responded if he accepts the answer or not.
Later on this has changed, so instead posting answers, all experts post comments. This way the question cannot be locked and new question momenum is not lost, nor people have to wait to give answer or post comment. The questioner then awards points to the comment he thinks is the best answer. If he thinks more than one answer deserves points, he can split points and award them to more than one answer.

Another change that occurred is that now you can pay to use Experts Exchange. You can still earn points and access site based on merit. If you pay for Premium Services you have unlimited number of points at your disposal and some other benefits, like private knowledge base. A maximum that can be gained from a single question is 2000 points. Besides points, there is also community rating system in place:

“To provide you with a tool to use, we rate our solutions based on various elements that most accurately determine if a solution is a quality solution. To explain what factors affect the solution rating, here are the elements we take into consideration when formulating our solution rating.
•    The Grade of the Solution
•    The Zone Rank of the Expert Providing the Solution
•    The Number of Author and Expert Comments
•    The Number of Experts Contributing
•    The Feedback of the Community”

I think ratings were also added to the site later on. Note that ratings are applied to solution (question + all the answers as a whole).  Ratings will by no means influence how many points are earned/given away/lost. I remember this worked well and I even participated answering questions made by others earning points. Going back recently to Experts Exchange, I got a feeling there has been some change in the way things are done since the early begginings and I don’t thinkg that comercialization is the only factor. I got a feeling there is somehow too much noise that happens before you get a valid answer. Often all you get is noise. Stackoverflow also has points and reputation, but it does not work quite the same. More on that later on, let me first finish telling you my experience with Experts Exchange.

Murky water fishermen and bulldozers

Before next section, a disclaimer is in order: What follows is by no means result of some methodological scientific investigation. The sample my experience is based upon is not significant; it is also quite probable that the nature and expertise level of my questions have changed (hopefully I progresses somewhat in the last 10 years!). Finally, maybe it is just my conspiracy theory loving mind playing tricks with me. Anyhow, here we go.
After I had this rather frustrating experience with unhelpful answers on more than one occasion, I started to note a certain pattern. One type of noise makers would make some general or seemingly inoffensive comment, something like “This is interesting question; I would also like to know how it is solved.” Or “I really don’t know how to do that, but you could try looking at documentation at msdn, here is the URL that talks about the subject…”. Here is a real-life example:

“have a look at the configuration api
http://www.hibernate.org/hib_docs/v3/api/org/hibernate/cfg/Configuration.html

sorry I can’t be more help”

All of this might sound quite inoffensive, in the end its just noise – maybe its not done on purpose. The trick is however, that after some time, if question is not closed by the questioner (and why should one close the question if no acceptable answer was provided?), moderator goes on and after some warning emails closes the question for you. If you do not react, the moderator generally awards the points to those that tried to answer your question. With time, just by posting some hardly useful comments, you start earning points. I call this kind of expert “murky water fishermen”.
Another type of noise was answers that tried to change some basic premise in the question, and to provide alternative way to solve your problem. For example “Why don’t you use Entity Framework instead of LINQ2SQL?” Admittedly, sometimes putting things on their head can be useful, many times the problems are not solved because you keep trying to find the solution along the same lines. Some solutions require that you take a look at the problem from different angle, like the matchstick problem; the solution is only possible if you start thinking in 3D space. This can be especially useful for a newbie, he might not be even aware of the alternative. Most of the time however, you did your homework before coming to Experts Exchange – the basic premise of the problem cannot be changed. I call these, often arrogant coercers – bulldozers. On one occasion, after providing unhelpful alternatives, one of these bulldozers told me “Such thing cannot be done”. After some time, I solved the problem and posted the answer. Bulldozer was quite annoyed, but had to accept the defeat. Moderator gave me back the points. Not that I really wanted the points back, I was just a bit annoyed that this bulldozer had a courage and arrogance to say that my problem was impossible to solve. What surprised me is that moderator also erased the question! I thought that Experts Exchange is also about building the knowledge base (one they often mention), the answer to my question could have been useful to someone else as well, even though the dude that asked and dude that answered the question are one same person.
The first thing that comes to mind is that some mechanism for reducing noise should be put in place. Probably some kind of punishment for creating noise. This is something that Stackoverflow considers. Everyone registered can vote on both the questions and the answers. So, you get more points and better rating even if someone has liked your question, not only the answer. Even more important is the fact that anyone registered can vote on any question or answer. If someone marks your answer as Helpful you get 10 points. If your answer has been accepted, you get only 5 points more. This way, any other member is just as important as questioner. If someone thinks your question/answer was not helpful you lose points. Probably there is less need for moderation; on a large scale any bias is neutralized, there is less possibility to try to hack things like creating ghost users etc. I have not yet registered at Stackoverflow, but the next time I need to ask some question, I will give it a try. And hopefully I will remember to post part 2 of this post, after having some personal experience on how Stackoverflow measures up to Experts Exchange.
Going back to Surowiecki’s book, it seems that Experts Exchange failed one step from unleashing the “wisdom of the crowds”. What lacks is what Surowiecki calls “Aggregation” (Wikipedia: “Some mechanism exists for turning private judgments into a collective decision.”) It is up to the person that asked the question to award the points. There is a rating system in place, but separated from point system. Why not let the community be the judge on the answer value and helpfulness and award points? Also, why not let the answer receive the points more than once? I’d argue that even answer to a simple question that helped people a hundred of times is more valuable than answer to more complex question that helped someone on one occasion. Finally, why not let the questions be awarded points and why not have points taken for bad answers/questions? Using ratings system separated from point system, where contributions are heavily point-driven seems to dilute the value of community feedback.

Wisdom or Maddness?

Surowiecki’s book is an interesting read. It is even close to being practical and I would definitely recommended it if you are into social aspects of web 2.0. The book however, takes pretty much one side to the phenomenon of the crowds. This is not necessarily biased; as a matter of fact even the title of the book alludes to “Extraordinary Popular Delusions and the Madness of Crowds” book by Charles Mackay. Indeed, the madness and hysteria are the sides of crowds we generally have heard of. But by no means have we investigated them enough. One who did investigate it is Wilhelm Reich. Fortunately, The Mass Psychology of Fascism is on my desk and thanks to Internet one of my favorite movies “W.R.: Mysteries of the Organism” just a piratebay away.

Edit: In the meantime I registered at the stackoverflow.com and started participating. While you can decide to offer some of your own points as a reward, this is seldom necessary. The community driven point system seems to be fearer and overall experience is favorable so far.

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • Reddit
  • Digg
  • StumbleUpon
  • Bloglines
  • Google Bookmarks
  • Y!GG

VB6 Migration Today

A few years ago I published an article for Visual System Journal on VB6 migration “Refactoring – elixir of youth for legacy VB code”.
I used “Engine-Collection-Class” pattern based application as a study case and applied different refactorings to upgrade VB6 code to .NET.
VSJ has just published an article follow-up where I deal with migration in context of .Net 3.5 and Visual Basic 2008. If you are interested in VB6 migration and upgrade process, check out my “Moving up the technology stack: VB6 migration reality check” article online at:
http://www.vsj.co.uk/articles/display.asp?id=756
and let me know what you think.

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • Reddit
  • Digg
  • StumbleUpon
  • Bloglines
  • Google Bookmarks
  • Y!GG