re-invent the wheel gifts, re-invent the wheel gift, re-invent the wheel merchandise, gifts for re-invent the wheel, gift for re-invent the wheel

In the recent town-hall meeting, our vice president of application development mentioned that in her experience, usually applications will only become great in their version 3.0. That’s in terms of feature richness, usability, stability and performance.

This insight fits well with my own experience and observation. The implications are developers are spending a lot of their times doing re-factory, which means rewriting the underline implementations for the existing functions.

At the first glance, re-factory seems to be an awful thing: tedious, time-wasting, boring, but unavoidable. However, if we really want to tame this beast, we need to take a deep look at its origins and natures. And it may come out as a necessary and valuable way to improve the quality and the business values of our applications.

Why not do things right at the first place?

It’s a question lingering in our minds when we take over a project for other people, either the consultants or colleagues who move on to other projects, and we feel the strong need to re-factory the codes. We feel if things are done in the right way, we don’t need to waste time and efforts to re-factory now.

We can blame the previous developers with their laziness or ignorance or lacking of skills or vision. However, if we retrospect our own experience, we will be more understanding.

There are several main reasons:

1. Pressure to market

This is especially true for first iteration of development. At that time, the business wants something out to the market as soon as possible to beat or match the competitors. The managers care about getting things done more than getting things right. They are running a very aggressive schedule and keep pushing things forward. There aren’t many time to waste in figuring out the best solutions.

And they are actually right. Innovation or “first in the market” matters most for new products.

2. No feedbacks

Before a product first goes to Production and welcomes its first user, it’s very hard to imagine how people will use it, how frequently they will use it and how many of them are going to use it simultaneously.

One way to overcome that is build over capacity and very flexible but very complicated solutions, which may eventually turn out as unnecessary.

Another extreme is build simple and straightforward solutions just to satisfy the limited requirements.

Either way will be considered “wrong” in the eyes of later developers who are equipped with more knowledge about the use patterns of the application.

3. Technology advancement

We used to write our own MVC framework cause at that time there is no popular ones available. Now, when we compare that house-made framework with Struts, we find it cumbersome, not flexible enough and not maintainable.

Traveling back on time, the technologies or best practices we have now are simple not available at the time when the application is first developed. We can hardly blame those developers with lacking of knowledge of something doesn’t exist at their times.

Why things are not working now?

The driven reason for re-factory is that things are not working smoothly any more.

Business are either changing to another direction or the same business simply flourishes, bringing hundreds of thousands users. New features are needed to earn big clients or match major competitors. The neglected requirements for performance, high availability and scalability are revived.

Even things aren’t broken. Once we have accumulated better knowledge of the business and the operation and the better technologies are available now, we know we can make things better and faster with cheaper cost.

The moment for re-factory comes!

Re-Factory is Painful

Whoever goes through the full process of a major re-factory project will agree it’s a very painful process.

Several main technical obstacles:

  • Large scale. The sheer number of codes that need to be touched is overwhelming for a large project. That manifest in resources, time and budget. That’s the very reason the re-factory project in one of the places I worked before failed three times.
  • Lack of documentation/knowledge: Consultants, out-sourcing and high turn-over rate make it hard to find people who know all the details of the project. And software developers are notorious in producing poor documentations.
  • Hard to test: Unless the tests are automated, it’s just time consuming to retest all the functional points with all the exceptional branches.

Usually, those technical obstacles will invite many suspicious voices in the management team before launching the project.

Here are some examples:

  • “Why fix it if it’s not broken?” – Lack of vision
  • “It will impact our delivery schedule.” – Resource conflicts – continuous delivery vs. architecture change
  • “Just throw in more hardware” – Hardware can resolve all the problems.
  • “This is huge investment with uncertain results” – Fear of unknown risks
  • “Developers just want to play with new tools” – lack of understanding of the new technology and no trust for developers

Those negative feedbacks are expectable and understandable. Without careful planning ahead of time, a large scale re-factory project can be a huge black-hole that drains up all your resources and times and eventually your credibility.

Solution

Several years ago, we did a re-factory on a large internet application. The project was pretty successful. We completed the majority of the work in 6 months and successfully went to Production in 12 months.

Here are some tips I learned from that project.

Prototype/Pilot

All the re-factory ideas should start small, with some prototype or pilot programs. Don’t just write down the ideas in documents. That means little to the management and the business partners.

A pilot program supported by solid improvement numbers will be more persuasive than any documents.

By the way, since it is small scale, you can easily squeeze some time to crack that out. Don’t need to wait for the green signal from the upper management team.

Tangible Improvements

The re-factory must have some tangible improvement. The vague terms of “improve architecture” or “use advanced technology” hardly fly.

Although the biggest improvements in our project is a more scalable and modern architecture, the actual improvement that earn the credit (and the investment) is the user interface. Our business partners in marketing department simply love the new user interface and it’s improved usability.

In additional to User Interface, anything vital to business and also measurable are ideal. For example, another re-factory project I am aware of improved the throughput by 17 times, cutting the operation duration for one business process from several hours to 10-20 minutes. That’s the tangible result you want to present to the management team.

Dedicated Resources

The worse thing for a re-factory project is a prolonged project cycle. I saw so many projects that last more than 6 months simply die out. Too many changes in the business environment can happen in 6 months. So is the management’s mind.

For a re-factory project to survive, it has to be done as quickly as possible. The key to make that happen is have a dedicated group of developers working on that. If possible, don’t have the same group of developers work on supporting the existing product and simultaneously working on the re-factory project. You will find them spent 60% of the time working on supporting issues. It will only slow things down and eventually kill the project.

 

Involve QA at very beginning

The biggest challenge for re-factory is not development but testing.

If fortunately you already have all your functional tests automated, congratulations, you are in the dream land and don’t need to worry about tests like rest of us. Otherwise, the early you have the testers involved, the better.

After all, re-factory is test-driven development. The QA can bring to the developers the very much needed test cases and business knowledge. Sometimes, developers are having hard time understanding why a piece of logic exists in the existing code. There the QA knowledge comes handy.

Pairing testers with developers along the development process is also very helpful. Usually, re-factory will apply the same improvement idea to the entire application. Having that idea tested early, finding out any catches and fixing them will save big times in later stages, when the same error will be repeated everywhere.

Co-Existence

You are done with the re-factory project. Time to kill the old bird, right? Absolutely not!

The chances are the re-factorized product, although new and shining, have some bugs. Maybe some less-frequently used functions are missing. Maybe some data-triggered issues are hiding somewhere. It’s a new product, not the one heavily tested over the years by the users.

The best choice is to deploy both the old and new applications. Select a small number of Beta clients and put them in the new application. The majority of the clients remain in the old application. For them, it’s business as usual.

Only after 3-6 months and several iterations of hot fixes, you can gradually switch clients from the old application to the new one. By doing so, you always have a fall-back plan. If a client has some serious problem in the new application, they can be easily switched back to the old one.

For The Long term

You are done with one successful re-factory project. Great! But unfortunately, the forces that drive for this re-factory are not stopping. They will continue to drive for next re-factory. It’s a non-stop game.

Don’t need to be disappointed. Just look at Microsoft Windows and Office. Basically, their business is “re-factory business”. They keep inventing new and better ways to do the same thing. And that’s how they earn their living.

Same to enterprise application development. We all need to be prepared to continue this cycle of reinventing ourselves.

Now, we have suffered one re-factory project. We know all the catches now. What we can do to make the next and the next after next re-factory project easier and faster?

First, obviously, we need to establish the culture of agility and continuously improvement. Just keep in mind, we cannot stop with what we have and we have to keep improving.

Second, force developers to write more documentation, in any forms, comments, word doc, wiki, notes, read me, etc. Even the best developer will forget about why he puts that weird piece of logics there after 2 years without some hints.

Then, once you have a chance, automate your test cases. It will be a life saver in re-factory project. It can cut the project time into half while improve the overall quality. Because developers won’t forget about any test cases any more. And they don’t need to go through the manual test again and again.

The last but not least, keep in touch with your business partners. Not only you will know what to improve, but also you will gain the very much needed support when you promote your next re-factory idea.

Good luck and Happy Re-factory!

Advertisements