Continous migration in software development

New (and soon to be deprecated) technologies are just flowing in the Software industry. Some people pointed out that you can't stop improving your product just to keep the pace with the release flow (that's the fire and motion theory). Yet, being an ISV, your options are quite limited. You have to rely on the latest (yet stable) technologies in order to maintain a highly competitive productivity.

Rewriting from scratch your application to support the latest Foo.NET release is a bad idea; no question asked. Yet, it must be taken into account that

• getting people interested (worse, training them) on deprecated technologies (let's say Classic ASP) is both hard and depressing.


• not beneficing from the latest tools means lower productivity. Ex: Classic ASP => ASP.Net 1.1 => ASP.Net 2.0, each new version being a huge time-saver compared to the previous one).

Lokad.com has been existing for less than a year and, we have already performed quite a lot of migrations.

• SQL Server 2000 => SQL Server 2005
  
  
• ASP.Net website => ASP.Net web application


• No AJAX => ATLAS (betas) => ASP.Net AJAX Extensions


• NAnt => MsBuild (when the MsBuild Community Tasks have been released)
  
  
• VS 2005 Setup Project => WiX 2.0
  
  
• Command Line => PowerShell (for our command-line tools)


• IE6 => IE7 and FF1.5 => FF2.0 (for javascripts and CSS)

Among the next planned migrations

• Visual Studio 2005 => Orcas


• WiX 2.0 => WiX 3.0


• Inline SQL in C# => LINQ


• NDoc => SandCastle


• NuSoap => PHP5 Web Services
  
  
• osCommerce 2.2 => osC 3.0 (currently alpha) => osC 3.1 (for the plugin framework)

Our processes at Lokad involve continuous migrations toward new technologies. Upgrading take time and efforts, yet this process seems quite necessary to maintain optimal development conditions.

Improved "Run external program" through environment variables

Dynacom is a Canadian accounting software that provides an add-on development framework named Synergy. We have started to work on an integration of the Lokad Desktop Sales Forecasting with Dynacom. This post might interest developers who want to integrate together several windows applications (we are considering Dynacom here, but the process would be quite similar for another application).

Basically, Dynacom provides build-in custom action Run external program; yet, this action has a huge drawback: it requires either your application to be part of the PATH on the client machine or it requires to provide an absolute file location (which is likely to vary from one machine to another). A lazy approach would consist in letting the user manually enter the application path; but this approach is likely to be a huge pain for the average user who may not be familiar with the location and the content of the Program Files directory.

Thus I have decided to create a Windows Environment Variable dedicated to Lokad Desktop Sales Forecasting. In the Windows Installer XML packaging script, I have added the following lines

<Environment Id="LokadDesktopPathVariable"
  Action="set"
  Name="LOKAD_DESKTOP_PATH"
  Part="all"
  Permanent="no"
  System="no"
  Value="[INSTALLLOCATION]" />

to get a new LOKAD_DESKTOP_PATH environment variable to be created at install-time (see Sourceforge for the full WiX script). Note that [INSTALLLOCATION] is a variable name; it might be different in your WiX script.

Then, in Dynacom, we retrieve the content of the LOKAD_DESKTOP_PATH environment variable though an Execute script action. The following VB-Script code illustrates how it is done.

Sub Main()
  Set wshShell = CreateObject("WScript.Shell")
  lokadPath = wshShell.ExpandEnvironmentStrings("%LOKAD_DESKTOP_PATH%")
  lokadPath = lokadPath & "Lokad.Windows.SalesForecasting.exe"

  ' HACK: triple-quotes are actually needed to support path with spaces.
  ' http://weblogs.asp.net/steveschofield/archive/2004/03/14/89240.aspx
  wshShell.Run """" & lokadPath & """"
End Sub

At the end, the Dynacom user will get an integration that is both straightforward and robust.

The quest of the fail-proof hosted service

There isn't many 100% reliable hosting providers; yet when I buy an hosted subscription plan, I expect no less than a 100% uptime services.

So far, I have discovered only two fail-proof hosting services

• Squarespace.com: blog hosting, not a single issue for more than 3 months of service.


• Hosted-projects.com: Subversion hosting, not a single issue for almost 1 year of service.

Such a quality of services is truly worth to be mentioned and praised.

VPS for continuous integration

Continuous integration is a cornerstone of our development processes at Lokad. We are currently relying on CruiseControl.Net to support continuous integration.

Several months ago, I did ask on various web forums if any company would sell hosting packages that would natively include CruiseControl.Net. The only answer that I did get was Get yourself a $300 PC and use it as your continuous integration server. I was totally unsatisfied with such answers because the maintenance costs associated with the management of an additional machine are terrible. Indeed, if you assume that your time is worth $50/hour (which is already quite a low estimate if you are a moderately skilled developer); then, this additional machine will cost you more than $200/month assuming only 1h of maintenance per week (which is also a very low estimate).

At this level, it becomes clearly profitable to go for a rented cheap dedicated server ($100/month). Yet, the idea of paying that much for a server that would be used at 5% of its capacity was not entirely satisfying. Thus, we have finally chosen to go for a Virtual Private Server (VPS) that are available at much cheaper rates ($30/month).

Bottom line: VPS is really the way to go for continuous integration involving small to midsize software projects.

Weird consequences of full transaction logs

Let say that you have an ASP.Net 2.0 web application running on top of MSSQL Server 2005. Guess what happen if you database transaction log get full? Well, you will get a large amount of weird side effects, most of them seeming totally unrelated to the saturation of the transaction log.

Among the problems that I have encountered

• The web services of your website start to send totally misleading error messages like authentication failed.


• You can not login through web form into your ASP.Net application any more. You will not get any error message, but the login control just tells you that your password is wrong.


• You look at your error logs (like ELMAH), but nothing gets recorded.


• You decide to go through the "recover password" (because you're still no suspecting the transaction logs), but actually it fails and no email is sent.

For the note, the following SQL question clears your transaction logs
DUMP TRANSACTION mydatabase WITH NO_LOG