Ideas and essays on code development

2007-12-07T07:38:00.000-08:00

Is JQuery now top dog in the JavaScript framework world?

A pretty fair and balanced review of Dojo, MochiKit, YUI, Scriptaculous (with Prototype), and JQuery (including the widgets) saw JQuery a winner by a short head.

Given the impact of Ruby and the Rails framework on modern web development – the effective endorsement of the JQuery client side framework represented by JRails (a ‘drop in’ Rails replacement for the prototype/scriptaculous combination) is an important marker for the growing importance of JQuery.

The Tuesday announcement of an early release of flot, a JQuery plotting widget, was pure icing on the cake for me. The flot code by the way is hosted at Google.

In an interesting twist Google also announced their Charting API which adds a new facility for those who want to use Google resources to display on line data. The approach is described at the link shown above and includes both a novel way of encoding values with a reasonably thorough treatment of 2D charts.

Regular Expressions

I was tweaking a couple of regular expressions used to validate user entry on a customer’s web site – playing with positive and negative “look ahead” to get the result I wanted from various trial input –

reminded of an old but good post by Mark Jason Dominus outlining how Regular expressions are implemented. Yes it’s a bit about perl but it should be interesting to anyone keen on knowing how these things are made to work.

2007-12-04T06:07:00.000-08:00

Shopping Online – the death of or dearth of, the small trader?

Only slightly off topic this one – stick with me for a few moments, it is about software.

In theory, the online shopping revolution gives the small specialist trader equal billing with the big online retail brands – in practice it is surprisingly difficult to buy anything from most smaller online traders.

Case in point – I just did a search for a particular kind of liquor chocolate that my wife is very partial to. The Google listing came up with two “paid for” search results that headed the list. I knew that one of them did not stock what I wanted, so tried the other. None of the links from the main landing page would work. This retailer had paid for what looked like an expensive design job – paid for near top Google ranking but could not sell me any chocolate. They have thrown all that money away.

I was using MS Internet Explorer 6 by the way – I know not to go looking for specialist items using a browser as exotic as FireFox.

Every year I always buy a side of hot smoked salmon for Christmas. If you have never tried this product by the way – take time out to give it a whirl – it is fantastic and nothing like the clammy insipid stuff that is most smoked salmon. I did my usual online round of the UK based suppliers – and they are collectively dreadful web sites – seemingly designed to ensure that you can’t locate the products you are looking for. Information – forget it. In the end I plumped for the Salar Smokehouse on South Uist because I know their product is usually superb – even if their web site misses the mark.

Everywhere you look, you find poorly designed web sites that require near fanatical perseverance on the user’s part to navigate to a successful purchase. Keyword searches that fail – mostly because there is no sensible content to search. There perhaps is the rub – instead of the web presence being central to the business it must be that it is perceived as “extra” or peripheral – bringing in additional (possibly marginal) sales.

Then there is the brief given to those responsible for building a given web site. As a favour, one of my colleagues reviewed a customer’s web site recently as it was failing one of the key metrics, as perceived by that customer. It was pretty obvious why it was failing and I don’t suppose it was the fault of the shop that built the web site – they were just given the wrong brief. Quite a lot of cash has already been spent on this web site but it has largely been wasted. There was no clear marketing idea behind it – well not one that was expressed in terms that the designers understood anyway. In this case completely the wrong site got built.

The hot smoked salmon thing got me thinking about small business online marketing as well. I seem to only buy this splendid food at Christmas – but I might well buy it at other times of the year (fantastic for a summer lunchtime garden party) if only I remembered it was there. So where are the “get 10% off and free shipping” offers through the rest of the year? They have my email address and postal address but you get the feeling they can’t be bothered – and that’s a shame. Smokehouses operate in a seasonal business climate – but they make so little effort to build a continuous trade – which is why so many fail at regular intervals.

Proper targeted, occasional emails that make relevant offers to people you know already buy your product is not spam – come on guys – let’s sell some product!

So if you are unsure about your software business web marketing then make a new year’s resolution to take a good long look at Eric Sink’s web site where there is masses of sound marketing advice and downloadable ebooks. All free stuff to get you thinking clearly about the subject. Then resolve to track Seth Godin’s blog to keep you thinking.

Seth posted today to point out that a very popular Reebok advertisement series running over at YouTube ends with a splash page sending viewers to a web page – which 404s. It’s the same with Vista – the OS screens have lots of links to Microsoft sites but each one I clicked ended up with a “Your page was not found” message. If the people who maintain the Microsoft web sites can’t be bothered to keep a list of the direct links from the company’s flagship product (whatever you might think of it) then what hope is there?

Well I got my fish but can’t find the chocolates. Santa’s sack is going to be light this year.

2007-11-29T04:54:00.000-08:00

JavaScript Dates

Sometimes I have to kick myself – I had better confess all.

I followed a link, proclaiming the “holy grail” of JavaScript Date functionality had been found, to a post by Mark Hopkins at Mashable demonstrating some of the functionality of date.js.

Very good I thought although I noted that the demonstration posited that 05/11/07 was a date in May (a peculiarly US idea). After I had followed the link to http://datejs.com to download the code I dropped by the project Issues page – but somehow completely missed the main project page (to be fair to myself these are often devoid of anything useful).

I reported my issue with the date although I had noted that when a date was unambiguous (say 23/11/07) date.js recognised the normal day, month, year order. My shame was that I had not looked at the code before jumping in – I just thought it was an issue that had been skipped while the project was still in Alpha.

I got a very nice email explaining about the culture settings which I quote in full below:

Please check out the Getting Started with Datejs tutorial.
http://www.datejs.com/2007/11/27/getting-started-with-datejs/

Currently the library supports 150+ cultures and including the appropriate
culture-specific date.js file the Parser will automatically switch to
the correct
parsing order for your country/culture/language.

The sample on the Datejs home page uses the "en-US" CultureInfo file
which expects a
dateElementOrder of "mdy". We're in Canada and use the "date-en-CA.js"
file in all
our internal apps. In Canada the expected dateElement order is "dmy".

The full download package includes all 150+ pre-compiled CultureInfo
files. See
http://www.datejs.com/download/

I hope my humble apology and this post makes amends for my wasting the team’s time.

What’s all the fuss about? Well date.js has a capacity to take just about any user input (no matter how formatted or descriptive) and turns it into a date.

date.js reminds me of some very old code that is still used in some VMS applications I maintain that was written (by my brother I think) to do a similar “trick”. The user can enter (say) “t” for today, “t+2” for the day after tomorrow, use “long” or “short” date formats and always (well nearly always) end up with a sensible date that can be stored or used by the application.

Date.js goes rather further than that. It offers some sophisticated date manipulation, date comparison and date formatting options – frankly I can’t see anything left out.

The code is published under the generous MIT license so you can use it as part of any sort of application.

There are some open issues with date.js (to be expected in an Alpha) but I am sure that this code is going to end up in my toolkit. Check it out or monitor the main project page to await a full release.

2007-11-27T07:46:00.000-08:00

Text Editing

Quite some time ago I took a look at the range of text editors for Windows with a view to getting something to replace the now moribund “Developer’s pad” and to meet a couple of additional needs for something rather more intelligent than notepad. The sheer range of options completely overwhelmed my enthusiasm and I got nowhere.

There was one feature in particular that I was looking for and that was the ability to select a column of data out of a text file – not something that is generally available but very useful when trying to extract specific data items from reports or other Customer generated input to the daily toil.

I picked up on a post that looked at a survey of Ruby developers and their tools organised by Tim Bray. This generally showed a preference for TextMate as a programmers editor. Now TextMate runs on an Apple Mac (where a lot of Ruby developers hang out) but I prefer the (slightly) more open Windows platform – well that’s what I keep telling myself anyway.

It turns out that the e Text Editor is a Windows clone of TextMate that can take advantage of all of the hard work contributed by the TextMate community to create settings for a wide variety of programming languages and developers “add-ins”. Unsurprisingly, VMS Basic is not in the list I saw but interestingly VB.NET is.

Oh yeah and the e Text Editor can select columns of data as well as provide effective support for the syntax of JavaScript , ASP.NET, CSS, HTML, PostScript, Python, XML and Ruby of course. Plus a host of others – there is a list you can check.

Add in a revisions tool and the ability to shell out to a language of your choice (via cygwin) then this really does look like the tool I’ve been ~~hunting for~~ sitting around waiting to magically turn up by itself.

e Text Editor has a free trial period – that is pretty gently enforced and a very reasonable purchase price. Now the thing is to make sure that I have enough work for a text editor right now so that it gets a fair evaluation.

2007-11-26T06:01:00.000-08:00

Ancient Vistas

OK – so I am a year late but I booted up the first Vista machine on the home network this weekend.

First impression – Windows Vista is the finest argument yet for a return to the command line. I am truly sorry for supporting Windows all those years ago – I thought it would make the user’s life simpler and let them get useful things done – I now see the error of my ways.

Seriously though, some good news – the d***d search dog has gone; but I am not yet sold on the replacement.

Goodish news – I was encouraged to set up a password protected administrator’s account and to then set up user accounts – in this instance for my children. Even better news – when logged in as one of the children I am prompted to select an administrators account (I created two) and to type in a password to install a game. Bad news – when I am logged in to the administrators account I keep getting hassled to “give Windows permission” to do what I just told it to do. So its a pain in the butt doing straightforward maintenance tasks after deliberately logging in as administrator to do them.

Plus the usual – “why do something with one click if you can get the dumb user to click four or five times” Microsoft syndrome. Usability – try clicking the “Add a new user” hyperlink from the Control Panel – you have to click three more times before you actually start adding a user – so what was the point of the link?

Plus - actually I don’t like the transparent bits around the edge of the app windows – sorry and all that but I think they look untidy and distracting.

If you boot a new XP machine hooked up to the network (or within sniffing distance of a wifi hub) then the initialisation process politely informs you that a network is out there and asks permission to get connected. The Vista machine stayed quiet on the subject of networks and I had to explicitly set up a connection. To be fair this might be a symptom of the pre-installed Acer crapware on the machine that seems to permanently waste CPU cycles (and memory) duplicating standard Windows control features.

The big question is – is Vista fit for business use? Probably too early for me to hazard an opinion. If suitable remote administrative tools exist to set up “standardised” installations and to get around the somewhat pathetic security layer apparently sewn into the fabric of the OS - then maybe. Is it worth the cost to upgrade existing XP machines – definitely not from what I’ve seen. Plus (warning – I am reporting other people’s findings here) it sounds like XP Service Pack 3 brings a performance boost with it – great for any machines that are beginning to feel a bit tired while Vista performance is little improved by it’s very own Service Pack.

Still – you don’t have to listen to me – I thought NT 3.51 was pretty good and that Windows 2000 was just about the pinnacle of the current design concept. So what do I know?

2007-11-21T02:34:00.000-08:00

More on Android

A very good first project piece from Reto Meier that nicely links the address book to the GPS to a map. A very good demonstration from a C# programmer (although with a background in Java).

For an intelligently critical viewpoint try Unqualified Reservations – a blog often more about politics than software. I think the most interesting point raised here is to do with Android graphics. Here the choice of pixel drawing functions looks plain wrong – despite the fact that such functions will certainly run faster on the current generation of devices than a vector based approach – the trick is to write for tomorrows devices f you want to be tomorrows winner. However it got me thinking about the browser end of the package – surely Google’s strength is Internet based - so why isn’t more of the SDK about Internet applications – with a brilliant JavaScript engine? Interesting that.

2007-11-21T02:33:00.000-08:00

Amazon saves the trees?

I think that Mark Pilgrim is exactly (and amusingly) on the money with his The Future of Reading (A Play in six acts). Ebooks just do not cut it when you factor in DRM and high device costs. I had been wondering how I could amortise a Kindle unit purchase over an anticipated device lifetime of two years on purchases of (say) two books a month – and that’s assuming that all the books I wanted would be available in the specified format. It just does not look a sensible option – and no electronic format available today is going to match the versatility of a simple book.

I suspect that the problem here is that Jeff Bezos and Amazon are trying to lead this market. The iPod came after the initial successes of MP3 and user driven format shifting for music. Apple made the format convenient and makes a bundle of money on the back of that convenience. There is no established format shifting mechanism for books. I suspect that “home copying” (more properly format shifting) of books will have to become a reality – fixing an effective digital format on the way – before eBook reading devices (if they turn out not to be an existing device) have a chance in the market place. It would be convenient to be able to carry around a couple of novels and a good portion of my technical library but the limitations (and price) of the current proposals from Amazon do not look that convenient.

The OLPC looks like a pretty good text reading device – so any ideas on how best to quickly and conveniently shift my newly purchased paper based books to a format that I can carry around on one of those?

2007-11-16T01:21:00.000-08:00

The Rise and Falter of Visual Basic

I read an excellent review of the history and current status of Visual Basic at DevTopics but I would change the post title from “The Rise and Fall of Visual Basic” by substituting “Falter” for “Fall” as this post’s title suggests.

I am not sure where the figures supposedly showing the death of Visual Basic come from. If you are happily working with VB then you are probably doing so without making a lot of noise about it – that could be the problem. However take hear from the following.

Visual Basic is the number one .NET language
Visual Basic is the number one downloaded and number one registered Express Edition (by a margin over the next of 20%)
Visual Basic is the number one language gauged by the MSDN centre and blog
The Visual Basic team blog is the number one MS blog.

Figures taken from Paul Vick’s post on the subject where he is now suggesting that it is time to “lose” the Mort persona used to label VB developers and replace him with Ben (Franklin). Ben is the guy you call when something needs to be done – a bit of a polymath – but practical through and through.

And perhaps there is the rub. VB.NET is for the Bens of this world and has left the Morts behind. I know plenty of lone developers and development team members who are Morts – making their contribution to those “Big balls of Mud” that serve Industry and Commerce so well. There are still a lot of Bens writing VB6 applications but they have the choice of future tools – what’s left for the guys who want a solid, pragmatic development platform that does not demand a theoretical model to hang the code on?

Me – I’m off to write a quick and dirty data analysis in VB6. The Visual Studio 6 IDE might feel somewhat lacking nowadays (you have to type in your own “End If” lines) but it’s still faster to hack a report than .NET and there are no deployment issues in my target domain.

2007-11-16T01:19:00.000-08:00

Google’s Android Again

Top class post by John Lombardo titled “A developers Perspective on Google’s Android” should be on the “must read” list for anyone serious about playing with this SDK.

2007-11-14T07:33:00.000-08:00

Android Backlash

Wow that was fast – the backlash to the Google Android SDK seems to have arrived with a slight lead on the fanfares – Oh, of course, this is Google – they do not do fanfares.

I suspect that the vast majority of the target audience of mobile developers is still reading the documentation and analysing the implications. Yes – we would all like to see a reference device available to get a better feel for how the various components (software and hardware) will fit together but for the moment the device emulator will have to do.

The negative posts seem to fall into two camps. One theme that is developer centric is personified by Rick Ross’ post at javalobby which complains that Android does not synchronise with existing Java standards and does not support a lot of pre-existing open source libraries I can understand those objections but I also think that they reflect one of the strengths of the Android SDK – it is not dependent upon a bunch of pre-existing code libraries (which have hardly taken the mobile world by storm) and can thus be much more agile and responsive to developer feedback and hardware advances.

The second negative theme is well typified by John C Dvorak who somehow misses the whole point. He sneers at industry alliances – perhaps rightly as such alliances often fail to deliver when the time comes to spend actual money. He even sneers at Google’s Andy Rubin because he has been associated with some PDA projects in the past. Dvorak fails to grasp the potential of mobile computing – and that’s rather the point I suppose.

All successful mobile computing projects have so far been niche applications working with small numbers of users. I first worked with a successful mobile computing project back in 1979 and yes that was two years before the IBM PC was launched – so prehistoric times then.

In these days of the ubiquitous MS Office applications, games and web browsers it is hard to remember that the IBM PC (and thus the PC) exploded into the marketplace because it allowed the low cost development and implementation of a host of niche business applications. The same has not yet happened in the PDA/Mobile phone marketplace because neither the hardware nor the development environments have provided that same catalyst to application development that the IBM PC represented. However successful applications are out there and they are generating device sales in their own right.

Mobile computing will “click” for a wide audience when you can do effective work over the Internet using smart-phones. Why smart-phones? Well because we only have the capacity to carry one electronic device around with us at any one time – so let’s make it a clever one.

When I am away from my home office my smart-phone becomes my active link to my business and the customers who pay for my time. It brings voice messages and email together at a single point while keeping track of the time, meetings and appointments. While these are vital roles I want my device to do more. With some fiddling with a bluetooth keyboard I can get it to take notes (just but it looks tricky and pretentious). I can just about use Terminal Services to log into a PC on my home/office network but I need to be very selective about what I run there. It makes quite a good job of navigating while I am driving but would be better if it connected up to a car radio/CD player so that I can listen to music and hear the driving directions clearly (actually some cars can do that). It plays MP3 tracks very well through headphones and (when lighting levels are low) makes a reasonable fist of showing recorded TV. You see it is nearly good at quite a lot of things but misses the mark of to many of them.

What I can’t do with my smart-phone is integration. If I take a snap with the camera I can’t directly include the shot into some notes I am typing along with the GPS location. It is also pretty naff at doing more than one thing at a time but this is partly the terrible navigation (or rather anti-navigation) features of Windows Mobile. Plus you can’t (as far as I can see) replace the worst of the bundled apps (like the ultra-simple calculator program) with better versions (like the one I wrote) You can have both but not just the one you want.

This list of what I can and can almost do is getting too long and boring so I will stop – but I am sure you get the idea.

Here I think you have to acknowledge the development tools available for the MS Windows Mobile platform. As a developer I have to say they make the MS software platform the most appealing – and that is where a great deal of good work is, in fact, being done.

However, as I have indicated before Windows Mobile is a terrible OS from the user’s standpoint. It is infinitely frustrating to use and cleverly blocks any attempt to manage truly joined up applications across a given device’s functionality. So, as I wrote yesterday, development tools are very important – but then so is the user interface and the level of integration offered by a given hardware manufacturer – and that is where it is still coming apart.

The simple functional approach of the IBM PC created a new hardware platform where before there was half functional anarchy. OK the brand helped but the device and basic OS covered the ground in a straightforward and compelling manner – it was adequate.

Defining an adequate device for mobile applications is not easy and you probably have to do it in relative terms. Examples might be “as big a screen as possible” or “an effective input device for near normal typing (or whatever)”. One absolute requirement would be an effective developer’s SDK and access through an API to all of the devices components. The last requirement is probably crucial and needs to be understood by device driver writers from the start – for instance more than one executable might want access to the output of a GPS component at the same time.

Is Android the answer – well it might be because it is starting from the right place – the SDK. The SDK defines the basic functionality of the devices that will run this OS. As a developer I have to feel that this is the right approach but (of course) we will have to wait and watch as things develop.

2007-11-13T04:36:00.000-08:00

November 12th. Mobile Action Day

It was fortuitous that I was chatting to our mobile platform development team yesterday – the day that Google released the first version of the SDK for their mobile phone operating system (codename Android). We had been discussing the fate of the PDA – and noted that this platform could now be considered officially dead. Nobody is making PDAs anymore (the very minor exceptions prove the point here) – all the action is with smart phone devices – oh and (perhaps) micro PCs. The obvious downside of the demise of the PDA is that average screen sizes have actually reduced on the available range of mobile devices – let us hope that is a temporary blip.

Yesterday also saw the effective launch of the limited One Laptop per Child (OLPC) “Give One, Get One” campaign (currently in the USA and Canada only) that will see the first of these devices delivered by Christmas – well to North American homes anyway. The current price is $200 each (interestingly the same price point as that chosen by Wallmart for their Linux based notebook – now sold out) which is under £100 (GBP) or 140€ (Euros) for what could turn out to be a smash hit device with a fantastic spec. I can’t wait to get my hands on one and will gladly pay for another to be given away by the project. It will be interesting to see what happens when real practical device reviews hit the Internet over the coming weeks and when the development community gets to write even more applications for what is a great machine. I suspect that individual teachers, schools and education authorities throughout the world are going to want to integrate these devices into school life – this is not going to be a “third world” device at all but an iconic educational tool for all.

For a short review of the OLPC device visit Bill Clementson’s blog or check out the donation web site for further details. These could turn out to be the portable device of choice for geeks – roll over Macbooks.

Back to Google’s Android. Existing phone operating systems are a pretty mixed bunch. From a developer’s standpoint Microsoft’s Windows Mobile is the most accessible – it’s just a shame that it is so terrible to use. Just how can version 6 of an Operating System be so broken? I could fill pages with the problems from both a developer’s and a user’s perspective. It is just about inconceivable that the development team at Microsoft actually use the OS on any of their personal devices. So, if the only plus point for the Windows Mobile platform is it’s support for development tools (primarily the .NET Framework of course) then a new smart phone OS built by a developer oriented company like Google is big news. Here at last we might have an open platform accessible to, nay designed for, the development community. With support coming from phone manufacturers this is going to be big and could well “claim” this key platform for Google.

Sitting at the heart of Google’s Android is a new virtual machine named Dalvik. Dalvik works in a very similar manner to Sun’s Java VM and the .NET Framework. As a strat point at least, Android programs are written in Google’s preferred language – Java. But instead of compiling that Java code to Java byte code the compiler targets Dalvik byte code. Thus, very neatly, Google have created a new platform which is not a Java platform and thus one that is not going to be restricted by Sun into the future.

Any bets on which language will be “ported” to the Dalvik platform to follow Java – will it be Python, Ruby or JavaScript? Will there be a Dalvik version of the mono project to open up the platform to .NET shops?

The Android SDK also includes Webkit and not Mozilla as the HTML rendering solution. This will give the mobile OS a lot in common with the Apple iPhone – well initially at least as this might well be subject to change in future releases.

Just in case there is any doubt – I think that Android is the Google OS that a lot of folks predicted a couple of years ago – it can clearly be ported to a whole range of chip sets – so coming to a desktop near you soon fully integrated into the Google “cloud computing” environment.

2007-10-25T00:58:00.000-07:00

Learning JQuery

In yesterday's post I mentioned that I was trialling the JQuery framework on some new interactive web development tasks. As is my want, I had purchased a copy of "Learning JQuery" by Jonathan Chaffer and Karl Swedberg and had started to write a review - when I spotted this excellent review over at SlashDot

The only thing I would add is that if you are already very familiar with DOM manipulation using JavaScript then you can probably manage to pick up JQuery from the documentation on the site - although a copy of the companion "JQuery Reference Guide" might make a welcome addition to your bookshelf.

For the lowest prices on these books and free shipping - buy directly from the publishers PACKT who will then also make a financial contribution to the project.

2007-10-24T08:25:00.000-07:00

JavaScript on demand

There is an interesting post over at Dr. Dobbs that looks at something I have been thinking about for a while – loading JavaScript functions on demand rather than loading a large library of functionality up front with the HTML page.

The idea is that instead of loading a complete library of JavaScript needed to supply the full functionality of a page you just load the actual functions required as and when they are first required in response to a user’s interaction with a page.

Thus instead of loading a script file with some code like this:


<HTML>
<HEAD>
 <SCRIPT type=”text/javascript” src=”MyBigJSFile.js”></SCRIPT>
</HEAD>
<BODY>
etc.

you use Ajax techniques to load JavaScript code on the fly as it is required. I have played with a trivial form of this in the past as it can be a simple solution to some complex problems where the nature of the script you actually want to run is heavily influenced by the current state. [It’s not the only solution or necessarily the best but…] One thing my early experiments confirmed is that JavaScript added to a page after it has loaded is always executed immediately – which can be just what you want but implies that techniques such as this are best attached to events that require some code to run.

The Dr Dobbs article suggests a technique where functions are downloaded and executed the first time they are required and the effectively cached for future use. This is managed by running what is effectively a prefix function that manages the download and then deletes itself – leaving the core function ready for a subsequent execution. The article features a library (AspectJS) that directly supports this approach. Now for the moment I am going to pass on the slightly bizarre idea of including a library within a page whose prime function is to save you including a library and consider some other aspects of this idea.

I suppose I start from the question of where you can afford for your web application (for surely we are talking applications here and not just regular pages) to take it’s time to download your scripts. The general user is used to the initial load overhead when a page is first accessed – we would have to be careful about introducing unnecessary pauses later in any given application process. In thinking about this issue I have changed my mind several times – and now I’m not sure.

If you have not read Joel Spolsky’s “Strategy letter IV” then I can recommend a good read. The key point however is that he envisages a “winner” in the apparent race to supply an Ajax browser SDK (to replace the Windows API). I am sure he is right in that life is too short to keep writing the same JavaScript code again and again – and way to short to keep writing your own way around the peculiarities of any given browser. For my part I am currently playing around with JQuery which I picked for it’s very small footprint and it’s clever approach to segmenting the library with a series of “plug-ins”. This could map quite well to the idea of loading JavaScript code on demand.

Anyway Joel’s post also said that the “winners” ignored current performance or resource restrictions and ploughed on building features. If they get their timing right then the restrictions will have melted away just about the time the new software hits the mainstream. The winners thus blow away the opposition who have wasted the intervening period writing code to manage around those now irrelevant restrictions.

Now I got to thinking about this whole issue when I started to plan a very complex vertical market program that we wanted to make available from our servers as a browser based application. I wanted to consider the loading of client JavaScript code on the fly to avoid too long a page start overhead but also as a form of obfuscation – as only a part of the client code base would be loaded into the browser at any one time and that code could be altered on the fly during code execution. Only time will tell if this approach has legs and remains as maintainable as the usual “big library” approach.

Sourceforge features another library loader called jsloader that has a narrower overall scope when compared to AspectJS and is more intended to deal with library versioning issues – but worth a look for anyone delivering applications that rely upon third party libraries or who envisages developing one or more libraries over time.

If you are checking out JavaScript libraries then you might like to consider Prototype as well as JQuery

2007-10-15T05:01:00.000-07:00

So just what is Anti-Virus software for?

I have had anti-virus software installed on all of my computers for years now and I’ve never caught a virus but I’m not at all sure that this is because anti-virus software actually does any good.

Sure, my incoming email often contains viral infections but none of them (get that – none of them) have ever passed the Spam filters ever since they were put in place more than two years ago. Plus, an irregular inspection of the messages carrying the software infections convinces me that I would never be dumb enough to click on the package even if my Spam filters let me down.

OK, I do get sent MS Office documents and bitmap files during the working day and I suppose one of those files could have become infected at source but truthfully when was the last time you got sent a virus laden email from a known corporate source? There was a time when one of my clients always sent me the latest and greatest in viruses just about as soon as they hit the Internet but those days are past.

Is anti-virus software any good at spotting viruses anyway? Well no – says a lot of recent research. Despite seeming glowing reviews in the computer press a great number of anti-virus products detect a worryingly low percentage of the software viruses presented to them in independent testing. None of them detect them all. There is frequently a high level of false positives to contend with – with your anti-virus software denying you access to perfectly innocent content.

See here for test results for a sample of anti-virus products. These results are not atypical.

Is anti-virus software an effective barrier to such threats as the “Storm Worm” – well no says no less an authority than Bruce Schneier. The only protection is to NOT CLICK ON THE ATTACHMENT on the email you just received.

What happens if your anti-virus software detects a virus? I have been called by friends who could not stop themselves clicking on that “see the bouncing bunnies” tag to help out when this has happened. What happens is that the anti-virus software detects the virus and is then unable to do anything about it – it gets stuck trying to delete the file in question – often failing even to remove registry entries. OK – someone with a reasonable level of technical expertise can deal with this but the general public can’t. What worries me here is that it looks to me like anti-virus software might have some value on a Windows machine being run by Joe Public but that it all to often fails to meet the requirements of that key user group.

The only time that my anti-virus software has detected a threat was when it spotted that some text files on my hard drive had a similar name to a known virus – so the software kindly tried to block my access to my own data. Potential threat – zero – inconvenience level – high.

What impact does anti-virus software have on all that expensive hardware you have deployed? If you are using a Mac or Linux probably none as you are probably not using such software. If you are using Windows then take a look at this PCSpy article the highlights of which I reproduce below – with additional thanks to Coding Horror for this quotation.

	Percent Slower
Software Name	Boot	CPU	Disk
Norton Internet Security 2006	46%	20%	2369%
McAfee VirusScan Enterprise 8	7%	20%	2246%
Norton Internet Security 2007	45%	8%	1515%
Trend Micro PC-cillin AV 2006	2%	0%	1288%
ZoneAlarm ISS	16%	0%	992%
Norton Antivirus 2002	11%	8%	658%
Windows Live OneCare	11%	8%	512%
Webroot Spy Sweeper	6%	8%	369%
Nod32 v2.5	7%	8%	177%
avast! 4.7 Home	4%	8%	115%
Windows Defender	5%	8%	54%
Panda Antivirus 2007	20%	4%	15%
AVG 7.1 Free	15%	0%	19%

The problem is that the “dancing bunnies” still attract the general public and the operating system security in general use (versions of MS Windows) are ineffective in protecting them from such foolishness. So perhaps there is a case to be made for encouraging “Aunt Maude” to persists in running up-to-date anti-virus software even if the protection is far from complete.

However the issue is rather different for us developers. There really seems no rational case for continuing to run (and presumably pay for) this class of software on our machines. For us, anti-virus software is all “downside” with no discernible benefits.

2007-10-04T03:24:00.000-07:00

Cat* and Mouse

or when to call a halt

I mentioned the other day that I had been reading an out of print book by Peter Grogono on the subject of the Mouse language that was designed to – well that’s interesting – just what was it designed for?

The language is a development of one that had been used to control audio equipment in a studio. The language was designed to be implemented by a (very) simple interpreter. In effect, the book explores many of the aspects of language design and implementation while avoiding the usual academic treatment. Mouse can be used to implement a number of useful algorithms but in reality is best considered a tool for learning the developer’s craft rather than as a tool for applying that craft.

The book describes the Mouse language and it’s design by exploring the development of an interpreter written in Pascal. However the code developed in the book deliberately avoids many of the features of Pascal (effectively using it as pseudo-code) to make it easy to translate the code into just about any assembly language.

I decided it would be equally straightforward to develop a Mouse language interpreter in VB.NET with the expectation that I would try my hand at building a compiler for the language once I fully understood it. OK a Mouse compiler for .NET was probably never going to be usefully applied (read never) but I thought it would teach me a few things about compiler writing.

The interpreter was finished quickly once I had debugged the odd error in Peter Grogono’s text plus revised the “test suite” to eliminate typos and to make it a little more structured. I then turned my attention to the .NET compiler.

If you take a look at the language structure at Wikipedia or here or here you will see that outputting assembly code for 90% of the language would hardly be taxing – and it proved a pretty simple undertaking to output IL Assembly code from a VB.NET program. Well simple until I got to the fascinating macro calls.

Mouse is a remarkably simple language and the macro parameter passing mechanism is also very simple but that simplicity allows remarkably subtle techniques to be applied and that subtlety opens up challenges for anyone why wants to “fix” the code at compile time.

Superficially, the language appears to present constants, variables and macros as part of a method call but in fact the arguments are interpreted by the method at the point in time they are referenced. The arguments are (normally) short pieces of Mouse code and not values per se. Some arguments might place a value on the stack and some might output a string constant to the “console”. There is also an interesting facility that allows values to be passed by reference as well as by value although the programmer has to acknowledge the difference in the macro code.

Part of the subtlety arises from the fact that the arguments are evaluated within the context of the calling routine (main program or another macro – perhaps even a recursive call) at the point in time that the argument is used in code and might well change between successive references to the same argument. I was faced with the choice of changing the language rules (allowed because there is no strict standard and multiple versions exist) or working very hard to emulate a subtle feature of an interpreted language within a compiled version.

When I thought about it I decided that reaching that level of understanding was the purpose of the exercise and that I could thus abandon the project and apply my compiler writing ambitions to a language of my own devising, and no-one wants to hear about yet another one of those.

* it turns out that cat is also a stack based r.p.n. language although it sports multiples data types and has some decidedly Lisp tendencies. Cat claims descent from a language called Joy but we should remember that no self respecting cat would acknowledge a mouse as even a distant cousin.

2007-09-27T04:59:00.000-07:00

Hashing passwords in JavaScript and .NET

If you are using the .NET framework to build a system with a web interface then you may be faced with the need to hash passwords at both the client and server.

Client side hashing of passwords makes sense as they are never then passed over the network/Internet in clear text. Plus passwords should always be stored in any continuous storage (such as a database) in a hashed or encrypted form.

If all passwords could be originated from the client end then you would just implement a JavaScript version of your preferred hashing algorithm. However, there is often need to create new passwords at the server and when this occurs it is necessary to ensure that any hash created by the server is exactly the same as the hash that would be generated within the client browser.

Popular hashing algorithms such as MD5, SHA-1 and SHA-2 always produce the same result for the same input don’t they? Well yes, but different implementations may not result in the same text representation of the computed hash.

Now MD5 is getting a bit long in the tooth and there are a lot of facilities on the web that provide look up tables for hundreds of thousands of possible MD5 password/hash combinations. See here. I decided to implement SHA256 as part of my security strategy for my next web project.

I located the excellent JavaScript SHA256 implementation made available by Angel Marin to run at the client end and created a simple server side .NET function to hash a given string.

The .NET documentation would suggest something like:


Private Function HashAString(ByVal StringToHash as string) as String
    Dim PlainTextBytes() as Byte
    PlainTextBytes = Encoding.UTF8.GetBytes(StringToHash)
    Dim shaM as New SHA256Managed()
    Dim HashBytes() as Byte
    HashBytes = shaM.ComputeHash(PlainTextBytes)
    Return Convert.ToBase64String(HashBytes)
End Function
not forgetting to import the System.Security.Cryptography namespace of course

Passing “Hello World” to this function results in
pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4=
but the JavaScript function returns a hex representation which is
a591a6d40bf420404a011733cfb7b190d62c65bf0bcda32b57b277d9ad9f146e

To get a match we need to convert the values in the .NET hashed byte array to a hex representation to match them to the result of the JavaScript function.

In order to produce a hex representation of the SHA256 hashing algorithm the Visual Basic .NET function should read something like:


Private Function HasAString(ByVal StringToHash as string) as String
    Dim PlainTextBytes() as Byte
    PlainTextBytes = Encoding.UTF8.GetBytes(StringToHash)
    Dim shaM as New SHA256Managed()
    Dim HashBytes() as Byte
    HashBytes = sham.ComputeHash(PlainTextBytes)
    Dim HexArray(Ubound(HashBytes)) as String
    For ByteLoop as Int16 = 0 to Ubound(HashBytes)
        HexArray(ByteLoop) = Hex(HashBytes(ByteLoop))
        If HexArray(ByteLoop).Length = 1 Then
            HexArray(ByteLoop) = “0” & HexArray(ByteLoop)
        End If
    Next
    Return Join(HexArray, “”).ToLower
End Function

which produces the required lowercase hexadecimal string representation that matches the output of the JavaScript function perfectly.

If you are still keen to use the MD5 hashing algorithm then you will find that the same solution works there when trying to get matching output from the .NET Framework to (say) the MD5 JavaScript function supplied by Paul Johnston. Paul also publishes a SHA-1 solution and has a lot of useful advice on his website for anyone designing a secure password access system. For another viewpoint on password storage try here

2007-09-21T06:54:00.000-07:00

Third party advertising on your web site

Disclaimer, one of our company web sites takes Google AdWord advertising on some selected general information pages but this blog does not show adverts and does not even provide affiliate links to Amazon for the books reviewed or mentioned. There is a reason for this cautious approach. Once you sign up for advertising to support your site costs you automatically lose control of part of the site content.

A recent example. When I visit the otherwise excellent User Friendly site to read the latest cartoon strip then most days the banner advertising at the top of the page for visitors using a French ISP is for scum ware. This would make it very hard to recommend that others visit this site as they might be tempted to click on the ad. I raised this issue with the team responsible for the site and got a response. Their view was that the site had to be supported by advertising and that they were not going to be bothered by ads that required the user to actively click on something before anything bad happened to that user’s PC. I accept the need for advertising revenue but I am not so sure I am comfortable with that moral position.

The same scum ware ads turn up from time to time on the Dilbert site as well if you are in France (and presumably if you visit from other parts of the world as well). However there is little chance that either the “User Friendly” or Dilbert sites will be blocked by Google because of dangerous content – but it could happen to you. The Register has a post today that tracks more than one web sites problems when they were flagged by Google as a site likely to be actively harmful. Now I would not want that stuff on a site of mine anyway but allowing a third party to plaster malicious advertising over your headlines just does not make Internet business sense.

What’s in an email address?

Well probably many more different characters than you would think.

A maintenance routine run by a customer choked on a new email address the other day – the format was one that lay outside of the range of the regular expression used to validate the entry field on the .NET windows form. The user logged a fault call and we had a tested update to the validation winging it’s way back again in minutes – not a big deal but…

Let us apply a little computational thinking to this area.

The validation problem was not strictly a bug. The regular expression being used was intended to help the user catch keying and transcription errors and was thus fairly strict. It probably validates more than 90% of email addresses correctly but was never intended to allow the full range of possible variations allowed by the specification RFC2822 .
The web site www.Regular-expressions suggests that the full specification would require the following regular expression to implement:
(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|"(?:[\x01-\ x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?: (?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:25[0-5]|2[0-4] [0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?: [\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])
and this would trap very few typos because it is much too wide ranging.
Out of interest we are currently using:
^([\w\.\-\&]+)@([\w\-.]+)[\.]((([a-zA-Z]){2,3})|((([a-zA-Z]){2,3})[\.](([a-zA-Z]){2,3})))$
and this covers most requirements for a UK based business recording internal and customer email addresses. However it will not validate some email addresses in use (such as mike@computer.museum). No I didn’t know that museum was a valid TLD (top level domain) either.
There are positive benefits in implementing validation routines that are technically faulty – it can make them more effective for daily use. However, they such restrictions only effective when they are applied at the interface between the user and a system. They should never be applied to data that has entered a system – that data should have a high level of trust after all. Plus – if it is a customer entering his or her email address then it might make sense to use a fairly strict validation to supply a warning that the email address might not be correct but a pretty dumb move to void a transaction if the customer’s actual email address contains unusual but valid characters.

Take a look at a post by Ragenwald explaining just what can go wrong (pretty forthrightly too).

2007-09-12T07:16:00.000-07:00

ILASM language specification

I am working to enhance my knowledge of the .NET IL assembly language and was looking around for a definitive specification for the assembly language elements. Fortunately I spotted the latest ECMA spec on the MSDN web site which was good because otherwise the MSDN site and Visual Studio documentation rather overlook this key .NET framework language.

I downloaded the CIL specification in MS Word format and that has been a great help. I found I still needed to do a little experimentation to remove any remaining ambiguities but that is probably all to the good as I “learn through play”.

I have also ordered a copy of Serge Lidin’s “Expert .NET 2 IL Assembler” which I am confident will fill in most gaps once it arrives. By the way, this book is listed at £40.99 at Amazon UK but is only $13.54 at Amazon .com – with shipping to France being just about the same cost from both locations. Guess where I purchased my copy given that the UK price was about six times the USA price? I am going to be checking the US on-line book stores more closely in future if prices vary as much as that – and the value of the dollar makes the spread even more attractive as days go by.

Books for .NET debutants

My last post that mentioned the new flush on interest in VB Classic over at Visual Studio Magazine reminded me that there are still a great many development shops yet to make the leap to .NET or even to test the waters. For anyone wondering which books might help them make the transition to the .Net world I would recommend Rod Stephens’ “Visual Basic 2005 with .NET 3.0” without reservation and would strongly suggest Rod’s “Visual Basic 2005 Design and Development” for the development team leader/software architect. Mind if your team includes one or more who like to look “under the hood” I would also recommend a copy of Andrew Troelsen’s “Pro VB 2005 and the .NET 2.0 Platform” – a book I wish I had discovered a good while ago. The three books represent three different approaches to .NET development but combined provide both fantastic learning resources but also a very solid reference base for a substantial initial project.

2007-09-10T03:16:00.000-07:00

Viva La VB

As a long term resident of France, how could I resist an article title like the one above?.

Karl E Peterson is back writing for the Visual Studio Magazine and his subject is Classic VB. His first piece (with that catchy title) can be seen at http://www.ftponline.com/channels/not/2007_09/peterson/ and his initial theme is going to be on solving the challenge of supporting Vista. He starts in on setting up Virtual PC 2007 so that you can do your developing and testing against a clean virtual machine.

If Visual Studio Magazine recognises VB Classic developers are a huge and enduring group (and presumably a substantial market) then, who knows, we might see more support from other sources into the future.

I suspect that Karl Peterson’s articles should be a “must read” for VB Classic development shops who are already having to fight problem areas such as ActiveX.exe modules on Windows XP, Windows 2003 and the dynamic duo or Terminal Servers and Citrix. The VB6 runtime might well have support on Vista but that is a long way from properly hosting VB Classic applications.

Virtual PC 2007

As Virtual PC 2007 is a free download from Microsoft and is such a useful tool – I had better include a link for it.

Those WPF dimensions

The current online pages of the Visual Studio Magazine have a short piece by Rockford Lhotka titled “Does WPF Matter?” which nicely dovetails into my musing on the same subject. I think I have decided that I will just keep an eye on WPF until either the dev tools mature or I need to write code that displays vector graphics.

2007-09-03T05:12:00.001-07:00

The other .Net language

It seems like hardly a month goes by without someone announcing a project to port yet another language to the .Net platform. There are good reasons for that of course. Both the CLR and the fast developing DLR offer an attractive foundation for language developers. The Windows OS represents the dominant business platform in any case but the amazing mono project is delivering on cross platform capability. A .NET based language is likely to attract new adherents and has an increased likelihood of acceptance among even relatively conservative corporate development teams.

But there has always been another .Net language – the target for some of those compiler writers – ILAsm – the .NET intermediate language assembler. The IL is the bytecode ultimately produced by all .Net languages. IL is the ‘lingua franca’ of the .Net platform and sits waiting in every .exe or .dll file ready for the JIT compiler to create machine code for the hardware to run just in time. ILAsm is the IL assembler language and is a low level language designed to describe every function of the CLR. ILAsm was one of the original .NET languages but remains the least known.

You can code to the .Net framework in ILAsm just as you can in any other .NET platform language. Forget the “C# is cooler than VB” meme – real programmers code in ILAsm!

Want to give it a go?

First let’s get the obligatory “Hello World” out of the way.

Grab your favourite text editor and type in the following code:


.assembly extern mscorlib { }
.assembly hellworld { }
.module hellworld.exe
.namespace hello
{
.class public auto ansi world extends [mscorlib]System.Object
{
.method public static void Main() cil managed
{
    .entrypoint
    ldstr "Hello World"
    call void [mscorlib]System.Console::WriteLine(string)
    call string [mscorlib]System.Console::ReadLine()
    pop
    ret
} //end of method
} //end of class
} //end of namespace

Save the file as hellworld.il

Open an .NET SDK Command prompt (see below) navigate to the folder holding the .il file created above.

Compile the ILAsm file to IL language by typing in the command “ilasm hellworld” (omitting the quotes of course). You will (all being well) then see a new file hellworld.exe. If you run this executable you will see the words “Hello World” displayed in the console window and the program will wait until you press the enter key before terminating.

First impressions? The code looks a lot like C or C# although there are some new keywords like ldstr and pop that seem much more what you might expect to see in an assembly language.

So what was going on in that ILAsm code file?

First off, just in case you did not know, you have to understand that the .Net framework is stack based and that ILAsm is strictly a stack base language. ILAsm instructions take parameters from the stack and return any results to the stack. Program variables can not be addressed directly by most instructions – values need to be moved to and from the stack by specialised “load” and “store” instructions to make them available for use by other instructions.

So let’s start with the “nitty gritty” of the public method Main()

.entrypoint logically enough marks the entry point for this program

ldstr “Hello World” takes the string and loads it onto the stack.

the call void Console.WriteLine(string) instructed that a string should be taken off the stack and written to the console with the void keyword indicating that no value was expected to be returned.

the call string Console.ReadLine() instruction has no arguments and thus nothing is taken from the stack but the string keyword indicates that a string will be returned and that this should be stored on the stack

pop simply clears the stack (just being polite as we don’t need it)

ret returns from the method and in this case that terminates the program.

Let’s write a longer (but still simple) program before reviewing the code from the top.

This program is going to prompt the user for a number and print out the square of that number before repeating the process (thus we include a loop, a data type conversion and some arithmetic). If the user enters a letter “E” then the loop terminates - so some conditional execution.


.assembly extern mscorlib { }
.assembly hellworld { }
.module hellworld.exe
.namespace hello
{
.class public auto ansi world extends [mscorlib]System.Object
{
.method public static void Main() cil managed
{
    .entrypoint
    .locals init (int32 MyInt, string MyVal)
    ldstr "Hello World - I can square numbers"
    call void [mscorlib]System.Console::WriteLine(string)
TopOfLoop:
    ldstr "Please enter an integer (E to end)"
    call void [mscorlib]System.Console::WriteLine(string)
    call string [mscorlib]System.Console::ReadLine()
    callvirt instance string [mscorlib]System.String::ToUpper()
    stloc MyVal
    ldloc MyVal
    ldstr "E"
    callvirt instance bool [mscorlib]System.String::StartsWith(string)
    brtrue LoopExitTest
    ldloc MyVal
    call int32 [mscorlib]System.Convert::ToInt32(string)
    stloc MyInt
    ldloc MyInt
    ldloc MyInt
    mul
    stloc MyInt
    ldstr "The Square of your number is "
    ldloca.s MyInt  //next call needs a pointer to the value
    call instance string [mscorlib]System.Int32::ToString()
    call string [mscorlib]System.String::Concat(string, string)
    call void [mscorlib]System.Console::WriteLine(string)
LoopExitTest:
    ldloc MyVal
    ldstr "E"
    callvirt instance bool [mscorlib]System.String::StartsWith(string)
    brfalse TopOfLoop
    ret
}
}
}

OK – if you are still with me, then let’s go into a bit more detail.

First though – I am aware that this demo code is inefficient and a touch repetitive. However I hope that it will be judged as a simple example and not as pro code. Plus the code has a gaping hole that needs to be fixed but we will get to error recovery a bit further on in this post.

So from the top:

.assembly extern mscorlib { }
.assembly hellworld { }
.module hellworld.exe
.namespace hello

Here we name current assembly as well as the global .NET Framework assembly mscorlib that I used extensively through the code and I am sure that all of the calls were for familiar functions (assuming you are a.NET programmer). This section also names the executable to be created by the compiler and declares a namespace for the project – although we do not end up making much use of this within this example.

.class public auto ansi world extends [mscorlib]System.Object

declares the class and this inherits from System.Object – as in fact all such components must ultimately do.

.method public static void Main() cil managed

declares the single method within the class – it is public, static, runs as managed code and does not return a value.

.entrypoint
.locals init (int32 MyInt, string MyVal)

After the entry point for this executable I have declared two variables – a four byte signed integer (MyInt) and a string (MyVal).


   ldstr "Hello World - I can square numbers"
   call void [mscorlib]System.Console::WriteLine(string)
TopOfLoop:
   ldstr "Please enter an integer (E to end)"
   call void [mscorlib]System.Console::WriteLine(string)
   call string [mscorlib]System.Console::ReadLine()

The program then announces itself in the same way as the original “Hello World”, then we get a label which we will need to branch to later to form a loop. We then prompt the user for an integer and read in whatever the user is going to give us.

callvirt instance string [mscorlib]System.String::ToUpper()
stloc MyVal
ldloc MyVal

The program then calls the ToUpper() method of the String class which acts on the string sitting on the top of the stack. The string value is then taken from the stack and stored in the variable and then (because we still need it) places that same value back on the stack.

ldstr "E"
callvirt instance bool [mscorlib]System.String::StartsWith(string)
brtrue LoopExitTest

The program then loads the string “E” onto the stack and calls the StartsWith() method of the String class which will return a Boolean true if the user string starts with “E” and false if it does not. The returned value is placed on the stack so the brtrue instruction will branch to the “LoopExitTest” label if it is true.

ldloc MyVal
call int32 [mscorlib]System.Convert::ToInt32(string)
stloc MyInt
ldloc MyInt
ldloc MyInt
mul
stloc MyInt

Here you can see that the program makes a call to the System.Convert class to convert the user entered string to an integer. The integer returned is stored in the variable MyInt and then that value is placed on the stack twice and the mul instruction called to multiply them together. The resulting squared integer is taken from the stack and stored back in the variable because…

ldstr "The Square Of your number is "
ldloca.s MyInt //next call needs a pointer to the value
call instance string [mscorlib]System.Int32::ToString()

well in fact we are going to concatenate the value to the string “The square of your number is “ but the Int32 ToString() method needs a pointer to the address of the integer in question (rather than the integer itself) so we have to use the ldloca.s instruction to put that pointer onto the stack ready for use.


  call string [mscorlib]System.String::Concat(string, string)
  call void [mscorlib]System.Console::WriteLine(string)
LoopExitTest:
  ldloc MyVal
  ldstr "E"
  callvirt instance bool [mscorlib]System.String::StartsWith(string)
  brfalse TopOfLoop
  ret

The string Concat() method ensures that the top of the stack has our concatenated string which is written to the console by the next line. We are now at the bottom of our loop. Here we have our repetitive test to check for the user having entered “E” to exit (or indeed “e”). If they have not then we loop back to prompt for another integer. Otherwise the program terminates.

This program has a big problem though – if the user enters a string that can’t be converted to an integer then an error will be generated. We should properly deal with this issue. We could call the excellent IsNumeric() function that sits in the Microsoft.VisualBasic assembly but to extend this demonstration we can implement some (so called) structured error handling into our code.

The code around the string conversion could be modified to read:


.try {
 ldloc MyVal
 call int32 [mscorlib]System.Convert::ToInt32(string)
 stloc MyInt
 leave IsNumeric
  }
catch [mscorlib]System.Exception
 {
 pop
 ldstr "You did not enter a valid number"
 call void [mscorlib]System.Console::WriteLine(string)
 leave LoopExitTest
  }
IsNumeric:

The .try catch blocks you see above have to follow some rules. One rule is that the stack has to be empty before the .try instruction and it has to be empty before the code “exits” the block. The code has to exit the block using a leave instruction and such instructions can only branch to forward locations (otherwise the leave instruction exiting the catch block could have branched to the TopOfLoop label). This is why the code introduces a new label following the try/catch.

The pop instruction at the start of the catch block takes the System.Exception object off the stack We have specified the most generic of exceptions as we are not really interested in the exact nature of the problem.

You can compile the final version if the .IL file and give the program a whirl.

The ILDASM utility – disassembling your assemblies.

You can also see more of what ILAsm looks like by running the ILDASM tool and pointing it at one of your existing project executables.

Starting the SDK Command prompt

The best way to start up this utility is from the SDK command prompt which presets the relevant .NET SDK paths for you. Click on your computer’s Start button, Click “All Programs” and navigate to “Microsoft .NET Framework SDK v2.0” (or for VS 2003, “Microsoft Visual Studio .NET 2003” and then “Visual Studio .NET tools”) and select the “Command Prompt” option.

Running the IL DASM utility.

When the SDK Command Prompt window opens then enter ILDASM as a command. This will open the IL DASM window and you can use the File Open menu option to navigate to one of your .NET executables.

Here you can see the ILDASM utility has opened a “Hello World” project executable originally written in VB.NET. Double clicking on the “Main: void()” line displays the code as below.

2007-08-31T00:41:00.000-07:00

Had a frustrating day fighting with security within a complex multi-domain network compounded by some annoying bugs during project deployment and testing.

clr20r3 system.io.filenotfoundexception

Fix for a .Net Windows Service installation error (or more properly a failure to run after installation).

The Application Log shows:

EventType clr20r3, P1 aservice.exe, P2 1.0.0.4, P3 43d8c1c7, P4 aservice, P5 1.0.0.4, P6 43d8c1c7, P7 1a, P8 19, P9 system.io.filenotfoundexception, P10 NIL

The service installs correctly but fails either at start-up or when called upon to execute one of it’s designated tasks. The failure occurs within the .NET runtime and is not trapped by any Try/Catch within the executing module. The above error (varied by the service executable name) is seen in the Application Event Log.

The problem is caused by a missing dependency – it is very probably a COM object or a third party DLL. The DLL may be installed on the target machine but that does not mean that the .NET manifest for the service points to it’s correct location.

The quickest fix is to track down the reference in the project and set the “Copy Local” value to true in the reference properties. Then re-build your project and re-deploy it and the, now local dlls, to the directory on the target machine where the executable is located. There should be no need to re-install the service.

<Addendem>
The problem almost certainly arrises from the use of one or more shared assembly. These should be located in the Global Assembly Cache (GAC) but the service installer component in your project takes no responsibility for installing such shared assemblies on the target machine - and indeed does not even provide a check on their availability.

The work-around described above simply treats such shared assemblies as private assemblies and these are located in the same folder as the executable.
</Addendum>

Sick Citrix not helping

Annoying Citrix Error when trying to copy a couple of files between two directories. The first directory had copies of files with names similar to:

MyProg.exe and
MyProg.vshost.exe

On copying the files to the target directory I was asked to confirm that I wanted to overwrite MyProg.vshost.exe and despite my best efforts the MyProg.exe file would not copy to the target directory once there was a copy of the MyProg.vshost.exe in residence, so to speak. The problem occurs with all similarly structured file names – not just executables.

And MS Remote Desktop keeps dying when asked to open a networked drive – a 10 minute job ends up taking all day.

2007-08-27T03:38:00.000-07:00

Weekend Reading

Over the weekend I read two books with two distinct purposes. The first was “Mouse: A language for microcomputers” by Peter Grogono (now out of print) and the second “DEC Is Dead, Long Live DEC” by Edgar H Schein.

Mouse

Peter Grogono’s book is an object lesson on how to write a book on a technical subject. It is beautifully paced, well illustrated with code and introduces an amazing range of computer language concepts in the easiest manner possible. If I were ever to have charge of trainee developers again then this book and it’s content would be on the course list.

The book introduces a simply structured (Reverse Polish Notation) language by taking the reader through the process of writing a stack based interpreter. The reader is introduced to new concepts almost by diffusion – by the end they will understand the relationship between “high level” languages and assembly codes, understand how variables are organised in memory, be confident with subroutines and functions – passing arguments by value and reference and well on their way to exploring a full range of data types.

The language, Mouse, was developed from a predecessor that was a Domain Specific language used to control a recording studio and running on a DEC PDP8.

The demise of DEC

Which brings me to the second book – one I had been looking forward to reading for a while. I started my professional career as a developer using DEC kit, was a customer of theirs for many years and between times almost became an employee. I was expecting to enjoy the recounting of the foundation, growth and the ultimate demise of this archetypal technology company. DEC was above all a technology business but unfortunately this book just about ignores the technology – giving it a background status at best.

The book represents an interesting viewpoint – perhaps some might say the viewpoint of the “lice” on the body corporate – that of management consultancy, and academic management consultancy at that.

I do not doubt that whenever or wherever a couple of ex DEC people get together (and that must be a frequent event given the numbers) opinion differs on just what happened. To be fair, this book has a couple of amazing insights. There is the 1976 internal report written by Larry Portner (VP of software development) that laid out every challenge facing the business at that time – and those challenges among them that were ignored all contributed to the demise of DEC. The second comes from 1989 when it became clear that Ken Olson (founder and CEO) was just unable to grasp that a $300 dollar semiconductor chip, built by one division in DEC, could outperform his $300,000 dollar top of the range Vax that was going to go up against IBM mainframes.

Lots of people point to DEC’s seeming inability to “get” the PC (something that indirectly allowed a start-up called Compaq to thrive) as a key element in their demise. It is my view that this was only symptomatic of a more fundamental problem – indeed PC based distributed processing only reached sufficient maturity to rival minicomputers well after Digital was in real decline.

For what it’s worth – it is my view that DEC senior management never understood why it was that DEC was successful. I am sure that they got the technology part – they knew that the exciting new computers they built were eagerly sought out by their customers and they had a vision based upon interactive processing – DEC computers were the first to make computing personal. What was happening though was that their low cost interactive computers were carving out a new market – one that IBM, ICL, UNYSIS etc. just did not address.

When DEC grew to be the second largest computer company in the world they got the funny idea that to be number one they needed to attack IBM’s core market – the mainframe.

DEC had thrived by disrupting the existing computer market place. They faltered when they stopped doing that – indeed when they allowed the next disruptive generation to stomp all over them. What is amazing is that with the alpha chip (backed by a new operating system being developed by Dave Cutler) they had the technology to keep on doing what they did so well. Here we are in 2007 (the year that some DEC senior managers originally predicted they would overtake IBM to become number one) IBM has been re-created as a services business (that just happens to sell a lot of mainframes) and DEC survive as little more than a memory and some HP product lines.

Thus DEC bet their own business on the continuation of a model that they had themselves originally disrupted but then became a part of. That model was apparently easy to live within – with it’s high margins and with technology moving forward at an (almost) predictable pace. However the clues were there. DEC had grown large enough to attract an OEM market place delivering alternate printers, back-up and storage solutions all working within the de facto standards created by DEC. Above all, of course, those standards included networking and the Ethernet network model.

What DEC needed to do was to continue down their original path – taking computing into new areas of business and daily life. Yes they should have bitten the bullet and allowed themselves to build PC “clones” and (at least as importantly) created a “commodity” distribution channel for them. With their unique value added approach to taking computers outside of the computer room they would then have become the network server supplier of choice. It is self evident that the Internet is not built on DEC technology – but it might well have been.

Oh yes, the book. It did not work for me but if you can stand reading the word "culture" more than one in every paragraph, it might be of interest to you.

A good yarn to gladden the heart of any Old Fogy.

I am not a big fan of books that use devices like an exchange of letters to tell their tale – although it can just work. The tale that unfolds here in a sequence of emails is however well worth following – even if the mathematical terms are from outside of your knowledge domain. It does not matter – the consequences of them are outside the knowledge domain of at least one of the protagonists.

2007-08-23T00:13:00.000-07:00

A chip of the Old Block?

Lovely post by Reg Braithwaite on bringing code blocks to JavaScript. The idea being to write blocks of code that contain variable declarations with local scope. The approach, which is ably demonstrated with code, tackles the issue in a manner that does not require the user of named functions to wrap code blocks which avoids a persistent code maintenance headache – “where else might this function be referenced?”.

If you are into JavaScript go check it out – if you are not then go and take a look anyway as you will be impressed by the flexibility and power of the language.

Does it run on Linux?

If your idea of “Open Source” on the Microsoft platform is that the genre is limited to “tricks and tweaks” on the web then you might like to review this post which is a list (edited by one person) of the top 25 project hosted at CodePlex. If you add this to the vast wealth of Windows based projects hosted at SourceForge and elsewhere you might well be persuaded that the open source movement has a long history and a wide scope on Microsoft platforms – and open source software is certainly not all about Linux.

Old Fogey

And if you are feeling just a little bit under-whelmed by the code just submitted by a junior software developer then, like J. Timothy King perhaps you just have to accept that you are now officially an old fogey.

A quote from the post: “I’ve discovered a new rule: When your junior colleague thinks his fancy object-oriented code is simpler than your straightforward procedural code, you know it’s time to raise your rates. Because they’re not listening to you, so you know you can’t be charging them enough.”

2007-08-17T05:06:00.000-07:00

Past programming futures

I was grateful to a link from the programming sub-reddit to the QB Express on-line magazine. Wow! A revelation. Flame wars, incoherent racist rants, Cellular Automata, maths, code listings and games, games, games. But what’s this – an article on the Singleton Pattern and another on project management. Quick Basic and the DOS window are alive and well and their proponents seem to be having lots of fun.

I suppose that the lure of Quick Basic and the DOS environment is both the straightforward nature of a procedural language matched to a relatively simple “machine”. Programming to the DOS machine cuts out a lot of options and probably a good few uncertainties – it’s just the code and the monitor – with perhaps a dash or two of DirectX magic.

Related link – http://freebasic.net announce v0.18.1b release. Hosted at SourceForge this is a project to construct a 32bit Basic compiler compatible with MS Quick basic but with a few extensions (like pointers) to make C libraries fully available to the developer. Sounds an interesting project.

In another direction I was reviewing Scott Hanselman’s diagram of the .NET ecosystem as I had hoped it would help me decide if I needed to get into WPF. I had read Eric Sink’s posts on 3D WPF and wondered if I should be reading Charles Petzold’s books on the subject. Is WPF the next big thing or is it destined to be a desktop niche in a Web 2+ world? I suppose part of the answer to that is tied up with Silverlight and it’s take up by developers but that just introduces another variable into the equation. There is also the issue of a Vista development machine for which office space must be found – and the jury is still out in Vista – certainly within corporate environments.

The .NET desktop UI and it’s attendant GUI development environment is essentially the same as the VB classic model – a wrapper for standard Win32 two dimensional bitmap windows and widgets. If I get this right, WPF starts afresh by describing the UI in vectors with a new graphical engine to translate those vectors to a bitmap for the current output device. WPF also allows us to describe “absolute” dimensions rather than measuring everything in pixels. <Asside> One of the genuine losses in moving from VB Classic to .NET was the loss of inbuilt absolute measurements as we had a smart backwards step to the Win32 “Any measurement you like as long as it is Pixels” approach. While twips were a bit weird it was nice to be able to “draw” a rectangle 10cm by 12 cm and know it would be rendered at that size on screen or to a printer (allowing for any slight rendering inaccuracies of up to a pixel)&tl;/Asside> Converting to a vector based graphical environment has many benefits – including smooth proportional zooming and (of course) 3D objects that can be rotated and “distorted” without complications or necessarily knowing a lot of math.

I suppose the big question hanging in the air is – is 3D going to enable new things in traditional application areas or deliver new sorts of applications? I might regret this but I rather suspect that 2D is going to continue to rule outside of CAD and modelling applications while the world of games is looking for rather more than can be delivered by WPF as it currently stands.

JQuery and JavaScript

Simon Willison has a nice write-up of the JQuery JavaScript tool set which is nicely aimed at experienced developers. Well worth a read if you are still undecided about such helper libraries or if you are thinking about changing. Simon ably demonstrates the utility of JQuery and illustrates the most interesting features with some snappy code samples.

2007-08-13T03:30:00.000-07:00

The lure of interesting problems

We can all get caught up in solving interesting problems – even when simple alternate solutions are staring us in the face. Example:

A program that had been running as part of a financial application since 1992. In the year 2000 I was asked to change the program so that it output a precisely formatted document alongside it’s existing functionality. As the program was running on a VMS system this document had to be achieved using ASCII character output only – so that left PostScript as the obvious option even though I knew nothing about it’s syntax. As one of the techies at that business had a PostScript manual of sorts at the back of a shelf I learned enough PostScript to produce the document – although I wisely appended a long description of how this had been done in comments within the program – a message to myself or anyone who followed. Seven years later – up popped a bug. How wise had I been? – well not very, as it turned out. The document output contained a name and an address and it had become customary among those running the system to use brackets within these fields – something like “United Consolidated Industries (trading as ACME)” perhaps.

Anyone who knows anything about PostScript is already laughing and (correctly) classifying me as a hopeless beginner. Let me explain. Text strings in PostScript are enclosed within brackets. Matched pairs of left and right hand brackets within those text delimiting brackets are treated as part of the string. The bug, of course, was the first time that an unmatched bracket turned up within such a string. This completely “threw” the PostScript interpreter of the printer and all subsequent output following the error was ignored.

It gets worse though. I had been doing a few things that week that involved parsing strings. So right away I saw this as a parsing problem. I wrote a function that used an array as a “stack” that parsed each string at risk – ensuring that only matched pairs of brackets – no matter how deeply nested – reached the PostScript print file. A neat elegant solution that I was feeling quite warm about until I started to laugh. The common sense bit of my brain had at last got my attention and pointed out that PostScript must have a string escape character and that it was probably the backslash (these things tend to get copied from one domain to another). Of course common sense was right as a Google search quickly confirmed (I told you I did not know any PostScript) – the true solution to the bug was simple and pragmatic – but nothing like as interesting as my function.

So apart from persuading potential customers that my consultancy services are to be avoided – is there any point to this little confession? If we can get caught up by an interesting problem (in my case) to the point of missing the obvious then might that apply to larger software architectural building blocks as well – perhaps even to whole systems. Yes – I am alluding to my post of last week that touched on the “Big Ball of Mud” (BBOM) pattern and it’s place as a de-facto standard for software architecture. Remember – systems based upon that pattern can be long lived and prove cost effective – above all they number among the projects that made it to implementation. A great many projects never make it. It is held that a common cause of project failure is a lack of good design – and I am assuming for the moment that proponents of “good design” are unlikely to include the BBOM pattern in the category of “good”. Might a deliberate lack of a coherent design in fact be good design? Are there characteristics of successful (defined as implemented) BBOM based systems that support their success and differentiate them from the (sometimes elegantly designed) failures that never make it into production? I suppose I am wondering if there is good design deep within the BBOM pattern – or is that just too inconsistent a thought – even for me?

2007-08-10T06:25:00.000-07:00

Stating the obvious.

Sometimes the obvious needs to be stated so that it becomes obvious and allows us to think more clearly about the consequences of the obvious.

Try this post at AddRef that makes the very clear case that classes are typically only available for re-use when compiled into a DLL or LIB and that most classes get “lost” to future use by being compiled into executables. The author is stating an obvious truth about how we actually build and implement classes as developers. The consequence of that truth is that we have to recognise that “in practice” we are re-using classes by copying and pasting them from one source code location to another – thus “forking” not re-using them in the true sense. The author of the post is right in thinking that re-usable code should be drawn off into a library even if that means that a good many executables contain little more than the UI functionality – but that’s not what happens.

So next time you hear someone arguing the benefits of OOP (and maybe design patterns) probably citing code re-use as a prime example – tackle them on the architectural implications and how they are crucial in delivering those benefits. As far as I can see OOP has delivered a lot less than was expected and that may be partly a function of a larger design pattern than those typically encapsulated by a class.

Continuing the theme you will probably enjoy a paper written by Brian Foote and Joseph Yoder on the most common design pattern of all – then one the paper’s authors title “The Big Ball of Mud”. The authors make the case that the deliberate or accidental exclusion of a planned (and implemented) architecture from most systems is a deliberate pattern that reflects the reality of the software development environment. The paper is well worth reading – as are the conclusions The chief conclusion from the paper is that developers build “Big Balls of Mud” because they work. I saw something else in the paper – I thought that the most interesting idea was that design/architecture was a project cost overhead that rarely made the budget. This being compounded (remember Brooke of the Mythical Man Month) by the fact that a good design often (maybe always) requires you to throw away your first effort and start again. Good design is expensive in time and money.

I was analysing a fairly typical Big Ball of Mud yesterday with the purpose of documenting the key functionality in order to help a customer decide if they should replace the existing system – and if so – what manner of replacement might best be implemented. I have a funny feeling though that the best option might be to slap a little lipstick on the pig and keep this fifteen year old system running a while longer. You see it’s not really broken at all and maintenance costs are tiny – usually associated with “cock-ups” in other systems and occasional small tweaks following the acquisition of other businesses – and thus inheriting some of their contractual agreements.

Corporate software is surprisingly long lived. It often outlives the original hardware and (this case in point) runs smoothly on as more recent generations of development tools and design methodologies have come and (in some cases) gone. I wish I could surmise that it is only systems with at least some past quality at their core that endure as (or to become) those “Big Muddy Balls” but I can site too many extant examples that should have been strangled at birth but have lived on for decades – their replacement costs always exceeding the perceived costs of their inefficiencies.

I am not quite sure what all this is telling me. Is “good design” worth the effort or is it just a way of engaging developers in their task? Do we overrate design as an attribute? Can I come up with a new development methodology that sets out to produce cheap and cheerful “Balls of Mud” and thus make a killing in the big name consultancy market? Maybe it’s just that Friday afternoon feeling again.

2007-08-09T08:50:00.000-07:00

Just how far is that?

I was taking a passing look at a piece posted to the Code Project titled GPS Receivers, Geodesy and Geo-caching by a Mike Gavaghan which looked at applying Vincenty’s formulae for calculating the distance between two points on the earth’s surface taking into account the fact that the earth is not a perfect sphere. Good, indeed great, stuff, but of course the answer produced still fails to provide a really useful result. Real world distances are about scale and route. Let me give a couple of examples.

A short while ago we were much amused to receive an unsolicited “offer” to provide a map for our web site showing our location and the distances to nearby hotels etc.. Part of our amusement stemmed from the fact that a lot of our revenue derives from mapping and location software but most came from the fact that all of the example distances supplied were wrong. You see, our company office is located on an island separated from the mainland by a stretch of water notoriously difficult to navigate. To get to all of the places selected by the proffered service you had to travel along the coast, take a bridge and then travel back along the mainland coast – yet all of the distances had been calculated arithmetically from the grid coordinates – presumably of the postal codes. This is why route is important and in a world of GPS supported driving directions pretty obvious one would have thought.

Now imagine that you plan a trip of (say) 10 miles or 25 kilometres across some mountainous or hilly countryside with the distance calculated using the map scale. Now if you were to walk that route pushing a measuring wheel in front of you on your journey you would quickly notice that the distance clocked was considerably greater than the distance on the map. This is the scale factor - you have to go up and down the hills as well as make progress across the notional plane of the map. If you were the size of an ant, the distance measured would be many orders of magnitude greater than the results you would obtain human sized. An ant has to climb every pebble and drop down into most fissures and cracks.

What has all this got to do with software? Well in the world of mapping and co-ordinate software you have to be able to calculate distances taking into account the shape of the world as otherwise the other software vendors will laugh at you and tell all of your customers that your software is rubbish. However, in reality there are very few times that such a calculation is actually useful. In most knowledge domains you want to know something else – like how long it will take you to get from A to B given a bunch of choices - like mode of transport and other preferences. On a more general front we have to be careful that our software is not just rigorous but practical. We developers can enjoy the beauty of an elegant calculation but the user might just want to know if they should take a bus.

2007-08-06T06:21:00.000-07:00

What now, VB6 Code?

Well I needed to locate a personal directory for users working via Citrix where all of the user processes are running on the same server. This preclude my using a directory associated with the application as all of the users would be working with the same one. An obvious choice is the “My Documents” folder for each user although it does have it’s snags. While locating this folder using VB.NET is pretty trivial getting the same value from the Registry using VB Classic and the Windows API takes a bit of setting up (plus some trawling through the registry) even if the code itself is pretty short.

So, in case anyone else wants to locate the My Documents folder using Visual Basic (VB) 6 – here is the code:

Caution – while this code only reads from the Windows Registry using the Win32 API any interaction with the Registry does have it’s risks – so go careful.

First you need to declare some functions and constants:


Private Declare Function RegOpenKeyEx Lib "advapi32.dll" Alias "RegOpenKeyExA" (ByVal hKey As Long, ByVal lpSubKey As String, ByVal ulOptions As Long, ByVal samDesired As Long, phkResult As Long) As Long
Private Declare Function RegCloseKey _
    Lib "advapi32.dll" _
    (ByVal hKey As Long) As Long

Private Declare Function RegQueryValueExString _
    Lib "advapi32.dll" Alias "RegQueryValueExA" _
    (ByVal hKey As Long, _
     ByVal lpValueName As String, _
     ByVal lpReserved As Long, _
     lpType As Long, _
     ByVal lpData As String, _
     lpcbData As Long) As Long
    
Private Declare Function RegQueryValueExNULL _
    Lib "advapi32.dll" Alias "RegQueryValueExA" _
    (ByVal hKey As Long, _
     ByVal lpValueName As String, _
     ByVal lpReserved As Long, _
     lpType As Long, _
     ByVal lpData As Long, _
     lpcbData As Long) As Long
Private Const HKEY_CURRENT_USER = &H80000001
Private Const REG_SZ = 1                         ' Unicode nul terminated string
Private Const KEY_QUERY_VALUE = &H1
Private Const KEY_ENUMERATE_SUB_KEYS = &H8
Private Const KEY_NOTIFY = &H10
Private Const READ_CONTROL As Long = &H20000
Private Const STANDARD_RIGHTS_READ = (READ_CONTROL)
Private Const SYNCHRONIZE = &H100000
Private Const KEY_READ = ((STANDARD_RIGHTS_READ Or KEY_QUERY_VALUE Or KEY_ENUMERATE_SUB_KEYS Or KEY_NOTIFY) And (Not SYNCHRONIZE))
Private Const ERROR_NONE As Long = 0
Private Const ERROR_KEY_DOES_NOT_EXIST As Long = 2

but the actual code for reading the key can be simplified to:


Private Sub GetMyDocsPath()
    Dim RegKey As String,   sValue As String

    Dim lRetVal As Long, lHandle As Long, iType As Long, iLength As Long   
   
    RegKey = "SOFTWARE\MICROSOFT\WINDOWS\CURRENTVERSION\EXPLORER\SHELL FOLDERS"
    lRetVal = RegOpenKeyEx(HKEY_CURRENT_USER, RegKey, 0, KEY_READ, lHandle)
    If lRetVal = ERROR_NONE Then
        lRetVal = RegQueryValueExNULL(lHandle, "PERSONAL", 0, iType, 0, iLength)
        If lRetVal = ERROR_NONE And iType = REG_SZ Then
            sValue = String(iLength, 0)
            lRetVal = RegQueryValueExString(lHandle, "PERSONAL", 0, iType, sValue, iLength)
            If lRetVal = ERROR_NONE Then
                MyDocPath = Left(sValue, iLength - 1) & "\"
            End If
        End If
        RegCloseKey lHandle
    End If
End Sub

2007-07-31T02:22:00.000-07:00

Data Entry from the Clipboard

One of my customers has a large retail arm with that retail group being managed within a multi-tear hierarchy. Acquisitions and disposals result in regular re-organisations of the management hierarchy and these can be a headache for some of the systems operated in the relevant division. Part of the problem is to do with timing – in the end the company’s central SAP system will reflect the changes but some systems need to be updated before the central accounting and reporting systems. They were therefore looking for a simple method to update an application that could be driven from a primary source such as an Excel spreadsheet.

It would be trivial to load Excel spreadsheet content into an SQL database as a table and then to run the relevant updates but as the source document structure would inevitably change (in a haphazard manner?) this would require the intervention of some technically able resources. Much better to go for a really simple approach – paste some selected columns of data into a grid, validate the content and then update the system from there.

I realised that it was a long time since I had written code to support pasting data and was happy to take a look and see what the .NET framework had to offer that advanced what was available using the Win32 API (this being an area where classic VB was never at the top of it’s game).

As this blog is supposed to be about code – let’s write some.

I started with the easy stuff – a filter to inspect the content of the clipboard to see if the data types available there were generically suitable for pasting into a table – in this instance a ListView control set to act like a grid. The paste option itself was a menu item that was a sub-item within an Edit menu item group so I put some code under the Select (not Click) event for the Edit menu item:


 Private Sub mnuEdit_Select(ByVal sender As Object, ByVal e As System.EventArgs) Handles mnuEdit.Select
     Dim DataObject As IDataObject = Clipboard.GetDataObject
     Dim IsHTML As Boolean = DataObject.GetDataPresent(DataFormats.Html)
     Dim IsCSV As Boolean = DataObject.GetDataPresent(DataFormats.CommaSeparatedValue)
     Dim IsText As Boolean = DataObject.GetDataPresent(DataFormats.Text)
     mnuPaste.Enabled = IsHTML Or IsText Or IsCSV
 End Sub

I had decided to support multiple data formats as I could not be sure of the source of the data the user might want to paste – not even sure that it would be from an MS Office document. There is a good range of native data formats available by default – perhaps oddly missing XML but including HTML, CSV and plain old vanilla TEXT). You can copy and paste custom data objects of course (see) but that is outside of this immediate context. Thus the code above enables the paste menu item when any or all of three data types are available on the clipboard. The HTML and Text data types are essentially strings but the detail of the content will change depending upon the data source so any code managing the actual paste event has to be flexible. The Comma Separated Variable (CSV) data type is a stream and needs slightly different treatment but boils down to the simplest data source when available – even if you have to allow for commas within delimited strings (which are a pain).

As you might expect, the content of the HTML data type posted to the clipboard by MS Office programs is full of layout and style information that needs to be ignored but fortunately the tags help you find the essentials – the “cell” values you are looking to paste. I experimented with what turned up when selecting and copying from both MS Excel and MS Word. The Excel contribution is fairly straightforward – if you ignore the crud and stick to the content of the <TR> and <TD> tags.

If you copy columns of data just typed into an MS Word page you get something like:

<P>data value<some whitespace>data value<some whitespace>data value</P>

for each “row” of data.

If you copy some data from an MS Word table then you get an HTML table but with the data cell content also wrapped in <P> tags so something like:


<TR><TD><P>data value</P></TD><TD><P>data value</P></TD></TR>

The plain text equivalents are pretty straightforward. For both Excel and MS Word Tables you get cell content separated by “tab” characters with a row being completed with carriage return and line feed characters. Text from a non-formatted area of an MS Word document is just as you would expect from the equivalent typed into NotePad.

Armed with all this choice, you have to decide on your personal “hierarchy” of data types to accept. I should sound a small note of caution here. Not all programs correctly clear the clipboard when adding data to it. This can mean that a user program could load (say) text data into the clipboard alongside existing data – perhaps in HTML format. This can cause unexpected results and a deal of user frustration. Apart from resolving to always copy data onto the clipboard properly yourself there is not much you can do about this problem although one answer might be to think carefully about your data hierarchy – perhaps taking the simplest data format in preference to those that might “persist” on the clipboard from unintended sources.

Pasting the clipboard content into your grid is a pretty straightforward process that extracts each row of cells in turn from the input string and then adds the rows to the grid. However this is where you also have to consider the “shape” of the data being pasted and perhaps decide how you are going to handle differences between the number of cells in each row being presented and the number of cells in your target grid. In my simple instance I am only looking for two cells in each row and will accept any number of rows – adding then to the grid. In other circumstances you might be writing code that pasts blocks of data into a specified portion of a grid or inserting data into a grid with (notionally) infinite dimensions. You might have to parse all of the rows represented in the paste string to determine the dimensions (in cells and rows) of the data being pasted before inserting the actual cells values – you might need the user to make some choices before proceeding.

For the purposes of this post I have encapsulated the paste functionality within a class that, to be honest, is ridiculously over-engineered for the task in hand but would serve as a platform for further development and exploration. I am sure that you have spotted that what we need to implement is a simple parser – simple because there is no need to support sub-expressions – all we need to do is to detect columns of data within rows. Classically parsers use stacks to parse a given statement but in this instance we are only parsing left to right so we can use a Queue object rather than a Stack object to store the resulting “grid” of data.

The HTML table parser is the most complex of the three parsing routines because its main task is to ignore most of the HTML content as irrelevant to the requirement. You might have thought that you could use the XMLReader class for this task as HTML is a mark-up language but unfortunately HTML is not a “well formed” mark-up language and will quickly cause errors. You could experiment with the SQMLReader available for download from the (still available) GotDotNet user samples site as an alternative. The code shown here for parsing the HTML is just about acceptable but is very close to the edge – probably needs a properly formed recursive parser to handle the task.

VB .NET class to manage pasting data from the clipboard into a grid on a form:


Imports System.IO
Imports System.Text
Public Class GridPaste
#Region " Public Enum"
 Public Enum PDataType
     CSV = 1
     HTML = 2
     Text = 4
 End Enum
#End Region
#Region " Private Declarations"
 Private HTMLString As String
 Private CSVString As String = ""
 Private TextString As String = ""
 Private gotType As Int16 = 0
 Private MaxCols As Integer = -1
 Private RowCount As Integer = -1
 Private GridRows As Queue

#End Region
#Region " Public Constructor Methods and Properties"
 Public Sub New()
     Dim DataObject As IDataObject = Clipboard.GetDataObject
     If DataObject.GetDataPresent(DataFormats.CommaSeparatedValue) Then
         Dim SReader As New StreamReader(CType(DataObject.GetData(DataFormats.CommaSeparatedValue), Stream))
         CSVString = SReader.ReadToEnd
         SReader.Close()
         gotType += PDataType.CSV
     End If
     If DataObject.GetDataPresent(DataFormats.Html) Then
         Dim HTMLPaste As New StringBuilder(DataObject.GetData(DataFormats.Html).ToString())
         'Upcase selected tags
         HTMLPaste.Replace("<td", "<TD")
         HTMLPaste.Replace("</td", "</TD")
         HTMLPaste.Replace("<tr", "<TR")
         HTMLPaste.Replace("</tr", "</TR")
         HTMLPaste.Replace("<p", "<P")
         HTMLPaste.Replace("</p", "</P")
         HTMLPaste.Replace("<table", "<TABLE")
         HTMLString = HTMLPaste.ToString
         gotType += PDataType.HTML
     End If
     If DataObject.GetDataPresent(DataFormats.Text) Then
         TextString = DataObject.GetData(DataFormats.Text).ToString().Trim
         gotType += PDataType.Text
     End If
 End Sub
 Public Function IsFormat(ByVal datatype As PDataType) As Boolean
     If gotType And datatype Then
         Return True
     Else
         Return False
     End If
 End Function
 Public Function GetRowCount() As Integer
     If RowCount = -1 Then
         BestParse()
     End If
     Return RowCount
 End Function
 Public Function GetColumnCount() As Integer
     If MaxCols = -1 Then
         BestParse()
     End If
     Return MaxCols
 End Function
 Public Sub PasteToGrid(ByRef aListView As ListView, ByVal datatype As PDataType)
     Select Case datatype
         Case PDataType.CSV
             ParseCSV()
         Case PDataType.HTML
             If HTMLString.IndexOf("<TABLE") > -1 Then
                 ParseHTMLTable()
             Else
                 ParseText() ' as that's all you are getting wraped up in <P> tags
             End If
         Case PDataType.Text
             ParseText()
     End Select
     AppendToGrid(aListView)
 End Sub
 Public Sub PasteToGrid(ByRef aListView As ListView)
     'alternate call that looks for best match that results in columns and rows being pasted
     BestParse()
     AppendToGrid(aListView)
 End Sub
#End Region
#Region " Private Methods"
 Private Sub BestParse()
     If gotType And PDataType.HTML Then
         If HTMLString.IndexOf("<TABLE") > -1 Then
             ParseHTMLTable()
         End If
     End If
     CountCells()
     If RowCount <= 0 Or MaxCols <= 0 Then
         'try next format
         If gotType And PDataType.CSV Then
             ParseCSV()
             CountCells()
         End If
         If RowCount <= 0 Or MaxCols <= 0 Then
             If gotType And PDataType.Text Then
                 ParseText()
                 CountCells()
             End If
         End If
     End If

 End Sub
 Private Sub CountCells()
     RowCount = -1
     MaxCols = -1
     If Not IsNothing(GridRows) Then
         Dim ThisRow As Queue
         Dim RowEnumerator As Collections.IEnumerator = GridRows.GetEnumerator
         RowCount = GridRows.Count
         While RowEnumerator.MoveNext
             ThisRow = RowEnumerator.Current
             If ThisRow.Count > MaxCols Then
                 MaxCols = ThisRow.Count
             End If
         End While
     End If
 End Sub
 Private Sub AppendToGrid(ByVal aListView As ListView)
     If Not IsNothing(GridRows) Then
         Dim ThisRow As Queue
         Dim NewRow As ListViewItem
         Dim ColNumber As Int16
         Dim RowEnumerator As Collections.IEnumerator = GridRows.GetEnumerator
         Dim ColEnumerator As Collections.IEnumerator
         While RowEnumerator.MoveNext
             ThisRow = RowEnumerator.Current
             ColEnumerator = ThisRow.GetEnumerator
             ColNumber = 0
             While ColEnumerator.MoveNext
                 ColNumber += 1
                 Select Case ColNumber
                     Case 1
                         NewRow = aListView.Items.Add(ColEnumerator.Current)
                     Case Is <= aListView.Columns.Count
                         NewRow.SubItems.Add(ColEnumerator.Current)
                 End Select
             End While
         End While
     End If
 End Sub

 Private Sub ParseText()
     PreProcessText()
     GridRows = New Queue
     Dim ThisRow As New Queue
     Dim NextChar As String, CellContent As String
     For CharPos As Integer = 0 To TextString.Length - 1
         NextChar = TextString.Substring(CharPos, 1)
         Select Case NextChar
             Case vbTab
                 ThisRow.Enqueue(CellContent)
                 CellContent = ""
             Case vbLf
                 ThisRow.Enqueue(CellContent)
                 CellContent = ""
                 GridRows.Enqueue(ThisRow)
                 ThisRow = New Queue
             Case Else
                 CellContent &= NextChar
         End Select
     Next
 End Sub
 Private Sub ParseCSV()
     GridRows = New Queue
     Dim ThisRow As New Queue
     Dim LineEnd As Boolean = False
     Dim NextChar As String, CellContent As String = ""
     Dim CharPos As Integer = 0, RowEnd As Integer, DelimEnd As Integer
     While CharPos < CSVString.Length
         NextChar = CSVString.Substring(CharPos, 1)
         CharPos += 1
         Select Case NextChar
             Case Chr(34)
                 'should be a string enclosed in quotes
                 RowEnd = CSVString.IndexOf(vbCrLf, CharPos)
                 If RowEnd = -1 Then
                     RowEnd = CSVString.Length
                 End If
                 DelimEnd = CSVString.IndexOf(Chr(34), CharPos, RowEnd - CharPos)
                 If DelimEnd > -1 Then
                     CellContent = CSVString.Substring(CharPos, DelimEnd - CharPos)
                     DelimEnd += 1
                     CharPos += DelimEnd - CharPos
                 Else
                     CellContent &= NextChar
                 End If
                 LineEnd = False
             Case ","
                 ThisRow.Enqueue(CellContent)
                 CellContent = ""
             Case vbCr, vbLf
                 If Not LineEnd Then
                     LineEnd = True
                     ThisRow.Enqueue(CellContent)
                     CellContent = ""
                     GridRows.Enqueue(ThisRow)
                     ThisRow = New Queue
                 End If
             Case Else
                 CellContent &= NextChar
                 LineEnd = False
         End Select
     End While
 End Sub
 Private Sub ParseHTMLTable()
     GridRows = New Queue
     Dim ThisRow As Queue
     Dim RowStart As Integer, RowEnd As Integer, NextCell As Integer
     Dim CellStart As Integer, CellEnd As Integer, PStart As Integer, PEnd As Integer
     Dim CellContent As String = "", TagCell As String
     Dim GoodRow As Boolean = False
     Dim NextRow As Integer = HTMLString.IndexOf("<TR")
     While NextRow > -1
         If GoodRow Then
             GridRows.Enqueue(ThisRow)
         End If
         ThisRow = New Queue
         RowEnd = HTMLString.IndexOf("</TR", NextRow)
         GoodRow = RowEnd > -1
         NextCell = HTMLString.IndexOf("<TD", NextRow, RowEnd - NextRow)
         While NextCell > -1
             'we need to find the start of the actual cell content as well as it's end
             CellStart = HTMLString.IndexOf(">", NextCell, RowEnd - NextCell)
             If CellStart > -1 Then
                 CellStart += 1
                 CellEnd = HTMLString.IndexOf("</TD", CellStart, RowEnd - CellStart)
                 If CellEnd > -1 Then
                     'we have a cell so
                     CellContent = HTMLString.Substring(CellStart, CellEnd - CellStart)
                     'but it might still be wrapped in a <P> tag set or contain even more junk tags yet so
                     PStart = CellContent.IndexOf("<P")
                     If PStart > -1 Then
                         PStart = CellContent.IndexOf(">", PStart)
                         PEnd = CellContent.IndexOf("</P", PStart)
                         If PEnd > -1 Then
                             PStart += 1
                             CellContent = CellContent.Substring(PStart, PEnd - PStart)
                         End If
                     End If
                     PStart = CellContent.IndexOf("<")
                     If PStart > -1 Then
                         PEnd = CellContent.IndexOf(">", PStart)
                         If PEnd > -1 Then
                             PEnd = CellContent.IndexOf(">", PEnd + 1)
                             If PEnd > -1 Then
                                 If PStart > 0 Then
                                     TagCell = CellContent.Substring(0, PStart)
                                 Else
                                     TagCell = ""
                                 End If
                                 If PEnd < CellContent.Length - 2 Then
                                     PEnd += 1
                                     TagCell &= CellContent.Substring(PEnd, CellContent.Length - PEnd)
                                 End If
                                 CellContent = TagCell
                             End If
                         End If

                     End If
                     ThisRow.Enqueue(CellContent)
                     CellContent = ""
                 Else
                     CellEnd = RowEnd
                 End If
             End If
             NextCell = HTMLString.IndexOf("<TD", CellEnd, RowEnd - CellEnd)
         End While
         NextRow = HTMLString.IndexOf("<TR", RowEnd)
     End While
     If GoodRow Then
         GridRows.Enqueue(ThisRow)
     End If
 End Sub
 Private Sub PreProcessText()
     Dim PasteText As New StringBuilder
     Dim IsWhite As Boolean = False, IsTerminal As Boolean = False
     Dim NextChar As String
     For Charloop As Integer = 0 To TextString.Length - 1
         NextChar = TextString.Substring(Charloop, 1)
         Select Case NextChar
             Case " ", vbTab
                 If Not IsWhite Then
                     IsWhite = True
                     PasteText.Append(vbTab)
                 End If
             Case vbCr, vbLf
                 If Not IsTerminal Then
                     IsTerminal = True
                     PasteText.Append(vbLf)
                 End If
                 IsWhite = False
             Case Else
                 IsWhite = False
                 IsTerminal = False
                 PasteText.Append(NextChar)
         End Select
     Next
     If PasteText.Chars(PasteText.Length - 1) <> vbLf Then
         PasteText.Append(vbLf) 'ensure last row is completed
     End If
     TextString = PasteText.ToString()
 End Sub
#End Region
End Class

Implementing the class functionality is pretty trivial – it could be as simple as

   Dim GridPaster As New GridPaste
 GridPaster.PasteToGrid(MyListView)

in the code block for the edit menu paste event – although you would probably want to make use of the row and column counting functionality or add additional content validation before proceeding.

Useful extensions – The class could be usefully extended to support DataGrids. Another feature you might like to consider would be “Drag and Drop” support for files. For more general applications, you might very well want to support the overwriting of user selected data in the target grid.

Links:
If you end up processing very large CSV encoded data files you might like to check out Sebastian Lorion’s “Fast CSV Reader” at The Code Project
Parsing HTML using the SQMLReader at GotDotNet

2007-07-27T04:52:00.000-07:00

I did not think I would ever do this but

I have translated some VB Classic code to VB.NET. I have always promoted the line that VB Classic applications should not be translated into VB.NET but should be redeveloped from scratch to take full advantage of the .NET Framework.

I needed to quickly move some existing VB6 functionality at a customer site into a Windows Service as the process needed to be run automatically to a regular time schedule and that was not happening reliably enough. I had a choice I suppose – I could have pasted the VB6 code into a new project that was compiled as a VB Classic DLL and then called the resulting COM object from my service. But then I though – how hard can it be? OK, the code reads data from MS Access databases, uses User Defined Types and even has a GoSub or two – but the two approaches looked likely to have similar time overheads and it is always interesting to try something new.

How did it go? Took a couple of hours – the MS Access database connections were still handled by DAO as I needed to support client side code to handle data locks within a very active set of databases. The User Defined types translated seamlessly into Structures and a little tidying sorted out the code structure. A combination of “My.Settings/app.config” and an editable XML file dealt with things like environment variables and INI files (yes this app went back a long way). And then I had a think about the number of changes I had made and the fact that my long experience with .NET had made those changes both predictable and straightforward. I wondered just how I would have coped if this exercise had been my first experience of VB.NET. This was not a hard task but the sheer number of seemingly meaningless error messages and squiggly lines after you pasted code into the new project would be capable of breaking the toughest heart.

Partly confirmed my prejudices – a translation project is no way to learn .NET although it is perfectly feasible if you are already very familiar with the .NET Framework – although that begs a question or two.

Finding Framework Functionality

I don’t know about you but every time I think I am getting “on top” of the .NET Framework I discover something new and get worried that there might be more to be found that I have learned so far. Example:

I was reading a post by Matt Gertz who is a member of the VB language development team where he ‘fessed up to abusing the List Of() functionality to create a list of lists to support “undo” functionality in a demo VB app. Another member of his team pointed out the Stack object and Stack Of() of which he had been previously unaware. Now a stack would have been ideal for Matt; as it is a LIFO (Last In First Out) object and that was just what he needed to support “undo” functionality. Well that got me thinking, because a few days before a colleague had been working on a FIFO (First In First Out) object. After a bit of searching I found the Queue class and Queue Of() – just what had been needed. Neither the Stack or Queue class appeared in the Visual Studio help files as links from the List class or the Array class from which they are all derived.

How are you supposed to find this stuff? Plus don’t get me started on the version changes that invalidate huge areas of help available via your search engine of choice. Yes – almost all changes are framework changes but those have code implications. If you are regularly developing using the .NET Framework then you are probably tracking the main changes but again it is tough on someone just starting out.

Language Bloat?

The Beta2 of Visual Studio 2008 is now available for download and arrives at a time when a good number of developers are beginning to wonder about language bloat. There is a worry that language development teams keep packing in new features to little avail – are they making the languages more effective or are we just seeing a form of language marketing with a “feature race” designed to appeal to corporate decision takers?

Anyone who reads my blogs (erm anyone out there?) will know that I am a bit of an enthusiast for the nitty gritty of languages and compilers but they will also have noted (I hope) that I like my tools to provide effective feature while minimising duplication of functionality and I just hate features that feel like they are just there to “dumb down” the craft of code development. So bring on Lamda expressions and operator overloading in VB.NET but do I really need yet another way to select data?

2007-07-24T08:09:00.000-07:00

This is totally off topic - or maybe more on topic than my usual stuff

Message en français ci-dessous

My nearest neighbour’s granddaughter Ilona is gravely ill and if a bone marrow transplant donor can’t be found in the next few days she is certain to die. Until illness struck her a few weeks ago she was a constant visitor to her grandparent’s house – and a playmate of my own children.

My request is simple – if you can, would you please sign up as a potential bone marrow donor. All it takes to register is a simple blood test. The chances of any individual being a good match for Ilona is small but if enough people register then there is just a chance that one of us will be able to save her life. If you register in Europe then your details can be matched to Ilona – if you live elsewhere then there may be others in equally desperate need.

This is what my neighbours have written

Notre petite fille Ilona (5 ans) est attente d’une tres grave maladie, dîtes: (maladie orpheline): ses chances de survie sont tres minces.
Seule une greffe de moelle osseuse (á 100%) peut la sauver.
Si je me permets de vous crier mon desespoir c’est que je pensais que ces malheurs ne se voyaint qu’à la télévision chez les autres mias aujourd'hui c’est à nous que cela arrive…

Faire un don de moelle c’est sauver une vie.

renseignez-vous à Bordeaux au : 05 56 90 83 70
Numero vert : 0 800 02 07 93

2007-07-12T03:15:00.000-07:00

Chinese whispers in the supply chain

I have just been working on a “one off” report for a customer that required crunching quite a few numbers extracted from various databases to prepare a report requested by one of their customers. Nothing unusual about that – but also – nothing unusual about the fact that the requested data was pretty meaningless. I regularly get asked to report meaningless metrics and statistics.

If I explain the set-up I am sure that you will begin to see where I am coming from. Somewhere in Company A (perhaps a successful retail chain) someone wants some data – it might be senior management reviewing strategy, the marketing department considering a new advertising campaign or it could be a management accountant preparing some statistics for an external requirement. Anyway, the request is passed eventually to the buying department to be passed on to all relevant suppliers – as a “must do” and with a deadline. So the request lands in the sales department of Company B – a manufacturer. The sales department consider the request carefully in case it contains a trap and then as all, or part, of the request is for data regarding items supplied in turn by other companies, a version of the request is passed to the buying department of Company B who send it out to the sales departments of all their relevant suppliers – again as a “must do” and with a tighter deadline. This may be repeated more than once. Eventually the data request ends up at the lowest point in the “food chain” a point from where the request can no longer be passed on.

If the data request posed eventually to my Customer’s sales department has much relevance to the original data requirement I would be surprised. Too many people have received it, misunderstood it and (gratefully) passed it on to someone further down the pecking order. The nature of the pecking order itself is relevant – it is the current practice of purchasing departments to act in a dictatorial manner when dealing with suppliers in almost all markets. “Just do it or lose the business.” is the norm – and let’s face it the buying departments in question don’t have a direct stake in the process anyway.

One of my favourite numbers in a supply chain request is “an average”. A recent request wanted an average unit weight for various categories of supply. Allowing for the fact that the request itself was probably an interpretation (or misinterpretation) of the original data requirement, we are still faced with a number of options. Did “average weight” mean the average of the unit weight of all of the different models in a given category or did it mean a (weighted?) average of the total quantity supplied? Given that numeracy is no longer one of the keys to business life it is entirely possible that the originator of the question does not know the answer in any case. If the alternatives were explained in straightforward terms the originator would probably plump for one meaning or another but that is never going to happen. Without knowing what the end user of the data wants to do with the data it is just about impossible to prepare any usable metrics. Worse – as the data makes it’s return journey up the food chain then it will be combined and aggregated on it’s way. What is going to become of an average? Will it get averaged with other averages? Probably!

Such data requests are a good test of a programmer’s integrity – they are also the darker side of the consultancy business. Motivation is low, as there is no direct benefit to your client in exchange for the money that you charge but you still have to deliver a result that is as honest and accurate as you can make it – no matter how sure you are that the numbers being prepared are unlikely to be what is wanted or needed. Decision will be made and money spent based upon data that is almost certainly irrelevant, wrong and misleading.

One solution would be for all requests for data to be accompanied by a short brief on the intended purpose of that data. However I suspect that in many circumstances such a brief would never survive to make its way down the supply chain. It is perfectly possible that the originator did not know that the original data request was in fact going to make such a journey before it was resolved. An alternative might be to reverse the process and provide a brief on how the data was gathered but, again, I suspect that such a brief would not make the return trip as, at every stage, there would be worries that “getting it wrong” might trigger an unfavourable review and a subsequent loss of business. No, the real problem is a social one and like many social problems very difficult to find solutions to. In the mean time I will keep writing code for “edge cases” and worry about numeric precision – grinding the numbers through the best code mill I can construct in the circumstances.

2007-07-09T05:40:00.000-07:00

My weekend reading on ASP.NET Ajax

Over the weekend I read “Professional ASP.NET 2.0 Ajax” by Matt Gibbs and Dan Wahlin – published by Wiley/Wrox. As the official documentation for ASP Ajax was found wanting as an introductory text this looked like just the book for someone looking to “catch up” with Atlas – and so it proved to be.

The authors of this book expect their readers to have a good working knowledge of ASP.NET 2.0 so I was a bit puzzled at first by the inclusion of a tutorial section introducing JavaScript. It did not quite seem to jell with the prerequisite or the word “Professional” in the title. I then realised that this book was targeted at ASP.NET developers working within, largely corporate, environments who, up until now, had been using server side functionality to process forms and pages. My reading of the book rather confirmed that ASP Ajax is also targeted at that same group of developers. This is not “faint praise” on my part – I would be happy to apply this framework extension if it were specified by one of my Customers for an Intranet style project.

There is the rub though. My feeling is that, in an attempt to provide a form of “instant Ajax” that can be applied quickly to pre-existing web based applications, Microsoft have supplied rather too much that is cumbersome and a lot that, for me, misses the point. I like the .NET Framework because it supplies a first class API that allows me to get on with designing and writing software. I tend to avoid the gimmicky bits. As an example I have never bound a DataGrid to directly to an SqlServer table – and I’m never likely to do either. Similarly, I can be pretty sure that I will never feel the need to implement a partial page update – the simplest Ajax style implementation offered by this framework extension. When applying Ajax technology, I want to write my own client side JavaScript code and I want a simple and straightforward way to call server side methods. I do not want a vast attendant baggage of classes and protocols.

If I were undertaking the role of an IT director I might very well appreciate the “standards” that are set by the Microsoft ASP Ajax approach – web based applications should be readily understandable, tweakable and maintainable by anyone who has worked on a similar project. That can count a lot but I find my personal enthusiasm for building applications via config file settings somewhat limited.

If you are building an Internet facing app then you would almost certainly want to design a mini-framework best suited to the particular need rather than take on the overhead of what remains, for Microsoft, a server centric approach to Ajax.

Back on the book. This is a highly readable book covering the ASP Ajax ground thoroughly. It covers all aspects of the ASP Ajax framework – even delving into the depths of deployment over multiple servers. It includes references to freely available tools that can be downloaded to assist in debugging and diagnostics. It encourages the reader to extend his or her knowledge into building client side custom controls supported by JavaScript functionality and Ajax networking to server side resources. This is definitely a “How To” book – only slight criticism being that it is rather light on the “Whats and Whys” – but that would have been a much bigger book and of only marginal interest to the majority of it’s readers.

2007-07-09T01:18:00.000-07:00

Perpetual Motion

I did enjoy following the purported exhibition of a working model demonstrating the Steorn Orbo This is the latest in a long line of perpetual motion machines that never quite work when anyone is looking. The back-story is that an Irish company, Steorn, announced to the world in August 2006 that they had appeared to have invented a device based upon “time variant magneto-mechanical interactions” that could generate work without the input of any energy. Even during such a slow period for news the announcement was largely greeted with snorts of amusement. As a follow up – Steorn announced that they would be demonstrating their wonder device in London and to the public from 4th July with “web cams” streaming the demonstration to the world who could not make the trip.

I’m a hopeful cynic I suppose. I would love the world’s energy problems to be solved by a simple piece of “magic” but in reality I sat and wrote a simple “script” or movie plot of how events would unfold on the assumption that it was all a con trick.

<Script>
Conman announces public demonstration of device.
Conman inexplicably decides to use brand new demo device never previously tested rather than one of the many impressive devices he (says he) has working in his Lab.
The brand new device does not work – initial technical fault – will be fixed tomorrow.
<Plot A>
Overnight someone breaks into exhibition and smashes machine – “Big Oil” blamed by Conman
</Plot A>

<Plot B>
Special local environmental problems preclude test without re-design – note unexplained impossibility of just using one of the Conman’s existing machines.
</Plot B>
Conman announces end of demonstration until further notice/development
</Script>

Well in this instance the “demonstration” followed movie plot variant B with camera light heat being blamed as the local environmental problem – this being the technology that was going to drive cars. Now apart from the obvious questions like “Are there really people so gullible that they would invest in this?” I can only note that modern video technology works very well indeed in even very low levels of lighting and ask why they did not just switch the lights off?

Still perhaps they will be back again next summer to amaze and entertain us. I rather hope so but in the mean time keep your “venture capital” in your piggy bank.

2007-07-06T04:22:00.000-07:00

A mild disappointment

OK – I indicated that I would no longer post on the subject of the GoTo statement but my much anticipated copy of “Structured Vax Basic A GOTO-less Approach” turned up this morning and I eagerly skimmed my way through looking for the magic formula that would replace that single last GoTo in my VMS Basic programs.

I was at first disappointed by the programming style – this book was published in 1988 and the use of prolific line numbers and non declared variables was well out of date by then – VMS Basic had come on a good ways from it’s PDP Basic+ antecedents but not much of that showed up in this book. Still – it’s nearly twenty years too late for a book review.

So – how did Messrs Price and Spitzer exclude that final GoTo in their book? – THEY CHEATED! They shoved a STOP statement into the code to replace the GOTO that would have taken them to the END statement. That's not code you can ship to customers. I’ve a good mind to claim my $2.79 back from the second hand bookseller as the book did not deliver honestly on the promise of the title.

Still never mind it’s Friday, my youngest children finish school for the summer today and the weekend weather looks likely to be warm.

2007-07-03T05:49:00.000-07:00

A Custom HTTPHandler

If you read my last post on building a base class that can be created automatically from the content of an SQL select statement and that can also serialise itself as a JSON string then you may be wondering how I will be handling my Ajax calls – given that I said I will not be using ASPAjax. The answer, of course, is through the use of a custom HTTPHandler.

My very first Ajax enabled ASP.NET applications used the excellent AjaxPro DLL from Michael Schwarz which implements a custom HTTPHandler to catch Ajax calls and to pass them to previously identified methods in a given ASP.NET page using reflection. In fact the earliest iterations of Michael Schwarz’s project were more than adequate to meet the needs of an ASP.NET developer looking to add Ajax capabilities to an application. However the code continued to develop – with a lot of additional functionality being added resulting in a framework rather similar in many ways to the ASPAjax offering from Microsoft. You can get the source code and find out how it is all done from it’s CodePlex repository .

There are lots of reasons why you might want to write a custom HTTPHandler – just another one popped up recently at the CodeProject where PrashantRishu has posted a method to support custom, user friendly Urls without having to write an ISAPI extension.

If you get enthused to write one for yourself then do take a look at both Phil Haack’s and Scott Hanselman’s boilerplate HTTPHandlers. These form an excellent foundation for any custom classes you might want to implement yourself.

You effectively implement an HTTP Handler by adding a couple of lines to the WebConfig file within your ASP.NET project. The snippet below implements the AjaxPro handler for requests looking for documents named *.ashx located in a (mythical) subdirectory /ajaxpro/


<httphandlers>
  <!-- Register the ajax handler -->
  <add verb="POST,GET" path="ajaxpro/*.ashx"
       type="AjaxPro.AjaxHandlerFactory, AjaxPro" />
 </httphandlers>

Custom HTTP handlers are certainly a cleaner approach to providing custom responses to HTTP requests (Ajax or otherwise) than stripping all of the HTML code from a standard ASP.NET page.

2007-07-02T06:11:00.000-07:00

A date with JSON

I was happily programming some base functionality for a set of classes to be used in a slightly complex, Ajax enabled, web based application when I came across a yawning gap in the JSON specification.

Let me start at the beginning.

I wanted to create a set of .NET classes that could set their own properties from data retrieved from a database without my writing specific code for each class. This proved a pretty straightforward use of reflection to match properties to field names. I was happy to ensure that field names and property names were consistent and was already thinking that table or view names should align with some class names – after all, those “Rails” chappies build whole systems on this simple foundation idea. Here is my simple class with 5 properties – four intended to match to field names in a database table with a fifth to represent any properties whose values are derived during the use of a given class.


Public Class Machines
    Private mMachineName As String = ""
    Private mMachineID As Integer = 0
    Private mCostPerHour As Decimal = 0
    Private mSpeed As Integer = 0
    Private mTestNotInDBase As String = ""

    Public Sub New()
        ‘a default constructor
    End Sub
    Public Property MachineName() As String
        Get
            Return mMachineName
        End Get
        Set(ByVal value As String)
            mMachineName = value
        End Set
    End Property
    Public Property MachineID() As Integer
        Get
            Return mMachineID
        End Get
        Set(ByVal value As Integer)
            mMachineID = value
        End Set
    End Property
    Public Property CostPerHour() As Decimal
        Get
            Return mCostPerHour
        End Get
        Set(ByVal value As Decimal)
            mCostPerHour = value
        End Set
    End Property
    Public Property Speed() As Integer
        Get
            Return mSpeed
        End Get
        Set(ByVal value As Integer)
            mSpeed = value
        End Set
    End Property
    Public Property testNotInDbase() As String
        Get
            Return mTestNotInDBase
        End Get
        Set(ByVal value As String)
            mTestNotInDBase = value
        End Set
    End Property
End Class

Here is the code for the constructor that takes a data table row as input and sets any properties with a name matching a given field name to the appropriate value.


    Public Sub New(ByVal dataRow As System.Data.DataRow)
        Dim myProperty As System.Reflection.PropertyInfo
        For Each myProperty In Me.GetType.GetProperties
            If myProperty.CanWrite Then
                Try
                    If Not dataRow.IsNull(myProperty.Name) Then
                        myProperty.SetValue(Me, dataRow(myProperty.Name), Nothing)
                    End If
                Catch ex As Exception
                End Try
            End If
        Next
    End Sub

It is a shame that the DataRow class does not expose a method or property that can be used to check for a given column name thus forcing the use of a Try/Catch block for a predictable exception but as you can see the code is very simple.

In the instance of this demonstration code, the classes in question are being create by another class that appends each instance to a collection using the results from a database select query - thus:


Public Class MachineSet
    Private myMachines As New Collection
    Public Sub New(ByVal SQLDb As SqlConnection)
        Dim SQL As String = "Select * From MACHINES"
        Dim TrialSQLCommand As New SqlCommand(Sql, SQLDb)
        Dim TrialDataAdapter As New SqlDataAdapter(TrialSQLCommand)
        Dim TrialDataTable As New DataTable
        TrialDataAdapter.Fill(TrialDataTable)
        Dim TrialDataRow As DataRow
        For Each TrialDataRow In TrialDataTable.Rows
            myMachines.Add(New Machines(TrialDataRow))
        Next
    End Sub
End Class

The constructor code above is passed an open SQLConnection to an SQL Server (or Express) database.

I now wanted to add a property to my Machines class that emitted a properly formatted JSON string so that the class properties could be re-created in an object at the client end in JavaScript. So I started writing something like:


    Public ReadOnly Property toJson() As String
        Get
            Dim RetVal As New StringBuilder("{ ")
            Dim FirstItem As Boolean = True
            Dim myProperty As System.Reflection.PropertyInfo
            For Each myProperty In Me.GetType.GetProperties
                If myProperty.CanRead Then
                    If Not FirstItem Then
                        RetVal.Append(", ")
                    Else
                        FirstItem = False
                    End If
                    RetVal.Append(Chr(34) & myProperty.Name & Chr(34) & ":")
                    Select Case Type.GetTypeCode(myProperty.GetType)
                        Case System.TypeCode.Boolean
                            RetVal.Append(IIf(myProperty.GetValue(Me, Nothing), "true", "false"))
                        Case TypeCode.DateTime
                        Case TypeCode.Decimal, TypeCode.Double, TypeCode.Int16, TypeCode.Int32, TypeCode.Int64, TypeCode.Single
                        Case TypeCode.String
                    End Select
                End If
            Next
            RetVal.Append("}")
            Return RetVal.ToString
        End Get
    End Property

When I got to the DateTime case I came to a halt – JSON does not have a date or date/time “type”. Odd, I thought. JavaScript has lots of ways of creating a Date and all of the server side technologies I have worked with know what the date is.

A stroll around the web confirmed the gap. You can’t transfer Date and Time data in a form that can be automatically loaded into a JavaScript object value using JSON. Those who had walked this way before me and published their answers seem to have generally gone for a string formed from the number of seconds since the beginning of 1st of January 1970 but delimited in some specific way. A typical example was the Microsoft ASP Ajax (nee Atlas) approach which is to wrap the number of seconds into a string like the following:

“\/Date(nnnnnn)\/” where nnnnnn is the magic number.

This approach requires some client side processing to complete the transformation of the string into a viable JavaScript Date object.

I then got to thinking about things like time zones and operating an application across multiple time zones. On balance, I think I am going to get my classes to emit a JSON string that can be converted to a date time using Greenwich Mean Time (UTC) as a standard and then supply client side support to convert displayed times to and from local time for a given user. This has the advantage that I can even ignore “summer time” issues if all of the recorded times in the database are based upon GMT/UTC.

If I am using ASP.NET then why am I not just using ASPAjax? I might well end up using that technology but at the moment it looks rather too “weighty”. It is also poorly documented so it is not easy to research (as an example) things like just how dates are transferred back and forth between client and server and I would prefer to explore it when I know just what I want to happen and how that can be achieved without using this extension to the framework.

2007-06-26T01:07:00.000-07:00

Back Buttons and Compilers

The last word on the GoTo statement goes to Edward G Nilges.

I look forward to understanding the full impact of the Steves'** translation of Rails from Ruby to JavaScript. I have banged on before about the potential joy of using JavaScript on both the server and client end of a .NET application and this sounds like positive steps although I had expected Rails to turn up in the .NET world via IronRuby - perhaps JavaScript will be a valid alternative.

**Steve Yegge announced his Google project to translate the Rails framework to JavaScript running on the Rhino engine (read Java VM) at Foo Camp. Steve Yen runs the TrimJunction project at code.google.com that (you guessed it) is a port of Rails to JavaScript. You can run (indeed use off-line) a working demo of Steve Yen's version.

Plus also - thank you Steve Yen for reminding me that I need to resolve the "Back Button" issue in my own Ajax enabled apps. Oh and thank you Steve Yegge for reminding us all of the importance of gaining a true understanding of compilers. Rich programmer food indeed that (sort of) closes the circle back to the "last word".

2007-06-22T06:04:00.000-07:00

Computational Thinking

The other day in the comment section of one of my own posts I was reaching for a suitable word or phrase to represent my concept of “computer literacy” and to avoid the association between that particular phrase and introductory classes on basic spreadsheets and word processors. I wanted a phrase that covered a whole lot more and implied a true facility with computational devices.

John Udell has recorded a podcast [aaargh ] of a conversation with Jeanette Wing, who is the head of the computer science department at Carnegie Mellon, on this very concept. She has coined the phrase “Computational Thinking” that covers what I had in mind - plus some. You can read her article on this (PDF warning) or (if you must) catch the mp3 at IT Conversations.

Ms Wing has a wide ranging definition for Computational Thinking but one aspect I particularly liked – “A way that humans, not computers think. Computational thinking is a way humans solve problems; it is not trying to get humans to think like computers. Computers are dull and boring; humans are clever and imaginative. We humans make computers exciting…”

The differences between the PDF and MP3 referred to above are near perfect examples of what I was trying to get over in my rant on the subject. The three pages of the PDF convey far more than the podcast and the text is truly thought provoking. Despite the obvious enthusiasm of both Jeanette Wing and John Udell for the concept, the podcast falls rather flat and tended to get caught up on the single aspect of undergraduate courses when there is rather more to "computational thinking" than that.

By contrast, I listened to Chris Spurgeon on "The best Geo Hacks of the last 2000 years" which is entertaining and interesting despite that fact that listeners have to imagine his "slides". It's a great listen but if you wanted to follow up on any of the "hacks" you would have to look for alternate sources.

Perhaps what I should have been saying was that podcasts can make you aware of some new technology or technical issue BUT they are a very poor media for communicating the detail.

</Adendum>

2007-06-20T02:53:00.000-07:00

It makes all the difference

I had just come to the conclusion that I needed a really good “diff” utility to check the set-up of a test version of a web component that normally runs within a portal when I came across a post at Eric Sink’s blog about a free (as in beer) Diff/Merge download from his SourceGear tools business. In a matter of moments I had downloaded the installation package, run the install and opened up the two relevant C# files (see I do write in C# as well as VB). I was then able to make a simple visual check of the two versions to make sure they were the same where they should be the same and different when I had to take account of the differences in their runtime environments. Excellent tool and available for Linux, Mac and Windows.

2007-06-20T02:44:00.000-07:00

A post on it’s own

Edward G. Nilges has posted a comment on my post titled “If not Basic, where should we GoTo” that is a post in it’s own right. Well worth reading.

There is one reference in his comment that may not be familiar to all – the one to Bohm-Jacopini control structures. In a paper in 1966 Bohm and Jacopini established a theorum that can be simply paraphrased as:

“Any computer program can be re-written as a ‘structured’ program using (recursively) only three control constructs. ‘Sequence’, ‘Iteration’ and ‘Alternative’.”

Nicely illustrated by a diagram I pinched from a Yourdon page on structured Analysis.

The following “structured programming” revolution, led as I recall by Ed Yourdon, demonised the usage of the GoTo statement and started a push towards better structure and a major step forward in program maintainability. I can remember one fall-out of the movement. My boss started to get keen on a new form of flow chart – data flow rather than the more mechanical process flow that preceded it. I think that my original IBM flow chart template saw it’s last use during this time when I was enjoined to drawing circles and curved arrows. I never did see the point in it (sorry Sid if you ever read this) but was happy to humour a man who taught me a great deal. I did not see business processes as data flows at all but simply as a series of data state changes towards a final end. Implementing highly efficient state changes fronted by human sensible interfaces was what interested me. Data flow seemed connected to the days of punch cards and tape drives and did not cognitively align with the relational databases that were the new “toys” of that era. Data had stopped moving and had no flow except perhaps towards aggregation and management reporting.

Just for fun, I tried to locate a reliable link to a copy of D. E. Knuth’s "Structured Programming with the GOTO Statement" but could only find a PDF that would not download. In the paper Knuth defended the elegance and efficiency of the GoTo and within his context, he was right. However few of us share his capability – yes you can use the GoTo to create well structured programs but it is all to easy to make a mess instead. The GoTo should remain in ghostly form only - such as the “break” statement in C based languages.

2007-06-19T05:06:00.000-07:00

Podcasts are ~~mostly~~ terrible

<Rant>

As someone who lives outside the current range of effective ADSL I am pretty picky about what media files I download. I have bemoaned the growing trend towards posting video as a means of communicating things as diverse as information on new technologies to a record of someone’s birthday party. OK, (me as case in point) we can’t all be great writers but text is a great communicator. If video is beyond my daily download budget then sound is just possible so I am regularly tempted to download a podcast or two from sites such as IT Conversations and even Google. I see something billed as being about a subject or technology of interest with good reviews so I go for it – almost always to be disappointed.

Podcasts can be great (sometimes). They are great when the material being presented is/was intended to be delivered largely by voice. Someone describing past events (say) can make a good tale of them and tales are for telling and being listened to. They can work as print but work exceptionally well as an oral presentation. Listen to Steve Wozniak doing his “stand-up” routine at Gnomedex 4 – it makes compelling and entertaining listening as does Paul Graham on Great Hackers BUT (that’s a big but) I prefer to read Paul Graham’s essays on the web. Paul Graham works hard at his essays and a good essay communicates more that just the sum of the words – I takes you on a journey and fosters thought and mental analysis as it develops it’s theme.

I had to sit on my hands for a moment here until I had decided not to site a specific podcast as an example of what is terribly wrong with most. Still you would not have to range too widely through the catalogue to come up with some dreadful examples. Picking one would be unfair on the participants and would only invite comment on the merits of that individual choice.

One common podcast theme that almost always fails miserably is three guys talking about some topic – in the worst cases over a telephone link. This is, superficially at least, a parallel to the TV “chat show”. You do not expect even the most entertaining chat show session to elicit much information but the audience can enjoy the interaction between a well briefed show host and the “personality” flogging whatever wares they currently have for sale. The podcast not only excludes the audience from most of that interaction but telephone links also mean that the participants are missing it too. Add in a generally feeble level of host preparation (or prior knowledge of the supposed podcast subject) and you are left with a subset of the words – and pretty sparse on the information front they tend to be. If people find it tough to get over their points in a text – most find it even tougher in conversation. Sure, a conversation is easier (less nerve racking) than a presentation but it is still a terribly limited form of communication for all but the most expert.

The last paragraph might give you the impression that I would advocate video as an improvement to sound only media but I fear that would not be so. Creating professional level video is even more expensive and demanding than sound and (in the mainstream) relies upon well trained professional presenters. I am aware that sites like YouTube have given a “voice” to many new talents – some of them may mature and endure but most are only really effective in short sessions where we can make allowances for production deficiencies.

Why am I worried? – I fear that podcast/video conversations may take over as the dominant form of technology communication on the web. Video is not searchable or effectively indexable and that is a major problem in it’s own right. The bigger problem is the loss of content a move to these new media would result in. I was looking at an excellent blog entry from Anthony Moore on the BCL team blog at MSDN. His subject is a new DateTime type in the pipeline for .NET 3.5. It is detailed, well argued and informative – cracking documentation. Let us hope it will not get lost in the myriad of MSDN pages. But here is the point. Provided the content is not deleted it will always be findable by search engines. The other side of the point is that the best job a podcast could have done with this subject matter would have been to make you aware of the deficiencies in the current .NET DateTime type and aware that a new type was being added to the framework. Such a podcast would soon be lost and un-findable and would not (indeed could not) include the content to be found in the text.

</Rant>

An MP3 download of this rant will not be available

2007-06-18T05:30:00.000-07:00

Why don’t you just try it?

I will come to that headline in a minute but in the spirit of encouraging trials I will post a couple of links:

A great one page JSON reference card – all you need t know.
and a nice post on how to convert XML to JSON using C#.

If you are as interested as me in the development of VB.NET then you should take a look at the fourth article posted on the MSDN blog site on Closures (and read the first three if you missed them).This post focuses on a subtle change in the scope and lifetime of a variable declared within a code block – in this instance a For Loop.

It was in February 2006 that Paul Vick asked a simple question – “What should this code do?”


Module Module1
   Sub Main()
       For i As Integer = 0 to 2
           Dim x As Integer
           Console.WriteLine(x)
           x += 1
       Next
       Console.ReadLine()
   End Sub
End Module

He did not ask what it did do – but what it should do and a good number of people (including me) posted their opinions.

A month later Paul explained why he had asked. The scope and lifetime of the variable x was going to be affected by the introduction of closures and it was important that any change reflected what people though the code SHOULD do as well as making sure that future changes did not break too much existing code.

Anyway read what Jared Parsons has to say on the actual implementation in the Orcas code base – it is remarkably subtle. Very powerful and very VB.

Paul’s most recent post raises the issue of the VB runtime DLL and the difficulty in getting it ported to all of the platforms the VB language team want to support with VB.NET now and into the future. Orcas will have an option that allows developers to exclude support from the VB runtime in an application. This would preclude using some of the more dated language features (such as Left()) but would allow VB developers to signal to the compiler that it should not break just because the VB runtime was not available as a “helper”. This helps future proof the language – making the development of portable modules ready to run on a wider selection of platforms possible.

Just Try It!

I took a very brief look at Google Maps for Mobiles last Friday and while playing around was very interested to see the app starting up my phone’s GPS when I clicked on the option to track a route I had selected. At the time I was looking at a route in the UK while sitting in my office in France so I knew that this was not going to work. However I was intrigued and wanted to know if a GPS enabled device could track a downloaded route without a continuous connection. So I took a look at the support group at Google and read a few “reviews” posted on the web – and I still did not know the answer.

The Google support form has a lot of questions that run like “Can anyone tell me if Google Maps will run on my Strawberry 8XTY?”. Which is pretty peculiar as it would take about five minutes to download, install and try the application. I was reminded that some of our support issues are quite similar – you end up asking yourself why the user does not just try it. I can remember a couple of years back we received an email from someone who had read a short article we had posted on our web site on SQL for Microsoft Access. He had been having trouble writing queries that included dates in the selection criteria and we had documented how this was done – not “rocket science”. What was interesting about the email was partly that it was pretty rude with the writer demanding to know the provenance of the information we had posted but also the fact that the writer could have checked that we were correct in less time that it had taken to write his email – and he was the one with the problem!

So I decided that I had better take my own medicine and just try it – fortunately Google Maps for mobiles supports France as well as other parts of the world – although low level satellite data is not available in most instances outside of the USA.

I wanted to answer two questions. Could Google Maps for Mobile track my progress without a continuous service connection? Plus could Google Maps for Mobile track me when I was walking as well as when driving?

<Backgrounder>
A brief bit of background on the second question. When small portable GPS devices were first introduced they were quickly taken up by walkers/hikers/mountaineers who used them off the beaten track. Much more recently a mass market has sprung up around driving directions and this has resulted in a subtle change in the way that GPS devices work. The driving directions applications wanted to reduce “noise” – particularly subtle changes in apparent position resulting from variations in signal quality and/or changes in the constellation of “visible” satellites. This means that GPS devices that are “expected” to be used in cars ignore slow changes in position. They need any application working with them to change their mode if the user wants to track progress at a walking pace.
</Backgrounder>

How did things go? To start off with, selecting my route start point was a bit “clunky” as there is a more famous place with the same name as my village near Paris – this made the route to the local town a bit long at over 400km. However once the GPS was activated I could ask for a route from where I was “now”. Once the map was downloaded then my phone was quite happy to track my route by car towards my destination. It even coped well with my deviating from the specified route even though there was no available network connection – although I remained well within the “scope” of the currently displayed map. So far, very good.

I then returned to my start point and “zoomed” the map in until the lane leading to my house was clearly visible – a good level of map detail. I then set off on foot – and was tracked nicely by the GPS. This may be device specific but it is a very positive result.

Conclusions – without sound and thus voice directions this is currently very much a tool to be used by a car passenger or by someone on foot. However the GPS integration was excellent as was the level of map detail. A network connection is needed if the map “zoom” needs adjusting (a likely requirement through a long journey) but otherwise the application works just fine “off-line”. Route finding is possible without a GPS although it is then up to the user to indicate when they have arrived at a given “way point” to get directions to the next.

Even if you already have a dedicated GPS device in your car or alternate software I would still classify this download as a must have – just in case you need it. After all, this will work when I am in New York even though my normal package only covers Europe. Available for Windows Mobile, Palm OS and the Blackberry.

More or less on topic - nice post by cricket nut Reto Meier on building your own Google Phone – well using your web browser plus some Google mobile apps – adds up to (Reto implies) more than the Apple iPhone is offering. Get it now while it is hot.

2007-06-08T09:59:00.000-07:00

In the right gear

OK I sat on my hands for a couple of days waiting to get a firm grip on Google’s latest offering – Google Gears. It was tough, but I wanted to be clear just what was there and how it worked before passing any comment.

I downloaded and installed the client side software, reviewed the documentation and ran the demos – just as a lot of people seem to have done. However I notice that the “backlash” has started in the blogsphere with critical reports now outnumbering the initial enthusiastic reviews. Which I suppose is about normal for anything new – particularly when it is an enabling technology and not a magic potion – yup people have begun to notice that an effective implementation of “off line” functionality for most on line systems is going to be hard work! While wondering why that came as a surprise I decided it was time to put in my two pence worth on the subject.

In fact my few words were further delayed – I had made a quick consultancy trip over to spend a day with a customer in Southern England when my home in France was treated to a really violent thunderstorm. On my return I found that we had lost the router that links us to the Internet, the WiFi unit and a couple of Ethernet switches – plus one of the PCs looks like it has a blown motherboard. Collectively that made any return to normal communications and work after my trip rather difficult. The switches and WiFi unit were quickly and cheaply replaced at the local supermarket but locating an ISDN router posed greater problems – just about unobtainable here in France. Fortunately the home office in North Wales had a retired one on a storage shelf and that is on it’s way to me. In the meantime I located an old BT (British Telecom) ISDN card in a box (together with an NT driver) and got that up and running on a Windows 2000 PC. I then downloaded the excellent first posted on an “offline” function to add credibility to an Internet based business application.

Why have Google launched this beta of Google Gears?

The beta itself is I think a clever move on Google’s part. They will quickly identify and solve all of the likely installation difficulties for this technology as well as pick a large number of bugs out of the code. This will place them in a very strong position for the launch of a full Business Service around their email and office offerings. So this is a big deal for Google and will accelerate the changes we are already seeing in the way applications are delivered to the end user and businesses alike.

Google gears itself is a masterly concoction – the very minimum possible to make it all work – the perfect basis for a de-facto standard and totally in keeping with the web development philosophy. SQLight is the client side database of choice but including this element as a data store is still a master stroke in it’s own right. We also have a JavaScript element (still the leading contender for the Next Big Language), JSON and a tiny proxy server (that requires installation of course).

So what does this bring to the table?

The proxy server brings an immediate capability to access web pages off line or without reference to the original server while providing automated functions to check for updates and to download them into the local cache. This is new and usable functionality straight away and requires minimal effort to get it up and running.

The database module provides a secure data service ensuring that application data is not available to other applications outside of the original domain. Data can be stored or extracted from the database. Extracted data is stored in a result set that will be immediately familiar to the vast majority of programmers who have worked with tools such as ADO (DAO etc). The database supports full text search and a couple of extensions that allow the specification of things such as character encoding, pages sizes and the like.

In may ways the most exciting element of Google Gears is the “worker pool” module. This allows the execution of JavaScript code in the background without blocking the execution of the main page script. The worker pool can manage multiple processes and individual processes can communicate between themselves. You can even run multiple worker pools allowing different page elements to be supported by their own pools without conflict.

But do we need it?

The Google Gears developers guide includes a good page that explores alternate architectures – this is a good starting point for deciding if your application would benefit from “off-line” functionality. Not every case is covered of course but I think this is here you will get your ideas from. Me? I’ve started coding.

Just a quick PS on my smartphone.

I did say that I would report back on just how well my Fujitsu Siemens LOOX performed as a phone – after all I have written a lot on it’s capabilities as pocket PC type device. It works just fine with none of the faults I have heard others have had with the Microsoft phone OS – signals were always clear, the volume was fine (and I am just a little deaf) and it was great working in a WiFi enabled office area to be able to keep bang up to date with my phone messages and emails all in one pocket.

So as a final comment on this device (don’t want to be too boring) – would I recommend it to others? Yes – IF they did exactly the same device with a minimalist phone keyboard and with a full VGA screen. I know - it was the keyboard that sold it to me in the first place but it is crap and so is having a small screen.

2007-05-25T04:15:00.000-07:00

Who says VB is not for grown-ups?

I am returning to a theme I have touched on before – my fascination for the intrinsic design and mechanics of languages and the sheer quality and care being put into Visual Basic by the language development team at Microsoft. It does not matter how much the idiots at the top of Microsoft annoy us, you have to acknowledge that their tool builders are top rank.

I was looking at Paul Vick’s blog and reviewing progress on some of the VB.NET language elements. Orcas is going to include a replacement for IIF – and guess what – it is If - but implemented as a true ternary operator. In true Visual Basic style it allows the two possible return values to have different types although it will only return the “wider” of the two. Thus, if your If statement was something like:

myVal = If(<some condition>, <integer value>, <long value>)

then the returned type will be a long in both instances. If the return types are completely incompatible then an error would be generated. Look out for If statements with function brackets in the Orcas Beta2.

We will also see hash tables that support multiple keys – just what I was looking for the other day to solve a little problem – but I had to take another longer route to a solution instead. This allows the quick storage and retrieval of data accessed by multiple keys simple and straightforward – hash tables were long overdue in Classic VB and it is great to see them continuing to develop now they are available to us in VB.NET

Paul also discusses the issue of anonymous types ** and how the C#9 team have decided to make them immutable. However, with an eye on the upcoming Dynamic Language Runtime (DLR) the VB team look likely to take the opposite line and that brings me on to the often argued differences between C# and VB.NET. I have previously taken the line that the differences between the two languages are immaterial and that if you want to program to the .NET framework then you should just choose the syntax style that you prefer (taking into account that C# programmers seem to get paid more). That C# developers attract higher salaries is odd at a functional level (and probably just reflects the language choices of higher paying organisations) as C# developers are always going to be less productive than a VB developer of similar experience and skill. C# developers will immediately start squealing about the supposed verbosity of the Visual Basic language and even site the supposedly poor aesthetics of the code on a screen but when you factor in the broader language and class library coverage with magnificent support from the IDE then it is clear that VB is a winner in terms of productivity.

There is another angle though and this is one where the generally perceived wisdom might just be “upside down”. Jeff Atwood at Coding Horror was on the case the other day in a post titled “C# and the compilation tax”. He was trying to balance the two languages – on the one hand C# seems to have become the “language of choice” (the de facto standard) for .NET and yet you have to accept the massive drain on productivity that is the very limited IDE support for C# compared to VB. On many .NET support sites (particularly those hosted by Microsoft) you will often find demonstration code written in VB but with a slightly snide or snobish comment that this is because C# programmers can probably decipher the VB syntax and yet somehow VB programmers can’t manage with C# examples. I think that Jeff’s view is that advanced code available on the net is going to all be in C# so you had better “suck up” the productivity overhead and not just learn C# but convert to it. I beg to differ.

If I were to sit back in my old IT manager’s chair (phew it was a good while back when I still wore a suite) and review the C#/VB choice then I might conclude from the evidence that Visual Basic was (superficially at least) simpler code to understand – and that would give me a code maintenance edge. Plus it would appear that VB is faster to write – almost slam dunk! Now about the issue of “the community” and it’s apparent fixation on C#. Is it true that C# is hard to understand if you are not a C# programmer – is there too much “cognitive friction” (as Jeff Atwood would have it) in writing one language while trying to read and understand code written in the other?

Certainly the syntax could appear stark – well unless you are used to writing JavaScript of course. So there is no syntactical dissonance for the large army of developers who are targeting the web as a software product platform. For the rest, well I am of the view that a couple of hours with a simple C# primer would cover most of the ground – quickly making sense of things like:

for(aLoop=0;aLoop<24;aloop++) style="color: rgb(51, 204, 0);"> \\some stuff
}

and the way that variables are declared:

bool upDown = false;

I can see that the closing brace is a poor substitute for a proper block closing statement (like “End Select”) but with proper indentation it is just not that hard.

C# interacts with most of the classes found in the .NET Framework in just the same way as VB.NET – it is just that Visual Basic is “richer” in terms of language elements and pre-defined support classes – so that makes C# smaller and simpler to understand doesn’t it?

There are a few potential “gotchas” around – say in the differences between the way a VB “Select Case” statement executes and the way that a C# switch statement runs but none of this is difficult to understand. Plus if you are just reading C# the issue of the benighted case sensitivity hardly arises – all in all I think that C# code is very accessible to the VB developer. It just takes a (very) little effort.

Visual Basic to author new Languages

Paul Vick also beat me to a comment on a throwaway line from Miguel de Icaza when he was describing work on the Mono version of the DLR that the new Microsoft JavaScript compiler for the DLR was written in Visual Basic. We already know that the Mono version of the Visual basic compiler was written in Visual Basic but it is good to see that the language is being correctly treated as a first class language at Microsoft as well. Paul also added that version ten of VB (currently dubbed VBx) will be written in VB as well.

** Anonymous Types
Anonymous types – what are they and why would I care?

Anonymous types are an essential by-product of the LINQ feature of the .NET languages and will prove very useful indeed as they enable developers to define types within code, without having to explicitly define a formal class declaration of the type. Developers will be able to “crack on” with the code and leave it up to the compiler to fill in the gaps.

Why should you care? Well I think that we all need to put some effort into understanding the structure and function of our languages as we face up to a multi-processor future. I was reminded of this during the week when we were working up a custom thread pool to service incoming asynchronous data on the Windows Mobile platform. The challenge was to manage the inflow of data in a manner that maximised the number of data items processed while ensuring that the program UI continued to perform at a level that would meet user expectations – plus it had to run on a wide variety of hardware devices with varying resources and processor speeds. If you do not understand your language, runtime and (to an extent) the hardware then you just can’t solve problems close to a system’s capacity. In a multi-processor future, a lot of individual process threads are likely to be approaching and reaching the capacity of a given device resource.

2007-05-14T08:32:00.000-07:00

The big bad wolf

Is Microsoft just huffing and puffing at the door of the Free Software movement and if so why? I just do not understand where they are coming from on the implicit threats they have made about some 235 unspecified breaches of their software patents supposedly to be found in current free software – including claims that this number includes 42 breaches within the Linux kernel alone.

Let’s be clear where I am coming from on this issue. I think that the whole idea of software patents is just plain stupid. I am fortunate to live and work within a region of the world that is not directly blighted by them but I can see other’s suffering and their existence indirectly constrains my marketing. My software is adequately protected by copyright and thus, in turn, I respect the copyright of others.

Now back at the issue. Many might assume that Microsoft are entitled (indeed enjoined by their shareholders) to maximise their profits – and that includes capitalising on their “intellectual property”. OK fine, but as even Steve Balmer says “It’s all about the developers”. Why is that important? Well, free software is created by our fellow developers with some (admittedly important in some cases) input from commercial enterprises. An attack on free software is a direct attack on developers. Just how many software patents have you potentially infringed upon in the last year writing code? I bet you have no idea – it would take more than a year of effort to find out. In all probability any individual patent might prove “obvious” or based upon “prior art” but can you afford to fund a defence against the might of Microsoft? Anyone who publishes or sells their code is vulnerable to inspection by others and this raises the potential of your being sued for a breach of someone’s software patent. The better the coder you are the more likely it is that you will infringe an unknown patent somewhere. I suppose you will be OK writing code in an “official” .NET language and only using the “widgets” in Visual Studio but there is the potential of Microsoft waiting in the wings to punish or tax your creative efforts if you should dare step outside of the “obvious”.

Paul Graham has suggested that the fear of software patents should not constrain start-ups. His essay makes the case that software patents are “big guns” used by successful businesses in their deals and manoeuvrings among themselves. It may be that Microsoft sees their current sabre rattling in the same way – perhaps targeting IBM. However this is different and could misfire badly for Microsoft. An attack on developers – even if they attempt to “spin” the attack as only directed to a small minority of developers – is a very dangerous strategy. I for one would have to re-evaluate my choice of development tools very quickly – certainly the .NET Framework (like the Java framework) makes it far to easy for a third party to analyse my code and to make a case that it could infringe upon some wildly generalised description of a software patent.

2007-05-11T02:59:00.000-07:00

If not Basic then where should we GOTO?

May I start with a quotation?

The act of focusing our mightiest intellectual resources on the elusive goal of goto-less programs has helped us get our minds off all those really tough and possibly un-resolvable problems and issues with which today's professional programmer would otherwise have to grapple.
John Brown (I am assuming John Seely Brown but please correct me if I’m wrong)

Many will be to young to remember the first flush of “structured programming” and the demonisation of the “GoTo” statement. I suppose it was true that in the absence of the “GoSub” (that early and soon forgotten “silver bullet” of programming) many programmers managed to produce a great quantity of un-maintainable spaghetti code that was clearly un-maintainable. All that happens now is that the mess written by poor programmers is hidden inside classes (OOP being a more recent silver/spent bullet) where we cant see it. Yet there was nothing fundamentally wrong with the unequivocal GoTo. It lives on in the Assembly code underlying most programming languages – without the Jump statement no branching of the code would be possible. And what after all is the “break” statement popular in so many languages (or say the VB Exit For) but a GoTo in disguise **.

Mark Guzdial of Amazon recently wrote a blog entry titled “Plea to Language Designers: Bring Back GoTo!” in which he made the persuasive case that there was an enduring need for programming languages accessible to a much wider audience than just the professional developer. Indeed he suggested that new languages might try and “span” the requirement providing a simple procedural syntax at one level while also supporting the complex needs of the professional with additional structures and syntax more suited to building large software applications. Mark identified the GoTo as a statement epitomising the simple procedural languages that were available everywhere at one time.

So where are the programming languages for non programmers?

I suppose that spreadsheets (and in particular the near ubiquitous MS Excel) provide an end user programmable facility meeting some of the need. Most people with reasonable numeracy skills can manage multi-stage calculations using a spreadsheet. However few try to manipulate text using such tools – indeed the complexity of text processing seems to be amplified by the spreadsheet paradigm. Even fewer venture into scripting languages like VBA – I suspect because they are hidden away “behind” the spreadsheet rather than brought to the foreground and into the focus of the average user. Spreadsheets have their limitations both in their functionality and in their user interface (VBA or no VBA) and I think what we are looking for should not come as a side effect of an application but should be a tool in it’s own right.

I think there is a genuine need for a general purpose programming language specifically designed to be accessible to all. A language that can run across multiple platforms and become the “natural” choice for authors writing books on general aspects of computing, magazine articles aimed at the hobbyist and further education tutors offering courses to adults. The “slot” that Basic filled so well for so many years.

I have been re-reading Edward Nilges’ “Build your own .NET Language and Compiler”. This is a thorough (yet non academic) introduction to language design and compiler writing, written in an entertaining style – a good read. The general theme of the book is the design and implementation of a “Quick Basic” clone to run against the .NET Common Language Runtime (CLR). However the book is not dominated by the Basic language and deals well with domain specific languages and many of the fundamental issues facing a language designer. The only disappointment of the book is that while it deals with the “front end” of the task of compiler writing in great detail it falls very short when it arrives at the “nitty gritty” of CLR code generation itself. One can surmise that this part of the book was deemed by the author or publishers too “dry” for the intended readership. However the supporting web site offers downloads of all of the code described in the book – together with the “finished” product. Reading the book can re-kindle the sense of excitement generated by the first brush each of us had with a computer programming language.

I mention this book here because one’s first thought for a new Basic would be – why not Basic? What were the strengths of Basic and what weaknesses might we avoid in a replacement? [I am using the past tense here to differentiate between the modern OO forms of Basic such as VB.NET and the original interpreted versions implemented on so many computer platforms over the years.]. Interestingly, some of the weaknesses of early Basics have the potential to become strengths. Early Basics were interpreted, line by line which was slow and precluded statements and data constructions that bridged multiple lines. With faster processors and modern JIT compilation techniques interpreted languages are now all the rage. Early Basics had a cavalier attitude to variable declarations (although they were almost all strongly typed) and that is a feature of some modern scripting languages. Basics have often been generous in allowing automatic “casting” between types and that does feel like a strength.

The early Basics had line numbers as logical labels and interpretation/execution proceeded in the order of the line numbers – allowing for the odd GoTo of course. This worked, but was clumsy – later Basics introduced labelled subroutines and then proper Functions (including void functions or subroutines with passed parameters and line numbers began to disappear. Originally variable scope was global with terse memory saving variable names and limited data types. Later Basics allowed some limitation of variable scope, user defined data types and long variable names to match the descriptive language keywords. Later Basic code could be written in a descriptive and often self documenting manner.

What are the “ideal” characteristics for a new basic language? (note the lower case b)

We could probably argue all day but we would need to keep things simple and accessible.

On balance I would plump for an interpreted case insensitive procedural language with descriptive keywords (that’s the Basic bit).
I’m ambivalent on the question of C style “braces” around statements verses the end of line character combined with block ending keywords like “End If” or “End Case”.
I think I would envisage targeting a subset of the .NET Framework CLR (or DLR) but the Java byte code runtime would be a valid alternative.
Again I am ambivalent about strong verses weak typing – weak typing gets “newbies” writing code faster but can demand a better understanding of data types when things go wrong that that required by a language with clear typing from the very start.
There are interesting issues arising around the natural UI of such a language – should it be the ubiquitous “desktop style” window with a full set of “widgets” or a graphical plane that also acts in a similar way to a “console” or terminal window?

** Bizarrely the GoTo lives on (just) in VB.NET whereas the oh-so-useful GoSub did not make it in the jump from VB6 – as the Return statement was claimed by the language designers for Functions I suppose .

2007-05-10T04:30:00.000-07:00

The eval() side of JSON

A very good post on “The Curious Schemer” points out one of the “features” of JSON I had rather glossed over – even if it is supported by my .NET JSON writing class. It is the JSON object without a “name” tag. These can be happily supported within an array of such objects (as you could see in my sample code) but any object outside of an array must have a name string. You can read the explanation if you follow the link above.

The Coding Horror at Microsoft.

I posted a couple of short comments recently that have (gently I hope) knocked Jeff Atwood but his “Giving Up on Microsoft” is right on the money and well written to boot.

I quote: “As a software developer, you're doing yourself a disservice by pledging allegiance to anything other than yourself and your craft-- whether it's Microsoft or the principle of free software. Stop with the us vs. them mentality. Let go of the partisanship. We're all in this thing together.” Spot On I say!

An interesting link from the article to Mike Gunderloy’s "A Fresh Cup". I can respect what Mike is doing but I can’t find respect for his apparent beliefs. There is no place for belief systems in software (or any other aspect of life) – we need to break out of this medieval thought pattern.

Installation kits

One of the joys of working with corporate clients on .NET projects is that you can rely upon your customer’s IT support team to ensure that your target client machines have the appropriate .NET Framework installed (or whatever runtime you are using) and that the required database technology is in place and functional. You can implement central error logging, silent code updates and installing an executable is the just a “copy” process. Outside of that world, you need a modern up-to-date installation program.

I have long been a fan of the Nullsoft Scriptable Install System available for download from SourceForge and have used it successfully for quite complex installation tasks. I can thoroughly recommend this package. I have often suggested it to others still struggling with the dreadful install packages generated by Visual Studio 6 for VB Classic applications. However, a link from Dr Dobbs site drew my attention to the WiX open source toolset for building .msi Windows installers. This again is available at SourceForge and looks a good candidate for even the most complex installation requirements – including creating SQL databases and installing Windows services. So two great free and open source installers ready to take on the toughest Windows installation challenges.

2007-05-09T05:07:00.000-07:00

Another class supporting JSON

After my post yesterday on a .NET class that writes JSON strings I spotted the existence of a free download from NewtonSoft that tackles the task in a different way. We all need choice – so check that one out as well.

Visual Basic 6 alive and well?

A very thorough post from Tim Anderson titled “Microsoft and the abandonment of Visual Basic 6” on the reasons behind the “end” of VB6 is worth reading if the issue continues to irk you. He makes the very valid point that the VB6 runtime is likely to be available a long way into the future – so you can keep on cranking out that VB Classic code. Certainly I have corporate customers still looking for new VB6 work with few signs that this choice causes them any doubts whatsoever.

A number to make your own

With the US movie industry (well the AACS) claiming rights over a 128 bit number (citing the DMCA) many people are now grabbing a number of their own. Visit Freedom To Tinker and get one for yourself. Mine is 02 03 C7 92 55 15 BC 96 A1 04 BB 0A 15 91 7F A4 and I intend to protect it with the full might of the law ;)

2007-05-08T05:21:00.000-07:00

JSON re-visited from .NET

In a longish post last month I tried to provide a full and complete introduction to JSON. In that post I said that I would follow up with a .NET class that would take care of creating the JSON string and inserting all of the string delimiters and braces as required. I have to admit that while my first attempt worked, on close inspection, it looked like a candidate for the Daily WTF. Sure it had recursion and thus had the attributes of cool code but… Anyway, a newer, simpler, version is now available – and it still has recursion - so it’s still cool.

I have tried to keep the class interface as simple as possible with just two constructors and just two methods – one being the toString() that returns the JSON string. All values are passed as an object and that probably accounts for about half of the code inside the class – most of it ensuring that the content of each passed object is properly stored and not just a reference back to the original object – which would have meant data “leaking” if external objects were re-used. This class does not currently support the JavaScript “null” and has missed some .NET data types but I think that it covers most needs while being easy to extend if required.

Here is the class (in VB.NET)


Imports System.Text
Public Class NetJSON
#Region " Private Declarations"
  Dim myName As String = ""
  Dim myContent As New Hashtable
#End Region
#Region " Enumerations"
  Private Enum dataType
      dt_Boolean
      dt_Decimal
      dt_Double
      dt_Integer
      dt_string
      dt_Array
      dt_NetJSON
  End Enum
#End Region
#Region " Public Constructors, Methods and Properties"
  Public Sub New()
  End Sub
  Public Sub New(ByVal nameString As String)
      myName = nameString
  End Sub
  Public Sub AddNameValue(ByVal nameString As String, ByVal Value As Object)
      StoreValue(nameString, Value)
  End Sub
  Public Overrides Function toString() As String
      Dim Resp As New StringBuilder
      Dim Firstcall As Boolean = True
      Dim TailBrace As String = "}"
      If myName.Length > 0 Then
          Resp.Append("{'" & myName & "': {")
          TailBrace &= "}"
      Else
          Resp.Append("{")
      End If

      Dim myEnumerator As IDictionaryEnumerator = myContent.GetEnumerator()
      While myEnumerator.MoveNext
          Resp.Append(IIf(Firstcall, "", ", ") &amp; "'" & myEnumerator.Key & "': " & MakeString(myEnumerator.Value))
          Firstcall = False
      End While
      Resp.Append(TailBrace)
      Return Resp.ToString()
      Exit Function
  End Function
#End Region
#Region " Private Functions and Subroutines"
  Private Function MakeString(ByVal ThisData As Object) As String
      Dim ThisType As dataType = GetDataType(ThisData)
      If ThisType = dataType.dt_Array Then
          Dim TestArray(ThisData.length) As Object
          Dim aLoop As Int16
          Dim ArrayStruct As New StringBuilder("[")
          Dim FirstCall As Boolean = True
          ThisType = GetDataType(ThisData(0))
          For aLoop = 0 To ThisData.Length - 1
              ArrayStruct.Append(IIf(FirstCall, "", ", ") & MakeElementString(ThisData(aLoop), ThisType))
              FirstCall = False
          Next
          ArrayStruct.Append("]")
          Return ArrayStruct.ToString()
      Else
          Return MakeElementString(ThisData, ThisType)
      End If

  End Function
  Private Function MakeElementString(ByVal ThisData As Object, ByVal ThisDataType As dataType)
      Select Case ThisDataType
          Case dataType.dt_Boolean
              Return IIf(CBool(ThisData), "true", "false")
          Case dataType.dt_Decimal
              Return CStr(ThisData)
          Case dataType.dt_string
              Return "'" & CStr(ThisData) & "'"
          Case dataType.dt_NetJSON
              Return ThisData.ToString()
      End Select
  End Function
  Private Function GetDataType(ByVal Value As Object) As dataType
      If TypeOf Value Is Array Then
          Return dataType.dt_Array
      ElseIf TypeOf Value Is Single Or TypeOf Value Is Double Then
          Return dataType.dt_Double
      ElseIf TypeOf Value Is Decimal Then
          Return dataType.dt_Decimal
      ElseIf TypeOf Value Is Boolean Then
          Return dataType.dt_Boolean
      ElseIf TypeOf Value Is String Then
          Return dataType.dt_string
      ElseIf TypeOf Value Is Integer Or TypeOf Value Is Int16 Or TypeOf Value Is Int32 Then
          Return dataType.dt_Integer
      ElseIf TypeOf Value Is NetJSON Then
          Return dataType.dt_NetJSON
      End If
  End Function
  Private Sub StoreValue(ByVal nameString As String, ByVal Value As Object)
      Select Case GetDataType(Value)
          Case dataType.dt_Array
              Dim copyArray(Value.length - 1) As Object
              For aLoop As Int16 = 0 To Value.length - 1
                  copyArray(aLoop) = Value(aLoop)
              Next
              myContent.Add(nameString, copyArray)
          Case dataType.dt_Boolean
              Dim wrkBoolean As Boolean = CBool(Value)
              myContent.Add(nameString, wrkBoolean)
          Case dataType.dt_Double, dataType.dt_Integer, dataType.dt_Decimal
              Dim wrkDecimal As Decimal = CDec(Value)
              myContent.Add(nameString, wrkDecimal)
          Case dataType.dt_string
              Dim wrkString As String = CStr(Value)
              myContent.Add(nameString, wrkString)
          Case dataType.dt_NetJSON
              myContent.Add(nameString, Value)
      End Select
  End Sub
#End Region
End Class

Now – how to use it?

Let’s start with the simple example from my original JSON article:


Dim aJSON As New NetJSON("visitor")
aJSON.AddNameValue("name", "EmilySmith")
aJSON.AddNameValue("phone", "0123 45678")
aJSON.AddNameValue("email", "emily@startup.com")

this creates a instance of the NetJSON class and populates it with three name/value pairs. To extract the JSON string – just call the toString() method thus:


myJSONStringValue =  aJSON.toString()

We can then extend this simple example by adding an array of un-named JSON objects:


Dim aJSON As New NetJSON("visitor")
aJSON.AddNameValue("name", "EmilySmith")
aJSON.AddNameValue("phone", "0123 45678")
aJSON.AddNameValue("email", "emily@startup.com")
Dim bArray(6) As Boolean
Dim aLoop As Int16
For aLoop = 0 To 6
   bArray(aLoop) = True
Next
Dim jArray(6) As NetJSON
For aLoop = 0 To 6
   bArray(aLoop) = False
   jArray(aLoop) = New NetJSON
   jArray(aLoop).AddNameValue("weekid", aLoop)
   jArray(aLoop).AddNameValue("wending", "07-05-07")
   jArray(aLoop).AddNameValue("visited", bArray)
Next
aJSON.AddNameValue("history", jArray)

Here I created a Boolean array to represent days of a given week together with an array of my NetJSON classes representing 7 weeks of history. Each history week has three name value pairs – one of which is an array of Booleans. The code then passes the array of NetJSON objects to the first NetJSON class.

The result would be something like:

{'visitor': {'email': 'emily@startup.com', 'history': [{'visited': [false, true, true, true, true, true, true], 'wending': '07-05-07', 'weekid': 0}, {'visited': [false, false, true, true, true, true, true], 'wending': '07-05-07', 'weekid': 1}, {'visited': [false, false, false, true, true, true, true], 'wending': '07-05-07', 'weekid': 2}, {'visited': [false, false, false, false, true, true, true], 'wending': '07-05-07', 'weekid': 3}, {'visited': [false, false, false, false, false, true, true], 'wending': '07-05-07', 'weekid': 4}, {'visited': [false, false, false, false, false, false, true], 'wending': '07-05-07', 'weekid': 5}, {'visited': [false, false, false, false, false, false, false], 'wending': '07-05-07', 'weekid': 6}], 'name': 'EmilySmith', 'phone': '0123 45678'}}

Which, as I said before, is about as complex as it gets - although I hope that with my .NET class writing that complex string gets a whole lot simpler.

2007-05-07T03:07:00.000-07:00

A language for all seasons

I find programming languages are at least as interesting as the applications we build using them. Part of the reason I find JavaScript so fascinating is its idiosyncratic but oh so effective approach to so many of the challenges facing any language developer. As a Visual basic classic user from the earliest versions it was great watching the language itself develop although I always got the feeling that the team responsible for Visual Basic within Microsoft never quite “got” how it was being used by the development community at large. The first version of VB.NET rather confirmed that feeling – it was a mess that failed to meet the need at so many levels I am certainly not going to attempt to list them. However, things got better and the current iteration of VB.NET is an excellent development language for a very wide range of applications. It is also clear that the Visual Basic team at Microsoft care about the language and about how it is applied. The Visual Basic IDE and language are now quite close to reclaiming the RAD (rapid application development) label lost in the confusion as VB migrated to the .NET platform. It is also great to watch the VB team continue to extend the language into new areas making VB.NET one of the “hottest” languages around.

Don’t believe me, then take a look at the series on closures (lambda expressions) on the VB team blog written by Jared parsons. A year ago I posted some notes on the use of closures in JavaScript and I for one am looking forward to having lambda expressions in Visual Basic and (by implication) the facility to treat functions as first class objects.

It looks as though Visual Basic is in for some interesting times. I am not just talking about the new language features, welcome though they are, but also about the approach that VB has to data typing. If you want, you can bolt your VB app right down when it comes to data types and implement a very strict typing regime. Alternately, you can take advantage of VB’s liberal nature and allow the automated casting of data types and take advantage of “late binding” to simplify your code – at a very moderate performance penalty. The reason I raise this is the fascinating challenge of the Dynamic Language Runtime (DLR) and the choice of VB as one of the prime scripting languages for the Silverlight project.

We have had VBScript around for a long time and VBA is arguable yet another scripted form of VB. VBSCript was the foundation of Classic ASP, popped up as a development tool for those building applications for the Windows CE (now Windows Mobile) platform and (of course) runs as a scripting language on Microsoft Operating systems. Heavens, there was even client browser support in Internet Explorer although here it lost heavily to JScript. But the days of Classic VBScript are probably coming to a close as a new scripting version of Visual Basic running against the .NET Framework comes to the fore. I love the idea of being able to run a loosely typed Visual Basic scripting edition in an environment where it will be able to co-operate with a strictly typed version of itself running against the CLR. However, things are expected to develop from there. The next version of Visual Basic (the one after VB9/Orcas) currently dubbed VBx looks likely to straddle both worlds – a language for all seasons capable of running seamlessly against both the CLR and the DLR.

2007-05-03T05:17:00.000-07:00

Has Jeff Atwood gone over to the dark side?

In a blog post dated 1st. May on Coding Horror, Jeff Atwood suggests that developers should learn to use a full featured graphics editor – and he very firmly does not believe it should be MS Paint. He says that we all need to learn the difference between JPG and PNG – Well Jeff I’ve news for you, many developers understand the difference very well – they also understand the difference between loss-less and lossy compressions and the various ways that an image can be represented in a file format. This is stuff they work with all the time – files, software – stuff like that.

Now for my part, I can use (yes) MS Paint to knock up a button or two – or even create a web page title bar as a gif image of a fancy font but when it comes down to real graphic design then I go to an expert. My expertise is application design coupled to a reasonable competence in several programming languages but I know to leave graphics to the experts - just as I trust those same experts will leave areas where I am competent to me.

By all means encourage developers to develop new skills – a new language would be favourite but let’s not belittle the skills of graphic designers by confusing those very real skills with competence in a graphic package’s user interface.

My developer’s graphics toolkit is:

MS Paint - for the basics although it is an amazing tool in the hands of an expert.
Formati 11 – to manipulate gifs
SmartDraw – for the odd diagram used in presentations or documentation
Easy Thumbnails – to resize and manipulate the compression/quality ratios of JPG’s

Each a simple tool and together they provide me with more than enough to ensure that I can deliver my designer’s graphics to the intended audience. That’s what it is about – delivering, not creating, graphical images.

2007-05-02T02:53:00.000-07:00

A Deep Understanding

Do take a look at Eric Sink’s take on “The Coding Horror’s” dismissal of Charles Petzold’s book on WPF. The basic premise of Jeff Atwood’s comments seemed to be that a snazzy page layout was an essential attribute of a good book on a given subject. Eric Sink is a much better writer than me so I will leave the analysis to him but I would like to support the point about “deep understanding”. I wrote about the potential dangers of developers assuming that they understood a technology on the basis of having read a single book – probably with a title like “Learn X in 21 days”. Great for an introduction but if you are going to start designing a new piece of software on the basis of such superficial knowledge then you had better be sure that it’s a prototype. Sometimes you are lucky and there will be a book written but someone as thorough and knowledgeable as Charles Petzold – and even luckier if they can write as well as he can. My venerable copy of his “Programming Windows” sits in pride of place on my office bookshelf.

Plumbing the depths of ASP.NET 2.0

It has been in pursuit of a deeper understanding of the ASP.NET 2/3 environment that I have been working my way through Stephen Walther’s “ASP.NET 2.0 Unleashed”. I am still doing lots of ASP.NET 1.1 work but I need to start in, very soon, on the architecture for an ASP.NET 2/3 application and I needed to further my understanding of the low level enhancements available in the current release – only two years late you might think. Let me say straight away that this is an excellent book and I am learning a lot – I can also say that this book covers the ground from the simplest start point to some very advanced features of ASP.NET – the stuff I needed to review and digest – not bad for a single volume – although this is a pretty hefty book.

Do I have a complaint? Well sort of. There is no way on earth that I would write my code the way that Stephen Walther seems to like to do. He mixes client and server side scripts with CSS and HTML all into a single file. Call me “anal retentive” or anything else you want but I like a clear separation between client side JavaScript, CSS, HTML, the page controls themselves and then the server side code – all in all I like to work with the Visual Studio IDE – it suits my style of working. If I hit a problem, I know exactly where to go in the IDE and fix it – I do not have to sift through a single file supporting multiple mark-up standards and multiple syntactical styles. Besides “Intellisense” does a lot of the hard work for me. However I do wonder if the book’s coding style actually supports the intended purpose better than one that revolved around the Visual Studio IDE. It needs fewer pictures for a start and almost certainly condenses the source code being described. It also makes me think – converting what I see to how I would implement the code in reality. That is a certain aid to true comprehension and understanding.

So, overall I am going to give this book top marks – even if I would shoot anyone working for me who wrote code in it’s style.

When I have finished this book it will be time to read Charles Petzold’s on WPF because – well see below:

More on VB and Silverlight

There is a great post on the VB Team blog

This one starts (more or less) with “Let me get this straight… You want to write a web plug-in VB.NET… that doesn’t require the full .NET framework to be installed… and runs on the Mac… and Firefox … but supports the full VB.NET language?”.

And the goes on to list the elements of VB being supported on the Silverlight platform.

Late Binding: One of the most definitive features of the VB language, I find late binding to be especially useful in writing Silverlight code-behind.
Conversions: Implicit and explicit conversion operators are available—Ctype, CStr, etc.
Linq: Currently, Linq over objects is supported
String utilities: Len(), InStr(), Mid(), UCase(), etc
VB Collection: Most folks either love it or hate it. A subset of generic collections is also supported. (Non-generic collections are considered obsolete for Silverlight.)

Dictionary(Of Key, Value)
List(Of T)
ArrayList (Obsolete)
BitArray (Obsolete)
Hashtable (Obsolete)
Queue (Obsolete)
SortedList (Obsolete)
Stack (Obsolete)

Math utilities: Rnd(), Random()
IIF()
Information utilities: Things like IsNumeric(), IsDate(), UBound(), LBound(), and so on
Date utilities: Now(), TimeOfDay(), Year(), etc
Constants: vbCrLf, vbTab, etc. Some of the more obscure constants have been removed, but the core set is included.
All core VB Language Constructs: Type Inference, anonymous delegates, Handles

There is also a complete list of the downloads needed to develop and run a Silverlight app using VB as your language of choice. It is still a bit complicated to set up but well worth it if you have the time and a spare development machine.

2007-05-01T05:10:00.000-07:00

The Dynamic Language Runtime

Rumours of a new .NET Framework layer dubbed the Dynamic Language Runtime (DLR) have been flying around for a few days – but the existence of this very important layer has now been confirmed at MIX 07 and you can download the alpha 1.1 of Silverlight with support for both IronPython and JScript. The DLR is planned to support Ruby and Visual Basic (presumably VBScript) by the time we get to a full release and interoperability with statically typed VB.NET and C# is a key feature of the DLR. So that’s great news for me as I have been looking for additional support for JScript for some time and great news for anyone investing time in building their JavaScript skills. OK, this is all at the alpha stage at the moment but is being built on the foundation of the work already done to integrate (Iron) Python into the .NET family of programming languages so I think we can have high expectations for the final release.

The Silverlight alpha also includes a new CLR (Common Language Runtime) – few details as yet but looks an interesting move to enable the CLR on a lot more OS platforms. Is this pro or anti mono I wonder?

New .NET Frameworks

MIX 07 also saw announcements of two new “productivity” frameworks providing for both dynamic data access and dynamic ASP.NET controls – sounds lime ASP.NET on rails to me. This had to come I suppose if ASP.NET is to continue to slug it out at the top as a toolkit for web based applications. This is not an empty space mind – I have been meaning to check out ActionPack at CodePlex for a while.

The MIX 07 announcements are laying down quite a challenge – it might make this year’s PDC well worth attending and it has been some years since I have felt that.

2007-04-23T04:11:00.000-07:00

Humiliation is the name of the game

I remember reading a great book by David Lodge titled “Changing Places”. In this novel (written in a sequence of literary styles – I know that sounds dreadful but it is subtle and funny) an English academic who has swapped places with a US professor introduces a game to his US university English Department hosts one evening over dinner. The game is called “Humiliation” and the object is to name a book that you have not read but that your peers would consider it humiliating to admit that you had not read. The dilemma is a choice between losing the game or admitting to a failing in such an august company. In the novel, the desire to win at all costs brings a young US academic to the end of his career.

At the risk of my reputation, it is time to admit that I have only just tried using JSON. I have always been quite comfortable with XML and while acknowledging that JSON looked like an interesting alternative I could not see a compelling reason to use it. I always try to promote a rich user experience within my web applications and this can often mean using Ajax techniques. Sometimes, Ajax is not possible (some portals do not work well with this technology) and sometimes it is not advisable for other reasons. One technique I have evolved is what I call “Fake Ajax”. This involves passing lots of support data to the client in hidden page fields ready to be loaded into JavaScript objects as and when required. The same technique can be used to pass data back again if it has been manipulated by the user. It looks slick, improves the user experience and all at a minor up front cost of adding some bytes to the size of the initial page load. An obvious candidate for JSON you would think. Certainly I was persuaded to give it a go when I needed to deliver what was, in effect, a two dimensional array of Booleans to a web page – just in case the user needed to review or edit the data. XML seemed a rather heavy approach to a few “bits” of data so it was time to try JSON.

If you need an alternate opinion on the joys of JSON then you should visit Dustin Diaz’s “JSON for the masses”. He slightly confuses the concept of ‘name spaces’ with ‘scope’ but it is a fun read Part of the problem with JSON is that applying it is so easy all examples look too simple and banal – leaving the reader with the idea that somehow JSON is more complex and difficult to understand. Thus I am going to show you the code for managing a complex block of data – just to demonstrate how simple it all is. The “fiddley” bit in building a Jason string is in ensuring that your data is in name/value pairs as it seems easy to lose track – particularly when you start creating arrays of objects. The values themselves can be strings, numbers, Booleans, arrays, null or another object described in the same format. As this is JavaScript you can also supply function definitions as values as well.

A simple, yet complete, introduction to JSON

My data scenario is that I have some related events that occur during a week but that may or may not coincide in a given day. I have to provide access to the most recent week’s record so that the user can keep things up to date. In addition, the user might want to review (and possibly) edit the previous n weeks of similar records. In my JSON object I have separated the current week from the historical data but it might be proper in another application to combine the two variations into a single list of values. My data is simple – just a list of Boolean values indicating if a given event occurred on a given day.

But let’s take it one step at a time.

I can define a relatively simple JavaScript object as a JSON string like this:

{‘visitor’: {
   ‘name’: ‘Emily Smith’,
   ‘phone’: ‘0123 56789’,
   ‘email’: ‘emily@hercompany.com’
   }
}

(the tabs, spaces and line feeds are to aid readability and are not needed)

What we have here is an initial name/object pair – the name being ‘visitor’, followed by a colon and then the related object wrapped up in its own pair of braces. This object , in turn, contains three name/data pairs. Each name and data element are separated by a colon and each pair is separated by a comma.

To turn this JSON string into a JavaScript object – all we need to do is use the JavaScript eval() function. The string might be passed to the client side web browser in an element of the page HTML or in response to an Ajax request.

The JavaScript code line would look something like:

var vData = eval(‘(‘ + request.responseText + ‘)’);

Assuming that your JSON string is stored in the request.responseText object then subsequent code could address the data elements of the newly created object (vData) using something like the code below:

var name = vData.visitor.name;
or
var email = vData.visitor.email;

which has the advantage of being clear straightforward and hopefully self documenting.

So let’s introduce an array to our JSON object.

{‘visitor’: {
    ‘name’: ‘Emily Smith’,
    ‘phone’: ‘0123 56789’,
    ‘email’: ‘emily@hercompany.com’,
    ‘payments’: [123.45, 234.56, 345.67]
    }
}

with the new data being available at the client with code something like:

alert(‘The second payment was ’ + vData.visitor.payments[1]);

showing that an array of numbers is just as accessible as our original object strings.

Now let’s try the object from my project:

{
 'thisweek': {'eventa': [true,true,true,false,false,false,true],
              'eventb':[true,false,true,false,false,false,false]},
 'history': [
             {'weekid': 8,
              'weekdate': '11-03-2007',
              'eventa': [false,false,true,true,false,true,true],
              'eventb': [true,false,true,true,false,true,true]
             },
             {'weekid': 7,
              'weekdate': '04-03-2007',
              'eventa': [false,false,true,true,false,true,false],
              'eventb': [true,false,true,true,true,true,true]
             }
             and so on…
            ]
}

So what does this consist of? At the highest level we have two name/value pairs (the names are ‘thisweek’ and ‘history’). The first value is an object and the second is an array – which contains more objects. Each object in the array is separated by a comma and contains 4 name/value pairs – with the second two such pairs being themselves arrays (of Booleans). Using integer values would have made the string shorter but the total string length here was not affecting performance.

I don’t think it can get any harder than this and at the client end there is almost nothing to do. Many prefer JSON to XML because it is more succinct, simple to address from client side JavaScript and yet the meaning is still clearly human sensible.

What’s the downside with JSON? Well I think that debugging an object string being created by code from server side data is more difficult than debugging the equivalent XML. I find it pretty easy to spot malformed XML tags and once any of those are tidied up it is simple to load an XML string into your web browser and inspect the structure. Long JSON strings are a mass of brackets, quote marks and commas that can take some sorting out. I suspect that a tool could be written to do this as a debugging aid but a casual run around Google did not turn up anything that went far enough. If the eval() function returns and error, it tends to be rather enigmatic.

For immediate use within an ASP.NET application it makes sense for me to write a new class to accept and store name/value pairs with a method that can serialise the stored content as a JSON string. I will publish my prototype shortly.

Conclusions from my first foray? Well I am going to make strategic use of JSON into the future as it simplifies the storage of data at the client end of the process. I will look for opportunities for JSON where that simplification can improve performance of simplify the code.

JSON has one key advantage in true Ajax enabled applications. It can be used to download data from a third party server. The XMLHttpRequest object is restricted to accessing the domain that served up the JavaScript file currently being executed. The Script element does not share this security restriction and thus you can request JSON format responses from alternate domains and from web services. Indeed, if you provide a web service then you should consider offering JSON as one of your response formats as this allows the consumption of your web service asynchronously from a client rather than just from your user’s servers.

You might well want to visit JSON.Org to pick up a copy of the JSON Parser (for use when accepting JSON from less trusted sources) and the JSON “Stringifier”. You might also want to download a version of Douglas Crockford’s JavaScript “minifier” to compress your JavaScript files if you do not already have one available on your development machine. It was Douglas Crockford, that guru of JavaScript, who created the JSON meme in the first place.

2007-04-19T01:04:00.000-07:00

Thoughtful Python

Kris Kowal has written a well researched and thoughtful review of the key differences and similarities of Python and JavaScript as an aid to determining how well they should work together. Anyone who reads this blog will know I am a JavaScript fan but Python is right up there as well.

2007-04-19T01:02:00.000-07:00

Adventures with the JIT compiler and Garbage Collector in .NET

This great analysis starts with a small mystery – how is it that a compiled .NET program can reduce it’s memory footprint in advance of the execution of the code lines freeing up a large data object? Unlike the author, you will probably not be bothered that it is – just glad that the task gets done – but stick with the investigation as it takes some interesting turns “down and dirty” in the assembly code. This is good technical writing and a fascinating view on the interesting relationship between what you code and what is subsequently executed – nothing like as straightforward as you might imagine.

2007-04-19T01:01:00.000-07:00

Binary Coded Decimal

Dr. Dobb’s Portal has the first of a two part article on a new IEEE standard defining a data type for integer, decimal and floating point numbers.

The article took me back a ways – remembering the tussle it was to produce accurate and (just as important) fast calculations on large decimal numbers on early machines with 16 bit registers and very limited data types.

This article sees a future where maths co-processors are revived – not as before to accelerate floating point arithmetic but to support accurate decimal arithmetic. I have long held the opinion that general purpose computer design should change to encompass an increased number of specialised processors dedicated to specific areas of computation – we might as well have added efficiency from the notion of multi-processor machines now we have “hit the power/heat wall” on current processor designs.

2007-04-19T00:59:00.000-07:00

Rich Internet Applications RIA

The Apollo - Silverlight - Ajax wars.

What’s that – Ajax techniques up there alongside the plug-ins from Adobe and Microsoft? Yes – because that is the alternative technique that already delivers a richer user experience and has a great deal more mileage left in it.

The initial reviews are of necessity rather shallow (and perhaps partisan) but they are coming in. Try:

Ted Patrick (partisan)

and Kevin Hoffman (bemused)

The consensus so far seems to be – nice marketing Microsoft but where is the product?

2007-04-17T05:49:00.000-07:00

Silverlight

Microsoft have released a CTP version of their WPF / E browser plugin designed to provide users with a richer multi media “experience”. Not sure about the experience but it is all about extending the capabilities of browser based applications and providing a new programmable platform for developers. We saw Adobe entering this arena with the alpha version of Apollo a couple of weeks ago – could be time for an interesting “head to head” soon.

Scheming

I picked up on a post by the Secret Geek and downloaded a version of scheme to run on my smart phone. This will make a nice companion to Daniel P Friedman and Matthias Felleisen’s “The Little Schemer” which I have enjoyed up to now as simply a mental exercise on long journeys. It will be great to be able to edit and run the code as well.

2007-04-17T01:48:00.000-07:00

First Class Rant

A not to be missed rant

2007-04-13T06:02:00.000-07:00

Customer Service – Non Merci, at CDG

I just spotted Joel Splosky’s comments on Charles De Gaul airport in Paris. OK I’m going to quote in full:

“I just got back from a trip to Israel, having, in a fit of stupidity, decided that it wouldn't be such a bad thing to take Air France and change planes at CDG. Nothing surprises the ground crew at CDG more than the arrival of the daily, scheduled flight from Kennedy. Every single thing about that airport is broken and it's the worst place in the world to change planes.”

His post is correct on every count. Air France is a nicer airline to fly the Paris/New York route than the US carriers (metal knives and forks, wine with dinner etc. etc.- I am talking tourist here not First Class) but the problem is CDG – it is truly dreadful.

2007-04-13T01:53:00.000-07:00

In Lieu of a Link

Yesterday Chris Swetenham kindly left a comment that suggested that “Lua” might be the programming language I was looking for as it had many similarities to JavaScript and had an implementation on the PSP. He did not leave a link but the site can be found at http://www.lua.org. As with many such projects there is extensive documentation available to those willing to invest a good few hours but precious little to persuade you to actually make such an investment. A little marketing can go a long way even in the esoteric world of language development – look at Haskell – Haskell developers seem to run the programming sub-reddit. However I did eventually find a “history” that gives a nice overview of the language as it developed. Worth a read in it’s own right.

2007-04-12T05:37:00.000-07:00

What is the Best First Language?

This question keeps coming around and there are a number of schools of thought. Each school has at least one very worthy proponent and a lot of good arguments based (often) upon personal experience or alternately on trying to recruit programmers with the “right” skill sets and approach to the task. There are views around that the wrong choices can “spoil” a nascent programmer (famously encapsulated by Edsger W Dijkstra as “The use of Cobol cripples the mind; its teaching should, therefore, be regarded as a criminal offence.” and “It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration.”).

As a newcomer to computer science I was first taught to program using Cobol and then moved quickly to writing in Basic running on DEC PDPs – so I was doubly damned. Perhaps I was saved (if saved I was) by the fact that I was also taught how operating systems worked and was exposed to the intimate relationship between code and hardware now largely hidden by compilers and runtime libraries. In addition, those of my generation were always strapped for computing resources and thus spent a lot of time tuning code to run efficiently on a given hardware platform. Having said that, a great many of today’s programmers who had the advantage of being the right age when the first home computer wave occurred, will admit to learning Basic as a first language without undue detriment. Perhaps they also had a closer relationship with the hardware? I can certainly remember helping out a neighbour’s 12 year old son with some assembler routines to support a time sharing routine that moved sprites around a screen and then, subsequently, with some more low level code to deal more effectively with hardware ports. That 12 year old is now a well respected IT professional.

A nice essay by Shlomi Fish got me thinking about this subject. The essay covers the ground well and comes to a well defended conclusion – although I am sure that any fellow developer will have his or her own ideas on the subject. Having said that, I would be surprised if, after consideration, Java was on the short list of many. One interesting point raised in the essay was the suggestion that XHTML might make a great introduction despite being a mark-up language. I tried this once when I came to the rescue of a school student who, for reasons beyond his control, had not secured a “work experience” placement and was in need of a two week opportunity. I needed to come up with something that would be sufficiently challenging, would provide some insight into the life of a professional developer and would not require too large a slice of my day to support. I plumped for HTML and that student not only mastered the fundamentals but wrote an introductory guide to HTML that he passed on to others. Shlomi’s conclusions by the way were Perl or Python or Ruby with Perl as first choice.

I posted a couple of weeks ago about making a choice between Logo and a new “upstart” called Robo to introduce to my nine year old son – a choice triggered by the nature of some mathematical tasks he had been set from school rather than as a considered choice of a language to introduce the craft of programming. We have started with Robo as it has an excellent visual appeal but I think that our work in that language will quickly run into its limitations. I have corresponded with Arvid Halma who is keen (certainly at this stage) to restrict Robo to a minimalist Turing Machine which is a fine exercise but (in my opinion) may thus fail to meet it’s apparent objectives of being a child oriented initial programming language. I suggested that something as simple as a stack might be all that was needed to extend the capabilities of Robo to more complex class of programming task. While I have a sympathy for the idea that an initial language need not be overburdened with functionality – no-one needs the myriad of overlapping choice presented by (say) the Java class library – I also feel it should introduce most of the fundamental concepts consistent with the design philosophy. I would not therefore expect to see OO classes in Robo but some mechanism to store and manipulate data (beyond painting squares which admittedly has some utility) looks like an essential from my viewpoint.

Self taught programmers have historically started with the programming language that was available to them. Thus it was Basic for a whole generation who first had access to home computers. If the One Laptop Per Child project succeeds in delivering the millions of machines it intends to children around the world then a new generation will grow up with the tools and languages “native” to that machine. For those who take to programming, the choices will be Python, JavaScript, an implementation of Smalltalk called Squeak, Logo and Csound (a programmable music and audio environment). A rich collection indeed and a choice that may have a profound impact upon the future development of programming languages as well as upon the CS courses being offered by universities around the world.

< Aside>
The OLPC project may prove to be “disruptive” on a great number of fronts. The “SUGAR” Interface will be the natural GUI for all those millions of children – will they want to move to the “desktop” we are all familiar with or will they blaze a trail to something completely new? The only interface those children will share by default with the rest of the world will be the Internet Browser. Is this likely to further reduce the power of Microsoft and does it mean that GNOME and KDE are doomed to never making it into the mainstream?
< /Aside>

To some extent I am deliberately confusing the issues of programming language choice for children and the choice of a first language for an aspiring professional. However I think that is at least partially valid – children may need an initial set of challenges that are more visual in their appeal than an adult but apart from that the fundamental requirements may have close parallels. The Internet provides us all with a wide choice of free (as in beer but often as in speech as well) programming language choices. Thus Shlomi’s choices of Perl, Python and Ruby are available to all (just about) irrespective of their operating system. Even Microsoft ships free versions of it’s Visual Basic .NET and C# languages to anyone with a broadband Internet connection. The options are wide – perhaps too wide when just about any opinion on the subject seems equally valid.

Economic considerations are at play here. A first language for children would ideally run on their platform of choice – in the affluent West that might be a Sony, Nintendo or Microsoft games console. However these consoles are sold at prices that require a cross subsidy from games software sales so they are closed environments to all but the most expert “hacker”. That is a shame although I am tempted to explore the “homebrew” market supporting the Nintendo DS as a potential option although I would prefer something that could support code development directly on the target device. So we end up back at our ubiquitous general purpose computing device – the PC. Economics may also play a part for aspiring professionals – there seems an urgent need to “get started” with a programming language popular for commercial development. This is surely why Java is such a popular component of university courses – it is reasonably modern and Java trained graduates are “in demand”. I think I would council a slower approach – work with a language that will teach you to program. The ideal programming language would probably help a beginner track the history of computing – learn about the structure of floating point numbers, learn how a stack works, memory allocation – oh dear I seem to be talking about C and that can’t be right can it?

Anyone who dips into this blog from time to time might predict that I would plump for JavaScript as a beginners language of choice. There is a lot to be said for it – it is both a procedural language and a functional programming language (you can use it like Scheme or as a C like language). JavaScript has an elegant approach to Objects and all objects are extendable. The only snag is that largely JavaScript is tied to the browser. The future looks good however and there is every chance that time spent learning this excellent language could turn out to be well invested if JavaScript truly becomes the Next Big Language. Just for the moment though, I might just have to plump for Python.

2007-04-11T08:53:00.000-07:00

Visual Basic for fun as well as profit

Mathew Gertz continues to bring a little more fun into the Visual Basic .NET world with his articles on the VB team blog. His latest posting – Simple Animation in Visual Basic is well up to standard but doubly worth it for this little item:

Note on transparency in .NET: In .NET forms, transparency is only one level deep, with that level always being the parent form. So, if you put a GIF containing transparent pixels into a PictureBox, the GIF will display correctly on the form (i.e., the form background will show where the transparent bits are). However, if you overlap a PictureBox onto another control, the transparency effect won’t stack – the transparent bits will continue to show the underlying form background, not any parts of the control “in between” them. This can look odd on the screen.

Hard or Soft?

Alex Popadimoulis at Worse Than failure raises an interesting coding issue - what should we hard code and what should be “soft” in our applications.

He starts with a nice example code snippet:

private void attachSupplementalDocuments()
{
if (stateCode == "AZ" || stateCode == "TX") {
//SR008-04X/I are always required in these states
attachDocument("SR008-04X");
attachDocument("SR008-04XI");
}

if (ledgerAmnt >= 500000) {
//Ledger of 500K or more requires AUTHLDG-1A
attachDocument("AUTHLDG-1A");
}

if (coInsuredCount >= 5 && orgStatusCode != "CORP") {
//Non-CORP orgs with 5 or more co-ins require AUTHCNS-1A
attachDocument("AUTHCNS-1A");
}
}

but then makes the telling (and likely to be misunderstood) point that this is good coding – the “business rules” are clearly stated and reflected in this code sample. Just because a value is a string literal or a numeric constant does not mean that it must be stored in a database or loaded from a program initialisation file. The business rules implemented above are intrinsically the same as all of the other rules coded into the same application. The other rules may not have such an obvious label to trigger their application but every program embodies a great many rules – why arbitrarily select some over others to become “soft”?

We all know that business rules change. What we have to do is identify those value driven rules that will change in a predictable manner (or to a predictable timetable) and separate those. You might also want to identify any value driven rules that are applied repeatedly throughout a code module (or even repeatedly within a given function). However these groups need different treatment. Values that you anticipate will change should certainly be soft and should ideally be maintained by the user. Values that are widely used with a code base might well be candidates for Global Variable status or (if you hate that idea) could be stored in a class that can be passed from method to method – the aim being to end up with a single point in the code where maintenance would need to be applied to change a given value.

As I said, business rules change. There is a limit to how much allowance for change it is sensible to build into an application. In my experience, when change comes it often takes an unexpected form and this can make any in-built provision redundant or can even be complicated by unnecessary program complexity written in advance to support change. Business change necessitates program code change – I think we are best supporting future change with clear well written routines and not with shards of pre-written code anticipating a specific change model. Any such code is almost bound to be untested (in the true sense) and may well be never executed – except in error.

2007-04-10T06:17:00.000-07:00

Faith in Methodology

In my post yesterday I “came out” as someone who strives to not believe in things. My post was deliberately confrontational with regard to some widely held belief systems but apart from reflecting an important part of my personality I was also trying to get across a point – developers should strive to exclude belief systems from their craft.

I think that a couple of recent posts regarding 37 signals could act as a good example of why I hold this opinion. It started with a post by Douglas Karr where he announced that he was cancelling his “BaseCamp” account. No big deal you might think and indeed it might not have been except that Douglas chose to share his reasons that extended beyond the fact that his development teams were simply not using the product. He was fussed that 37Signals seemed to be more a “cult” than a commercial proposition. Based upon his analysis (and some specific 37Signals posts like (watch out for the potentially (deliberately?) offensive word in the headline))) he was of the opinion that the BaseCamp service was unlikely to develop in a way that was going to reflect his (the Customer’s) needs so it was time to say goodbye.

Some of the comments attached to Douglas Kerr’s post will probably strike you as reflecting a belief system in place in a number of the contributor’s minds. If you don’t believe then you are the antichrist (to take the Christian term as it will be familiar to most Anglophones). This post at Uncov.com reached the same conclusions – using similar keywords such as cult and faith. 37Signals do seem to demand an adherence to a certain dogma from their customers and fan base which is OK up to a point. The worry is that some of the 37Signals fans will slavishly follow the dogma to the detriment of their end users and their own craftsmanship.

This post is in no way an attack on 37Signals – I respect their opinions (indeed share some of them) – I just do not believe in them. So my opinion on one of their key tenets might run something like: If in doubt – keep it simple. If simple doesn’t cut it for the user then complex is what you have to implement.

2007-04-09T03:28:00.000-07:00

I don’t believe in things

There I’ve come out publicly and said it. I don’t believe in things. There are some things I “know” but only in very limited domains where it is possible to really do so – otherwise I have a set of reasonably firm opinions based upon knowledge and another set of opinions (you would probably describe them as “forming”) where the available knowledge base is tenuous or difficult to evaluate. OK, I risk your derision, who is to be the arbiter of what is tenuous and just how many times have we had to demolish knowledge bases as new evidence has come along? In my defence I can only try and convince you that I try to test and re-evaluate my opinions in an honest effort to eliminate belief systems from my life.

I am certain that eliminating belief systems has helped me be a better professional developer and I am pretty sure that it has shielded my children from the excesses of other peoples belief systems as they have grown up. As a simple example, I do not believe that any of the triumvirate of Windows, Linux or Apple operating systems are inherently “better” but I can list the pluses and minuses associated with each for any given purpose. That is to say I can give you an opinion based upon my current knowledge but (and this is important) I expect that opinion to change on a continuous basis as the things (Operating Systems in this case) evolve and change as well.

So no, I do not believe in your god either – who or whichever it might be. As far as I can see all religions are based upon one or more ludicrous premise coupled to the usual enjoinders to not be too unpleasant to those of the same faith (you can usually be pretty unspeakable to the rest of mankind) plus a few “gotchas” to make sure that everyone commits a “sin” or two on a regular basis to keep them in fear of the super being (or beings) in the sky. There is also, usually, “cash in the bank” (or near currency) for those that claim to interpret the will of the sky dwelling whatevers. You can keep those belief systems well away from me thank you.

To be more up-to-date (religions usually having a purposely vague beginning – often coupled to book rights) I am not convinced by “Global Warming”. That is not to say that I think it is impossible, it is just that on the evidence presented (rather than the hysteria) I am no more convinced that by the year 2100 we will be appreciably warmer than we are today than that we will be seeing the first effects of a new ice age. Even if Global warming is real – what is the alternative? Check with the climatologists – it’s the next ice age. Even the most pessimistic acolytes of Global Warming can’t threaten you with effects as truly devastating (for human civilisation you understand – the planet is used to this stuff) as mile thick ice sheets covering the Northern hemisphere.

Global warming is a wonderful banner for governments, organisations and individuals of all kinds. Governments love it because they can use the words to justify new taxes carefully designed to not modify peoples behaviour in any way likely to threaten the extra income promised by the tax (examples abound – airport tax increases?). I expect that most government is the world are trying to think of a way of welding the messages of “Global Warming” to “Terrorism” in some way that will allow them to continue to erode freedoms at a pace that would otherwise be unthinkable. Organisations? – well the (UK) National Union of Teachers in their annual conference demanded (in the name of Global Warming you understand) additional days off when it was nice and sunny as it was unreasonable to expect them to work in temperatures above 24C (75F). If that idea catches on then Global Warming can be used as an excuse to terminate education programs around many parts of the world where the Global Warming nazis would prefer that the populace be maintained in poverty rather than risk their demand for basic energy supplies, clean water and even basic consumer goods.

As I do not operate a “belief system” around the issue of Global Warming and as climatology is outside of my area of expertise (although the contribution of computer models of climate might be an area where I could apply critical evaluation) than I will leave this specific area of debate to others. If nothing else I enjoin you to scepticism in all things – plus the willingness to invest (via taxes or whatever) in true scientific research willing – nay required – to openly publish negative as well as positive results.

One or two in the world may have started to believe that “Microsoft is dead” after reading Paul Graham’s piece titled just that. But Paul Graham does not believe that (indeed had to post again quickly to make that clear. Any rational analysis of Paul’s original post would have simply added evidence towards making the twin cases that Microsoft itself is no longer to be feared (it has lost it’s agility and apparently the ability to innovate) and that the MS desktop platform is no longer the necessary platform of choice for new developments. I am sure the “hoo hah” surrounding Paul Graham’s post was all driven by belief systems.

Actually this is about where I was going to start this post. Paul Graham was clearly keen to be one of the first to acknowledge the decline in the power of Microsoft and it’s dominance of modern software development. He is right – the signs are clear on two fronts. Web based applications can now rival the “desktop experience” of recent times and will now probably play a dominant role. Plus Microsoft seem driven by a desire to maximise “income now” at the loss of innovation or any apparent clear vision for the future. If you don’t believe me – and you should not - then how about some evidence upon which you can form an opinion? The Firefox development team have demonstrated to all that you can outperform Microsoft even in an arena that they have made their own. Here is a good post on how the retro-fitting of WGA to bought and paid for copies of Windows seeks to maximise the Microsoft tax at the risk of pushing yet another developer more quickly into a new camp where innovation still leads the way forward. My own business is currently researching a new software product and all the signs are that it will be web based rather than delivered as a desktop product. Microsoft development tools are still up there and in the running but this is no longer a “slam dunk” even for software aimed at the business market – the alternatives are compelling and look exciting to the development team. What is certain is that the final product will not require a Microsoft operating system at the client end – what was once a prerequisite is now; well not an irrelevance given the fractured nature of the client end browser DOM objects but certainly not the most important element in the planned architecture.

So sorry Easter Bunny I don’t believe in you (spoiler alert – generally people get their chocolate from the supermarket) and my Easter Message is to cast off your belief systems and start to form evidence based opinions. You and the world will be the better for it.

2007-03-30T04:56:00.000-07:00

Is security supposed to be a pain?

Anyone who has travelled on an aeroplane in the last few months is well aware that airport security is a right royal pain in the arse. You could also describe it as pretty pointless and almost certainly laughably ineffective but I will leave that to experts on security like Bruce Schneier . Network security seems to be going the same way – a lot of it is pure theatre and seems only to be there to hamper productive work and annoy system users. A lot has been written about the Vista security model of late and that does seem to favour the theatrical message dialogue over practical security designed to protect data and executables from external attack and end users from inadvertent error.

We are at the point of implementing a web based system for a customer when suddenly they wanted to include regular forced password changes for end users (built in but effectively turned off), minimum password lengths (built in already), no repeat use of passwords and the blacklisting of IP addresses after n failures to log in. I suppose their users should be thankful that they did not want to specify mix cased passwords including compulsory numeric and alpha characters as well. None of these requirements will appear at all strange to many business network computer users but this was for a customer facing web based application.

Why are frequent password changes a good idea anyway? Our security objective is to persuade the users to select a password that is unique to our system and sufficiently long as to defy most “dictionary attacks”. We present them with on screen advice about selecting and composing memorable passwords. If we succeed in this then why do we want them to throw away this carefully selected password a month later? The only argument I can see is that if a third party knows the password today then they will not in (say) 30 days time. However, if an end user has chosen to share a password with a third party then they will probably want to continue to share it into the future – if it has to change then inevitably it will be written down and displayed in a prominent location.

Too much of modern system security seems to be tailor made to annoy and frustrate those who should be “protected” by the security and yet frequently fails to inconvenience those who would invade our systems for nefarious purposes.

As an aside – why do most systems that require frequent password changes start nagging users days before a given password expires? I have one customer whose systems start nagging from 14 days ahead – every time I log-in. I currently have 6 active username/password combinations for different parts of their systems – plus a bunch of secure FTP logins fortunately stored in the relevant applications. Now explain to me what advantage I get from changing any password ahead of time unless it is just to stop the f****** annoying dialogues asking me if I want to.

Radar Alert

I was reading Dave Thomas’ post on The RADAR Architecture and was just getting ready to start disagreeing with him when I read on and started thinking. I am not sure that I would describe Ajax technology as “applying lipstick to a pig” (I said I started by disagreeing with him) and I have so far found it impossible to get excited about the (so called) REST issues but he has an interesting idea that just struck a chord.

Dave’s idea (put simply) is to put a proxy server into place on the client PC (or local network) to handle all of the “clever” interaction with the browser thus leaving the interaction with the web server as the standard GET/PUT over HTML or supporting REST but only via the local application proxy server. As this all dovetails rather neatly into where I think the future of application development is anyway I decided to explore the idea and see if a straightforward and practical technical implementation would be possible using current tools. Dave himself asks the same question in his post. I will be running some experiments but I suspect that the only obvious problem is that the additional “layer” of a proxy server is going to be operating system specific although it could be made to work well on a network of mixed platforms.

Co-incidentally we are at the “feet up on desk staring into space” stage of working out how we would go about a particular project. The essence is that we want to offer a solution to a complex business process and are minded to construct it on a web server foundation. We are also aware that our prospective customers are likely to be concerned about the availability of such an application and would be mistrustful of their internet connections and (possibly) the availability of our web servers. This is an issue facing all ISVs looking to promote new applications on a web based platform. We want the simplicity of managing a web server based application but also want to offer what is effectively “off-line” functionality, at least until the market matures. The RADAR approach might be the basis of a solution and would have the added benefit of allowing compute intensive stage of the application to be run selectively at the client or server end with the potential to share such tasks between the processes.

I trust that my experiments will be productive enough for me to post on at least the basics of such an approach.

2007-03-26T05:07:00.000-07:00

Floating Point Arithmetic – a Snip?

Rod Stephen’s VB Helper newsletter this week contained a useful link for VB .NET 2005 developers. A Code Snippet Editor that makes editing and developing code snippets a breeze. The newsletter also had a very nice section (by Bruce Deam I think) on the significant errors that can be introduced into floating point arithmetic by subtraction. Those of us with long working lives have battled with floating point arithmetic errors many times – and developed some techniques for dealing with them but the problems are not going away anytime soon.

Lesson Cards

Mathew Gertz has continued his excellent series of articles on the Visual Basic team blog on the subject of creating a new card game using Visual Basic. I’ve mentioned it before but it deserves a wider audience. The articles form a great tutorial with sections of interest to everyone – even if you have no intention of writing a card game. The articles introduce many lesser used aspects of Visual basic development as well as challenging popular viewpoints on things such as “modal forms”. Matt emailed me to say that the source code with be released with the final part of the series – I have a particular interest as the version of the game I learned and played in my youth is not one of the many versions his code is supporting. He gave me some tips on changing the code to meet my needs and I am looking forward to tweaking the AI engine to handle my “local” variant.

2007-03-23T03:06:00.000-07:00

Falling by the wayside

Rumour has it that Appforge who supplied programming tools for the smart phone and PDA developer’s market have closed their doors. This is a tough market place to thrive in with mobile device sales never fulfilling their promise and with even fewer software houses or in house development teams focusing on this platform for new applications.

I remember reviewing the first Appforge product – a VB6 “add in” that allowed you to use a subset of the VB Classic language to develop for Palm OS based platforms. I wrote the review as a “head to head” in 2001 for the UK magazine “PC Pro” between Appforge targeting the Palm OS and developing for the Windows CE platform using the free EMBedded Visual tools provided by Microsoft.- in this case targeting the new Compaq iPaQ that was wowing the PDA market place at the time. Both products did quite well and produced workmanlike applications on their respective devices. I recall that when I reached the “conclusions” in my review I was sounding warnings about the potential impact of the .NET platform on development tools, mobile operating systems and runtimes. I did not foresee the rapid “loss” of the Pam OS although my review noted the dominance of the Symbian OS in the phone market. Reviewing my article reminded me that Sony was briefly a “player” in the consumer end of this market.

I later wrote a review of the developer products for the same platforms produced by NS Basic who happily have continued to work with their loyal developer fan base and still offer an alternative toolset for developers targeting the “mobile platform” market place.

Procedural Robots or Functional Turtles?

My younger son brought some homework back from school the other day (French schools like to keep kids (and parents) busy late into the evening from the age of 6 onwards). The assignment was represented as an exercise in symmetry although in fact it was a vector “transformation”. I immediately began thinking that we could replicate the exercise in something like Logo and start the process of learning the venerable craft of computer programming at the same time. I took a look around for a suitable Logo implementation but at the same time stumbled upon Robo. Both languages are designed as first or educational languages and both take a similar tack with a conceptual turtle (or robot for Robo) being moved around a two dimensional space. While Logo is a functional programming language with it’s roots in Lisp, Robo is a procedural programming language with a C like syntax and using English language keywords.

Robo has the more immediate visual appeal matched by an ability to interact with a virtual environment so I suspect it will be a winner with my son – I will just have to give him a side course in Scheme over the summer break.

2007-03-20T04:12:00.000-07:00

It’s all Greek to me

Not another posting on that new Hollywood film although you may have caught mention of Adobe’s Alpha release of Apollo. Apollo is a development platform incorporating HTML, JavaScript, Flash, Ajax and Adobe Flex designed to build desktop applications using browser technologies that can run outside of an Internet Browser.

This is yet another strong challenge to the Microsoft dominance of the desktop as Apollo is intended to run “cross platform”. In fact it will probably worry Apple as much as the guys in Redmond. This is a particularly strong challenge to Microsoft’s WPF (Windows Presentation Foundation) which is currently rather poorly supported by Visual Studio and which has cross platform functionality still sitting well down the “to do list”. Adobe will also be challenging Microsoft in the developer’s tools arena with the introduction of Creative Suite 3 – probably later this year.

Kevin Lynch has a blog entry that covers the launch from a technical viewpoint and you can download the tools from the Adobe Labs website.

Note the building blocks – HTML for layout mark up, the (NBL of course) JavaScript, Ajax for asynchronous links to web services, Flash for graphics (in fact it also provides the JavaScript virtual machine) and the development framework Flex. And there was me thinking that I did not need to learn Flash. Oh well – it keeps you young – a good thought for the day as it is my birthday.

Down to Earth Architects

I enjoyed Jason Kester’s post a couple of weeks back on working on the road – indeed deliberately taking teams away to pleasant locations for intensive periods of development; an idea which I was also alluding to yesterday. If you are a fan of the Daily WTF now renamed “Worse Than Failure” you will surely love Jason’s latest post “Example Code != Production Code”. His points there are very important – it is easy for a developer or development team to think that an introductory book to (say) Visual Basic is also an education in the .NET Framework, or worse, a primer on designing anything more than some simple programs using a given framework. The development of an application can and should demand a level of understanding that goes way beyond what is normally available from the standard IT bookshelf. If Visual Basic is your bag then you can do a whole lot worse than read Rod Stephen’s book “Visual Basic 2005 Design and Development” that takes developers a big step towards a proper understanding of the task of building .NET Framework applications. I reviewed it here when my copy first arrived but further reading of some of the chapters has strengthened my view that this is a book that is a “must read” for everyone new to .NET or anyone who wants to move up from just dabbling to becoming an effective developer. The problem for authors writing a book like this one is connecting with the target readership – all of those developers in company IT departments everywhere – too few of them read technical blogs, visit sites like slash dot or even keep up to date with the Amazon book list. So if you know someone who should read this book – buy them a copy as a gift.

Amazon.com link
Amazon.co.uk link

2007-03-19T06:06:00.000-07:00

More Vision in Basic

I was a bit slow picking up the details on some of the new features scheduled for Visual Basic 9 (currently in the “Orcas” CTP). The VB team always work particularly hard to imbed new features firmly into the IDE so it is usually C# that gets the starring role when it comes to announcing new features but (as usual) the VB team are right behind and deliver a better integrated language element. The VB team are working hard to re-earn the RAD title so richly earned by the VB Classic versions that preceded .NET.

I particularly liked the new Extension methods – you can check out some detail at the MSDN “Blogs” site. Now this great new feature has something of the feel of the way that you can extend the JavaScript built in objects using “prototyping”. I also think it is interesting that XML will become a “built in” data type for VB. I trust that is implemented in a way that works for large XML data-sets but I am content to wait and see how this will work in practice. XML has the potential to match the utility of the VB Classic variant array for passing complex and asymmetrical data between methods and classes.

<aside>
The MSDN link above threw up some links to a series of posts by Matt Gertz of the VB development team on coding a Euchre game. As a young man living in North Kent I developed a great fondness for this clever pub card game and I look forward to following this series and maybe having a chance to play the game again – even if it is with my PC rather than a table of companions and a pint of good English ale on the side. You can find all of the posts at http://blogs.msdn.com/vbteam/
</aside>

Language Wars

I have always held myself somewhat aloof from the flame wars supporting or criticising development language X or Y – juvenile postings nearly always exposing ignorance of the fundamentals of computer science even where they showed a worryingly details grasp of some of the minor syntactical peculiarities of any given favourite or foe. OK, I will admit to favouring a subset of the available development tools (Visual Basic classic, Visual Basic .NET, C# and JavaScript in particular) but they are not above criticism and I can see the merits an almost any other programming language around – well, with the possible exception of Mumps – but that’s another story.

Now it’s confession time. When I saw a post that said that Microsoft had decided that FoxPro had reached the end of it’s days I was reminded that I have always looked down on Xbase applications and have always harboured the feeling that Xbase applications somehow “short changed” their users. Now I must admit that I have not knowingly seen anything developed in any of the Xbase clones in recent years but that inbuilt “sneer” persists. I suppose it all stems from the fact that the early tools of this nature restricted what you could do more than more general programming languages of the same era. Plus I have always had a distrust of tools that do really simple things just that bit more easily but then demand a convoluted expertise to prise the slightly more complex requirements out of the functionality of the toolset. All of the (so called) 4GLs had this problem – usually forcing the developer to compromise the underlying database design in order to achieve what should have been pretty straightforward requirements in the data presentation layer.

Working on the Move

I rather liked the idea of the lifestyle of those the San Francisco Chronicle dubbed the “neo-nomads” although I could not imagine swapping my calm and quiet workspace (in a barn opposite my house) for the noise and distractions of a coffee house. However I did like the notion that you could travel the world and continue working on your customer’s projects – I suppose the infrastructure is now falling into place so we will see more of this in the future. Certainly, I have taken my trusty lap-top (you can’t call it a notebook with it’s large screen and 7kg+ weight) with me when I have been travelling for leisure purposes with my family and continued to provide support services for selected customers. I have not yet experimented with working full time away from my home base but I am sure it is just a matter of time or opportunity. Time zones could represent a bit of a problem although I have found that the Eastern seaboard of the USA is OK being just 5 hours behind the UK – that worked quite well with no-one really noticing that my responses “lagged” a bit.

2007-03-13T07:37:00.000-07:00

Office Productivity or “Let’s just get the job done”

Richard Hillesley’s blog entry “A pig through the python” caught some attention this week – if you have not read it then the suggestion is that tools such as Microsoft’s (already much maligned) PowerPoint actually reduce productivity and organisations that have rejected the charms of bullet points and clip art have seen enhanced productivity as a result.

We are all probably aware that the vast majority of the documentation developers are required to generate is designed to manage us rather than the task in hand – it is inherently unproductive but gives business managers the illusion of some measurement of productivity. This bizarre conundrum is interesting – we simulate productivity by not being productive and “wasting” our time on tasks that are not directly project related.

<aside>
I recalled seeing a Scott Adams’ Dilbert carton that was very pertinent to this and then immediately thought that what I needed was some way of searching through the Dilbert back catalogue using a word search on the text. No such tool appeared to exist but I accidentally used the “search” facility on the United Media web site that used something called “MySearch” from myway.com – life is too short to wait for “also ran” web search tools – believe me, the minute and a half I waited for a response had me so amazed I did not just close the window and start again with Google. The best laugh of all was that when the search page finally resolved itself in my browser it proudly boasted that the search had taken a mere 0.46 seconds. On some other planet it probably did but here on planet earth it took a very long time indeed.
</aside>

I was sent some documentation for a project I am working on the other day and invited to contribute to the “Outstanding Issues” section. I quickly realised that it would be quicker to actually resolve all of my outstanding issues than it would be to document them and see them subsequently circulated to all “interested” parties. So that is just what I did in just a few minutes on the phone. Of course, those managing the project at the “Business “ level will be the poorer for my precipitate action and will never know about those issues or indeed how they were resolved – actually nine times out of ten that is a good thing – too many managers need to interfere in due process in order to feel they have made a contribution.

When I lived the life of a “corporate” developer I quickly learned that while meetings had their place most project fact finding was best accomplished outside of any meetings. If you rang someone up to arrange a meeting on a given subject then they would find you a “slot” in a fortnight or so. If you grabbed a cup of their favourite mix from the coffee machine and proffered this gift before sitting on the corner of their desk then you got the information or agreements that you needed then and there – project advanced – maximised productivity. Again this was work “off the radar” as far as the project management was concerned but it was productive – and more important it was “done”.

So if real programmers don’t use flowcharts then perhaps they also don’t use agendas or minutes or project documentation.

2007-03-09T07:35:00.000-08:00

Is Vista ME 2?

I have a friend whose first PC was nearing the end of it’s useful life. He made the decision to buy a new machine last autumn but also decided to hold on until Microsoft’s Vista became available in the UK this February. I gave him a spec for an acceptable “Home Premium” machine and once the Vista machines were shipping he started to shop around. He came across quite a good bargain from a major specialist retailer – the spec was just about in bounds so I persuaded him to invest some of his “savings” in a copy of MS Office 2007 for Home/Student users – now that product is sensibly marketed and priced.

My friend is very happy with his purchase. He quickly got the machine up and running on the Internet (well done Microsoft as I had not been looking forward to talking him though a sequence of screens I had never seen to help him out if he got stuck). He was able to transfer his photographs and documents from his old machine using a USB “thumb” drive and all is well. Why the post? Well – my friends old machine ran Windows ME which was a buggy design mess and the last of it’s development line. I am just a bit worried that he has brought the Windows ME of 2007. Nobody should have to suffer that life experience twice in a row should they?

IE Version hell

A customer called with a problem yesterday. They have added 50 or so new retail outlets following a takeover and some of these outlets were having problems with a particular web based application supplied by us. The page registering problems makes extensive use of AJAX technologies and all of the outlets having a problem were running MS Internet Explorer version 5. As far as I could recall, all version 5.n copies of IE for Windows had support for the XMLHTTP ActiveX Object so that did not sound like the obvious answer. Now I no longer have a copy of IE 5 running anywhere – even on my old Windows 2000 box, so it was difficult to simulate and diagnose any potential local set-up problems.

Ever since version 3.0 of Internet explorer when Microsoft (unwisely) pushed the browser deep into the OS it has been difficult , if not impossible, to maintain multiple versions of IE on a given machine. Indeed IE has been so deeply tied into the OS that the OS would break if you succeeded in installing a version of IE that preceded the “base” for a given Windows version. This clearly represents a problem for web site developers looking to support the range of Microsoft browsers found in the wild alongside the myriad varieties on other browsers on Windows, Linux and Apple boxes.

I wondered if there was a solution. In fact Joe Maddalone published a technique in 2003 that allowed for multiple copies of IE to be run on a given machine (Windows 2000/ XP) in “stand alone” mode. He built the idea from the “developers preview” version of IE6 that was distributed in a “stand alone” form by Microsoft. The idea has been extended by Manfred Staudinger and you can download versions of Internet Explorer from the evolt.org archive. You can also download an installer for multiple copies of IE from Yousif Al Saif at tredsoft . This is great stuff and an invaluable resource for web site developers even if there are warnings that not all version run perfectly in all circumstances.

NBL Libraries

Is it me or is there currently an explosion of great NBL (JavaScript to it’s friends) libraries. There seems to be a new one every day. Not that I am complaining – it just goes to underline the fact that so many programmers are beginning to appreciate the subtleties and strengths of this great language.

Today I spotted http://ajax3d.sourceforge.net/
and http://script.aculo.us/
also http://www.hunlock.com/blogs/Ten_Javascript_Tools_Everyone_Should_Have
The excellent Prototype as been around for a while and Yahoo have a library beloved of many. We are talking about some excellent infrastructure here and anyone who follows my blog will know that I am a fan and expect this to be the way of the future BUT – without standards we are always going to be scrabbling to develop and test to multiple browser versions – heavens even the introduction of standards compliant browsers will simply add another layer of complexity at each iteration.

2007-03-07T03:35:00.000-08:00

Real programmers don’t use flow charts

I enjoyed Kevin Barnes’ Code Craft posting titled “Real programmers don’t use pencils”. I came to it after he had “softened” his position a bit but it makes some telling points. Like all such “strong” statements it always gets me examining my own processes to see if I agree with the basic premise or not. So what do I write down – if anything? I looked around – there was certainly a fair pile of bits of paper on my desk and quite a few had scribbles and notes on them arising from various current development tasks. I was interested to discover that it was all UI stuff – sketches of parts of windows to get help organise the relationship between options and occasionally lists of prompts to help me get the wording aligned with the user’s terminology. Nothing that reflected code and when I thought about it I could not recall ever writing out a code block except in some very odd circumstances when I was unable to get anywhere near a computer.

I do not think that the issue is about speed (even the speed of computation) it is all about one’s ability to maintain all of the details of a given process in your head – if you can’t do that then you will not be able to write effective code. You can’t keep part of a process in your head and lodge the rest on paper or even typed into your favourite language IDE – well you can try but it’s not going to work at any level that I would recognise as actual programming. Kevin picks up this point when he gets to de-bugging. I am amazed to meet developers who, when faced by a bug, do not immediately know where in their code the problem must lie – to the nearest one or perhaps two functions or methods. The ability to “trap” aberrant (and by definition unexpected) behaviour in an executing code block is, I suppose, partly as a result of a certain programming style that does not allow errors to “leak” from one code block to another but then again you probably can’t write code that way if you can’t maintain an effective mental model of the whole program in the first place.

Back in the days when mainframe computers held sway over everything, trainee programmers and analysts were issued with a plastic stencil that included the basic elements of the standard IBM flowcharting symbols. I know because I started out as a trainee systems analyst in the days when early Apple and Commodore PET computers were making their first appearance in the UK. Ignoring the coming computing revolution we were taught the basics of programming by writing code (COBOL of course) onto coding sheets which was then transferred to punch cards by “key punch operators”. Your program might or might not be allocated an overnight run slot on any given night – the lucky ones were rewarded with their “core dumps” the following mornings. You will understand that the “elapsed time” for code development was pretty extensive but that equally the process of transferring your code to a machine readable format was also relatively expensive. Programmers were expected to sort out their program logic in advance of the “code, key punch, overnight run, morning debug” cycle by flowcharting the whole thing before committing any of it to code. In truth this flowcharting was probably some sort of displacement activity – it kept the programmers busy doing something – after all management would not have wanted to be paying these expensive guys just for the ten or fifteen minutes of actual productive work they managed each day.

Perhaps it is me but I quickly found that I could not “describe” to myself the process of a program in terms of a standard flow chart. OK, I have since used flow charts to document the interaction and flow of data between processes but I have never found the language of a flow chart capable of describing those processes themselves. This is probably why attempts to generate code from flow charts and other related attempts to find the “Silver Bullet” of programming in that arena have never come to much. I wonder if it was that early “flowchart” training that was part responsible for the terrible structure of early program code as it encouraged a linear thought process and those “dreaded” off page links would have translated into instance “goto” statements and code spaghetti.

So there you have it – real programmers do not use flow charts or pencils.

2007-03-05T03:09:00.000-08:00

More on the revolution

Only one post last week. This was partly because I am working on a lengthy piece that still needs a little ‘wrestling’ to come to it’s conclusions. Also partly because a project I am working on was going through some (very) last minute changes in specification and also because I got hit by a virus that reduced my capacity to near zero on Thursday and for a fair bit of Friday.

If anyone is following my thoughts on the “end of desktop computing” and the coming shift to online/offline web based systems then they might like to look at the following two links.

Mary Jo Foley takes a look at a start-up called Xcerion which has some hot shot ex-Microsofties as investors. Xcerion is planning to supplement their “me to” web based Office Apps with a storage layer designed to handle that online/offline divide. They are also looking for developers interested in building applications to work within their OS layer. This arena is going to get competitive. Google are doing their thing, Microsoft are working on it, the next major release of Firefox will include another approach and we have start-ups like Xcerion hoping to be the winners as this disruptive approach to delivering applications takes affect.. My only comment is that “It’s not all about Office Apps”. MS Office might be the number one shrink wrapped application in the world but it is not the only key to the future. The alternate key here is to attract the ISVs (Independent Software Vendors), get them to write the next version of their vertical market applications for your platform and then business will buy in.

What got me thinking though was an article by Peter Seebach on an IBM web site suggesting that it makes sense to develop applications intended for local use on a single PC as a Web application. Now I am not entirely convinced by the argument that building a web application is “easier” than (say) a Windows thick client application although I will accept that if the interface itself need only be very functional then the web approach might have an edge. Then there is the question of your preferred language and platform – if you want to write in (say) Python and you want flexibility for your platform choice then the browser based approach could after all be a ‘slam dunk’.

One implication of Peter Seeebach’s suggestion is that the web server (say Apache) represents an alternate “Framework” to (say) .NET. Now that is an interesting idea. I will have to give some thought to this as a potential solution for ISVs reluctant to expose their proprietary algorithms to the world as .NET CLR code. With the browser representing the client process and taking on the large proportion of the code that is required by the UI the task of coding the truly proprietary code as a DLL could become manageable. My notional ISV could then deliver his application more readily across different platforms while retaining the security of compiled code.

2007-02-26T00:47:00.000-08:00

The solutions for Error Code -2147217884 Rowset does not support fetching backward

This refers to SQL Server 2000 and ADO .NET and is a short lesson on getting properly prepared.

“Rowset does not support fetching backward” is one of those “obvious” errors once you know what is causing the problem but as it seems to be a common one I will tell the story of how I found the solution.

I was running the first tests on a slightly modified routine that accessed records from an SQL Server 2000 database using ADO when I hit this error. Given that almost exactly the same code in the original routine runs without a problem I spent some time looking for errors in my code – but (on this occasion anyway) I did not find any.

Now in fact, the modified routine was running against an apparently identical table to the original sitting in a database that was supposed to be a good copy of the original.

The error occurred when I wanted to .MoveLast() on the Recordset to check the number of rows returned to support a progress bar providing feedback to the user.

A quick google returned a huge number of results for the error message and a great deal of debate over cursor types and lock types. Interestingly, as it turned out, many posts asking for help with this error indicated that the developer was working with a newly set-up database. Now I was setting the cursor type to Static and my Lock type to Pessimistic so in theory I was avoiding the obvious pit falls known to cause this problem. My Google search did not come up with the solution I needed although it would have been hard to find it in that volume or cries for help.

So I added some code to my routine to check on the cursor type and lock type that I was actually getting after the recordset had been created. The results (sort of) explained the error – the actual cursor type returned was “Forward Only” and (just as bad) the Lock type was “Read Only”. The only obvious difference I could see between the working and non-working routines was in the complexity of the SQL Select statement.

One routine got the correct Recordset attributes with an SQL statement like:

“Select * From STAFFRECS Where STATUS > 0 And STATUS < 4 Order By LOCATIONID, STAFFID”

but everything appeared to be going wrong in the new routine when the SQL was slightly extended like this:

“Select * From STAFFRECS Where STATUS > 0 And STATUS < 4 And (PAYTYPE = 'H' Or PAYTYPE = 'W') Order By LOCATIONID, STAFFID” – although that was not quite it.

Experimenting with a drastically simplified SQL statement of “Select * From STAFFRECS” returned a Dynamic Cursor type and the correct lock and should have given me the answer instantly

I'm a bit slow so I needed a bit of a think about how locks might be applied to selected records and how cursors actually work. That got me to the solution. I checked and the new table did not have any indexes set up – in particular there was no primary key even though the table had an “auto-increment” integer field ready and waiting to stand that duty. Adding the appropriate indexes solved the problem. This also explained why so many developers had hit the problem with new databases and copies and upgrades of pre-existing databases.

I started this post with the promise of a solution to this error so to cover the ground:

First check your Cursor Type – if you are not specifying a cursor type then do so – something like

MyRecordset.Open(SQLString, SQLConnection, ADODB.CursorTypeEnum.adOpenStatic, ADODB.LockTypeEnum.adLockReadOnly)

will work great for reports and listings – so make sure you are using the correct Cursor type for your purposes.

Check you are requesting and actually getting a cursor that can step backwards as well as forwards. It is simple to add something like:

Dim Dummy as Integer = MyRecordset.CursorType

as a code line and inspect it each time you step through the code = or to inspect that same attribute of the ADODB.Recordset using the “Watch” pane in the IDE.

If that is not your problem then check that your table’s indexes and (if necessary) specify a primary key. So that’s a reminder to the DBAs out there to set up indexes even in a scratch development database.

So no more “Rowset does not support fetching backward” errors in your applications please.

2007-02-23T00:51:00.000-08:00

An ultra low cost power adaptor for your Fujitsu Siemens L00X or any other PDA or phone that can be charged from a USB port.

FS charge a hefty £35 in the UK for their car power adaptor – just a 12 volt to 5 volt step and delivering power through the standard USB pins. On a visit to Woolworths I saw they had a super little travel kit for the Nintendo DS Lite that includes just such an adaptor – for just £3.99. Discard the cable and connect the adapter through the cable that came with the phone. Now what shall I do with the rest of the goodies that came in the same pack?

OK I doubt the ultra low cost USB power adapter includes any sort of sophisticated power surge protection so I would not have this plugged in while starting your car (can be a problem with some cars) but a little caution is a small price to pay in order to take advantage of this generic solution.

2007-02-23T00:23:00.000-08:00

Customer Service

Joel Spolsky wrote a great piece on Customer Service / Product support and plugged his company and FogBugz (probably deservedly) in the process but hey that’s what bloggs are for. Go read the piece if you have not already. Seth Godin chimed in with the sensible suggestion that Customer Service is broken and proposed how to fix it by resetting our unrealistic expectations and delivering a “next day” guaranteed fix. A nice idea, although scary as most of us know that a problem not fixed “now” is almost universally forgotten by all but a tiny minority of support services by tomorrow.

I think that the key here really is cost. Although cost has distorted customer service practices to the point where they actually can’t function at a more effective level. Back in the old days when a mini-computer from DEC cost tens, indeed hundreds, of thousands of pounds (the days of expensive hardware are coming back by the way but that’s another post) you were also obliged to pay some pretty steep maintenance charges as well. But, when you called in a fault, an engineer arrived within four hours – even if you phoned at five minutes to five in the afternoon. The support service had already made an initial diagnosis of the problem (interacting remotely with the hardware if necessary – or indeed possible) and the spare parts required were routed to your site independently of the engineer. The engineer then stayed on site until the problem was fixed – yes that did mean some pretty late nights sometimes. We also got regular PM (preventative maintenance) visits designed to replace parts that had failed on other similar machines of a similar age before they failed at our site. The service cost a lot but it delivered a lot - and you could negotiate (and pay for) better deals than the basic one I just described.

More recently, I have a couple of times worked with consulting clients eager to secure a hardware and networking support contract that matched their operational requirements. The response from just about every company in that market place has been blank incomprehension – even when we have offered to match my client’s support requirements with cash. They can all do “Ring between 9 and 5 and someone will wander along the next day and not bring any parts to fix it” and will happily compete with each other on price to offer that single level of so called support. See what I mean – low price – low price mindset.

So sadly chastened by such failed negotiations most businesses respond to the reality of the marketplace by building in hardware redundancy. That way they can build in some “time” to allow the tardy support functions to manage any given repair. Some go for two computer rooms, certainly duplicated servers with “fail over” – plus RAID drives everywhere. They spend capital where they should have spent revenue and once that capital has been spent they then get to enjoy at least twice the number of component failures than they experienced before – twice as much kit means twice as many problems after all. Twice as many problems demands increased management time and expense; tracking and escalating issues that should/could have been dealt with at a “Customer Service” level. Software upgrades grow from being problematic to become a scheduling (and execution) nightmare. More low cost (and therefore relatively unskilled) maintenance work for the low cost and under achieving support suppliers though.

Now I’m not saying it is easy to find customers willing to pay for effective service – its not. I know that well enough from the software marketplace. Most small businesses in particular only like to pay money for something tangible and hate paying for service or support they might not take up. This is the other face of the problem – significantly more effective service and support can only be managed by a supplier set up to achieve that level of support – and you have to have some idea where your customers are going to be coming from to make that strategic step. Not an easy choice.

RAID drives

Just a quick aside on RAID drives. Some recent reviews (including one by Google) of the life, times and more importantly deaths of many thousands of hard drives has an important lesson for us all. The myth is that drives have a high initial failure rate followed by a long period of stability with increased failures seen again towards the end of the anticipated lifetime. However the truth is simple - hard drive failures simply increase exponentially over time – with failures likely to occur in about one third the time specified at MTBF by the manufacturer. This should impact your RAID strategy. RAID arrays have for a long time been considered an effective strategy for dealing with disk failures as it was thought that it was unlikely that a second drive in any given array would fail before a first failing drive was replaced and operational again. The research on actual drives would indicate that the likelihood of a second failure occurring during the process of re-synchronising a new drive is high enough to be taken into account – probably about four times as likely as anyone had previously supposed. Sensible system managers should continue to make strategic use of RAID drives but should perhaps seek to limit the capacity of the drives in any given array. Smaller drives take less time to resynchronise and thus leave smaller time slots for a second and disastrous failure to occur.

The analyses also indicate that it would be advisable to keep a record of the type and installation dates for each disk in a RAID array in order to predict arrays likely to have increasing failure rates – and apply some preventative maintenance.

Keep doing the backups guys.

2007-02-22T02:53:00.000-08:00

Google Apps

Even faster than I anticipated we see Google positioning itself for a full assault on the MS Office monopoly. With big companies like Proctor & Gamble already signed up and others like General Electric eager to transfer the management and support of key business tools like email, document production and spreadsheets to a world class partner this is the true start of the shift to web based applications.

2007-02-22T02:49:00.000-08:00

Technology makes us dumber

I had been thinking about a post on some key attributes that every software tool should have when I came across this post by Kathy Sierra that covers a lot of the ground.

I think the theme of “literacy” from a few days ago is much wider in scope than I had previously considered. So I will just leave my own headlines for the moment ad think about the topic some more.

Keep the UI as simple as possible without dumbing down
Leave the user in charge
Ensure the user understands the process

2007-02-22T01:00:00.000-08:00

Not another text editor?

I am prepared to admit that most of my text editing needs can normally be met by Notepad. I am also aware that super geeks would probably sneer at anyone not using Emacs and writing macros and extensions daily in Emacs Lisp. On one of my machines I have an old VB Classic open source program called “Developer’s Pad” that serves very well for certain sorts of code development – just the thing for knocking up some VMS Basic code. Sadly, Developer’s Pad would now appear to be “abandonware” although I think I have a copy of the source somewhere so I suppose I could revise and extend it to meet new needs. Now what I want is a Windows based programmable text editor and there are lots of them. You could spend a ridiculous amount of time just evaluating the front runners – which is why I suspect they continue to accumulate – it could well be quicker to write a new one than pick the best of those available already.

Searching at Sourceforge for “Text Editor” got me 5366 results! Restricting a Google search to “Programmers Editor” got equally stratospheric results.

My requirements:

Some ability to support the syntax of different languages (with some pre-defined because I am as lazy as the next guy)
Support for “scripting”/extending the functionality
Runs on Windows – GUI but with keyboard shortcuts (I like to work both ways).

Any suggestions?

Visual Basic on Linux and the Apple Mac

The mono project have announced that they have completed the development of support for Visual Basic .NET in version 1.2.3 and that this version almost completes support for the .NET Framework version 2.0.

The mono project has thus enabled the porting of unchanged VB.NET code to all of the platforms supported by that project.

This is potentially very big news indeed. I have long argued that any OS without it’s “Visual Basic” was doomed to languish in the backwaters – now VB is available across multiple platforms. This opens up wider markets for software houses and development teams in every sphere.

Oh, and the new Mono Visual Basic compiler was written in Visual Basic! Good style guys.

The .NET Framework 3.5 and “Orcas”

The next version of Visual Studio codenamed Orcas will support multiple version of the .NET Framework (specifically version 2.0, 3.0 and 3.5). This is great as that means that developers will be able to switch to the new IDE while continuing to be able to support systems running against earlier versions of the framework.

[I love the way Microsoft like to push the version numbers ever upwards but sometimes strangely – after all the .NET Framework 3.0 is really 2.1 although it might be that 3.5 is nearly a whole step forwards.]

Other features include intellisense for JavaScript and what looks like truly effective support for CSS (Cascading Style Sheets). The JavaScript intellisense will also be able to make use of commenting within your own client side JavaScript objects and methods to offer effective support elsewhere in your code – just as it does with classes and objects written in (say) C# or VB.

Which just leaves one big thing on my “wish list” - full server side support for JScript in ASP.NET – IronPython would be good too.

2007-02-21T00:14:00.000-08:00

A taste of NBL honey

or how to write a Windows application using JScript (JavaScript) and .NET

The closest you can get to Steve Yegge’s NBL (Next Big Language) right now is Microsoft’s JScript .NET. JScript has already implemented many of the features to be found in the ECMAScript version 4 specification. As the more advanced features of JScript will not yet run within an Internet Browser I decided to try out a simple program running from the next best thing – a Windows form. More than that, this small code sample constructs the form using the .NET Framework. The simplicity and brevity of the code rather demonstrates the advantages of the .NET Framework over the old Windows API.

The real purpose of this little code module is to demonstrate the mixing of strongly typed variables with native JavaScript variables. So what we have here is a program that calculates factorials (well it’s a step up from “Hello World” so give me a break) wrapped up in a simple but workmanlike (not beautiful) Windows interface – all in 60 odd lines of code.

I wrote the following code using Notepad and saved it to a file called fact.js

import System;
import System.Drawing;
import System.Windows.Forms;

public class MyForm extends Form {
//declare the form controls
private var tbInput:System.Windows.Forms.TextBox;
private var lblResult:System.Windows.Forms.Label;
private var btnCalc:System.Windows.Forms.Button;
private var btnExit:System.Windows.Forms.Button;

public function MyForm(){
InitializeComponent();
}
private function InitializeComponent() : void {
//lay the form out
this.Text = "Calculate Factorials";
this.Height = 160;
this.Width = 260;
this.WindowState = FormWindowState.Normal;
this.StartPosition = FormStartPosition.CenterScreen;

this.tbInput = new TextBox();
this.tbInput.Text = "5"; // show an initial value
this.tbInput.Top = 10;
this.tbInput.Left = 10;
this.tbInput.Width = 40;
this.tbInput.Anchor = AnchorStyles.Top | AnchorStyles.Left;

this.lblResult = new Label();
this.lblResult.Text = "0"; // show an initial value
this.lblResult.Top = 70;
this.lblResult.Left = 10;
this.lblResult.Width = 150;
this.lblResult.Anchor = AnchorStyles.Top | AnchorStyles.Left;

this.btnCalc = new Button();
this.btnCalc.Text = "Calculate";
this.btnCalc.Top = 40;
this.btnCalc.Left = 10;
this.btnCalc.Width = 80;
this.btnCalc.Anchor = AnchorStyles.Top | AnchorStyles.Left;
this.btnCalc.add_Click(this.btnCalc_Click); //attach a handler to the Click event

this.btnExit = new Button();
this.btnExit.Text = "&Exit";
this.btnExit.Top = 100;
this.btnExit.Left = 200;
this.btnExit.Width = 40;
this.btnExit.Anchor = AnchorStyles.Top | AnchorStyles.Left;
this.btnExit.add_Click(this.btnExit_Click);

this.Controls.Add(this.tbInput);
this.Controls.Add(this.lblResult);
this.Controls.Add(this.btnCalc);
this.Controls.Add(this.btnExit);

}
public STAThreadAttribute() static function Main(Args:String[]) : void {
Application.Run(new MyForm());
}
// control event methods
private function btnExit_Click(sender:Object, e:EventArgs) : void {
this.Close();
}
private function btnCalc_Click(sender:Object, e:EventArgs) : void {
var testVal=parseInt(this.tbInput.Text, 10);
if(isNaN(testVal)){
this.lblResult.Text = "Error";
} else {
this.lblResult.Text = calcFact(Math.abs(testVal)).ToString();
}
}
// the function that calculates factorials
private function calcFact(inVal : int){
if (inVal==0 || inVal == 1){
return 1;
} else {
return (inVal * calcFact(inVal - 1));
}
}

}

MyForm.Main(Environment.GetCommandLineArgs());

(Sorry about the indentation but after a fight with Blogger I gave up)

The code was compiled with the following command line - check your path to the JScript compiler jsc.exe:

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\jsc.exe /t:winexe /r:System.dll;System.Drawing.dll;System.Windows.Forms.dll;Accessibility.dll fact.js

The variable used in the calcFact() function is defined as an integer (32 bit) and so the compiler can construct an efficient version of this recursive function while the variable used in the button event handler btnCalc_Click() is a traditional JavaScript variable that makes it easy to parse and validate the user’s value before passing it on to the calculation function itself. A perfect combination in my book.

I suppose this little demo could have gone a step further but there again it does represent a foundation for experiment – so copy the code, tweak away at the .Net bits if you are familiar with the relevant properties or the JavaScript code if that is more your thing.

Links

The Mozilla Tamarin project is working on an open source implementation of the ECMAScript 4th edition language specification (yup that’s the NBL) for the Firefox web browser.

You can get a list of the ECMAScript 4 features already supported by JScript from msdn and it looks like Microsoft are committed to supporting the whole standard – although knowing them they will probably also continue to support the odd “private” extension. Full JScript support within Visual Studio is embarrassingly absent although I think that this is likely to change – although that will leave the complication of separating that which runs in the browser as part of ASP.NET applications and the fuller spec that needs the support on the .NET framework Perhaps they need a new name – I know – why not call the browser language JavaScript? Or I suppose IronScript for the full shebang?

2007-02-20T02:28:00.000-08:00

Windows Mobile 5

This starts as just a moan so you can skip to the full rant if you want.

Navigation Software

In my review of the Fujitsu Siemens L00X smart-phone I mentioned that my existing Dell navigation software (which was pretty good overall by the way) could not cope with the restricted screen area available and so I had to cast around for a replacement that would run on a phone.

The AA Navigator looked like a candidate but it was not perfectly clear from the web site that the software was adapted to a 240 by 240n pixel screen. I therefore used the only available contact facility at their web site to pose the question. I did not get a reply so must assume that the answer was probably no and that no-one could be bothered to say so.

I turned to the “Pocket Navigator 6” software from Directions Ltd and this explicitly mentions support for my screen size on the web page. Thus I ordered the package plus a 2gig SD card from Amazon France to hold the maps. Note – the L00x does not support some types of SD card. The software arrived promptly by international post so I started the installation. On the installation start-up the user has to guess they are being prompted for a “security code” and also guess where they might find it. After that things improved a bit. I installed the software on the L00X but elected to install my selected maps directly on the SD card plugged into a port on my Notebook PC. I selected the UK, France, Belgium, Nederland and Germany for an initial map set and started the process off – wow did it take a long time. I lost track of time as the CD ROM hummed into life every minute or so and the progress bar edged forward in what I took to be microns – it certainly did not inch. And then it all ground to a halt – the web site had lied about the SD card space required. The site claimed 2.5 gigs for the whole of Europe but had filled a 2gig SD part way through France and well before getting to Germany. OK we know that 2gig SD cards don’t really hold two gigabytes (aprox. 1.8 in fact) but this was way out. I settled for France and the UK as my initial map database.

Once installed I ran the software on the phone – nice touch – it wanted the product key again – I just love that. Let us hope that my next system back-up stores this away OK – I suppose the only way to check will be to do a backup and then re-set the L00X to “factory settings” and try a restore. What I don’t need is m navigation software to refuse to work just because I had an OS crash.

The good news is that Pocket Navigator 6 works on the small screen and that “it knows where I live”. The Lat/Long came up reasonably quickly (good GPS) and the map showed the nearest road that has a number but not the lane that is adjacent (well fairly) to my house which indicates a lower “definition” than the Dell software I have used previously which is worrying as “back roads” are a normal mode of travel here in rural France. The operating menus are presented in groups of icons that make an effective job of managing the screen area but there are a lot of them and the labelling leaves something to be desired so this necessitates a good read of the CD based “manual”. Despite the nice icons this has the feel of a programmer designed interface – I should know I’m a programmer myself. By that I mean the menus are clustered by function and not by task – good job this was not my first experience of navigation software.

Practical problem – left to itself, the L00X switches itself off before the GPS has a chance to get a location fix. Hmn… now how did I get the device “mode” list displayed? However none of the available modes seem to apply – I want the box to “stay on” until I have finished using the navigation software or the GPS – looks like I have to explicitly set the power mode to stop it turning off and (of course) remember to re-set it when I have finished. So a faintly bizarre missing option for a GPS enabled device. Plus I don’t quite understand why the phone stays functional when it is playing an MP3 track but shuts down if it is waiting for the GPS to get it’s location – I suppose that process is “off chip” but that is the nature of these devices is it not? More than one hardware “package” inside the box. Sounds like another huge gap in the provisions within the OS.

The Rant

Did I mention how much I hated Windows Mobile 5? I am sure I must have – the UI is so fundamentally flawed that I completely fail to understand how it can have survived in the current form from it’s first incarnation – although the UI might partly explain the relative market failure of all PDA type devices. Don’t get me wrong – there are lots of clever applications and facilities; it is just that the whole is somehow massively less than the sum of the parts. That clearly indicates that it is the OS that is to blame.

My Pocket L00X has 64 megabytes of RAM, 128 megabytes of ROM plus whatever I stick into the SD card slot for it to play with. My first real Windows development machine used to run Windows NT version 3.1 (later 3.5) – yes I know it was just me and no one else running that OS. The NT machine was fitted with 16 megabytes of RAM and most people who knew that were gob smacked – why would you need that much RAM? Now NT 3.1 was a stable and secure 32bit operating system and that ran comfortably in 16 megabytes – Windows Mobile 5 has just about zero security functionality, a GUI that reminds me of those old CPM and DOS Shells that predated The Apple Mac and MS Windows, and it collapses at the slightest provocation. Some might argue that I am making an unfair memory comparison here as Windows NT had access to a hard drive for memory paging – well allowing for arguments about the size of the available swap files on early NT systems I would like to point out that it is not beyond the wit of man to devise a virtual memory system that could operate within a fixed memory allocation – but that rather brings me to the next point.

Most programs (read all the Microsoft ones) running under WM 5 have no direct mechanism to shut them down. This is mostly because of the crappy windowing system (next paragraph) and is, I suppose, intended to make provision for multiple applications to execute simultaneously. So why no choice between “minimise” (or hide in this case) and “stop”? If you want to stop a program you have to pull down the main menu, select “settings” and then the “system tab and then click the “memory” icon and then the “running programs” tab and then select the program from the scrollable list and then click the “stop” button. Simple! The MS Media Player is a good example – it has a useful menu of options for managing it’s options but they do not include the option to “shut this damn program down so it’s not still running for days and days until I next want to listen to some music or a pod-cast”. Strangely missing. There is also no provision (other than the 5 or 6 clicks listed above) to see a list of running programs so that you can navigate directly back to it – you are supposed to carry out the same process as you used to start the program in the first place which feels like flawed to me. There is an iconic representation of your most recently run programs but that is not an effective substitute for the desktop “system tray” and proper control over the running executables.

The Windows Mobile OS only presents one window at a time – even on a VGA compatible screen – no overlaps. The OS is so fundamentally truncated that it can’t actually support overlapping windows. You can check this out if you have an appropriate development tool by creating a simple form with a couple of controls that overlap – if you click the overlapped area then both controls get the click. You might be tempted to think that this was working in a similar way to the “event cascade” that occurs within the DOM in your Internet Browser – but it is not. It’s just broken. The programmer is not choosing where to respond to an event in a hierarchical sequence – the OS just generates a “click event” for every rectangular object under the click. If the window management is as feeble as that it seems hard to know how it will ever be fixed.

I could go on but I think you get the picture. Definitely time to invest some effort into the mobile platform web browsers and see if we can’t move the processes off this platform and onto an Internet based server.

I know genuine intelligent hard working people who are trying to build a business creating functionality meeting real end user needs on the back of this great steaming pile or ordure that we know as the Windows Mobile OS. I feel sorry for then and their struggle to make it usable and effective in the hands of normal members of the public but they have my respect. Hats Off – as they say.

2007-02-19T03:03:00.000-08:00

One for the team leaders

I spent some time over the weekend dipping into Rod Stephens’ latest book “Visual Basic 2005 Design and Development” one of the “expert one-on-one” series from Wrox. This is an excellent book and has an interesting target audience. This book is not designed to provide tuition in the finer points of VB.NET but would be a great resource for software houses and programming teams considering the switch from VB Classic (or other development tools of the same generation) to VB.NET and the .NET framework. It would also be a great resource for anyone new to the task of leading a programming team using VB.NET. The book covers aspects of managing and executing development projects that are rarely mentioned at all in technical books and even more rarely covered in such an authoritative and entertaining style.

I have been a fan of Rod Stephens’ books since I first discovered his “Visual Basic Graphics Programming” which was a book I wish I had had to hand when I was writing my first mapping tool using VB3 and Windows 3.1 back in the year dot – but unfortunately it had not yet been written. I keep buying his books because they always deliver a different insight and approach to the code development task even when you might expect to be already well versed in the subject matter. I am also hoping that continued support for Rod will persuade his publishers to let him write the next definitive graphics book – for GDI+ and the myriad formats and technologies from Microsoft and others. Unless of course my predictions of last week mean that SVG (scalable vector graphics) running in your browser will sweep away all desktop vector technologies as we move wholesale to web based applications.

This book starts with an honest review of the Visual Basic language itself – listing it’s disadvantages and then it’s advantages. It takes a look a project lifecycle methodologies and presents a chapter on “Agile” development. Now I am not convinced by many aspects of the agile promotion machine although I will admit that parts of this methodology can work well with the right team and the right project. There are sections on UI design and even design patterns – a great way of catching up on what the Java brigade waffle on about even if you (like me) suspect that patterns are an invention designed to fill in fatal flaws in a given language or environment. OK – I concede that even if that is true it does not invalidate their use – you just need to be aware that patterns are a “bug fix” and not a design philosophy. In a similar vein the book has sections on documentation, coding standards and testing that are well worth reviewing and putting into practice if there are flaws within your own processes.

Where the book really starts to deliver is when it covers some of the more technical aspects of the language and the .NET framework and explains why you might want to build these technologies into any given project. Thus the chapters on scripting, deployment and reflection will truly serve Rod’s target audience. Plus the short chapter on .NET printing, that solves so many real world project issues in this area, makes the price of the book worthwhile by itself.

So if you are in the position where you need a good review of Visual Basic and the .NET framework from the viewpoint of a lead programmer or programming manager then this book will deliver just what you need and give you a clear insight into the finer points of the technology. Yes the book includes lots of source code that will be eagerly reviewed by those with a technical bent but you can skip the details of any given implementation if you are just after the concepts. Recommended reading for any programming shop still “sitting on the fence” and cutting VB Classic code.

2007-02-16T03:47:00.000-08:00

It’s not the clicks stupid

Yesterday a Reddit contributor posted a link to a web site www.dontclick.it that runs a Macromedia Flash presentation of a web interface that does not require any mouse button clicks to navigate – mouse cursor motion and direction manages the navigation and the selection of options. At one level it is a remarkably effective demonstration of a different approach to web page and (by implication) desktop application control within a GUI (Graphical User Interface). I think the demonstration failed at the point that it got to the on screen representations of buttons and their operation. It was a bit weird that this development group still felt the need to include buttons within the interface – after all they are GUI artefacts that have only a single purpose – to be clicked. The site even showed some fairly unintuitive ways of clicking their buttons without clicking them if you follow me. I just could not see why they were there at all but if you need buttons why not click `em?

GUI buttons like GUI menu options exist to allow the user to decide when to trigger a UI “event” that is typically used to indicate that some data entry is complete or that the user wants to navigate to an alternate display, option or even application. In the case of menus we are used to being able to review the options before using the left button click (for Apple users this is a “no brainer” poor things) to make a definitive choice. Is the click a bad thing? Not necessarily, although I am excited by navigation techniques which are dynamic – when they are applied to a dynamic process. If you are reviewing and analysing a displayed report then you want a static or near static interface while that review progresses but if you are guiding a process towards a solution then dynamic navigation looks like a new and interesting area to explore.

A few days ago I mentioned that I was taking a specific look at Dasher and Dasher applies just such a dynamic UI to a dynamic process. This non click interface allows you to generate text – applying statistical techniques to present you with the most likely candidates for the next letter or group of letters (like the predictive text on your cell phone but way, way more than that) as you move continuously across the working surface. The technique is of particular interest wherever the operation of a normal keyboard is problematic – in the case of phones or PDAs or in the case of personal restrictions on mobility or dexterity. You can try it in a number of formats – on your PDA, within a web browser or on your desktop PC. The latter has the largest scale and may perhaps help you to pick up the technique more quickly but ten or fifteen minutes will see anyone quite adept and capable of creating text at a reasonable pace.

Jef Raskin described in his book “The Human Interface” the concept of a computer driver user interface that broke down the barriers we perceive between “applications” and that supported infinite undo and redo. Imagine all of the documents and data on your computer as part of a continuous process rather than representing discrete tasks and you will get a hint of the underlying ideas here. Despite being the true father of the Apple Mac, Jef was not a huge mouse fan – he preferred the keyboard as the control device of choice because the keyboard is faster and frees the user from having to locate the correct “control” in the GUI to achieve a specific action. I think however that he would have been interested in a more dynamic UI – particularly where the keyboard could not achieve the goals of speed and accuracy he identified as crucial to the interactive process.

If you are interested in where Jef Raskin’s legacy (sadly he is no longer among us) has progressed then you might like to take a look at http://humanized.com (large Macromedia Flash file warning if you want to see the movie). The “Humanized” product Enso is part of the “less is more” approach to computing. I firmly approve and what a great use for the now derided Caps Lock key. We need much less in our UI although that will always raise the issue of – “how do I find what I am looking for?” Effective desktop search (that’s Google, not that insultingly ineffective MS dog) has allowed us to flatten our file structures but we need the same approach for our applications. Early efforts like hiding supposedly unused menu items were a complete failure – who would want to hide the “Print” option of the File menu – it just makes navigating the menu slower not faster. It is a good sign that creative people are getting bored with windows (small w as that includes the whole current GUI mess) and have started looking at new interfaces for new and existing devices. In the mean time I’m off to hook up that old teletype terminal to the MicroVax sleeping in the corner – now that was a minimalist interface – but do you know what? It worked!

2007-02-14T03:29:00.000-08:00

NBL I think I get it now

An interesting juxtaposition of posts yesterday caught my eye and got me re-thinking about something I had been musing on a while back. Jeff Atwood’s coding horror had a post consigning the “Work Offline” option for Internet browsers to an historical footnote in this increasingly connected world but on the same day we (well me anyway) get first notice of the plans for Firefox 3 and they include a measure of local storage so that “on-line” applications can continue to work “off-line” which is something I think that Google have already “cracked” for their internal use and will hopefully be making available to the world at large soon. I also factored in Steve Yegge’s oddly toned denial that his post on the Next Big Language (NBL) had anything to do with Google. Steve normally does not mind his words and Google is normally enigmatic in their response to any speculation on their internal plans.

So how might offline/online applications work and what skills will we need to develop applications for this new platform model? Yes it is new – startlingly new. This platform is founded on web based applications that continue to function when you are not connected to the Internet server. Where collaboration or document sharing are desirable features – well they are built right into the Internet based model but that model becomes a “killer app” when you mix in the ability to continue to work away from an Internet connection. Toss in automated backups and file storage and this platform could meet all or a substantial part of the computing needs of a huge majority of today’s PC users.

PCs have been around for a good while now but squint a bit and you might just be able to see their end, as a consumer purchase anyway, in sight. If you are a games player then your Xbox or Sony Play Station is all you need to access your email, instant messaging and the world of on line applications – not just “office” style applications but the vast range of commercial and business applications currently only available on your company network. Your phone or even your USB thumb drive could become the repository holding the links to your personal choice of computer applications with processing power being supplied by a whole range of simple devices served up to rent or just borrow – located on desks in your office, on trains and planes and in every café and restaurant.

I am not sure that we even need a smarter web browser to achieve this – the required functionality could probably be delivered through a local proxy server. The proxy server passes all traffic to your applications through to the Internet servers when you are on-line and then fields the functional requirements locally when you are off line. You would need to be able to download the server side code to make it available to your local server and you would need to know that this code was executing within a secure virtual machine or robust “sand box”. This is where the NBL comes in of course – next generation JavaScript. The code would be pre-compiled to byte code and probably well obfuscated before it is trusted to client side servers but it would be almost the same code running locally as on the main application server. The only major change in the “rules”, under which JavaScript is run, required would be some ability to write to local data storage. Now if we wrap up and lock that requirement within a client based relational database we can use the security built into the relational model to ensure the security and privacy of each application’s data; while also ensuring the security of any “host” running the client process. When next your proxy server is linked to the Internet it can resynchronise your local and remote data sets and even update the server application code if it has changed since the last synchronisation.

I am assuming that you would not carry around or keep local copies of everything although that would be technically possible I suppose if memory devices continue on their current price/capacity curve. I would expect the applications to maintain an automatic “recent documents” list that would include contributions from collaborators and colleagues and that the user would be free to add to that mix.

Is this going to be big? Yes – revolutionary. Will it change the marketplace? Yes – although Apple will continue to be Apple and invest in proprietary systems that fit into the various consumer niches (like phones and TV sets) that will support the new model. Microsoft will continue in server operating systems and portable devices (Zune phone coming soon) but their home user OS will disappear into your games console and music centre. Dell and HP will sell even more servers and introduce cheap home devices that combine a processor, simple OS, Firewall and an ADSL connection all in one box for everyone without a network of games machines and music centres in their homes. The big disruption will be felt in the software market place. Today’s big players will be slow to respond to the new dynamic – patching and bodging to make things look like they are working. Opportunity abounds for new entrants ready to ride the new wave.

So – like I said on Monday – get learning JavaScript and start looking at the Internet server model for delivering applications.