The trials and tribulations of MS Soap Toolkit 2

13th May, 2002

Recently during the course of my work I found myself writing a Windows based application that needed to be able to communicate over the internet with a server running on our company network. I investigated several possible alternatives to achieve this, and in the end came up with what appeared to be an ideal solution: SOAP.

SOAP (Simple Object Access Protocol) offers a simple-looking system that allows a client application to communicate with a server over TCP/IP port 80 (as used by web browsers). Using SOAP seemed to give me everything I needed, no problems with firewalls, an established standard and example code that I could get running without any problems.

The only initial downside with the Microsoft SOAP Toolkit that I was using was that it didn't support Windows 95. I decided we could live with that: it's an old OS anyway and most of our clients would be running Windows NT or 2000.

There are two ways of using the MS Soap Toolkit: the "high level" and "low level" APIs. The high level API uses two files on the web server to describe the service to the SOAP client, which then pretty much does all the difficult stuff automatically. The low level API requires a lot more of the wiring between the server and client to be written. Of the two, the high level API seemed to do everything I needed and was a lot less work to program, so that's the API I decided to use.

Before I got too involved with the full complex implementation in my application, I set about performing some tests to ensure it worked how I wanted it to. I run simple connectivity tests over our network. I tried running tests with my computer removed from the company domain. I tested from my computer at home. We even had clients visiting from Hong Kong, and I gave them a beta copy of the software to take back with them. Everything worked just fine, so I set about getting the program developed.

And six months later I had finished. And I was very pleased with it! At last a truly portable application which our clients could run from anywhere they could get an internet connection.

And it was at this point I realised that I had missed one small element in my testing. In each of the cases, I had been using a direct internet connection that didn't need a proxy server. I had briefly tested the software on a computer using a proxy server and it worked fine so I thought nothing more about it. But when we gave the software to our main client we found that we had a problem. They not only had a proxy that they used to access the internet (and therefore our server), but they needed to authenticate before they could use it.

Now surely that couldn't be too much of a problem, could it? Microsoft must have thought about this before they released their software?

At first sight it seemed that they had. The SOAP client objects were able to be instructed of a proxy server and the authentication details required for that server. But there was one small problem. The proxy details could only be supplied after the object was initialised, but in order to initialise the object required some data from the internet. Which it couldn't get because I hadn't given it the proxy authentication details. I let out a slow sigh at this point, realising that there may still be a lot of work ahead.

Trawling the web and posting to the newsgroups revealed that this really was as dumb as it looked. For my purposes, the high level API was completely useless.

All was not lost, however, as there was always the low level API. Because this relied on the server to handle requests at a lower level, there was no need for the service description files that the client was trying to read before it could initialise with the high level API. And wonder of wonders, it worked with proxy authentication. Great stuff -- except that now I needed to recode every function in both the client and server components of my application.

Getting this to work was not a lot of fun. The first problem I encountered was that the low level SOAP components that run on the server would not work in the VB IDE. This was confirmed within a MS knowledge base article that I eventually found. I spent some time looking for ways to work around this shortcoming but had one problem after another until I eventually decided on a different solution. Instead of using the SOAP components to encode the response to the client, I just build the response by hand. This was a bit fiddly but worked a treat once I had the XML content correct. Now if I could get that to work in VB, why couldn't Microsoft? It was nothing more complex than a bit of XML manipulation.

I started to add the functionality into my SOAP server and I started to move the calls in my client across to the low level API. At last it seemed there was light at the end of the tunnel. Sadly I was wrong again, though, and it turned out that the light was actually the headlights of an oncoming train...

Yes, yet again I was let down by the MS Soap Toolkit. It turns out that the low level API has a problem of its own. If too many requests are sent within a small space of time (by which I mean more than about one per second), the client crashed with YAME (Yet Another Meaningless Error). And that is that for MS Soap Toolkit. I have finally had enough of it. Much as I would like it to be the answer to me problem, the only thing it is actually an answer to is the question "Which programming toolkit should you never use?"

I believe there are still a couple of possibilities. I have briefly played with Simon Fell's PocketSOAP and, whilst not quite as pleasant to use (from the client end, anyway) as MS Soap it does seem to work with my existing server code and without any of the stupid errors that the MS Soap Toolkit seemed to enjoy raising. This is a definite option which requires some investigation.

The only alternative to that as far as I am aware is to drop SOAP completely. No bad thing in my opinion. A much better alternative would seem to be to just post a request to an .asp page, using standard URL conventions to encode all of the parameters etc. The .asp page would just respond with an XML-encoded reply instead of an HTML page. If I could just find a way of getting the Microsoft Winsock control to handle proxy authentication (or find another control that could do it) then I'd try this. Ever get the feeling you're going around in circles?

I'll conclude this by saying that if you ae considering using SOAP for your application, particularly if you're looking at the MS Soap Toolkit, then think very carefully about it. Before you get too far into development, ensure you have tried your client under every set of conditions you can think of -- including on a network that needs to authenticate to use their proxy server! I note that a beta version of MS Soap Toolkit 3 is now available, whether this will resolve any of these problems I cannot say but judging on the quality of the last full version of MS Soap, I think I'll leave the beta for someone else to evaluate.

If you have any comments or suggestions regarding this article, please don't hesitate to contact me.

This article is copyright © Adam Dawes, 2002.