Posted by: Peter Scott | October 9, 2007

Diagnosing problems

Sometimes, when things are not working as they should the human brain goes into straw clutching mode. In extreme cases this includes guilt-by-association scenario where a benign event that occurred at around the same time is blamed for something happening or if not blamed causes a large amount of lost time working out what is going on. For example, in yesterday’s posting on a database upgrade and the inability for a query tool (in this case Business Objects Web Intelligence (or whatever it is called these days)) to connect to a data warehouse. The development team could use SQL/Plus and SQL-Developer, the customer could use Business Objects clients on their desktop PCs, my team could edit, deploy and run OWB maps in the database, SQL loader worked, exp worked. The only thing that did not work was this one key piece of software – it was the way in which the majority of users accessed the data warehouse. So what was it that made the customer go down the route of thinking the only possible cause was a defective upgrade of a database? Their gut reaction was to back-out the upgrade by restoring the whole file system to the pre-upgrade state. This seemed a little over the top as to me something had gone wrong with one connection – the error message the users saw referred to TNSNAMES – the obvious place to look was TNSNAMES on the Business Objects server Oracle client. I explained my diagnosis based on log files, error messages and information about an ‘extra’ change that customer put in to the database connect string. I explained what needed to happen to make it work properly and work it did. I left the conference call so that the customer could finish validating the web reporting suite. Five minutes later a senior manager from the customer called me to ask how I sorted out a problem in five minutes that took his whole support group three hours to get to the stage of backing out the upgrade.

There are some frightening things posted on the Internet. One of my college friends has posted up our 1977 graduation class group photo and a couple of shots of the class socializing in one of the Oxford colleges. I had big-hair those days – Google already has the URL, but do you think I would give out clues where to find it?

About these ads


  1. Experience and a great ability to debug.

    It seems to me most people want to complicate things. We have a situation were we have deployed a new external facing web application. For 2 weeks since launch, they have been trying to get all the load balancers and everything configured properly because they can’t handle the load. How big is the load? Perhaps 5,000 page views a day…I think I’ve read on asktom that his site receives roughly 40,000 a day and it’s on a single box (both database and web server).

    Anyway, I just want to scream at all these people. Get it working on one box and then worry about making it more scalable, which is reasonable.

    Cheers to you for looking to the simple solution. The down-grade would have been a nightmare!


  2. Oxford college graduation in 1977? Big hair? Hmmm… What colour was your crushed velvet suit?

  3. crushed velvet? – I went for made-to-measure… I had the flares taken out a couple of years later [and put into the jacket :-( ]

    BTW – a bit of jump of faith to assume an Oxford graduation – my life is never that simple

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s



Get every new post delivered to your Inbox.

%d bloggers like this: