Archive for December, 2005

The Accidental Utility of Slashdot

I've been reading Slashdot for the last year and a half or so. I stayed away from it until I downloaded an aggregator that had it preinstalled, and, well, it just kind of stayed there. I don't post comments myself, but I find find those who do interesting. Of course, it's probably not for the reasons they had intended.

I think Slashdot is a great way to measure my own susceptibility to argumentum ad verecundiam. I do this (retrospectively) by reflecting on to what degree I have agreed with the posts scoring "5, Informative" or "5, Insightful" when attached to topics I know relatively little about. I contrast that with how ignorant posts with the same rating seem when in reply to topics I know inside and out.

What is more likely? That only the high-scoring commentators on "my" topics make serious errors? Or that the overall intelligence and ability of the commentators is fairly uniform, and I attribute too much credit to them when I don't know the material? Meeting one or two posters in real life might bias your response, but I digress.

In Humans First Arose in Asia, for example, some of the comments currently scored as fives give away obvious misconceptions of the time frame of human evolution. There are good comments too, but the scoring system does a poor job of differentiating them. Competitors like Digg and Reddit suffer from similar problems with varying degrees of severity. (I can't read Digg at all, and I'm currently giving Reddit a "time-out" after it linked to one too many pseudoscientific/conspiracy-theory articles.)

So what's the point here? Just that adequate English skills combined with an argument that sounds logical can easily be mistaken for an argument that is correct.

I’m All Sixes and Sevens and Nines

The following is a review of A Mathematician Plays the Stock Market, by John Allen Paulos.

One of my inaugural tasks at my current job was developing a technical analysis package for market data. I have to admit I rather enjoyed this, for a few reasons only tangentially related to the specific technology at hand.

First, I like solving math problems–always have. Writing programs that do this for me are more enjoyable still. Second, there was the shameful thrill of scrawling some equations involving capital sigmas (the kind of thing those of us destined to be computer scientists are doing by eighth grade) on a whiteboard and watching the panicked expressions of the business and finance people present.

I guiltily concede that the latter motivation was the dominant one. I still keep a sheet with sigmas painted all over it within reach. Anytime I’m asked about the output of my analysis package, I produce it from deep within my desk which by the way, overwhelmingly contains only ketchup (Heinz) and straws (plastic, non-bending). I’ll scribble some new symbols on it and say something like, “so as you can see, the limit of this term as phi approaches infinity is…” and before I’ve finished the sentence the person has muttered something in bewilderment and shuffled away.

dice

I’m not necessarily doing this out of malice or contempt, it’s just that I realized a long time ago that the technical analysis of market data is largely a crock. I’ve always carried this nagging little fact with me, and at times I’ve pondered the morality of having this job at all. So I’m not really doing the user a disservice here, unless I’m somehow expected to explain to everyone in the world that you can find meaningful patterns in any set of data–words in the bible, petals on a flower, sand on the beach, or the price of Superconductor (NasdaqSC:SCON). Chances are, the pattern that you discover holds no predictive power.

So it’s really not important to the person asking what the answer is, it’s just reassuring to believe that I possess one. In the end, they will probably make about as much money as chance would dictate. Maybe a little more, maybe a little less. If they do happen upon the holy grail, that ineffable oracle of a model that really can forecast the future, it is as likely to be in spite of my explanation as it is to be caused by it. Populus vult decipi; decipiatur.

How odd it is, I found myself thinking while reading this book, that there exists a Nobel prize for Economics yet none for Mathematics. Is my support for this book just another example of confirmation bias? I’m obviously not qualified to say. I suggest you judge it for yourself.

Quick Debugger Tips

In my javascript debugging post I showed a stack trace and mentioned only that it seemed to be running javascript code. Here it is:

0:009> ~0k
ChildEBP RetAddr
WARNING: Stack unwind information not available. Following frames may be wrong.
0012f97c 6009db8b js3250!JSLL_MinInt+0x45e2
0012f9c8 6009cdd9 js3250!js_GetSrcNoteOffset+0x5358
0012f9f0 600c7c01 js3250!js_GetSrcNoteOffset+0x45a6
0012fa08 600856b5 js3250!js_GetScriptLineExtent+0x39e6
0012fa28 600b036f js3250!JS_NewStringCopyZ+0x44
0012fa40 600b3e93 js3250!js_FindProperty+0x26c5

This is a good stack to use to point out a few things about debugging without symbols.

If you're used to stepping through debug code in Visual Studio, your first instict may be to trust the stack printed out here. This would be wrong, for a few reasons.

First, there's the line telling us: "Stack unwind information not available. Following frames may be wrong." Stop me if I've covered this before, but it probably means that we're looking at optimized code. Optimized stack frames often have the pointers to each previous stack frame left out (you'll see this called "frame pointer omission," or FPO) which makes walking the stack much more difficult for the debugger.

Second, notice the offsets following the function names - most are very high. For example,

js3250!js_GetSrcNoteOffset+0x5358

The +0×5358 indicates that the return address is to a point more than 20KB after the export function given. 20KB is a lot of machine code, so this almost definitely an internal function call and not a return address into the body of js_GetSrcNoteOffset.

I say "almost definitely" instead of "definitely" with good reason. I have personally seen some VB6 functions several thousands of lines long that would certainly generate more than 20KB of machine code. But I guess if you're dealing with that then you have enough of your own problems without worrying about symbol subtleties in the Windows debuggers.

So the debugger here is showing us the closest symbol to the return address available, which turns out to not be very close at all.

What can we say about this thread's stack, given just this information? Only that it is very likely to be running javascript code. Not much else.

Conventional Javascript Debugging is for Wimps

Recently, Firefox has been maxing out my CPU "when I have more than three tabs open." As is typical with bug reports from users, this one is very poorly worded and essentially useless. It turns out that that was not a good explanation of what was going on. I messed around a little more and figured out that the problem had these characteristics:

  • The CPU was maxed out on any ESPN.com page.
  • Thread #0 was the offending thread.
  • Firefox's working set increased steadily while looking at ESPN.
  • Firefox eventually crashes with an A/V if the browser is left on ESPN.

Firefox 1.5 CPU Spike

I'm not sure if this is some strange interaction of my specific combination of extensions and settings, or if this happens for all Firefox users. This seems like a possible security vulnerability, but I can't say one way or the other.

I suppose I could have downloaded and compiled the source code for Firefox to figure things out, but that was way more effort than I felt like giving. I did managed to debug and fix this issue for myself–without source or symbols–which I think makes for an interesting writeup.

Since the problem was on every page on the website, and continued after the page finished loading, I assumed it had to be a javascript or Flash problem (both of which are abused by ESPN to an irritating degree). I don't even have the Flash plugin for Firefox installed, so that narrowed the scope of my investigation.

The first thing I did was try to debug the Javascript normally; this was hampered by several factors:

  • The Javascript debugger for Firefox, Venkman, hasn't officially come out for version 1.5 yet. I've complained about broken extensions before, but let me just reiterate how stupid and unprofessional I think broken extensions are.
  • Although this kind person has done his own workaround for 1.5, it was either brought to its knees by the ESPN site or was broken by something else.

To summarize the story to this point, I had a (likely) javascript problem on a page I visit very frequently, coupled with a browser bug that maxed the CPU and made the issue difficult to diagnose through normal means.

I attached to Firefox.exe in Windbg and started randomly breaking in and checking on thread zero while the spike was in progress. Sometimes, frames in the js3250 module were on the stack:

0:009> ~0k
ChildEBP RetAddr
WARNING: Stack unwind information not available. Following frames may be wrong.
0012f97c 6009db8b js3250!JSLL_MinInt+0x45e2
0012f9c8 6009cdd9 js3250!js_GetSrcNoteOffset+0x5358
0012f9f0 600c7c01 js3250!js_GetSrcNoteOffset+0x45a6
0012fa08 600856b5 js3250!js_GetScriptLineExtent+0x39e6
0012fa28 600b036f js3250!JS_NewStringCopyZ+0x44
0012fa40 600b3e93 js3250!js_FindProperty+0x26c5

The information for js3250 confirms (if the module name and the names of the export functions weren't enough for you) that it is the Mozilla Javascript implementation.

0:009> lmv m js3250
start    end        module name
60080000 600e9000   js3250     (export symbols)
   C:\Program Files\Mozilla Firefox\js3250.dll
    Loaded symbol image file:
        C:\Program Files\Mozilla Firefox\js3250.dll
    Image path: C:\Program Files\Mozilla Firefox\js3250.dll
    Image name: js3250.dll
    Timestamp:        Fri Nov 11 20:05:34 2005 (43753FDE)
    CheckSum:         00073A1C
    ImageSize:        00069000
    File version:     4.0.0.0
    Product version:  4.0.0.0
    File flags:       0 (Mask 3F)
    File OS:          10004 DOS Win32
    File type:        2.0 Dll
    File date:        00000000.00000000
    Translations:     0409.04e4
    CompanyName:      Netscape Communications Corporation
    ProductName:      NETSCAPE
    InternalName:     JS3240
    OriginalFilename: js3240.dll
    ProductVersion:   4.0
    FileVersion:      4.0
    FileDescription:  Netscape 32-bit JavaScript Module
    LegalCopyright:   Copyright Netscape Communications. 1994-96
    LegalTrademarks:  Netscape, Mozilla

All of this is reasonably good evidence that javascript is the right place to start. I took a look at the exports for js3250 ("x js3250!*" in Windbg) — it appeared as though this module was implemented as a few dozen C-style exports.

I thought that a logical thing to look for was an "execute a javascript function" function of some kind, and there were a few exports named some variation of "Call function."

0:009> x js3250!*Call*Function*
6008541f js3250!JS_CallFunction (<no parameter info>)
60085464 js3250!JS_CallFunctionName (<no parameter info>)
600854ff js3250!JS_CallFunctionValue (<no parameter info>)

I set a breakpoint on all of these.

0:009> bm js3250!*Call*Function*
  1: 6008541f @!"js3250!JS_CallFunction"
  2: 60085464 @!"js3250!JS_CallFunctionName"
  3: 600854ff @!"js3250!JS_CallFunctionValue"

After resuming the program, I started hitting these breakpoints constantly. I did a sanity check and made sure that this didn't occur on a cleaner site (google), and this was the case. I didn't have any immediate success figuring out what script functions were being called. You can dig up the source for these API's if you like, but I'm guessing that the script is already processed into different data structures by the time the javascript engine gets here.

I looked at some of the stacks when these functions were called, and I noticed this pretty far down:

0012fc7c 00534e18 js3250!JS_EvaluateUCScriptForPrincipals+0x70

I hoped this would lead me to some bits of javascript pointed to from nearby locations on the stack, so I set a breakpoint there. When it was hit, I started dumping out strings (using "dda @esp" and and "ddu @esp") and found this:

http://espn-att.starwave.com/motion/fsp/fsp.js

As an only slightly educated guess, I used the Adblock extension to block this script.

After reloading the page, the problem was gone! There don't seem to be any site-breaking problems associated with turning this script off. This is all or part of the "ESPN Motion" business that tries to display sound and video on the site. Hey, ESPN: this is a terrible idea in the first place. Your website shouldn't start yelling at me when I visit it. It's no excuse for Firefox to A/V, but I thought everyone with a three-digit IQ stopped doing this in 1996.

SQL Server 2005 Database Diagrams PSA

If you find yourself struggling with the following error message in SQL Server Management Studio 2005:

Database diagram support objects cannot be installed because this database does not have a valid owner. To continue, first use the Files page of the Database Properties dialog box or the ALTER AUTHORIZATION statement to set the database owner to a valid login, then add the database diagram support objects.

And you are sure that the database does in fact have a valid owner, check the "Compatibility Level" on the Options page. Make sure this is set to "SQL Server 2005."

alt="SQL Server 2005 compatibility level"/>

The Canned Chili Cookoff

I had a lot of vacation days left over this year, and not much to do with them. Not much, that is, until I decided to buy ALL of the spicy chili brands in the grocery store and have an EXTREME canned chili cookoff!

Hormel Chili - "Hot With Beans"

Total size - 15oz

Servings - 2

Calories / Serving - 260 (on the can), 270 (website)

General Observations

The can has some vintage 50s appeal. I have something of a soft spot for the Hormel brand, having once witnessed a can of beef stew turn into a flaming campfire projectile. Think "every slow-motion hand grenade war movie clip you've ever seen," except with a bigass can of beef stew.

Ingredients

No wild cards here. Kidney beans, ground sirloin, tomato base.

Heat

I've had "mild" salsas with a bigger kick than this. I added 30 shakes of Tobasco just to make it palatable.

Consistency

I was relatively pleased with the thickness. I think that's this chili's strong point. Being forced to add Tobasco made it a little watery, of course. In retrospect I should have used a few drops of Dave's Insanity.

Overall Score / Recommendations

3/10. Consider serving on a hot dog. Otherwise, not worth the time.


Campbell's Chunky "Firehouse" Chili with Beans

Total size - 19oz

Servings - 2

Calories / Serving - 220

Campbell's Chunky Chili

General Observations

I lost my fantasy football league this year because of the Philadelphia Eagles' disgraceful performance against the Seahawks. In the aftermath, many a chunky soup epithet spewed from my drunken visage.

Ingredients

Some decent chunks of beef and visible peppers. Recognizable tomatoes remain. Still, only kidney beans are used here. It would be nice to get some more variety in that area.

Heat

A very slight twang in the back of your throat. No discomfort by any means. Not bland, but definitely not exciting.

Consistency

Despite the boastful moniker, this one is a bit watery for my taste.

Overall Score / Recommendations

6/10. Definitely better than having no chili. Aftertaste is suspiciously reminiscient of Campbell's tomato soup.

Coincidence?


Health Valley Spicy Black Bean Chili

Total Size - 15oz

Servings - 2

Calories / Serving - 160

Health Valley Chili

General Observations

I was excited about this chili due to my experiences at the yearly Ithaca Commons Chili cookoff. The multi-bean vegetarian chilis were always the best in show - there was of course the Moosewood and also some latecomers like the Lost Dog Cafe. This dominance could be attributed to two factors: hippie know-how and a truly staggering inferiority complex.

Ingredients

Finally! Some pinto and black beans. Unfortunately, I think the decision to include soy protein ultimately proves to be disappointing and ill-conceived. The chili is also labeled "low sodium," which is not well compensated for. It needs pepper and some more cumin.

Heat

Basically nonexistent. I added Tobasco and chili powder.

Consistency

The most watery of the chilis being reviewed. I was disappointed that it was not a heartier bean broth.

Overall Score / Recommendations

4/10. I had high hopes for this chili, but ultimately the seasoning just falls short. It was quite bland. It scores points for being the healthiest option here, but it should not have sacrificed taste.

The winner: Campbell's Chunky "Firehouse" Chili with Beans.

I was disappointed with all of these chilis. Although each is promoted as "spicy" in some way, none stray into triple Scoville digits. Campbell's gets the nod by default.

A Meandering Static Locals Story

I was writing some C# code the other day and attempted to use a static local variable. This may only be because I've been spending so much time with C++ recently, but I was very surprised to find that it didn't work. "That's a shame," I thought. It's not as though I use static locals every day. I've been using C# for a few years and I was just getting around to discovering that it doesn't support them, after all. Nevertheless, I would say I use static locals exactly when I need them.

I was sure that VB.NET supported static locals. I thought that perhaps an Object Orientation zealot decreed that they weren't going into C#, even though that would not be typical of the attitude exhibited in the most recent C# whitepapers. As it turns out, that's not what happened at all. Static locals are a compiler feature, and not something implemented on the CLR level. Paul Vick explains:

Implementing static locals in the compiler was a gigantic pain in the ass, and we had a lot of arguments about whether they were worth the effort. There are times that I haven't been entirely convinced about it, but I think my thoughts on this are changing…

I suppose I have already conceded that there are more important things to spend time on. In the same post Paul discusses the relative merits of the keywords in either language.

We also felt that “Shared” is a lot more descriptive than “Static,” and I wonder whether C# would have chosen that term if they hadn't been stuck with the legacy of C, but… This is probably the most painful keyword divergence between the two languages in terms of documentation.

Something of a silly discussion, I think, since VB is teeming with its own historical oddities (to wit: Dim intArray(0) As Integer). But this does remind me of a story.

A while back I was debugging an odd issue where users of a web application were seeing each other's data "randomly." Our application has a set of links on one side of it that are customized to relate to the entity the user is viewing. A typical bug would be:

The top half of the links relate to entity 'A' (which is the right one), but the bottom half of the links relate to entity 'B,' which I was not looking at.

Compared to some other bugs I have worked on, this one was a piece of cake. I figured out pretty quickly that the developer of the control in question had written a big section of variables that looked like this:

Public Shared A As Integer
Public Shared B As String
Public Shared C As String
' …

These were used in the process of rendering the hyperlinks. I asked the developer why he'd done this, but he didn't know why. He had just seen the word "Shared" associated with member declarations in the past and thought that he needed to put it there.

He was typically more informed about keywords than this, so telling him that Shared was like "static" in other languages brought instant recognition. In fact he changed the code so that it looked like this (I'm doing this from memory, but I think this is pretty faithful to the original treasure):

' Shared == Static!! Shared variables are static!
Public A As Integer
Public B As String
Public C As String
' …

Interview Questions for Eccentrics

In my last post I alluded to interview quizzers. These are [straw man alert] egotistical people who assume that any smart person must know the highly-specific information that they themselves have learned in the last month. "I don't know, but I'll google it and tell you in a few minutes," is what I would recommend saying to such a person. While there are also many really excellent guides out there that will help you become a good interviewer, I think there is one area in which all are lacking. Naturally, I am referring to completely askew interview questions.

We've all had interviews like this. Maybe the interviewees just don't know anything. Maybe they're biology majors that your boss scheduled for you because "their resume looked kind of interesting." Maybe they're Operations Research people interviewing for your programming job, and they made it through HR because "all engineers are the same." Maybe they have a year of experience, yet somehow six pages of bold acronyms ("HTML, DHTML, ASP, VB6, J2EE, XML" - I haven't seen one with AJAX yet but I'll let you know) and this successfully fooled someone upstream.

You could gripe and moan about getting some better resume screeners around here, dammit, or you could make the most of it. Once you've gotten the deer-in-the-headlights blank stare trying to work through an algorithm or write a C function, you still have the awkward remainder of the interview to worry about. You could waste time explaining the position or talking about their fraternity, or you could politely say you don't think things will work out (I've never had the heart to do this). There is a better way.

In an ideal world, all interviews are an uplifting experience. Even if the candidate isn't qualified, you still want them to walk away with something–a warm fuzzy feeling, perhaps. Personally, I try to apply my own philosophy of making everybody's day just a little more surreal.

To that end, I thought I would share two of my favorite questions for this purpose. The good thing about each of these is that they leave a slight (yet tangible) opening for a clever candidate to find redemption.

Where is November?

This is the great grandaddy of them all, when it comes to programmer interviews. This question is lifted from The Mythical Man Month, a classic of software writing. Fred Brooks explains that the point of this question is to determine whether or not the person thinks in three dimensions.

The idea that great programmers think in (at least) three dimensions is quite valid, if you ask me, but I'm not sure that all people capable of doing so make great programmers. The connection here is that 3D-thinkers will have a mental model of the calendar whether they realize as much or not. The smartest people will instinctively recognize what you're asking.

Anyway, the nice thing about this question is that it can be answered even if the candidate calls you a sociopath and storms out. If they motion with their hands as they parse the question, that is probably an unconscious answer.

There is no correct response. My "November" is located to the lower left. The calendar forms a very disproportionate ellipse with summer on the top and winter on the bottom. June, July, and August are overhead and roughly equivalent in length to November through May.

Name the Nine Planets

I really want to use this as a basic litmus test, but this is foiled by the fact that I know some very competent programmers who would not have answered it correctly. I really can't believe this–if you ask me, this is a trivia question only if "the sun revolves around the Earth, true or false" is a trivia question. It’s inconceivable to me that someone can know all of the words to Bird Dream of the Olympus Mons without caring where Olympus Mons is.

This question is deceptively open-ended. The candidate can get some major bonus points if they tell you that there are actually eight, ten, or eleven planets, you retard.

Disclaimer: this is mostly a humor piece, and I actually do make an attempt to keep things professional. So in other words, don't come crying to me if acting crazy gets you sued. This should not be attempted by amateurs.

Faking Multiple Inheritance with Event Handlers

I'm going to take a page from the Book of Darrel and post some design patterns whenever I realize that I'm using them. I'd like to be careful because I think that there are a few camps when it comes to design patterns:

  1. Design patterns are very important, very specific, and are a great thing to quiz people about in an interview.
  2. Design patterns are more of a feel thing, something that is figured out when you're factoring code. There is no way you will not discover these on your own, if you are experienced and smart.

I lean to the second category. The quizzers tend to be solipsists and may know little else (you'll see the same attitude with plenty of other development topics).

Now that you know how to take my writing about design patterns, let's suppose that you want to make a Windows form that becomes a little translucent when it is inactive. That may or may not be annoying, but there is a very straightforward way to make it happen:

// Form that becomes transparent when it is inactive
public class Foo : Form
{
    private const double c_InactiveOpacity = 0.25d;
    private double _prevOpacity;

    protected override void OnDeactivate(EventArgs e)
    {
        _prevOpacity = this.Opacity;
        this.Opacity = c_InactiveOpacity;
        base.OnDeactivate(e);
    }

    protected override void OnActivated(EventArgs e)
    {
        this.Opacity = _prevOpacity;
        base.OnActivated(e);
    }

    public Foo()
    {
        _prevOpacity = 1.0d;
    }
}

Which works fine, unless you want TWO forms that do that, the second with a special base class not related to the heirarchy of the first.

public class Foo2 : SpecialForm
{
    // criminy
}

This conundrum could be resolved in a few ways:

  1. Ctrl+C, Ctrl+V.
  2. Multiple base classes.
  3. What I am going to call the "Extender Pattern."

If you picked the first option, I think it'd be better for both of us if you unsubscribed from my feed. The second option would work fine, but CLR languages don't support it. Some days this bothers me, but most days I think it's a good thing. Especially when I factor a solution that is just as elegant.

I'm calling that solution the "Extender Pattern" without bothering to find out if anyone has used this term to describe anything else. As I explained above, this is more about artistic sensibility than it is about hard rules and book learning. Not that I'm anti-book learning.

The basic idea is this: you have a set of components that support certain properties, methods, and events. The forms, in this case. You add in to the mix a set of small objects whose only responsibility is to wait for events from the components and make very directed changes to them.

So here is such a class for the problem stated above:

// Alters the target form's transparency level
// when it is activated/deactivated.
public class TransparencyExtender
{
    private Form _target;
    private double _prevOpacity;
    private const double c_opacity = 0.25d;

    public TransparencyExtender(Form target)
    {
        _target = target;
        _prevOpacity = target.Opacity;
        target.Activated +=
             new EventHandler(Activated);
        target.Deactivate +=
             new EventHandler(Deactivate);
    }

    void Deactivate(object sender, EventArgs e)
    {
        _prevOpacity = _target.Opacity;
        if (_prevOpacity >= c_opacity)
        {
            _target.Opacity = c_opacity;
        }
    }

    void Activated(object sender, EventArgs e)
    {
        _target.Opacity = _prevOpacity;
    }
}

Given this, we can write our forms very easily:

public class Foo : Form
{
    private TransparencyExtender _tExt =
        new TransparencyExtender(this);
}

public class Foo2 : SpecialForm
{
    private TransparencyExtender _tExt =
        new TransparencyExtender(this);
}

Keep in mind that I am not claiming that this is original. It's really the same as a small secondary base class in C++. However, I think there's value in looking at the same problem in different ways.

In a real application I'm working on, I've written a handful of other window/control styles that can be applied and removed on the fly or from the designer. Again, not original, but I like my little framework. Perhaps I'll post this someday, or turn it into a product.

I like event/message-driven programming the more I try to use it. I'm using Windows Forms as an example here, but it would be wrong to assume that's the only place this could be applied. I've done the same kinds of things in ASP.NET.

Exploit Natural Mappings in Interface Design

The on-screen guide for my Direct TV box looks roughly like this:

DirectTV blows

The control on my remote that causes the current channel to move up and down looks like this:

DirectTV blows

If the problem here is obvious to you, you too may have a future in user interface design. If you thought to yourself, "hey, they screwed up an obvious natural mapping," then you are probably already involved on some level.

Just to make it perfectly clear what I am talking about, the orientation of the controls are out of whack. From my perspective, the channels increase going down on the screen but going up on the remote. The fact that the two don't match causes me to change the channel up when I meant to change it down, and vice versa, pretty much constantly.

You might call me a moron, and most days you might be right, but not here. This design is broken. It should do what I want without requiring me to stop to think about it.

It's amazing that decades after the publication of The Design of Everyday Things that it is still possible for a well-funded company to make this mistake. This company has enough money to buy the NFL through 2010, but apparently it isn't able to find a decent interface designer.