Archive for the ‘Programming’ Category

How to Debug Python in GDB

Although I haven't found this to be necessary nearly as often as Windbg was for me on Windows, it's still somewhat handy to be able to look at a Python daemon or multithreaded program in GDB. I've had to set this up a few times now, and I've forgotten the steps each time, so here they are in one place.

Step 1 - Get a debug build of Python.

You need a debug version of the interpreter. This is what I do from source on *nix:

./configure --prefix=/usr/local/dbg
make OPT=-g
sudo make install

sudo ln -s /usr/local/bin/dbgpy /usr/local/dbg/bin/python

Step 2 - Fix the botched .gdbinit that comes with the Python source.

The Python source comes with a .gdbinit file (Misc/gdbinit), but it's broken in 2.5. Apply these changes, which will fix pystack to 1) work and 2) work on threads other than the main thread.

@@ -119,8 +122,8 @@

 # print the entire Python call stack
 define pystack
-    while $pc < Py_Main || $pc > Py_GetArgcArgv
-        if $pc > PyEval_EvalFrame && $pc < PyEval_EvalCodeEx
+    while ($pc < Py_Main || $pc > Py_GetArgcArgv) && ($pc < t_bootstrap || $pc > thread_PyThread_start_new_thread)
+        if $pc > PyEval_EvalFrameEx && $pc < PyEval_EvalCodeEx
         pyframe
         end
         up-silently 1
@@ -130,8 +133,8 @@

 # print the entire Python call stack - verbose mode
 define pystackv
-    while $pc < Py_Main || $pc > Py_GetArgcArgv
-        if $pc > PyEval_EvalFrame && $pc < PyEval_EvalCodeEx
+    while ($pc < Py_Main || $pc > Py_GetArgcArgv) && ($pc < t_bootstrap || $pc > thread_PyThread_start_new_thread)
+        if $pc > PyEval_EvalFrameEx && $pc < PyEval_EvalCodeEx
         pyframev
         end
         up-silently 1

Note that the locals still won't work. I'll post a diff that fixes those when I get around to doing that. In the meantime it's not too hard to poke through the python structures to get them.

On OS X you can just copy the result of this to ~/.gdbinit. Some linux distros might be cranky about where you put this and where you launch gdb.

Step 3 - Launch your program with the debug interpreter and attach.

From the command line you can attach with "gdb /usr/local/bin/dbgpy [pid]." I like gdba mode in emacs (see below - M-x gdba, then the same kind of command line).

gdba mode in Emacs

New MacBook Pro, Jealous?

I suppose I finally had enough of the carpal tunnel caused by typing '\' to delimit directories (yes, powershell helps a little, but not enough).

Or, I decided I knew enough about Windows for now and needed to branch out.

Or maybe I was just in a rut.

Well, I'm happy about whatever it was that caused me to make my latest impulse purchase.

Haskell program in Aquamacs

I've had it less than a week. Great stuff so far:

  • Having a unix-based OS to develop on is fantastic. The compilers for the languages I prefer to use in my spare time these days seem to work much better, presumably because many of the people who work on those languages also use Macs. GHC, for one, isn't terribly well supported on Windows.
  • It only took me a second to figure out how to swap caps lock and control for emacs.
  • It's great to have a terminal that doesn't suck. (Though, it'd be hard to do worse than cmd.exe. I'm aware of several valiant attempts at replacing it, but none compare.)
  • GarageBand - watch for my band to make a splash on the indie hate folk scene sometime in 2007. That's a genre I just invented, by the way.

I guess that's mostly developer stuff, which is odd since Apple's marketing actively makes fun of programmers.

I don't see myself ceasing Windows development anytime in the next decade, though, so if you're only here for .NET debugging stuff I wouldn't head for the hills just yet.

The reports of my madness may not be so exaggerated

Sometimes I think my quest to get my functional programming skills up to snuff is turning me into a raving lunatic. I keep writing code that inflicts physical pain when kept to myself. Here's a fresh cut and paste out of my emacs:

  applyAll [] = id
  applyAll fs = foldl1 (.) fs

After writing that, I immediately opened up all of my instant messenger programs in search of an imperative programmer to accost. "Don't you see? Don't you see how beautiful this is?" I would say. They almost never see.

Wheels I Have Wasted Time Re-inventing

This site has been silent for quite a while. My lame excuse for this is as follows.

Sometime around the beginning of the summer I started trying to learn functional programming in earnest. I had some exposure to this in school (ML, if I remember correctly), but not nearly enough, as it turns out. C#'s anonymous methods provided the "eureka" moment that made me want to revisit the topic. I had also been asked to contribute to The Hub, but I had felt like I was lost in the world of F# and that fizzled out altogether (maybe now I might have a few things to say).

So that's my story–I have been exploring strange new(-ish) topics and I've found that I haven't had anything insightful to contribute to the world.

Recently I've been trying to complete a few large projects in Common Lisp. I'm pleasantly surprised to find myself saying that most or all of the highfalutin crap that is spouted about Lisp is true. As to why it isn't the most-used language in the world right now, I am too much of a novice to speculate. But purely from a language standpoint it is certainly making all of the others feel, well, tedious.

Lisp seems to supersede every single neat trick I have ever come up with programming professionally. It either contains them outright, or it makes them trivial through macros or completely unnecessary for some other reason. Let me give you one simple example.

Writing applications in C#, I've occasionally had a singleton or some kind of global object that I wanted to be redefined within a certain call stack. Let's say I have this API:

class Foo
{
    public static Foo Default { get; }
}

And I want this to be redefined for a little while while I call some method called Bar(). Impersonation of users is one concrete example of where this is useful.

My neat-trick solution to this problem is to make the implementation of Default use thread local storage (and maybe fall back on a static instance too), then create a "scope" class that allows me to write code that looks like this:

// Original value of SomeOtherFoo out here
using(FooScope s = new FooScope(someOtherFoo))
{
    // In here, Foo.Default gives someOtherFoo
    Bar();
}
// Original value of SomeOtherFoo out here

(I am skipping over a lot of irritating details here and let me be clear, I don't claim to have invented this. I just mean that I burned a lot of hours arriving at it basically independently of the ten million other programmers who probably recognized the same need and did the same thing. It is purely my own ignorance and stupidity that made "re-discovering" it necessary.)

Anyway, I thought that was clever at the time. But with dynamic scope (as opposed to global scope, which is what static fields have in C#) Lisp has this as part of the language. Writing this:

(let ((*default-foo* some-other-foo))
  (bar))

Is basically the same thing with no effort. The Lisp syntax has the added advantage of working with objects other than the ones I author myself.

This reminds me of the first time I was exposed to .NET events. I realized that I had implemented the same god-damned thing in other languages a handful of times (albeit as a less-seasoned programmer) and it never worked as beautifully or as simply.

Naturally, CLOS seems to have some features that I now realize I have been mimicking imperfectly with events. It also has features that will probably replace events in many of the problems where I have applied them. That is an altogether different story.

Thinking Better Tech Designs

Where I work, there are programmers of widely varying skill levels all working on a single project. Some pieces tend to get behind, and some finish much more quickly. There are a seemingly endless number of integration problems. I am daily pondering what the differences are in the way people develop, and how I can communicate good ideas in a way that will help people reach that “eureka” moment where they get it.

Generally speaking, I’ve been pretty bad at it. I’ve tried explaining inheritance with stories, shaking people by their collars, etc, nothing really seems to work. The result is that people write code the way they always have: messy, hard to understand, hard to transfer to another owner, hard to change without creating new bugs.

Today this SLAR on System::Console got me thinking. Particularly, this line:

This class is a classic example of “design in reverse.” Before we designed this class we knew what we wanted the Hello World example to look like:

Console.WriteLine ("Hello World");

Well, there it is. That might be my own “eureka moment.” Write an API for yourself as you go. For every task you have to accomplish, ask yourself this:

If I had access to an oracle that could do my task for me, how would I want to use it?

Then start building that oracle. It’ll have steps that seem hard too, but use more oracles and more black boxes.

Another important point is to avoid thinking about code as much as you can.

Now, there are (obviously) a lot of steps between being a programming neophyte and being in the top few percent of software engineers. Most people need to get a lot of bad code out of their system before they’re much good. And this is certainly an oversimplification of thinking like a good developer.

Thinking of the best questions to ask your oracles can be a subjective, soft science too–but hopefully this is a decent starting point.

Moore’s Law and the Free Lunch

This article was brought to my attention from a few sources. The general theme here is, “concurrency is going to be really important as processors begin to hit physical limits, and these kinds of programs are harder to write.” I thought I would give my spin on it. Here is another one-sentence summary of that article, childishly represented using Windows Paint.

Your friend Jack Albertson

At the admitted risk of sounding too much like a doomsday prophet, my prediction can be summed up as follows: despite attempts at tool and language support (for instance, ), this is going to be painful for a large percentage of developers. Software cycles will probably take a big turn for the worse, so you might be better off working on the quantum computers in your garages now.

My reasoning is heavily influenced by the alleged “object revolution.” The fact is, you can’t claim to be doing object-oriented development just by virtue of using a language that has object-oriented features. And you can’t reap the benefits of doing object-oriented development in that case, either. In this day and age, I still see a ton (scores… hundreds.. maybe thousands) of methods that are 200 lines long and take twelve arguments. Now you can put a “virtual” in front of a method like that, but there’s obviously still a problem.

I like this quote from Object Thinking:

Both software engineering and object orientation have achieved a strange status - everyone claims to be doing them without really doing so.

The thesis of that book is that OOP and traditional programming take drastically different mindsets. Changes in mindsets can be really difficult to accomplish. Tools help you but they don’t automatically make you good at the task at hand. We’ve got great OOP tools now, but I think a lot of people still work on teams where deadlines are missed, integrating code written by different people is hard, and any number of shortcuts leads to a mess of spaghetti. We get away with it to an extent, mostly because it is accepted that software projects are late and contain bugs. We work way too hard in the process, though.

And as accurate as I think that is for the “object revolution”, it is only more accurate for the upcoming “concurrency revolution.” Writing a multithreaded app is a lot different than writing a single threaded app.

It could be good news for the highly motivated / educated, but as in any field that is the minority. I’m looking forward to the challenge, but purely for selfish reasons.