Python stack trace saga

How hard can it be to obtain a stack trace in a mature language like Python?

The task turns out to be non-trivial. Python gives you the following dump:

Traceback (most recent call last):
  File "/frickingly/long/path/google/appengine/ext/webapp/__init__.py", line 503, in __call__
    handler.post(*groups)
  File "/another/path/controllers/base.py", line 55, in decoration
    return original_func(self, *args)
  File "/another/path/controllers/product.py", line 42, in post
    self.product.reopening_mode = NAMES_TO_REOPENING_MODES[self.valid_string('reopening_mode')]
KeyError: u'on'

There is something mind-bogglingly odd here if you look closely enough. Python does not provide class names as part of a stack trace. One should expect to see WSGIApplication.__call__ and ProductSettingsHandler.post, while the stack trace only lists __call__ and post. I have, like, a few dozen post functions in my project, and at least 3 of them are in product.py. This hip new OOP thing has never reached this corner of the language.

The way we deal with it is similar to the problem itself, that is, stupid and mind-boggling. A simple heuristic checks whether the first argument of a function is an object that has an attribute pointing to the function. Decorators make the problem worse, and there's another heuristic that deals with some function-based decorators. Net result: CrashKit reports correct class names in most cases, even for decorated functions. (See the code at http://gist.github.com/152398)

Another problem is that “/another/path/controllers/product.py” is a very verbose way of saying “controllers.product”. The path might differ across various machines, and it's crucial to obtain machine-independent stack traces for CrashKit bug grouping to work.

Good news on calculating package names is that you can simply query the value of __name__ from the global context of the frame of interest. Bad news is that sometimes you get None in return, and one notable case is when an import statement has not yet finished execution. In such cases, one has to guess a possible package name by iterating through all sys.path entries.

Again, the implementation (taken from CrashKit Python) is available at http://gist.github.com/152407. Both fragments of code mentioned here are known to work in production environments and are tested on Mac OS X, Linux and Windows. The whole stack trace collection code is available under BSD license as part of CrashKit Python client.