Access violation errors with callbacks in ctypes

I’ve just spent a few hours trying to solve this bug, so I’m publishing this so maybe it will help someone with this issue…

Assume that you’re working with a DLL/.so library through ctypes in Python, and this library allows you to set a callback for some other function. In my case, I was working with unrar.dll. The code was something among these lines:

UNRARCALLBACK = ctypes.WINFUNCTYPE(ctypes.c_int, ctypes.c_uint, ctypes.c_long, ctypes.c_long, ctypes.c_long)
#in a class...
RARSetCallback(self.handle, UNRARCALLBACK(self.callback_fn), 0)
RARProcessFile(self.handle, RAR_TEST, None, None)

The first lines constructs the function prototype, the second sets the callback in a function of the DLL file, and the third calls a function in the DLL which will call the callback.

Can you spot the error?

The code worked fine in Python 2.5, but then I changed to 2.6 and it stopped working. I got a “WindowsError: exception: access violation reading…” (or writing) exception in the third call.

The reason, which is obvious in hindsight, is cleared explained in the docs:

Make sure you keep references to CFUNCTYPE objects as long as they are used from C code. ctypes doesn’t, and if you don’t, they may be garbage collected, crashing your program when a callback is made.

(Though it’s not explicit, it applies to WINFUNCTYPE objects too)

The WINFUNCTYPE object created in the second line no longer exists in the third line, so when the callback was called, it no longer pointed to a valid address. The solution is simple — just keep a reference to the object:

UNRARCALLBACK = ctypes.WINFUNCTYPE(ctypes.c_int, ctypes.c_uint, ctypes.c_long, ctypes.c_long, ctypes.c_long)
#inside a class...
self.callback_ref = UNRARCALLBACK(self.callback_fn)
RARSetCallback(self.rarFile.RAR._handle, self.callback_ref, 0)
RARProcessFile(self.rarFile.RAR._handle, RAR_TEST, None, None)

The only mystery left is why the old code worked on 2.5!

Quivi 1.0 released

I’ve just released the new version of my image viewer and manga / comic reader, Quivi (only for Windows right now, but it will be ported to Linux and maybe to Mac). I’ve rewritten it from scratch, check my last post for more info.

The new features added were:

  • Prefetching of the next image
  • Wallpaper dialog
  • Start directory option

But that is only the beginning; now it will be much more easy for me to work on Quivi.

There are many comic readers around; like CDisplay (outdated, but very popular), Comical and Comix. They have much more features than Quivi, and I must admit that I often thought on giving up on Quivi. But I just couldn’t.

I wrote it mainly for myself; when I started there was only CDisplay. The first version was written with Visual Basic (go on, laugh). Then I changed to C++. And now I changed to Python, and I hope this is the last language change 🙂 It’s a hobby project that I grew to love. It suits my needs, but I’d really like to suit the needs of other people too!

It is scary when you realize that people are using your software. I feel responsible; wherever I find someone criticizing it, I feel guilty for not attending those people needs. Yes, that’s stupid, but it’s how I am. At the same time, though, it’s awesome to know that there are people who like it and find it useful.

If you have any suggestion, comment, complain about Quivi, please send it to me!

Why I rewrote Quivi from scratch

Joel Spolsky, popular software engineering, said in this blog:

(…) They [Netscape] did it by making the single worst strategic mistake that any software company can make:

They decided to rewrite the code from scratch.

I agree, mostly. Many projects market that they’ve been “rewritten from scratch” as if it was something marvelous, and most of time, it’s just a sign that the new version probably has more bugs than the previous. But, wait, I just rewrote Quivi (a image viewer and comic / manga reader) from scratch! Well, why?

Well, I thought I really had to justify this. So here are the reasons:

  • It is a small project. Rewriting it certainly isn’t as hard as rewriting a browser!
  • I couldn’t stand C++ anymore (the programming language I used before). Of course, this isn’t C++ fault per se. It has its uses — the right tool for the right job, and so on. But to write an desktop application? It’s overkill. If you manage to pull it of, hey, kudos to you. But I had no motivation to work on it anymore.
  • I love Python (the programming language I use now), and of course, I’m not the only one. Programming is fun in Python, so much that had the motivation to rewrite Quivi from scratch in the first place! And of course, it will be much more easier for me to keep working on Quivi.
  • I could change GUI libraries. SmartWin is a nice library, and uses templates in very interesting ways; when programming Quivi I ended up involved with its development too. But it has its bugs, and because it uses templates extensively, it’s awful slow to compile an application that uses it. I’ve changed to wxPython, which is a very mature GUI library – and cross platform to boot.

Of course, there are some downsides with the change. The whole software package is much bigger due to the dependencies (the installer jumped from 900K to 5MB!). And the program is a little bit slower, mainly when starting up, and uses more memory (9MB to 30MB with no images loaded). But I think it was a good enough trade-off.

Rewriting from scratch must be considered carefully, and in this case, I think it was a good idea. Quivi is a hobby project, and I guess the main point of it is to have fun writing it, and to make users happy. I hope I can do both with this new version (which will be released soon).

"Access is denied" on "My Pictures" in Windows Vista

Just for the record, in case anyone gets stuck with this issue.

The “C:/Users/username/Documents” folder in Vista has three hidden junctions named “My Pictures”, “My Music” and “My Videos” which point to the folders “Pictures”, “Music” and “Video” inside the “C:/Users/username” folder. But they have the “List folder / read data” permission denied for all users as default. So any attempt to open them will result in a “Access denied” error.

It’s possible to “fix” this, but I don’t recommend it. Just open the pointed folders.

The question is, of course, why the permission is set, and why there are those hidden junctions in the first place?

I’m only guessing here, but it makes sense. The junctions are there for compatibility. If a program was written with a reference to a file inside the “(My Documents Folder)/My Pictures” path (assuming the “My Pictures” folder would always be inside the My Documents folder), it would break on Vista. With the junction, you can write to or read any files inside this (now non existent) folder.

(That is why there is also a junction “C:\Documents and Settings” pointing to “C:\Users”)

And why is the list folder permission denied? At first I thought it was there to prevent an infinite loop on programs who list the directory tree recursively, but those junctions don’t point to parent directories…

And the last question is, of course, why they changed those folders locations in the first place? Well, I must admit it’s much better this way (spaces inside paths are a pain).