So I was inspired (distracted) by the python functional programming module, and got to thinking couldn't things like partial application and function composition be a bit more natural in the syntax.
So it turns out that with decorators and a bit of evil introspection I came up with something that works (at least for functions that don't have keyword arguments). So you can do something like:
@Functional
def add(a, b): return a + b
add3 = add.add
assert(add3(1)(2)(3) == add3(1, 2, 3) == add3(1, 2)(3) == add3(1)(2, 3) == 6)
So what is the magic (or evilness)? Well first we (ab)use decorators to decorate things as Functional, which is you probably know, is just a shortcut for function = Functional(function).
The evil is in the Functional class:
class Functional:
def __init__(self, fn, nargs = None):
self.fn = fn
if not nargs:
_args, _, _, _ = inspect.getargspec(self.fn)
self.nargs = len(_args)
else:
self.nargs = nargs
def __call__(self, *args, **kargs):
if len(args) > self.nargs:
res = self.fn(*args[:self.nargs], **kargs)
if res.__class__ == Functional:
return res(*args[self.nargs:])
if type(res) != type((0,)):
res = (res, )
return res + args[self.nargs:]
elif len(args) == self.nargs:
return self.fn(*args, **kargs)
else:
def newfunc(*fargs, **fkeyw):
return self.fn(*(args + fargs))
newfunc.func = self.fn
newfunc.args = args
return Functional(newfunc, self.nargs - len(args))
def __getattr__(self, name):
if hasattr(self.fn, name):
return getattr(self.fn, name)
func_1 = self.fn
func_2 = globals()[name]
def composition(*args, **kwargs):
res = func_2(*args, **kwargs)
if type(res) != type((0,)):
res = (res, )
return self(*res)
return Functional(composition, func_2.nargs)
I totally abuse the __getattr__ method so that dot becomes a composition operator. This returns a new function (which is also Functional), which when called will pass on the return value from the first function to the second function. If the first function returns multiple results each of these is passed as arguments to the second function.
The real magic comes in the overload __call__, which is where
partial functions are hacked in. Basically if not enough arguments are
passed to the function is returns a new function that accumulates the
arguments already passed and once it gets enough calls the original function.
Of course the function returned from partial application is itself. The real
evil is in supporting the case add3(1, 2, 3) which means
we detect if too many arguments are passed and then call with only the first
arguments, then if the called function returns another functional we apply the remaining
arguments to it.
Oh yeah, I wouldn't use this in any real python code, as it is likely to confuse everyone!
Ok, pexif 0.4 is out (even after saying th other day that I wouldn't do any more releases for a while!), and I've updated the pyannodex 0.7.3 tarball to include a missing header file. (I didn't really think that deserved a version bump!).
I've released 0.7.3 of the pyannodex bindings. This fixes some bugs, importantly includes the config_unix.py script which is kind of essential for installing it. It also includes anxgrep.py tool.
pexif hits 0.3. Ok, there is now an easy way to get and set GPS coordinates on a photo, which was really my aim in writing this library in the first place. There is also a setgps.py and getgps.py script to do it on the command line.
I don't plan on doing any more major stuff to this library in the near future unless someone other than me is actually using it, so more than likely this will be the final pexif release. Itch scratched.
of source code is what Gernot Heiser, John Ferlito, Conrad Parker, Jamie Wilkinson, myself, and a host of around 20 other other, mostly ex-UNSW students just paid for a copy of the revered Lions' Book.
That's right, between us, we kicked in $10,000 Australian dollars to buy a copy of the book which has been signed by both Ken Thompson and Dennis Ritchie of UNIX fame, along with Kirk McKusick who wrote the Design and implementation of the 4.3BSD UNIX Operating System, Eric Allman author of sendmail, and Linus Torvalds creator of the Linux kernel. It also had the signatures of all the speakers from the 2006 linux.conf.au.
Of course with any LCA bidding process there were some other non material rewards on offer. With the $10,000 bid we also won a lot of hair, including Rusty's mustache, Jdub's hair, and Greg "Groggy" Lehey's beard! OMG! LOL!
Oh yeah... so why were people happy to offer such money and go to such lengths, on one side, it is the awesome respect people have to John Lions, who is a leading light in the idea of open source UNIX. And also I think the respect graduates have for UNSW's School of Computer Science and Engineering. The money from the auction will go towards creating a perpetual chair in John Lions' name. And even better, Linux Australia is contributing massively by matching the bid, raising the donation to $20,000 and USENIX is doubling this again to give a grand total of $40,000 towards the Opearting Systems chair at UNSW. This is pretty impressive! Hopefully by the time linux.conf.au rolls around next year, we will be able to announce that the chair is fully funded!
Oh, and a big thanks to John maddog Hall for organising the auction and the prizes.
Short version: I release some code. It edits EXIF data.
Long version: All I wanted to do was write some simple code to add a GPS location take to my photos. It should have been easy. A couple of hours of scripting at the most. But it wasn't.
Most JPEG images have some meta data at the front of them telling you stuff like, when it was taken, what camera took it, wether the flash was used, and so on, and so on. The latest version of the spec also has a bunch of fields for storing GPS information. Unfortunately my cheapo camera doesn't store that kind of information, so I want to add it in after the fact. I thought I could just grab an existing EXIF editing library and away I would go.
Unfortunately the two Python libraries out there pyexif and EXIF.py, only handle the parsing of EXIF data, not the updating of it. The also seemed to be a C library libexif that claimed to do editing, but it was basically undocumented, I couldn't work out how to use it, and besides I wanted to write in python not C. (And I didn't feel like writing a Python wrapper for a library I didn't uderstand.)
This led me to writing some of my own Python code to try and just do enough to insert the GPS tags and nothing else. Unfortunately the way EXIF works things aren't quite that easy. Inside the file format you end up with internal offset pointers, which means if you change the layout (by inserting some extra tags), you end up needing to change all these pointers to work with the new offsets.
I thought this wouldn't be too bad because I'd just append to the end of the section I was working with, (which was unfortuantely first), and then the other sections could move freely. Unfortuantely these pointers don't reference from the start of a section, they reference from the start of the file. This basically means that I need to parse everything into memory, and then know how to write it all back out again. Pain, pain, pain. (This was of course a bit of a simplication, so please don't email me saying that isn't exactly how EXIF is structured.)
Anyway, I now have a library, which can read all the EXIF data, including the thumbnail section, edit and insert values into it, and then spit out a valid file again. Now I can start writing the code to do the actual tagging, which should be more interesting than this stuff, but right now, time for the LCA dinner.
In the morning I attended the writing a gcc front-end tutorial. This was really interesting to find out about the internals of GCC, and also got into some of the new features gcc will have in the future. While being intersting, I think if I was writing a new language right now, I would still do the classic approach of outputing C (or maybe C++) and then passing that to the normal compiler.
Jimi Xenidis gave a cool talk about the new Cell processor and (failed) attempts to optimise the zeroing of pages. The basic idea of this work is to zero out pages in the background and keeping a set of zeroed pages ready, rather than zeroing on demand.
Unfortunately, it is really quick to zero a page, because of the
dcbz instruction, which invalidates cache-lines without
having to touch memory. This instruction has a 1 cycle latency so for
4kb pages, they can be zeroed in 32 cycles. Unfortunately the
instructions to get the DMA engine to zero pages is almost 400
cycles. It was actually pretty encouraging that the overhead was only
around 1%, which is really just in the noise.
The really interesting thing would be if they used bigger
pages. For example, if you were using 4MB pages, instead of 4kb pages
the dcbz aprroach would be slower, and would actually
suck more because it would probably require writeback. Hopefully once
the Gelato guys will get
their super page stuff into the kernel soon.
The afternoon had the excitement of the Linux australia AGM, where it was announced that linux.conf.au 2007 would be held in Sydney. This was followed up by a dinner meeting between the LA and linux.conf.au committees at Table 7.
Dinner drinks were hosted by the Google hiring squad, who basically put a big tab on a bar in the city for us, which basically meant free beer for the whole conference.
For a hastily prepared talk this went pretty well. I demoed the PLEB -- showing off hardware is always good for getting the interested level up. Of course I managed to screw up the demo, by failing to actually turn the external power supply on! There were enough questions to think that people were actually listening to the talk, which is always encouraging.
I have the slides available if anyone is interested.
On Tuesday I attended the miniconf, which was pretty fun. Mark Philips provided a great inspiration talk about why working in the embedded is the cool place to be. Most of the talk focussed on robotics, and had great examples of current robotics stuff.
Paul Campbell from Digeo presented on set-top boxes using Linux. The main interesting thing from this talk was the use of a secure boot loader to get into Linux so people can't ripoff any content stored on the set-top box.
Ian Walters from Trolltech gave an update on Qtopia. This stuff looked kind of interesting, they have basically come up with their own programming environment which totally abstracts away POSIX which is underneath. This would be an interesting system to run directly on top of Iguana.
Yutaka Niibe gave a really cool talk on using a USB hub to control AC devices. In Japan they have a much higher respect than we do here, (they signed the Kyoto protocol after all), and care about things like turning off peripheral devices when they are not in use, so they have USB controlled power cables.
There was a bunch of cool hardware shown off, but probably the most interesting one is the gumstix hardware. It would definately nice to get some of these with L4 running on it.
This year the embedded miniconf was more a microconf and only ran for the one morning. So in the afternoon I went along to the GNOME miniconf, it was really cool to see CryoPID which does process checkpointing in Linux.
Dinner was at the Terrace where they serve beer in 3 litre glass tubes, which have there own taps. Pretty damn cool! The meat is served raw on hot rocks which is a pretty cool idea, but I'm not too sure if serving people 3 litres of beer and then giving them boiling rocks is the best idea!