Android Emulator Internals — Bus Scanning

Tue, 09 Feb 2010 15:14:27 +0000
tech android

Wow, I can’t believe it has been over two years since I last wrote about Android’s for of the QEMU emulator. Turns out there have been some changes since I last looked at it.

The most important is that the Android emulator no longer has a fixed layout of devices in the physical memory address space. So, while it may have previously been the case that the event device was always at 0xff007000, now it might be at 0xff008000, or 0xff009000, depending on what other devices have been configured for a particular device configuration.

Now, if a device may exist at some random physical address, how does the OS know how to setup the devices drivers? Well, as I’m sure you’ve guessed, the addresses and really random, they are located at page-offset addresses through a restricted range of memory. OK, so how does the OS know what the range is? Well, there is the goldfish_device_bus device.

Basically, this device provides a mechanism to enumerate the devices on the bus. The driver writes PDEV_BUS_OP_INIT to the PDEV_BUS_OP register, the goldfish_device_bus then raises an interrupt. The driver the reads the PDEV_BUS_OP register. Each time the value is PDEV_BUS_OP_ADD_DEV, the driver can read the other registers such as PDEV_BUS_IO_BASE, PDEV_BUS_IO_SIZE, PDEV_BUS_IRQ, to determine the properties of the new device. It continues doing this until it reads a PDEV_BUS_OP_DONE, which indicates the bus scan has finished.

The driver can determine what type of device it has found by writing a pointer to the PDEV_BUS_GET_NAME register. When this happens the device writes an the device’s name (as an ASCII string) to the pointer.

Linux uses these strings to perform device to driver matching as described in the Platform Devices and Drivers document.

IMAP passwords in Android

Sun, 04 Jan 2009 08:47:54 +0000
tech android article

I’ve been setting up my new Android phone and found out that the password handling code in the IMAP client doesn’t work so well with my Dovecot IMAP server.

Now, I really can’t be bothered working out which side is doing it wrong, but Dovecot expects your password to be contained in double-quotes if it contains special character. (No, I don’t precisely know what those characters are!). And, of course, if you are double-quoing the string, you then need to escape and double-quotes in the password itself. Now, like I said, no idea of this is the correct (according to the standard) behaviour, but it is the behaviour that I have to deal with, since I can’t mess with the server. Of course, the Android IMAP client doesn’t do this escaping, and so you get an error message indicating your username / password is incorrect. Frustratingly, the error message doesn’t pass on the information from the server giving detail on exactly what the problem is so you are left guessing.

Anyway, it turns out if you manually escape the password that you give to the Android email client things work fine. Of course, the SMTP server doesn’t need this escaping, and fails if you do have it, so you need to type in a different, unescaped, password for SMTP. Fun and games!

Looking at the lastest source code for Android, it looks like this has been properly fixed, so hopefully and upgrade to the Cupcake branch in the near future will solve the problem.

Android booting on Neo 1973

Sun, 02 Nov 2008 16:03:45 +0000
android arm tech

Well, it started almost a year ago, but I finally now have Android booting on my Neo 1973 phone:

It ain’t exactly running fast yet, and not everything it working 100%, but I think most of the tricky bits are done. I’m starting to push most of these changes back to the android project. It seems that while I’ve been working on this Sean McNeill has been having similar successes getting Android up on the latest Freerunner phones.

Android on ARMv4 (take 2)

Mon, 27 Oct 2008 21:36:32 +0000
android tech article arm

So, my earlier post on this was a little premature; anyone who has tried out the code has found out that it pretty much doesn’t work (hey I did warn you!). Now there are a range of fun reasons why this didn’t work, most of which I’ve now solved.

Firstly, it turns out that EABI and ARMv4T are pretty much incompatible. (I’ll post separately about that!). In short, thumb interworking doesn’t (can’t) work, so I’ve reverted back to plain old ARMv4 architecture as my target (the only difference between ARMv4 and ARMv4T is the thumb stuff, which we can’t use until the compiler / spec is fixed.). So I’ve updated the linux-arm.mk to support ARMv4 for now as well.

Of course the next problem that this introduces is that the bx instruction doesn’t exist on ARMv4, and GCC (helpfully) complains and stops the compilation. Now a BX without thumb support is simply a mov pc, instruction, so I went through and provided a BX macro that expands to either bx or mov pc,. This is a little bit nasty/invasive because it touches all the system call bindings, thankfully these are generated anyway, but it makes the diff quite large. (When I have time I’ll make it so that generation is part of the buid system, not a manual process.)

The next problem is that the provided compiler’s libgcc library is build for ARMv5, and has instructions that just don’t exist on ARMv4 (shc as clz), so I went and built a new compiler targeted to ARMv4. There is no reason why this couldn’t be set up as a multi-lib compiler that supports both, but I don’t have enough GCC wizardry in me to work that out right now. So a new compiler.

This got things to a booting stage, but not able to mount /system or /data. Basically, Android by default uses yet another flash file-system (YAFFS), but for some reasons, which I couldn’t fully work out initially, the filesystem just didn’t seem to cleanly initialise and then mount. So, without diving too deep, I figured I could just use jffs2 instead, which I know works on the target. So I upgraded the Android build system to support allowing you to choose which filesystem type to use, and providing jffs2 as an option. This was going much better, and I got a lot further, far enough that I needed to recompile my kernel with support for some of the Android specific drivers like ashmem, binder and logger. Unfortunately I was getting a hang on an mmap call, for reasons that I couldn’t quite work out. After a lot of tedious debugging (my serial console is broken, so I have to rely on graphics console, which is really just an insane way to try and debug anything), anyway, it turns out that part of what the Dalvik virtual machine does when optimising class files is to mmap the file as writable memory. This was what was failing, with the totally useless error invalid argument. Do you know how many unique paths along the mmap system call can set EINVAL? Well it’s a lot. Anyway, long story short, it turns out that the jffs2 filesystem doesn’t support writable mmaps! %&!#.

After I finished cursing, I decided to go back to using yaffs and working out what the real problem is. After upgrading u-boot (in a pointless attempt to fix my serial console), I noticed a new write yaffs[1] command. This wasn’t there in the old version. Ok, cool, maybe this has something do to with the problem. But what is this the deal with yaffs versus yaffs1? Well it turns out that NAND has different pagesize, 512 bytes, and 2k (or multiples thereof, maybe??). And it turns out that YAFFS takes advantage of this and has different file systems for different sized NAND pages, and of course, everything that can go wrong will so, the filesystem image that the build system creates is YAFFS2 which is for 2k pages not 512b pages. So, I again updated the build system to firstly build both the mkyaffs2image and the mkyaffsimage tool, and then set off building a YAFFS file system.

Now, while u-boot supports yaffs filesystem, device firmware update doesn’t (appear to). So this means I need to copy the image to memory first, then on the device copy it from memory to flash. Now, the other fun thing is that dfu can only copy 2MB or so to RAM at a time, and the system.img file is around 52MB or so, which means that it takes around 26 individual copies of 2MB sections.... very, very painful. But in the end this more or less worked. So now I have a 56MB partition for the system, and a 4MB partition for the user and things are looking good.

Good that is, right up until the point where dalvik starts up and writes out cached version of class files to /data. You see, it needs more than 4MB, a lot more, so I’m kind of back to square one. I mean, if I’d looked at the requirements I would have read 128MB of flash, but meh, who reads requirements? The obvious option would be some type of MMC card, but as it turns out the number of handy Fry’s stores on Boeing 747 from Sydney to LA number in the zeroes.

So the /system partition is read-only, and since the only problem with jffs2 was when we were writing to it, it seems that we could use jffs2 for the read-only system partition, which has the advantage of jffs2 doing compression, and fitting in about 30MB, not about 50MB, leaving plenty of room for the user data partition, which is where the Dalvik cached files belong. This also has the advantage of being able to use normal DFU commands to install the image (yay!). So after more updates to the build system to now support individually setting the system filesystem type and the user filesystem type things seem a lot happier.

Currently, I have a system that boots init, starts up most of the system services, including the Dalvik VM, runs a bunch of code, but bombs out with an out-of-memory error in the pixelflinger code which I’m yet to have any luck tracing. Currently my serial console is fubar, so I can’t get any useful logging, which makes things doubly painful. The next step is to get adb working over USB so I have at least an output of the errors and warning, which should give me half a chance of tracking down the problem.

So if you want to try and get up to this point, what are the steps? Well, firstly go and download the android toolchain source code. and compile it for a v4 target. You use the --target=armv4-android-eabi argument to configure if I remember correctly.

Once you have that done, grab my latest patch and apply it to the Android source code base. (That is tar file with diffs for each individual project, apply these correctly is left as an exercise for the reader). Then you want to compile it with the new toolchain. I use a script like this:

#!/bin/sh

make TARGET_ARCH_VERSION=armv4 \
     MKJFFS2_CMD="ssh nirvana -x \"cd `pwd`; mkfs.jffs2\""  \
     SYSTEM_FSTYPE=jffs2 \
     USERDATA_FSTYPE=yaffs \
     TARGET_TOOLS_PREFIX=/opt/benno/bin/armv4-android-eabi- $@

Things you will need to change it the tools prefix, and the mkjffs2 command. The evil-hackery above is to run it on my linux virtual machine (I’m compiling the rest under OS X, and I can’t get mkfs.jffs2 to compile under it yet.)

After some time passes you should end up with a ramdisk.img, userdata.img and system.img files. The next step is to get a usable kernel.

I’m using the OpenMoko stable kernel, which is 2.6.24 based. I’ve patched this with bits of the Android kernel (enough, I think, to make it run). Make sure you configure support for yaffs, binder, logger and ashmem. Here is the kernel config I’m currently using.

At this stage it is important you have a version of u-boot supporting the yaffs write commands, if you don’t your next step is to install that. After this the next step is to re-partition your flash device. In case it isn’t obvious this will trash your current OS. The useful parts from my uboot environment are:

mtdids=nand0=neo1973-nand
bootdelay=-1
mtdparts=mtdparts=neo1973-nand:256k(uboot)ro,16k(uboot-env),752k(ramdisk),2m(kernel),36m(system),24m(userdata)
rdaddr=0x35000000
kaddr=0x32000000
bootcmd=setenv bootargs ${bootargs_base} ${mtdparts} initrd=${rdaddr},${rdsize}; nand read.e ${kaddr} kernel; nand read.e ${rdaddr} ramdisk; bootm ${kaddr}
bootargs_base=root=/dev/ram rw console=tty0 loglevel=8

Note the mtdparts which defines the partitions, and the bootcmd. (I’m not entirely happy with the boot command, mostly because when I install new RAM image I need to manually update $rdsize, which is a pain).

With this in place you are ready to start. The first image to move across is your userdata image. Now to make this happen we first copy it into memory using dfu-util:

sudo dfu-util -a 0 -R -D source/out/target/product/generic/userdata.img  -R

Then you need to use the nand write.yaffs1 command to copy it to the data partition. Note, at this stage I get weird behaviour, I’m not convinced that the yaffs support truly works yet! Afterwards I get some messed up data in other parts of the flash (which is why we are doing it first). After you have copied it in, I suggest reseting the device, and you may find you need to reinitialise u-boot (using dyngen, and resetting up the environment as above.

After this you are good to use dfu-util to copy accross the kernel, system.img and ramdisk.img. After copying the ramdisk.img across update the rdsize variable with the size of the ramdisk.

Once all this is done, you are good to boot, I wish you luck! If you have a working serial console you can probably try the logcat command to see why graphics aren’t working. If you get this far please email me the results!

Compiling the Android source code for ARMv4T

Thu, 23 Oct 2008 23:02:13 +0000
tech article android arm

After a lot of stuffing around installing new hard drives so I had enough space to actually play with the source code, getting screwed by Time Machine when trying to convert my filesystem from case-insenstive to case-insensitive (I gave up and am now usuing a case-sensitive disk image on top of my case-insenstive file system.. sigh), I finally have the Android source code compiling, yay!.

Compiling is fairly trivial, just make and away it goes. The fun thing is trying to work out exactly what the hell the build system is actually doing. I’ve got to admit though, it is a pretty clean build system, although it isn’t going to win any speed records. I’m going to go into more details on the build sstem when i have more time, and I’ve actually worked out what the hell is happening.

Anyway, after a few false starts I now have the build system compiling for ARMv4T processors (such as the one inside the Neo1973), and hopefully at the same time I haven’t broken compilation from ARMv5TE.

For those interested I have a patch available. Simply apply this to the checked out code, and the build using make TARGET_ARCH_VERSION=armv4t. Now, of course I haven’t actually tried to run this code yet, so it might not work, but it seems to compile fine, so that is a good start! Now once I work out how to make git play nice I'll actually put this into a branch and make it available, but the diff will have to suffice for now. Of course I’m not the only one looking at this, check out Christopher’s page for more information. (Where he actually starts solving some problems instead of just working around them ;)

The rest of this post documents the patch. For those interested it should give you some idea of the build system and layout, and hopefully it is something that can be applied to mainline.

The first changes made are to the linux-arm.mk file. A new make variable TARGET_ARCH_VERSION is added. For now this is defaulted to armv5te, but it can be overridden on the command line as shown above.

project build/
diff --git a/core/combo/linux-arm.mk b/core/combo/linux-arm.mk
index adb82d3..a43368f 100644
--- a/core/combo/linux-arm.mk
+++ b/core/combo/linux-arm.mk
@@ -7,6 +7,8 @@ $(combo_target)TOOLS_PREFIX := \
 	prebuilt/$(HOST_PREBUILT_TAG)/toolchain/arm-eabi-4.2.1/bin/arm-eabi-
 endif
 
+TARGET_ARCH_VERSION ?= armv5te
+
 $(combo_target)CC := $($(combo_target)TOOLS_PREFIX)gcc$(HOST_EXECUTABLE_SUFFIX)
 $(combo_target)CXX := $($(combo_target)TOOLS_PREFIX)g++$(HOST_EXECUTABLE_SUFFIX)
 $(combo_target)AR := $($(combo_target)TOOLS_PREFIX)ar$(HOST_EXECUTABLE_SUFFIX)

The next thing is to make the GLOBAL_CFLAGS variable dependent on the architecture version. The armv5te defines stay in place, but an armv4t architecture version is added. Most of the cflags are pretty similar, except we change the -march flag, and change the pre-processor defines. These will become important later in the patch as they provide the mechanism for distinguishing between versions in the code.

@@ -46,6 +48,7 @@ ifneq ($(wildcard $($(combo_target)CC)),)
 $(combo_target)LIBGCC := $(shell $($(combo_target)CC) -mthumb-interwork -print-libgcc-file-name)
 endif
 
+ifeq ($(TARGET_ARCH_VERSION), armv5te)
 $(combo_target)GLOBAL_CFLAGS += \
 			-march=armv5te -mtune=xscale \
 			-msoft-float -fpic \
@@ -56,6 +59,21 @@ $(combo_target)GLOBAL_CFLAGS += \
 			-D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ \
 			-D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__ \
 			-include $(call select-android-config-h,linux-arm)
+else
+ifeq ($(TARGET_ARCH_VERSION), armv4t)
+$(combo_target)GLOBAL_CFLAGS += \
+			-march=armv4t \
+			-msoft-float -fpic \
+			-mthumb-interwork \
+			-ffunction-sections \
+			-funwind-tables \
+			-fstack-protector \
+			-D__ARM_ARCH_4__ -D__ARM_ARCH_4T__ \
+			-include $(call select-android-config-h,linux-arm)
+else
+$(error Unknown TARGET_ARCH_VERSION=$(TARGET_ARCH_VERSION))
+endif
+endif
 
 $(combo_target)GLOBAL_CPPFLAGS += -fvisibility-inlines-hidden

The next bit we update is the prelink-linux-arm.map file. The dynamic libraries in android are laid out explicitly in virtual memory according to this map file. If I’m not mistaken those address look suspiciously 1MB aligned, which means they should fit nicely in the pagetable, and provides some opportunity to use fast-address-space-switching techniques. In the port to ARMv4 I have so far been lazy and instead of fixing up any assembler code I’ve just gone with existing C code. One outcome of this is that I need the libffi.so for my foreign function interface, so I’ve added this to the map for now. I’m not 100% sure that when compiling for ARMv5 this won’t cause a problem. Will need to see. Fixing up the code to avoid needing libffi is probably high on the list of things to do.

diff --git a/core/prelink-linux-arm.map b/core/prelink-linux-arm.map
index d4ebf43..6e0bc43 100644
--- a/core/prelink-linux-arm.map
+++ b/core/prelink-linux-arm.map
@@ -113,3 +113,4 @@ libctest.so             0x9A700000
 libUAPI_jni.so          0x9A500000
 librpc.so               0x9A400000 
 libtrace_test.so        0x9A300000 
+libffi.so               0x9A200000


The next module is the bionic module which is the light-weight C library that is part of Android. This has some nice optimised routines for memory copy and compare, but unfortunately they rely on ARMv5 instructions. I’ve changed the build system to only use the optimised assembler when compiling with ARMv5TE, and falling back to C routines in the other cases. (The strlen implementation isn’t pure assembly, but the optimised C implementation has inline asm, so again it needs to drop back to plain old dumb strlen.)

project bionic/
diff --git a/libc/Android.mk b/libc/Android.mk
index faca333..3fb3455 100644
--- a/libc/Android.mk
+++ b/libc/Android.mk
@@ -206,13 +206,9 @@ libc_common_src_files := \
 	arch-arm/bionic/_setjmp.S \
 	arch-arm/bionic/atomics_arm.S \
 	arch-arm/bionic/clone.S \
-	arch-arm/bionic/memcmp.S \
-	arch-arm/bionic/memcmp16.S \
-	arch-arm/bionic/memcpy.S \
 	arch-arm/bionic/memset.S \
 	arch-arm/bionic/setjmp.S \
 	arch-arm/bionic/sigsetjmp.S \
-	arch-arm/bionic/strlen.c.arm \
 	arch-arm/bionic/syscall.S \
 	arch-arm/bionic/kill.S \
 	arch-arm/bionic/tkill.S \
@@ -274,6 +270,18 @@ libc_common_src_files := \
 	netbsd/nameser/ns_print.c \
 	netbsd/nameser/ns_samedomain.c
 
+
+ifeq ($(TARGET_ARCH),arm)
+ifeq ($(TARGET_ARCH_VERSION),armv5te)
+libc_common_src_files += arch-arm/bionic/memcmp.S \
+		arch-arm/bionic/memcmp16.S \
+		arch-arm/bionic/memcpy.S \
+		arch-arm/bionic/strlen.c.arm
+else
+libc_common_src_files += string/memcmp.c string/memcpy.c string/strlen.c string/ffs.c
+endif
+endif
+
 # These files need to be arm so that gdbserver
 # can set breakpoints in them without messing
 # up any thumb code.

Unfortunately, it is clear that this C only code hasn’t been used in a while as there was a trivial bug as fixed by the patch below. This makes me worry about what other bugs that aren’t caught by the compiler may be lurking.

diff --git a/libc/string/memcpy.c b/libc/string/memcpy.c
index 4cd4a80..dea78b2 100644
--- a/libc/string/memcpy.c
+++ b/libc/string/memcpy.c
@@ -25,5 +25,5 @@
  * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  */
-#define MEM_COPY
+#define MEMCOPY
 #include "bcopy.c"

Finally, frustratingly, the compiler’s ffs() implementation appears to fallback to calling the C library’s ffs() implementation if it can’t doing something optimised. This happens when compiling for ARMv4, so I’ve added an ffs() implementation (stolen from FreeBSD).

#include 
#include 

/*
 * Find First Set bit
 */
int
ffs(int mask)
{
        int bit;

        if (mask == 0)
                return (0);
        for (bit = 1; !(mask & 1); bit++)
                mask = (unsigned int)mask >> 1;
        return (bit);
}

The next module for attention is the dalvik virtual machine. Again this has some code that relies on ARMv5, but there is a C version that we fall back on. In this case it also means pulling in libffi. This is probably the module that needs to most attention in actually updating the code to be ARMv4 assembler in the near future.

project dalvik/
diff --git a/vm/Android.mk b/vm/Android.mk
index dfed78d..c66a861 100644
--- a/vm/Android.mk
+++ b/vm/Android.mk
@@ -189,6 +189,7 @@ ifeq ($(TARGET_SIMULATOR),true)
 endif
 
 ifeq ($(TARGET_ARCH),arm)
+ifeq ($(TARGET_ARCH_VERSION),armv5te)
 	# use custom version rather than FFI
 	#LOCAL_SRC_FILES += arch/arm/CallC.c
 	LOCAL_SRC_FILES += arch/arm/CallOldABI.S arch/arm/CallEABI.S
@@ -204,6 +205,16 @@ else
 		mterp/out/InterpC-desktop.c \
 		mterp/out/InterpAsm-desktop.S
 	LOCAL_SHARED_LIBRARIES += libffi
+	LOCAL_SHARED_LIBRARIES += libdl
+endif
+else
+	# use FFI
+	LOCAL_C_INCLUDES += external/libffi/$(TARGET_OS)-$(TARGET_ARCH)
+	LOCAL_SRC_FILES += arch/generic/Call.c
+	LOCAL_SRC_FILES += \
+		mterp/out/InterpC-desktop.c \
+		mterp/out/InterpAsm-desktop.S
+	LOCAL_SHARED_LIBRARIES += libffi
 endif
 
 LOCAL_MODULE := libdvm

Next is libjpeg, which again, has assembler optimisation that we can’t easily use without real porting work, so we fall back to the C

project external/jpeg/
diff --git a/Android.mk b/Android.mk
index 9cfe4f6..3c052cd 100644
--- a/Android.mk
+++ b/Android.mk
@@ -19,6 +19,12 @@ ifneq ($(TARGET_ARCH),arm)
 ANDROID_JPEG_NO_ASSEMBLER := true
 endif
 
+# the assembler doesn't work for armv4t
+ifeq ($(TARGET_ARCH_VERSION),armv4t)
+ANDROID_JPEG_NO_ASSEMBLER := true
+endif
+
+
 # temp fix until we understand why this broke cnn.com
 #ANDROID_JPEG_NO_ASSEMBLER := true
 

For some reason compiling with ARMv4 doesn’t allow the prefetch loop array compiler optimisation, so we turn it off for ARMv4.

@@ -29,7 +35,10 @@ LOCAL_SRC_FILES += jidctint.c jidctfst.S
 endif
 
 LOCAL_CFLAGS += -DAVOID_TABLES 
-LOCAL_CFLAGS += -O3 -fstrict-aliasing -fprefetch-loop-arrays
+LOCAL_CFLAGS += -O3 -fstrict-aliasing
+ifeq ($(TARGET_ARCH_VERSION),armv5te)
+LOCAL_FLAGS += -fprefetch-loop-arrays
+endif
 #LOCAL_CFLAGS += -march=armv6j
 
 LOCAL_MODULE:= libjpeg

Next up is libffi, which is just a case of turning it on since we now need it for ARMv4.

project external/libffi/
diff --git a/Android.mk b/Android.mk
index f4452c9..07b5c2f 100644
--- a/Android.mk
+++ b/Android.mk
@@ -6,7 +6,7 @@
 # We need to generate the appropriate defines and select the right set of
 # source files for the OS and architecture.
 
-ifneq ($(TARGET_ARCH),arm)
+ifneq ($(TARGET_ARCH_VERSION),armv5te)
 
 LOCAL_PATH:= $(call my-dir)
 include $(CLEAR_VARS)

The external module opencore contains a lot of software implemented codecs. (I wonder about the licensing restrictions on these things...). Not surprisingly these too are tuned for ARMv4, but again we fall back to plain old C.

project external/opencore/
diff --git a/codecs_v2/audio/aac/dec/Android.mk b/codecs_v2/audio/aac/dec/Android.mk
index ffe0089..6abdc2d 100644
--- a/codecs_v2/audio/aac/dec/Android.mk
+++ b/codecs_v2/audio/aac/dec/Android.mk
@@ -150,7 +150,7 @@ LOCAL_SRC_FILES := \
 LOCAL_MODULE := libpv_aac_dec
 
 LOCAL_CFLAGS := -DAAC_PLUS -DHQ_SBR -DPARAMETRICSTEREO  $(PV_CFLAGS)
-ifeq ($(TARGET_ARCH),arm)
+ifeq ($(TARGET_ARCH_VERSION),armv5te)
  LOCAL_CFLAGS += -D_ARM_GCC
  else
  LOCAL_CFLAGS += -DC_EQUIVALENT
diff --git a/codecs_v2/audio/gsm_amr/amr_wb/dec/Android.mk b/codecs_v2/audio/gsm_amr/amr_wb/dec/Android.mk
index e184178..3223841 100644
--- a/codecs_v2/audio/gsm_amr/amr_wb/dec/Android.mk
+++ b/codecs_v2/audio/gsm_amr/amr_wb/dec/Android.mk
@@ -48,7 +48,7 @@ LOCAL_SRC_FILES := \
 LOCAL_MODULE := libpvamrwbdecoder
 
 LOCAL_CFLAGS :=   $(PV_CFLAGS)
-ifeq ($(TARGET_ARCH),arm)
+ifeq ($(TARGET_ARCH_VERSION),armv5te)
  LOCAL_CFLAGS += -D_ARM_GCC
  else
  LOCAL_CFLAGS += -DC_EQUIVALENT
diff --git a/codecs_v2/audio/mp3/dec/Android.mk b/codecs_v2/audio/mp3/dec/Android.mk
index 254cb6b..c2430fe 100644
--- a/codecs_v2/audio/mp3/dec/Android.mk
+++ b/codecs_v2/audio/mp3/dec/Android.mk
@@ -28,8 +28,8 @@ LOCAL_SRC_FILES := \
 	src/pvmp3_seek_synch.cpp \
 	src/pvmp3_stereo_proc.cpp \
 	src/pvmp3_reorder.cpp
-	
-ifeq ($(TARGET_ARCH),arm)
+
+ifeq ($(TARGET_ARCH_VERSION),armv5te)
 LOCAL_SRC_FILES += \
 	src/asm/pvmp3_polyphase_filter_window_gcc.s \
 	src/asm/pvmp3_mdct_18_gcc.s \
@@ -46,7 +46,7 @@ endif
 LOCAL_MODULE := libpvmp3
 
 LOCAL_CFLAGS :=   $(PV_CFLAGS)
-ifeq ($(TARGET_ARCH),arm)
+ifeq ($(TARGET_ARCH_VERSION),armv5te)
  LOCAL_CFLAGS += -DPV_ARM_GCC
  else
  LOCAL_CFLAGS += -DC_EQUIVALENT

Unfortunately it is not just the build file that needs updating in this module. I need to manually go and update the headers so that some optimised inline assembler is only used in the ARMv5 case. To be honest this messes these files up a little bit, so a nicer solution would be preferred.

diff --git a/codecs_v2/video/m4v_h263/enc/src/dct_inline.h b/codecs_v2/video/m4v_h263/enc/src/dct_inline.h
index 86474b2..41a3297 100644
--- a/codecs_v2/video/m4v_h263/enc/src/dct_inline.h
+++ b/codecs_v2/video/m4v_h263/enc/src/dct_inline.h
@@ -22,7 +22,7 @@
 #ifndef _DCT_INLINE_H_
 #define _DCT_INLINE_H_
 
-#if !defined(PV_ARM_GCC)&& defined(__arm__)
+#if !(defined(PV_ARM_GCC) && defined(__arm__) && defined(__ARCH_ARM_5TE__))
 
 #include "oscl_base_macros.h"
 
@@ -109,7 +109,7 @@ __inline int32 sum_abs(int32 k0, int32 k1, int32 k2, int32 k3,
 #elif defined(__CC_ARM)  /* only work with arm v5 */
 
 #if defined(__TARGET_ARCH_5TE)
-
+#error
 __inline int32 mla724(int32 op1, int32 op2, int32 op3)
 {
     int32 out;
@@ -266,7 +266,7 @@ __inline int32 sum_abs(int32 k0, int32 k1, int32 k2, int32 k3,
     return abs_sum;
 }
 
-#elif defined(PV_ARM_GCC) && defined(__arm__) /* ARM GNU COMPILER  */
+#elif defined(PV_ARM_GCC) && defined(__arm__) && defined(__ARCH_ARM_5TE__) /* ARM GNU COMPILER  */
 
 __inline int32 mla724(int32 op1, int32 op2, int32 op3)
 {
diff --git a/codecs_v2/video/m4v_h263/enc/src/fastquant_inline.h b/codecs_v2/video/m4v_h263/enc/src/fastquant_inline.h
index 6a35d43..fbfeddf 100644
--- a/codecs_v2/video/m4v_h263/enc/src/fastquant_inline.h
+++ b/codecs_v2/video/m4v_h263/enc/src/fastquant_inline.h
@@ -25,7 +25,7 @@
 #include "mp4def.h"
 #include "oscl_base_macros.h"
 
-#if !defined(PV_ARM_GCC) && defined(__arm__) /* ARM GNU COMPILER  */
+#if !(defined(PV_ARM_GCC) && defined(__arm__) && defined(__ARCH_ARM_V5TE__)) /* ARM GNU COMPILER  */
 
 __inline int32 aan_scale(int32 q_value, int32 coeff, int32 round, int32 QPdiv2)
 {
@@ -423,7 +423,7 @@ __inline int32 coeff_dequant_mpeg_intra(int32 q_value, int32 tmp)
     return q_value;
 }
 
-#elif defined(PV_ARM_GCC) && defined(__arm__) /* ARM GNU COMPILER  */
+#elif defined(PV_ARM_GCC) && defined(__arm__) && defined(__ARCH_ARM_V5TE__) /* ARM GNU COMPILER  */
 
 __inline int32 aan_scale(int32 q_value, int32 coeff,
                          int32 round, int32 QPdiv2)
diff --git a/codecs_v2/video/m4v_h263/enc/src/vlc_encode_inline.h b/codecs_v2/video/m4v_h263/enc/src/vlc_encode_inline.h
index 69857f3..b0bf46d 100644
--- a/codecs_v2/video/m4v_h263/enc/src/vlc_encode_inline.h
+++ b/codecs_v2/video/m4v_h263/enc/src/vlc_encode_inline.h
@@ -18,7 +18,7 @@
 #ifndef _VLC_ENCODE_INLINE_H_
 #define _VLC_ENCODE_INLINE_H_
 
-#if !defined(PV_ARM_GCC)&& defined(__arm__)
+#if !(defined(PV_ARM_GCC) && defined(__arm__) && defined(__ARCH_ARM_V5TE__))
 
 __inline  Int zero_run_search(UInt *bitmapzz, Short *dataBlock, RunLevelBlock *RLB, Int nc)
 {
@@ -208,7 +208,7 @@ __inline  Int zero_run_search(UInt *bitmapzz, Short *dataBlock, RunLevelBlock *R
     return idx;
 }
 
-#elif defined(PV_ARM_GCC) && defined(__arm__) /* ARM GNU COMPILER  */
+#elif defined(PV_ARM_GCC) && defined(__arm__) && defined(__ARCH_ARM_V5TE__) /* ARM GNU COMPILER  */
 
 __inline Int m4v_enc_clz(UInt temp)
 {

A similar approach is needed in the skia graphics library.

project external/skia/
diff --git a/include/corecg/SkMath.h b/include/corecg/SkMath.h
index 76cf279..5f0264f 100644
--- a/include/corecg/SkMath.h
+++ b/include/corecg/SkMath.h
@@ -162,7 +162,7 @@ static inline int SkNextLog2(uint32_t value) {
     With this requirement, we can generate faster instructions on some
     architectures.
 */
-#if defined(__arm__) && !defined(__thumb__)
+#if defined(__arm__) && defined(__ARM_ARCH_5TE__) && !defined(__thumb__)
     static inline int32_t SkMulS16(S16CPU x, S16CPU y) {
         SkASSERT((int16_t)x == x);
         SkASSERT((int16_t)y == y);

The sonivox module (no idea what that is!), has the same requirement of updating the build to avoid building ARMv5 specific code.

project external/sonivox/
diff --git a/arm-wt-22k/Android.mk b/arm-wt-22k/Android.mk
index 565c233..a59f917 100644
--- a/arm-wt-22k/Android.mk
+++ b/arm-wt-22k/Android.mk
@@ -73,6 +73,7 @@ LOCAL_COPY_HEADERS := \
 	host_src/eas_reverb.h
 
 ifeq ($(TARGET_ARCH),arm)
+ifeq (($TARGET_ARCH),armv5)
 LOCAL_SRC_FILES+= \
 	lib_src/ARM-E_filter_gnu.s \
 	lib_src/ARM-E_interpolate_loop_gnu.s \

The low-level audio code in audioflinger suffers from the same optimisations, and we need to dive into the code on this occasion to fix things up.

project frameworks/base/
diff --git a/libs/audioflinger/AudioMixer.cpp b/libs/audioflinger/AudioMixer.cpp
index 9f1b17f..4c0890c 100644
--- a/libs/audioflinger/AudioMixer.cpp
+++ b/libs/audioflinger/AudioMixer.cpp
@@ -400,7 +400,7 @@ void AudioMixer::process__validate(state_t* state, void* output)
 static inline 
 int32_t mulAdd(int16_t in, int16_t v, int32_t a)
 {
-#if defined(__arm__) && !defined(__thumb__)
+#if defined(__arm__) && defined(__ARCH_ARM_5TE__) && !defined(__thumb__)
     int32_t out;
     asm( "smlabb %[out], %[in], %[v], %[a] \n"
          : [out]"=r"(out)
@@ -415,7 +415,7 @@ int32_t mulAdd(int16_t in, int16_t v, int32_t a)
 static inline 
 int32_t mul(int16_t in, int16_t v)
 {
-#if defined(__arm__) && !defined(__thumb__)
+#if defined(__arm__) && defined(__ARCH_ARM_5TE__) && !defined(__thumb__)
     int32_t out;
     asm( "smulbb %[out], %[in], %[v] \n"
          : [out]"=r"(out)
@@ -430,7 +430,7 @@ int32_t mul(int16_t in, int16_t v)
 static inline 
 int32_t mulAddRL(int left, uint32_t inRL, uint32_t vRL, int32_t a)
 {
-#if defined(__arm__) && !defined(__thumb__)
+#if defined(__arm__) && defined(__ARCH_ARM_5TE__) && !defined(__thumb__)
     int32_t out;
     if (left) {
         asm( "smlabb %[out], %[inRL], %[vRL], %[a] \n"
@@ -456,7 +456,7 @@ int32_t mulAddRL(int left, uint32_t inRL, uint32_t vRL, int32_t a)
 static inline 
 int32_t mulRL(int left, uint32_t inRL, uint32_t vRL)
 {
-#if defined(__arm__) && !defined(__thumb__)
+#if defined(__arm__) && defined(__ARCH_ARM_5TE__) && !defined(__thumb__)
     int32_t out;
     if (left) {
         asm( "smulbb %[out], %[inRL], %[vRL] \n"
diff --git a/libs/audioflinger/AudioResamplerSinc.cpp b/libs/audioflinger/AudioResamplerSinc.cpp
index e710d16..88b8c22 100644
--- a/libs/audioflinger/AudioResamplerSinc.cpp
+++ b/libs/audioflinger/AudioResamplerSinc.cpp
@@ -62,7 +62,7 @@ const int32_t AudioResamplerSinc::mFirCoefsDown[] = {
 static inline 
 int32_t mulRL(int left, int32_t in, uint32_t vRL)
 {
-#if defined(__arm__) && !defined(__thumb__)
+#if defined(__arm__) && defined(__ARCH_ARM_5TE__) && !defined(__thumb__)
     int32_t out;
     if (left) {
         asm( "smultb %[out], %[in], %[vRL] \n"
@@ -88,7 +88,7 @@ int32_t mulRL(int left, int32_t in, uint32_t vRL)
 static inline 
 int32_t mulAdd(int16_t in, int32_t v, int32_t a)
 {
-#if defined(__arm__) && !defined(__thumb__)
+#if defined(__arm__) && defined(__ARCH_ARM_5TE__) && !defined(__thumb__)
     int32_t out;
     asm( "smlawb %[out], %[v], %[in], %[a] \n"
          : [out]"=r"(out)
@@ -103,7 +103,7 @@ int32_t mulAdd(int16_t in, int32_t v, int32_t a)
 static inline 
 int32_t mulAddRL(int left, uint32_t inRL, int32_t v, int32_t a)
 {
-#if defined(__arm__) && !defined(__thumb__)
+#if defined(__arm__) && defined(__ARCH_ARM_5TE__) && !defined(__thumb__)
     int32_t out;
     if (left) {
         asm( "smlawb %[out], %[v], %[inRL], %[a] \n"

The AndroidConfig.h header file is included on every compile. We mess with it to convince it that we don’t have an optimised memcmp16 function.

project system/core/
diff --git a/include/arch/linux-arm/AndroidConfig.h b/include/arch/linux-arm/AndroidConfig.h
index d7e182a..76f424e 100644
--- a/include/arch/linux-arm/AndroidConfig.h
+++ b/include/arch/linux-arm/AndroidConfig.h
@@ -249,8 +249,9 @@
 /*
  * Do we have __memcmp16()?
  */
+#if defined(__ARCH_ARM_5TE__)
 #define HAVE__MEMCMP16  1
-
+#endif
 /*
  * type for the third argument to mincore().
  */

Next up is the pixelflinger, where things get interesting, because all of a sudden we have armv6 code. I’ve taken the rash decision of wrapping this in conditionals that are only enabled if you actually have an ARMv6 version, not a pesky ARMv5E, but I really need to better understand the intent here. It seems a little strange.

diff --git a/libpixelflinger/Android.mk b/libpixelflinger/Android.mk
index a8e5ee4..077cf47 100644
--- a/libpixelflinger/Android.mk
+++ b/libpixelflinger/Android.mk
@@ -5,7 +5,7 @@ include $(CLEAR_VARS)
 # ARMv6 specific objects
 #
 
-ifeq ($(TARGET_ARCH),arm)
+ifeq ($(TARGET_ARCH_VERSION),armv6)
 LOCAL_ASFLAGS := -march=armv6
 LOCAL_SRC_FILES := rotate90CW_4x4_16v6.S
 LOCAL_MODULE := libpixelflinger_armv6
@@ -39,7 +39,7 @@ PIXELFLINGER_SRC_FILES:= \
 	raster.cpp \
 	buffer.cpp
 
-ifeq ($(TARGET_ARCH),arm)
+ifeq ($(TARGET_ARCH_VERSION),armv5te)
 PIXELFLINGER_SRC_FILES += t32cb16blend.S
 endif
 
@@ -67,7 +67,7 @@ ifneq ($(BUILD_TINY_ANDROID),true)
 LOCAL_MODULE:= libpixelflinger
 LOCAL_SRC_FILES := $(PIXELFLINGER_SRC_FILES)
 LOCAL_CFLAGS := $(PIXELFLINGER_CFLAGS) -DWITH_LIB_HARDWARE
-ifeq ($(TARGET_ARCH),arm)
+ifeq ($(TARGET_ARCH_VERSION),armv6)
 LOCAL_WHOLE_STATIC_LIBRARIES := libpixelflinger_armv6
 endif
 include $(BUILD_SHARED_LIBRARY)

Finally scanline has an optimised asm version it calls in preference to doing the same thing inline with C code. Again, I take the easy way out, and use the C code.

diff --git a/libpixelflinger/scanline.cpp b/libpixelflinger/scanline.cpp
index d24c988..685a3b7 100644
--- a/libpixelflinger/scanline.cpp
+++ b/libpixelflinger/scanline.cpp
@@ -1312,7 +1312,7 @@ void scanline_t32cb16blend(context_t* c)
     const int32_t v = (c->state.texture[0].shade.it0>>16) + y;
     uint32_t *src = reinterpret_cast(tex->data)+(u+(tex->stride*v));
 
-#if ((ANDROID_CODEGEN >= ANDROID_CODEGEN_ASM) && defined(__arm__))
+#if ((ANDROID_CODEGEN >= ANDROID_CODEGEN_ASM) && defined(__arm__) && defined(__ARCH_ARM_5TE__))
     scanline_t32cb16blend_arm(dst, src, ct);
 #else
     while (ct--) {

And that my friends, is that! Now to see if I can actually run this code!

Android Source Code Released

Wed, 22 Oct 2008 09:14:34 +0000
tech android

Usually when companies say release 4th quarter 2008 you usually see something around January 2009, and to be honest that was when I was expecting the Android source code to finally drop. So I was a little surprised to see that the code was released early this morning.

Stay tuned, more to come as I start playing.

Nokia acquires Symbian, Open Sources Symbian OS

Tue, 24 Jun 2008 20:04:11 +0000
tech article android symbian

On the 10th anniversary of the creation of Symbian Ltd, Nokia has announced that they will be acquiring Symbian Ltd with the aim of opening up the Symbian OS under an Eclipse based license. The mobile operating system market is really getting a shake up at the moment!

The Symbian Foundation has been created to build a new platform for mobile phones based on Symbian OS, S60, UIQ and MOAP(S). The foundation is expected to launch in H1 2009.

The new Symbian Foundation Platform will be made up of a common setup application suites, runtimes, UI framework, middleware, OS, tools and SDK, with Foundation members able to provide differentiated experiences on top. The platform is expected to be released to foundation members in 2009, and eventually open sourced over the following two years.

This obviously makes a huge change in the market place. It wll be interesting to see how Symbian Platform, vs. Android, vs. LiMo, vs. Windows Mobile vs. iPhone.

Of course it is all about developers, developers, developers, and it will be extremely interesting to see where developers will want to go.

linux.conf.au 2008 Day 1

Mon, 28 Jan 2008 19:01:51 +0000
tech lca linux android

After two weeks in California, I spent two days in Sydney, before flying down the sunny Melbourne yesterday for linux.conf.au 2008.

Monday and Tuesday at linux.conf.au are the miniconf days. The wide variety of topics on display make things a little difficult. I was back and forward between the embedded and virtualisation mini-confs.

I gave two presentations today, the first this morning was on how to port OKL4 to a new system-on-a-chip. The chip in question is the virtual Goldfish SoC, which forms the core of the emulated platform in Google's Android SDK.

The second presentation on a more high-level talk on why virtualisation is a useful technology not just for large data-centers and server applications, but also for embedded systems, such a mobile phone handsets.

Unfortunately because I was presenting, I didn't really have much time to focus on some of the other great presentations that went on today. With any luck I'll be able more attentively attend some talks tomorrow.

For those interested, the talks were filmed, so hopefully videos will be up online in the near future.

Updating the Android system image

Thu, 29 Nov 2007 08:34:00 +0000
tech android article

A lot of people on the groups page seem to be wanting to update the /system directory for a bunch of sane (and sometimes insane) reasons. As they and others have found out, changes to /system do not persist across reboots. This explains how to get around this.

The easiest way is to make a simple change to the Qemu source. First you download the source from the Android open source page. Then it is a very simple matter of chaning one line:

sprintf(tmp, "system,size=0x4200000,initfile=%s", arg_nand0);

Replace initfile, with simply file:

sprintf(tmp, "system,size=0x4200000,file=%s", arg_nand0);

Recompile and start using your shiny new emulator. Note: You probably want to make a backup of the system.img file so when you hose it you can start fresh.

As far as I can tell there isn't anyway to do this without hacking the code. This is probably easier than trying to get yaffs filesystem generatiogn tools working though.

A quick look inside the Android emulator

Thu, 29 Nov 2007 00:37:01 +0000
tech android article

I've been working on getting OKL4 up and running on the Android simulator. In doing so I've ended up getting quite involved with the simulator code base and thought I would share some of my findings.

At the core the simulator is Qemu version 0.8.2. The Android team has provided a wrapper main(), which does some slightly nicer argument naming and parsing, and ends up calling the traditional Qemu entrypoint. Inside, the main change is that a new platform called goldfish has been added to supplement the existing Integrator and Versatile platforms.

When porting a different OS to a new platform, the first thing you need to do is get some basic device drivers, such as interrupt controller, serial console and timer, up and running. Usually, the way to do this is find the published spec sheet, and go off that. Unfortunately there is no published spec sheet for the goldfish, but we have something infinitely more useful; the actual source code to the simulated device. (The number of times I've found bit errors in device documentation is pretty amazing!).

This post will share some of the details of the simulated platform (as it stands at this point in time!), along with some commentary. I'm only covering the bits that I have needed in bootstrapping OKL4, so keypads, framebuffers, etc will have to wait for another day. (Hopefully, tomorrow).

Physical memory layout

The physical memory layout is about as simple as it gets. RAM starts at address 0, and continues up to size of ram, in one contiguous block.

Cache

The data cache is 16KiB, 4 way set associative, with 32 byte lines. This is pretty standard, although it would be nice to have higher associativity. (Of course, it makes not very much difference in terms of simulation, so one can only guess that this cache layout is going to be similar to some real system-on-chip being used in an actual phone.)

Interrupt controller

The interrupt controller has a 4KiB block of registers residing at 0xff000000. It consist of 5 32-bit registers.

STATUS at offset 0x0 contains the number of pending interrupt. It is a read-only register.

NUMBER at offset 0x4 contains the lowest pending, enabled interrupt number. It is a read-only register.

DISABLE_ALL at offset 0x8 is a write-only register. Writing any value to it will disable all interrupts.

DISABLE at offset 0xC is a write-only register. Writing an interrupt number to it will disable to specified interrupt.

ENABLE at offset 0x10 is a write-only register. Writing an interrupt number to it will enable to specified interrupt.

This has to be the best interface to an interrupt controller ever. No messy shifting, or updating multiple registers to get the job done. Every function I need to implement in my driver ends up being just a register read or write. Bliss!

Serial

The serial controller has a 4KiB block of registers residing at 0xff002000. It consist of 5 32-bit registers.

PUT_CHAR at offset 0x0 is a write-only register. Writing a value to it puts a character onto the console.

BYTES_READY at offset 0x4 returns the number of characters waiting to be read from the console. This register is read-only.

CMD at offset 0x8 is a write-only register. Writing a command performs one of four actions.

DATA_PTR at offset 0x10 is a write-only register. The value in this register is the virtual address used in read and write buffer commands.

DATA_LEN at offset 0x14 is a write-only register. The value in this register is the number of bytes to copy on the read or write buffer commands.

This is a really nice interface. My one reservation is that it would be really nice if performing a read from PUT_CHAR returned a character if available. (Of course then it should be renamed from PUT_CHAR.) It was an interesting decision to use virtual addresses for the buffers, rather than a physical address. This will be different from most hardware out there.

Timer

The serial controller has a 4KiB block of registers residing at 0xff003000. It consist of 5 32-bit registers. Time is represented by a flowing 64-bit counter.

TIME_LOW at offset 0x0 return the lowest 32-bit from the 64-bit counter. It also latches the high 32-bits into TIME_HIGH. You must read TIME_LOW, before reading TIME_HIGH to get consistent values. It is a read-only register.

TIME_HIGH at offset 0x4 is a read-only register storing the top 32-bits of the 64-bit counter. It should only be read after reading the TIME_LOW value.

ALARM_LOW at offset 0x8 is a write-only register storing the lowest 32-bits of the next alarm value. When written it takes the top 32-bits for the alarm value from ALARM_HIGH and stores the value in an internal register. To get consistent results the ALARM_HIGH should be stored first when setting an alarm. When the counter value reaches the alarm value and interrupt is triggered.

ALARM_HIGH at offset 0xc is a write-only register storing the top 32-bits of the next alarm value. Writing to this register does not update the internal 64-bit alarm register. This is done on writes to ALARM_LOW

CLEAR_INTERRUPT at offset 0x10 is a write-only register. When written to it will clear an interrupt previously posted by the alarm.

This is a nice simple way to access an OS timer. The only thing missing is a periodic mode so the next alarm value doesn't need to be calculated each time. (Of course, periodic ticks are on the way out, so this isn't very critical.)

Bug fixes

The first bug fix is to actually make the thing compile on my machine. This mostly involved removing what seems to be dead code. SDL is used, but the build system is set up to only use SDL on specific files, and on those files, the correct include would be #include <SDL.h>. It turns out this code is unused, so we can just get rid of it entirely. This isn't really a problem for anyone using Android, just if you want to try and recompile.

--- android-emulator-20071111.orig/qemu/vl.c 2007-11-12 17:58:42.000000000 +1100
+++ android-emulator-20071111/qemu/vl.c   2007-11-29 00:29:35.000000000 +1100
@@ -78,12 +78,6 @@
 extern void  android_emulation_setup( void );
 extern void  android_emulation_teardown( void );
 
-#ifdef CONFIG_SDL
-#ifdef __APPLE__
-#include 
-#endif
-#endif /* CONFIG_SDL */
-
 #ifdef CONFIG_COCOA
 #undef main
 #define main qemu_main

The next bug is a little odd. This was found during a run of L4 test. Basically, the code goes to a lot of trouble to register a real-time alarm, and hook up a signal handler to service this. (This is how timer interrupts end up being injected into the emulated machine.) The problem is that the default sigprogmask seems to have SIGALRM blocked, which means we don't end up getting timer interrupts, or at least not if the emulated code is running in a tight loop. This bug could actually affect people using Android. It is possible that in this case timer interrupts are missed and the system becomes unresponsive. I'm not sure if it is something strange in my setup that makes SIGALRM blocked by default, or if this is a general problem. It should probably be investigated further.

diff -ru android-emulator-20071111.orig/qemu/vl.c android-emulator-20071111/qemu/vl.c
--- android-emulator-20071111.orig/qemu/vl.c 2007-11-12 17:58:42.000000000 +1100
+++ android-emulator-20071111/qemu/vl.c   2007-11-29 00:43:34.000000000 +1100
@@ -1282,6 +1276,7 @@
     {
         struct sigaction act;
         struct itimerval itv;
+        sigset_t nset;
 
         sigfillset(&act.sa_mask);
         act.sa_flags = 0;
@@ -1304,6 +1299,10 @@
         /* we probe the tick duration of the kernel to inform the user if
            the emulated kernel requested a too high timer frequency */
         getitimer(ITIMER_REAL, &itv);
+
+        sigemptyset(&nset);
+        sigaddset(&nset, SIGALRM);
+        sigprocmask(SIG_UNBLOCK, &nset, NULL);
     }
 #endif
 }

The next bugs are nasty. Really nasty. And I didn't really debug them, I kind of guessed and looked at diffs to find them. They deal with the innards of the ARM MMU, and are only really exposed by kernels that make full use of the memory management unit (e.g: domains for fast context switching, super pages, PID relocation). OKL4 does this, which I why I'm hitting these bugs, where as Linux doesn't right now, so it avoids them. The first of these problems was fixed upstream in Qemu 0.9.0, the second was actually found by another engineer at OK Labs, Matt Warton, and has been pushed upstream already.

diff -ru android-emulator-20071111.orig/qemu/target-arm/helper.c android-emulator-20071111/qemu/target-arm/helper.c
--- android-emulator-20071111.orig/qemu/target-arm/helper.c 2007-11-12 17:58:42.000000000 +1100
+++ android-emulator-20071111/qemu/target-arm/helper.c   2007-11-29 00:26:44.000000000 +1100
@@ -247,7 +247,7 @@
 
   switch (ap) {
   case 0:
-      if (access_type != 1)
+      if (access_type == 1)
           return 0;
       switch ((env->cp15.c1_sys >> 8) & 3) {
       case 1:
@@ -428,6 +428,7 @@
         break;
     case 3: /* MMU Domain access control.  */
         env->cp15.c3 = val;
+        tlb_flush(env, 1);
         break;
     case 4: /* Reserved.  */
         goto bad_reg;

New features

Since I was having to modify the emulator anyway, I decided to add a feature to make my live just a little easier. Qemu expects to be loading and a Linux kernel, but not all kernels out there are Linux, and they have different expectations about where they should be loaded and what data if any should be passed to them.

I've extended Qemu, and the android wrapper to support a new -os-type flag, so that you can specify what type of OS is being emulated. By default this is set to Linux, and the normal Linux kernel loading algorithm applies. If it is set to anything else, then it simply loads the specified kernel directly in at the start of memory and doesn't do any string or command line passing.

diff -ru android-emulator-20071111.orig/qemu/android_sdl.c android-emulator-20071111/qemu/android_sdl.c
--- android-emulator-20071111.orig/qemu/android_sdl.c 2007-11-12 17:58:41.000000000 +1100
+++ android-emulator-20071111/qemu/android_sdl.c   2007-11-29 00:59:24.000000000 +1100
@@ -3537,6 +3537,7 @@
 static char *arg_nand0 = 0;
 static char *arg_nand1 = 0;
 static char *arg_sdcard = 0;
+static char *arg_os_type = 0;
 static char *arg_kernel = 0;
 static char *arg_ramdisk = 0;
 static char *arg_tracefile = 0;
@@ -3573,6 +3574,7 @@
     const char *help;   /* description text for this option */
 } argmap[] = {
     { "-system",   &arg_sysdir,    0,              0,                   "<dir>",    "search system, ramdisk and userdata images in <dir>" },
+    { "-os-type",  &arg_os_type,   0,             "linux",              "<os-type>","kernel image is of given OS type. E.g: linux, okl4" },
     { "-kernel",   &arg_kernel,    0,             "kernel-qemu",        "<file>",   "use <file> as the emulated kernel" },
     { "-ramdisk",  &arg_ramdisk,   0,             "ramdisk.img",        "<file>",   "use <file> as the ramdisk image (default is <system>/ramdisk.img)" },
     { "-image",    &arg_nand0,     0,             "system.img",         "<file>",   "use <file> as the system image (default is <system>/system.img)" },
@@ -4270,6 +4272,10 @@
 
     n = 1;
     /* generate arguments for the underlying qemu main() */
+    if(arg_os_type && arg_os_type[0]) {
+        args[n++] = "-os-type";
+        args[n++] = arg_os_type;
+    }
     if(arg_kernel && arg_kernel[0]) {
         args[n++] = "-kernel";
         args[n++] = arg_kernel;
--- android-emulator-20071111.orig/qemu/vl.c 2007-11-12 17:58:42.000000000 +1100
+++ android-emulator-20071111/qemu/vl.c   2007-11-29 00:56:47.000000000 +1100
@@ -199,6 +193,8 @@
 int dcache_store_miss_penalty = 5;
 #endif
 
+char *os_type = "linux";
+
 extern void  dprint( const char* format, ... );
 
 /***********************************************************/
@@ -6005,6 +6006,7 @@
     QEMU_OPTION_smb,
     QEMU_OPTION_redir,
 
+    QEMU_OPTION_os_type,
     QEMU_OPTION_kernel,
     QEMU_OPTION_append,
     QEMU_OPTION_initrd,
@@ -6095,6 +6097,7 @@
     { "redir", HAS_ARG, QEMU_OPTION_redir },
 #endif
 
+    { "os-type", HAS_ARG, QEMU_OPTION_os_type },
     { "kernel", HAS_ARG, QEMU_OPTION_kernel },
     { "append", HAS_ARG, QEMU_OPTION_append },
     { "initrd", HAS_ARG, QEMU_OPTION_initrd },
@@ -6564,6 +6567,9 @@
                 pstrcpy(serial_devices[0], sizeof(serial_devices[0]), "stdio");
                 nographic = 1;
                 break;
+            case QEMU_OPTION_os_type:
+                os_type = optarg;
+                break;
             case QEMU_OPTION_kernel:
                 kernel_filename = optarg;
                 break;
--- android-emulator-20071111.orig/qemu/hw/arm_boot.c 2007-11-12 17:58:41.000000000 +1100
+++ android-emulator-20071111/qemu/hw/arm_boot.c   2007-11-29 00:57:04.000000000 +1100
@@ -64,6 +64,8 @@
     stl_raw(p++, 0);
 }
 
+extern char *os_type;
+
 void arm_load_kernel(int ram_size, const char *kernel_filename,
                      const char *kernel_cmdline, const char *initrd_filename,
                      int board_id)
@@ -71,19 +73,27 @@
     int kernel_size;
     int initrd_size;
     int n;
+    int linux_image = (strcmp(os_type, "linux") == 0);
 
     /* Load the kernel.  */
     if (!kernel_filename) {
         fprintf(stderr, "Kernel image must be specified\n");
         exit(1);
     }
-    kernel_size = load_image(kernel_filename,
-                             phys_ram_base + KERNEL_LOAD_ADDR);
+
+    if (linux_image) {
+      kernel_size = load_image(kernel_filename,
+               phys_ram_base + KERNEL_LOAD_ADDR);
+    } else {
+      kernel_size = load_image(kernel_filename,
+               phys_ram_base);
+    }
     if (kernel_size < 0) {
         fprintf(stderr, "qemu: could not load kernel '%s'\n", kernel_filename);
         exit(1);
     }
-    if (initrd_filename) {
+
+    if (linux_image && initrd_filename) {
         initrd_size = load_image(initrd_filename,
                                  phys_ram_base + INITRD_LOAD_ADDR);
         if (initrd_size < 0) {
@@ -94,12 +104,14 @@
     } else {
         initrd_size = 0;
     }
-    bootloader[1] |= board_id & 0xff;
-    bootloader[2] |= (board_id >> 8) & 0xff;
-    bootloader[5] = KERNEL_ARGS_ADDR;
-    bootloader[6] = KERNEL_LOAD_ADDR;
-    for (n = 0; n < sizeof(bootloader) / 4; n++)
-        stl_raw(phys_ram_base + (n * 4), bootloader[n]);
-    set_kernel_args(ram_size, initrd_size, kernel_cmdline);
+    if (linux_image) {
+        bootloader[1] |= board_id & 0xff;
+        bootloader[2] |= (board_id >> 8) & 0xff;
+        bootloader[5] = KERNEL_ARGS_ADDR;
+        bootloader[6] = KERNEL_LOAD_ADDR;
+        for (n = 0; n < sizeof(bootloader) / 4; n++)
+            stl_raw(phys_ram_base + (n * 4), bootloader[n]);
+        set_kernel_args(ram_size, initrd_size, kernel_cmdline);
+    }
 }