So, my earlier post on this was a little premature; anyone who has tried out the code has found out that it pretty much doesn’t work (hey I did warn you!). Now there are a range of fun reasons why this didn’t work, most of which I’ve now solved.
Firstly, it turns out that EABI and ARMv4T are pretty much
incompatible. (I’ll post separately about that!). In short,
thumb interworking doesn’t (can’t) work, so I’ve reverted back to
plain old ARMv4 architecture as my target (the only difference
between ARMv4 and ARMv4T is the thumb stuff, which we can’t use
until the compiler / spec is fixed.). So I’ve updated the
linux-arm.mk to support ARMv4 for now as well.
Of course the next problem that this introduces is that the
bx instruction doesn’t exist on ARMv4, and GCC
(helpfully) complains and stops the compilation. Now a BX
without thumb support is simply a mov pc, instruction,
so I went through and provided a BX macro that expands
to either bx or mov pc,. This is a little
bit nasty/invasive because it touches all the system call bindings,
thankfully these are generated anyway, but it makes the diff quite
large. (When I have time I’ll make it so that generation is part of
the buid system, not a manual process.)
The next problem is that the provided compiler’s
libgcc library is build for ARMv5, and has instructions
that just don’t exist on ARMv4 (shc as clz), so I went and built a new compiler
targeted to ARMv4. There is no reason why this couldn’t be set up as a
multi-lib compiler that supports both, but I don’t have enough GCC
wizardry in me to work that out right now. So a new compiler.
This got things to a booting stage, but not able to mount
/system or /data. Basically, Android by
default uses yet another flash
file-system (YAFFS), but for some reasons, which I couldn’t
fully work out initially, the filesystem just didn’t seem to cleanly
initialise and then mount. So, without diving too deep, I figured I
could just use jffs2 instead, which I know works on the
target. So I upgraded the Android build system to support allowing you
to choose which filesystem type to use, and providing jffs2 as an
option. This was going much better, and I got a lot further, far
enough that I needed to recompile my kernel with support for some of
the Android specific drivers like ashmem, binder and
logger. Unfortunately I was getting a hang on an mmap
call, for reasons that I couldn’t quite work out. After a lot of
tedious debugging (my serial console is broken, so I have to rely on
graphics console, which is really just an insane way to try and debug
anything), anyway, it turns out that part of what the Dalvik virtual
machine does when optimising class files is to mmap the
file as writable memory. This was what was failing, with the totally
useless error invalid argument. Do you know how many unique
paths along the mmap system call can set EINVAL? Well it’s a lot. Anyway,
long story short, it turns out that the jffs2 filesystem doesn’t support writable
mmaps! %&!#.
After I finished cursing, I decided to go back to using yaffs and
working out what the real problem is. After upgrading u-boot (in a
pointless attempt to fix my serial console), I noticed a new
write yaffs[1] command. This wasn’t there in the old
version. Ok, cool, maybe this has something do to with the
problem. But what is this the deal with yaffs versus yaffs1? Well it
turns out that NAND has different pagesize, 512 bytes, and 2k (or
multiples thereof, maybe??). And it turns out that YAFFS takes
advantage of this and has different file systems for different sized
NAND pages, and of course, everything that can go wrong will so, the
filesystem image that the build system creates is YAFFS2 which is for
2k pages not 512b pages. So, I again updated the build system to
firstly build both the mkyaffs2image and the
mkyaffsimage tool, and then set off building a YAFFS file
system.
Now, while u-boot supports yaffs filesystem, device firmware update
doesn’t (appear to). So this means I need to copy the image to memory
first, then on the device copy it from memory to flash. Now, the other
fun thing is that dfu can only copy 2MB or so to RAM at a time, and
the system.img file is around 52MB or so, which means that
it takes around 26 individual copies of 2MB sections.... very, very painful.
But in the end this more or less worked. So now I have a 56MB partition
for the system, and a 4MB partition for the user and things are looking good.
Good that is, right up until the point where dalvik starts up and
writes out cached version of class files to /data. You
see, it needs more than 4MB, a lot more, so I’m kind of back to square
one. I mean, if I’d looked at the requirements I would have read 128MB
of flash, but meh, who reads requirements? The obvious option would be
some type of MMC card, but as it turns out the number of handy Fry’s
stores on Boeing 747 from Sydney to LA number in the zeroes.
So the /system partition is read-only, and since the
only problem with jffs2 was when we were writing to it, it seems that
we could use jffs2 for the read-only system partition, which has the
advantage of jffs2 doing compression, and fitting in about 30MB, not
about 50MB, leaving plenty of room for the user data partition, which
is where the Dalvik cached files belong. This also has the advantage
of being able to use normal DFU commands to install the image
(yay!). So after more updates to the build system to now support
individually setting the system filesystem type and the user
filesystem type things seem a lot happier.
Currently, I have a system that boots init, starts up
most of the system services, including the Dalvik VM, runs a bunch of
code, but bombs out with an out-of-memory error in the pixelflinger
code which I’m yet to have any luck tracing. Currently my serial
console is fubar, so I can’t get any useful logging, which makes
things doubly painful. The next step is to get adb
working over USB so I have at least an output of the errors and
warning, which should give me half a chance of tracking down the
problem.
So if you want to try and get up to this point, what are the steps?
Well, firstly go and download the android
toolchain source code. and compile it for a v4 target. You use the
--target=armv4-android-eabi argument to configure if I
remember correctly.
Once you have that done, grab my latest patch and apply it to the Android source code base. (That is tar file with diffs for each individual project, apply these correctly is left as an exercise for the reader). Then you want to compile it with the new toolchain. I use a script like this:
#!/bin/sh
make TARGET_ARCH_VERSION=armv4 \
MKJFFS2_CMD="ssh nirvana -x \"cd `pwd`; mkfs.jffs2\"" \
SYSTEM_FSTYPE=jffs2 \
USERDATA_FSTYPE=yaffs \
TARGET_TOOLS_PREFIX=/opt/benno/bin/armv4-android-eabi- $@
Things you will need to change it the tools prefix, and the mkjffs2 command. The evil-hackery above is to run it on my linux virtual machine (I’m compiling the rest under OS X, and I can’t get mkfs.jffs2 to compile under it yet.)
After some time passes you should end up with a ramdisk.img, userdata.img and system.img files. The next step is to get a usable kernel.
I’m using the OpenMoko stable kernel, which is 2.6.24 based. I’ve patched this with bits of the Android kernel (enough, I think, to make it run). Make sure you configure support for yaffs, binder, logger and ashmem. Here is the kernel config I’m currently using.
At this stage it is important you have a version of u-boot supporting the yaffs write commands, if you don’t your next step is to install that. After this the next step is to re-partition your flash device. In case it isn’t obvious this will trash your current OS. The useful parts from my uboot environment are:
mtdids=nand0=neo1973-nand
bootdelay=-1
mtdparts=mtdparts=neo1973-nand:256k(uboot)ro,16k(uboot-env),752k(ramdisk),2m(kernel),36m(system),24m(userdata)
rdaddr=0x35000000
kaddr=0x32000000
bootcmd=setenv bootargs ${bootargs_base} ${mtdparts} initrd=${rdaddr},${rdsize}; nand read.e ${kaddr} kernel; nand read.e ${rdaddr} ramdisk; bootm ${kaddr}
bootargs_base=root=/dev/ram rw console=tty0 loglevel=8
Note the mtdparts which defines the partitions, and the bootcmd. (I’m not entirely happy with the boot command, mostly because when I install new RAM image I need to manually update $rdsize, which is a pain).
With this in place you are ready to start. The first image to move across is your userdata image. Now to make this happen we first copy it into memory using dfu-util:
sudo dfu-util -a 0 -R -D source/out/target/product/generic/userdata.img -R
Then you need to use the nand write.yaffs1 command to copy it to the data partition. Note, at this stage I get weird behaviour, I’m not convinced that the yaffs support truly works yet! Afterwards I get some messed up data in other parts of the flash (which is why we are doing it first). After you have copied it in, I suggest reseting the device, and you may find you need to reinitialise u-boot (using dyngen, and resetting up the environment as above.
After this you are good to use dfu-util to copy accross the kernel, system.img and ramdisk.img. After copying the ramdisk.img across update the rdsize variable with the size of the ramdisk.
Once all this is done, you are good to boot, I wish you luck! If you have a working
serial console you can probably try the logcat command to see why
graphics aren’t working. If you get this far please email me the results!