Showing posts with label Kernel. Show all posts
Showing posts with label Kernel. Show all posts

Friday, February 24, 2012

UBIFS

http://www.linux-mtd.infradead.org/doc/ubi.html

** need to pay attention to kernel version and utiltools version.


mtd-utils# mkfs.ubifs/mkfs.ubifs -r filesystem/ -F -o ubifs.img -m 2048 -e 126976 -c 1580


  • Create ubinize.cfg file and write the contents into it
mtd-utils# vi ubinize.cfg
  [ubifs]                <== Section header
  mode=ubi              <== Volume mode (other option is static)
  image=ubifs.img       <== Source image
  vol_id=0              <== Volume ID in UBI image
  vol_size=192MiB       <== Volume size
  vol_type=dynamic      <== Allow for dynamic resize
  vol_name=rootfs       <== Volume name
  vol_flags=autoresize  <== Autoresize volume at first mount
mtd-utils# ubi-utils/ubinize -o ubi.img -m 2048 -p 128KiB -s 512 -O 2048 ubinize.cfg






Symbol
MeaningValue for XO test case
SPPEB Size128KiB
SLLEB Size128KiB - 2 * 2KiB = 124 KiB
PTotal number of PEBs on the MTD device200MiB / 128KiB = 1600
BNumber of PEBs reserved for bad PEB handling1% of P = 16
OThe overhead related to storing EC and VID headers in bytes, i.e. O = SP - SL4KiB

Monday, August 23, 2010

Linux Process management

http://www.comptechdoc.org/os/linux/howlinuxworks/linux_hlprocess.html

Process control and the ability for inter process communication is handled by the Linux kernel.

Tools for working with processes

  • accton - Turns process accounting on and off. Uses the file /var/log/pacct. To turn it on type "accton /var/log/pacct". Use the command with no arguments to turn it off.
  • kill - Kill a process by number
  • killall - Send a signal to a process by name
  • lastcomm (1) - Display information about previous commands in reverse order. Works only if process accounting is on.
  • nice - Set process priority of new processes.
  • ps(1) - Used to report the status of one or more processes.
  • pstree(1) - Display the tree of running processes.
  • renice(8) - Can be used to change the process priority of a currently running process.
  • sa(8) - Generates a summary of information about users' processes that are stored in the /var/log/pacct file.
  • skill - Report process status.
  • snice - Report process status.
  • top - Displays the processes that are using the most CPU resources.

Monday, June 21, 2010

Others failure on porting android

http://www.mail-archive.com/android-porting@googlegroups.com/msg05626.html


Hi guys,
I am porting Android to AT91sam9261,however, I failed.
I share my porting process in here, and eager to obtain your
suggestions.
-------------------------------
My development environment:
* Fedora-9 full install
* arm-none-linux-gnueabi-gcc (arm-2008q1)
* u-boot-1.3.4, and official bootstrap file (from www.linux4sam.org)
--------------------------------
My porting steps:
1. download android source codes linux-2.6.25-android-1.0_r1.tar.gz
from http://code.google.com/p/android/downloads/list

2. download linux-2.6.25-at91 patch file from (
http://www.linux4sam.org/twiki/bin/view/Linux4SAM/LinuxKernel )

3. Compile the Android Kernel,
This step is successful and working on my at91sam9261ek.

4. Prepare Android file system, I got these information from Google.
Download the Android 1.0 SDK ( from
http://developer.android.com/sdk/1.0_r2/index.html
)
and execute this emulator.
#./emulator&
set the [Menu] -> [Dev. Tools] -> [Develop Settings] ->
apply these functions : [Wait for debugger], [Show running process],
[Show screen updates]
Then, following the Benno's Blog - Busybox for android (
http://benno.id.au/blog/2007/11/14/android-busybox )
Operating on Fedora terminal, I successfully generated the android
file system based on Benno's blog.

5. Boot my at91sam9261ek with NFS after setting the u-boot env. and
downloading the kernel,
However, it's not working.

When I set the u-boot env. parameter "init=/init", the boot message
was shown that
========
Starting kernel ...
Uncompressing
Linux...............................................................................................
done, booting the kernel.
Linux version 2.6.25-at91sam9261ek-android-20090624
(r...@localhost.localdomain)
(gcc version 4.2.3 (Sourcery G++ Lite 2008q1-126)) #6 Wed Jun 24
16:04:34 CST 2
009
CPU: ARM926EJ-S [41069265] revision 5 (ARMv5TEJ), cr=00053177
Machine: Atmel AT91SAM9261-EK
Ignoring unrecognised tag 0x54410008
Memory policy: ECC disabled, Data cache writeback
Clocks: CPU 198 MHz, master 99 MHz, main 18.432 MHz
CPU0: D VIVT write-back cache
CPU0: I cache: 16384 bytes, associativity 4, 32 byte lines, 128 sets
CPU0: D cache: 16384 bytes, associativity 4, 32 byte lines, 128 sets
Built 1 zonelists in Zone order, mobility grouping on. Total pages:
16256
Kernel command line: root=/dev/mtdblock3 mem=64M console=ttyS0,115200
init=/syst
em/bin/sh
AT91: 96 gpio irqs in 3 banks
PID hash table entries: 256 (order: 8, 1024 bytes)
Console: colour dummy device 80x30
console [ttyS0] enabled
Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
Memory: 64MB = 64MB total
Memory: 61628KB available (2668K code, 423K data, 140K init)
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
net_namespace: 152 bytes
android_power_init
android_power_init done
NET: Registered protocol family 16
AT91: Power Management
AT91: Starting after user reset
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 2048 (order: 2, 16384 bytes)
TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
TCP reno registered
NetWinder Floating Point Emulator V0.97 (double precision)
JFFS2 version 2.2. (NAND) (SUMMARY) 穢 2001-2006 Red Hat, Inc.
yaffs Jun 24 2009 15:35:49 Installing.
io scheduler noop registered
io scheduler anticipatory registered (default)
atmel_lcdfb atmel_lcdfb.0: 160KiB frame buffer at 00300000 (mapped at
c4880000)
Console: switching to colour frame buffer device 30x40
atmel_lcdfb atmel_lcdfb.0: fb0: Atmel LCDC at 0x00600000 (mapped at
c485c000), i
rq 21
atmel_usart.0: ttyS0 at MMIO 0xfefff200 (irq = 1) is a ATMEL_SERIAL
brd: module loaded
loop: module loaded
ssc ssc.1: Atmel SSC device at 0xc4860000 (irq 15)
dm9000 Ethernet Driver, V1.30
eth0: dm9000 at c485e000,c4866044 IRQ 107 MAC: 00:1f:3a:22:c2:30
(chip)
Driver 'sd' needs updating - please use bus_type methods
NAND device: Manufacturer ID: 0xec, Chip ID: 0xf1 (Samsung NAND 128MiB
3,3V 8-bi
t)
AT91 NAND: 8-bit, Software ECC
Scanning device for bad blocks
Bad eraseblock 256 at 0x02000000
Creating 4 MTD partitions on "at91_nand":
0x00020000-0x00060000 : "Bootstrap"
0x00060000-0x00200000 : "Parameters"
0x00200000-0x00400000 : "Kernel"
0x00400000-0x08000000 : "rootfs-1"
atmel_spi atmel_spi.0: Atmel SPI Controller at 0xfffc8000 (irq 12)
mtd_dataflash spi0.0: AT45DB321x (4224 KBytes)
usbmon: debugfs is not available
at91_ohci at91_ohci: AT91 OHCI
at91_ohci at91_ohci: new USB bus registered, assigned bus number 1
at91_ohci at91_ohci: irq 20, io mem 0x00500000
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
Initializing USB Mass Storage driver...
usbcore: registered new interface driver usb-storage
USB Mass Storage support registered.
udc: at91_udc version 3 May 2006
mice: PS/2 mouse device common for all mice
input: gpio-keys as /class/input/input0
ads7846 spi0.2: touchscreen, irq 29
input: ADS784x Touchscreen as /class/input/input1
rtc-at91sam9 at91_rtt.0: rtc core: registered at91_rtt as rtc0
rtc-at91sam9 at91_rtt.0: rtc0: SET TIME!
Registered led device: ds7
Registered led device: ds8
Registered led device: ds1
usbcore: registered new interface driver usbhid
drivers/hid/usbhid/hid-core.c: v2.6:USB HID core driver
logger: created 64K log 'log_main'
logger: created 64K log 'log_events'
logger: created 64K log 'log_radio'
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
rtc-at91sam9 at91_rtt.0: hctosys: unable to read the hardware clock
yaffs: dev is 32505859 name is "mtdblock3"
yaffs: passed flags ""
yaffs: Attempting MTD mount on 31.3, "mtdblock3"
yaffs: auto selecting yaffs2
block 225 is bad
VFS: Mounted root (yaffs filesystem).
Freeing init memory: 140K
init: cannot open '/initlogo.rle'
(Stop and die here)
====================

When I set the u-boot env. parameter "init=/system/bin/sh", the boot
message was shown that:

....... (the same boot message, ignored) .......
VFS: Mounted root (yaffs filesystem).
Freeing init memory: 140K
sh: can't access tty; job control turned off
#
#

The shell is working. But I still don't know how to run the Android.
Has anybody got ideas for this porting tasks?
Please share your experience if you succeed in porting to
at91sam9261ek?
Many thanks.

Android Porting Resources

Thursday, April 01, 2010

How to build kernel

Some tips are useful in this slide.

<object id="_ds_23919583" name="_ds_23919583" width="670" height="550"
type="application/x-shockwave-flash"
data="http://viewer.docstoc.com/"><param name="FlashVars"
value="doc_id=23919583&mem_id=2670372&showrelated=1&showotherdocs=1&doc_type=pdf&allowdownload=1"
/><param name="movie" value="http://viewer.docstoc.com/"/><param
name="allowScriptAccess" value="always" /><param name="allowFullScreen"
value="true" /></object><br /><font size="1"><a
href="http://www.docstoc.com/docs/23919583/Linux-Kernel-Hacking-Free-Course">Linux
Kernel Hacking Free Course</a> - </font>

Tuesday, March 09, 2010

Linux device drivers

Check out this SlideShare Presentation:

Friday, January 29, 2010

Steps to Init Block Device Driver on Linux

from Essential Linux Device Driver
1.
Registers the block device using register_blkdev(). This block library
routine assigns an unused major number to myblkdev and adds an entry for
the device in /proc/devices.
2.
Associates a request method with the block device. It does this by
supplying the address of myblkdev_request() to blk_init_queue(). The
call to blk_init_queue() returns the request_queue for myblkdev. Refer
back to Figure 14.2 to see how the request_queue sits relative to the
driver. The second argument to blk_init_queue(), myblkdev_lock, is a
spinlock to protect the request_queue from concurrent access.
3.
Hardware performs disk transactions in units of sectors, whereas
software subsystems, such as filesystems, deal with data in terms of
blocks. The common sector size is 512 bytes; the usual block size is
4096 bytes. You need to inform the block layer about the sector size
supported by your storage hardware and the maximum number of sectors
that your driver can receive per request. myblkdev_init() accomplishes
these by invoking blk_queue_hardsect_size() and blk_queue_max_sectors(),
respectively.
4.
Allocates a gendisk corresponding to myblkdev using alloc_disk() and
populates it. One important gendisk field that myblkdev_init() supplies
is the address of the driver's block_device_operations. Another
parameter that myblkdev_init() fills in is the storage capacity of
myblkdev in units of sectors. This is accomplished by calling
set_capacity(). Each gendisk also contains a flag that signals the
properties of the underlying storage hardware. If the drive is
removable, for example, the gendisk's flag field should be marked
GENHD_FL_REMOVABLE.
5.
Associates the gendisk prepared in Step 4 with the request_queue
obtained in Step 2. Also, connects the gendisk with the device's
major/minor numbers and a name.
6.
Adds the disk to the block I/O layer by invoking add_disk(). When this
is done, the driver has to be ready to receive requests. So, this is
usually the last step of the initialization sequence.

Wednesday, January 27, 2010

What users comment on this bonding technic

Ya, I this product is definitely using bonding technique. For the uploading problem, it's quite reasonable. Because packages from the same TCP connection go to different path to reach the destination with different delay. So in the destination side, the package may fell out of TCP window and cause the retransmit of the same TCP package. It's not easy to solve the problem I think. One way is to bind the TCP connection to a specified interface so that the packages cease to reach in order. But user can not benefit from this design if there is only one connection.

in reference to:

"We have a portabella and really love it. We rent the porcini (the other magic mushroom that makes this possible) from mushroom networks and it really is very reasonably priced compared to the other limited options out there. The only flaw is it needs UDP to do solid video streaming right now which is not possible for some reason with flash video streaming. We get INCREDIBLE download speeds and file transfers but less than impressive upload speed via TCP. Its an issue that they are working on very hard. The biggest benefit to something like this is for tradeshows. Those bastards will rape you for $1,000 a day for a decent connection. We have bandwidth to spare on our own WiFi now, run skype, live broadcast, make sales, play videos etc etc while our competition spends more for internet in one day than we spend for 4 months."
- Mushroom Networks announces new wireless "Broadband Bonding" technology -- Engadget (view on Google Sidewiki)

Multiple links bonding

try to bond links from different ISP

in reference to:

"External BGP neighbors normally sit on directed connected networks.When one has multiple parallel links the a load balancing technique is to use loop back address in the configuration for the EBGP neighbor. Static routes are configured pointing to the loopback address on the far-side neighbor.I would not recommend you try to bundle links from different ISP if using over these links. You have to use the EBGP-multi-hop feature and trying to figure out the hop count to prove to be a very interesting exercise."
- Which Router will be suitable for bonding 2 or 3 Dsl cable connection? - Networking & Security (view on Google Sidewiki)

Port Android to ARM926-ej-s

I think i can try this on my SAM9261 Board.

in reference to:

"下面是一些有用的参考,希望有助于对此感兴趣的开发人员: (1)Ben “Benno” Leslie的关于andorid移植到openmoko的个人博客地址: http://benno.id.au/blog/ (2)早期宣布成功移植android到zauraus-sl-c760的详细方法描述的链接: http://euedge.com/blog/2007/12/06/google-android-runs-on-sharp-zaurus-sl-c760/ (3)后续的根据上述先行者们的工作,成功移植android到zauraus-c3000的方法: http://androidzaurus.seesaa.net/article/74237419.html (4)本文是参考下面的wiki,接合个人的实践写出来的,对原文的作者表示一下感谢: http://wiki.droiddocs.net/Compilation_of_Android_kernel"
- Android内核编译方法_嵌入式Android_Linux开发_百度空间 (view on Google Sidewiki)

Monday, January 25, 2010

P2P Streaming Process

this also applies to multiple connections between two endpoints.

in reference to:

"A streaming process can be separated into three stages that overlap in time (Figure 1): data acquisition, data delivery and data presentation. Data acquisition is the stage that determines how the streaming content is acquired, packetized and distributed for streaming. The data presentation stage represents the methods on how to buffer, assemble and render the received data. Data delivery is the process of how the stream data is transported from the source to the destination. The source, the destination and all the intermediate nodes in a streaming system participate in a topology that is constructed based on the specific system’s protocol. In a P2P streaming system, this network architecture exhibits peer-to-peer characteristics."
- Peer-to-Peer Streaming - Streaming P2P Architectures, Streaming Process, Peer-to-Peer System Operation (view on Google Sidewiki)

Wednesday, January 06, 2010

Byte Alignment Rule

I dont know the 16 bytes thing before -_-!

in reference to:

"General Byte Alignment Rules The following byte padding rules will generally work with most 32 bit processor. You should consult your compiler and microprocessor manuals to see if you can relax any of these rules. Single byte numbers can be aligned at any address Two byte numbers should be aligned to a two byte boundary Four byte numbers should be aligned to a four byte boundary Structures between 1 and 4 bytes of data should be padded so that the total structure is 4 bytes. Structures between 5 and 8 bytes of data should be padded so that the total structure is 8 bytes. Structures between 9 and 16 bytes of data should be padded so that the total structure is 16 bytes. Structures greater than 16 bytes should be padded to 16 byte boundary."
- Byte Alignment and Ordering in Message Definition (view on Google Sidewiki)

Tuesday, December 29, 2009

convert vmlinuz to vmlinux

this is a way. actually, oprofile needs symbols in vmlinux. but they were stripped when converted from vmlinux to vmlinuz to save space. so there will be no use at all.


The vmlinuz isn't just the compressed kernel, it's complete bootable image including the decompressor. To get just the image search for the GZ signature - 1f 8b 08 00. Now i'm sure there are scripts for it somewhere, but you can do it old-fashioned way - in my case:

> od -A d -t x1 vmlinuz | grep '1f 8b 08 00'
0024576 24 26 27 00 ae 21 16 00 1f 8b 08 00 7f 2f 6b 45

so the image begins at 24576+8 => 24584 . Then just copy the image from the point and decompress it -

>dd if=vmlinuz bs=1 skip=24584 | zcat > vmlinux
1450414+0 records in
1450414+0 records out
1450414 bytes (1.5 MB) copied, 6.78127 s, 214 kB/s

in reference to:

"The vmlinuz isn't just the compressed kernel, it's complete bootable image including the decompressor. To get just the image search for the GZ signature - 1f 8b 08 00. Now i'm sure there are scripts for it somewhere, but you can do it old-fashioned way - in my case: > od -A d -t x1 vmlinuz | grep '1f 8b 08 00' 0024576 24 26 27 00 ae 21 16 00 1f 8b 08 00 7f 2f 6b 45 so the image begins at 24576+8 => 24584 . Then just copy the image from the point and decompress it - >dd if=vmlinuz bs=1 skip=24584 | zcat > vmlinux 1450414+0 records in 1450414+0 records out 1450414 bytes (1.5 MB) copied, 6.78127 s, 214 kB/s"
- convert vmlinuz to vmlinux [Archive] - CodeGuru Forums (view on Google Sidewiki)

Oprofile Configure

default event for ARM series is different from Pentium.

in reference to:

"ARM/XScale PMU1 arm/xscale1 CPU_CYCLES:100000:0:1:1 ARM/XScale PMU2 arm/xscale2 CPU_CYCLES:100000:0:1:1 ARM/MPCore arm/mpcore CPU_CYCLES:100000:0:1:1 AVR32 avr32 CPU_CYCLES:100000:0:1:1"
- Chapter 3. Controlling the profiler (view on Google Sidewiki)

Tuesday, November 10, 2009

Understanding ELF using readelf and objdump (zz)

http://www.linuxforums.org/articles/understanding-elf-using-readelf-and-objdump_125.html

What is ELF? ELF (Executable and Linking Format) is file formatthat defines how an object file is composed and organized. With thisinformation, your kernel and the binary loader know how to loadthe file, where to look for the code, where to look the initializeddata, which shared library that needs to be loaded and so on.

First of all, you should know about different kind of ELF object:

  • Relocatable file: an object file that holds code and data suitable for linking with other object files to create an executable or a shared object file. In other word, you can say that relocatable file is a foundation for creating executables and libraries.

    This is kind of file you get if you compile a source code like this:

    $ gcc -c test.c

    That will produce test.o, which is a relocatable file.

    Kernel module (either suffixed with .o or .ko) is also a form of relocatable file.

  • Executable file: object file that holds a program suitable for execution. Yes, that means, your XMMS mp3 player, your vcd software player, even your text editor are all ELF executable files.

    This is also a familiar file if you compile a program:

    $ gcc -o test test.c

    After you make sure the executable bit of "test" is enabled, you can execute it. The question is, what about shell script? Shell script is NOT ELF executable, but the interpreter IS.

  • Shared object file: This file holds code and data suitable for linking in two contexts:

    1. The link editor may process it with other relocatable and shared shared object file to create another object file.
    2. The dynamic linker combines it with an executable file and other shared objects to create a process image.

    In simple words, these are the files that you usually see with suffix .so (normally located inside /usr/lib on most Linux installation).

Is there any other way to detect the ELF type? Yes there is. In every ELF object, there is a file header that explains what kind file it is. Assuming you have installed binutils package, you can use readelf to read this header. For example (command results are shortened to show related fields only):

$ readelf -h /bin/ls Type:  EXEC (Executable file)  $ readelf -h  /usr/lib/crt1.o Type:  REL (Relocatable file)  $ readelf -h /lib/libc-2.3.2.so Type:  DYN (Shared object file)       

"File" command works too for object file identification, but I won't discuss it further. Let's focus on readelf and objdump, since we will use both of them.

To make us easier to study ELF, you can use the following simple C program:

/* test.c */ #include  int global_data = 4; int global_data_2;  int main(int argc, char **argv) {         int local_data = 3;         printf("Hello Worldn");         printf("global_data = %dn", global_data);         printf("global_data_2 = %dn", global_data_2);         printf("local_data = %dn", local_data);         return (0); } 

And compile it:

$ gcc -o test test.c

A. Examining ELF header.

The produced binary will be our examination target. Let's start with the content of the ELF header:

$ readelf -h test
ELF Header:   Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00   Class:                             ELF32   Data:                              2's complement, little endian   Version:                           1 (current)   OS/ABI:                            UNIX - System V   ABI Version:                       0   Type:                              EXEC (Executable file)   Machine:                           Intel 80386   Version:                           0x1   Entry point address:               0x80482c0   Start of program headers:          52 (bytes into file)   Start of section headers:          2060 (bytes into file)   Flags:                             0x0   Size of this header:               52 (bytes)   Size of program headers:           32 (bytes)   Number of program headers:         7   Size of section headers:           40 (bytes)   Number of section headers:         28   Section header string table index: 25       

What does this header tell us?

  • This executable is created for Intel x86 32 bit architecture ("machine" and "class" fields).

  • When executed, program will start running from virtual address 0x80482c0 (see entry point address). The "0x" prefix here means it is a hexadecimal number. This address doesn't point to our main() procedure, but to a procedure named _start. Never felt you had created such thing? Of course you don't. _start procedure is created by the linker whose purpose is to initialize your program.

  • This program has a total of 28 sections and 7 segments.

What is section? Section is an area in the object file that contains information which is useful for linking: program's code, program's data (variables, array, string), relocation information and other. So, in each area, several information is grouped and it has a distinct meaning: code section only hold code, data section only holds initialized or non-initialized data, etc. Section Header Table (SHT) tells us exactly what sections the ELF object has, but at least by looking on "Number of section headers" field above, you can tell that "test" contains 28 sections.

If section has meaning for the binary, our Linux kernel doesn't see it the same way. The Linux kernel prepares several VMA (virtual memory area) that contains virtually contigous page frames. Inside these VMA, one or more sections are mapped. Each VMA in this case represents an ELF segment. How the kernel knows which section goes to which segment? This is the function of Program Header Table(PHT).

Figure 1. ELF structure in two different point of view.

{mospagebreak title=Examining Section Header Table(SHT)}

B. Examining Section Header Table(SHT).

Let's see what kind of sections that exist inside our program (output is shortened):

$ readelf -S test

There are 28 section headers, starting at offset 0x80c:

Section Headers: [Nr] Name      Type      Addr     Off    Size   ES Flg Lk Inf Al ........ [ 4] .dynsym   DYNSYM    08048174 000174 000060 10   A  5   1  4 ........ [11] .plt      PROGBITS  08048290 000290 000030 04  AX  0   0  4 [12] .text     PROGBITS  080482c0 0002c0 0001d0 00  AX  0   0  4 ........ [20] .got      PROGBITS  080495d8 0005d8 000004 04  WA  0   0  4 [21] .got.plt  PROGBITS  080495dc 0005dc 000014 04  WA  0   0  4 ........ [22] .data     PROGBITS  080495f0 0005f0 000010 00  WA  0   0  4 [23] .bss      NOBITS    08049600 000600 000008 00  WA  0   0  4 ........ [26] .symtab   SYMTAB    00000000 000c6c 000480 10     27  2c  4 ........        

.text section is a place where the compiler put executables code. As the consequence, this section is marked as executable ("X" on Flg field). In this section, you will see the machine codes of our main() procedure:

$ objdump -d -j .text test

-d tells objdump to diassembly the machine code and -j tells objdump to focus on specific section only (in this case, .text section)

08048370 
: ....... 8048397: 83 ec 08 sub $0x8,%esp 804839a: ff 35 fc 95 04 08 pushl 0x80495fc 80483a0: 68 c1 84 04 08 push $0x80484c1 80483a5: e8 06 ff ff ff call 80482b0 80483aa: 83 c4 10 add $0x10,%esp 80483ad: 83 ec 08 sub $0x8,%esp 80483b0: ff 35 04 96 04 08 pushl 0x8049604 80483b6: 68 d3 84 04 08 push $0x80484d3 80483bb: e8 f0 fe ff ff call 80482b0 .......

.data section hold all the initialized variable inside the program which doesn't live inside the stack. "Initialized" here means it is given an initial value like we did on "global_data". How about "local_data"? No, local_data's value isn't in .data since it lives on process's stack.

Here is what objdump found about .data section:

$ objdump -d -j .data test ..... 080495fc :  80495fc:       04 00 00 00           ....  .....

One thing that we can conclude so far is that objdump kindly does address-to-symbol transformation for us. Without looking into symbol table, we know that 0x08049424 is the address of global_data. There, we clearly see that it is initialized with 4. Please note that common executables installed by most Linux distribution has been striped out, thus there is no entry in its symbol table. It makes objdump difficult to interpret the addresses.

And what is .bss? BSS (Block Started by Symbol) is a section where all unitialized variables are mapped. You might think "everything surely has an initial value". True, in Linux case, all unitialized variables are set as zero, that's why .bss section is just bunch of zeroes. For character type variables, that means null character. Knowing this fact, we know that global_data_2 is assigned 0 on runtime:

$ objdump -d -j .bss test-lagi Disassembly of section .bss: ..... 08049604 :  8049604:       00 00 00 00               .... .....

Previously, we mentioned a bit about symbol table. This table is useful to find the correlation between a symbol name (non external function, variable) and an address. Using -s, readelf will decode the symbol table for you:

$ readelf -s ./test
Symbol table '.dynsym' contains 6 entries:    Num:    Value  Size Type    Bind   Vis      Ndx Name .....      2: 00000000    57 FUNC    GLOBAL DEFAULT  UND printf@GLIBC_2.0 (2) .....  Symbol table '.symtab' contains 72 entries:    Num:    Value  Size Type    Bind   Vis      Ndx Name .....         49: 080495fc     4 OBJECT  GLOBAL DEFAULT   22 global_data .....     55: 08048370   109 FUNC    GLOBAL DEFAULT   12 main .....     59: 00000000    57 FUNC    GLOBAL DEFAULT  UND printf@@GLIBC_2.0 .....     61: 08049604     4 OBJECT  GLOBAL DEFAULT   23 global_data_2 .....       

"Value" denotes the address of the symbol. For example, if an instruction refers to this address (e.g: pushl 0x80495fc), that means it refers to global_data. Printf() is treated differently, since it is a symbol that refers to an external function. Remember that printf is defined in glibc, not inside our program. Later, I will explain how our program calls printf.

C. Examining Program Header Table(PHT).

Like I explained previously, segment is the way operating system "sees" our program. Thus, let's see how will our program be segmented:

$ readelf -l test
..... There are 7 program headers, starting at offset 52  Program Headers:      Type     Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align [00] PHDR     0x000034 0x08048034 0x08048034 0x000e0 0x000e0 R E 0x4 [01] INTERP   0x000114 0x08048114 0x08048114 0x00013 0x00013 R   0x1 [02] LOAD     0x000000 0x08048000 0x08048000 0x004fc 0x004fc R E 0x1000 [03] LOAD     0x0004fc 0x080494fc 0x080494fc 0x00104 0x0010c RW  0x1000 [04] DYNAMIC  0x000510 0x08049510 0x08049510 0x000c8 0x000c8 RW  0x4 [05] NOTE     0x000128 0x08048128 0x08048128 0x00020 0x00020 R   0x4 [06] STACK    0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x4  Section to Segment mapping:   Segment Sections...    00         01     .interp     02     .interp .note.ABI-tag .hash .dynsym .dynstr .gnu.version  	  .gnu.version_r .rel.dyn .rel.plt .init .plt .text .fini  	  .rodata .eh_frame     03     .ctors .dtors .jcr .dynamic .got .got.plt .data .bss     04     .dynamic     05     .note.ABI-tag     06       
Note:I add numbers on the left of each PHT entries to make the reader easier to study the section to segment mapping.

The mapping is quite straightforward. For example, inside segment number 02, there are 15 sections mapped. .text section is mapped in this segment. Its flags are R and E, which means it is Readable and Executable. If you see W in segment's flag, that means it is writable.

By looking on "VirtAddr" column, we can discover the virtual start address of each segment. Back to the segment number #2, the start address is 0x08048000. Later in this section, we will discover that this address isn't the real address of the segment on memory. You can ignore the PhysAddr, because in Linux always operate in protected mode (on Intel/AMD 32 bit and 64 bit) thus virtual address is the thing that matters.

Segment has many types, but let's focus on two types:

  • LOAD: The segment's content is loaded from the executable file. "Offset" denotes the offset of the file where the kernel should start reading the file's content. "FileSiz" tells us how many bytes must be read from the file.

    For example,segment #2 is actually the content of the file starting from offset 0 to 4fc (offset+filesiz). To speed up the execution, the file's content is read on demand, thus it is only read from the disk if it is referenced at runtime.

  • STACK : The segment is stack area. Interesting to see that all the fields except "Flg" and "Align" are given 0. Is it an error? No, it is valid. It is the kernel's job to decide where the stack segment starts from and how big it is. Remember that on Intel compatible processor, stack grows downward (address is decremented each time a value is pushed).

Courious to see the real layout of process segment? We can use /proc/<pid>/maps file to reveal it. <pid> is the PID of the process we want to observe. Before we move on, we have a small problem here. Our test program runs so fast that it ends before we can even dump the related /proc entry. I use gdb to solve this. You can use another trick such as inserting sleep() before it calls return().

In a console (or a terminal emulator such as xterm) do:

$ gdb test (gdb) b main Breakpoint 1 at 0x8048376 (gdb) r Breakpoint 1, 0x08048376 in main ()       

Hold right here, open another console and find out the PID of program "test". If you want the quick way, type:

$ cat /proc/`pgrep test`/maps

You will see an output like below (you might get different output):

[1]  0039d000-003b2000 r-xp 00000000 16:41 1080084  /lib/ld-2.3.3.so [2]  003b2000-003b3000 r--p 00014000 16:41 1080084  /lib/ld-2.3.3.so [3]  003b3000-003b4000 rw-p 00015000 16:41 1080084  /lib/ld-2.3.3.so [4]  003b6000-004cb000 r-xp 00000000 16:41 1080085  /lib/tls/libc-2.3.3.so [5]  004cb000-004cd000 r--p 00115000 16:41 1080085  /lib/tls/libc-2.3.3.so [6]  004cd000-004cf000 rw-p 00117000 16:41 1080085  /lib/tls/libc-2.3.3.so [7]  004cf000-004d1000 rw-p 004cf000 00:00 0 [8]  08048000-08049000 r-xp 00000000 16:06 66970    /tmp/test [9]  08049000-0804a000 rw-p 00000000 16:06 66970    /tmp/test [10] b7fec000-b7fed000 rw-p b7fec000 00:00 0 [11] bffeb000-c0000000 rw-p bffeb000 00:00 0 [12] ffffe000-fffff000 ---p 00000000 00:00 0       
Note: I add number on each line as reference.

Back to gdb, type:

(gdb) q

So, in total, we see 12 segment (also known as Virtual Memory Area--VMA). Focus on the first and the last field. First field denotes VMA address range, while last field shows the backing file. Do you see the similarity between VMA #8 and segment #02 listed in PHT? The difference is, SHT said it is ended on 0x080484fc, but on VMA #8, we see that it ends on 0x08049000. Same thing happens between VMA #9 and segment #03; SHT said it starts at 0x080494fc, while the VMA starts at 0x0804900.

There are several facts we must observe:

  1. Even though the VMA started on different address, the related sections are still mapped on exact virtual address.

  2. The kernel allocate memory on per page basis and the page size is 4KB. Thus, every page address is actually a multiple of 4KB e.g: 0x1000, 0x2000 and so on. So, for the first page of VMA #9, the page's address is 0x0804900. Or technically speaking, the address of the segment is rounded down (aligned) to the nearest page boundary.

Last, which one is the stack? That is VMA #11. Usually, the kernel allocate several pages dynamically and map to the highest virtual address possible in user space to form stack area. Simply speaking, each process address space is divided into two part (this assume Intel compatible 32 bit processor): user space and kernel space. User space is in 0x00000000-0xc0000000 range, while kernel space starts on 0xc0000000 onwards.

So, it is clear that stack is assigned address range near the 0xc0000000 boundary. The end address is static, while the start address is changing according to how many values are stored on stack.{mospagebreak title=How a function is referenced?}

D. How a function is referenced?

If a program calls a function that resides within its own executable, all it has to do is simple: just call the procedure. But what happens if it calls something like printf() that is defined inside glibc shared library?

Here, I won't discuss deeply about how the dynamic linker really works, but I focus on how the calling mechanism is implemented inside the executable itself. With this assumption in mind, let's continue.

When a program wants to call a function, it actually does following flow:

  1. It made a jump to relevant entry in PLT (Procedure Linkage Table).

  2. In PLT, there is another jump to an address mentioned in related entry in GOT (Global Offset Table).

  3. If this is the first the function is called, follow step #4. If this isn't, follow step #5.

  4. The related GOT entry contains an address that points back to next instruction in PLT. Program will jump to this address and then calls the dynamic linker to resolve the function's address. If the function is found, its address is put in related GOT entry and then the function itself is executed.

    So, another time the function is called, GOT already holds its address and PLT can jump directly to the address. This procedure is called lazy binding; all external symbols are not resolved until the time it is really needed (in this case, when a function is called). Jump to step #6.

  5. Jump to the address mentioned in GOT. It is the address of the function thus PLT is no longer used.

  6. Execution of the function is finished. Jump back to the next instruction in the main program.

As always, looking inside the executable is the best way to explain it. If you do:

$ objdump -d -j .text test  

You will see the following code fragment:

..... 08048370 
: ..... 804838f: e8 1c ff ff ff call 80482b0

What we have on 0x80482b0 is:

 080482b0 :  80482b0:       ff 25 ec 95 04 08       jmp    *0x80495ec  80482b6:       68 08 00 00 00          push   $0x8  80482bb:       e9 d0 ff ff ff          jmp    8048290 <_init+0x18>	 

As you see, the jump on 0x80482b0 is indirect jump ('*' in front of the address). So, to see where it will jump, we must peek into 0x80482b0. The guesses are, either this address is in .got section or in .got.plt. Looking back in SHT, it is clear that we must check .got.plt. I use readelf to do hexadecimal dump because it does number reordering for us:

$ readelf -x 21 test Hex dump of section '.got.plt':   0x080495dc 080482a6 00000000 00000000 08049510 ................   0x080495ec                            080482b6 ....       
(Note: first column is virtual address. The data in this address is described at the 5th column, not the second one! So, from right to left, the address is in ascending order.)

Bingo! We have "080482b6" here. In other word, we go back to PLT and there we eventually jump another address. This is where the work of the dynamic linker is started, so we will skip it. Assuming the dynamic linker has finished its magic work, the related GOT entry now holds the address of printf().

E. Alternative tool to inspect ELF structure.

Besides counting on readelf and objdump, there is another tool called Biew. This is actually a file viewer but it is capable to parse the ELF structure.. You can grab the source from http://biew.sourceforge.net/ and compile it by yourself. Usually biew is included in hacking oriented Linux Live CD such as Phlak. Refer to the website and the packaged documents on how to compile and install Biew.

I personally like Biew because it offers curses based GUI display. Navigation between sections, checking ELF header, listing symbols and other tasks are now just a matter of pressing certain keyboard shortcut and you're done.

For example, you can list the symbols and directly jump to the symbol's address. Here, we try to jump to main(). First execute biew:

$ biew test

Press Ctrl+A followed by F7 to view symbol table. To avoid wasting time traversing the table, press again F7 to open "Find string" menu. Type "main" and press Enter. Once the highlighted entry is what you're looking for, simply press Enter and Biew will jump to the address of main(). Don't forget to switch to the disassembler mode (press F2 to select it) so you can see the high level interpretation of the opcodes.

Figure 2. Biew lists all symbols

Since we usually refer to virtual address, not file offset, it is better to switch to virtual address view. Press Ctrl+C followed by F6 and select "Local". Now, what you see in the leftmost column is the virtual address.

Conclusion

This article is just an overview on how to study ELF structure. Using readelf and objdump, you are ready to take your first journey. If needed, tool like Biew can help you to explore the binary internal faster. Use any arsenal you have, be creative and practice it regularly, then soon you can master the technique. Happy exploring.

Further reading

Tuesday, September 22, 2009

Sunday, August 09, 2009

generate uboot uImage

Porting Linux to U-Boot based systems:
---------------------------------------

U-Boot cannot save you from doing all the necessary modifications to
configure the Linux device drivers for use with your target hardware
(no, we don't intend to provide a full virtual machine interface to
Linux :-).

But now you can ignore ALL boot loader code (in arch/ppc/mbxboot).

Just make sure your machine specific header file (for instance
include/asm-ppc/tqm8xx.h) includes the same definition of the Board
Information structure as we define in include/asm-<arch>/u-boot.h,
and make sure that your definition of IMAP_ADDR uses the same value
as your U-Boot configuration in CONFIG_SYS_IMMR.


Configuring the Linux kernel:
-----------------------------

No specific requirements for U-Boot. Make sure you have some root
device (initial ramdisk, NFS) for your target system.


Building a Linux Image:
-----------------------

With U-Boot, "normal" build targets like "zImage" or "bzImage" are
not used. If you use recent kernel source, a new build target
"uImage" will exist which automatically builds an image usable by
U-Boot. Most older kernels also have support for a "pImage" target,
which was introduced for our predecessor project PPCBoot and uses a
100% compatible format.

Example:

make TQM850L_config
make oldconfig
make dep
make uImage

The "uImage" build target uses a special tool (in 'tools/mkimage') to
encapsulate a compressed Linux kernel image with header information,
CRC32 checksum etc. for use with U-Boot. This is what we are doing:

* build a standard "vmlinux" kernel image (in ELF binary format):

* convert the kernel into a raw binary image:

${CROSS_COMPILE}-objcopy -O binary \
-R .note -R .comment \
-S vmlinux linux.bin

* compress the binary image:

gzip -9 linux.bin

* package compressed binary image for U-Boot:

mkimage -A ppc -O linux -T kernel -C gzip \
-a 0 -e 0 -n "Linux Kernel Image" \
-d linux.bin.gz uImage


The "mkimage" tool can also be used to create ramdisk images for use
with U-Boot, either separated from the Linux kernel image, or
combined into one file. "mkimage" encapsulates the images with a 64
byte header containing information about target architecture,
operating system, image type, compression method, entry points, time
stamp, CRC32 checksums, etc.

"mkimage" can be called in two ways: to verify existing images and
print the header information, or to build new images.

In the first form (with "-l" option) mkimage lists the information
contained in the header of an existing U-Boot image; this includes
checksum verification:

tools/mkimage -l image
-l ==> list image header information

The second form (with "-d" option) is used to build a U-Boot image
from a "data file" which is used as image payload:

tools/mkimage -A arch -O os -T type -C comp -a addr -e ep \
-n name -d data_file image
-A ==> set architecture to 'arch'
-O ==> set operating system to 'os'
-T ==> set image type to 'type'
-C ==> set compression type 'comp'
-a ==> set load address to 'addr' (hex)
-e ==> set entry point to 'ep' (hex)
-n ==> set image name to 'name'
-d ==> use image data from 'datafile'

Right now, all Linux kernels for PowerPC systems use the same load
address (0x00000000), but the entry point address depends on the
kernel version:

- 2.2.x kernels have the entry point at 0x0000000C,
- 2.3.x and later kernels have the entry point at 0x00000000.

So a typical call to build a U-Boot image would read:

-> tools/mkimage -n '2.4.4 kernel for TQM850L' \
> -A ppc -O linux -T kernel -C gzip -a 0 -e 0 \
> -d /opt/elsk/ppc_8xx/usr/src/linux-2.4.4/arch/ppc/coffboot/vmlinux.gz \
> examples/uImage.TQM850L
Image Name: 2.4.4 kernel for TQM850L
Created: Wed Jul 19 02:34:59 2000
Image Type: PowerPC Linux Kernel Image (gzip compressed)
Data Size: 335725 Bytes = 327.86 kB = 0.32 MB
Load Address: 0x00000000
Entry Point: 0x00000000

To verify the contents of the image (or check for corruption):

-> tools/mkimage -l examples/uImage.TQM850L
Image Name: 2.4.4 kernel for TQM850L
Created: Wed Jul 19 02:34:59 2000
Image Type: PowerPC Linux Kernel Image (gzip compressed)
Data Size: 335725 Bytes = 327.86 kB = 0.32 MB
Load Address: 0x00000000
Entry Point: 0x00000000

NOTE: for embedded systems where boot time is critical you can trade
speed for memory and install an UNCOMPRESSED image instead: this
needs more space in Flash, but boots much faster since it does not
need to be uncompressed:

-> gunzip /opt/elsk/ppc_8xx/usr/src/linux-2.4.4/arch/ppc/coffboot/vmlinux.gz
-> tools/mkimage -n '2.4.4 kernel for TQM850L' \
> -A ppc -O linux -T kernel -C none -a 0 -e 0 \
> -d /opt/elsk/ppc_8xx/usr/src/linux-2.4.4/arch/ppc/coffboot/vmlinux \
> examples/uImage.TQM850L-uncompressed
Image Name: 2.4.4 kernel for TQM850L
Created: Wed Jul 19 02:34:59 2000
Image Type: PowerPC Linux Kernel Image (uncompressed)
Data Size: 792160 Bytes = 773.59 kB = 0.76 MB
Load Address: 0x00000000
Entry Point: 0x00000000


Similar you can build U-Boot images from a 'ramdisk.image.gz' file
when your kernel is intended to use an initial ramdisk:

-> tools/mkimage -n 'Simple Ramdisk Image' \
> -A ppc -O linux -T ramdisk -C gzip \
> -d /LinuxPPC/images/SIMPLE-ramdisk.image.gz examples/simple-initrd
Image Name: Simple Ramdisk Image
Created: Wed Jan 12 14:01:50 2000
Image Type: PowerPC Linux RAMDisk Image (gzip compressed)
Data Size: 566530 Bytes = 553.25 kB = 0.54 MB
Load Address: 0x00000000
Entry Point: 0x00000000