A paper
from David Woodhouse on JFFS2The David Woodhouse's
presentation
on JFFS2. He explains how works the Garbage Collector with JFFS2HOW TO USE MTD/JFFS2 UNDER µClinux
Patrice KADIONIK, Professor Assistant at the ENSEIRB School of Electrical Engineering, Telecommunication, and Computer Science
UPDATED
This HOWTO presents step by step how to enable and use MTD/JFFS2 features under µClinux.
I'm currently teaching embedded system programming and propose to my students practical exercices on µClinux on Motorola M5407C3 ColFire boards. In every embedded system, you have some FLASH memory. We can use it for saving configuration parameters in a raw binary format. With embedded Linux, it's now possible to put a file system in FLASH and mount it after from the embedded Linux OS. The technique used here is to enable the MTD (Memory Technology Device) technology in order to hide the physical FLASH programming methods (reading, erasing sector, writing...) and put abose a JFFS(2) file system (Journalling Flash File System) supported by Linux and µClinux. JFFS2 is a robust file system supporting power failure without fsck checking after reboot.
|
JFFS
|
JFFS2
|
|
MTD
|
|
|
FLASH, RAM
|
|
Another way of using FLASH memory is to put in it a Linux kernel with a bootloader that decompresses the Linux kernel into the RAM memory and launches it after. This point will be not discussed here.
I strongly recommand to read, read and read these following documents that should become familiar to you before reading this HOWTO:
A paper
from David Woodhouse on JFFS2The David Woodhouse's
presentation
on JFFS2. He explains how works the Garbage Collector with JFFS2I will also thank here Massimo Calo who helped me through his threads posted to the µClinux news server and his mails on MTD/JFFS.
This point is just a resume extracted from the preceding documents (from the www.embeddedlinuxworks.com site, an article from Vipin Malik).
"In the year 2000, Axis Communications AB (www.axis.com), released the first version of the JFFS file system. It was also Open Sourced. This was a system designed from the get-go for truly embedded Linux systems. JFFS is implemented directly on the FLASH devices, cognizant of the erase sector boundaries, and size of the FLASH device.
The adoption of the file system was further sped up by the fact that a raw FLASH chip translation layer was already available- in the form of MTD (Memory Technology Device) device drivers. Using this HAL (Hardware Abstraction Layer) JFFS could be mounted on any raw random access device (i.e. any RAM, FLASH from any mfg.) as long as a MTD driver was available for it.
If one was not available, all you had to do was take an existing driver and modify the read/write/erase routine to your particular chip and mount JFFS on it- without knowing anything about JFFS itself. Combining the JFFS system with the device specific MTD (Memory Technology Device) raw FLASH chip device drivers already being developed for Linux, you could now have a complete solution with the MTD driver layer providing a level of abstraction for the JFFS file system layer.
In this manner JFFS was (is) not tied directly with any particular memory technology. Any device that can support, random (or even pseudo random- like NAND FLASH) can be interfaced with MTD, and hence with JFFS. By design, JFFS (and subsequently JFFS2) is designed to guarantee 1 (more on this in a minute) meta-data or "formatting" reliability of the file system layer. This means that if your write() system call returned, the data are guaranteed to be there even if power fails. This also means that if the power failed during the write() command and complete data did not get written to the FLASH chip, then either older data, newer data or a combination is guaranteed to be there. Your file will not get corrupted, successful write or not- power fail or not!
The original JFFS file system was designed as a "append only" type of file system. What this means that good data was never overwritten, even if you did a rewind() on an open file and then fwrite() to it. New data was always appended to the end of the last write to the fs. Some meta data written along with the block would make sure that the written block was written "logically" to the correct place in the file being written to. On restart (or mount), the entire file system would be scanned and the scattered blocks be re-arranged so that newer "stamped" blocks, that have logically overwritten older blocks were returned when that portion of the file was read. The older blocks would be marked for "garbage collection", to be deleted at a later time. One beneficial side effect of this append-only structure is that it provides a natural wear leveling on the FLASH, regardless if the file is being written to or not (i.e. even the FLASH being used by the static portions of the file system are wear levelled). Power Down Reliability of JFFS- the Reality I have done some extensive testing on the system and have submitted fixes (that are in the latest version of the CVS) that increases the power fail reliability of the JFFS file system from about 10 power fails (when I found it) to about 500+ reliable power fails. There is still a bug in the system, that causes the JFFS file system to loose some files (even static files) at random! I would NOT recommend this file system (as it currently stands) for production.
The solution? JFFS2. JFFS2 is the second version in the JFFS anthology. It was based on the design concepts of JFFS, but was implemented by Redhat (www.redhat.com). It takes a different track to reliability. In this system, each "erase sector" is managed independently and can be addressed out-of-order. A collection of known erase sectors is always kept available to use to add new files to the system (or to even overwrite older files on the system). Again, to guarantee power down reliability, no part of an existing file is ever overwritten, before the part that overwrites it is successfully (with CRC and version stamp) stored on the FLASH device. Then the older part can be marked for garbage collection and deleted during the erase of the sector in which it resides when all its neighbors have been either similarly marked or moved to another sector.
As an added bonus, JFFS2 also supports compression. All file data being written to the system gets compressed using zlib (and some custom mod's to it). Data being read out of the system are also (obviously) decompressed on the fly. So you can never really tell that your data are being compressed. Now you can use ASCII files for log or config files rather than binary files. Of course your binary files will also be compressed. Plus if your files are very "sparse", i.e. with lots of empty space in them, you can store them on JFFS2 without a penalty. One down side to this is that if you are writing compressed data to the system, the CPU will spend time trying to compress it further. At this time of writing, compression cannot be turned off. There are some plans to implement this feature however (even turn it off on a directory by directory basis!)."
On my M5407C3 Motorola board, I have a 16 bit FLASH memory (AMD Am29PL160C) present in the memory mapping between $7FE00000 and $7FFFFFFF.
You have to read carefully the technical data on your FLASH memory chip and find its sector address table:
My FLASH memory has 11 sectors with different sizes. Its contains the dBUG Motorola monitor in the first 256 Kbytes (SA0 to SA3). The rest of the memory is available to the user (SA4 to SA10, 256 Kbytes each). It is also possible to erase the dBUG monitor and use the entire memory (the dBUG monitor can be installed again with the BDM cable). I've choosen to define 2 MTD partitions:
If you erase the dBUG monitor, you have just one big user partition:
It's important to notice (read in the µClinux archive) that you have to create each MTD partition with at least 6contiguous sectors in order to use JFFS(2) (for garbage collection).
You have first to enable MTD/JFFS2 during the µClinux configuration.
% cd uClinux-dist
% make xconfig
These are the MTD options that I've enabled:
% grep MTD linux-2.4.x/.config
# Memory Technology Devices (MTD)
CONFIG_MTD=y
CONFIG_MTD_DEBUG=y
CONFIG_MTD_DEBUG_VERBOSE=3
CONFIG_MTD_PARTITIONS=y
CONFIG_MTD_CHAR=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
CONFIG_MTD_JEDECPROBE=y
CONFIG_MTD_GEN_PROBE=y
CONFIG_MTD_CFI_AMDSTD=y
CONFIG_MTD_PHYSMAP=y
CONFIG_MTD_PHYSMAP_START=0x7fe00000
CONFIG_MTD_PHYSMAP_LEN=0x200000
CONFIG_MTD_PHYSMAP_BUSWIDTH=2
With these MTD options, you have enabled the CONFIG_MTD_PHYSMAP
option (see the uClinux-dist/linux-2.4.x/drivers/mtd/maps/physmap.c
file), and you access to the entire FLASH memory with just one MTD partition.
For defining several MTD partitions, I've created a special physmap.c file (m5407c3.c file under the uClinux-dist/linux-2.4.x/drivers/mtd/maps/ directory).
I've also enabled the JFFS2 support during the µClinux kernel configuration:
I've also enabled MTD/JFFS2 tools in the userland area:
If you want several MTD partitions, you may modify the uClinux-dist/linux-2.4.x/drivers/mtd/maps/physmap.c file (see the Phil Wildshire's document for more explanations). I've choosen to create my own file that I've included in the µClinux distribution. You have several examples under the uClinux-dist/linux-2.4.x/drivers/mtd/maps/ directory. I've used the m5272c3.c as an example (written for a M5272C3 Motorola board).
I've obtained the m5407c3.c file:
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <asm/io.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/map.h>
#include <linux/mtd/partitions.h>
#include <linux/config.h>
#define WINDOW_ADDR 0x7fe00000
#define WINDOW_SIZE 0x200000
#define BUSWIDTH 2
static struct mtd_info *mymtd;
__u8 m5407c3_read8(struct map_info *map, unsigned long ofs)
{
return __raw_readb(map->map_priv_1 + ofs);
}
__u16 m5407c3_read16(struct map_info *map, unsigned long ofs)
{
return __raw_readw(map->map_priv_1 + ofs);
}
__u32 m5407c3_read32(struct map_info *map, unsigned long ofs)
{
return __raw_readl(map->map_priv_1 + ofs);
}
void m5407c3_copy_from(struct map_info *map, void *to, unsigned long from, ssize_t len)
{
memcpy_fromio(to, map->map_priv_1 + from, len);
}
void m5407c3_write8(struct map_info *map, __u8 d, unsigned long adr)
{
__raw_writeb(d, map->map_priv_1 + adr);
mb();
}
void m5407c3_write16(struct map_info *map, __u16 d, unsigned long adr)
{
__raw_writew(d, map->map_priv_1 + adr);
mb();
}
void m5407c3_write32(struct map_info *map, __u32 d, unsigned long adr)
{
__raw_writel(d, map->map_priv_1 + adr);
mb();
}
void m5407c3_copy_to(struct map_info *map, unsigned long to, const void *from, ssize_t len)
{
memcpy_toio(map->map_priv_1 + to, from, len);
}
struct map_info m5407c3_map = {
name: "MCF5407C3 flash device",
size: WINDOW_SIZE,
buswidth: BUSWIDTH,
read8: m5407c3_read8,
read16: m5407c3_read16,
read32: m5407c3_read32,
copy_from: m5407c3_copy_from,
write8: m5407c3_write8,
write16: m5407c3_write16,
write32: m5407c3_write32,
copy_to: m5407c3_copy_to
};
/*
* MTD 'PARTITIONING' STUFF
*/
static struct mtd_partition m5407c3_partitions[] = {
{
name: "dBUG (256K)",
size: 0x40000,
offset: 0x0
},
{
name: "user (1792K)",
size: 0x1c0000,
offset: 0x40000
}
};
int __init init_m5407c3(void)
{
printk(KERN_NOTICE "m5407c3 flash device: %x at %x\n", WINDOW_SIZE, WINDOW_ADDR);
m5407c3_map.map_priv_1 = (unsigned long)ioremap(WINDOW_ADDR, WINDOW_SIZE);
if (!m5407c3_map.map_priv_1) {
printk("Failed to ioremap\n");
return -EIO;
}
mymtd = do_map_probe("cfi_probe", &m5407c3_map);
if (mymtd) {
mymtd->module = THIS_MODULE;
mymtd->erasesize = 0x40000;
return add_mtd_partitions(mymtd, m5407c3_partitions,
sizeof(m5407c3_partitions) /
sizeof(struct mtd_partition));
}
iounmap((void *)m5407c3_map.map_priv_1);
return -ENXIO;
}
static void __exit cleanup_m5407c3(void)
{
if (mymtd) {
del_mtd_partitions(mymtd);
map_destroy(mymtd);
}
if (m5407c3_map.map_priv_1) {
iounmap((void *)m5407c3_map.map_priv_1);
m5407c3_map.map_priv_1 = 0;
}
}
module_init(init_m5407c3);
module_exit(cleanup_m5407c3);
The most important thing is the mtd_partition m5407c3_partitions[] structure that defines my 2 MTD partitions. (the rest was modified by a sed command under vi :-)).
In order to integrate this file in the µClinux distribution, I have:
if [ "$CONFIG_M5407C3" ]; then dep_tristate ' CFI Flash device mapped on Motorola M5407C3' CONFIG_MTD_M5407C3 $CONFIG_MTD_CFI fi
obj-$(CONFIG_MTD_M5407C3) += m5407c3.o
If you perform again a make xconfig, you have now:
% cd uClinux-dist
% make xconfig
I don't use now the CONFIG_MTD_PHYSMAP default option:
# Memory Technology Devices (MTD)
CONFIG_MTD=y
CONFIG_MTD_DEBUG=y
CONFIG_MTD_DEBUG_VERBOSE=3
CONFIG_MTD_PARTITIONS=y
CONFIG_MTD_CHAR=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
CONFIG_MTD_JEDECPROBE=y
CONFIG_MTD_GEN_PROBE=y
CONFIG_MTD_CFI_AMDSTD=y
CONFIG_MTD_M5407C3=y
You have to retrieve the Linux kernel version 2.4.19 (from here for example) and copy the files pushpull.c, zlib.c, zlib.h from the linux-2.4.19/fs/jffs2/ Linux kernel source directory under the uClinux-dist/linux-2.4.x/fs/jffs2/ µClinux source directory.
you have to modify the BLKMEM_MAJOR value from 31 to 30 in the files:
in order to have no confusion on major numbers between MTD and BLKMEM.
You have to add to the file uClinux-dist/vendors/Motorola/M5407C3/Makefile the following lines:
DEVICES = \
tty,c,5,0 console,c,5,1 cua0,c,5,64 cua1,c,5,65 \
mtd0,c,90,0 mtd1,c,90,2 mtd2,c,90,4 mtd3,c,90,6 \
mtd4,c,90,8 mtd5,c,90,10 mtd6,c,90,12 mtd7,c,90,14 \
mtdblock0,b,31,0 mtdblock1,b,31,1 mtdblock2,b,31,2 mtdblock3,b,31,3 \
mtdblock4,b,31,4 mtdblock5,b,31,5 mtdblock6,b,31,6 mtdblock7,b,31,7 \
mem,c,1,1 kmem,c,1,2 null,c,1,3 \
. . .
in order to create the device files used by MTD and JFFS2 under the /dev/ µClinux directory on the target device.
You have now to compile all the µClinux distribution:
% cd uClinux-dist
% make dep
% make
The last but not the least. You download the µClinux image file into the M5407C3 board by the network and launch the µClinux kernel:
Hard Reset
DRAM Size: 32M
Copyright 1995-2001 Motorola, Inc. All Rights Reserved.
ColdFire MCF5407 EVS Firmware v2e.1a.1b (Build 18 on Apr 20 2001 11:57:55)
Enter 'help' for help.
dBUG> dn
Eth Mac Addr is 00:00:00:00:00:01
Downloading Image 'image.bin' from 192.168.4.1
Read 1891552 bytes (3695 blocks)
dBUG> go 20000
These are the traces I've had on screen :
Linux version 2.4.19-uc1 (root@localhost.localdomain) (gcc version 2.95.3
uClinux/COLDFIRE(m5407)
COLDFIRE port done by Greg Ungerer, gerg@snapgear.com
Flat model support (C) 1998,1999 Kenneth Albanowski, D. Jeff Dionne
. . .
Starting kswapd
kmem_create: Forcing size word alignment - file lock cache
JFFS2 version 2.1. (C) 2001 Red Hat, Inc., designed by Axis Communications AB.
ColdFire internal UART serial driver version 1.00
ttyS0 at 0x100001c0 (irq = 73) is a builtin ColdFire UART
ttyS1 at 0x10000200 (irq = 74) is a builtin ColdFire UART
kmem_create: Forcing size word alignment - blkdev_requests
ne.c:v1.10 9/23/94 Donald Becker (becker@scyld.com)
Last modified Nov 1, 2000 by Paul Gortmaker
NE*000 ethercard probe at 0x40000300: 00 00 00 00 00 01
eth0: NE2000 found at 0x40000300, using IRQ 27.
SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
CSLIP: code copyright 1989 Regents of the University of California.
Blkmem copyright 1998,1999 D. Jeff Dionne
Blkmem copyright 1998 Kenneth Albanowski
Blkmem 1 disk images:
0: 107FA4-2013A3 [VIRTUAL 107FA4-2013A3] (RO)
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
PPP generic driver version 2.4.2
m5407c3 flash device: 200000 at 7fe00000
Amd/Fujitsu Extended Query Table v1.0 at 0x0040
number of CFI chips: 1
Creating 2 MTD partitions on "MCF5407C3 flash device":
0x00000000-0x00040000 : "dBUG (256K)"
mtd: partition "dBUG (256K)" doesn't end on an erase block -- force read-only
mtd: Giving out device 0 to dBUG (256K)
0x00040000-0x00200000 : "user (1792K)"
mtd: Giving out device 1 to user (1792K)
init_mtdchar: allocated major number 90.
init_mtdblock: allocated major number 31.
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP
. . .
Command: dhcpcd -p -a eth0 &
[14]
Command: cat /etc/motd
. . .
For further information check:
http://www.uclinux.org/
Execution Finished, Exiting
Sash command shell (version 1.1.1)
You can see that JFFS2 and MTD are enabled. The FLASH memory has been found by CFI probing and 2 MTD partitions (/dev/mtd0 and /dev/mtd1) have been created.
You can see the MTD partition listing:
/> cd /proc
/proc> cat mtd
dev: size erasesize name
mtd0: 00040000 00038000 "dBUG (256K)"
mtd1: 001c0000 00040000 "user (1792K)"
You can now create a JFFS2 image that you copy on the user MTD partition (/dev/mtd1):
/proc> cd /tmp
/var/tmp> mkdir fs
/var/tmp> mkdir jffs2
/var/tmp> mkdir jffs2/bin
/var/tmp> cd jffs2
/var/tmp/jffs2> vi file1
/var/tmp/jffs2> cat file1
coucou
/var/tmp/jffs2> cd ..
/var/tmp> mkfs.jffs2 -d jffs2 -o jffs2.img
/var/tmp> erase /dev/mtd1
/var/tmp> cp jffs2.img /dev/mtd1
MTD_open
MTD_write
MTD_close
You can now mount the JFFS2 partition:
/var/tmp> mount -t jffs2 /dev/mtdblock1 /mnt
mtdblock_open
ok
/var/tmp> cd /mnt
/mnt> ls
bin
file1
toto
/mnt> cd /proc
/proc> cat mounts
rootfs / rootfs rw 0 0
/dev/root / romfs ro 0 0
/proc /proc proc rw 0 0
/dev/ram1 /var ext2 rw 0 0
/dev/mtdblock1 /mnt jffs2 rw 0 0
You may control that you have all the requested device files under the /dev/ directory:
/proc> ls -l /dev
. . .
crw------- 1 0 0 90, 0 Jan 01 1970 mtd0
crw------- 1 0 0 90, 2 Jan 01 1970 mtd1
crw------- 1 0 0 90, 6 Jan 01 1970 mtd3
crw------- 1 0 0 90, 8 Jan 01 1970 mtd4
crw------- 1 0 0 90, 10 Jan 01 1970 mtd5
crw------- 1 0 0 90, 12 Jan 01 1970 mtd6
crw------- 1 0 0 90, 14 Jan 01 1970 mtd7
brw------- 1 0 0 31, 0 Jan 01 1970 mtdblock0
brw------- 1 0 0 31, 1 Jan 01 1970 mtdblock1
brw------- 1 0 0 31, 2 Jan 01 1970 mtdblock2
brw------- 1 0 0 31, 3 Jan 01 1970 mtdblock3
brw------- 1 0 0 31, 4 Jan 01 1970 mtdblock4
brw------- 1 0 0 31, 5 Jan 01 1970 mtdblock5
brw------- 1 0 0 31, 6 Jan 01 1970 mtdblock6
brw------- 1 0 0 31, 7 Jan 01 1970 mtdblock7
. . .
And finally, you can umount the JFFS2 partition:
/proc> umount /mnt
mtdblock_release
ok
That's all folks!
In this article, I've described step by step how to enable and use MTD/JFFS2 under µClinux. It was tested on a M5407C3 Motorola board.
It was tested with:
I've also used a Dell laptop (Inspiron 8200, P IV at 1,7 GHz) under RedHat 8.0.
Nobody is perfect. If you find inaccurate informations, you can contact me.