[OpenWrt-Devel] [PATCH] brcm2708: add squashfs rootfs image

Discussion:

Christian Lamparter

2018-03-27 17:42:18 UTC

This patch adds a image with squashfs as the root filesystem.
A rootfs_data partition will be generated on the first boot
and placed inside the rootfs partition (just after the squashfs
image).

advantages:
- it is possible to migrate from an existing -ext4
installation and back via sysupgrade.
- existing partition layout will not be lost.
- slightly smaller image size.
- support for attendedsysupgrade

disadvantages:
- needs f2fs + tools as well. This is because fs-tools decides on the
blocksize of the sdcard. So either f2fs or ext4 can get choosen as
the rootfs_data filesystem (depends on the size of the root partition).
- rootfs_data is placed into the rootfs partition. This makes
it difficult for tools that expect a /dev/mmc0pX device.
It also makes it difficult for data recovery tools since they
might not expect to find a embedded partition or will be
confused.

For people with existing build configurations: make sure to include mkf2fs
and f2fsck package into the image... Otherwise the new -squashfs image will
boot of a ram-overlay and won't keep the configurations after a reboot.

Cc: Álvaro Fernández Rojas <***@gmail.com>
Cc: Paul Spooren <***@informatik.uni-leipzig.de>
Cc: Daniel Golle <***@makrotopia.org>
Signed-off-by: Christian Lamparter <***@gmail.com>
---
Hello Álvaro,

The squashfs image was requested by Paul:
<https://github.com/aparcar/attendedsysupgrade-server/issues/88#issuecomment-365036459>
I'm not sure why, but he hardcoded his tool to only work with "squashfs".
<https://github.com/aparcar/attendedsysupgrade-server/blob/c8684b12d7a0b95222eec8aea20608d2595ce0ed/worker/worker.py#L209>

I've tested this with a raspberrypi model b. But I'm confident the
other targets rpis will work as well. What's your take on this?

Regards,
Christian
---
target/linux/brcm2708/Makefile | 5 +++--
target/linux/brcm2708/bcm2708/config-4.9 | 6 +++++-
target/linux/brcm2708/bcm2709/config-4.9 | 6 +++++-
target/linux/brcm2708/bcm2710/config-4.9 | 6 +++++-
target/linux/brcm2708/image/Makefile | 1 -
target/linux/brcm2708/image/cmdline.txt | 2 +-
6 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/target/linux/brcm2708/Makefile b/target/linux/brcm2708/Makefile
index 244513dfc6..870029c4c8 100644
--- a/target/linux/brcm2708/Makefile
+++ b/target/linux/brcm2708/Makefile
@@ -10,7 +10,7 @@ include $(TOPDIR)/rules.mk
ARCH:=arm
BOARD:=brcm2708
BOARDNAME:=Broadcom BCM27xx
-FEATURES:=ext4 audio usb usbgadget display gpio fpu
+FEATURES:=ext4 audio usb usbgadget display gpio fpu squashfs
MAINTAINER:=Álvaro Fernández Rojas <***@gmail.com>
SUBTARGETS:=bcm2708 bcm2709 bcm2710

@@ -27,7 +27,8 @@ DEFAULT_PACKAGES += \
kmod-usb-hid \
kmod-sound-core kmod-sound-arm-bcm2835 \
kmod-fs-vfat kmod-nls-cp437 kmod-nls-iso8859-1 \
- brcmfmac-firmware-43430-sdio kmod-brcmfmac wpad-mini
+ brcmfmac-firmware-43430-sdio kmod-brcmfmac wpad-mini \
+ partx-utils mkf2fs e2fsprogs

KERNELNAME:=Image dtbs

diff --git a/target/linux/brcm2708/bcm2708/config-4.9 b/target/linux/brcm2708/bcm2708/config-4.9
index dccf85fac3..116dd257dd 100644
--- a/target/linux/brcm2708/bcm2708/config-4.9
+++ b/target/linux/brcm2708/bcm2708/config-4.9
@@ -141,6 +141,11 @@ CONFIG_ENABLE_MUST_CHECK=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
+# CONFIG_F2FS_CHECK_FS is not set
+CONFIG_F2FS_FS=y
+# CONFIG_F2FS_FS_SECURITY is not set
+CONFIG_F2FS_FS_XATTR=y
+CONFIG_F2FS_STAT_FS=y
CONFIG_FB=y
CONFIG_FB_BCM2708=y
CONFIG_FB_CFB_COPYAREA=y
@@ -342,7 +347,6 @@ CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
CONFIG_SERIAL_OF_PLATFORM=y
CONFIG_SG_POOL=y
CONFIG_SPARSE_IRQ=y
-# CONFIG_SQUASHFS is not set
CONFIG_SRCU=y
# CONFIG_STRIP_ASM_SYMS is not set
CONFIG_SUSPEND=y
diff --git a/target/linux/brcm2708/bcm2709/config-4.9 b/target/linux/brcm2708/bcm2709/config-4.9
index 71884a5fa1..2571a4dc0a 100644
--- a/target/linux/brcm2708/bcm2709/config-4.9
+++ b/target/linux/brcm2708/bcm2709/config-4.9
@@ -155,6 +155,11 @@ CONFIG_ENABLE_MUST_CHECK=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
+# CONFIG_F2FS_CHECK_FS is not set
+CONFIG_F2FS_FS=y
+# CONFIG_F2FS_FS_SECURITY is not set
+CONFIG_F2FS_FS_XATTR=y
+CONFIG_F2FS_STAT_FS=y
CONFIG_FB=y
CONFIG_FB_BCM2708=y
CONFIG_FB_CFB_COPYAREA=y
@@ -372,7 +377,6 @@ CONFIG_SG_POOL=y
CONFIG_SMP=y
CONFIG_SMP_ON_UP=y
CONFIG_SPARSE_IRQ=y
-# CONFIG_SQUASHFS is not set
CONFIG_SRCU=y
# CONFIG_STRIP_ASM_SYMS is not set
CONFIG_SUSPEND=y
diff --git a/target/linux/brcm2708/bcm2710/config-4.9 b/target/linux/brcm2708/bcm2710/config-4.9
index d402f4b573..87e0289347 100644
--- a/target/linux/brcm2708/bcm2710/config-4.9
+++ b/target/linux/brcm2708/bcm2710/config-4.9
@@ -162,6 +162,11 @@ CONFIG_ENABLE_MUST_CHECK=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
+# CONFIG_F2FS_CHECK_FS is not set
+CONFIG_F2FS_FS=y
+# CONFIG_F2FS_FS_SECURITY is not set
+CONFIG_F2FS_FS_XATTR=y
+CONFIG_F2FS_STAT_FS=y
CONFIG_FB=y
CONFIG_FB_BCM2708=y
CONFIG_FB_CFB_COPYAREA=y
@@ -393,7 +398,6 @@ CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM_VMEMMAP=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPARSE_IRQ=y
-# CONFIG_SQUASHFS is not set
CONFIG_SRCU=y
# CONFIG_STRIP_ASM_SYMS is not set
CONFIG_SUSPEND=y
diff --git a/target/linux/brcm2708/image/Makefile b/target/linux/brcm2708/image/Makefile
index 9ea9a07a9d..28dfa70628 100644
--- a/target/linux/brcm2708/image/Makefile
+++ b/target/linux/brcm2708/image/Makefile
@@ -47,7 +47,6 @@ endef

### Devices ###
define Device/Default
- FILESYSTEMS := ext4
KERNEL := kernel-bin | kernel-img
KERNEL_IMG := kernel.img
IMAGES := sdcard.img.gz
diff --git a/target/linux/brcm2708/image/cmdline.txt b/target/linux/brcm2708/image/cmdline.txt
index 24f4f0c276..c1099afb00 100644
--- a/target/linux/brcm2708/image/cmdline.txt
+++ b/target/linux/brcm2708/image/cmdline.txt
@@ -1 +1 @@
-dwc_otg.lpm_enable=0 console=serial0,115200 kgdboc=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait
+dwc_otg.lpm_enable=0 console=serial0,115200 kgdboc=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=squashfs,ext4 rootwait

--
2.16.3

Daniel Golle

2018-03-29 00:22:48 UTC

Permalink

Post by Christian Lamparter
This patch adds a image with squashfs as the root filesystem.
A rootfs_data partition will be generated on the first boot
and placed inside the rootfs partition (just after the squashfs
image).
- it is possible to migrate from an existing -ext4
installation and back via sysupgrade.
- existing partition layout will not be lost.
- slightly smaller image size.
- support for attendedsysupgrade
- needs f2fs + tools as well. This is because fs-tools decides on the
blocksize of the sdcard. So either f2fs or ext4 can get choosen as
the rootfs_data filesystem (depends on the size of the root partition).
- rootfs_data is placed into the rootfs partition. This makes
it difficult for tools that expect a /dev/mmc0pX device.
It also makes it difficult for data recovery tools since they
might not expect to find a embedded partition or will be
confused.
For people with existing build configurations: make sure to include mkf2fs
and f2fsck package into the image... Otherwise the new -squashfs image will
boot of a ram-overlay and won't keep the configurations after a reboot.

Acked-by: Daniel Golle <***@makrotopia.org>

Daniel Golle

2018-04-12 13:30:18 UTC

Permalink

Álvaro, please review the patch or at least give us a sign of life.

Cheers

Daniel

Stijn Tintel

2018-04-24 11:29:17 UTC

Permalink

Tried to sysupgrade one of my RPi0W from ext4 to squashfs, and it
appeared to work at first, until I rebooted it again. Apparently the
overlay is completely missing, the SD card only has /boot (FAT32) and /
(squashfs). Can you please look into this? If you can get it resolved I
will merge the patch.

Stijn

Christian Lamparter

2018-04-24 14:59:34 UTC

Permalink

Post by Stijn Tintel

I think know the problem. The commit-message mentions it and the symptoms
you are describing would fit. But just in case: can you please provide the
image's diffconfig (or full-config ... and I guess a bootlog wouldn't
hurt either - but it would be optional) to confirm?

(I also tested it again on an uptodate "OpenWrt SNAPSHOT, r6759+2-9e848f901c"
with my rpi-b - which should be in the same family/subtarget as your RPi0W so
I can try your generated images as well.)

Post by Stijn Tintel
If you can get it resolved I will merge the patch.

The original issue [0] has been addressed by this patch [1]. And indeed
the luci-app will now successfully generate a image (currently, only for the
rpi-3 though!):
"openwrt-dbbb09d2658d5c2-brcm2708-bcm2710-rpi-3-ext4-sdcard.img.gz"
and install it.

this patch is therefore no longer necessary since it the existing ext4
images work as well. :) ... Plus the ext4 images don't confuse tools and
brains as much.

Regards,
Christian

[0] <https://github.com/aparcar/attendedsysupgrade-server/issues/88>
[1] <https://github.com/aparcar/attendedsysupgrade-server/commit/d477a87db41ae1e44325acf22fd4cb37d269f8b9#diff-b7368b46572bf04e88e8fcd94d072a1d>

Stijn Tintel

2018-04-24 16:17:33 UTC

Permalink

Post by Christian Lamparter

Post by Stijn Tintel

Please find diffconfig attached. The machine was running headless so I
don't have a bootlog.

Post by Christian Lamparter
(I also tested it again on an uptodate "OpenWrt SNAPSHOT, r6759+2-9e848f901c"
with my rpi-b - which should be in the same family/subtarget as your RPi0W so
I can try your generated images as well.)

Post by Stijn Tintel
If you can get it resolved I will merge the patch.

The original issue [0] has been addressed by this patch [1]. And indeed
the luci-app will now successfully generate a image (currently, only for the
"openwrt-dbbb09d2658d5c2-brcm2708-bcm2710-rpi-3-ext4-sdcard.img.gz"
and install it.
this patch is therefore no longer necessary since it the existing ext4
images work as well. :) ... Plus the ext4 images don't confuse tools and
brains as much.

I still find it useful, I've had way too many corrupted ext4 filesystems
on my RPis, and I'm hoping squashfs + f2fs would be more reliable than
ext4 on SD cards...

Thanks,
Stijn

Stijn Tintel

2018-04-24 17:56:38 UTC

Permalink

Post by Stijn Tintel

Post by Christian Lamparter

Post by Stijn Tintel

Please find diffconfig attached. The machine was running headless so I
don't have a bootlog.

Post by Stijn Tintel
If you can get it resolved I will merge the patch.

I still find it useful, I've had way too many corrupted ext4 filesystems
on my RPis, and I'm hoping squashfs + f2fs would be more reliable than
ext4 on SD cards...

Read the commit message again, and I see what you mean. Must have
misread CONFIG_DEFAULT_mkf2fs=y as CONFIG_PACKAGE_mkf2fs=y. Too bad make
defconfig doesn't enable DEFAULT_PACKAGES. Sorry for the noise, I'll do
some more testing and accept this patch.

Thanks,
Stijn

Christian Lamparter

2018-04-24 18:07:40 UTC

Permalink

Post by Stijn Tintel

Post by Christian Lamparter

Post by Stijn Tintel

Please find diffconfig attached. The machine was running headless so I
don't have a bootlog.

Post by Stijn Tintel
If you can get it resolved I will merge the patch.

I still find it useful, I've had way too many corrupted ext4 filesystems
on my RPis, and I'm hoping squashfs + f2fs would be more reliable than
ext4 on SD cards...

Well, if you want to go through with it, you could always add more
"WARNING - DANGER - ATTENTION" stickers and point out that
CONFIG_DEFAULT_mkf2fs=y != CONFIG_PACKAGE_mkf2fs=y . So people will
really watch out.

Regards,
Christian