OpenBSD 7.1 on PINE64 RockPro64

June 7, 2022

This is a small write-up about installing OpenBSD 7.1 on a PINE64 RockPro64 SBC.

Preamble

RockPro64 is a beefy single-board computer made by a company that brought us awesome devices like Pinebook Pro (laptop), Pinecil (soldering iron), PineTime (smartwatch) and of course PinePhone. The board utilizes the same hexa-core processor as Pinebook Pro - Rockchip RK3399, and 4 gigabytes of LPDDR4 RAM.

One of the distinct features of that computer is a PCI-express X4 socket. Unfortunately I wasn’t able to use any video card there even with “stock” GNU/Linux - particular PCIe implementations don’t work with memory mappings required for GPUs to work¹. The slot is often being used for a network cards and SATA controllers - there is even an official case for RP64 with 3.5" hard drives spots inside, quite handy for a homemade NAS or something of sorts.

I am using a different case that doubles as a heat sink for the RK3399 as it runs relatively hot under heavy load especially.

Prerequisites

A microsd card of sufficient size (I’d recommend 8+gb) or an eMMC chip.
In case of eMMC an USB adapter is quite handy.
An USB UART adapter based on FT232RL or CH341 like this one.
A separate computer with USB, obviously.
A wired LAN connection (since RP64 doesn’t have onboard WiFi module by default).

Storage choice

Since I’ve obtained the board I’ve ran OpenBSD on a microsd card there and while it has its perks if used as a disposable/read-only storage it is usually not a good choice to run an operating system from, especially because FFS (OpenBSD default FS) is not particularly optimized for that kind of flash storage.

Apart from microsd as boot/main drive the board supports eMMC chips up to 128gb, in my opinion this is a far better option for using with any SBC long-term (except SSD/HDD obviously).

There is no difference in the installation procedure between those variants though.

Brief boot sequence theory

Unlike x86_64 ARM ecosystem does not have unified boot routine - hardware and especially memory initialization process is vastly different between CPUs, unlike old BIOS or modern UEFI. Das U-Boot is the most successful attempt to fix that and nowadays even UEFI is a viable option for some arm64 devices (notably Raspberry Pi 4/400).

Unfortunately that means that it is necessary to use a specific u-boot builds for each different board out there. Fortunately for us OpenBSD already has prebuilt images to write onto storage to successfully boot on RockPro64; having mainline support is awesome in general.

I’ll try to not dive deep into RK3399 process since I’m no expert; essentially the internal CPU boot ROM will execute the code from eMMC/microsd at the 32768 bytes offset ( 64 sectors of 512 bytes)². At this point though the RAM is not fully initialized and only small amount is available to the CPU so the whole bootloader cannot be fit there; it inits “main” DDR memory and chainloads the next piece of puzzle at the offset of 16384 sectors.

Once U-Boot is fully chainloaded it fires off proper OpenBSD bootloader.

The process described goes only for eMMC/MicroSD boot, not SPI flash - I avoid using that because the “u-boot flavor” there would be hardwired for certain OS, so if you flash u-boot made for Linux you won’t be able to boot other OS.

Required downloads as of OpenBSD v7.1

If you have any OpenBSD installation you can get the bootloader binaries from u-boot-aarch64 and dtb binary packages. Otherwise just download their tarballs from here: https://cdn.openbsd.org/pub/OpenBSD/7.1/packages/aarch64/

Obviously we wouldn’t be able to install anything without an install image itself - the usual place to grab it is from the official website: https://cdn.openbsd.org/pub/OpenBSD/7.1/arm64/install71.img

Preparing the boot media

In examples below I’ll be using sd5 as target disk and OpenBSD as a host system.

First we dump the install image onto the drive - microsd or eMMC connected via adapter:

dd if=install71.img of=/dev/rsd5c bs=1024M

Next thing is to dump both parts of bootloader:

dd if=/usr/local/share/u-boot/rockpro64-rk3399/idbloader.img of=/dev/rsd5c bs=512 seek=64
dd if=/usr/local/share/u-boot/rockpro64-rk3399/u-boot.itb of=/dev/rsd5c bs=512 seek=16384

(Be sure to provide valid paths in case you’re not on OpenBSD and unpacked u-boot tarball somewhere else.)

DTB is an acronym for the “Device Tree Binary”. It is a blob that has a board hardware description for operating system to use - SPI/flash storage, memory mappings and so on. We have that prebuilt from a package mentioned above - all is need to unpack specific files needed to the boot partition.

Install image contains a FAT32 partition that we have to mount and copy files onto:

mount /dev/sd5i /mnt
mkdir -p /mnt/dtb/rockchip
cp /usr/local/share/dtb/arm64/rockchip/rk3399-rockpro64* /mnt/dtb/rockchip/
umount /mnt

Hooking up the serial console

Default install media OpenBSD kernel cannot use the GPU on the RockPro64 so it is necessary to be able to interact with the installer, that’s where serial console comes into play. We need 3 wires connected from USB UART to the board:

UART GND -> BOARD PIN 6 (GND)
UART RXD -> BOARD PIN 8 (TXD)
UART TXD -> BOARD PIN 10 (RXD)

Using serial adapter is trivial using many tools (screen, minicom, picocom, cu), I’ll be using cu since it is default and preinstalled OpenBSD program:

cu -l cuaU0 -s 115200

You can check dmesg in your OS after plugging in the adapter to get the name of the device (cuaU0, ttyUSB0 or something else).

OS installation tips

General installation procedure is the same boring OpenBSD stuff that just always works anyway.

Notable detail is that when the installer suggests wiping whole storage to write down a whole new partition table - don’t let it, we have precious code at offsets 64 and 16384 and definitely don’t want to lose that. So in the fdisk question it is needed to only edit the 4th partition (OpenBSD one) and extend its boundaries to the end of the storage. Don’t touch the default “begin” sector though, only move the last one so partition takes the majority of disk.

Reusing an existing partition is the last tricky thing; disklabel-ing it can be done as usual with automatic partitioning.

Some older guides³ suggested copying DTB files over to the fat32 boot partition all over again after the installation is finishes; I found this redundant because after-install scripts copied those files onto sd0i without my assist just fine.

This is a full dmesg of an installed system:

OpenBSD 7.1 (GENERIC.MP) #3: Sun May 15 09:18:22 MDT 2022
    root@syspatch-71-arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP
real mem  = 4038729728 (3851MB)
avail mem = 3839430656 (3661MB)
random: good seed from bootblocks
mainbus0 at root: Pine64 RockPro64 v2.1
psci0 at mainbus0: PSCI 1.1, SMCCC 1.2
cpu0 at mainbus0 mpidr 0: ARM Cortex-A53 r0p4
cpu0: 32KB 64b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache
cpu0: 512KB 64b/line 16-way L2 cache
cpu0: CRC32,SHA2,SHA1,AES+PMULL,ASID16
cpu1 at mainbus0 mpidr 1: ARM Cortex-A53 r0p4
cpu1: 32KB 64b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache
cpu1: 512KB 64b/line 16-way L2 cache
cpu1: CRC32,SHA2,SHA1,AES+PMULL,ASID16
cpu2 at mainbus0 mpidr 2: ARM Cortex-A53 r0p4
cpu2: 32KB 64b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache
cpu2: 512KB 64b/line 16-way L2 cache
cpu2: CRC32,SHA2,SHA1,AES+PMULL,ASID16
cpu3 at mainbus0 mpidr 3: ARM Cortex-A53 r0p4
cpu3: 32KB 64b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache
cpu3: 512KB 64b/line 16-way L2 cache
cpu3: CRC32,SHA2,SHA1,AES+PMULL,ASID16
cpu4 at mainbus0 mpidr 100: ARM Cortex-A72 r0p2
cpu4: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
cpu4: 1024KB 64b/line 16-way L2 cache
cpu4: CRC32,SHA2,SHA1,AES+PMULL,ASID16
cpu5 at mainbus0 mpidr 101: ARM Cortex-A72 r0p2
cpu5: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
cpu5: 1024KB 64b/line 16-way L2 cache
cpu5: CRC32,SHA2,SHA1,AES+PMULL,ASID16
efi0 at mainbus0: UEFI 2.8
efi0: Das U-Boot rev 0x20211000
apm0 at mainbus0
agintc0 at mainbus0 sec shift 3:3 nirq 288 nredist 6 ipi: 0, 1: "interrupt-controller"
agintcmsi0 at agintc0
syscon0 at mainbus0: "qos"
syscon1 at mainbus0: "qos"
syscon2 at mainbus0: "qos"
syscon3 at mainbus0: "qos"
syscon4 at mainbus0: "qos"
syscon5 at mainbus0: "qos"
syscon6 at mainbus0: "qos"
syscon7 at mainbus0: "qos"
syscon8 at mainbus0: "qos"
syscon9 at mainbus0: "qos"
syscon10 at mainbus0: "qos"
syscon11 at mainbus0: "qos"
syscon12 at mainbus0: "qos"
syscon13 at mainbus0: "qos"
syscon14 at mainbus0: "qos"
syscon15 at mainbus0: "qos"
syscon16 at mainbus0: "qos"
syscon17 at mainbus0: "qos"
syscon18 at mainbus0: "qos"
syscon19 at mainbus0: "qos"
syscon20 at mainbus0: "qos"
syscon21 at mainbus0: "qos"
syscon22 at mainbus0: "qos"
syscon23 at mainbus0: "qos"
syscon24 at mainbus0: "qos"
syscon25 at mainbus0: "power-management"
"power-controller" at syscon25 not configured
syscon26 at mainbus0: "syscon"
"io-domains" at syscon26 not configured
rkclock0 at mainbus0
rkclock1 at mainbus0
syscon27 at mainbus0: "syscon"
"io-domains" at syscon27 not configured
"usb2phy" at syscon27 not configured
"usb2phy" at syscon27 not configured
rkemmcphy0 at syscon27
"pcie-phy" at syscon27 not configured
rktcphy0 at mainbus0
rktcphy1 at mainbus0
rkpinctrl0 at mainbus0: "pinctrl"
rkgpio0 at rkpinctrl0
rkgpio1 at rkpinctrl0
rkgpio2 at rkpinctrl0
rkgpio3 at rkpinctrl0
rkgpio4 at rkpinctrl0
pwmreg0 at mainbus0
rkdrm0 at mainbus0
drm0 at rkdrm0
"pmu_a53" at mainbus0 not configured
"pmu_a72" at mainbus0 not configured
agtimer0 at mainbus0: 24000 kHz
"xin24m" at mainbus0 not configured
rkpcie0 at mainbus0
rkpcie0: link training timeout
dwge0 at mainbus0: rev 0x35, address 3e:16:08:b2:57:d5
rgephy0 at dwge0 phy 0: RTL8169S/8110S/8211 PHY, rev. 6
dwmmc0 at mainbus0: 50 MHz base clock
sdmmc0 at dwmmc0: 4-bit, sd high-speed, dma
dwmmc1 at mainbus0: 50 MHz base clock
sdmmc1 at dwmmc1: 4-bit, sd high-speed, dma
sdhc0 at mainbus0
sdhc0: SDHC 3.0, 200 MHz base clock
sdmmc2 at sdhc0: 8-bit, sd high-speed, mmc high-speed, dma
ehci0 at mainbus0
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Generic EHCI root hub" rev 2.00/1.00 addr 1
ohci0 at mainbus0: version 1.0
ehci1 at mainbus0
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Generic EHCI root hub" rev 2.00/1.00 addr 1
ohci1 at mainbus0: version 1.0
rkdwusb0 at mainbus0: "usb"
xhci0 at rkdwusb0, xHCI 1.10
usb2 at xhci0: USB revision 3.0
uhub2 at usb2 configuration 1 interface 0 "Generic xHCI root hub" rev 3.00/1.00 addr 1
rkdwusb1 at mainbus0: "usb"
xhci1 at rkdwusb1, xHCI 1.10
usb3 at xhci1: USB revision 3.0
uhub3 at usb3 configuration 1 interface 0 "Generic xHCI root hub" rev 3.00/1.00 addr 1
"saradc" at mainbus0 not configured
rkiic0 at mainbus0
iic0 at rkiic0
escodec0 at iic0 addr 0x11
rkiic1 at mainbus0
iic1 at rkiic1
com0 at mainbus0: dw16550, 64 byte fifo
com1 at mainbus0: dw16550, 64 byte fifo
com1: console
"spi" at mainbus0 not configured
rktemp0 at mainbus0
rkrng0 at mainbus0
rkiic2 at mainbus0
iic2 at rkiic2
rkpmic0 at iic2 addr 0x1b: RK808
fanpwr0 at iic2 addr 0x40: SYR827, 1.00 VDC
fanpwr1 at iic2 addr 0x41: SYR828, 1.00 VDC
rkiic3 at mainbus0
iic3 at rkiic3
fusbtc0 at iic3 addr 0x22
rkpwm0 at mainbus0
rkpwm1 at mainbus0
rkpwm2 at mainbus0
"video-codec" at mainbus0 not configured
"iommu" at mainbus0 not configured
"video-codec" at mainbus0 not configured
"iommu" at mainbus0 not configured
"rga" at mainbus0 not configured
"efuse" at mainbus0 not configured
"dma-controller" at mainbus0 not configured
"dma-controller" at mainbus0 not configured
"watchdog" at mainbus0 not configured
"rktimer" at mainbus0 not configured
rkiis0 at mainbus0
rkiis1 at mainbus0
rkiis2 at mainbus0
rkvop0 at mainbus0: RK3399 VOPL
"iommu" at mainbus0 not configured
rkvop1 at mainbus0: RK3399 VOPB
"iommu" at mainbus0 not configured
"iommu" at mainbus0 not configured
"iommu" at mainbus0 not configured
simpleaudio0 at mainbus0
rkdwhdmi0 at mainbus0: HDMI TX
rkdwhdmi0: version 2.11a, phytype 0xf3
"gpu" at mainbus0 not configured
"opp-table0" at mainbus0 not configured
"opp-table1" at mainbus0 not configured
"opp-table2" at mainbus0 not configured
"external-gmac-clock" at mainbus0 not configured
gpiokeys0 at mainbus0: "GPIO Key Power"
"ir-receiver" at mainbus0 not configured
gpioleds0 at mainbus0: "work", "diy"
pwmfan0 at mainbus0: no cooling levels
"sdio-pwrseq" at mainbus0 not configured
graphaudio0 at mainbus0
graphaudio1 at mainbus0
"spdif-dit" at mainbus0 not configured
"vcc12v-dcin" at mainbus0 not configured
"vcc1v8-s3" at mainbus0 not configured
"vcc3v0-sd" at mainbus0 not configured
"vcc3v3-pcie-regulator" at mainbus0 not configured
"vcc3v3-sys" at mainbus0 not configured
"vcc5v0-host-regulator" at mainbus0 not configured
"vcc5v0-typec-regulator" at mainbus0 not configured
"vcc5v0-sys" at mainbus0 not configured
"vcc5v0-usb" at mainbus0 not configured
"framebuffer" at mainbus0 not configured
usb4 at ohci0: USB revision 1.0
uhub4 at usb4 configuration 1 interface 0 "Generic OHCI root hub" rev 1.00/1.00 addr 1
usb5 at ohci1: USB revision 1.0
uhub5 at usb5 configuration 1 interface 0 "Generic OHCI root hub" rev 1.00/1.00 addr 1
audio0 at graphaudio0
sdmmc0: can't enable card
scsibus0 at sdmmc2: 2 targets, initiator 0
sd0 at scsibus0 targ 1 lun 0:  removable
sd0: 118000MB, 512 bytes/sector, 241664000 sectors
uhidev0 at uhub4 port 1 configuration 1 interface 0 "HAILUCK CO.,LTD USB KEYBOARD" rev 1.10/1.00 addr 2
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0 mux 1
uhidev1 at uhub4 port 1 configuration 1 interface 1 "HAILUCK CO.,LTD USB KEYBOARD" rev 1.10/1.00 addr 2
uhidev1: iclass 3/0, 9 report ids
ums0 at uhidev1 reportid 1: 5 buttons, Z and W dir
wsmouse0 at ums0 mux 0
uhid0 at uhidev1 reportid 2: input=1, output=0, feature=0
ucc0 at uhidev1 reportid 3: 24 usages, 13 keys, enum
wskbd1 at ucc0 mux 1
uhid1 at uhidev1 reportid 5: input=0, output=0, feature=5
uhid2 at uhidev1 reportid 6: input=0, output=0, feature=255
uhid3 at uhidev1 reportid 9: input=1, output=0, feature=0
uhidev2 at uhub5 port 1 configuration 1 interface 0 "Hoksi Technology DURGOD Taurus K320" rev 2.00/1.04 addr 2
uhidev2: iclass 3/1
ukbd1 at uhidev2: 8 variable keys, 6 key codes
wskbd2 at ukbd1 mux 1
uhidev3 at uhub5 port 1 configuration 1 interface 1 "Hoksi Technology DURGOD Taurus K320" rev 2.00/1.04 addr 2
uhidev3: iclass 3/1, 4 report ids
uhid4 at uhidev3 reportid 1: input=1, output=0, feature=0
ucc1 at uhidev3 reportid 2: 573 usages, 18 keys, array
wskbd3 at ucc1 mux 1
ukbd2 at uhidev3 reportid 3: 104 variable keys, 0 key codes
wskbd4 at ukbd2 mux 1
ums1 at uhidev3 reportid 4: 5 buttons, Z and W dir
wsmouse1 at ums1 mux 0
uhidev4 at uhub5 port 1 configuration 1 interface 2 "Hoksi Technology DURGOD Taurus K320" rev 2.00/1.04 addr 2
uhidev4: iclass 3/0
uhid5 at uhidev4: input=64, output=64, feature=64
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
root on sd0a (eec5a595eb8339a3.a) swap on sd0b dump on sd0b
WARNING: bad clock chip time
WARNING: CHECK AND RESET THE DATE!
rkvop0: using CRTC 0 for RK3399 VOPL
rkvop1: using CRTC 1 for RK3399 VOPB
rkdrm0: 1920x1080, 32bpp
wsdisplay0 at rkdrm0 mux 1
wskbd0: connecting to wsdisplay0
wskbd1: connecting to wsdisplay0
wskbd2: connecting to wsdisplay0
wskbd3: connecting to wsdisplay0
wskbd4: connecting to wsdisplay0
wsdisplay0: screen 0-5 added (std, vt100 emulation)

Lobsters’ denizens kindly corrected me on the topic https://lobste.rs/s/vxoulp/openbsd_7_1_on_pine64_rockpro64#c_govyh6 ↩︎
https://github.com/sigmaris/u-boot/wiki/RockPro64-boot-sequence ↩︎
https://github.com/jasperla/openbsd-rockpro64 ↩︎