How to compile OpenWrt and still use the official repository

Overview

We all know what OpenWrt is. The amazing Linux distro built specifically for embedded devices.

What you can achieve with a rather cheap router running OpenWrt, is mind-boggling.

OpenWrt also gives you a great control over its build system. For normal cases, you probably don’t need to build OpenWrt from source yourself. That has been done for you already and all you need to do, is to just download the appropriate compiled firmware image and then upload it to your router¹.

But for more advanced usages, you may find yourself needing to build OpenWrt images yourself. This could be due wanting to make some changes to the code, add some device specific options, etc.

Building OpenWrt from source is easy, well-documented, and works great. That is, until you start using opkg to install some new packages.

opkg will by default fetch new packages from the official repository (as one might expect), but depending on the package, the installation may or may not fail.

The problem

In almost all cases, opkg fails when it tries to install a kernel module (or the so called kmods) and any packages that depend on them.

So assuming that you’ve built OpenWrt from lets say v18.06.4 source, opkg will use the v18.06.4 online dedicated repository to install new packages. Now if your package happens to depend on a kernel module, the installation will likely fail.

For example, openvpn-openssl package, depends on kmod-tun. Trying to install it on a self-build image using the official repository, could result in an error like this:

root@OpenWrt:~# opkg update && opkg install openvpn-openssl
Installing openvpn-openssl (2.4.5-4.2) to root...
Downloading http://downloads.openwrt.org/releases/18.06.4/packages/x86_64/base/openvpn-openssl_2.4.5-4.2_x86_64.ipk
Collected errors:
 * satisfy_dependencies_for: Cannot satisfy the following dependencies for openvpn-openssl:
 * 	kernel (= 4.14.131-1-7f44b66739e15d9ee98957787ab1e200)
 * opkg_install_cmd: Cannot install package openvpn-openssl.

Depending on the platform your router is running on, the message might be slightly different (namely the download source and the so called vermagic value), but either way opkg fails even though you have used the same exact build source.

As a safeguard, the kernel itself and all kernel modules, come with a vermagic value that represents the environment they’ve been built on. If that value differs between the kernel and a module, it is an error.

For example in the above message, 7f44b66739e15d9ee98957787ab1e200 is the official OpenWrt version magic value for this specific target system and release. If you want to use kmod-tun module from that build, your kernel vermagic should have the same value as the official one. In our case it did not, hence the error.

Now to fix this, you may decide to build the offending packages at the build time. You can either embed them in the image or just install them as separate packages later on, and that works.
But what if you need another extra kernel module a few months down the road? Not to mention that if a new version of one of the packages with a kernel module comes along, you may have a hard time upgrading it.

There is a better solution…

Behind the scenes

The trick is to find out how version magic is generated in OpenWrt so we can build it in the same environment, hence reproducing the same number.

in OpenWrt, “./include/kernel-defaults.mk” is where this magic value is being generated:

grep '=[ym]' $(LINUX_DIR)/.config.set | LC_ALL=C sort | mkhash md5 > $(LINUX_DIR)/.vermagic

Now this tells us pretty much everything we need to know. As part of building kernel, .config.set file is created. This file includes all the applied kernel settings. What the above line of code does, is basically making a MD5 hash out of all the settings (including selected modules).

To get the same vermagic value, not only we need to use the same exact OpenWrt source version, but also build our image with the same exact set of kernel settings and modules.

This of course is possible using config.seed file which we will discuss later.

It’s also worth noting that making changes to normal packages, does not change the version magic. Only related kernel settings could affect that.

config.seed

In newer versions of OpenWrt, When you build an image, a diffconfig file gets automatically generated. This file is called config.seed.

In a nutshell, config.seed includes all the required changes for building the same image again. In other words, it contains all the changes that’s been applied to an image compared to the default configuration.

Using This file makes it possible to re-create the exact configuration that’s been used to build an image.

The config.seed file used for each individual official build, is available in the same place as the firmware for that build. For example assuming your target is x86-64, both the firmware and the config.seed file is available at this link:

https://downloads.openwrt.org/releases/18.06.4/targets/x86/64/

kmods

One of the most important options in any official config.seed file, is CONFIG_ALL_KMODS=y.

That single line, tells the build system to build every kernel module there is (and as we’ve discussed, this affects the magic value).

After applying the config.seed settings to your own build (we will cover this soon), you will quickly realize that all kernel modules are selected for the build. This is what you want.

Now you may decide to change the state of some of these kernel modules (that is setting them to built-in (<*>) as opposed to being external (<M>)). This can also happen automatically if you set a package as built-in and the package itself happens to depend on a kernel module (e.g, openvpn-openssl). As long as you do not deselect any of them, switching their state, does not change the magic value.

The solution

Now that we know what’s going on, we can come up with a build procedure that is compatible with the official repository so we could install/upgrade any packages later on.

1. Prepare the build system

Well, this is an easy one and I’m sure you already know how to. If not, take look at the official documentation.

2. Get the source

Now it’s time to get the official OpenWrt source:

git clone https://git.openwrt.org/openwrt/openwrt.git && cd openwrt

And then checkout the specific version that you want to build.

For example, assuming that we want to use v18.06.4, we do the followings:

git checkout v18.06.4

3. Update and install feeds

Now you need to update all the feeds and make all packages available to the build system:

./scripts/feeds update -a && ./scripts/feeds install -a

4. Use config.seed as your .config file

Find the correct config.seed file for the same OpenWrt version that you’re building. This file differs for each architecture so be sure to use the right one. This will serve as our .config file starting point.

Assuming that we are building OpenWrt v18.06.4 for x86-64 architecture, the required command for doing so, would be something like this:

wget -O .config https://downloads.openwrt.org/releases/18.06.4/targets/x86/64/config.seed

4.1 Exclude LuCi from the build

At this stage you are able to completely exclude LuCi from the build if you want so.

Take a look at my next post on How to properly remove LuCi from OpenWrt, for further information on how to do so.

5. Expand .config file

We need to expand our compact .config file now. We do this by issuing this command:

make defconfig

6. Apply all necessary changes

At this point, you are pretty much done with the initial setup. You may now use make menuconfig to:

• Include some new packages
• potentially exclude some normal packages from the build
• Change some non-kernel options

You may also use quilt to patch some packages.

In a nutshell, whatever you do as long as it’s not kernel related, should be fine.

7. Check the vermagic

Building full image can take a long time. You don’t want to build the whole image and apply it your router just to discover that the changes you’ve made in the previous step, affected the version magic value.

Before building the full image, it is possible to only build the kernel and spot the vermagic there.

You build the kernel by issuing:

make target/linux/{clean,compile}

You may now check the version magic value of your soon-to-be-made image:

find build_dir/ -name .vermagic -exec cat {} \;

Note that for things to work, this value must be the same as the official builds (as we’ve discussed before, you can check the official build vermagic, by searching for kernel entry in the .manifest file. This file is available in the same location as config.seed).

If the value did not match, you need to go back and find out what kind of changes causing this, fix it and check again.

8. Download and build

Following the official guide for building images, you will use something like this to build your image:

make download && make -j5

Final words

Congratulation! You are done. You should now be able to upload your image to your router and use opkg to update/install packages without any limitation.

It goes without saying that if you’ve applied some custom changes to a package in the build time, you should be careful when upgrading it from the official repository (as it may revert your changes).

And as always, I’d appreciate your thoughts and comments on this post. Please share them with me below.