This blog post demonstrates how to set up a Zephyr project in a reproducible
manner. Additionally, it provides some Makefile tricks and best practices for
using this powerful tool effectively.
While you can set up a Zephyr project manually, following the Getting Started
Guide, a reproducible and automatic approach has several advantages. Firstly,
any changes made to the project will be automatically documented in GIT.
Furthermore, it is easier to move the project onto CI/CD servers or into Docker
containers.
(Ab)use of Makefiles
The entire setup is primarily managed by a Makefile. Despite the fact that
Zephyr utilizes CMake and Ninja, Makefiles offer a more convenient way to
consolidate numerous shell commands into a single location. You can consider
this Makefile as a repository of knowledge or as a mechanism for ensuring
replicability.
The full Makefile is accessible as
https://github.com/holgerschurig/zephyr-multi-board/blob/main/Makefile.zephyr_init
Basic project setup
Make sure you have all dependencies installed
Execution: either “make init” or, as a single step, “make debs”.
How?
| |
In this section, we employ a trick using the Makefile to detect the user ID of
the current user in line 1. Line 4 is used to verify if the
Makefile is running as a user or root. If it’s running as root, we can utilize
“apt” in line 5 to install all necessary dependencies.
If we’re non-root, we use “sudo” in line 43 to become root and execute the
“debs” Makefile target again. The “--no-print-directory” command-line argument
is employed to remove visual clutter from the output.
Lastly, as a normal user, we create the directory “.west” if it doesn’t exist
("-p") and place a stamp file inside it. The “make init” command checks the
existence of the stamp, preventing unnecessary re-execution of this part if it
already exists. In contrast, “make debs” does not check for the stamp and
always runs “apt”. This can be used if you want to install additional Debian
packages in an existing project setup.
Setting up a python virtual environment
Zephyr requires a tool named “west” that is written in Python and is installed
using “pip3”. Along with several Python modules. To prevent these modules from
conflicting with those installed by Debian (or Ubuntu), we need to create a
virtual environment.
Execution: either “make init” or, as a single step, “make debs”.
How?
| |
In line 5, we verify if the environment already exists. While Make’s
dependency checking can be used for this purpose, it would check not only for
file existence, but also for the timestamp. In this case, this is undesirable.
If the environment does not exist, we use the Python “venv” module in line
6 to create one. While we could source “.venv/bin/activate” to activate
this within Make, unfortunately, it has to be done outside of Make. Instead, we
ask to source “.env” so that we can also set up the required Zephyr
environment variables.
Pro tip: On my development PCs, I have a shell function “pro” that
automatically changes into a project directory and sources “.env” if it
exists. It looks like this:
pro ()
{
cd ~/src/$1 2> /dev/null || cd ~/d/$1 2> /dev/null || cd /usr/src/$1;
test -f .env && . .env
}
So now I can do “pro cool-zephyr-project” and my environment is automatically
setup.
(This shell function assumes that you have your projects in your home directory
below the “d” (like development) or “src” directories. Adjust as needed.)
Install the “west” tool
Now that we have a virtual environent, we can install the “west” tool.
Execution: either “make init” or, as a single step, “make west”.
How?
.PHONY:: west
init:: .west/config
west .west/config:
@type west >/dev/null || pip3 install west pyelftools
mkdir -p .west
/bin/echo -e "[manifest]\npath = zephyr\nfile = west.yml\n[zephyr]\nbase = zephyr" >.west/config
Actually this does 3 steps:
- install west
- install pyelftools (needed on Debian Bookworm, as the distro provided ones are too old)
- configure Zephyr via “
.west/config”
Install Zephyr
Now we require the source of Zephyr. On some projects, you may want to use the
current development version, while on others, you may wish to pin yourself to a
specific version. Additionally, you might have local patches for Zephyr that you
don’t want to publish upstream and that you want to apply automatically. This
step accomplishes all of this!
Execution: either “make init” or, as a single step, “make zephyr”.
How?
| |
The first step is a typical “git clone”. If you don’t care about Zephyr’s
commit history (e.g., you don’t want to run things like “git log” or “git blame”), you can also add “--depth 1”. This reduces the size of the cloned
“zephyr/” directory.
Specific version: you can uncommend and modify ZEPHYR_VERSION in line 1 to your liking.
This will pin Zephyr to the specified version. This is done by creating a branch “my”
in line 7.
BTW, the value of ZEPHYR_VERSION is the output of “git describe --tags”.
Background: when should you start to lock Zephyr? This depends on your
circumstances. When a project is still in EVT phase, I tend to follow Zephyr
closely, e.g. use development version so it. “ZEPYHR_VERSION” would be
uncommented then. But then the projects enters DVT phase, or even MP phase, I’ll
certainly lock Zephyr to a well-known version.
Local patches: in one of my projects, I have patches that will probably never
be accepted by upstream Zephyr. I could put them directly into Zephyr, in my own
branch … but I prefer to have them in my own GIT project. So I use the
“quilt” tool to manage a stack of patches.
The existence of quilt patches is checked in line 10 and, if they
exist, line 12 rolls them in.
Final note: It’s worth mentioning that due to version pinning and local
patches, we intentionally don’t use “west init” in this step.
Install needed Zephyr modules, e.g. HALs from the µC vendor
Some (actually almost all) of the SOCs that Zephyr supports need HALs (hardware
abstraction layers) provided by the chip vendor. If they don’t exist, we cannot
compile at all. So let’s install them!
Execution: either “make init” or, as a single step, “make modules”.
How?
.PHONY:: modules
init:: modules/hal/stm32/.git/HEAD
.PHONY:: module_stm32
update modules module_stm32 modules/hal/stm32/.git/HEAD:: .west/config
mkdir -p modules
west update hal_stm32
touch --no-create modules/hal/stm32/.git/HEAD
init:: modules/hal/st/.git/HEAD
.PHONY:: module_st
update modules module_st modules/hal/st/.git/HEAD:: .west/config
mkdir -p modules
west update hal_st
touch --no-create modules/hal/st/.git/HEAD
init:: modules/hal/cmsis/.git/HEAD
.PHONY:: module_cmsis
update modules module_cmsis modules/hal/cmsis/.git/HEAD:: .west/config
mkdir -p modules
west update cmsis
touch --no-create modules/hal/cmsis/.git/HEAD
As usual, I made the Makefile so that “make init” only pulls in the modules
once. However “make modules” will always pull them in, should the vendor have
changed them.
Theoretically one could pin the modules also to specific version, like in the
step above. I however noticed that they are quite stable and this was never
needed. And also I need to have something to assign to you as homework, didn’t I
????
Getting help
If you look at the actual Makefile, you’ll notice that I ommited a whole lot of lines like
help::
@echo " modules install Zeyphr modules (e.g. ST and STM32 HAL, CMSIS ...)"
from above. They aren’t strictly necessary, but nice. They allow you to run “make help” and
see all the common makefile targets meant for users. Like so:
(.venv) holger@holger:~/src/multi-board-zephyr$ make -f Makefile.zephyr_init help
init do all of these steps:
debs only install debian packages
venv create and check Python3 virtual environment
west install and configure the 'west' tool
zephyr clone Zephyr
modules install Zeyphr modules (e.g. ST and STM32 HAL, CMSIS ...)
module_stm32 update only STM32 HAL
module_st update only ST HAL
module_cmsis update only CMSIS
All of the above
The individual targets like “make venv” or “make debs” are mostly only for
debugging. Once you know they are working, simply run: “make init”.
Using this makefile in your project
You can simply add your own clauses at the end of this Makefile … your you can include it from
a main Makefile. This is demonstrated in the Github project https://github.com/holgerschurig/zephyr-multi-board/:
Main “Makefile”
PWD := $(shell pwd)
UID := $(shell id -u)
.PHONY:: all
all::
# Include common boilerplate Makefile to get Zephyr up on running
include Makefile.zephyr_init
# ... many more lines ...
First at the top we set two environment variables that we often use, PWD
(working directory) and UID (user id). You can then later just use them via
“$(PWD)” — note that Make want’s round brances here, not curly braces like
Bash.
Then I set a default target, to be executed if you just run “make” without specifying
a target by yourself.
The double colon here needs to be used for all targets that are defined more
than once in a Makefile. As you see, here the target is empty. It’s fleshed out
in much more complexity below, but this is beyond this blog post.
Also note the “.PHONY:: all” line. It helps Make to understand that “make”
or “make all” isn’t supposed to actually create file called “all”. This
helps it’s dependency resolvement engine, and is good style. My makefile uses
“.PHONY::” liberally, for each pseudo-target (shell script snippet) basically.
Finally, we use Make’s “include” clause to include our boilerplate Makefile.
You could also run the Boilerplate makefile itself, with “make -f Makefile.zephyr_init”, e.g. for debugging purposes. But oh … now PWD and UID
aren’t set. So at the top of this makefile I set these variables if they don’t exist:
ifeq ($(PWD),"")
PWD := $(shell pwd)
endif
ifeq ($(UID),"")
UID := $(shell id -u)
endif