2020年10月29日 星期四

yocto 概念


這篇文章紀錄我對於 yocto 的概念. 以下用的版本為 2.4 (rocko). 一開始介紹幾個用詞. 接著述說架構的運作流程. 最後再介紹 bitbake 指令用法.

名詞定義

[2]摘錄比較常看到的名詞定義
  • BitBake: The task executor and scheduler used by the OpenEmbedded build system to build images. For more information on BitBake, see the BitBake User Manual.

  • Task: A unit of execution for BitBake (e.g. do_compile, do_fetch, do_patch, and so forth).

  • Recipe: A set of instructions for building packages. A recipe describes where you get source code, which patches to apply, how to configure the source, how to compile it and so on. Recipes also describe dependencies for libraries or for other recipes. Recipes represent the logical unit of execution, the software to build, the images to build, and use the .bb file extension.
    P.S recipe 在 yocto 用 .bb 檔案表示. 而 .bbappend 用於修改 .bb 的設定

  • Metadata: The files that BitBake parses when building an image. In general, Metadata includes recipes, classes, and configuration files. In the context of the kernel ("kernel Metadata"), the term refers to the kernel config fragments and features contained in the yocto-kernel-cache Git repository.

我對於後三個的關係是. metadata 包含許多 recipes. recipe 則記載著哪些 tasks 可以使用. recipe 在根據規範或需求實作 tasks. 所以 bitbake 運行時候會在適當時機呼叫 recipe 所屬的 task 來執行其工作.
e.g. meta layer 目錄底下有許多的 recipe-bsp, recipe-core, etc., 每個 recipe-xxx 底下有許多 recipe 檔案. 這些檔案皆有實作所需的 tasks, 如 do_compile, do_fetch, 等等.

概述流程

下方以 bitbake 編譯流程和使用yocto文件[2]的架構流程圖互相搭配解說. 首先我們依照官網[1]所說的, 經由 oe-init-build-env script 配置好環境以及工作目錄 build. build 目錄為設定檔以及編譯所存放檔案的地方

[yijyun@localhost Github]$ git clone -b rocko git://git.yoctoproject.org/poky.git
[yijyun@localhost Github]$ cd poky/
[yijyun@localhost poky]$ . oe-init-build-env

### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
    core-image-minimal
    core-image-sato
    meta-toolchain
    meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86'

[yijyun@localhost build]$ ls
bitbake-cookerdaemon.log  cache  codes  conf  downloads  sstate-cache  tmp  workspace

在看一下官方文件[2]第三章所述說的架構流程, 如下所示.


圖片來自於[2]第三章

執行 bitbake core-image-minimal 來進行編譯動作, 產生的資訊如下.

[yijyun@localhost build]$ bitbake core-image-minimal
Parsing recipes: 100% |##########################################################################################################| Time: 0:00:41
Parsing of 820 .bb files complete (0 cached, 820 parsed). 1277 targets, 44 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.36.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "centos-7"
TARGET_SYS           = "i586-poky-linux"
MACHINE              = "qemux86"
DISTRO               = "poky"
DISTRO_VERSION       = "2.4.2"
TUNE_FEATURES        = "m32 i586"
TARGET_FPU           = ""
meta
meta-poky
meta-yocto-bsp       = "rocko:50189fdf620bc9ca42065998ce8c5a796ad8c331"

NOTE: Fetching uninative binary shim from http://downloads.yoctoproject.org/releases/uninative/1.7/x86_64-nativesdk-libc.tar.bz2;sha256sum=ed033c868b87852b07957a4400f3b744c00aef5d6470346ea1a59b6d3e03075e
Initialising tasks: 100% |#######################################################################################################| Time: 0:00:03
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
WARNING: libmpc-native-1.0.3-r0 do_fetch: Failed to fetch URL http://www.multiprecision.org/mpc/download/mpc-1.0.3.tar.gz, attempting MIRRORS if available
Currently  4 running tasks (99 of 2470)   3% |###                                                                                              |
0: binutils-cross-i586-2.29.1-r0 do_fetch (pid 2411)   6% |####                                                                        | 1.59M/s
1: gcc-source-7.3.0-7.3.0-r0 do_fetch (pid 4633)  17% |#############                                                                    | 746K/s
2: xz-native-5.2.3-r0 do_configure - 2s (pid 18637)
3: gmp-native-6.1.2-r0 do_configure - 2s (pid 18769)                   
or                                                    
0: binutils-cross-i586-2.29.1-r0 do_fetch (pid 2411)  25% |###################                                                          | 583K/s
1: linux-libc-headers-4.12-r0 do_fetch (pid 21075)  22% |#################                                                             | 1.02M/s
2: glibc-initial-2.26-r0 do_fetch (pid 21452)   3% |##                                                                                  | 390K/s
3: sqlite3-native-3_3.20.0-r0 do_compile - 1s (pid 2812)

接下來就描述這些步驟與上方架構流程圖的關係.

  1. 執行 source oe-init-build-env 來設置環境. 此 script 會產生一個 build 目錄. 底下的 conf 目錄存放 user config(local.conf) 與 layers (bblayers.conf). local.conf 可以設定編譯的target(kernel, machine)以及加入相關套件 image 等等之類的設定. bblayers.conf 則紀錄所要使用的 layers.

  2. 執行 bitbake core-image-minimal 也就是執行 core-image-minimal recipe. 此 recipe 擁有產生 image 相關的 tasks. 但是在執行之前, bitbake 根據 bblayer.conf. 把所有 layers 裡面的 recipes 都給讀進來做分析. 之後才正式執行 core-image-minimal. 這部分如上方圖裡面的 Parsing of 820 .bb files complete (0 cached, 820 parsed). 1277 targets, 44 skipped, 0 masked, 0 errors.

  3. bitbake 執行 target recipe(core-image-minimal) 時會尋找其依賴的 recipes, 並執行其所擁有的 tasks (do_fetch, do_compile, etc.). 這邊 recipe 所擁有的 tasks 可參閱[2]第九章. 基本上的流程都是先抓取 source codes, 接著 configure 和編譯. 把編譯完的檔案存放於 ${WORKDIR}. 然後把這些檔案封包成 package 的格式 (rpm, deb, etc.) 存放於 ${WORKDIR}/deploy-xxxx. 同樣也會存放於 ${DEPLOY_DIR}

  4. [yijyun@localhost build]$ ls  tmp/work/i586-poky-linux/acl/2.2.52-r0/
    0001-Added-configure-option-to-enable-disable-static-libr.patch  pkgdata
    acl-2.2.52                                                       pseudo
    acl-fix-the-order-of-expected-output-of-getfacl.patch            recipe-sysroot
    acl.spec                                                         recipe-sysroot-native
    configure.sstate                                                 relative-libdir.patch
    debugsources.list                                                run-ptest
    deploy-rpms                                                      sysroot-destdir
    image                                                            temp
    license-destdir                                                  test-fix-directory-permissions.patch
    Makefile-libacl-should-depend-on-include.patch                   test-fix-insufficient-quoting-of.patch
    package                                                          test-fixups-on-SELinux-machines-for-root-testcases.patch
    packages-split
    
    [yijyun@localhost build]$ ls  tmp/work/i586-poky-linux/acl/2.2.52-r0/deploy-rpms/i586/
    acl-2.2.52-r0.i586.rpm      acl-doc-2.2.52-r0.i586.rpm        acl-locale-fr-2.2.52-r0.i586.rpm  acl-locale-sv-2.2.52-r0.i586.rpm
    …
    
    [yijyun@localhost build]$ ls  tmp/deploy/rpm/i586/ | grep acl
    acl-2.2.52-r0.i586.rpm
    acl-dbg-2.2.52-r0.i586.rpm
    acl-dev-2.2.52-r0.i586.rpm
    …
    
  5. do_compile task 以及 do_package task 完畢之後, 接著就是執行 image 的部分, bitbake 會執行 do_image 相關的部分, 來產生所需要的 image 檔案. 相關檔案會存放於 ${DEPLOY_DIR_IMAGE}

  6. [yijyun@localhost build]$ ls tmp/deploy/images/qemux86/
    bzImage
    bzImage--4.12.19+git0+44a22d45cb_257f843ea3-r0-qemux86-20180219113211.bin
    bzImage-qemux86.bin
    core-image-minimal-qemux86-20180219113211.qemuboot.conf
    core-image-minimal-qemux86-20180219113211.rootfs.ext4
    core-image-minimal-qemux86-20180219113211.rootfs.manifest
    core-image-minimal-qemux86-20180219113211.rootfs.tar.bz2
    core-image-minimal-qemux86-20180219113211.testdata.json
    core-image-minimal-qemux86.ext4
    core-image-minimal-qemux86.manifest
    core-image-minimal-qemux86.qemuboot.conf
    core-image-minimal-qemux86.tar.bz2
    core-image-minimal-qemux86.testdata.json
    modules--4.12.19+git0+44a22d45cb_257f843ea3-r0-qemux86-20180219113211.tgz
    modules-qemux86.tgz
    

bitbake 常用指令

從[3]摘要一些小技巧, 用來快速查詢 recipe, 檔案位置, 依賴關係, 等等一些常用的方法.

bitbake [recipename/target recipe:do_task ...]

執行某個 recipe (or target) 所有的 tasks

[yijyun@localhost build]$ bitbake acl

bitbake [recipename/target recipe:do_task ...] -c [cmd]

執行 recipe 裡的某個 task. 可以先使用 listtasks 列出 recipe 可用的 tasks

# -c CMD, --cmd=CMD     Specify the task to execute. The exact options available depend on the metadata. Some examples might be 'compile' or 'populate_sysroot' or 'listtasks' may give a list of the tasks available.
[yijyun@localhost build]$ bitbake acl -c listtasks
Loading cache: 100% |###################################################################################| Time: 0:00:00
...
Initialising tasks: 100% |##################################################################################| Time: 0:00:01
NOTE: Executing RunQueue Tasks
do_build                       Default task for a recipe - depends on all other normal tasks required to 'build' a recipe
do_checkuri                    Validates the SRC_URI value
...
do_unpack                      Unpacks the source code into a working directory
NOTE: Tasks Summary: Attempted 1 tasks of which 0 didn't need to be rerun and all succeeded.

bitbake -s

根據設定檔, 列出所有 recipe 的版本資訊. 可以搭配 grep 來找是否存在某個 recipe 以及版本.

# -s, --show-versions   Show current and preferred versions of all recipes.
[yijyun@localhost build]$ bitbake -s | grep linux-yocto
linux-yocto                         :4.12.19+gitAUTOINC+44a22d45cb_257f843ea3-r0

bitbake -e [recipename/target recipe:do_task ...]

列出此 recipe 相關的環境變數資訊. 此方式可以找出 ${S}, ${WORKDIR}, 等等的資訊.

# -e, --environment     Show the global or per-recipe environment complete with information about where variables wereset/changed.
[yijyun@localhost build]$ bitbake -e acl | grep ^WORKDIR=
WORKDIR="/media/disk1-1/Github/poky/build/tmp/work/i586-poky-linux/acl/2.2.52-r0"

Reference

  1. Yocto Project Quick Start 2.4.1
  2. Yocto Project Reference Manual 2.4.1
  3. Yocto实用技巧

 

資料來源: http://yi-jyun.blogspot.com/2018/02/yocto-1.html

沒有留言: