Linux mainline fork with MSM8998 patches | https://mainline.space | Currently supported devices: OnePlus 5/5T, Xiaomi Mi 6, F(x)tec Pro¹ (2019 QX1000 model) & Sony Xperia XZ Premium (UNTESTED!)
Find a file
Andrii Nakryiko 477bb4c1ba Merge branch 'libbpf: Implement BTFGen'
Mauricio Vásquez <mauricio@kinvolk.io> says:

====================

CO-RE requires to have BTF information describing the kernel types in
order to perform the relocations. This is usually provided by the kernel
itself when it's configured with CONFIG_DEBUG_INFO_BTF. However, this
configuration is not enabled in all the distributions and it's not
available on kernels before 5.12.

It's possible to use CO-RE in kernels without CONFIG_DEBUG_INFO_BTF
support by providing the BTF information from an external source.
BTFHub[0] contains BTF files to each released kernel not supporting BTF,
for the most popular distributions.

Providing this BTF file for a given kernel has some challenges:
1. Each BTF file is a few MBs big, then it's not possible to ship the
eBPF program with all the BTF files needed to run in different kernels.
(The BTF files will be in the order of GBs if you want to support a high
number of kernels)
2. Downloading the BTF file for the current kernel at runtime delays the
start of the program and it's not always possible to reach an external
host to download such a file.

Providing the BTF file with the information about all the data types of
the kernel for running an eBPF program is an overkill in many of the
cases. Usually the eBPF programs access only some kernel fields.

This series implements BTFGen support in bpftool. This idea was
discussed during the "Towards truly portable eBPF"[1] presentation at
Linux Plumbers 2021.

There is a good example[2] on how to use BTFGen and BTFHub together
to generate multiple BTF files, to each existing/supported kernel,
tailored to one application. For example: a complex bpf object might
support nearly 400 kernels by having BTF files summing only 1.5 MB.

[0]: https://github.com/aquasecurity/btfhub/
[1]: https://www.youtube.com/watch?v=igJLKyP1lFk&t=2418s
[2]: https://github.com/aquasecurity/btfhub/tree/main/tools

Changelog:
v6 > v7:
- use array instead of hashmap to store ids
- use btf__add_{struct,union}() instead of memcpy()
- don't use fixed path for testing BTF file
- update example to use DECLARE_LIBBPF_OPTS()

v5 > v6:
- use BTF structure to store used member/types instead of hashmaps
- remove support for input/output folders
- remove bpf_core_{created,free}_cand_cache()
- reorganize commits to avoid having unused static functions
- remove usage of libbpf_get_error()
- fix some errno propagation issues
- do not record full types for type-based relocations
- add support for BTF_KIND_FUNC_PROTO
- implement tests based on core_reloc ones

v4 > v5:
- move some checks before invoking prog->obj->gen_loader
- use p_info() instead of printf()
- improve command output
- fix issue with record_relo_core()
- implement bash completion
- write man page
- implement some tests

v3 > v4:
- parse BTF and BTF.ext sections in bpftool and use
  bpf_core_calc_relo_insn() directly
- expose less internal details from libbpf to bpftool
- implement support for enum-based relocations
- split commits in a more granular way

v2 > v3:
- expose internal libbpf APIs to bpftool instead
- implement btfgen in bpftool
- drop btf__raw_data() from libbpf

v1 > v2:
- introduce bpf_object__prepare() and ‘record_core_relos’ to expose
  CO-RE relocations instead of bpf_object__reloc_info_gen()
- rename btf__save_to_file() to btf__raw_data()

v1: https://lore.kernel.org/bpf/20211027203727.208847-1-mauricio@kinvolk.io/
v2: https://lore.kernel.org/bpf/20211116164208.164245-1-mauricio@kinvolk.io/
v3: https://lore.kernel.org/bpf/20211217185654.311609-1-mauricio@kinvolk.io/
v4: https://lore.kernel.org/bpf/20220112142709.102423-1-mauricio@kinvolk.io/
v5: https://lore.kernel.org/bpf/20220128223312.1253169-1-mauricio@kinvolk.io/
v6: https://lore.kernel.org/bpf/20220209222646.348365-1-mauricio@kinvolk.io/

Mauricio Vásquez (6):
  libbpf: split bpf_core_apply_relo()
  libbpf: Expose bpf_core_{add,free}_cands() to bpftool
  bpftool: Add gen min_core_btf command
  bpftool: Implement "gen min_core_btf" logic
  bpftool: Implement btfgen_get_btf()
  selftests/bpf: Test "bpftool gen min_core_btf"
====================

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
2022-02-16 10:15:47 -08:00
arch Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2022-02-09 18:40:56 -08:00
block block: add bio_start_io_acct_time() to control start_time 2022-01-28 12:28:15 -07:00
certs certs: Fix build error when CONFIG_MODULE_SIG_KEY is empty 2022-01-23 00:08:44 +09:00
crypto lib/crypto: add prompts back to crypto libraries 2022-01-18 13:03:55 +01:00
Documentation Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2022-02-09 18:40:56 -08:00
drivers net: dsa: qca8k: fix noderef.cocci warnings 2022-02-10 10:56:00 -08:00
fs Notable bug fixes: 2022-02-02 10:14:31 -08:00
include net: make net->dev_unreg_count atomic 2022-02-10 15:30:26 +00:00
init Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2022-02-09 18:40:56 -08:00
ipc proc: remove PDE_DATA() completely 2022-01-22 08:33:37 +02:00
kernel libbpf: Split bpf_core_apply_relo() 2022-02-16 10:05:42 -08:00
lib Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2022-02-09 18:40:56 -08:00
LICENSES LICENSES/LGPL-2.1: Add LGPL-2.1-or-later as valid identifiers 2021-12-16 14:33:10 +01:00
mm Revert "mm/gup: small refactoring: simplify try_grab_page()" 2022-02-03 06:51:42 -08:00
net net/switchdev: use struct_size over open coded arithmetic 2022-02-10 15:37:47 +00:00
samples samples/bpf: Get rid of bpf_prog_load_xattr() use 2022-02-03 16:32:25 +01:00
scripts Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2022-02-09 18:40:56 -08:00
security Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2022-02-03 17:36:16 -08:00
sound proc: remove PDE_DATA() completely 2022-01-22 08:33:37 +02:00
tools selftests/bpf: Test "bpftool gen min_core_btf" 2022-02-16 10:14:34 -08:00
usr kbuild: remove include/linux/cyclades.h from header file check 2022-01-27 08:51:08 +01:00
virt Two larger x86 series: 2022-01-28 19:00:26 +02:00
.clang-format genirq/msi: Make interrupt allocation less convoluted 2021-12-16 22:22:20 +01:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.get_maintainer.ignore Opt out of scripts/get_maintainer.pl 2019-05-16 10:53:40 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for dts files 2019-12-04 19:44:11 -08:00
.gitignore .gitignore: ignore only top-level modules.builtin 2021-05-02 00:43:35 +09:00
.mailmap mailmap: update Christian Brauner's email address 2022-02-01 11:21:31 -08:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: Removing Ohad from remoteproc/rpmsg maintenance 2021-12-08 10:09:40 -07:00
Kbuild kbuild: rename hostprogs-y/always to hostprogs/always-y 2020-02-04 01:53:07 +09:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2022-02-09 18:40:56 -08:00
Makefile Linux 5.17-rc2 2022-01-30 15:37:07 +02:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.