Layer basics 1

Best Practices for Bitbake Layers in Yocto

Brief Background

The Yocto Project (i.e. bitbake) enables custom embedded Linux builds using a flexible ”Layer Model,” where layers are collections of metadata, recipes, and configurations grouped by purpose (e.g., hardware support, distribution policy, or software)[1][2]. This modularity supports collaboration, customization, and re-usability.

Organizing Layers: Git Repositories and Separation

Layer Types and Functional Separation

  • Base Layers:
    Layers provided by OpenEmbedded or The Yocto Project, e.g. openembedded-core (i.e. driectory poky/meta) & meta-oe,
  • BSP (Board Support Package) Layers:
    Contain hardware-specific support (e.g., machine configurations, kernel recipes).
  • Distro Layers:
    Define distribution policies, global configurations, and may add or override recipes for a specific Linux distribution.
  • Software Layers:
    Provide additional software recipes, middleware, or application stacks, generally hardware-agnostic.
  • Miscellaneous Layers:
    Layers that doesn’t fit the descriptions above.

This separation of layers isn’t strict and bitbake will allow allow will allow any mix, but being strict with this ensures re-usability, and mixing distro and bsp layers is actually considered to be an error by the yocto-check-layer script/tool, that can be used to check your layers for Yocto Project compatibility.

Layer Curiosity/specifics

  • Container Layers
    Describe this somehow
  • Bootstrap Layers
    Also describe this

Directory and Repository Structure

  • According to Yocto Project Best Known Methods you should ”Group Your Layers Locally”, which in this context means ”Clone your repository alongside other cloned meta directories”, see 3.2 Following Best Practices When Creating Layers, in the Yocto Documentation. Elsewhere in the documentation, it is also stated that you should never make changes in the poky directory, and this is why this line is present in poky’s .gitignore file.
  • Each layer should be a separate Git repository, enabling independent versioning, easier sharing, and clear ownership.

– Default Tool Behavior: `bitbake-layers layerindex-fetch` and similar tools expect layers to be cloned or placed in directory `poky`. This is also reflected in the `poky/.gitignore` file, which ignores `meta-*` directories by default, preventing accidental inclusion of external layers in your version control.
– Consistency: Placing all layers at the top level ensures compatibility with official documentation and community practices, making it easier to collaborate and share projects.

General Tips

  • Keep layers focused and minimal; avoid mixing hardware, distro, and software logic in a single layer.
  • Use the `yocto-check-layer` script to verify compatibility and structure before publishing or sharing your layer.

Maintaining Maintainability

  • Layer Dependencies:
    Use `LAYERDEPENDS` and `BBFILE_PRIORITY` in `layer.conf` to manage dependencies and priorities.
  • Automation:
    Make use of tools like `bitbake-layers` for layer inspection and management.
  • Regular Testing:
    Test new layers before integrating them into your build environment.

Leveraging the OpenEmbedded Layer Index

What is the Layer Index?

The OpenEmbedded Layer Index catalogs hundreds of layers, categorized by type (BSP, distro, software, etc.), with metadata, Git URLs, and descriptions[6]. It is the authoritative source for discovering, comparing, and fetching layers.

bitbake-layers layerindex-fetch Usage

  • `bitbake-layers layerindex-fetch` is a tool that automates fetching layers by querying the Layer Index, resolving dependencies, and cloning the required repositories.
  • This streamlines setup, ensures you get official sources, and reduces manual errors.

REST API

  • The Layer Index provides a REST API for programmatic access to layer metadata, enabling automation in CI/CD pipelines, custom tooling, or integration with company portals.
  • Typical use cases: searching for layers, retrieving layer dependencies, tracking updates.

Setting Up a Company Layer Index: Benefits

Why Maintain Your Own Layer Index?

  • Centralized Metadata:
    Catalog all company-specific layers, including internal BSPs, distros, and software stacks.
  • Access Control:
    Restrict visibility and access to proprietary layers.
  • Consistency:
    Enforce naming conventions, metadata quality, and dependency tracking.
  • Automation:
    Integrate with internal tools for build automation, layer validation, or release management.
  • Upstream Sync:
    Optionally mirror public layers, making it easy to mix internal and external sources.

Best Practices

  • Regularly update your index with new or modified layers.
  • Use the REST API to integrate with internal dashboards or CI/CD systems.
  • Document layer submission and review processes to ensure high quality and maintainability.

Summary

  • Separate BSP, Distro, and Software layers into distinct, version-controlled Git repositories.
  • Use the OpenEmbedded Layer Index and `bitbake-layers layerindex-fetch` for efficient layer discovery and management.
  • Consider running a company-specific Layer Index for better control, automation, and collaboration.

This approach ensures scalable, maintainable, and robust Yocto-based development workflows:
[1] https://docs.yoctoproject.org/2.5/overview-manual/overview-manual.html
[2] https://www.openembedded.org/wiki/Layers_FAQ
[3] https://docs.yoctoproject.org/1.5/dev-manual/dev-manual.html
[4] https://docs.yoctoproject.org/dev-manual/layers.html
[5] https://layers.openembedded.org
[6] https://docs.yoctoproject.org/toaster-manual/reference.html
[7] https://github.com/yoctoproject/layerindex-web
[8] https://docs.yoctoproject.org/3.3.2/what-i-wish-id-known.html
[9] https://docs.yoctoproject.org/4.0.8/dev-manual/common-tasks.html
[11] https://www.embeddedrelated.com/showarticle/1693.php
[12] https://docs.yoctoproject.org/overview-manual/concepts.html

Mix of good and bad stuff, i.e. skip from repo and kas
[13] https://bootlin.com/pub/conferences/2020/elce/belloni-yocto-best-practices/belloni-yocto-best-practices.pdf
[14] https://pretalx.com/yocto-project-summit-2020/talk/DSGCRC/

Bad recommendations that IMO only add confusions:
[15] https://siliconbladeconsultants.com/2024/05/16/structure-of-a-yocto-project/
[16]  https://www.reddit.com/r/embedded/comments/tryzdf/yocto_layer_handling_via_git/