Brief Background

The Yocto Project (i.e. bitbake) enables custom embedded Linux builds using a flexible "Layer Model," where layers are collections of metadata, recipes, and configurations grouped by purpose (e.g., hardware support, distribution policy, or software)[1][2]. This modularity supports collaboration, customization, and re-usability.

Organizing Layers: Git Repositories and Separation

Layer Types and Functional Separation

• Base Layers:
  Layers provided by OpenEmbedded or The Yocto Project, e.g. openembedded-core (i.e. driectory poky/meta) & meta-oe,
• BSP (Board Support Package) Layers:
  Contain hardware-specific support (e.g., machine configurations, kernel recipes).
• Distro Layers:
  Define distribution policies, global configurations, and may add or override recipes for a specific Linux distribution.
• Software Layers:
  Provide additional software recipes, middleware, or application stacks, generally hardware-agnostic.
• Miscellaneous Layers:
  Layers that doesn't fit the descriptions above.

This separation of layers isn't strict and bitbake will allow allow will allow any mix, but being strict with this ensures re-usability, and mixing distro and bsp layers is actually considered to be an error by the yocto-check-layer script/tool, that can be used to check your layers for Yocto Project compatibility.

Layer Curiosity/specifics

• Container Layers
  Describe this somehow
• Bootstrap Layers
  Also describe this

Directory and Repository Structure

• According to Yocto Project Best Known Methods you should "Group Your Layers Locally", which in this context means "Clone your repository alongside other cloned meta directories", see 3.2 Following Best Practices When Creating Layers, in the Yocto Documentation. Elsewhere in the documentation, it is also stated that you should never make changes in the poky directory, and this is why this line is present in poky's .gitignore file.
• Each layer should be a separate Git repository, enabling independent versioning, easier sharing, and clear ownership.

- Default Tool Behavior: `bitbake-layers layerindex-fetch` and similar tools expect layers to be cloned or placed in directory `poky`. This is also reflected in the `poky/.gitignore` file, which ignores `meta-*` directories by default, preventing accidental inclusion of external layers in your version control.
- Consistency: Placing all layers at the top level ensures compatibility with official documentation and community practices, making it easier to collaborate and share projects.

General Tips

• Keep layers focused and minimal; avoid mixing hardware, distro, and software logic in a single layer.
• Use the `yocto-check-layer` script to verify compatibility and structure before publishing or sharing your layer.

Maintaining Maintainability

• Layer Dependencies:
  Use `LAYERDEPENDS` and `BBFILE_PRIORITY` in `layer.conf` to manage dependencies and priorities.
• Automation:
  Make use of tools like `bitbake-layers` for layer inspection and management.
• Regular Testing:
  Test new layers before integrating them into your build environment.

Leveraging the OpenEmbedded Layer Index

What is the Layer Index?

The OpenEmbedded Layer Index catalogs hundreds of layers, categorized by type (BSP, distro, software, etc.), with metadata, Git URLs, and descriptions[6]. It is the authoritative source for discovering, comparing, and fetching layers.

bitbake-layers layerindex-fetch Usage

• `bitbake-layers layerindex-fetch` is a tool that automates fetching layers by querying the Layer Index, resolving dependencies, and cloning the required repositories.
• This streamlines setup, ensures you get official sources, and reduces manual errors.

REST API

• The Layer Index provides a REST API for programmatic access to layer metadata, enabling automation in CI/CD pipelines, custom tooling, or integration with company portals.
• Typical use cases: searching for layers, retrieving layer dependencies, tracking updates.

Setting Up a Company Layer Index: Benefits

Why Maintain Your Own Layer Index?

• Centralized Metadata:
  Catalog all company-specific layers, including internal BSPs, distros, and software stacks.
• Access Control:
  Restrict visibility and access to proprietary layers.
• Consistency:
  Enforce naming conventions, metadata quality, and dependency tracking.
• Automation:
  Integrate with internal tools for build automation, layer validation, or release management.
• Upstream Sync:
  Optionally mirror public layers, making it easy to mix internal and external sources.

Best Practices

• Regularly update your index with new or modified layers.
• Use the REST API to integrate with internal dashboards or CI/CD systems.
• Document layer submission and review processes to ensure high quality and maintainability.

Summary

• Separate BSP, Distro, and Software layers into distinct, version-controlled Git repositories.
• Use the OpenEmbedded Layer Index and `bitbake-layers layerindex-fetch` for efficient layer discovery and management.
• Consider running a company-specific Layer Index for better control, automation, and collaboration.

This approach ensures scalable, maintainable, and robust Yocto-based development workflows:
[1] https://docs.yoctoproject.org/2.5/overview-manual/overview-manual.html
[2] https://www.openembedded.org/wiki/Layers_FAQ
[3] https://docs.yoctoproject.org/1.5/dev-manual/dev-manual.html
[4] https://docs.yoctoproject.org/dev-manual/layers.html
[5] https://layers.openembedded.org
[6] https://docs.yoctoproject.org/toaster-manual/reference.html
[7] https://github.com/yoctoproject/layerindex-web
[8] https://docs.yoctoproject.org/3.3.2/what-i-wish-id-known.html
[9] https://docs.yoctoproject.org/4.0.8/dev-manual/common-tasks.html
[11] https://www.embeddedrelated.com/showarticle/1693.php
[12] https://docs.yoctoproject.org/overview-manual/concepts.html

Mix of good and bad stuff, i.e. skip from repo and kas
[13] https://bootlin.com/pub/conferences/2020/elce/belloni-yocto-best-practices/belloni-yocto-best-practices.pdf
[14] https://pretalx.com/yocto-project-summit-2020/talk/DSGCRC/

Bad recommendations that IMO only add confusions:
[15] https://siliconbladeconsultants.com/2024/05/16/structure-of-a-yocto-project/
[16]  https://www.reddit.com/r/embedded/comments/tryzdf/yocto_layer_handling_via_git/

1. Standard Layer Placement

For many official examples, tutorials, and `layerindex-fetch` workflows,
layers are indeed placed inside the `poky` directory.**
This structure looks like:
poky/
├── meta-my-bsp/
├── meta-my-distro/
├── meta-my-software/
├── meta-openembedded/
└── meta-other-third-party/

Why this structure?

• Simple and Standard:
  Easy to set up and maintain, especially for new users and small projects.
• Tool Compatibility:
  Tools like `layerindex-fetch` and yocto-check-layer expect to find layers in the `poky` directory.
• Documentation Alignment:
  Official Yocto Project document Best Known Methods and examples use this layout.

3. What Does the Official BKM Say?

• The Yocto Project documentation is not always 100% prescriptive about
  layer placement.
• For simplicity and tool compatibility, the BKM often suggest placing layers inside
  `poky/`.

4. Best Practice Summary

For most users and standard setups:
Place layers **inside the `poky` directory.
poky/
├── meta-my-bsp/
├── meta-my-distro/
├── meta-my-software/
├── meta-openembedded/
└── meta-other-third-party/

5. Layer Management Tips

• Use `bitbake-layers` to add, remove, and inspect layers.
• Keep layer dependencies clear** using `LAYERDEPENDS` in `layer.conf`.
• Document your structure** so team members understand where to find each
  layer.
• Test new layers** before integrating them into your build.
• Use bootstrap layers to manage your projects/products

6. Conclusion

• Official BKM and many examples show layers inside `poky/`.
• Your structure is correct for standard, tool-compatible setups.
• Advanced users may place layers outside `poky/` for larger projects, but
  this is not the default or simplest path.

Final Structure Example (Official BKM-style)

poky/
├── meta-my-bsp/
├── meta-my-distro/
├── meta-my-software/
├── meta-openembedded/
└── meta-other-third-party/

This is the recommended structure for most Yocto Project users following the
Best Known Methods.

The Official Best Practice: Layers Inside the `poky` Directory

The Yocto Project Best Known Methods (BKM) and official documentation are clear:
All meta-layers — whether BSP, distro, software, or third-party—should reside directly inside the `poky` directory.
This is not just a convention but a practical requirement for many Yocto tools and scripts, including `bitbake-layers` and the `yocto-check-layer` compatibility script (which, until recently, required this structure to function properly).

Example structure:

poky/
├── meta-my-bsp/
├── meta-my-distro/
├── meta-my-software/
├── meta-openembedded/
└── meta-other-third-party/

• Layer creation:
  The recommended way to create layers is with the `bitbake-layers create-layer` command, run from inside `poky/`. This ensures the layer lands in the correct location and is properly initialized.
• Layer management:
  Use `bitbake-layers add-layer` and related commands to manage which layers are active in your build.

Common Misconceptions: Alternative (and Faulty) Structure

A widespread mistake—especially in larger or long-running projects—is to invent alternative directory layouts, such as grouping layers under `layers/project/` or `layers/third-party/`, or placing them outside the `poky` directory entirely.
This is not recommended by Yocto documentation BKM.

Why did this happen? Probably a combination of not reading the official Yocto Documentation, and that these BKM's have evolved over time and therefore a number of improvised own structures appeared. Over time, these alternatives spread, leading to confusion and compatibility problems.

•  Many Yocto tools and scripts assume layers are in `poky/` and may not work correctly otherwise.
•  The `poky/.gitignore` is set up to ignore `meta-*` directories in `poky/`, not elsewhere.
•  The official documentation and community support expect the standard layout, making troubleshooting harder for non-standard setups.

Managing Multiple Projects and Builds: Build Directories

The "correct"" way to handle multiple projects or products is not to scatter layers, but to use separate build directories.

• Each project or product gets its own build directory e.g:
  `build-project_one/`
  `build-project_two/`),
  each with its own `conf/local.conf` and `conf/bblayers.conf`.
• All builds share the same set of layers in:
  `poky/`**, but each build directory can enable or disable layers as needed by editing its own `bblayers.conf`.

Example:

poky/
├── meta-my-bsp/
├── meta-my-distro/
├── meta-my-software/
├── meta-openembedded/
build-project_one/
└── conf/
├── local.conf
└── bblayers.conf
build-project_two/
└── conf/
├── local.conf
└── bblayers.conf

This allows for clean separation of build configurations without duplicating or scattering layers.

Layer Management with BitBake and Bootstrap Layers

bitbake-layers utility:
- Use for creating (`create-layer`), adding (`add-layer`), and managing layers[1][2][4].
- Also supports advanced commands like `layerindex-fetch` for pulling layers from the OpenEmbedded Layer Index.

• Bootstrap layers:
  A **bootstrap layer** is a minimal layer that helps manage project/product/release configurations, often containing only the essential settings and dependencies to “bootstrap” a build.
• Use `bitbake-layers create-layer meta-bootstrap` to set up such a layer in `poky/`.
• Enable the bootstrap layer first in your `bblayers.conf` so it can set up the environment for other layers.
• This is especially useful when managing multiple products or releases, as it allows you to encapsulate project-specific logic cleanly and portable.
• Use bitbake--alyers save-build-conf, to create local.conf.template & bblayers.conf.template files in your bootstrap layer, to be used with TEMPLATECONF
  when sourcing the environment
  TEMPLATECONF=path/to/bootstrap-layer source oe-init-build-env

In summary

For maintainable, compatible Yocto projects, always place your layers directly inside `poky/`, use separate build directories for each project/product, and leverage bootstrap layers for scalable configuration management. Avoid inventing alternative layer layouts—these are a main source of confusion and break tool compatibility.

[1] https://docs.yoctoproject.org/brief-yoctoprojectqs/index.html
[2] https://kickstartembedded.com/2022/01/19/yocto-part-5-creating-adding-a-new-layer-to-your-image/
[3] https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/57836605/Creating+a+Custom+Yocto+Layer
[4] https://github.com/maxpeng/openstlinux_add_new_recipe/blob/master/add_the_layer_by_bitbake-layers.md
[5] https://docs.yoctoproject.org/dev-manual/layers.html
[6] https://docs.yoctoproject.org/2.1/ref-manual/ref-manual.html
[7] https://docs.yoctoproject.org/1.5/dev-manual/dev-manual.html
[8] https://www.reddit.com/r/embedded/comments/1dprl19/those_who_work_with_yocto_how_do_yall_manage_it/
[9] https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/How-to-add-a-new-layer-and-a-new-recipe-in-Yocto/ta-p/1102230
[10] https://www.embeddedrelated.com/showarticle/1693.php
[11] https://docs.yoctoproject.org/4.0.4/dev-manual/common-tasks.html
[12] https://bootlin.com/~mike/yocto/doc/overview-manual/yp-intro.html
[13] https://www.yoctoproject.org/development/technical-overview/
[14] https://github.com/jynik/ready-set-yocto
[15] https://yannik520.github.io/create_a_new_yocto_layer.html
[16] https://stackoverflow.com/questions/78808744/how-to-use-dynamic-layers-in-yocto
[17] https://www.youtube.com/watch?v=yuE7my3KOpo
[18] https://docs.yoctoproject.org/overview-manual/concepts.html
[19] https://community.nxp.com/pwmxy87654/attachments/pwmxy87654/connects/289/1/AMF-AUT-T2310.pdf