include/nuttx: Add link-time iterable sections infrastructure

Documentation/components/index.rst

@@ -13,6 +13,7 @@ case, you can head to the :doc:`reference <../reference/index>`.
 
    binfmt.rst
    concurrency/index.rst
+   iterable_sections.rst
    drivers/index.rst
    nxflat.rst
    nxgraphics/index.rst

Documentation/components/iterable_sections.rst

@@ -0,0 +1,170 @@
+=================
+Iterable Sections
+=================
+
+Iterable sections provide **link-time registration** of ``struct``
+instances: an instance defined with :c:macro:`STRUCT_SECTION_ITERABLE` in
+any compilation unit is placed in a dedicated linker input section.  The
+linker collects all instances into a contiguous, name-sorted array
+delimited by ``_<type>_list_start``/``_<type>_list_end`` symbols, which
+the code can then iterate like a plain C array -- no runtime registration
+calls, no central list to maintain.
+
+This is the same mechanism used by the Zephyr RTOS ``STRUCT_SECTION_*``
+macros.  The first user of this infrastructure is the zbus message bus
+port (``apps/system/zbus``, from nuttx-apps).
+
+C API
+=====
+
+The macros are provided by ``include/nuttx/iterable_sections.h``:
+
+.. code-block:: c
+
+   #include <nuttx/iterable_sections.h>
+
+   struct my_entry
+   {
+     const char *name;
+     int value;
+   };
+
+   /* In any .c file (const places the instance in ROM): */
+
+   const STRUCT_SECTION_ITERABLE(my_entry, entry_foo) =
+   {
+     .name = "foo",
+     .value = 42,
+   };
+
+   /* In the file that iterates: declare the section boundaries once, at
+    * file scope, then loop with a caller-declared pointer.
+    */
+
+   STRUCT_SECTION_DECLARE(my_entry);
+
+   void print_entries(void)
+   {
+     FAR struct my_entry *entry;
+
+     STRUCT_SECTION_FOREACH(my_entry, entry)
+       {
+         printf("%s = %d\n", entry->name, entry->value);
+       }
+   }
+
+Available macros:
+
+* ``STRUCT_SECTION_ITERABLE(type, varname)`` -- define an instance inside
+  the iterable section ``._<type>.static.<varname>``.  The variable name
+  is part of the input section name, so the linker's ``SORT_BY_NAME()``
+  defines the iteration order (instances may encode ordering in their
+  names).
+* ``STRUCT_SECTION_DECLARE(type)`` -- declare the boundary symbols (file
+  scope), required before iterating.
+* ``STRUCT_SECTION_FOREACH(type, iterator)`` -- for-loop over all
+  instances; ``iterator`` is a pointer declared by the caller, as with
+  ``list_for_every_entry()``.
+* ``STRUCT_SECTION_GET(type, i, dst)`` -- random access by index.
+* ``STRUCT_SECTION_COUNT(type, dst)`` -- number of instances.
+* ``STRUCT_SECTION_START/END/START_EXTERN/END_EXTERN`` -- direct access
+  to the boundary symbols.
+
+Linker integration
+==================
+
+The collection step needs linker script support.  Two mechanisms are
+available; both rely on the fact that the linker scripts listed in
+``ARCHSCRIPT`` are preprocessed with CPP (arm, arm64, risc-v, xtensa,
+x86_64 and tricore), so ``#include`` and ``#ifdef CONFIG_*`` work inside
+them.
+
+Board script include (first-class mechanism)
+--------------------------------------------
+
+The board linker script includes the central fragments, which expand to
+nothing unless a subsystem using iterable sections is enabled:
+
+.. code-block:: text
+
+   .text :
+   {
+       ...
+       *(.gnu.linkonce.r.*)
+   #include <nuttx/linker/common-rom.ld>
+       _etext = ABSOLUTE(.);
+   } > flash
+
+   .data :
+   {
+       _sdata = ABSOLUTE(.);
+       ...
+   #include <nuttx/linker/common-ram.ld>
+       . = ALIGN(4);
+       _edata = ABSOLUTE(.);
+   } > sram AT > flash
+
+* ``common-rom.ld`` collects the read-only (``const``) iterable sections
+  and must be included inside the read-only output section (typically
+  ``.text``, before ``_etext``).
+* ``common-ram.ld`` collects mutable *initialized* iterable sections and
+  must be included inside ``.data`` (between ``_sdata`` and ``_edata``)
+  so the startup FLASH-to-RAM copy initializes the entries.
+* Subsystems add their sections to these central files, guarded by their
+  Kconfig option (see ``include/nuttx/linker/common-rom.ld`` for the zbus
+  example).
+
+Supplementary INSERT script (zero-touch mode)
+---------------------------------------------
+
+With ``CONFIG_ITERABLE_SECTIONS_LINKER_INSERT`` the central script
+``include/nuttx/linker/common-insert.ld`` is added to the ``ARCHSCRIPT``
+list by ``tools/Config.mk`` and supplements the board script through the
+GNU ld ``INSERT AFTER`` command, so **no board script modification is
+needed**.  Subsystems add their fragment (a ``SECTIONS { ... } INSERT
+AFTER .text`` block, see ``include/nuttx/linker/zbus.ld``) to that
+central file, guarded by their Kconfig option.
+
+This mode has constraints, discovered the hard way and worth knowing
+before choosing it:
+
+* GNU ld only (``INSERT`` is not supported by the macOS ld64).
+* The INSERT script must come *before* the board script on the linker
+  command line.  Adding it via ``ARCHSCRIPT`` from ``tools/Config.mk``
+  guarantees that, because ``Config.mk`` is included by the board
+  ``Make.defs`` before it appends its own script.  (The reversed order
+  fails with ``.text not found for insert``.)
+* GNU ld assigns an INSERTed output section to a ``MEMORY`` region by
+  *attribute matching in declaration order*, not by inheriting the anchor
+  section's region.  The ROM/flash region must therefore be the first
+  region compatible with read-only sections.  Boards declaring a generic
+  ``rwx`` region at a lower address first (e.g. an ITCM at ``0x0``) are
+  incompatible with this mode and must use the board script include.
+* Giving the inserted section an explicit address is **not** a fix: a
+  section with an explicit address does not consume the memory region,
+  so the next region-allocated section overlaps it.
+
+Alignment rules
+===============
+
+Instances are aligned to the natural alignment of their type
+(``STRUCT_SECTION_ITERABLE`` adds ``__aligned__(__alignof__(type))``), and
+``sizeof`` is always a multiple of ``alignof``, so the collected section
+can be indexed as a plain array with no padding between entries from
+different compilation units.  The fragments additionally align the list
+boundaries to 4 bytes.
+
+Adding a new iterable type
+==========================
+
+1. Define the instances with ``STRUCT_SECTION_ITERABLE(mytype, name)``.
+2. Add ``ITERABLE_SECTION(mytype)`` to
+   ``include/nuttx/linker/common-rom.ld`` (const) or ``common-ram.ld``
+   (mutable initialized), guarded by the subsystem Kconfig option.
+3. Iterate with ``STRUCT_SECTION_FOREACH(mytype, it)`` after
+   ``STRUCT_SECTION_DECLARE(mytype);`` at file scope.
+
+Caveat on generated linker scripts: the preprocessed ``.ld.tmp`` files
+only depend on the board script and ``.config``; after editing the
+central fragments during development, remove the ``.tmp`` files (or run
+``make clean``) to force regeneration.

Kconfig

@@ -2957,6 +2957,29 @@ config DEBUG_LINK_MAP
 		and debugging magic section games, and for seeing which
 		pieces of code get eliminated with DEBUG_OPT_UNUSED_SECTIONS.
 
+config ITERABLE_SECTIONS_LINKER_INSERT
+	bool "Collect iterable sections through a supplementary INSERT linker script"
+	default n
+	depends on ARCH_TOOLCHAIN_GNU
+	---help---
+		Zero-touch mode for link-time iterable sections
+		(include/nuttx/iterable_sections.h): instead of the board linker
+		script including <nuttx/linker/common-rom.ld>, the supplementary
+		script <nuttx/linker/common-insert.ld> is added to ARCHSCRIPT and
+		supplements the board script with GNU ld "INSERT AFTER .text";
+		subsystems add their INSERT fragments to that central file.
+
+		Leave disabled for boards whose linker script already includes
+		the common fragments.
+
+		Constraints: requires GNU ld, a board script with an output
+		section named ".text", and a MEMORY layout where the ROM/flash
+		region is the first region compatible with read-only sections
+		(GNU ld assigns INSERTed sections to a region by attribute
+		matching, in declaration order).  Boards declaring a generic
+		rwx region at a lower address first (e.g. ITCM at 0x0) must use
+		the common-rom.ld include instead.
+
 config CCACHE
 	bool "Use ccache"
 	default n

include/nuttx/iterable_sections.h

@@ -0,0 +1,121 @@
+/****************************************************************************
+ * include/nuttx/iterable_sections.h
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.  The
+ * ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
+ * License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ *
+ ****************************************************************************/
+
+/* Iterable sections: link-time registration of struct instances.
+ *
+ * A struct instance defined with STRUCT_SECTION_ITERABLE() in any
+ * compilation unit is placed in a dedicated input section named
+ * "._<struct_type>.static.<varname>".  The board linker script collects
+ * these input sections (sorted by name) into a contiguous array delimited
+ * by the _<struct_type>_list_start/_<struct_type>_list_end symbols by
+ * including <nuttx/linker/common-rom.ld> (const data, inside the .text or
+ * .rodata output section) and <nuttx/linker/common-ram.ld> (mutable
+ * initialized data, inside the .data output section, so that the startup
+ * FLASH-to-RAM copy initializes it).
+ *
+ * The collection is only available on architectures whose linker scripts
+ * are preprocessed with CPP (arm, arm64, risc-v, xtensa, x86_64, tricore)
+ * and on boards whose scripts include the common-*.ld fragments.
+ */
+
+#ifndef __INCLUDE_NUTTX_ITERABLE_SECTIONS_H
+#define __INCLUDE_NUTTX_ITERABLE_SECTIONS_H
+
+/****************************************************************************
+ * Included Files
+ ****************************************************************************/
+
+#include <nuttx/config.h>
+#include <nuttx/compiler.h>
+
+/****************************************************************************
+ * Pre-processor Definitions
+ ****************************************************************************/
+
+/* Define a struct instance inside an iterable section.  A "const"
+ * qualifier may be prepended at the point of use to place the instance in
+ * ROM.  Each instance is aligned to the natural alignment of its type so
+ * that the collected section can be indexed as a plain C array.  The
+ * variable name is part of the input section name so that the linker's
+ * SORT_BY_NAME() defines the iteration order (instances may encode
+ * ordering in their names).
+ */
+
+#define STRUCT_SECTION_ITERABLE(struct_type, varname) \
+  struct struct_type varname \
+  used_data \
+  aligned_data(__alignof__(struct struct_type)) \
+  locate_data("._" #struct_type ".static." #varname)
+
+/* Start/end symbols provided by the linker script fragments */
+
+#define STRUCT_SECTION_START(struct_type) _##struct_type##_list_start
+#define STRUCT_SECTION_END(struct_type)   _##struct_type##_list_end
+
+#define STRUCT_SECTION_START_EXTERN(struct_type) \
+  extern struct struct_type STRUCT_SECTION_START(struct_type)[]
+#define STRUCT_SECTION_END_EXTERN(struct_type) \
+  extern struct struct_type STRUCT_SECTION_END(struct_type)[]
+
+/* Declare both boundary symbols of an iterable section.  Place it at file
+ * scope (followed by a semicolon) in every file that iterates with
+ * STRUCT_SECTION_FOREACH.
+ */
+
+#define STRUCT_SECTION_DECLARE(struct_type) \
+  STRUCT_SECTION_START_EXTERN(struct_type); \
+  STRUCT_SECTION_END_EXTERN(struct_type)
+
+/* Iterate over every instance of an iterable section.  "iterator" is a
+ * pointer variable (FAR struct struct_type *) declared by the caller, as
+ * with list_for_every_entry(); the boundary symbols must be in scope
+ * (STRUCT_SECTION_DECLARE).
+ */
+
+#define STRUCT_SECTION_FOREACH(struct_type, iterator) \
+  for ((iterator) = STRUCT_SECTION_START(struct_type); \
+       (iterator) < STRUCT_SECTION_END(struct_type); \
+       (iterator)++)
+
+/* Get the i-th element of an iterable section (no bounds checking) */
+
+#define STRUCT_SECTION_GET(struct_type, i, dst) \
+  do \
+    { \
+      STRUCT_SECTION_START_EXTERN(struct_type); \
+      *(dst) = &STRUCT_SECTION_START(struct_type)[i]; \
+    } \
+  while (0)
+
+/* Number of elements in an iterable section */
+
+#define STRUCT_SECTION_COUNT(struct_type, dst) \
+  do \
+    { \
+      STRUCT_SECTION_START_EXTERN(struct_type); \
+      STRUCT_SECTION_END_EXTERN(struct_type); \
+      *(dst) = STRUCT_SECTION_END(struct_type) - \
+               STRUCT_SECTION_START(struct_type); \
+    } \
+  while (0)
+
+#endif /* __INCLUDE_NUTTX_ITERABLE_SECTIONS_H */

include/nuttx/linker/common-insert.ld

@@ -0,0 +1,34 @@
+/****************************************************************************
+ * include/nuttx/linker/common-insert.ld
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.  The
+ * ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
+ * License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ *
+ ****************************************************************************/
+
+/* Supplementary linker script for the iterable sections "zero-touch" mode
+ * (CONFIG_ITERABLE_SECTIONS_LINKER_INSERT): added to ARCHSCRIPT by
+ * tools/Config.mk, it supplements -- does not replace -- the board linker
+ * script through the GNU ld INSERT command, so boards need no edit.
+ *
+ * Subsystem fragments are included below, each guarded by its Kconfig
+ * option; every fragment provides its own SECTIONS { ... } INSERT AFTER
+ * block.  See Documentation/components/iterable_sections.rst for the
+ * constraints of this mode (GNU ld, command-line ordering, MEMORY layout).
+ */
+
+#include <nuttx/config.h>