head 1.24; access; symbols netbsd-10-0-RELEASE:1.24 netbsd-10-0-RC6:1.24 netbsd-10-0-RC5:1.24 netbsd-10-0-RC4:1.24 netbsd-10-0-RC3:1.24 netbsd-10-0-RC2:1.24 netbsd-10-0-RC1:1.24 netbsd-10:1.24.0.2 netbsd-10-base:1.24 netbsd-9-3-RELEASE:1.19 cjep_sun2x-base1:1.23 cjep_sun2x:1.23.0.2 cjep_sun2x-base:1.23 cjep_staticlib_x-base1:1.23 netbsd-9-2-RELEASE:1.19 cjep_staticlib_x:1.22.0.2 cjep_staticlib_x-base:1.22 netbsd-9-1-RELEASE:1.19 phil-wifi-20200421:1.21 phil-wifi-20200411:1.21 is-mlppp:1.21.0.2 is-mlppp-base:1.21 phil-wifi-20200406:1.21 netbsd-8-2-RELEASE:1.11 netbsd-9-0-RELEASE:1.19 netbsd-9-0-RC2:1.19 netbsd-9-0-RC1:1.19 phil-wifi-20191119:1.20 netbsd-9:1.19.0.2 netbsd-9-base:1.19 phil-wifi-20190609:1.19 netbsd-8-1-RELEASE:1.11 netbsd-8-1-RC1:1.11 pgoyette-compat-merge-20190127:1.14.2.3 pgoyette-compat-20190127:1.18 pgoyette-compat-20190118:1.18 pgoyette-compat-1226:1.16 pgoyette-compat-1126:1.15 pgoyette-compat-1020:1.15 pgoyette-compat-0930:1.15 pgoyette-compat-0906:1.15 pgoyette-compat-0728:1.15 netbsd-8-0-RELEASE:1.11 phil-wifi:1.15.0.2 phil-wifi-base:1.15 pgoyette-compat-0625:1.15 netbsd-8-0-RC2:1.11 pgoyette-compat-0521:1.15 pgoyette-compat-0502:1.14 pgoyette-compat-0422:1.14 netbsd-8-0-RC1:1.11 pgoyette-compat-0415:1.14 pgoyette-compat-0407:1.14 pgoyette-compat-0330:1.14 pgoyette-compat-0322:1.14 pgoyette-compat-0315:1.14 pgoyette-compat:1.14.0.2 pgoyette-compat-base:1.14 matt-nb8-mediatek:1.11.0.6 matt-nb8-mediatek-base:1.11 netbsd-8:1.11.0.4 netbsd-8-base:1.11 prg-localcount2-base3:1.11 prg-localcount2-base2:1.11 prg-localcount2-base1:1.11 prg-localcount2:1.11.0.2 prg-localcount2-base:1.11 pgoyette-localcount-20170426:1.11 bouyer-socketcan-base1:1.11 pgoyette-localcount-20170320:1.11 bouyer-socketcan:1.10.0.2 bouyer-socketcan-base:1.10 pgoyette-localcount-20170107:1.9 pgoyette-localcount-20161104:1.8 localcount-20160914:1.5 pgoyette-localcount:1.4.0.2 pgoyette-localcount-20160806:1.4; locks; strict; comment @# @; 1.24 date 2021.08.09.20.49.08; author andvar; state Exp; branches; next 1.23; commitid DVtnVgF9HMHM3l4D; 1.23 date 2021.05.30.02.37.42; author pgoyette; state Exp; branches; next 1.22; commitid oYg25MDDpqAbj7VC; 1.22 date 2020.07.05.02.04.25; author pgoyette; state Exp; branches 1.22.2.1; next 1.21; commitid R5iRRTUKoJqgHPeC; 1.21 date 2019.12.08.15.51.49; author pgoyette; state Exp; branches; next 1.20; commitid OMXkNpagyp3H1VNB; 1.20 date 2019.09.29.00.57.11; author pgoyette; state Exp; branches; next 1.19; commitid LibCzM52oCAxkQEB; 1.19 date 2019.02.20.04.32.51; author pgoyette; state Exp; branches; next 1.18; commitid TxxzFn90pZEOCscB; 1.18 date 2018.12.28.21.51.49; author pgoyette; state Exp; branches; next 1.17; commitid AEgUD0yzHmAO6C5B; 1.17 date 2018.12.28.21.37.56; author pgoyette; state Exp; branches; next 1.16; commitid Ip1GzUGqtlte2C5B; 1.16 date 2018.12.13.11.28.00; author pgoyette; state Exp; branches; next 1.15; commitid 9WqHn9wLmpPM8D3B; 1.15 date 2018.05.04.00.25.26; author pgoyette; state Exp; branches 1.15.2.1; next 1.14; commitid 7SlHB7ZuzlwZDUAA; 1.14 date 2017.12.15.21.57.09; author pgoyette; state Exp; branches 1.14.2.1; next 1.13; commitid rRrG2Q5K2Rh4i2jA; 1.13 date 2017.08.21.10.38.19; author pgoyette; state Exp; branches; next 1.12; commitid SxFiL0WBNCUeg44A; 1.12 date 2017.08.04.11.55.06; author pgoyette; state Exp; branches; next 1.11; commitid un6mJQ39RCzueT1A; 1.11 date 2017.01.26.04.24.20; author pgoyette; state Exp; branches; next 1.10; commitid vYR6uJcSoxpzQqDz; 1.10 date 2017.01.14.21.18.40; author pgoyette; state Exp; branches 1.10.2.1; next 1.9; commitid jw3Ip1l0l05jQYBz; 1.9 date 2016.12.15.03.24.43; author pgoyette; state Exp; branches; next 1.8; commitid VJ2bFXF5yEMzR1yz; 1.8 date 2016.09.28.06.47.55; author wiz; state Exp; branches; next 1.7; commitid 1pNCKi3aSHu9v1oz; 1.7 date 2016.09.27.22.54.57; author christos; state Exp; branches; next 1.6; commitid MqOBQposPmwNSYnz; 1.6 date 2016.09.27.22.27.50; author pgoyette; state Exp; branches; next 1.5; commitid ZOD9JHeU1VnsJYnz; 1.5 date 2016.08.06.00.30.57; author pgoyette; state Exp; branches; next 1.4; commitid ZXYthzGJW3rk7bhz; 1.4 date 2016.08.05.04.21.01; author pgoyette; state Exp; branches 1.4.2.1; next 1.3; commitid AaoF50tL1Jnbq4hz; 1.3 date 2016.08.05.02.27.14; author pgoyette; state Exp; branches; next 1.2; commitid NSDqlcEOQwxiN3hz; 1.2 date 2016.08.04.22.12.31; author pgoyette; state Exp; branches; next 1.1; commitid gJhTnd3A4zdVn2hz; 1.1 date 2016.08.04.10.45.52; author pgoyette; state Exp; branches; next ; commitid fBdS0dpsES6vxYgz; 1.22.2.1 date 2021.05.31.22.06.55; author cjep; state Exp; branches; next ; commitid eWz9SBW0XqKjJlVC; 1.15.2.1 date 2019.06.10.21.42.38; author christos; state Exp; branches; next 1.15.2.2; commitid jtc8rnCzWiEEHGqB; 1.15.2.2 date 2020.04.13.07.45.37; author martin; state Exp; branches; next ; commitid X01YhRUPVUDaec4C; 1.14.2.1 date 2018.05.21.04.35.50; author pgoyette; state Exp; branches; next 1.14.2.2; commitid X5L8kSrBWQcDt7DA; 1.14.2.2 date 2018.12.26.14.01.13; author pgoyette; state Exp; branches; next 1.14.2.3; commitid xUhK8IAeBM1azj5B; 1.14.2.3 date 2019.01.18.08.48.34; author pgoyette; state Exp; branches; next ; commitid Lmlzg3OVT2cd6f8B; 1.10.2.1 date 2017.04.21.16.51.16; author bouyer; state Exp; branches; next ; commitid dUG7nkTKALCadqOz; 1.4.2.1 date 2016.08.05.04.21.01; author pgoyette; state dead; branches; next 1.4.2.2; commitid da8LmcQp9HeG2bhz; 1.4.2.2 date 2016.08.06.00.18.40; author pgoyette; state Exp; branches; next 1.4.2.3; commitid da8LmcQp9HeG2bhz; 1.4.2.3 date 2016.11.04.14.42.40; author pgoyette; state Exp; branches; next 1.4.2.4; commitid 2m1JRwYmpwPkOOsz; 1.4.2.4 date 2017.01.07.08.53.45; author pgoyette; state Exp; branches; next 1.4.2.5; commitid uEL0C1YuiJrlV0Bz; 1.4.2.5 date 2017.03.20.06.52.12; author pgoyette; state Exp; branches; next ; commitid jjw7cAwgyKq7RfKz; desc @@ 1.24 log @fix various typos in compatibility, mainly in comments. @ text @/* $NetBSD: TODO.modules,v 1.23 2021/05/30 02:37:42 pgoyette Exp $ */ Some notes on the limitations of our current (as of 7.99.35) module subsystem. This list was triggered by an Email exchange between christos and pgoyette. 1. Builtin drivers can't depend on modularized drivers (the modularized drivers are attempted to load as builtins). The assumption is that dependencies are loaded before those modules which depend on them. At load time, a module's undefined global symbols are resolved; if any symbols can't be resolved, the load fails. Similarly, if a module is included in (built-into) the kernel, all of its symbols must be resolvable by the linker, otherwise the link fails. There are ways around this (such as, having the parent module's initialization command recursively call the module load code), but they're often gross hacks. Another alternative (which is used by ppp) is to provide a "registration" mechanism for the "child" modules, and then when the need for a specific child module is encountered, use module_autoload() to load the child module. Of course, this requires that the parent module know about all potentially loadable children. 2. Currently, config(1) has no way to "no define" drivers XXX: I don't think this is true anymore. I think we can undefine drivers now, see MODULAR in amd64, which does no ath* and no select sppp* 3. It is not always obvious by their names which drivers/options correspond to which modules. 4. Right now critical drivers that would need to be pre-loaded (ffs, exec_elf64) are still built-in so that we don't need to alter the boot blocks to boot. This was a conscious decision by core@@ some years ago. It is not a requirement that ffs or exec_* be built-in. The only requirement is that the root file-system's module must be available when the module subsystem is initialized, in order to load other modules. This can be accomplished by having the boot loader "push" the module at boot time. (It used to do this in all cases; currently the "push" only occurs if the booted filesystem is not ffs.) 5. Not all parent bus drivers are capable of rescan, so some drivers just have to be built-in. 6. Many (most?) drivers are not yet modularized 7. There's currently no provisions for autoconfig to figure out which modules are needed, and thus to load the required modules. In the "normal" built-in world, autoconfigure can only ask existing drivers if they're willing to manage (ie, attach) a device. Removing the built-in drivers tends to limit the availability of possible managers. There's currently no mechanism for identifying and loading drivers based on what devices might be found. 8. Even for existing modules, there are "surprise" dependencies with code that has not yet been modularized. For example, even though the bpf code has been modularized, there is some shared code in bpf_filter.c which is needed by both ipfilter and ppp. ipf is already modularized, but ppp is not. Thus, even though bpf_filter is modular, it MUST be included as a built-in module if you also have ppp in your configuration. Another example is sysmon_taskq module. It is required by other parts of the sysmon subsystem, including the "sysmon_power" module. Unfortunately, even though the sysmon_power code is modularized, it is referenced by the acpi code which has not been modularized. Therefore, if your configuration has acpi, then you must include the "sysmon_power" module built-in the kernel. And therefore you also need to have "sysmon_taskq" and "sysmon" built-in since "sysmon_power" rerefences them. 9. As a corollary to #8 above, having dependencies on modules from code which has not been modularized makes it extremely difficult to test the module code adequately. Testing of module code should include both testing-as-a-built-in module and testing-as-a-loaded-module, and all dependencies need to be identified. 10. The current /stand/$ARCH/$VERSION/modules/ hierarchy won't scale as we get more and more modules. There are hundreds of potential device driver modules. 11. There currently isn't any good way to handle attachment-specific modules. The build infrastructure (ie, sys/modules/Makefile) doesn't readily lend itself to bus-specific modules irrespective of $ARCH, and maintaining distrib/sets/lists/modules/* is awkward at best. Furthermore, devices such as ld(4), which can attach to a large set of parent devices, need to be modified. The parent devices need to provide a common attribute (for example, ld_bus), and the ld driver should attach to that attribute rather than to each parent. But currently, config(1) doesn't handle this - it doesn't allow an attribute to be used as the device tree's pseudo-root. The current directory structure where driver foo is split between ic/foo.c and bus1/foo_bus1.c ... busn/foo_busn.c is annoying. It would be better to switch to the FreeBSD model which puts all the driver files in one directory. 12. Item #11 gets even murkier when a particular parent can provide more than one attribute. 13. It seems that we might want some additional sets-lists "attributes" to control contents of distributions. As an example, many of our architectures have PCI bus capabilities, but not all. It is rather painful to need to maintain individual architectures' modules/md_* sets lists, especially when we already have to conditionalize the build of the modules based on architecture. If we had a single "attribute" for PCI-bus-capable, the same attribute could be used to select which modules to build and which modules from modules/mi to include in the release. (This is not limited to PCI; recently we encounter similar issues with spkr aka spkr_synth module.) 14. As has been pointed out more than once, the current method of storing modules in a version-specific subdirectory of /stand is sub-optimal and leads to much difficulty and/or confusion. A better mechanism of associating a kernel and its modules needs to be developed. Some have suggested having a top-level directory (say, /netbsd) with a kernel and its modules at /netbsd/kernel and /netbsd/modules/... Whatever new mechanism we arrive at will probably require changes to installation procedures and bootstrap code, and will need to handle both the new and old mechanisms for compatibility. One additional option mentioned is to be able to specify, at boot loader time, an alternate value for the os-release portion of the default module path, i.e. /stand/$MACHINE/$ALT-RELEASE/modules/ The following statement regarding this issue was previously issued by the "core" group: Date: Fri, 27 Jul 2012 08:02:56 +0200 From: To: Subject: Core statement on directory naming for kernel modules The core group would also like to see the following changes in the near future: Implementation of the scheme described by Luke Mewburn in to allow a kernel and its modules to be kept together. Changes to config(1) to extend the existing notion of whether or not an option is built-in to the kernel, to three states: built-in, not built-in but loadable as a module, entirely excluded and not even loadable as a module. 15. The existing config(5) framework provides an excellent mechanism for managing the content of kernels. Unfortunately, this mechanism does not apply for modules, and instead we need to manually manage a list of files to include in the module, the set of compiler definitions with which to build those files, and also the set of other modules on which a module depends. We really need a common mechanism to define and build modules, whether they are included as "built-in" modules or as separately-loadable modules. (From John Nemeth) Some sort of mechanism for a (driver) module to declare the list of vendor/product/other tuples that it can handle would be nice. Perhaps this would go in the module's .plist file? (See #17 below.) Then drivers that scan for children might be able to search the modules directory for an "appropriate" module for each child, and auto-load. 16. PR kern/52821 exposes another limitation of config(1) WRT modules. Here, an explicit device attachment is required, because we cannot rely on all kernel configs to contain the attribute at which the modular driver wants to attach. Unfortunately, the explicit attachment causes conflicts with built-in drivers. (See the PR for more details.) 17. (From John Nemeth) It would be potentially useful if a "push" from the bootloader could also load-and-push a module's .plist (if it exists. 18. (From John Nemeth) Some sort of schema for a module to declare the options (or other things?) that the module understands. This could result in a module-options editor to manipulate the .plist 19. (From John Nemeth) Currently, the order of module initialization is based on module classes and declared dependencies. It might be useful to have additional classes (or sub-classes) with additional invocations of module_class_init(), and it might be useful to have a non-dependency mechanism to provide "IF module-A and module-B are BOTH present, module-A needs to be initialized before module-B". 20. (Long-ago memory rises to the surface) Note that currently there is nothing that requires a module's name to correspond in any way with the name of file from which the module is loaded. Thus, it is possible to attempt to access device /dev/x, discover that there is no such device so we autoload /stand/.../x/x.kmod and initialize the module loaded, even if the loaded module is for some other device entirely! 21. We currently do not support "weak" symbols in the in-kernel linker. It would take some serious thought to get such support right. For example, consider module A with a weak reference to symbol S which is defined in module B. If module B is loaded first, and then module A, the symbol gets resolved. But if module A is loaded first, the symbol won't be resolved. If we subsequently load module B, we would have to "go back" and re-run the linker for module A. Additional difficulties arise when the module which defines the weak symbol gets unloaded. Then, you would need to re-run the linker and _unresolve_ the weak symbol which is no longer defined. 22. A fairly large number of modules still require a maximum warning level of WARNS=3 due to signed-vs-unsigned integer comparisons. We really ought to clean these up. (I haven't looked at them in any detail, but I have to wonder how code that compiles cleanly in a normal kernel has these issues when compiled in a module, when both are done with WARNS=5). 23. The current process of "load all the emulation/exec modules in case one of them might handle the image currently being exec'd" isn't really cool. (See sys/kern/kern_exec.c?) It ends up auto-loading a whole bunch of modules, involving file-system access, just to have most of the modules getting unloaded a few seconds later. We don't have any way to identify which module is needed for which image (ie, we can't determine that an image needs compat_linux vs some other module). 24. Details are no longer remembered, but there are some issues with building xen-variant modules (on amd4, and likely i386). In some cases, wrong headers are included (because a XEN-related #define is missing), but even if you add the definition some headers get included in the wrong order. One particular fallout from this is the inability to have a compat version of x86_64 cpu-microcode module. PR port-xen/53130 This is likely to be fixed by Chuck Silvers on 2020-07-04 which removed the differences between the xen and non-xen module ABIs. As of 2021-05-28 the cpu-microcode functionality has once again been enabled for i386 and amd64 compat_60 modules. @ 1.23 log @Note that compat_60 modules on i386 and amd64 once again include the microcode-update functionality. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.22 2020/07/05 02:04:25 pgoyette Exp $ */ d132 1 a132 1 both the new and old mechanisms for compatability. @ 1.22 log @Note that the xen vs non-xen issue has been resolved by recent commit from Chuck Silvers. (Also add xref to PR port-xen/53130 for history.) @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.21 2019/12/08 15:51:49 pgoyette Exp $ */ d242 2 @ 1.22.2.1 log @sync with head @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.23 2021/05/30 02:37:42 pgoyette Exp $ */ a241 2 As of 2021-05-28 the cpu-microcode functionality has once again been enabled for i386 and amd64 compat_60 modules. @ 1.21 log @Add another issue that I just remembered, from the time working on the [pgoyette-compat] branch, regarding inability to get some XEN-related modules to build. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.20 2019/09/29 00:57:11 pgoyette Exp $ */ d236 1 a236 1 included in the wrong order. On particular fallout from this is d238 4 a241 1 module. @ 1.20 log @Another issue, as identified on IRC/ICB @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.19 2019/02/20 04:32:51 pgoyette Exp $ */ d231 8 @ 1.19 log @Add an entry to remind someone(tm) to review the need for WARNS=3 in more than 100 modules' Makefile. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.18 2018/12/28 21:51:49 pgoyette Exp $ */ d222 9 @ 1.18 log @Expand the weak-symbol section to mention module unload issues. As noted by martin@@ on source-chages-d list. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.17 2018/12/28 21:37:56 pgoyette Exp $ */ d215 7 @ 1.17 log @Add an entry regarding weak symbols @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.16 2018/12/13 11:28:00 pgoyette Exp $ */ d211 4 @ 1.16 log @Add a note about the (lack of) correspondence between a module's name and the name of the file from which it is loaded. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.15 2018/05/04 00:25:26 pgoyette Exp $ */ d203 8 @ 1.15 log @Add some comments/suggestions from John Nemeth. Thanks! @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.14 2017/12/15 21:57:09 pgoyette Exp $ */ d195 8 @ 1.15.2.1 log @Sync with HEAD @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.19 2019/02/20 04:32:51 pgoyette Exp $ */ a194 27 20. (Long-ago memory rises to the surface) Note that currently there is nothing that requires a module's name to correspond in any way with the name of file from which the module is loaded. Thus, it is possible to attempt to access device /dev/x, discover that there is no such device so we autoload /stand/.../x/x.kmod and initialize the module loaded, even if the loaded module is for some other device entirely! 21. We currently do not support "weak" symbols in the in-kernel linker. It would take some serious thought to get such support right. For example, consider module A with a weak reference to symbol S which is defined in module B. If module B is loaded first, and then module A, the symbol gets resolved. But if module A is loaded first, the symbol won't be resolved. If we subsequently load module B, we would have to "go back" and re-run the linker for module A. Additional difficulties arise when the module which defines the weak symbol gets unloaded. Then, you would need to re-run the linker and _unresolve_ the weak symbol which is no longer defined. 22. A fairly large number of modules still require a maximum warning level of WARNS=3 due to signed-vs-unsigned integer comparisons. We really ought to clean these up. (I haven't looked at them in any detail, but I have to wonder how code that compiles cleanly in a normal kernel has these issues when compiled in a module, when both are done with WARNS=5). @ 1.15.2.2 log @Mostly merge changes from HEAD upto 20200411 @ text @d1 1 a1 1 /* $NetBSD$ */ a221 17 23. The current process of "load all the emulation/exec modules in case one of them might handle the image currently being exec'd" isn't really cool. (See sys/kern/kern_exec.c?) It ends up auto-loading a whole bunch of modules, involving file-system access, just to have most of the modules getting unloaded a few seconds later. We don't have any way to identify which module is needed for which image (ie, we can't determine that an image needs compat_linux vs some other module). 24. Details are no longer remembered, but there are some issues with building xen-variant modules (on amd4, and likely i386). In some cases, wrong headers are included (because a XEN-related #define is missing), but even if you add the definition some headers get included in the wrong order. On particular fallout from this is the inability to have a compat version of x86_64 cpu-microcode module. @ 1.14 log @Add reference to uwe's PR kern/52821 @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.13 2017/08/21 10:38:19 pgoyette Exp $ */ d167 7 d180 15 @ 1.14.2.1 log @Sync with HEAD @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.15 2018/05/04 00:25:26 pgoyette Exp $ */ a166 7 (From John Nemeth) Some sort of mechanism for a (driver) module to declare the list of vendor/product/other tuples that it can handle would be nice. Perhaps this would go in the module's .plist file? (See #17 below.) Then drivers that scan for children might be able to search the modules directory for an "appropriate" module for each child, and auto-load. a172 15 17. (From John Nemeth) It would be potentially useful if a "push" from the bootloader could also load-and-push a module's .plist (if it exists. 18. (From John Nemeth) Some sort of schema for a module to declare the options (or other things?) that the module understands. This could result in a module-options editor to manipulate the .plist 19. (From John Nemeth) Currently, the order of module initialization is based on module classes and declared dependencies. It might be useful to have additional classes (or sub-classes) with additional invocations of module_class_init(), and it might be useful to have a non-dependency mechanism to provide "IF module-A and module-B are BOTH present, module-A needs to be initialized before module-B". @ 1.14.2.2 log @Sync with HEAD, resolve a few conflicts @ text @d1 1 a1 1 /* $NetBSD$ */ a194 8 20. (Long-ago memory rises to the surface) Note that currently there is nothing that requires a module's name to correspond in any way with the name of file from which the module is loaded. Thus, it is possible to attempt to access device /dev/x, discover that there is no such device so we autoload /stand/.../x/x.kmod and initialize the module loaded, even if the loaded module is for some other device entirely! @ 1.14.2.3 log @Synch with HEAD @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.14.2.2 2018/12/26 14:01:13 pgoyette Exp $ */ a202 12 21. We currently do not support "weak" symbols in the in-kernel linker. It would take some serious thought to get such support right. For example, consider module A with a weak reference to symbol S which is defined in module B. If module B is loaded first, and then module A, the symbol gets resolved. But if module A is loaded first, the symbol won't be resolved. If we subsequently load module B, we would have to "go back" and re-run the linker for module A. Additional difficulties arise when the module which defines the weak symbol gets unloaded. Then, you would need to re-run the linker and _unresolve_ the weak symbol which is no longer defined. @ 1.13 log @Add previous statement from core@@ and add reference to earlier E-mail discussion. OK martin@@ @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.12 2017/08/04 11:55:06 pgoyette Exp $ */ d80 1 a80 1 module built-in the kernel. And therefore your also need to d166 7 @ 1.12 log @Add a note regarding the need for a common mechanism for defining and building modules, whether built-in or separately-loadable. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.11 2017/01/26 04:24:20 pgoyette Exp $ */ d138 20 @ 1.11 log @Add comment about possibly prompting for "release" portion of module path at boot-loader time. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.10 2017/01/14 21:18:40 pgoyette Exp $ */ d138 8 @ 1.10 log @Add an entry to discuss association of a kernel with its specific modules. Prompted by recent Email discussion started by wiz (there have been many earlier discussions on this topic, too). @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.9 2016/12/15 03:24:43 pgoyette Exp $ */ d133 5 @ 1.10.2.1 log @Sync with HEAD @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.11 2017/01/26 04:24:20 pgoyette Exp $ */ a132 5 One additional option mentioned is to be able to specify, at boot loader time, an alternate value for the os-release portion of the default module path, i.e. /stand/$MACHINE/$ALT-RELEASE/modules/ @ 1.9 log @Note desire to have a way to selectively build modules (and include them from the modules/mi sets-list) based on "attributes" rather than having to enumerate individual architectures on which to build and include. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.8 2016/09/28 06:47:55 wiz Exp $ */ d123 10 @ 1.8 log @Fix typo. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.7 2016/09/27 22:54:57 christos Exp $ */ d112 11 @ 1.7 log @Minor edit. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.6 2016/09/27 22:27:50 pgoyette Exp $ */ d101 1 a101 1 provide a common attribute (for example, ld_bud), and the ld driver @ 1.6 log @Add some additional comments resulting from my recent efforts to provide ld(4) modularization. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.5 2016/08/06 00:30:57 pgoyette Exp $ */ d7 2 a8 2 1. Builtin drivers can't depend on modularized drivers (the modularized drivers are attempted to load as builtins). d28 11 a38 8 2. Currently, config(1) has no way to "no define" drivers 3. It is not always obvious by their names which drivers/options correspond to which modules. 4. Right now critical drivers that would need to be pre-loaded (ffs, exec_elf64) are still built-in so that we don't need to alter the boot blocks to boot. d49 2 a50 2 5. Not all parent bus drivers are capable of rescan, so some drivers just have to be built-in. d52 1 a52 1 6. Many (most?) drivers are not yet modularized d54 2 a55 2 7. There's currently no provisions for autoconfig to figure out which modules are needed, and thus to load the required modules. d64 2 a65 2 8. Even for existing modules, there are "surprise" dependencies with code that has not yet been modularized. d84 25 a108 21 9. As a corollary to #8 above, having dependencies on modules from code which has not been modularized makes it extremely difficult to test the module code adequately. Testing of module code should include both testing-as-a-built-in module and testing-as-a-loaded-module, and all dependencies need to be identified. 10.The current /stand/$ARCH/$VERSION/modules/ hierarchy won't scale as we get more and more modules. There are hundreds of potential device driver modules. 11.There currently isn't any good way to handle attachment-specific modules. The build infrastructure (ie, sys/modules/Makefile) doesn't readily lend itself to bus-specific modules irrespective of $ARCH, and maintaining distrib/sets/lists/modules/* is awkward at best. Furthermore, devices such as ld(4), which can attach to a large set of parent devices, need to be modified. The parent devices need to provide a common attribute (for example, ld_bud), and the ld driver should attach to that attribute rather than to each parent. But currently, config(1) doesn't handle this - it doesn't allow an attribute to be used as the device tree's pseudo-root. d110 2 a111 2 12.Item #11 gets even murkier when a particular parent can provide more than one attribute. @ 1.5 log @Expand discussion a bit, and provide ppp as an example way to do things. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.4 2016/08/05 04:21:01 pgoyette Exp $ */ d87 18 @ 1.4 log @The ppp compressors are already being handled correctly. No new work will be needed here. @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.3 2016/08/05 02:27:14 pgoyette Exp $ */ d19 8 a26 1 load code), but they're gross hacks. @ 1.4.2.1 log @file TODO.modules was added on branch pgoyette-localcount on 2016-08-06 00:18:40 +0000 @ text @d1 79 @ 1.4.2.2 log @Sync with HEAD @ text @a0 79 /* $NetBSD: TODO.modules,v 1.4 2016/08/05 04:21:01 pgoyette Exp $ */ Some notes on the limitations of our current (as of 7.99.35) module subsystem. This list was triggered by an Email exchange between christos and pgoyette. 1. Builtin drivers can't depend on modularized drivers (the modularized drivers are attempted to load as builtins). The assumption is that dependencies are loaded before those modules which depend on them. At load time, a module's undefined global symbols are resolved; if any symbols can't be resolved, the load fails. Similarly, if a module is included in (built-into) the kernel, all of its symbols must be resolvable by the linker, otherwise the link fails. There are ways around this (such as, having the parent module's initialization command recursively call the module load code), but they're gross hacks. 2. Currently, config(1) has no way to "no define" drivers 3. It is not always obvious by their names which drivers/options correspond to which modules. 4. Right now critical drivers that would need to be pre-loaded (ffs, exec_elf64) are still built-in so that we don't need to alter the boot blocks to boot. This was a conscious decision by core@@ some years ago. It is not a requirement that ffs or exec_* be built-in. The only requirement is that the root file-system's module must be available when the module subsystem is initialized, in order to load other modules. This can be accomplished by having the boot loader "push" the module at boot time. (It used to do this in all cases; currently the "push" only occurs if the booted filesystem is not ffs.) 5. Not all parent bus drivers are capable of rescan, so some drivers just have to be built-in. 6. Many (most?) drivers are not yet modularized 7. There's currently no provisions for autoconfig to figure out which modules are needed, and thus to load the required modules. In the "normal" built-in world, autoconfigure can only ask existing drivers if they're willing to manage (ie, attach) a device. Removing the built-in drivers tends to limit the availability of possible managers. There's currently no mechanism for identifying and loading drivers based on what devices might be found. 8. Even for existing modules, there are "surprise" dependencies with code that has not yet been modularized. For example, even though the bpf code has been modularized, there is some shared code in bpf_filter.c which is needed by both ipfilter and ppp. ipf is already modularized, but ppp is not. Thus, even though bpf_filter is modular, it MUST be included as a built-in module if you also have ppp in your configuration. Another example is sysmon_taskq module. It is required by other parts of the sysmon subsystem, including the "sysmon_power" module. Unfortunately, even though the sysmon_power code is modularized, it is referenced by the acpi code which has not been modularized. Therefore, if your configuration has acpi, then you must include the "sysmon_power" module built-in the kernel. And therefore your also need to have "sysmon_taskq" and "sysmon" built-in since "sysmon_power" rerefences them. 9. As a corollary to #8 above, having dependencies on modules from code which has not been modularized makes it extremely difficult to test the module code adequately. Testing of module code should include both testing-as-a-built-in module and testing-as-a-loaded-module, and all dependencies need to be identified. @ 1.4.2.3 log @Sync with HEAD @ text @d1 1 a1 1 /* $NetBSD$ */ d7 2 a8 2 1. Builtin drivers can't depend on modularized drivers (the modularized drivers are attempted to load as builtins). d21 8 a28 11 2. Currently, config(1) has no way to "no define" drivers XXX: I don't think this is true anymore. I think we can undefine drivers now, see MODULAR in amd64, which does no ath* and no select sppp* 3. It is not always obvious by their names which drivers/options correspond to which modules. 4. Right now critical drivers that would need to be pre-loaded (ffs, exec_elf64) are still built-in so that we don't need to alter the boot blocks to boot. d39 2 a40 2 5. Not all parent bus drivers are capable of rescan, so some drivers just have to be built-in. d42 1 a42 1 6. Many (most?) drivers are not yet modularized d44 2 a45 2 7. There's currently no provisions for autoconfig to figure out which modules are needed, and thus to load the required modules. d54 2 a55 2 8. Even for existing modules, there are "surprise" dependencies with code that has not yet been modularized. d74 5 a78 25 9. As a corollary to #8 above, having dependencies on modules from code which has not been modularized makes it extremely difficult to test the module code adequately. Testing of module code should include both testing-as-a-built-in module and testing-as-a-loaded-module, and all dependencies need to be identified. 10. The current /stand/$ARCH/$VERSION/modules/ hierarchy won't scale as we get more and more modules. There are hundreds of potential device driver modules. 11. There currently isn't any good way to handle attachment-specific modules. The build infrastructure (ie, sys/modules/Makefile) doesn't readily lend itself to bus-specific modules irrespective of $ARCH, and maintaining distrib/sets/lists/modules/* is awkward at best. Furthermore, devices such as ld(4), which can attach to a large set of parent devices, need to be modified. The parent devices need to provide a common attribute (for example, ld_bus), and the ld driver should attach to that attribute rather than to each parent. But currently, config(1) doesn't handle this - it doesn't allow an attribute to be used as the device tree's pseudo-root. The current directory structure where driver foo is split between ic/foo.c and bus1/foo_bus1.c ... busn/foo_busn.c is annoying. It would be better to switch to the FreeBSD model which puts all the driver files in one directory. a79 2 12. Item #11 gets even murkier when a particular parent can provide more than one attribute. @ 1.4.2.4 log @Sync with HEAD. (Note that most of these changes are simply $NetBSD$ tag issues.) @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.9 2016/12/15 03:24:43 pgoyette Exp $ */ a104 11 13. It seems that we might want some additional sets-lists "attributes" to control contents of distributions. As an example, many of our architectures have PCI bus capabilities, but not all. It is rather painful to need to maintain individual architectures' modules/md_* sets lists, especially when we already have to conditionalize the build of the modules based on architecture. If we had a single "attribute" for PCI-bus-capable, the same attribute could be used to select which modules to build and which modules from modules/mi to include in the release. (This is not limited to PCI; recently we encounter similar issues with spkr aka spkr_synth module.) @ 1.4.2.5 log @Sync with HEAD @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.11 2017/01/26 04:24:20 pgoyette Exp $ */ a115 15 14. As has been pointed out more than once, the current method of storing modules in a version-specific subdirectory of /stand is sub-optimal and leads to much difficulty and/or confusion. A better mechanism of associating a kernel and its modules needs to be developed. Some have suggested having a top-level directory (say, /netbsd) with a kernel and its modules at /netbsd/kernel and /netbsd/modules/... Whatever new mechanism we arrive at will probably require changes to installation procedures and bootstrap code, and will need to handle both the new and old mechanisms for compatability. One additional option mentioned is to be able to specify, at boot loader time, an alternate value for the os-release portion of the default module path, i.e. /stand/$MACHINE/$ALT-RELEASE/modules/ @ 1.3 log @Add some info regarding ppp @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.2 2016/08/04 22:12:31 pgoyette Exp $ */ a79 9 10.Work is currently under way on creating a ppp module. Once this is complete, it will require additional work to possibly auto-load the compression routines. Until then, users would need to manually load the ppp_deflate and ppp_bsdcompress modules. Currently the compressors are separate modules which reside on-top-of ppp, and which call back to ppp for "registration"; thus, ppp itself cannot list the compressors as dependencies. See #1 above! @ 1.2 log @Add some more notes @ text @d1 1 a1 1 /* $NetBSD: TODO.modules,v 1.1 2016/08/04 10:45:52 pgoyette Exp $ */ d79 10 @ 1.1 log @Add a list of "module issues" based on an Email discussion between myself and christos@@ @ text @d1 1 a1 1 /* $NetBSD$ */ d54 25 @