diff --git a/nuttx/ChangeLog b/nuttx/ChangeLog index c0e2a2e9a..65fcd4ebd 100644 --- a/nuttx/ChangeLog +++ b/nuttx/ChangeLog @@ -773,7 +773,7 @@ * The NXP LPC2148 and STR711 targets can now also be built using the CodeSourcery or devkitARM Windows native toolchains. -0.4.8 2009-xx-xx Gregory Nutt +0.4.9 2009-06-26 Gregory Nutt * Add strtoll() and strtoull(); Add macros for atol() and atoll(). * dup() and dup2() will now clone socket descriptors @@ -802,6 +802,7 @@ This file has been around for a long time, but I don't think it has every been used before (i.e., prior to the NXFLAT logic) +0.4.10 2009-xx-xx Gregory Nutt diff --git a/nuttx/Documentation/NuttX.html b/nuttx/Documentation/NuttX.html index 0dfc4d0e0..36eba419b 100644 --- a/nuttx/Documentation/NuttX.html +++ b/nuttx/Documentation/NuttX.html @@ -8,7 +8,7 @@

NuttX RTOS

-

Last Updated: June 25, 2009

+

Last Updated: June 26, 2009

@@ -411,6 +411,15 @@
  • ROMFS filesystem support.

    • + +
    + +

    +

  • NXFLAT. + A new binary format call NXFLAT that can be used to + execute separately linked programs in place in a file system. +

    + @@ -687,8 +696,8 @@ -

    nuttx-0.4.8. - This 40th release of NuttX (nuttx-0.4.8) was made on June 13, 2009 +

    nuttx-0.4.9. + This 41st release of NuttX (nuttx-0.4.9) was made on June 26, 2009 and is available for download from the SourceForge website. @@ -698,21 +707,16 @@

    This release adds:
      -
    • - Support for the Olimex STRP711 board. That board is based on the STMicro - STR711 MCU (ARM7TDMI). Integration is complete on the basic port (boot logic, - system time, serial console). Two configurations have been verified: (1) The - board boots and passes the OS test with console output visible on UART0, and - the NuttShell (NSH) is fully functional with interrupt driven serial console. - An SPI driver is available but untested (because the Olimex card slot appears - to accept only MMC cards; I have only SD cards). Additional needed: USB and - driver, MMC integration. -
      • -
    • - Support for the CodeSourcery and devkitARM Windows-native GNU toolchains. - Makefiles have been modified for the LM3S6918, LPC2148, and STR711 to support - these toolchains under Cygwin. -
      • +
    • + Support for a new binary format call NXFLAT that can be used to + execute separately linked programs in place in a file system. + See NXFLAT Documentation + for detailed information. +
      • +
    • + Several important bugs were files related to networking and ROMFS + (see the ChangeLog for a complete list). +

    @@ -1418,57 +1422,7 @@ Other memory: 
      
-nuttx-0.4.8 2009-06-13 Gregory Nutt <spudmonkey@racsa.co.cr>
-
-	* lib/lib_*stream.c: Extend internal stream logic to support incoming streams.
-	* arch/arm/src/str71x: Serial output is now correct and timer interrupts are
-	  working.  The test at configs/olimex-strp711/ostest passes.  This means that
-	  the basic STR-P711 port is complete.
-	* configs/olimex-strp711/nsh:  Add and verifed a NuttShell (NSH) configuration
-	  for the STR-P711.
-	* arch/arm/str71x/str71x_serial.c:  The STR711 interrupt driven serial driver
-	  finally works after some extradinary measures to handle missed interrupts.
-	  NSH is fully functional on the Olimex STR-P711 board.
-	* example/nsh: Moved architecture specific files from NSH directory to board-
-	  specific directories.
-	* config/olimex-strp711/src/up_nsh.c:  Add an NSH board specific directory for
-	  for the Olimex STR7P11 board.
-	* Fixed build of LM3X6918 using the CodeSourcery Windows native toolchain.  There
-	  were lots of issues with Cygwin paths and Cygwin symbolic links.  These changes
-	  may work with the devarmKIT as well, but that remains untested.
-	* The NXP LPC2148 and STR711 targets can now also be built using the CodeSourcery
-	  or devkitARM Windows native toolchains.
-
-pascal-0.1.2 2008-02-10 Gregory Nutt <spudmonkey@racsa.co.cr>
-
-	* Add logic to build and link with the ZDS-II toolchain
-	  use with the z16f.
-	* Make sure that POFF header structures are aligned
-	* Standardized POFF file format to big-endian
-	* Break up large switch statements to lower complexity
-	  and eliminate a compiler bug
-	* Changes so that runtime compiles with SDCC.
-
-buildroot-0.1.6 2009-xx-xx <spudmonkey@racsa.co.cr>
-
-	* Added config/arm7tdmi-defconfig-4.2.4
-	* Added config/arm920t-defconfig-4.3.3
-	* Correct error in arm-defconfig gcc-3.4.6 build. The gcc-3.4.6 configuration
-	  does not not take --with-abi
-	* Correct error in gcc-3.4.6/gcc/collect.c.  Calls open with O_CREAT but
-	  does not specify mode.  Newer host compilers can error out on this.
-
    - - - - - -
    - Unreleased Changes -
    - -
      
-nuttx-0.4.9 2009-xx-xx Gregory Nutt <spudmonkey@racsa.co.cr>
+nuttx-0.4.9 2009-06-26 Gregory Nutt <spudmonkey@racsa.co.cr>
 
 	* Add strtoll() and strtoull(); Add macros for atol() and atoll().
 	* dup() and dup2() will now clone socket descriptors
@@ -1494,9 +1448,17 @@ nuttx-0.4.9 2009-xx-xx Gregory Nutt <spudmonkey@racsa.co.cr>
 	  This file has been around for a long time, but I don't think it has
 	  every been used before (i.e., prior to the NXFLAT logic)
 
-pascal-0.1.3 2009-xx-xx Gregory Nutt <spudmonkey@racsa.co.cr>
+pascal-0.1.2 2008-02-10 Gregory Nutt <spudmonkey@racsa.co.cr>
 
-buildroot-0.1.7 2009-xx-xx <spudmonkey@racsa.co.cr>
+	* Add logic to build and link with the ZDS-II toolchain
+	  use with the z16f.
+	* Make sure that POFF header structures are aligned
+	* Standardized POFF file format to big-endian
+	* Break up large switch statements to lower complexity
+	  and eliminate a compiler bug
+	* Changes so that runtime compiles with SDCC.
+
+buildroot-0.1.7 2009-06-26 <spudmonkey@racsa.co.cr>
 
 	* configs/avr-defconfig-4.3.3: Added support for AVR to support a NuttX
 	  port of the ATmega128.
@@ -1504,6 +1466,24 @@ buildroot-0.1.7 2009-xx-xx <spudmonkey@racsa.co.cr>
 	* toolchain/genromfs:  Added support for the genromfs tool
    + + + + +
    + Unreleased Changes +
    + +
      
+nuttx-0.4.10 2009-xx-xx Gregory Nutt <spudmonkey@racsa.co.cr>
+
+nuttx-0.4.10 2009-xx-xx Gregory Nutt <spudmonkey@racsa.co.cr>
+
+pascal-0.1.3 2009-xx-xx Gregory Nutt <spudmonkey@racsa.co.cr>
+
+buildroot-0.1.8 2009-xx-xx <spudmonkey@racsa.co.cr>
+
    + + + + + diff --git a/nuttx/Documentation/NuttXNxFlat.html b/nuttx/Documentation/NuttXNxFlat.html new file mode 100644 index 000000000..289c8a35c --- /dev/null +++ b/nuttx/Documentation/NuttXNxFlat.html @@ -0,0 +1,549 @@ + + +NXFLAT + + + +
    +
    @@ -1539,6 +1519,10 @@ buildroot-0.1.7 2009-xx-xx <spudmonkey@racsa.co.cr> NuttShell (NSH)
    NXFLAT Binary Format
    NX Graphics Subsystem
    + + + +
    +

    NXFLAT

    +

    >>> Under Construction <<<

    +

    Last Updated: June 26, 2009

    +
    +
    + + + + +
    +

    Table of Contents

    +
    + +
    + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + 1.0 Overview +

    		+ 1.1 Functionality +

    		+ 1.2 Background +

    		+ 1.3 Limitations +

    		+ 1.4 Supported Processors +

    		+ 1.5 Development Status +
    + 2.0 NXFLAT Toolchain +

    		+ 1.2 Building the NXFLAT Toolchain +

    		+ 1.2 mknxflat +

    		+ 1.3 ldnxflat +

    		+ 1.4 Making an NXFLAT module +
    + Appendix A. No GOT Operation +
    + Appendix B. PIC Text Workaround +
    +
    + + + + + +
    +

    1.0 Overview

     +
    f + +

    1.1 Functionality

     + +

    + NXFLAT is a customized and simplified version of binary format implemented a few years ago called + XFLAT + With the NXFLAT binary format you will be able to do the following: +

    +
      +
    • Place separately linked programs in a file system, and
      • +
    • Execute those programs by dynamically linking them to the base NuttX code.
      • +
    +

    + This allows you to extend the NuttX base code after it has been written into FLASH. + One motivation for implementing NXFLAT is support clean CGI under an HTTPD server. +

    +

    + This feature is especially attractive when combined with the NuttX ROMFS support: + ROMFS allows you to execute programs in place (XIP) in flash without copying anything + other than the .data section to RAM. + In fact, the initial NXFLAT release will work only on ROMFS. +

    +

    + This NuttX feature includes: +

    +
      +
    • + A dynamic loader that is built into the NuttX core + (See CVS). +
      • +
    • + Minor changes to RTOS to support position independent code, and +
      • +
    • + A linker to bind ELF binaries to produce the NXFLAT binary format + (See CVS). +
      • +
    + +

    1.2 Background

     + +

    + NXFLAT is derived from XFLAT. + XFLAT is a toolchain add that provides full shared library and XIP executable + support for processors that have no Memory Management Unit (MMU). + NXFLAT is greatly simplified for the deeply embedded environment targeted by Nuttx: +

    +
      +
    • NXFLAT does not support shared libraries, because
      • +
    • NXFLAT does not support exportation of symbol values from a module
      • +
    +

    + Rather, the NXFLAT module only imports symbol values. + In the NXFLAT model, the (PIC1) NXFLAT module resides in a FLASH file system and + when it is loaded at run time, it is dynamically linked only to the (non-PIC) base NuttX + code: + The base NuttX exports a symbol table; the NXFLAT module imports those symbol value + to dynamically bind the module to the base code. +

    +

    1"Position Independent Code"

    + +

    1.3 Limitations

     + +
      +
    • ROMFS Only. + The initial NXFLAT release will work only with executable modules residing on a ROMFS + file system. + That is because the loader depends on the capability to mmap() the code segment. + NUTTX does not provide any general kind of file mapping capability. + In fact, true file mapping is only possible with RTOSs and MCUs that provide an MMU1 + and only ROMFS supports that kind of XIP execution from FLASH. + It is possible to simulate file mapping by allocating memory and copy the file into memory. + NXFLAT would work that kind of file mapping to and that feature could easily be added to NuttX. +
      • +
    • GCC/ARM/Cortex-M3 Only + At present, the NXFLAT toolchain is only available for ARM and Cortex-M3 (thumb2) targets. +
      • +
    • Read-Only Data in RAM + Read-only data must reside in RAM. + In code generated by GCC, all data references are indexed by the PIC2 base register + (that is usually R10 or sl for the ARM processors). + The includes read-only data (.rodata). + Embedded firmware developers normally like to keep .rodata in FLASH with the code sections. + But because all data is referenced with the PIC base register, all of that data must lie in RAM. + A NXFLAT change to work around this is under investigation3. +
      • +
    • Globally Scoped Function Function Pointers + If a function pointer is taken to a statically defined function, then (at least for ARM) GCC will + generate a relocation that NXFLAT cannot handle. + The workaround is make all such functions global in scope. + A fix would involve a change to the GCC compiler as described in Appendix B. +
      • +
    • No Callbacks + Callbacks through function pointers must be avoided or, when then cannot be avoided, handled very specially. + The reason for this is that the PIC module requires setting of a special value in a PIC register. + If the callback does not set the PIC register, then the called back function will fail because + it will be unable to correct access data memory. + Special logic is in place to handle: Signal callbacks, watchdog timer callbacks. + But other callbacks (like those used with qsort() must be avoided in an NXFLAT module. +
      • +
    + +

      + 1"Memory Management Unit"
      + 2"Position Independent Code"
      + 3A work around is under consideration: + At run time, the .rodata offsets will be indexed by a RAM address. + If the dynamic loader were to offset those .rodata offsets properly, it + still might be possible to reference .rodata in ROM. + That work around is still a top of investigation at this time. +

    + +

    1.4 Supported Processors

     + +

    + As mentioned above, the NXFLAT toolchain is only available for ARM and + Cortex-M3 (thumb2) targets. + Furthermore, NXFLAT has only been tested on the Eagle-100 LMS6918 Cortex-M3 board. +

    + +

    1.5 Development Status

     + +

    + The initial release of NXFLAT was made in NuttX version 0.4.9. + Testing is limited to the tests found under examples/nxflat in the source tree. + Some known problems exist + (see the TODO list). + As such, NXFLAT is currently in an early alpha phase. +

    + + + + + +
    +

    2.0 NXFLAT Toolchain

     +
    + +

    1.2 Building the NXFLAT Toolchain

     + +

    + In order to use NXFLAT, you must use special NXFLAT tools to create the binary module in FLASH. + To do this, you will need to download the buildroot package and build it on your Linux or Cygwin machine. + The buildroot can be downloaded from + Sourceforge. + You will need version 0.1.7 or later. +

    +

    + Here are some general build instructions: +

    +
      + +
    • + You must have already configured Nuttx in <some-dir>/nuttx +
    • +
    • + Download the buildroot package buildroot-0.x.y into <some-dir> +
    • +
    • + Unpack <some-dir>/buildroot-0.x.y.tar.gz using a command like tar zxf buildroot-0.x.y. + This will result in a new directory like <some-dir>/misc/buildroot-0.x.y +
      • +
    • + Move this into position: mv <some-dir>/misc/buildroot-0.x.y <some-dir>/buildroot +
      • +
    • + cd <some-dir>/buildroot +
      • +
    • + Copy a configuration file into the top buildroot directory: cp configs/abc-defconfig-x.y.z .config. +
      • +
    • + Enable building of the NXFLAT tools by make menuconfig. + Select to build the NXFLAT toolchain with GCC (you can also select omit building GCC with and only build the + NXFLAT toolchain for use with your own GCC toolchain). +
      • +
    • + Make the toolchain: make. + When the make completes, the tool binaries will be available under + <some-dir>/buildroot/build_abc/staging_dir/bin +
      • +
    + +

    1.2 mknxflat

     + +

    + mknxflat is used to build a thunk file. + See below for usage. + +

      +Usage: mknxflat [options] <bfd-filename>
+
+Where options are one or more of the following.  Note
+that a space is always required between the option and
+any following arguments.
+
+  -d Use dynamic symbol table. [symtab]
+  -f <cmd-filename>
+      Take next commands from <cmd-filename> [cmd-line]
+  -o <out-filename>
+     Output to  [stdout]
+  -v Verbose output [no output]
+  -w Import weakly declared functions, i.e., weakly
+     declared functions are expected to be provided at
+     load-time [not imported]
+
    + +

    1.3 ldnxflat

     + +

    + ldnxflat is use to link your object files along with the thunk file + generated by mknxflat to produce the NXFLAT binary module. + See below for usage. +

    + +
      +Usage: ldnxflat [options] <bfd-filename>
+
+Where options are one or more of the following.  Note
+that a space is always required between the option and
+any following arguments.
+
+  -d Use dynamic symbol table [Default: symtab]
+  -e <entry-point>
+     Entry point to module [Default: _start]
+  -o <out-filename>
+     Output to <out-filename> [Default: <bfd-filename>.nxf]
+  -s <stack-size>
+     Set stack size to <stack-size> [Default: 4096]
+  -v Verbose output. If -v is applied twice, additional
+     debug output is enabled [Default: no verbose output].
+
    + +

    1.4 Making an NXFLAT module

     + +

    + Below is a snippet from an NXFLAT make file (simplified from NuttX + + Hello, World! example. +

    +

      + +
        + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
        Target 1hello.r1:hello.o


        		abc-elf-ld -r -d -warn-common -o $@ $^
        Target 2hello-thunk.S:hello.r1


        		mknxflat -o $@ $^
        Target 3hello.r2:hello-thunk.S


        		 + abc-elf-ld -r -d -warn-common -T binfmt/libnxflat/gnu-nxflat.ld -no-check-sections -o $@ hello.o hello-thunk.o +
        Target 4hello:hello.r2


        		ldnxflat -e main -s 2048 -o $@ $^
      + +

      Target 1. + This target links all of the object files together into one relocation object. + In this case, there is only a single object file, hello.o, and it is linked to produce hello.r1. +

      + +

      Target 2. + Given hello.r1, this target will invoke mknxflat + to make the thunk file, hello-thunk.S. + This thunk contains all of the information needed to create the imported function list. +

      + +

      Target 3 + This this target is similar to Target 1. + In this case, it will link together the object file, hello.o along with the assembled thunk file, + hello-thunk.o to create another, single relocatable object, hello.r2. + The linker script, gnu-nxflat.ld is required at this point to correctly position the sections. +

      + +

      Target 4. + Finally, this target will use the hello.r2 relocatable object to create the final, NXFLAT module + hello by calling ldnxflat. +

      + + + + + +
      +

      Appendix A. No GOT Operation

       +
      + +

      + When GCC generate position independent code, new code sections will appear in your programs. + One of these the the GOT (Global Offset Table) and, in ELF environments, the PLT (Procedure + Lookup Table. + For example, if your C code generated (ARM) assembly language like this without PIC: +

      +

        +        ldr     r1, .L0         <-- Fetch the offset to 'x'
+        ldr     r0, [r10, r1]   <-- Load the value of 'x' with PIC
+                                offset
+        ...
+.L0:    .word   x               <-- Offset to 'x'
+
      + +

      + Then when PIC is enabled (say with the -fpic compiler option), it will generate code like + this: +

      + +
        +        ldr     r1, .L0         <-- Fetch the offset to the GOT entry
+        ldr     r1, [r10,r1]    <-- Fetch the (relocated) address
+                                of 'x' from the GOT
+        ldr     r0, [r1, #0]    <-- Fetch the value of 'x'
+        ...
+.L1     .word   x(GOT)          <-- Offset to entry in the GOT
+
      + +

      See reference

      + +

      + Notice that the generates an extra level of indirection through the GOT. + This indirection is not needed by NXFLAT and only adds more RAM usage and + execution time. +

      +

      + NXFLAT (like XFLAT) can work even better with + the GOT. + Patches again older version of GCC exist to eliminate the GOT indirections. + Several are available here + if you are inspired to port them to a new GCC version. +

      + + + + + + +
      +

      Appendix B. PIC Text Workaround

       +
      + +

      + There is a problem with the memory model in GCC that prevents it from + being used as you need to use it in the NXFLAT context. + The problem is that GCC PIC model assumes that the executable lies in a flat, contiguous (virtual) address space like: +

      +

        + + + + + + + + + + + + + +
        Virtual
        .text
        .got
        .data
        .bss
      +

      + It assumes that the PIC base register (usually r10 for ARM) points to the base of .text + so that any address in .text, .got, .data, .bss + can be found with an offset from the same base address. + But that is not the memory arrangement that we need in the XIP embedded environment. + We need two memory regions, one in FLASH containing shared code and on per task in RAM containing task-specific data: +

      + +
        + + + + + + + + + + + + + +
        FlashRAM
        .text.got

        		.data

        		.bss
      + +

      + The PIC base register needs to point to the base of the .got and only + addresses in the .got, .data, and .bss + sections can be accessed as an offset from the PIC base register. + See also this + XFLAT discussion. +

      +

      + Patches again older version of GCC exist to correct this GCC behavior. + Several are available here + if you are inspired to port them to a new GCC version. +

      + + + diff --git a/nuttx/ReleaseNotes b/nuttx/ReleaseNotes index f421c5365..d8c04ed2a 100644 --- a/nuttx/ReleaseNotes +++ b/nuttx/ReleaseNotes @@ -923,3 +923,17 @@ This is the 40th release of NuttX. This release adds: This tarball contains a complete CVS snapshot from June 13, 2009. +nuttx-0.4.9 +^^^^^^^^^^^ + +This is the 41st release of NuttX. This release adds: + + * Support for a new binary format call NXFLAT that can be used to + execute separately linked programs in place in a file system. + See http://www.nuttx.org/NuttXNxFlat.html. + + * Several important bugs were files related to networking and ROMFS + (see the ChangeLog for a complete list). + +This tarball contains a complete CVS snapshot from June 26, 2009. + diff --git a/nuttx/TODO b/nuttx/TODO index da9356e26..3370e58c0 100644 --- a/nuttx/TODO +++ b/nuttx/TODO @@ -1,4 +1,4 @@ -NuttX TODO List (Last updated June 19, 2009) +NuttX TODO List (Last updated June 26, 2009) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (5) Task/Scheduler (sched/) @@ -6,15 +6,16 @@ NuttX TODO List (Last updated June 19, 2009) (1) Signals (sched/, arch/) (1) pthreads (sched/) (1) C++ Support + (5) Binary loaders (binfmt/) (12) Network (net/, netutils/) (1) USB (drivers/usbdev) (4) Libraries (lib/) (8) File system/Generic drivers (fs/, drivers/) (2) Graphics subystem (graphics/) (1) Pascal add-on (pcode/) - (0) Documentation (Documentation/) + (2) Documentation (Documentation/) (6) Build system / Toolchains - (2) NuttShell (NSH) (examples/nsh) + (3) NuttShell (NSH) (examples/nsh) (3) Other Applications & Tests (examples/) (2) Linux/Cywgin simulation (arch/sim) (3) ARM (arch/arm/) @@ -102,6 +103,39 @@ o C++ Support constructor logic will probably have to be performed by user logic in user_start(). +o Binary loaders (binfmt/) + ^^^^^^^^^^^^^^^^^^^^^^^^ + + Description: Not all of the NXFLAT test under examples/nxflat are working. + Most simply do not compile yet. tests/mutex runs okay but + outputs garbage on completion. + Status: Open + Priority: High + + Description: The ARM up_getpicbase() does not seem to work. This means + the some features like wdog's might not work in NXFLAT modules. + Status: Open + Priority: Medium-High + + Description: At present, all .rodata must be put into RAM. There is a + tentative design change that might allow .rodata to be placed + in FLASH (see Documentation/NuttXNxFlat.html). + Status: Open + Priority: Medium + + Description: If the function pointer to a statically defined function is + taken, then GCC generates a relocation that cannot be handled + by NXFLAT. There is a solution described in Documentataion/NuttXNxFlat.html, + by that would require a compiler change (which we want to avoid). + The simple workaround is to make such functions global in scope. + Status: Open + Priority: Low (probably will not fix) + + Description: In the NXFLAT symbol tables... Using a 32-bit hash value instead + of a string to identify a symbol should result in a smaller footprint. + Status: Open + Priority: Low + o Network (net/, netutils/) ^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -325,6 +359,10 @@ o Documentation (Documentation/) Status: Open Priority: Low + Description: Need to document binary loader APIs + Status: Open + Priority: Low + o Build system ^^^^^^^^^^^^ @@ -378,6 +416,10 @@ o NuttShell (NSH) (examples/nsh) Status: Open Priority: Med-High + Description: Add support to NSH to run NXFLAT programs from a ROMFS file system + Status: Open + Priority: Low + o Other Applications & Tests (examples/) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/nuttx/examples/README.txt b/nuttx/examples/README.txt index b31c37490..e228de207 100644 --- a/nuttx/examples/README.txt +++ b/nuttx/examples/README.txt @@ -143,7 +143,7 @@ examples/nxflat This example builds a small NXFLAT test case. This includes several test programs under examples/nxflat tests. These tests are build using the NXFLAT format and installed in a ROMFS file system. At run time, - each program int eh ROMFS file system is executed. + each program in the ROMFS file system is executed. Requires CONFIG_NXFLAT. examples/null ^^^^^^^^^^^^^