Bare-metal and freestanding programming#

This chapter is for people who build firmware or kernels with EmbCC: code that runs with no operating system underneath it. It covers, per board family, what an image needs before main runs (vector table, reset handler, stack pointer, .data and .bss), how interrupt and trap handlers are written, how an image is linked for a memory map with EmbLD, which runtime routines the compiler expects to find, how to size the stack, and how to run an image under QEMU the way EmbCC's own test harnesses do. Target triples, data models and -m options are listed in Targets; this chapter refers to them rather than repeating them.

Overview#

The aliases each triple accepts are listed in Targets.

Compile, then link#

A firmware build compiles each file with -c and links the objects, as a Makefile written for arm-none-eabi-gcc does. The driver links them, adding the target's librt.a (and libc.a, if built) itself; the memory map is a linker script (--link-script) or -Wl,-Ttext/-Wl,-Tdata:

embcc --target=thumbv7m-none-eabi --opt=size -c startup.c -o startup.o
embcc --target=thumbv7m-none-eabi --opt=size -c main.c    -o main.o
embcc --target=thumbv7m-none-eabi --link-script=board.ld startup.o main.o -o fw.elf

or, naming every input to embld explicitly:

embld -e Reset_Handler -Ttext 0x0 -Tdata 0x20000000 --rom-limit 262144 \
      startup.o main.o build/libc/thumbv7m-none-eabi/librt.a -o fw.elf

Without a memory map the driver refuses the link (embcc: error: linking a thumbv7m-none-eabi image needs its memory map).

embld reads ARM, RV32, RV64, MIPS32, AVR and x86-64 objects. It does not read AArch64 objects; a bare-metal AArch64 image is linked with another toolchain's linker (see Running images under QEMU).

--no-stdlib, --no-default-libs and --no-start-files mean what they mean to gcc, and -lc, -lm and -lgcc name EmbCC's own libc.a and librt.a. -Wl, options are ignored by a compile with -c, so a build system that passes them to every command still compiles. See Libraries.

Freestanding compilation#

--freestanding and --no-builtins#

GCC: -ffreestanding, -fno-builtin.

Both are accepted and change nothing. EmbCC makes no hosted assumptions to drop: it recognises no library function by name, only the __builtin_ spellings, and it never turns a loop into a call to memset or memcpy.

__STDC_HOSTED__ is defined as 1 on every target, with or without --freestanding. Code that tests it to detect a freestanding build gets the hosted answer.

Headers#

The headers a freestanding program may use are found without any -I:

<stdint.h> and <limits.h> derive every width from the target's predefined macros, so int32_t is long on AVR (where int is 16 bits) and int64_t is long long on every 32-bit and 16-bit target.

The other C library headers (<stdio.h>, <stdlib.h>, <math.h> and the rest) also resolve, but no C library is built for the embedded targets: a firmware image that calls printf or malloc has to supply them. --no-std-include removes EmbCC's directories from the search; see Invoking.

What the compiler emits calls to#

Code compiled for an embedded target refers to symbols outside the program in exactly these cases:

• Runtime helpers for operations the machine has no instruction for. These are libgcc's names and are listed per target below; they are in the target's librt.a.
• memcpy, memmove, memset, memcmp and the other string functions, only where the program calls them, by name or as __builtin_memcpy and so on. Struct assignment, struct arguments and return values, and zero or partial initialisation of arrays are always expanded inline; EmbCC never introduces a call to memcpy or memset that the source does not contain.
• C++ runtime routines, from C++ code (see C++).

It emits no stack-protector calls (-fstack-protector is refused), no __aeabi_* calls, no __atomic_* or __sync_* library calls (atomic operations are inline or refused; see each board's section), no profiling or sanitizer runtime calls (--sanitize= checks trap in place), and no abort: __builtin_trap() is udf #0 on ARM, unimp on RISC-V and break on MIPS32, and on AVR a jump to itself.

Which operations become helper calls depends on the target:

The complete list of what each librt.a defines is in Libraries. The names are libgcc's, so an object compiled by EmbCC links against another toolchain's libgcc, and objects from GCC or Clang that call the libgcc names link against EmbCC's librt.a. On ARM it also has the run-time ABI's names, which other ARM compilers call and EmbCC's own code does not (__aeabi_ldivmod, __aeabi_fadd, __aeabi_h2f, ...), all weak, so an object from one of them links against it too.

The runtime archive#

make rt-embedded builds one librt.a per embedded triple with tools/build-rt.sh, at --opt=size, into build/libc/TRIPLE/librt.a: avr, thumbv6m-none-eabi, thumbv7m-none-eabi, thumbv7em-none-eabi, thumbv7em-none-eabihf, thumbv8m.main-none-eabi, thumbv8m.main-none-eabihf, riscv32-unknown-elf and mipsel-none-elf. make install copies each to PREFIX/lib/embcc/VERSION/TRIPLE/librt.a. To build one by hand:

sh tools/build-rt.sh thumbv7em-none-eabihf out/    # writes out/librt.a

The hard-float triples have archives of their own because their objects do not link with soft-float ones (see The float ABI).

There is no archive for riscv64-unknown-elf: two of the runtime files use 128-bit integers, which the RV64 backend does not lower. An RV64 image compiles the files it needs itself:

embcc --target=riscv64-unknown-elf --opt=size -c lib/rt/softfp.c  -o softfp.o
embcc --target=riscv64-unknown-elf --opt=size -c lib/rt/complex.c -o complex.o

Put the archive after the objects that use it. A helper missing from the link is reported with what it is:

embld: undefined symbol '__divdi3' (referenced by main.o)
  note: this is a compiler-runtime helper (libgcc's __muldi3 family) — the routine a backend calls for an operation the machine has no instruction for, such as 128-bit multiply or divide. EmbCC ships these in librt.a (lib/rt); the driver puts it on the link line by itself, and a hand-written link has to name it after libc.a

Each runtime source file is one archive member, so an image pays for a file when it uses any routine in it. On Cortex-M and RISC-V all of binary32 and binary64 is one member (softfp.o, about 8 KB). On AVR the float routines are split into one member per group (add, multiply, divide, compare, conversions), because the whole set does not fit in the part's flash.

Startup code and the linker's symbols#

A C startup on every embedded target does the same four things before main: set the stack pointer, copy .data from flash to RAM, zero .bss, and run static constructors if the program has any. EmbLD provides the symbols for the middle two when -Tdata is given:

These are defined only when something references them, and a definition in the program wins. They are all a C runtime needs:

extern unsigned __data_load, __data_start, __data_end;
extern unsigned __bss_start, __bss_end;
typedef void (*initfn)(void);
extern initfn __init_array_start[], __init_array_end[];

static void crt_init(void)
{
    unsigned *d = &__data_start, *s = &__data_load;
    while (d < &__data_end)
        *d++ = *s++;
    for (d = &__bss_start; d < &__bss_end; d++)
        *d = 0;
    for (initfn *f = __init_array_start; f < __init_array_end; f++)
        (*f)();
}

The EmbLD options that shape a firmware image:

An input section named .vectors or .isr_vector is placed first in the image, ahead of .text. Other sections are grouped by prefix (.text.* with .text, .data.* with .data); a section with any other name is kept as a group of its own, after .rodata if it is read-only and after .data if it is writable, and is bracketed by __start_NAME/__stop_NAME (for a C-identifier name) or __NAME_start/__NAME_end (for .NAME). There is no garbage collection of unused sections.

A project that comes with a GNU ld linker script links with it instead: embld -T SCRIPT replaces all of the options above, and the startup uses whatever symbols the script defines (_sidata, _sdata, _ebss in STM32CubeMX's, for example). See Linker scripts. A section of your own that holds only const objects (a command table, a list of drivers) is read-only, as gcc makes it, so a script's orphan rule stores it in flash after .rodata.

An image too large for --rom-limit is refused:

embld: the image needs 9072 bytes of flash and the part has 4096 (--rom-limit): 9055 of text, 16 of initial data

The output is an ELF executable. Its second PT_LOAD has the RAM address as its virtual address and the flash address as its physical address, so llvm-objcopy -O binary fw.elf fw.bin produces the flash image with .data's initial values in place, and -O ihex an Intel HEX file. EmbCC ships no objcopy. The symbol table is kept in the image, outside every loadable segment, so it costs no flash. All EmbLD options are in EmbLD.

volatile and memory-mapped registers#

An access through a volatile lvalue is performed every time the source makes it, at every optimization level, so a peripheral register can be declared the usual way:

struct uart {
    volatile unsigned data;
    volatile unsigned status : 8;
    volatile unsigned mode : 3;
};
#define UART0 ((struct uart *)0x4000C000u)

void put(char c)
{
    while (UART0->status & 0x20)    /* one read of the register per test */
        ;
    UART0->data = c;                /* one store */
}

• A read or write through a pointer to volatile is never removed, merged with another, moved out of a loop or replaced by a value already known. (void)UART0->data; performs the read.
• A volatile bit-field is accessed through its storage unit. Reading the field loads the unit once; assigning to it loads the unit once and stores it once. The value of an assignment such as x = (r->mode = 5) is computed from the stored value, without reading the register again, which matters for a register that clears when read.
• A volatile local variable stays in memory, so a delay loop such as for (volatile int i = 0; i < 1000; i++) ; performs every read and write of i.

See Optimization.

Sizing the stack: --stack-usage#

A microcontroller has no guard page and no room to grow a stack, so the deepest call path has to be added up before the image runs. --stack-usage writes one line per emitted function to a .su file beside the object (the output name with its extension replaced), in GCC's format: FILE:LINE:FUNCTION, a tab, the frame size in bytes, a tab, static.

embcc --target=thumbv7m-none-eabi --opt=size --stack-usage -c su.c -o su.o

su.c:1:leaf	0	static
su.c:2:buf_user	264	static
su.c:3:caller	8	static

The number is that function's own frame and nothing it calls:

Functions the inliner absorbed or that were never emitted are left out rather than reported as 0. Combine the numbers along the call graph to get a worst case; embcc inspect callgraph FILE.c prints the graph the optimizer sees at the same -O level. Add the hardware's own exception frame for each level of interrupt nesting (on Cortex-M, 32 bytes per level, more with an active FPU context).

Leaving out what is never called: --gc-sections#

Flash is the limit a firmware build meets first. Compile with --sections=functions --sections=data, so each function and object is a section of its own, and link with --gc-sections: the linker keeps what the program can reach and drops the rest -- a driver's functions the program never calls, a table nothing reads.

CF="--target=thumbv7em-none-eabi -O2 -ffunction-sections -fdata-sections"
embcc $CF -c main.c -o main.o
embcc $CF -c drivers.c -o drivers.o
embcc --target=thumbv7em-none-eabi --link-script=board.ld \
      -Wl,--gc-sections,--print-memory-usage,-Map=fw.map \
      startup.o main.o drivers.o -o fw.elf

Memory region         Used Size  Region Size  %age Used
             RAM:       1968 B        64 KB      3.00%
           FLASH:        844 B       256 KB      0.32%

EmbCC's own libraries are built the same way: lib/libc and the runtime have a section per function and per object, so --gc-sections drops what a program does not reach inside them too, not only in its own files. On a Cortex-M4, printf brings 8 KB of code where it brought 16, and strtod with snprintf's %f 19 KB where it brought 78. Linked without --gc-sections, an image is the size it always was.

What is kept is what the entry symbol, -u and the script's KEEP() reach, through every relocation of everything kept. Some things are reached without a reference, and are kept too:

• the constructor and destructor arrays (.init_array, .fini_array, .preinit_array, .ctors, .dtors) and .init/.fini;
• a section the program walks with __start_NAME/__stop_NAME;
• without a linker script, the vector table (.vectors, .isr_vector); with one, keep it with KEEP(*(.isr_vector)), as every vendor script does;
• notes, and a section marked SHF_GNU_RETAIN.

An interrupt handler is reached from the vector table, and a function from a table of pointers, so nothing else is needed for them. A function that is entered only from outside the image -- by a debugger script, say -- needs -Wl,-u,NAME. --print-gc-sections lists what was dropped; the map file (-Map) lists it too, with every section that was kept, its address, and the symbols in it; --print-memory-usage is the table above, region by region. The details are in embld.

ARM Cortex-M#

Choosing the target#

--cpu=cortex-m0, cortex-m0plus, cortex-m1, cortex-m23, cortex-m3, cortex-m4, cortex-m7 and cortex-m33 select the sub-architecture as GCC's options do. The Secure side of a TrustZone-M build is --cmse (Targets). --isa=thumb is accepted and has no effect; --isa=arm is refused (-marm is not supported: a Cortex-M has no ARM instruction set, only Thumb). Plain char is unsigned, long double is 8 bytes, and an enum is int-sized unless its values need a wider type, or the smallest type that holds them under -fshort-enums (arm-none-eabi-gcc's default) or __attribute__((packed)). The rest of the data model is in Targets.

Vector table and reset handler#

A Cortex-M reads its initial stack pointer from the first word of the vector table and the reset handler's address from the second, so the whole startup is C: the table is an array placed in .vectors, and the reset handler is an ordinary function.

/* startup.c -- for a part with 64 KiB of SRAM at 0x20000000 */
extern unsigned __data_load, __data_start, __data_end;
extern unsigned __bss_start, __bss_end;

int main(void);
void Reset_Handler(void);
void SysTick_Handler(void);

#define SRAM_TOP 0x20010000u

__attribute__((section(".vectors"), used))
void *const vectors[16] = {
    (void *)SRAM_TOP,          /* 0: initial stack pointer */
    (void *)Reset_Handler,     /* 1: reset */
    [15] = (void *)SysTick_Handler,
};

void Reset_Handler(void)
{
    unsigned *d = &__data_start, *s = &__data_load;
    while (d < &__data_end)
        *d++ = *s++;
    for (d = &__bss_start; d < &__bss_end; d++)
        *d = 0;
    main();
    for (;;)
        ;
}

used keeps the table even though nothing refers to it. The linker sets the Thumb bit in every function address it writes into the table. CMSIS's section name .isr_vector is placed the same way as .vectors.

A function that must run with no prologue -- a context switch, or a reset handler that sets up the stack itself -- is written as __attribute__((naked)), whose body is its asm (see Naked functions); on a Cortex-M a reset handler needs none, since the processor loads the stack pointer from the table. FreeRTOS's Cortex-M3 port, whose PendSV and SVC handlers are naked functions, builds unmodified and runs (tests/golden/freertos-cm3.sh).

Interrupt handlers#

The processor saves r0-r3, r12, lr, pc and xPSR on exception entry and puts an EXC_RETURN value in lr, so an ordinary function is a correct handler: its prologue saves the rest and its bx lr performs the exception return. __attribute__((interrupt)) is accepted on the Cortex-M targets, as CMSIS headers write it, and changes nothing in the generated code.

volatile unsigned ticks;

__attribute__((interrupt)) void SysTick_Handler(void) { ticks++; }

Put the handler's address in its slot of the vector table. On a part with an FPU, the exception entry also preserves s0-s15 and FPSCR when the FPU's automatic state preservation is enabled, which it is at reset.

__attribute__((signal)) is refused on ARM: it is AVR's spelling. Inline assembly covers the special registers a handler or a critical section needs (mrs/msr on PRIMASK, BASEPRI and the others, cpsid/cpsie, dsb, dmb, isb, wfi, wfe, sev, bkpt); see Inline assembly.

Atomics#

32-bit and narrower atomic operations are inline: ldrex/strex loops with dmb barriers. A 64-bit atomic is a call to __atomic_*_8, which lib/rt implements with PRIMASK set; see Eight-byte atomics on a 32-bit target.

Enabling the FPU#

The FPU is off at reset. Grant full access to coprocessors CP10 and CP11 in CPACR before the first floating-point instruction runs, and follow the write with dsb and isb:

#define CPACR (*(volatile unsigned *)0xE000ED88u)

void Reset_Handler(void)
{
    unsigned *d = &__data_start, *s = &__data_load;
    CPACR |= (3u << 20) | (3u << 22);     /* CP10 and CP11: full access */
    __asm__ volatile("dsb");
    __asm__ volatile("isb");
    /* ... copy .data, zero .bss, call main ... */
}

Do this first: the optimizer may use a VFP register for a value anywhere after it, including inside the copy loops. Without it the first VFP instruction takes a UsageFault.

The float ABI#

--fpu= and --float-abi= are read together after the whole command line, in either order:

The -eabihf triples mean the part's unit with --float-abi=hard: the Cortex-M4F's for thumbv7em-none-eabihf, or the Cortex-M7's when --cpu=cortex-m7 is given too. An explicit --float-abi= overrides that, and --print-target reports the result (thumbv7em-none-eabihf --float-abi=soft prints thumbv7em-none-eabi).

The Cortex-M4F and M33 units are single-precision: float arithmetic becomes VFP instructions and double arithmetic stays a call into librt.a. The Cortex-M7's computes both. With --fpu=fpv5-d16, double addition, subtraction, multiplication, division, negation, fabs, __builtin_sqrt, comparisons, and conversions to and from float and 32-bit integers are VFP .f64 instructions, and a double that needs a register lives in d8-d15, which a function that uses them saves with vpush. Only the conversions between double or float and a 64-bit integer remain calls (__fixdfdi, __floatdidf and their kin); lib/rt/softfp.c built for that unit keeps those eight routines and nothing else. The runtime helpers use the base (core-register) convention under every float ABI.

The FPU's predefined macros follow: __ARM_FP is 0x4 for the single-precision units and 0xc for FPv5-D16, which also defines __ARM_FPV5__. (clang says 0x6 and 0xe; EmbCC leaves out bit 1, hardware half-precision conversion, which it does not emit.)

Invalid combinations are refused:

embcc: error: --float-abi=hard needs an FPU to use: add --fpu=fpv4-sp-d16 (Cortex-M4F), --fpu=fpv5-d16 (Cortex-M7) or --fpu=fpv5-sp-d16 (Cortex-M33)
embcc: error: --fpu=fpv4-sp-d16 is an ARMv7E-M unit, and the part is ARMv7-M (a Cortex-M3 has no FPU); add --cpu=cortex-m4
embcc: error: --fpu=fpv5-d16 is the Cortex-M7's double-precision unit, and --cpu=cortex-m4 does not have it; the Cortex-M4F's is --fpu=fpv4-sp-d16
embcc: error: --float-abi=bogus is not an ARM float ABI: it is one of soft, softfp and hard

Each object records its FPU (Tag_FP_arch: VFPv4-D16 for the M4F's unit, FPv5-D16 for the M7's and the M33's, with Tag_ABI_HardFP_use saying single precision on the two -sp- units), its float ABI and its enum size in its ARM attributes, and EmbLD refuses to link objects that disagree about the float ABI or the enum size:

embld: 'startup.o' and 'main-hf.o' disagree about where floating-point arguments go: one passes them in the core registers (-mfloat-abi=soft) and the other in s0-s15 (-mfloat-abi=hard). Linking them would leave every float argument read from a register the caller never wrote

That includes the runtime archive: link -eabihf objects with the -eabihf librt.a. The one built for thumbv7em-none-eabihf is the Cortex-M4F's, and links with Cortex-M7 objects -- the convention is the same, and the M7 runs every M4F instruction -- but an image for the M7 that converts a 64-bit integer then carries the whole of software binary64. lib/rt/softfp.c compiled with --cpu=cortex-m7 is the eight conversion routines alone.

Linking for a memory map#

Flash is -Ttext, RAM is -Tdata, and the flash size is --rom-limit. For an STM32F4-class part with 512 KiB of flash at 0x08000000 and SRAM at 0x20000000:

embld -e Reset_Handler -Ttext 0x08000000 -Tdata 0x20000000 --rom-limit 524288 \
      startup.o main.o librt.a -o fw.elf

Or with the linker script the part's project template provides, which also checks RAM: a script's MEMORY regions are enforced, and an image that does not fit stops with region RAM overflowed by N bytes.

embld -T STM32F407VGTx_FLASH.ld startup.o main.o librt.a -o fw.elf

The vector table's stack-pointer entry is the top of RAM. Without a script EmbLD has no RAM-size check, so keep .data, .bss and the stack inside the part's SRAM yourself (llvm-size fw.elf gives data and bss). -Tstack is a RISC-V option and is refused here:

embld: -Tstack is a RISC-V option: every other target here starts with a stack pointer already set (a Cortex-M reads its own from the vector table)

CMSIS and vendor files#

A Cortex-M project's vendor files build unmodified: ARM's CMSIS-Core headers (core_cm4.h, cmsis_gcc.h and their intrinsics), a vendor's device header and system_*.c, its GNU-syntax startup file, and a CubeMX-style linker script. CMSIS chooses its compiler support by __GNUC__, so compile the C with -fgnuc-version=4.2.1:

CF="-O2 -ICMSIS/Core/Include -Icmsis-device-f4/Include -DSTM32F405xx -fgnuc-version=4.2.1"
embcc --target=thumbv7em-none-eabi $CF -c system_stm32f4xx.c -o system.o
embcc --target=thumbv7em-none-eabi $CF -c main.c -o main.o
embcc --target=thumbv7em-none-eabi -c startup_stm32f405xx.s -o startup.o
embcc --target=thumbv7em-none-eabi --link-script=STM32F405RGTx_FLASH.ld \
      startup.o system.o main.o -o fw.elf
qemu-system-arm -M netduinoplus2 -nographic -kernel fw.elf

tests/golden/cmsis-stm32f4.sh builds that from ST's startup and system files as they ship, with a linker script in CubeMX's shape, and runs it on QEMU's netduinoplus2 (an STM32F405) with SysTick interrupts, when ARM's CMSIS_5 and ST's cmsis-device-f4 are cloned under ~/EmbRef.

Starting from the vendor's SVD#

A vendor that ships no CMSIS device pack still publishes an SVD file for its part. embsvd writes the device header, a startup file and a linker script from it:

embsvd STM32F405.svd --header STM32F405.h --startup startup.c \
       --ld STM32F405.ld --flash 0x08000000:1M --ram 0x20000000:128K

The header has CMSIS's shape -- USART1->CR1, USART1_IRQn, RCC_APB2ENR_USART1EN_Msk -- and includes CMSIS-Core for the SVD's core. embsvd --show USART1 prints the registers and their fields, and embsvd --json describes the whole device -- every register at its absolute address, with its fields -- for tools rather than compilers. tests/golden/svd-stm32f405.sh checks the generated layout against ST's own header, and runs a firmware built from the generated files; tests/golden/svd-clusters.sh checks clusters, arrays and the JSON.

Stopping and printing under a debugger or emulator#

bkpt is in the inline-assembly vocabulary, so ARM semihosting works. With QEMU's -semihosting, operation 0x04 (SYS_WRITE0) prints a string on the host and operation 0x20 (SYS_EXIT_EXTENDED) ends the emulator with an exit status:

static int semihost(int op, const void *arg)
{
    int r;
    __asm__ volatile("mov r0, %1\n\tmov r1, %2\n\tbkpt #0xab\n\tmov %0, r0"
                     : "=r"(r) : "r"(op), "r"(arg) : "r0", "r1", "memory");
    return r;
}

void exit_qemu(int code)
{
    static unsigned block[2];
    block[0] = 0x20026;            /* ADP_Stopped_ApplicationExit */
    block[1] = (unsigned)code;
    semihost(0x20, block);
}

On real hardware a bkpt with no debugger attached escalates to a HardFault; remove semihosting calls from production images.

An RTOS kernel#

What a preemptive kernel on Cortex-M asks of its compiler, and where EmbCC provides it:

FreeRTOS as the worked example#

FreeRTOS's kernel and its GCC Cortex-M ports compile unmodified, and run on QEMU at --opt=none, --opt=basic, --opt=speed and --opt=size (tasks, a queue, a mutex, a software timer and time slicing; on the FPU parts, two tasks doing floating-point arithmetic through every switch):

The files the Cortex-M3 test uses, in tests/golden/freertos-cm3/, are a starting point for a board of your own (tests/golden/freertos-ports/ has the same for the MPS2 boards):

• FreeRTOSConfig.h, whose interrupt priorities must match the part: configPRIO_BITS is 3 on the LM3S6965 and 4 on most STM32 parts, and configMAX_SYSCALL_INTERRUPT_PRIORITY is shifted by 8 - configPRIO_BITS;
• startup.c, whose vector table sends SVC to vPortSVCHandler, PendSV to xPortPendSVHandler and SysTick to xPortSysTickHandler;
• lm3s.ld, the memory map.

With K the FreeRTOS-Kernel directory and the three files above in board/:

CF="--target=thumbv7m-none-eabi -Os -Iboard -I$K/include -I$K/portable/GCC/ARM_CM3"
for f in board/startup.c main.c $K/tasks.c $K/queue.c $K/list.c $K/timers.c \
         $K/portable/MemMang/heap_4.c $K/portable/GCC/ARM_CM3/port.c; do
    embcc $CF -c "$f" -o "$(basename "$f" .c).o"
done
embcc --target=thumbv7m-none-eabi --link-script=board/lm3s.ld *.o -o fw.elf
qemu-system-arm -M lm3s6965evb -cpu cortex-m3 -nographic -kernel fw.elf

For a Cortex-M4F, use --target=thumbv7em-none-eabihf --fpu=fpv4-sp-d16 and the ARM_CM4F port, whose PendSV handler also saves s16-s31; the startup code must enable the FPU (CPACR) before main when anything before the scheduler may use it. The ARMv8-M ports name their handlers as CMSIS does (SVC_Handler, PendSV_Handler, SysTick_Handler), and want configENABLE_FPU, configENABLE_MPU, configENABLE_TRUSTZONE and configRUN_FREERTOS_SECURE_ONLY set in FreeRTOSConfig.h.

The RISC-V port is port.c and portASM.S, compiled with -I naming its chip-specific directory (chip_specific_extensions/RISCV_MTIME_CLINT_no_extensions for a CLINT timer). Its FreeRTOSConfig.h gives the timer's addresses (configMTIME_BASE_ADDRESS, configMTIMECMP_BASE_ADDRESS; on QEMU's virt 0x0200BFF8 and 0x02004000), and configCPU_CLOCK_HZ is the timer's rate, 10 MHz there. The startup points mtvec at freertos_risc_v_trap_handler before calling main (see tests/golden/freertos-ports/riscv-start.S).

Under QEMU, add -icount shift=2 when the program measures time: without it SysTick follows the host's clock, and a loaded host delivers fewer ticks than the guest's code expects.

Writing your own#

The port's three hard parts are the same for any kernel:

• The context switch. A naked PendSV handler saves r4-r11 (and, with an FPU, s16-s31 when lr says the task used it) on the process stack, stores the stack pointer in the current task's control block, calls the scheduler with interrupts masked, and restores the next task the same way. Nothing in a naked function may need a frame: only basic asm is allowed in its body, and EmbCC refuses anything else by name.

• Critical sections. Raise BASEPRI rather than masking everything, so that interrupts above the kernel's priority still run:

  static inline __attribute__((always_inline)) unsigned raise_basepri(void)
  {
      unsigned old, pri = 5 << (8 - 3);
      __asm__ volatile("mrs %0, basepri\n\tmsr basepri, %1\n\tisb\n\tdsb"
                       : "=r"(old) : "r"(pri) : "memory");
      return old;
  }
  

  The "memory" clobber is what keeps the compiler from moving loads and stores of shared data across the boundary.

• Shared data. A flag or counter an interrupt writes is volatile; a structure the kernel protects with a critical section is not, and must not need to be.

RISC-V#

The target#

riscv32-unknown-elf and riscv64-unknown-elf generate code for the RV32IMAC and RV64IMAC instruction sets with the soft-float ABIs (ilp32 and lp64) and the medany code model by default. --isa= and --abi= select the F and D extensions and their calling conventions -- --isa=rv32imafc --abi=ilp32f for an ESP32-P4 or CH32V3 class part, --isa=rv64gc --abi=lp64d -- or leave out C; M and A are always used, so EmbCC's output does not run on a core without them. A program built for F or D must turn the FPU on before its first floating-point instruction (mstatus.FS, which is 0 at reset: csrs mstatus, 0x2000 in machine mode); tests/harness/riscv/boot.c does. The predefined macros say what was chosen (__riscv_a, __riscv_c, __riscv_compressed, __riscv_flen, __riscv_float_abi_single, __riscv_cmodel_medany, ...).

Plain char is unsigned. long double is 16 bytes (binary128) and has no arithmetic: any operation on one, like any 128-bit integer on RV64, is refused with the RV32 backend cannot lower a 128-bit value yet (RV64 on the other width). The full data model is in Targets.

Boot#

A RISC-V core starts with every register zero and has no vector table to load a stack pointer from, and C cannot set sp. EmbLD supplies the missing instructions: -Tstack ADDR places a stub at the start of the image that loads ADDR into sp and jumps to the -e symbol, and makes the stub the ELF entry point. Everything after that is C, and the core stays in machine mode.

/* boot.c */
extern unsigned __data_load, __data_start, __data_end;
extern unsigned __bss_start, __bss_end;
int main(void);

void _start(void)
{
    unsigned *d = &__data_start, *s = &__data_load;
    while (d < &__data_end)
        *d++ = *s++;
    for (d = &__bss_start; d < &__bss_end; )
        *d++ = 0;
    main();
    for (;;)
        ;
}

embcc --target=riscv32-unknown-elf --opt=size -c boot.c -o boot.o
embld -e _start -Ttext 0x80000000 -Tstack 0x80800000 boot.o main.o \
      build/libc/riscv32-unknown-elf/librt.a -o fw.elf

An image loaded wholly into RAM, as on QEMU's virt board, needs no -Tdata: the bracket symbols are still defined, __data_load equals __data_start, and the copy loop moves nothing. A part that executes from flash uses -Ttext FLASH -Tdata RAM as on Cortex-M.

Interrupt handlers#

__attribute__((interrupt)) makes a C function a machine-mode interrupt handler, as with GCC and clang; interrupt("machine") is the same, and interrupt("supervisor") makes a supervisor-mode one:

#define CLINT_MTIMECMP_LO (*(volatile unsigned *)0x2004000u)
#define CLINT_MTIMECMP_HI (*(volatile unsigned *)0x2004004u)
#define CLINT_MTIME_LO    (*(volatile unsigned *)0x200bff8u)

volatile unsigned ticks;

__attribute__((interrupt)) void timer_isr(void)
{
    ticks++;
    CLINT_MTIMECMP_HI = 0xffffffffu;               /* re-arm: no early match */
    CLINT_MTIMECMP_LO = CLINT_MTIME_LO + 10000;
    CLINT_MTIMECMP_HI = 0;
}

void timer_start(void)
{
    __asm__ volatile("csrw mtvec, %0" :: "r"(timer_isr));
    CLINT_MTIMECMP_LO = CLINT_MTIME_LO + 10000;
    CLINT_MTIMECMP_HI = 0;
    __asm__ volatile("csrs mie, %0" :: "r"(1ul << 7));       /* MTIE */
    __asm__ volatile("csrs mstatus, %0" :: "r"(1ul << 3));   /* MIE */
}

The handler saves and restores every caller-saved register it writes -- ra, t0-t6, a0-a7 -- and, when it calls any function (a runtime helper such as a soft-float routine counts), all of them; with an FPU, the floating-point registers a call may clobber are included on the same terms, at the full register width: ft0-ft11 and fa0-fa7, and fs0-fs11 too where the ABI keeps fewer bits of them than the registers hold (ilp32f on a part with D, or a soft-float ABI on a part with an FPU). This is clang's rule, register for register. It returns with mret, or sret for a supervisor-mode handler, and is placed at a 4-byte boundary even with the C extension, because mtvec's and stvec's low two bits select the mode. fcsr is not saved, as by GCC and clang. A handler makes no tail call.

A handler takes no parameters and returns void; anything else is refused, as is interrupt("user") (user-mode interrupts were never ratified) and any other argument:

embcc: isr.c:3: error: interrupt handler 'isr' takes parameters: the hardware calls it, so nothing passes them, and they would be read out of whatever the interrupted code left in the argument registers
embcc: isr.c:3: error: __attribute__((interrupt("user"))) is not supported: user-mode interrupts (the N extension and its uret) were never ratified and are gone from the privileged spec, and GCC and clang no longer accept them

The attribute may be on a prototype, the definition, or both; two different kinds on one function are refused. In C++ it is refused (not supported in C++ yet): write the handler in C.

tests/golden/riscv-isr.sh runs a leaf handler, a calling handler, one with a frame larger than 2 KiB and a supervisor-mode handler (a delegated software interrupt the machine timer raises) against code that keeps a value in every caller-saved register, at RV32 and RV64, soft and hard float.

Trap handlers#

A handler that needs the trapped state -- a synchronous exception whose mepc must be advanced, or a context switch that saves the interrupted registers into a task's frame -- is an entry written in assembly. It saves the caller-saved registers, calls a C function, restores them and returns with mret. EmbCC assembles .S files for RISC-V itself, and CSRs may be named (csrr a0, mcause) or numbered (csrr a0, 0x342), in a .S file as in inline assembly.

/* trap.S -- machine-mode trap entry, RV32 */
	.text
	.globl	trap_entry
	.p2align 2
trap_entry:
	addi	sp, sp, -64
	sw	ra, 0(sp)
	sw	t0, 4(sp)
	sw	t1, 8(sp)
	sw	t2, 12(sp)
	sw	a0, 16(sp)
	sw	a1, 20(sp)
	sw	a2, 24(sp)
	sw	a3, 28(sp)
	sw	a4, 32(sp)
	sw	a5, 36(sp)
	sw	a6, 40(sp)
	sw	a7, 44(sp)
	sw	t3, 48(sp)
	sw	t4, 52(sp)
	sw	t5, 56(sp)
	sw	t6, 60(sp)
	csrr	a0, 0x342		/* mcause */
	csrr	a1, 0x341		/* mepc */
	call	trap_handler
	lw	ra, 0(sp)
	lw	t0, 4(sp)
	lw	t1, 8(sp)
	lw	t2, 12(sp)
	lw	a0, 16(sp)
	lw	a1, 20(sp)
	lw	a2, 24(sp)
	lw	a3, 28(sp)
	lw	a4, 32(sp)
	lw	a5, 36(sp)
	lw	a6, 40(sp)
	lw	a7, 44(sp)
	lw	t3, 48(sp)
	lw	t4, 52(sp)
	lw	t5, 56(sp)
	lw	t6, 60(sp)
	addi	sp, sp, 64
	mret

.p2align 2 asks for 4-byte alignment, which mtvec's direct mode requires; .align 2 means the same, since EmbCC's assembler reads .align N as 2^N bytes on every target it assembles files for, as GNU as does (.balign N is a byte count). For RV64, store and load with sd/ld in 8-byte slots.

The C side installs the entry in mtvec and enables interrupts with inline assembly:

extern void trap_entry(void);

#define CLINT_MTIMECMP (*(volatile unsigned long long *)0x2004000u)
#define CLINT_MTIME    (*(volatile unsigned long long *)0x200bff8u)

volatile unsigned ticks;

void trap_handler(unsigned long cause, unsigned long epc)
{
    (void)epc;
    if (cause == 0x80000007ul) {               /* machine timer interrupt */
        ticks++;
        CLINT_MTIMECMP = CLINT_MTIME + 10000;
    }
}

void timer_start(void)
{
    __asm__ volatile("csrw mtvec, %0" :: "r"(trap_entry));
    CLINT_MTIMECMP = CLINT_MTIME + 10000;
    __asm__ volatile("csrs mie, %0" :: "r"(1ul << 7));       /* MTIE */
    __asm__ volatile("csrs mstatus, %0" :: "r"(1ul << 3));   /* MIE */
}

The CLINT addresses are those of QEMU's virt board. For a synchronous exception, the handler must advance mepc past the faulting instruction (csrw mepc) before returning, or the instruction runs again.

Atomics: the A extension#

Atomic operations up to the register width are inline A-extension instructions: atomic_fetch_add on an int is one amoadd.w.aqrl, a compare-exchange is an lr.w.aqrl/sc.w.rl loop, and atomic_thread_fence is fence rw, rw. No library is involved. Those are the seq_cst forms; an explicit memory order sets the .aq and .rl bits as clang's does (atomic_fetch_add_explicit(p, 1, memory_order_relaxed) is a bare amoadd.w, an acquire compare-exchange lr.w.aq/sc.w).

Without the A extension (--isa=rv32imc, an ESP32-C3) there is no atomic read-modify-write instruction, so each one is a call to libatomic's sized routine, as GCC and clang make it there: __atomic_fetch_add_4, __atomic_compare_exchange_1 and the rest. An aligned load or store is one instruction either way, and stays inline. EmbCC's runtime defines the routines (lib/rt/atomic-riscv.c) by masking interrupts: mstatus.MIE is cleared and read in one csrrci, and set again only if it was set, so they nest. That makes them atomic on one core with no other bus master, which is what such a part is. They need machine mode, and they are not atomic against a second core or DMA, so every one is weak, and an SDK or RTOS with another way replaces them by defining its own. The C library's locks are atomics too, so a program for such a part links the runtime and C library built without A, from build/libc/<triple>/noa; the driver picks them from --isa=.

A one- or two-byte atomic works on the aligned word around it, as GCC's and LLVM's do: AND, OR and XOR are one AMO with the other lanes neutral, and the rest an lr.w/sc.w loop that rewrites only its lane. That is atomic against the neighbouring bytes too, because a write to any of them breaks the reservation and the loop runs again; a fetch-and-add on a byte is

    andi  t4, a0, -4        # the aligned word
    andi  t2, a0, 3
    slli  t2, t2, 3         # the lane's shift
    li    t5, 255
    sll   t5, t5, t2        # the lane's mask
    sll   t1, a1, t2        # the operand, in the lane
1:  lr.w.aqrl t0, (t4)
    add   t6, t0, t1
    xor   t6, t6, t0
    and   t6, t6, t5
    xor   t6, t6, t0        # old ^ ((new ^ old) & mask)
    sc.w.rl t6, t6, (t4)
    bnez  t6, 1b

which is clang's sequence instruction for instruction. A 64-bit atomic on RV32 is a call to __atomic_*_8, which lib/rt implements with mstatus.MIE clear; see Eight-byte atomics on a 32-bit target.

The C extension#

Compressed (16-bit) instructions are always emitted, and every object carries EF_RISCV_RVC in its ELF header. EmbLD does not relax: it ignores R_RISCV_RELAX and R_RISCV_ALIGN and keeps every sequence at the length the compiler or assembler chose.

With the C extension, functions are aligned to 2 bytes, and __attribute__((aligned(4))) on a function does not change that. An interrupt handler is the exception: it is 4-byte aligned, as a direct-mode mtvec target must be. Any other address that must be 4-byte aligned belongs in an assembly file with .p2align 2, as in the trap entry above.

Runtime#

riscv32-unknown-elf has a librt.a with the 64-bit division routines, software float and double, and complex multiply and divide. RV64 divides 64-bit integers in hardware and needs only the floating-point and complex routines, which have to be compiled by hand (see The runtime archive).

MIPS32#

The target#

mipsel-none-elf generates MIPS32 Release 2 code, little-endian, with the o32 ABI and soft float: the configuration of a PIC32MX or PIC32MZ part's core used without its FPU. The code is not abicalls code and puts nothing in small data: addresses are built with lui/addiu, calls are jal, and $gp is never read. Plain char is signed, long double is double, and a misaligned word access is avoided rather than trapped: a packed member is read with lwl/lwr. The options and the full data model are in Targets.

Boot#

As on RISC-V, C cannot set sp, and embld -Tstack ADDR puts a stub at the start of the image that loads ADDR into sp and jumps to the -e symbol through t9. The startup is then C:

embcc --target=mipsel-none-elf --opt=size -c boot.c -o boot.o
embld -e _start -Ttext 0x80100000 -Tstack 0x80800000 boot.o main.o \
      build/libc/mipsel-none-elf/libc.a build/libc/mipsel-none-elf/librt.a \
      -o fw.elf

tests/harness/mips/boot.c is a complete startup for QEMU's malta board: it copies .data, zeroes .bss, runs the static constructors and calls main. Its image is linked in KSEG0 (cached, unmapped), so .data needs no -Tdata. A PIC32 runs from flash in KSEG1 or KSEG0 and keeps its data in RAM: link it with -Ttext at the flash address and -Tdata at the RAM address, as on Cortex-M. A jal keeps the caller's top four address bits, so a call must stay in its 256 MiB region: code in the boot flash (KSEG1, 0xbfc00000) calling program flash in KSEG0 (0x9d000000) must go through a function pointer, and EmbLD refuses a jal that does not reach (a jal or j at ADDR to 'f' at ADDR, which is in another 256 MiB region).

Interrupt handlers#

__attribute__((interrupt)) makes a C function an interrupt handler, as GCC defines it for MIPS32 (clang's is the same):

The handler saves EPC and Status (a nested interrupt overwrites both), writes Status with EXL, ERL and KSU cleared as the table says, saves every caller-saved register it writes -- $at, v0-v1, a0-a3, t0-t9, ra -- and HI/LO if it multiplies or divides, and all of them and gp when it calls any function (a soft-float or 64-bit division helper counts). It ends with di; ehb (unless masked), puts EPC and Status back and returns with eret. k0 and k1 are the only registers it uses unsaved, as the ABI reserves them for this.

A PIC32, MX or MZ, has an EIC: its interrupt controller hands the requested priority to the core in Cause.RIPL (Config3.VEIC is set), so the plain form is the one to use there, as XC32's __ISR is. On a core without one -- malta's 24Kc -- Cause[15:10] are the pending interrupt lines, so the plain form unmasks the very line that is pending and re-enters at once, as GCC's does: use vector=hwN (hw5 for the CP0 timer on line 7) or keep_interrupts_masked there.

__attribute__((interrupt("vector=hw5"))) void timer_isr(void)
{
    unsigned c;
    ticks++;
    __asm__ volatile("mfc0 %0, $9" : "=r"(c));             /* Count */
    __asm__ volatile("mtc0 %0, $11" :: "r"(c + PERIOD));   /* Compare: ack */
}

The handler is reached from the vector through k0: four words at EBase + 0x180 (or at EBase + 0x200 + n * spacing with vectored interrupts, Cause.IV) that load its address and jr to it, as the exc_stub below. A handler takes no parameters and returns void; use_shadow_register_set and use_debug_exception_return are refused, as is the attribute in C++ and on MIPS64. tests/golden/mips-isr.sh runs a masked, a calling and a nested pair of handlers against code that keeps a value in every caller-saved register and HI/LO, in both byte orders.

Exceptions#

A handler for a synchronous exception (syscall, an address error) that must read or edit the trapped registers is an entry written in a .S file (or a naked function, or a file-scope asm block) that saves the registers C may change, calls a C function, restores them and returns with eret. tests/golden/mips-exc/vector.S is a complete one, run on malta by tests/golden/mips-exc.sh, and its shape is:

/* vector.S */
#define FRAME (16 + 4 * 24)
    .text
    .set noat                       /* $at is saved by hand */

    .globl exc_stub, exc_stub_end   /* copied to EBase + 0x180 */
    .set push
    .set noreorder
exc_stub:
    lui     $k0, %hi(exc_entry)
    addiu   $k0, $k0, %lo(exc_entry)
    jr      $k0
    nop
exc_stub_end:
    .set pop

exc_entry:
    addiu   $sp, $sp, -FRAME
    sw      $at, 16($sp)            /* ...and v0-v1, a0-a3, t0-t9, ra */
    mfhi    $t0                     /* ...and HI, LO */
    sw      $t0, 88($sp)
    mfc0    $a0, $13                /* Cause */
    mfc0    $a1, $14                /* EPC */
    addiu   $a2, $sp, 16            /* the saved words, for C to edit */
    jal     exc_c                   /* returns where to resume */
    mtc0    $v0, $14
    ehb
    lw      $t0, 88($sp)            /* restore in reverse */
    mthi    $t0
    lw      $at, 16($sp)
    addiu   $sp, $sp, FRAME
    eret

The file is in .set reorder mode outside the stub, so the assembler puts a nop after jal and jr; the stub is in .set noreorder so it is exactly four words. The C side copies the stub to the vector and clears Status.BEV, ERL and EXL (tests/harness/mips/boot.c does the same for its fault report):

extern char exc_stub[], exc_stub_end[];

unsigned exc_c(unsigned cause, unsigned epc, unsigned *frame)
{
    switch ((cause >> 2) & 31) {
    case 8:                                 /* syscall: resume after it */
        frame[1] = handle_syscall(frame[3]);    /* v0 from a0 */
        return epc + 4;
    case 0:                                 /* an interrupt: resume at it */
        if (cause & (1u << 15))             /* IP7, the CP0 timer */
            timer_tick();                   /* writes Compare: the ack */
        return epc;
    }
    return epc + 4;
}

void install(void)
{
    volatile unsigned *vec = (volatile unsigned *)0x80000180u;
    for (unsigned k = 0; k < (unsigned)(exc_stub_end - exc_stub) / 4; k++)
        vec[k] = ((unsigned *)exc_stub)[k];
}

A real core caches the vector's words: write them back from the data cache and invalidate the instruction cache (cache, or synci) before the first exception; QEMU models no caches. Interrupts are enabled and disabled with inline ei, di and ehb, and CP0 registers are read and written with mfc0 and mtc0; see MIPS32 in Inline Assembly. cache and synci are not in the assembler's vocabulary yet.

Atomics#

4-byte atomic read-modify-write operations are inline ll/sc loops bracketed by sync. An atomic load of 4 bytes or fewer is one lw (lhu, lbu) followed by sync, and an atomic store one sw (sh, sb) with a sync before and after: never the two-instruction lwl/lwr form a packed member gets, which an interrupt could split. A 1- or 2-byte read-modify-write is an ll/sc loop on the aligned word around it that rewrites only its lane. An 8-byte atomic is a call to __atomic_*_8, which lib/rt implements with di/ei; see Eight-byte atomics on a 32-bit target. __GCC_HAVE_SYNC_COMPARE_AND_SWAP_1, _2 and _4 are defined, and _8 is not.

Runtime#

make rt-embedded libc-embedded builds librt.a (the 64-bit division routines, soft float and double, complex multiply and divide) and libc.a (EmbCC's C library on its bare-metal backend: write is a weak function the program overrides to reach its UART) for mipsel-none-elf.

AVR (ATmega328P)#

The target and its data model#

--target=avr generates code for the ATmega328P (__AVR_ATmega328P__, AVR architecture 5). --mcu=atmega328p names it, and --mcu=atmega328, atmega168p and atmega168 the other parts of its family with the same core (Targets lists their memories); any other part is refused by name.

double is binary32, as in avr-gcc's default configuration: DBL_DIG is 6 and DBL_MAX is about 3.4e38.

Memory map and limits#

Link with -Ttext 0x0 -Tdata 0x100 --rom-limit 32768. Program space and data space are separate on this machine, and the instructions EmbCC emits for a pointer dereference read data space only. EmbLD therefore puts .rodata in the writable segment on AVR: string literals and const data are stored in flash after the text, copied to SRAM by the startup, and occupy SRAM for the life of the program. .data, .rodata, .bss and the stack all share the 2 KiB; --stack-usage and llvm-size are how to keep track. --rom-limit counts the stored copy of .data and .rodata as well as the text.

Startup and the interrupt vector table#

With -mmcu=, the driver links what avr-gcc's does for the part:

embcc --target=avr --mcu=atmega328p --opt=size main.c -o fw.elf

• The startup, crt<part>.o (lib/avr/crt.S), first, unless --no-start-files or --no-stdlib. It has avr-libc's and libgcc's names and sections, so a program written for avr-gcc, and its own .S files, find what they expect:
  • .vectors: __vectors at address 0, the part's 26-entry table, one jmp each. Entry 0 jumps to __init; entry n to __vector_n, which is a weak alias of __bad_interrupt, so an ISR() replaces it and any other vector goes to __bad_interrupt. That jumps to __vector_default: the program's ISR(BADISR_vect), or else address 0, a restart.
  • .init0 __init; .init2 clears r1 and SREG and sets the stack pointer to __stack (weak, RAMEND unless the link defines it); .init4 __do_copy_data (.data and .rodata from their copy in flash, __data_load_start to __data_start..__data_end) and __do_clear_bss (__bss_start..__bss_end); .init6 __do_global_ctors, each function in .init_array (__attribute__((constructor))) when the script brackets it with __init_array_start/__init_array_end; .init9 calls main and then exit.
  • .fini9 _exit, with exit a weak name for it, as avr-libc's; .fini0 disables interrupts and stops in a loop.
  • A program adds a step to the sequence with code in .init1, .init3, .init5, .init7, .init8 or .fini8...fini1, without a ret: the sections run into each other. In a .S file, or in C as avr-libc writes it: a naked function placed in the section, __attribute__((naked, used, section(".init3"))), whose body is its asm and nothing after it.
• The linker script, when the link names none: <part>.ld (lib/avr/avr5.ld with the part's memories), avr-ld's layout: flash from 0 with the vectors, PROGMEM data, .init0...init9, the code and .fini9...fini0; the data space from 0x800000 in the linker's view, with .data, .rodata and .init_array stored after the code and copied, then .bss and .noinit; .eeprom from 0x810000.
• -Tdata 0x800100, SRAM's start, unless the link gives -Tdata: avr-gcc's device specs pass it beside any script, and the default avr5 script and the scripts derived from it rely on it (their data region starts at 0x800060, under the ATmega328P's extended I/O).

A script of the program's own works as with avr-gcc (--link-script=arduino_uno.ld); -Wl,-Ttext=, -Wl,-Tdata= and -Wl,--rom-limit= apply over it. tests/golden/avr-crt.sh links one program by the part's script, by an avr5-style one, and by that with the flags, and runs all three on QEMU's ATmega328P.

Without --mcu=, nothing is added: the program brings its startup, as tests/harness/avr/boot.S is, and its memory map:

embcc --target=avr -c tests/harness/avr/boot.S -o boot.o
embcc --target=avr --opt=size -c main.c -o main.o
embld -e __vectors -Ttext 0x0 -Tdata 0x100 --rom-limit 32768 \
      boot.o main.o build/libc/avr/librt.a -o fw.elf

The entry symbol must be at address 0: QEMU refuses an AVR image whose ELF entry point is anything else, and the part starts there regardless.

Interrupt handlers#

Both of avr-gcc's attributes are implemented, and the handler names are avr-libc's, so __vector_13 lands in vector 13 of a table like the one above:

volatile unsigned char ticks;

/* TIMER1_OVF is vector 13 on the ATmega328P. */
__attribute__((signal)) void __vector_13(void) { ticks++; }

A handler saves r0, SREG, r1, every call-clobbered register (r18-r27, r30, r31) and the frame pointer, clears r1 for its own body, and returns with reti. signal runs the body with interrupts disabled. interrupt re-enables them with sei at entry, so the handler can be interrupted. Interrupts are enabled globally with sei() (<avr/interrupt.h>) or __builtin_avr_sei(), and disabled with cli() or __builtin_avr_cli(); avr-gcc's other __builtin_avr_* are there too (Extensions).

avr-libc-compatible headers#

EmbCC has its own headers with avr-libc's interface, so code written for avr-gcc and avr-libc compiles unchanged. They are in include/avr and on the include path for --target=avr alone (installed under freestanding/avr); another target does not find <avr/io.h>.

The text is EmbCC's own; the names and the addresses are the part's. tests/golden/avr-io.sh checks every register address and bit number against a table taken from the datasheet, for each of the four parts.

Where they differ from avr-libc's:

• _SFR_MEM_ADDR(PORTB) and _SFR_IO_ADDR(PORTB) are constants where the code needs one -- an asm "I" or "n" operand, a static initializer -- but not in an integer constant expression: a case label or a _Static_assert refuses them.
• The timed sequences (wdt_enable, wdt_disable, sleep_bod_disable) are single asm statements, so they hold at every optimisation level.
• _delay_ms and _delay_us count loops of four cycles (_delay_loop_2) and wait at least the time asked. With a constant argument the count is worked out at compile time from --opt=basic up; at --opt=none it is computed at run time in software floating point, which makes the delay longer. For a delay of exactly n cycles, which avr-libc's _delay_ms builds on, call __builtin_avr_delay_cycles(n).
• <avr/io.h> in a .S file gives each register as its data-space address (sts UDR0, r24, out _SFR_IO_ADDR(PORTB), r24), as avr-libc's does; EmbCC defines __ASSEMBLER__ for a .S file as GCC does.
• Not there: the _far reads (no part here has more than 64 KiB of flash), <util/atomic.h> (its ATOMIC_BLOCK needs the cleanup attribute, which EmbCC refuses), <avr/boot.h>, <avr/power.h>, <util/setbaud.h>, <util/crc16.h>, <util/twi.h>, <avr/fuse.h>, <avr/lock.h>.

Data in program memory#

__flash keeps const data in program memory instead of copying it to SRAM at start-up, as avr-gcc's does. On a part with 2 KiB of SRAM, every table that stays in flash is RAM the program gets back.

static const __flash char banner[] = "ready\n";
static const __flash unsigned char crc8_table[256] = { 0x00, 0x07, /* ... */ };

static void puts_P(const __flash char *s)
{
    char c;
    while ((c = *s++) != 0)
        uart_putc(c);
}

A read through a __flash lvalue compiles to lpm through Z. That includes an element of a __flash array, a member of a __flash struct, and *p for a const __flash T *p. The object goes to .progmem.data. EmbLD places that in flash with the code, and so do avr-libc's linker scripts (*(.progmem*) in .text).

The rules are GCC's, and each is refused by name:

• A __flash object must be const and have static storage: at file scope, or static in a function. Program memory is written when the part is flashed, not by the program.
• A store through a __flash lvalue is refused.
• A __flash pointer and a generic pointer are in different address spaces. Converting one to the other needs a cast: a generic pointer to a flash address would read SRAM at the same number.
• A whole __flash struct cannot be copied yet (struct pin p = pins[i];). Read its members instead.

__flash may stand wherever const can, before or after the type, and after a * for a pointer that is itself in flash (const __flash char *const __flash names[]). __memx is not supported.

__attribute__((progmem)), avr-libc's PROGMEM, also puts an object in .progmem.data, but leaves its type alone: the object is an ordinary const one to the compiler, and the program reads it with pgm_read_byte() and the other <avr/pgmspace.h> functions, which use lpm. A plain *p of such an object reads SRAM at the same number, as with avr-gcc. As avr-gcc does, progmem wins over a section() on the same declaration, and the object must be const and static ('x' is in program memory (progmem, section .progmem.data) and must be const ..., progmem on 'x', which is on the stack ...). On other targets progmem is an unknown attribute, ignored with a -Wattributes warning, as GCC does there.

The calling convention#

EmbCC follows avr-gcc's convention, so EmbCC and avr-gcc objects call each other:

• Arguments are assigned from r25 downwards, each rounded up to an even number of registers, down to r8. Once one argument does not fit, it and every later argument go on the stack, packed at their natural size.
• All arguments of a call to a variadic function go on the stack, the named ones included.
• A value is returned in r24 (1 or 2 bytes), r22-r25 (up to 4) or r18-r25 (up to 8), its size rounded up to a power of two. A larger struct is returned through a pointer the caller passes in r24:r25.
• r0 is a scratch register, r1 is always zero, r2-r17 and r28-r29 are preserved by the callee, and r18-r27, r30-r31 are not.

The AVR library#

make rt-embedded builds build/libc/avr/libc.a from lib/avr, and the driver links it for --target=avr as it links libc.a elsewhere. It is not lib/libc: that library's formatter prints every double exactly, in 17 KB of AVR code, and its streams keep 8 KB of buffers. What it has, each function an archive member of its own, so an image carries only what it calls:

• memcpy, memmove, memset, memcmp, memchr, strlen, strnlen, strcpy, strncpy, strcat, strncat, strcmp, strncmp, strchr, strrchr, strstr, strspn, strcspn, strpbrk;
• the _P functions <avr/pgmspace.h> declares;
• sprintf, snprintf, vsprintf, vsnprintf and avr-libc's sprintf_P, snprintf_P, vsprintf_P, vsnprintf_P, whose format is in flash (snprintf_P(buf, n, PSTR("%u"), x); <stdio.h> declares them for AVR). The conversions are avr-libc's default ones: flags, width, precision, hh h l ll z t j, d i u o x X c s p n % and %S, a string in flash. A floating-point conversion prints ?, as avr-libc's does without -lprintf_flt; EmbCC has no printf_flt;
• the <avr/eeprom.h> functions;
• setjmp and longjmp, with a 24-byte jmp_buf: r2-r17, Y, the stack pointer, SREG and the return address.

Not there: streams (printf, puts, FILE, fdevopen), malloc, <stdlib.h>'s conversions and <ctype.h>'s functions. Their declarations come from the C library's headers, and a program that calls one gets an undefined symbol at the link.

--warn=format compares a printf argument with the conversion by the target's own sizes: on AVR %d reads 2 bytes and %ld 4.

Runtime helpers#

EmbCC does not use the part's mul instruction, and the part has no divide, so every integer multiply, divide and remainder is a call: __mulsi3, __divsi3, __udivsi3, __modsi3, __umodsi3 for 8-, 16- and 32-bit operands, and __muldi3, __divdi3, __udivdi3, __moddi3, __umoddi3 for 64-bit. A quotient and a remainder of the same operands are two calls. The 64-bit __udivdi3 and __umoddi3 are assembly, about 2500 cycles a division (the signed ones call them). Every floating-point operation is a call into the binary32 routines (__addsf3, __mulsf3, __divsf3, __ltsf2, __fixsfsi, ...). EmbCC does not call avr-gcc's __divmodsi4 family, and its librt.a does not define it.

The stack#

Variable-length arrays, alloca and locals with an alignment move the stack pointer below the frame, which stays at Y. A call made after one copies its stack arguments down to the new stack pointer, where the callee looks for them, and the stack pointer is restored at the end of a VLA's scope and at the return.

Atomics of one, two, four and eight bytes compile: every read-modify-write, compare-exchange and test-and-set, the _Atomic operators, and a load or store of two to eight bytes. Each is done with interrupts masked, as avr-libc's ATOMIC_BLOCK does it: SREG is saved, cli, the access, SREG restored -- which puts the I flag back as it was, so an atomic inside an interrupt handler or a critical section does not turn interrupts on. On one core that is all the atomicity there is to have, and no library is called at any size (clang calls __sync_fetch_and_add_8 and the like for four and eight bytes). A load reads and writes nothing back; a one-byte load or store is a single instruction. Fences compile to nothing.

x86-64 kernels and EmbLinkOS#

x86_64-elf is the freestanding x86-64 target and the default when no --target= is given. x86_64-emblink is the same code generator with __EmbLinkOS__, __emblink__ and __emblink predefined, for EmbLinkOS userland.

Code generation options for a kernel#

Most kernel code needs no runtime routines. __int128 multiply, divide and shifts, conversions between __int128 and floating point, and _Complex multiply and divide are calls (__multi3, __udivti3, __ashlti3, __floattidf, __muldc3, ...). No librt.a is built for x86_64-elf; link libgcc, or compile the needed files of lib/rt for the target (see Libraries).

__attribute__((interrupt)) is refused on x86-64. Interrupt and exception entry stubs are written in assembly and call C handlers. EmbCC assembles NASM-syntax .asm files (embcc -c isr.asm -o isr.o, or the standalone embas); GNU-syntax .s/.S files are refused on x86-64.

section .text
global isr_stub
extern isr_handler
isr_stub:
    push rax
    push rcx
    call isr_handler
    pop rcx
    pop rax
    iretq

Linking a kernel#

EmbLD links a kernel without a linker script:

embld -e _start -Ttext 0xFFFFFFFF80100000 -o kernel.elf *.o

--embx writes an EmbLinkOS EMBX executable instead of ELF, with the capabilities given by --cap NAME. See EmbLD.

The test harness's x86-64 kernel (tests/harness/x86_64) shows the boot path QEMU's -kernel needs: a Multiboot header, the climb from 32-bit protected mode to long mode, SSE enabled in CR0/CR4 before any C that uses it, and .bss zeroed. It is assembled and linked with a GNU cross toolchain; only the program under test comes from EmbCC.

C++ on the embedded targets#

C++ is compiled without exceptions, with or without RTTI, for the 32-bit embedded targets: the Cortex-M triples and armv7a-none-eabi, riscv32-unknown-elf, MIPS, PowerPC, SPARC, ColdFire, Xtensa, TriCore and RX (C++ targets has the exact list). A firmware build looks like this:

embcc --target=thumbv7em-none-eabihf --opt=size --no-exceptions --no-rtti \
      -c driver.cc -o driver.o
embld -e reset -Ttext 0x0 -Tdata 0x20000000 boot.o driver.o \
      build/libcxx/thumbv7em-none-eabihf/libcxx.a \
      build/libc/thumbv7em-none-eabihf/libc.a \
      build/libc/thumbv7em-none-eabihf/librt.a -o image.elf

The startup code must call the functions between __init_array_start and __init_array_end before main: that is where the constructors of namespace-scope objects are. make libcxx-embedded builds the C++ runtime (operator new and delete, the guard functions, __aeabi_atexit, the RTTI classes and __dynamic_cast). On ARM the objects follow the ARM C++ ABI and link with clang++'s. Exceptions are refused by name, as there are no unwind tables for these machines:

embcc: error: C++ exceptions are not supported for thumbv7m-none-eabi yet: EmbCC writes no ARM EHABI unwind tables (.ARM.exidx); compile with -fno-exceptions

On AVR EmbCC refuses to generate code for a C++ unit:

embcc: error: C++ is not yet supported for avr: the C++ lowering assumes a 32-bit int and 4-byte pointers (integral promotions, member pointers, RTTI), and this target's are 16 bits

--syntax-only, -E, -M and --deps-only still accept C++ there. See C++ targets for what the ABI is on each target.

C++ compiles for riscv64-unknown-elf, but no C++ runtime is built for it. Code that needs a landing pad (a try block, or a local object whose destructor must run while an exception unwinds) does not compile:

embcc: cx.cc:7: error: the RV64 backend cannot lower this operation yet (function _Z1gi) [landing w=8 size=4]

Compile with --no-exceptions, and with --no-rtti unless the program supplies the __cxxabiv1 type-information vtables. The program must then define what the compiled code refers to: operator new and operator delete if it uses them, __cxa_pure_virtual for abstract classes, and __cxa_atexit and __dso_handle for static objects with destructors. See C++.

Eight-byte atomics on a 32-bit target#

No 32-bit target moves eight bytes atomically -- ARMv7-M and ARMv8-M Baseline have no ldrexd, RV32's A extension no .d forms -- so on every one (Cortex-M, ARMv7-A, RV32, MIPS32, SPARC, PowerPC, ColdFire, TriCore, Xtensa, RX) an eight-byte atomic is a call to libatomic's sized routine, exactly as GCC and clang make it: __atomic_load_8, __atomic_store_8, __atomic_exchange_8, __atomic_compare_exchange_8 and __atomic_fetch_{add,sub,and,or,xor,nand}_8, with the memory order as the last argument(s). The names, arguments and orders are listed in Extensions. An object compiled by GCC or clang that makes the same calls links against the same routines. (AVR does its eight-byte atomics inline, with interrupts masked.)

librt.a (lib/rt/atomic8.c) defines them all -- plus GCC's __atomic_OP_fetch_8, and libatomic's generic __atomic_load, __atomic_store, __atomic_exchange and __atomic_compare_exchange, which take the size first and which clang calls for an _Atomic long long or double -- by masking interrupts around the access:

The mask is saved and put back as it was, never simply enabled, so a call inside an interrupt handler or a critical section leaves interrupts masked. With interrupts masked the memory-order argument changes nothing. On SPARC, PowerPC, ColdFire, Xtensa and RX, where EmbCC has no inline assembler yet, the two mask routines are machine code (lib/rt/atomic8.h), checked against the backends' encoders and llvm-mc by tests/golden/atomic8.sh.

What this does not cover. Masking interrupts makes the operation atomic on ONE core with no other bus master writing the object:

• It is not atomic against a second core or a DMA engine. A multi-core part (the RP2040, the ESP32's two cores, an AURIX with several TriCore CPUs) needs a lock that both cores honour.
• The instructions are privileged: machine mode on RISC-V, supervisor mode on SPARC, PowerPC, ColdFire and RX, privileged mode on Cortex-M (CPSID is ignored in unprivileged thread mode, so the operation is not atomic there), kernel mode on MIPS. Code running in user mode traps or silently loses atomicity.
• An interrupt of a priority that masking does not reach -- a Cortex-M fault or NMI, a SPARC level-15 interrupt -- can still run in the middle.

Every routine is weak, so a program, an RTOS or a vendor SDK replaces them by defining its own: the RP2040's Pico SDK defines __atomic_*_8 with its hardware spinlocks, and an RTOS whose tasks run unprivileged defines them with its own critical section. Define all the routines you use, with the signatures in Extensions; tests/golden/atomic8/order.c is an example that replaces every one.

Running images under QEMU#

EmbCC's tests run every embedded image on QEMU. The harnesses under tests/harness/ are working examples of a startup, a UART driver and a link line for each board:

Board details the harnesses encode:

• On the M33 board the core starts in the Secure state, so code is linked at the Secure alias of SSRAM1: -Ttext 0x10000000 -Tdata 0x10100000, stack top 0x10200000. An image linked at 0 fails with QEMU's Lockup: can't escalate 3 to HardFault.
• On virt, -bios none keeps OpenSBI out, so the image runs in machine mode from 0x80000000.
• On uno, the image is given with -bios, not -kernel.
• On malta, -kernel loads the image at its KSEG0 link address (0x80100000) and QEMU's own reset code jumps to its entry.
• isa-debug-exit can only report (v << 1) | 1, which is why the x86-64 harness prints the real exit code first and run.sh reads it back (124 for a timeout, 125 for a crash).

A bare-metal image that does not end QEMU itself runs until killed. tests/harness/qrun.sh SECONDS [--until TEXT] COMMAND... runs a QEMU command under a hard timeout; with --until, it watches the output and kills QEMU as soon as TEXT appears, so a test program that prints a sentinel such as DONE as its last line costs only its own run time:

tests/harness/qrun.sh 10 --until DONE \
    qemu-system-avr -M uno -nographic -bios fw.elf

The harness scripts read these environment variables:

Debugging a board#

-g works on every embedded target, and EmbLD carries the DWARF sections into the image and writes a .embdbg index beside it. The image keeps its symbol table, so a debugger can resolve function names even without -g.

EmbDBG connects to anything that speaks the GDB remote serial protocol: QEMU's gdb stub, or OpenOCD in front of a real part. Its remote client knows the register layouts of ARM M-profile, RV32, RV64, x86-64 and AArch64; AVR is not among them.

qemu-system-riscv32 -M virt -bios none -nographic -m 8 -kernel fw.elf -S -gdb tcp::3333 &
embdbg fw.elf remote :3333

Breakpoints, stepping, registers and memory are described in Debugging.