From 396701808d83d21f9b292dd752daa2495ba90acb Mon Sep 17 00:00:00 2001 From: George Jiglau Date: Tue, 21 Nov 2017 02:11:51 +0000 Subject: [PATCH] genlink: Rewrite genlink script from awk to python This removes the dependency on awk and should fix #732 python was already a dependency, for the irq generation --- README.md | 4 +- ld/README | 31 +++------- ld/devices.data | 2 +- mk/genlink-config.mk | 10 ++-- mk/genlink-rules.mk | 2 +- scripts/genlink.awk | 77 ------------------------- scripts/genlink.py | 125 +++++++++++++++++++++++++++++++++++++++++ scripts/genlinktest.sh | 13 +++-- 8 files changed, 151 insertions(+), 113 deletions(-) delete mode 100644 scripts/genlink.awk create mode 100755 scripts/genlink.py diff --git a/README.md b/README.md index fa067929..5f9c4e50 100644 --- a/README.md +++ b/README.md @@ -40,9 +40,7 @@ _TIP_: Include this repository as a Git submodule in your project to make sure Prerequisites ------------- -Building requires Python (some code is generated). -If your user application uses the (optional) dynamic linker script generator, -you will (presently) need GNU AWK. Please see https://github.com/libopencm3/libopencm3/issues/732 +Building requires Python (Some code is generated). **For Ubuntu/Fedora:** diff --git a/ld/README b/ld/README index f270afbf..5694c6e6 100644 --- a/ld/README +++ b/ld/README @@ -13,9 +13,8 @@ File contents * {ROOT}/ld/tests/* - Prepared tests for the testing of the script * {ROOT}/ld/devices.data - Device database file -* {ROOT}/ld/Makefile.linker - Common makefile part * {ROOT}/ld/linker.ld.S - Linker script template -* {ROOT}/scripts/genlink.awk - Device database file search script +* {ROOT}/scripts/genlink.py - Device database file search script * {ROOT}/scripts/genlinktest.sh - Device database file search test script Principle of operation @@ -29,26 +28,15 @@ is case insensitive. DEVICE=stm32f407vgt6 -Then, the user includes the file /ld/Makefile.linker exactly after the first -target (usually all) has been defined. - -The script automatically modifies the $(LDSCRIPT) variable to meet the new -generated script with .ld in the project directory, and adds -a rule to make it from the scratch. - -Making the script is done by looking to device database file for the needed -definitions, and applying those definitions to the C preprocessor source -linker.ld.S outputting the preprocessed ld file. - Device database contains definitions of common sections and its origins for the linker preprocessor. Every definition is interpreted in the linker script template as a macro, and it can be used for conditional insertion of the device dependent stuff. -The search in the device database is pattern-based, and using awk script -genlink.awk. The awk script traverses the file as a tree, joining the options -for the preprocessor together by single space. The awk script adds -D to each -parameter for you. +The search in the device database is pattern-based, and using python script +genlink.py. The python script traverses the file as a tree, joining the options +for the preprocessor together by single space. The python script adds -D to +each parameter for you. Testing ------- @@ -69,7 +57,7 @@ The testing stops after all test cases are valid, or at first error found. Example of use -------------- -* See Makefile.example file +* Check the documentation for the genlink module in /mk/README. Device database file structure ------------------------------ @@ -127,9 +115,8 @@ stm32 END --- queried chip name --- stm32f051c4t6 ---- output of the awk script --- --D_ROM=16K -D_RAM=4K -D_ROM_OFF=0x08000000 -D_RAM_OFF=0x20000000 \ --mcpu=cortex-m0 -mthumb +--- output of the python script --- +-D_ROM=16K -D_RAM=4K -D_ROM_OFF=0x08000000 -D_RAM_OFF=0x20000000 The generated linker script file will contain sections rom and ram with appropriate initialization code, specified in linker file source linker.ld.S @@ -154,4 +141,4 @@ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License -along with this library. If not, see . \ No newline at end of file +along with this library. If not, see . diff --git a/ld/devices.data b/ld/devices.data index 034cd8a1..3815e7a4 100644 --- a/ld/devices.data +++ b/ld/devices.data @@ -55,7 +55,7 @@ # --- queried chip name --- # stm32f051c8t6 # -# --- output of the awk script --- +# --- output of the python script --- # -DROM=16K -DRAM=4K -DROM_OFF=0x08000000 -DRAM_OFF=0x20000000 # # The generated linker script file will contain sections rom and ram with diff --git a/mk/genlink-config.mk b/mk/genlink-config.mk index ae480513..b2dfb3c6 100644 --- a/mk/genlink-config.mk +++ b/mk/genlink-config.mk @@ -24,11 +24,11 @@ endif LDSCRIPT = generated.$(DEVICE).ld DEVICES_DATA = $(OPENCM3_DIR)/ld/devices.data -genlink_family :=$(shell gawk -v PAT="$(DEVICE)" -v MODE="FAMILY" -f $(OPENCM3_DIR)/scripts/genlink.awk $(DEVICES_DATA)) -genlink_subfamily :=$(shell gawk -v PAT="$(DEVICE)" -v MODE="SUBFAMILY" -f $(OPENCM3_DIR)/scripts/genlink.awk $(DEVICES_DATA)) -genlink_cpu :=$(shell gawk -v PAT="$(DEVICE)" -v MODE="CPU" -f $(OPENCM3_DIR)/scripts/genlink.awk $(DEVICES_DATA)) -genlink_fpu :=$(shell gawk -v PAT="$(DEVICE)" -v MODE="FPU" -f $(OPENCM3_DIR)/scripts/genlink.awk $(DEVICES_DATA)) -genlink_cppflags :=$(shell gawk -v PAT="$(DEVICE)" -v MODE="CPPFLAGS" -f $(OPENCM3_DIR)/scripts/genlink.awk $(DEVICES_DATA)) +genlink_family :=$(shell $(OPENCM3_DIR)/scripts/genlink.py $(DEVICES_DATA) $(DEVICE) FAMILY) +genlink_subfamily :=$(shell $(OPENCM3_DIR)/scripts/genlink.py $(DEVICES_DATA) $(DEVICE) SUBFAMILY) +genlink_cpu :=$(shell $(OPENCM3_DIR)/scripts/genlink.py $(DEVICES_DATA) $(DEVICE) CPU) +genlink_fpu :=$(shell $(OPENCM3_DIR)/scripts/genlink.py $(DEVICES_DATA) $(DEVICE) FPU) +genlink_cppflags :=$(shell $(OPENCM3_DIR)/scripts/genlink.py $(DEVICES_DATA) $(DEVICE) CPPFLAGS) CPPFLAGS += $(genlink_cppflags) diff --git a/mk/genlink-rules.mk b/mk/genlink-rules.mk index 64b37be0..0136c6f8 100644 --- a/mk/genlink-rules.mk +++ b/mk/genlink-rules.mk @@ -19,4 +19,4 @@ $(LDSCRIPT): $(OPENCM3_DIR)/ld/linker.ld.S $(OPENCM3_DIR)/ld/devices.data @printf " GENLNK $(DEVICE)\n" - $(Q)$(CPP) $(ARCH_FLAGS) $(shell gawk -v PAT="$(DEVICE)" -v MODE="DEFS" -f $(OPENCM3_DIR)/scripts/genlink.awk $(OPENCM3_DIR)/ld/devices.data 2>/dev/null) -P -E $< > $@ + $(Q)$(CPP) $(ARCH_FLAGS) $(shell $(OPENCM3_DIR)/scripts/genlink.py $(DEVICES_DATA) $(DEVICE) DEFS) -P -E $< > $@ diff --git a/scripts/genlink.awk b/scripts/genlink.awk deleted file mode 100644 index 2b7e9d12..00000000 --- a/scripts/genlink.awk +++ /dev/null @@ -1,77 +0,0 @@ -# This awk program generates parameters for the linker script generator feature. -# -# See ld/README file for more info. -# - -# This file is part of the libopencm3 project. -# -# Copyright (C) 2013 Frantisek Burian -# Copyright (C) 2013 Werner Almesberger -# -# This library is free software: you can redistribute it and/or modify -# it under the terms of the GNU Lesser General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public License -# along with this library. If not, see . - -BEGIN { - PAT = tolower(PAT); - family = PAT; -} -!/^#/{ - #remove cr on windows - gsub(/\r$/,""); - - tmp = "^"$1"$"; - gsub(/\?/, ".", tmp); - gsub(/\*/, ".*", tmp); - gsub(/\+/, ".+", tmp); - tolower(tmp); - - if (PAT ~ tmp) { - if ("CPPFLAGS" == MODE) - printf "-D%s ",toupper(PAT); - if ($2 != "+") - PAT=$2; - for (i = 3; i <= NF; i = i + 1) { - if ($i ~ /^CPU=/) { - if ("CPU" == MODE){ - sub(/[^=]*=/,"",$i); - printf "%s",$i; - exit; - } - } - else if ($i ~ /^FPU=/) { - if ("FPU" == MODE){ - sub(/[^=]*=/,"",$i); - printf "%s",$i; - exit; - } - } - else if ($i ~ /[[:upper:]]*=/) { - if ("DEFS" == MODE) - printf "-D_%s ",$i; - } - } - if (PAT=="END"){ - if ("FAMILY" == MODE) - printf "%s",family; - else if ("SUBFAMILY" == MODE) - printf "%s",subfamily; - exit; - } - else{ - subfamily = family; - family = PAT; - if("DEFS" == MODE) - printf "-D%s ",toupper(PAT); - } - } -} diff --git a/scripts/genlink.py b/scripts/genlink.py new file mode 100755 index 00000000..96c27e08 --- /dev/null +++ b/scripts/genlink.py @@ -0,0 +1,125 @@ +#!/usr/bin/env python +# This python program generates parameters for the linker script generator feature. + +# This file is part of the libopencm3 project. +# +# 2017 George-Cristian Jiglau +# +# This library is free software: you can redistribute it and/or modify +# it under the terms of the GNU Lesser General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this library. If not, see . +from __future__ import print_function +import fnmatch +import sys +import re +import os + +if len(sys.argv) != 4: + print("usage: %s " % sys.argv[0], file=sys.stderr) + sys.exit(1) + +data_file_path = sys.argv[1] +find = sys.argv[2].lower() +mode = sys.argv[3].upper() + +device = { + 'info': {}, + 'defs': [], + 'family': [], +} + +# open device data file +with open(data_file_path, 'r') as data_file: + # iterate lines + for line in data_file: + # strip whitespace from the beginning and end of line + line = line.strip() + + # skip empty lines and comments + if line == '' or line.startswith('#'): + continue + + # split line into it's parts: + parts = line.split() + pattern, parent, data = parts[0], parts[1], parts[2:] + + # skip line if pattern did not match first element + if not fnmatch.fnmatch(find, pattern): + continue + + # extract data + for d in data: + # split into K=V + try: + (k, v) = d.split('=') + except: + continue + + # skip invalid datas + if not re.match('^[A-Z0-9_]+$', k): + continue + + # add FPU and CPU to info, not defs + if k in ('FPU', 'CPU'): + device['info'][k.lower()] = v + continue + + device['defs'].append((k, v)) + + # if parent is +, there's more data for this pattern + if parent == '+': + continue + + # device family + device['family'].append(find) + + # break if this was the last line in this chain + if parent == 'END': + break + + # look for the parent + find = parent + +# reverse device list +device['family'] = device['family'][::-1] + +# device was not found +if len(device['family']) == 0: + sys.exit(1) + +# for CPPFLAGS and DEFS, define device family +if mode in ('CPPFLAGS', 'DEFS'): + sys.stdout.write(' '.join('-D%s' % d.upper() for d in device['family'])) + +# defines +if mode == 'DEFS': + if len(device['defs']) > 0: + defs = ' '.join('-D_%s=%s' % d for d in device['defs']) + sys.stdout.write(' ' + defs) + +# device family +elif mode == 'FAMILY': + if len(device['family']) > 0: + sys.stdout.write(device['family'][0]) + +# device subfamily +elif mode == 'SUBFAMILY': + if len(device['family']) > 1: + sys.stdout.write(device['family'][1]) + +# device info +else: + info = mode.lower() + if info in device['info']: + sys.stdout.write(device['info'][info]) + +sys.stdout.flush() diff --git a/scripts/genlinktest.sh b/scripts/genlinktest.sh index 148eae16..2c8d4a8b 100755 --- a/scripts/genlinktest.sh +++ b/scripts/genlinktest.sh @@ -1,6 +1,6 @@ #!/bin/sh -# This script is intended to test the awk program genlink.awk for the linker +# This script is intended to test the python program genlink.py for the linker # script generator feature. # # See ld/README file for more info. @@ -25,8 +25,13 @@ # along with this library. If not, see . # run test -PAAT=`basename $1`; -gawk -v PAT="$PAAT" -f scripts/genlink.awk $1.data > $1.out; +DEVICE=`basename $1`; +(scripts/genlink.py $1.data $DEVICE CPPFLAGS; echo) > $1.out +(scripts/genlink.py $1.data $DEVICE DEFS; echo) >> $1.out +(scripts/genlink.py $1.data $DEVICE FAMILY; echo) >> $1.out +(scripts/genlink.py $1.data $DEVICE SUBFAMILY; echo) >> $1.out +(scripts/genlink.py $1.data $DEVICE CPU; echo) >> $1.out +(scripts/genlink.py $1.data $DEVICE FPU; echo) >> $1.out #check test if ! diff -q $1.out $1.result >/dev/null; then @@ -36,4 +41,4 @@ fi #remove workout only if it is OK rm -f $1.out -exit 0 \ No newline at end of file +exit 0