Not hashing CWD is potentially unsafe since it involves deliberately
poisoning the cache in certain situations in exchange for performance
gain. It can lead to debug information pointing out either no longer
existing or much worse incorrect source files, possibly leading
developers onto a false track and wasting a lot of time.
If one wishes to save build time by sharing the cache between multiple
source trees, this can be achieved safely by enabling reproducible
debug information, like this:
CONFIG_CCACHE_DIR="$(HOME)/.ccache"
CONFIG_REPRODUCIBLE_DEBUG_INFO=y
Note that CWD hashing gets disabled implicitly when reproducible debug
information is enabled. The CCACHE_NOHASHDIR option is only for
disabling CWD hashing in cases where it is not safe to do so.
Signed-off-by: Erik Karlsson <erik.karlsson@iopsys.eu>
Link: https://github.com/openwrt/openwrt/pull/20317
Signed-off-by: Robert Marko <robimarko@gmail.com>
(cherry picked from commit 11820ef801)
Update the compiler check for ccache so we don't
end up with the wrong binaries. Right now the
compiler check will not be able to correctly
distinguish the compiler used for build
ARMv8 binaries from the one used to build
ARMv7 binaries.
Signed-off-by: Markus Gothe <markus.gothe@genexis.eu>
Link: https://github.com/openwrt/openwrt/pull/16290
Signed-off-by: John Crispin <john@phrozen.org>
(cherry picked from commit 3859e8eeb8)
- Disable compression for ccache's cached files.
- Disable the hashing of the CWD inside debug information. This
increases the cache hits drastically.
Signed-off-by: Markus Gothe <markus.gothe@genexis.eu>
(cherry picked from commit 1f11912138)
Add a variable that stores the original value of $PATH
in the host system's shell, before Make alters it.
This can be useful for when it is necessary
to ignore symlinks and programs made by the build system.
Define this new variable before all instances of
'export PATH:=' or similar.
Signed-off-by: Michael Pratt <mcpratt@pm.me>
(cherry picked from commit d87a8aa148)
By having a local copy of gnulib, we can:
import the latest macro fixes into any package,
get rid of some statically stored macros that were otherwise missing,
bootstrap GNU tools with the latest relevant source
without having to wait for a release or rely on git submodules,
and possibly more...
The patch assists in bootstrapping by ignoring
the building of po files using gettext,
and also to allow a user-defined path to a program
to include parameters.
Signed-off-by: Michael Pratt <mcpratt@pm.me>
Some uses cases, like with autotools, need a path for 'true'
if we use them to force something to skip.
This will work by default on Linux hosts,
and require MacOS hosts to get coreutils,
which is currently installed in the CI
for 'macos-latest' host.
In the future, prereq stage can be reworked
to search for the actual binary
instead of relying on env.
Signed-off-by: Michael Pratt <mcpratt@pm.me>
Some packages using Libtool have the possibility
of using this environment variable if set,
instead of the direct command for "file" using PATH,
or the need to patch a hard-coded path.
This is a new feature of Libtool 2.4.7
Ref: bf261073d ("tools/libtool: bump to 2.4.7")
Signed-off-by: Michael Pratt <mcpratt@pm.me>
These wrappers are not needed as CC doesn't need to be a single word.
a53b084e49 which introduced the wrappers
doesn't explain why they were really needed and why only for the target
and not for the host.
Moreover, name of the wrappers breaks a ccache assumption: since
v4.0-3-g6a92b4cd3a67 it has special handling for "chained" invocation
such as "ccache ccache gcc" where it skips all the "ccache*" names in
the middle and proceeds to run as if it was started as "ccache
gcc"[1][2].
This becomes important when a build system sees ccache in the PATH and
automatically enables it by prepending to CC. An example of such a
system would be autosetup as used by jimtcl. With the wrappers it breaks
as the command line ends up being just "ccache -Os..." because
"ccache_cc" gets skipped as it starts with "ccache".
[1] https://github.com/ccache/ccache/blob/master/src/ccache.cpp#L2105
[2] https://github.com/ccache/ccache/blob/master/src/Util.cpp#L802
Reported-by: Karl Palsson <karlp@etactica.com>
Signed-off-by: Paul Fertser <fercerpav@gmail.com>
STAGING_DIR may be provided from command line. We currently hardcoded
STAGING_DIR_HOST and STAGING_DIR_HOSTPKG to the default location but we
currently have some relocatable patch that derive the path from
STAGING_DIR.
Fix this and correctly derive STAGING_DIR_HOST and STAGING_DIR_HOSTPKG
from STAGING_DIR.
The intention is to fix inconsistency from the relocatable patch and the
use of STAGING_DIR_HOST that is always hardcoded.
This with a wrong configuration may end up in broken state with some
host tools expecing a PATH from STAGING_DIR and others using library
from the default staging_dir/host path.
To save downstream project the original implementation is saved while
fixing the inconsistency between patch and .mk.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
We currently enable -Wno-error=unused-but-set-variable and
-Wno-error=unused-result by default on every compile package.
While this is (relatively) unharmful, we should follow other project
direction and starts enforcing good code quality. For example the linux
kernel recently started to enforce Wall by default and clean code is
mandatory for inclusion.
Drop for good these flags and and make it mandatory to correctly handle
return values at least with a warning log if they are not strictly error
condition.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
To generate commitcount we use grep --max-count. This is not present on
alpine grep and cause wrong generation. Use -m as it's just the short
version of --max-count and more portable.
Fixes: #11200
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
(cherry picked from commit eb7ffeafbf)
To generate commitcount we use grep --max-count. This is not present on
alpine grep and cause wrong generation. Use -m as it's just the short
version of --max-count and more portable.
Fixes: #11200
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Move DOWNLOAD_CHECK_CERTIFICATE to include/download.mk as it's a better
place than exporting it in the global rules.mk makefile.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Don't add wrapped bin to the TARGET_PATH as it does cause compilation
error.
cmake.mk will use the "command -v" and will use the wrapped bin instead
of the external toolchain bin as they have the same name and command
will select the first result.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
(cherry picked from commit a90eabf602)
When using the OpenWrt toolchain as an external toolchain the build
failed due to missing LTO support. By choosing the GCC wrappers of
the tools this commit makes sure that the LTO-enabled executables
are being used.
Signed-off-by: Vincent Wiemann <vincent.wiemann@ironai.com>
[ wrap the commit description to 72 char ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
(cherry picked from commit 2555ffb453)
Some packages download a single raw file or set of raw files
instead of an archive.
The filenames of these download files have names and version numbers
that do not directly correspond to the name and version of the package
as it is defined in its Makefile.
To better organize downloaded files, this adds the option of
creating a subdirectory in the download directory for these files
that can be named with package specific variables.
DL_DIR is now a recursively expanded variable
since the value now depends on whether DL_SUBDIR is defined.
Signed-off-by: Michael Pratt <mcpratt@pm.me>
Remove flags from wget and curl instructing them to ignore bad server
certificates. Although other mechanisms can protect against malicious
modifications of downloads, other vectors of attack may be available
to an adversary.
TLS certificate verification can be disabled by turning oof the
"Enable TLS certificate verification during package download" option
enabled by default in the "Global build settings" in "make menuconfig"
Signed-off-by: Josh Roys <roysjosh@gmail.com>
[ add additional info on how to disable this option ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
(cherry picked from commit 90c6e3aedf)
Remove flags from wget and curl instructing them to ignore bad server
certificates. Although other mechanisms can protect against malicious
modifications of downloads, other vectors of attack may be available
to an adversary.
TLS certificate verification can be disabled by turning oof the
"Enable TLS certificate verification during package download" option
enabled by default in the "Global build settings" in "make menuconfig"
Signed-off-by: Josh Roys <roysjosh@gmail.com>
[ add additional info on how to disable this option ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Don't add wrapped bin to the TARGET_PATH as it does cause compilation
error.
cmake.mk will use the "command -v" and will use the wrapped bin instead
of the external toolchain bin as they have the same name and command
will select the first result.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
When using the OpenWrt toolchain as an external toolchain the build
failed due to missing LTO support. By choosing the GCC wrappers of
the tools this commit makes sure that the LTO-enabled executables
are being used.
Signed-off-by: Vincent Wiemann <vincent.wiemann@ironai.com>
[ wrap the commit description to 72 char ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
This uses kernel's generic variable and doesn't require patching it with
a custom Makefile change. It's expected *not* to change any behaviour.
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
(cherry picked from commit 1d42af720c)
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
(cherry picked from commit 24e27bec9a)
Compiler option -no-plt will break kernel builds on some architectures
eg. (x86) Filter this option from the recently introduced handling of
KCFLAGS vs EXTRA_OPTIMISATION
Fixes: 1d42af720c ("kernel: use KCFLAGS for passing EXTRA_OPTIMIZATION flags")
Suggested-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
On recent macOS, /usr/bin/python3 is a wrapper that finds the right python executable
It checks argv[0] to determine if python2 or python3 should be called. Always execute
it as python3 to ensure it calls the right version
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Added HOST_CXXFLAGS to specify CXXFLAGS during host-compile
(e.g. to specify c++ standard: HOST_CXXFLAGS += -std=c++11)
Signed-off-by: Sergey V. Lobanov <sergey@lobanov.in>
ninja is faster at building cmake packages than make, and according to reports
also more reliable at handling parallel builds
This commit includes a patch that adds GNU make jobserver support, in order to
allow more precise control over the number of parallel tasks
Enable parallel build by default for packages using ninja
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Before this commit, it was assumed that mkhash is in the PATH. While
this was fine for the normal build workflow, this led to some issues if
make TOPDIR="$(pwd)" -C "$pkgdir" compile
was called manually. In most of the cases, I just saw warnings like this:
make: Entering directory '/home/.../package/gluon-status-page'
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
[...]
While these were only warnings and the package still compiled sucessfully,
I also observed that some package even fail to build because of this.
After applying this commit, the variable $(MKHASH) is introduced. This
variable points to $(STAGING_DIR_HOST)/bin/mkhash, which is always the
correct path.
Signed-off-by: Leonardo Mörlein <me@irrelefant.net>
Fixes issue openwrt/packages#14921, whereby inline ASM wasn't getting
built as PIC; look at gmp-6.2.1/mpn/x86/pentium/popcount.asm for
example:
ifdef(`PIC',`
...
for a routine that exists in both PIC and non-PIC versions.
Make sure that wherever $(FPIC) gets passed as a variable expansion
that it gets quoted where necessary (such as setting environment
variables in shell commands).
Signed-off-by: Philip Prindeville <philipp@redfish-solutions.com>
(cherry picked from commit af22991e03)
Fixes issue openwrt/packages#14921, whereby inline ASM wasn't getting
built as PIC; look at gmp-6.2.1/mpn/x86/pentium/popcount.asm for
example:
ifdef(`PIC',`
...
for a routine that exists in both PIC and non-PIC versions.
Make sure that wherever $(FPIC) gets passed as a variable expansion
that it gets quoted where necessary (such as setting environment
variables in shell commands).
Signed-off-by: Philip Prindeville <philipp@redfish-solutions.com>
This reverts commit b12288fa69.
The patchelf approach is too fragile, and the only users of this have been
converted to make patching unnecessary
Leave the abi_version_str variable in place in rules.mk
Signed-off-by: Felix Fietkau <nbd@nbd.name>
This makes it possible to declare a package ABI_VERSION independent from the
upstream soname by setting PKG_ABI_VERSION in the package makefile.
The library filename is fixed up for files installed to packages and to the
staging dir. References to the original from executables within the same
package are also fixed up
Signed-off-by: Felix Fietkau <nbd@nbd.name>
The license folder is a core part of OpenWrt and all GPL-2.0 licensed.
Use SPDX license tags to allow machines to check licenses.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[rebase, keep some Copyright lines, sharpen commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Packages that are in-tree only often lack a PKG_VERSION and only use the
PKG_RELEASE to mark changes. Using COMMITCOUNT/AUTORELEASE variables
causes an issue as both variables are empty during the metadata DUMP
phase.
Instead of leaving these variables empty and causing an error message
like below, set the variables to 0 during dumping. On actual building
the variable is evaluated causing in a value above 0.
ERROR: please fix package/utils/px5g-wolfssl/Makefile - \
see logs/package/utils/px5g-wolfssl/dump.txt for details
Makefile:48: *** Package/px5g-wolfssl is missing the VERSION field. Stop.
Reported-by: Daniel Golle <daniel@makrotopia.org>
Reported-by: Stijn Segers <foss@volatilesystems.org>
Reported-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: Paul Spooren <mail@aparcar.org>
The newly added $(COMMITCOUNT) variable was wrongly increased by plus
one. The addition should have been only added to $(AUTORELEASE) as
OpenWrt traditionally starts counting at one rather than zero.
$(AUTORELEASE) counts the commits since the last bump, which is zero on
the version bump commit itself.
This commit increases $(AUTORELEASE) by one while leaving $(COMMITCOUNT)
as is.
The base-files package is the only package using $(COMMITCOUNT) so far
and requires a pseudo commit to keep the PKG_RELEASE correct. A non
functional change (Copyright bump) is done in the next commit.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The lack of bumped PKG_RELEASE variables is a recurring theme on the
mailing list and in GitHub comments. This costs precious review time,
a rare good within the OpenWrt project.
Instead of relying on a manually set PKG_RELEASE this commit adds a
`commitcount` function that uses the number of Git commits to determine
the release. The function is called via the variables `$(AUTORELEASE)`
or `$(COMMITCOUNT)`. The `PKG_RELEASE` variable can be set to either of
the two.
- $(AUTORELEASE):
Release is automagically set to the number of commits since the last
commit containing either ": update to " or ": bump to ".
Example below:
$ git log packages/foobar/
foobar: fixup file location
foobar: disable docs
foobar: bump to 5.3.2
foobar: fixup copyright
Resulting package name: foobar_5.3.2-3_all.ipk, two package changes
since the last upstream version change, using a 1 based counter.
- $(COMMITCOUNT):
For non-traditional versioning (x.y.z), most prominent `base-files`,
this variable contains the total number of package commits.
The new functionality can also be used by other feeds like packages.git.
In case no build information is available, e.g. when using release
tarballs, the SOURCE_DATE_EPOCH is used to have a reproducible release
identifier.
Suggested-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
Some packages fail to build on arm64 when PKG_ASLR_PIE_ALL=y, due to
machine-specific size restrictions on the global offset table.
While the manual instructs to recompile with -fPIC if it fails with
-fpic, by doing this per package, there is still a risk of random
breakage due to version bumps or other changes, so let's use -fPIC on
arm64 by default.
While comparing the sizes of 141 packages built with -fpic vs -fPIC,
most packages are either equal or smaller in size. Only 9 of the
compared packages turned out slightly larger.
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
Commit f98878e4c1 ("cmake.mk: set C/CXX compiler for host builds as
well") has introduced regression as it didn't taken usage of ccache into
the account so fix it by handling ccache use cases as well.
In order to get this working we need to export HOSTCXX_NOCACHE in
rules.mk as well.
Fixes: f98878e4c1 ("cmake.mk: set C/CXX compiler for host builds as well")
Reported-by: Ansuel Smith <ansuelsmth@gmail.com>
Tested-by: Ansuel Smith <ansuelsmth@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit 524fb5646e)
Commit f98878e4c1 ("cmake.mk: set C/CXX compiler for host builds as
well") has introduced regression as it didn't taken usage of ccache into
the account so fix it by handling ccache use cases as well.
In order to get this working we need to export HOSTCXX_NOCACHE in
rules.mk as well.
Fixes: f98878e4c1 ("cmake.mk: set C/CXX compiler for host builds as well")
Reported-by: Ansuel Smith <ansuelsmth@gmail.com>
Tested-by: Ansuel Smith <ansuelsmth@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Drop our local sstrip copy and use the current ELFKickers upstream
version.
Patch the original makefile in order to avoid building elftoc, since it
fails with musl's elf.h. This is fine, since we only need sstrip anyway.
Finally, add the possibility to pass additional arguments to sstrip and
pass -z (remove trailing zeros) by default, which matches the behaviour
of the previous version.
Signed-off-by: Rui Salvaterra <rsalvaterra@gmail.com>
[shorten long commit msg lines]
Signed-off-by: Paul Spooren <mail@aparcar.org>
Since fakeroot is patched to discover related ressources relative to the
STAGING_DIR_HOST environment variable, there is no need to pass the path
to faked or the preload library manually anymore.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
