Commit Graph

16135 Commits

Author SHA1 Message Date
Andrew Kelley
694f424fc7 glibc: update abilists file
This one includes riscv32 and riscv64 targets.
2021-12-15 17:50:44 -07:00
Andrew Kelley
d9fe0d2d6b tools/update_glibc: ignore some more files 2021-12-15 15:28:37 -07:00
Andrew Kelley
3dcd3612dc glibc: use linux-specific files for nonshared
Upstream, some of the nonshared functions moved to be different for hurd
and for linux. Since our glibc is linux-only we update to use the
linux-specific files.

This fixes std lib tests for x86_64 when linking glibc.
2021-12-15 15:23:56 -07:00
Andrew Kelley
19ca2415f2 update glibc start files to 2.34
This commit introduces tools/update_glibc.zig to update the start files
for next time.

Some notable changes in recent glibc:

 * abi-note.S has been changed to abi-note.c but we resist the change to
   keep it easier to compile the start files.
 * elf-init.c has been deleted upstream. Further testing should be done
   to verify that binaries against glibc omitting elf-init.c still run
   properly on oldel glibc linux systems.

Closes #4926
2021-12-15 14:30:03 -07:00
Andrew Kelley
1edf8efa42 update glibc headers to 2.34
closes #10308
2021-12-15 14:30:03 -07:00
Andrew Kelley
877a1f2a29 std.os: fix error codes for execve
execve can return EBADLIB on Linux. I observed this when passing
an x86_64 interpreter path to qemu-i386.

This error code is Linux and Solaris-only. I came up with an improved
pattern for dealing with OS-specific error codes.
2021-12-15 14:23:32 -07:00
Andrew Kelley
3532abe0c6 compiler_rt: reorganize in a way that stage2 understands
Before this commit, stage2 behavior tests are regressed; it cannot build
compiler-rt.
2021-12-15 14:23:28 -07:00
Andrew Kelley
e45680cab5
Merge pull request #10341 from ziglang/stage2-tests
stage2: refactor x86_64 and macOS tests
2021-12-15 12:47:11 -08:00
Arnav Singh
09f70bdd91
std.bounded_array: support inserting a new value at the end (#10340)
Since `BoundedArray.insert` internally reserves space for the element
to be inserted, it can support inserting at the position that is
the current length of the array. Change the check for the insertion position
to allow this.
2021-12-15 20:10:34 +01:00
Jakub Konka
0478446377 stage2: fix register size selection
This actually needs proper rework, and I'll get to that when refactoring
MIR.
2021-12-15 17:28:48 +01:00
Jakub Konka
510357355a stage2: fix premature atom deinit in macho linker 2021-12-15 17:28:48 +01:00
Jakub Konka
4b5f8bca5e stage2: clean up tests
* move darwin tests into respective architecture test files: `x86_64`
and `aarch64`
* run majority of `x86_64` tests on macOS
2021-12-15 17:28:48 +01:00
Jakub Konka
87b843ef08
Merge pull request #10338 from ziglang/macho-missing-feats
macho: refactor consts in std.macho, and fix two bugs in MachO linker backend
2021-12-15 15:54:48 +01:00
Jakub Konka
bd926e5ea0 add standalone tests for the new linker bug fixes
This is just a temp addition until I figure out how to tweak
the stage2 test harness to add the ability to test the linker too.
2021-12-15 10:31:29 +01:00
Jakub Konka
3ff05b79b9 macho: handle TLS imported from dylib
This is a missing feature which requires `__thread_ptrs` section
to be synthesised for any extern reference to a global TLS variable.
2021-12-15 08:59:20 +01:00
Jakub Konka
2e7a48d6bf macho: fix section size calculation when no prealloc
Add test for common symbol alignment.
2021-12-15 08:59:20 +01:00
Jakub Konka
ab328aca33 macho: put LC_* consts in a typed enum(u32) LC
repeat for `PLATFORM_*` and `TOOL_*` sets
2021-12-15 08:59:20 +01:00
Andrew Kelley
ff93486d0c test: remove testing for tools/update_glibc.zig
Fixes test failures introduced by
5da013e39c.
2021-12-14 23:41:35 -07:00
Andrew Kelley
5da013e39c remove tools/update_glibc.zig
It has been replaced by https://github.com/ziglang/glibc-abi-tool/ and
should have been removed in 1442aa7dc0.
2021-12-14 21:44:44 -07:00
Jan Philipp Hafer
20328e976f compiler_rt: add __cmpXi2 and __ucmpXi2
- adds __cmpsi2, __cmpdi2, __cmpti2
- adds __ucmpsi2, __ucmpdi2, __ucmpti2
- use 2 if statements with 2 temporaries and a constant
- tests: MIN, MIN+1, MIN/2, -1, 0, 1, MAX/2, MAX-1, MAX if applicable

See #1290
2021-12-14 14:21:30 -08:00
Jan Philipp Hafer
0550198c98 compiler_rt: simplify popcount "magic constants"
- magic constants are nicer to construct ie with
  (~@as(unsigned type, 0) / 3) == 0x55...55
- thanks to Stefan Kanthak for the idea
2021-12-14 14:19:51 -08:00
Jan Philipp Hafer
eb1e75b2b8 compiler_rt: refactor __mulodi2 and __muloti2 to get __mulosi2
- use comptime instead of 2 identical implementations
- tests: port missing tests and link to archived llvm-mirror release 80

See #1290
2021-12-14 14:16:24 -08:00
Andrew Kelley
93c6ab4952
Merge pull request #10322 from vrischmann/more-io_uring
implement more io_uring opcodes
2021-12-14 14:15:12 -08:00
Jan Philipp Hafer
c56663dee8 compiler_rt: add __negsi2, __negdi2, __negti2
- use negXi2.zig to prevent confusion with negXf2.zig
- used for size optimized builds and machines without carry instruction
- tests: special cases 0, -INT_MIN
  * use divTrunc range and shift with constant offsets

See #1290
2021-12-14 14:14:31 -08:00
Luuk de Gram
50201e1c30 wasm-linker: Allow specifying symbols to be exported
Notating a symbol to be exported in code will only tell the linker
where to find this symbol, so other object files can find it. However, this does not mean
said symbol will also be exported to the host environment. Currently, we 'fix' this by force
exporting every single symbol that is visible. This creates bigger binaries and means host environments
have access to symbols that they perhaps shouldn't have. Now, users can tell Zig which symbols
are to be exported, meaning all other symbols that are not specified will not be exported.

Another change is we now support `-rdynamic` in the wasm linker as well, meaning all symbols will
be put in the dynamic symbol table. This is the same behavior as with ELF. This means there's a 3rd strategy
users will have to build their wasm binary.
2021-12-14 14:02:23 -08:00
Stephen Gutekanst
e563b166b2 Fix compilation cache updating bug leading to slow builds
While investigating slow build times with [a large project](https://github.com/hexops/mach/issues/124),
I found that the compiler was reading from disk nearly every C source file in my project
when rebuilding despite no changes having been made. This accounted for several seconds of
time (approx. 20-30% of running `zig build` without any changes to the sources.)

The cause of this was that comparisons of file mtimes would _always_ fail (the mtime of the file on
disk was always newer than that stored in the cache manifest), and so the cache logic would always
fall back to byte-for-byte file content comparisons with what is on disk vs. in the cache-reading every
C source file in my project from disk during each rebuild. Because file contents were the same, a cache
hit occurred, and _despite the mtime being different the cache manifest would not be updated._

One can reproduce this by building a Zig project so the cache is populated, and then changing mtimes
of their C source files to be newer than what is in the cache (without altering file contents.)

The fix is rather simple: we should always write the updated cache manifest regardless of
whether or not a cache hit occurred (a cache hit doesn't indicate if a manifest is dirty) Luckily,
`writeManifest` already contains logic to determine if a manifest is dirty and becomes no-op if no
change to the manifest file is necessary-so we merely need to ensure it is invoked.

Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
2021-12-14 13:56:35 -08:00
Andrew Kelley
5d1aab72d9 std.zig.system: improve native glibc version detection
When the Zig compiler is statically linked, it inspects the
/usr/bin/env ELF file to determine the native glibc version, by checking
the DT_RUNPATH, and then calling readlink() on the libc.so file, because
typically the symlink will have e.g. libc-2.33.so in the name, revealing
the glibc version.

Fortunately, this information is also in readlink() of ld.so, which is
available as the "INTERP" file path. This commit looks for e.g.
`ld-2.33.so` on the symlink data for the dynamic linker.

In theory a more complete solution would also look at `/etc/ld.so.cache`
if necessary, and finally fall back to some hard coded paths, in order
to resolve the location of libc.so, in order to do this readlink() trick
on the resulting path. You can find that flow chart with `man ld.so`.
But I think this logic will be enough to get a correct answer in all real
world cases.

This has been tested on Debian Buster and glibc-based Void Linux.

Fixes #6469
2021-12-13 18:05:02 -07:00
Andrew Kelley
1442aa7dc0 stage2: improved glibc stubs
This commit upgrades glibc shared library stub-creating code to use the
new abilists file which is generated by the new glibc-abi-tool project:
https://github.com/ziglang/glibc-abi-tool/

The abilists file is different in these ways:
 * It additionally encodes whether a symbol is a function or an object,
   and if it is an object, it additionally encodes the size in bytes.
 * It additionally encodes migrations of symbols from one library to
   another between glibc versions.
 * It is binary data instead of ascii.
 * It is one file instead of three.
 * It is 165 KB instead of 200 KB.

This solves three bugs:

Fixes #7667
Fixes #8714
Fixes #8896
2021-12-13 15:07:53 -08:00
Vincent Rischmann
f9b8808d74 os/linux/io_uring: implement linkat 2021-12-12 18:49:17 +01:00
Vincent Rischmann
5dd53c1986 os/linux/io_uring: implement symlinkat 2021-12-12 18:43:31 +01:00
Vincent Rischmann
1fd0542bee os/linux/io_uring: implement mkdirat 2021-12-12 18:33:17 +01:00
Vincent Rischmann
4e647dee9f os/linux/io_uring: implement unlinkat 2021-12-12 18:32:08 +01:00
Vincent Rischmann
088c1fab4d os/linux/io_uring: implement renameat 2021-12-12 18:12:15 +01:00
Vincent Rischmann
0229fb7c62 os/linux/io_uring: implement shutdown 2021-12-12 18:04:15 +01:00
Vincent Rischmann
331b8e892a os/linux: add more io_uring opcode 2021-12-12 16:55:51 +01:00
Jakub Konka
a76910b691 Merge branch 'motiejus-framework-not-found' 2021-12-12 14:03:13 +01:00
Jakub Konka
4ce6208750 macho: if lib or framework not found, wait until syms resolved
This way, we will inform the user that there are unresolved symbols
in addition to missing library/framework as requested on the linker
line. If all symbols were resolved on the other hand, we still
flag up that the library/framework cannot be found.

Example behaviour:

```
$ zig cc hello.c -framework MyFoundation --verbose
warning(link): framework not found for '-framework MyFoundation'
warning(link): Framework search paths:
warning(link):   /Library/Frameworks
warning(link):   /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks
thread 1079397 panic: attempt to unwrap error: FrameworkNotFound
...stack trace...
```

and

```
❯ zig cc hello.c -lWAT --verbose
warning(link): library not found for '-lWAT'
warning(link): Library search paths:
warning(link):   /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib
warning(link):   /usr/local/lib
thread 1079824 panic: attempt to unwrap error: LibraryNotFound
...stack trace...
```
2021-12-12 13:53:59 +01:00
Jakub Konka
1abae06432 Merge branch 'framework-not-found' of git://github.com/motiejus/zig into motiejus-framework-not-found 2021-12-12 13:31:26 +01:00
Jakub Konka
dbfcebf8d8 macho: allow undefined symbols in dylibs
We now respect both `-fallow-shlib-undefined` and
`-Wl,"-undefined=dynamic_lookup"` flags. This is the first step
towards solving issues #8180 and #3000. We currently do not expose
any other ld64 equivalent flag for `-undefined` flag - we basically
throw an error should the user specify a different flag. Support for
those is conditional on closing #8180. As a result of this change,
it is now possible to generate a valid native Node.js addon with Zig
for macOS.
2021-12-12 00:32:03 +01:00
Jan Philipp Hafer
efdb94486b compiler_rt: add __bswapsi2, __bswapdi2 and __bswapti2
- each byte gets masked, shifted and combined
- use boring masks instead of comptime for readability
- tests: bit patterns with reverse operation, if applicable

See #1290
2021-12-11 01:43:37 -08:00
Motiejus Jakštys
82a7069344 macho: fail if requested -framework is not found
If `-framework` is requested, but not found, the linker will err
instead of creating a strange executable.

https://github.com/ziglang/zig/issues/10299#issuecomment-990404953

Refs #9542
Refs #10299
Refs #10158
2021-12-11 10:40:57 +02:00
Andrew Kelley
97c0373fa7
Merge pull request #10295 from ifreund/prefetch
Implement @prefetch()
2021-12-10 18:54:56 -08:00
Isaac Freund
516945d7d9
langref: document @prefetch() builtin 2021-12-11 00:29:31 +01:00
Isaac Freund
7bb6393b59
stage1: implement @prefetch() builtin 2021-12-11 00:29:31 +01:00
Isaac Freund
175463d75d
AstGen: implement @prefetch() builtin 2021-12-10 23:09:02 +01:00
Isaac Freund
47c309c34a
AstGen: increase zig fmt off/on granularity
This enables automatic formatting for a significant amount of code that
currently doesn't deviate from the standard zig fmt enforced style.
2021-12-10 23:09:01 +01:00
Jakub Konka
75f3e7a4a0
Merge pull request #10310 from ziglang/macho-common-functions
macho: move load command wrappers and parsing utils to std.macho
2021-12-10 21:55:21 +01:00
Jakub Konka
828f61e8df macho: move all helpers from commands.zig into std.macho
This way we will finally be able to share common parsing logic
between different Zig components and 3rd party packages.
2021-12-10 18:18:28 +01:00
Jakub Konka
81e7d8505c macho: move helper functions to libstd
Helper functions such as `commands.sectionName`, etc. should really
belong in `std.macho.section_64` extern struct.
2021-12-10 11:56:51 +01:00
Andrew Kelley
77836e08a2
Merge pull request #9930 from PhaseMage/fix-cache-timestamps
Fix isProblematicTimestamp
2021-12-09 21:11:51 -08:00