Commit Graph

18808 Commits

Author SHA1 Message Date
Jakub Konka
80790be309 compiler_rt: compile each unit separately for improved archiving 2022-06-17 16:38:59 -07:00
Andrew Kelley
33cf6ef621
Merge pull request #11881 from Vexu/stage2
Stage2: fixes for bugs found while looking for miscompilations
2022-06-17 19:23:08 -04:00
Veikka Tuominen
28986a0590 stage2: check that struct is a tuple when value tags differ in eql 2022-06-17 21:04:21 +03:00
Veikka Tuominen
a224dfceee std.tz: fix function returning pointer to local variable 2022-06-17 21:04:21 +03:00
Xavier Bouchoux
b66247c97a stage2: coerce tuple to vector 2022-06-17 19:06:17 +03:00
Omar Alhammadi
69e2cac0d3
stage2: comptime @bitCast packed struct bug fix 2022-06-17 19:04:51 +03:00
Veikka Tuominen
b9dcbe6b4c Sema: handle sentinels in tupleToArray 2022-06-17 18:57:02 +03:00
Veikka Tuominen
ffa6f895ff Sema: validateArrayInit detect bitcast before store 2022-06-17 18:57:02 +03:00
Motiejus Jakštys
d506275a06 [elf] understand -no-pie
This passes -Wl,-no-pie linker arg. Golang uses that. From the `ld(1)`
man page:

   Create a position dependent executable.  This is the default.

Not adding to the help text, because this is the default.
2022-06-16 15:27:30 -04:00
Jakub Konka
d3caacfab7
Merge pull request #11864 from jedisct1/wasi-libc-update
Update the WASI libc to 30094b6ed05f19cee102115215863d185f2db4f0
2022-06-15 12:33:17 +02:00
Frank Denis
27610b0a0f
std/crypto: add support for ECDSA signatures (#11855)
ECDSA is the most commonly used signature scheme today, mainly for
historical and conformance reasons. It is a necessary evil for
many standard protocols such as TLS and JWT.

It is tricky to implement securely and has been the root cause of
multiple security disasters, from the Playstation 3 hack to multiple
critical issues in OpenSSL and Java.

This implementation combines lessons learned from the past with
recent recommendations.

In Zig, the NIST curves that ECDSA is almost always instantied with
use formally verified field arithmetic, giving us peace of mind
even on edge cases. And the API rejects neutral elements where it
matters, and unconditionally checks for non-canonical encoding for
scalars and group elements. This automatically eliminates common
vulnerabilities such as https://sk.tl/2LpS695v .

ECDSA's security heavily relies on the security of the random number
generator, which is a concern in some environments.

This implementation mitigates this by computing deterministic
nonces using the conservative scheme from Pornin et al. with the
optional addition of randomness as proposed in Ericsson's
"Deterministic ECDSA and EdDSA Signatures with Additional Randomness"
document. This approach mitigates both the implications of a weak RNG
and the practical implications of fault attacks.

Project Wycheproof is a Google project to test crypto libraries against
known attacks by triggering edge cases. It discovered vulnerabilities
in virtually all major ECDSA implementations.

The entire set of ECDSA-P256-SHA256 test vectors from Project Wycheproof
is included here. Zero defects were found in this implementation.

The public API differs from the Ed25519 one. Instead of raw byte strings
for keys and signatures, we introduce Signature, PublicKey and SecretKey
structures.

The reason is that a raw byte representation would not be optimal.
There are multiple standard representations for keys and signatures,
and decoding/encoding them may not be cheap (field elements have to be
converted from/to the montgomery domain).

So, the intent is to eventually move ed25519 to the same API, which
is not going to introduce any performance regression, but will bring
us a consistent API, that we can also reuse for RSA.
2022-06-15 08:55:39 +02:00
Andrew Kelley
0e9458a3fc test-cases: avoid using realpath since it is not portable
For example FreeBSD does not support this syscall.
2022-06-14 22:15:22 -07:00
Andrew Kelley
8caa206417 test-cases: fix race with zig run on C backend tests
Also avoid redundantly doing compile-error checks on multiple targets
for test cases where that is not helpful.
2022-06-14 15:27:43 -07:00
Frank Denis
8c63037695 Update the WASI libc
Update our copy of wasi-libc up to the commit
30094b6ed05f19cee102115215863d185f2db4f0 from the upstream repository.
2022-06-14 23:20:39 +02:00
Andrew Kelley
22690efcc2 multi-thread zig build test-cases
Instead of always using std.testing.allocator, the test harness now follows
the same logic as self-hosted for choosing an allocator - that is - it
uses C allocator when linking libc, std.testing.allocator otherwise, and
respects `-Dforce-gpa` to override the decision. I did this because
I found GeneralPurposeAllocator to be prohibitively slow when doing
multi-threading, even in the context of a debug build.

There is now a second thread pool which is used to spawn each
test case. The stage2 tests are passed the first thread pool. If it were
only multi-threading the stage1 tests then we could use the same thread
pool for everything. However, the problem with this strategy with stage2
is that stage2 wants to spawn tasks and then call wait() on the main
thread. If we use the same thread pool for everything, we get a deadlock
because all the threads end up all hanging at wait() and nothing is
getting done. So we use our second thread pool to simulate a "process pool"
of sorts.

I spent most of the time working on this commit scratching my head trying
to figure out why I was getting ETXTBSY when spawning the test cases.
Turns out it's a fundamental Unix design flaw, already a known, unsolved
issue by Go and Java maintainers:

https://github.com/golang/go/issues/22315
https://bugs.openjdk.org/browse/JDK-8068370

With this change, the following command, executed on my laptop, went from
6m24s to 1m44s:

```
stage1/bin/zig build test-cases -fqemu -fwasmtime -Denable-llvm
```

closes #11818
2022-06-14 17:15:13 -04:00
Mikael Berthe
47c4d44502
std.math.big.int: update Managed.toString() to use provided allocator (#11839) 2022-06-13 17:19:37 +02:00
Frank Denis
7c660d17cd
crypto/pcurves: compute constants for inversion at comptime (#11780) 2022-06-13 08:13:52 +02:00
Andrew Kelley
13f02c30e6 stage2: fix some inline asm incompatibilities with stage1 2022-06-12 14:46:05 -07:00
Andrew Kelley
ffa700ee58
Merge pull request #11837 from Vexu/stage2
Fix (nearly) all stage2 crashes when testing stdlib
2022-06-12 17:45:57 -04:00
Andrew Kelley
6e42d45dcc
Merge pull request #11851 from ziglang/stage2-comptime-store
Sema: rework beginComptimePtrMutation
2022-06-12 15:19:15 -04:00
Veikka Tuominen
0a9d6956e7 Sema: add missing set_union_tag 2022-06-12 19:17:41 +03:00
Andrew Kelley
e64d5a0753 Sema: rework beginComptimePtrMutation
This comment is now deleted because the task is completed in this
commit:

```
// TODO: Update this to behave like `beginComptimePtrLoad` and properly check/use
// `container_ty` and `array_ty`, instead of trusting that the parent decl type
// matches the type used to derive the elem_ptr/field_ptr/etc.
//
// This is needed because the types will not match if the pointer we're mutating
// through is reinterpreting comptime memory.
```

The main strategy is to change the ComptimePtrMutationKit struct so that
instead of `val: *Value` it now returns a tagged union which can be one
of three possibilities:

 * The pointer type matches the actual comptime Value so a direct
   modification is possible. Before this commit, the implementation
   incorrectly assumed this was always the case.

 * In the case of needing to write through a reinterpreted pointer, a
   mutable base Value pointer is provided along with a byte offset
   pointing to the element value in virtual memory.

 * Otherwise, it means a compile error must be emitted because one or
   both of the types (the owner of the value, or the pointer type being
   used to write through) do not have a well-defined memory layout.

After calling beginComptimePtrMutation, the one callsite now switches on
this tagged union and does the appropriate thing. The main new logic is
for the second case, which involves pointer reinterpretation, which now
takes this strategy:

 1. write the base value to a memory buffer.
 2. perform the pointer store at the proper byte offset, thereby
    modifying a subset of the buffer.
 3. read the base value from the memory buffer, overwriting the old base
    value.
2022-06-12 01:33:56 -07:00
Andrew Kelley
c29746aa55 add std.debug.Trace.format
This makes it show up in some useful places; for example in the
self-hosted compiler we already print it now with
--debug-compile-errors.
2022-06-12 00:57:59 -07:00
Andrew Kelley
85492f2b91 std.mem.zeroes: remove call to std.meta
everybody is so horny for std.meta
2022-06-12 00:56:59 -07:00
Veikka Tuominen
6b36774adc std: disable failing tests, add zig2 build test-std to CI 2022-06-12 10:43:28 +03:00
Veikka Tuominen
35c7e376b8 stage2: improve anon name strategy for local variables 2022-06-11 23:49:33 +03:00
Veikka Tuominen
0333ff4476 stage2: make error{} the same size as anyerror
Having `error{}` be a zero bit type causes issues when it interracts
with empty inferred error sets which are the same size as `anyerror`.
2022-06-11 23:49:33 +03:00
Veikka Tuominen
488e1e5f51 stage2: small fixes + adjustments to std tests 2022-06-11 23:49:33 +03:00
Veikka Tuominen
95ab942184 Sema: make @src give absolute paths 2022-06-11 23:12:52 +03:00
Andrew Kelley
9360cfebc7 Sema: type safety for "runtime_index" field
This commit does not change any behavior, but changes the type of
the runtime_index field from u32 to a non-exhaustive enum. This allows
us to put `std.math.maxInt(u32)` only in the enum type definition and
give it an official meaning.
2022-06-11 12:18:04 -07:00
Luuk de Gram
13123afedb wasm: implement @ceil, @floor and @trunc 2022-06-11 19:38:00 +02:00
Luuk de Gram
f05e09a0cf wasm: optimize & simplify sign extension
Rather than storing all the shifts in temporaries, we perform the correct
shifting without temporaries. This makes the runtime code more performant
and also the backend code is simplified as we have a singular abstraction.
2022-06-11 19:38:00 +02:00
Luuk de Gram
18afcc34c6 wasm: implement @divFloor for signed integers 2022-06-11 19:38:00 +02:00
Luuk de Gram
3011ef2d82 wasm: signed integer division (non-floor)
Implements the non-floor variants of signed integer division.
2022-06-11 19:38:00 +02:00
Luuk de Gram
9b84f29503 wasm: support all @div{trunc/floor/exact} ops
This does however not support floats of bitsizes
different than 32 or 64. f16, f80, f126 will require
support for compiler-rt and are out-of-scope for this commit.

Signed integers are currently not supported either.
2022-06-11 19:38:00 +02:00
Luuk de Gram
180baa0546 wasm:@byteSwap for 24 bit integers 2022-06-11 19:38:00 +02:00
Luuk de Gram
bc499de328 wasm: implement @byteSwap for 16/32bit integers 2022-06-11 19:38:00 +02:00
Veikka Tuominen
eaa6b04c3c Sema: skip decl causing namespace lookup when doing lookup 2022-06-11 11:02:56 +03:00
Veikka Tuominen
0f820d0bdf stage2: improve debugging tools
llvm: dump failed module when -femit-llvm-ir set
print_air:
 * print fully qualified name
 * use Type.fmt and Value.fmtValue, fmtDebug is useless

TypedValue
 * handle anon structs and tuples
 * fix bugs
2022-06-11 11:02:56 +03:00
Veikka Tuominen
002df65b6e Sema: handle tuple and anon_struct in resolveTypeFully 2022-06-11 11:02:56 +03:00
Andrew Kelley
c1eb6c30e8
Merge pull request #11835 from ziglang/stage2-behavior
stage2: fix handling of aggregates with mixed comptime-only fields
2022-06-11 00:27:41 -04:00
Cody Tapscott
9b05474d79 ThreadPool: Make join() a no-op in single-threaded mode
This comptime gate is needed to make sure that purely single-threaded
programs don't generate calls to the std.Thread API.

WASI targets successfully build again with this change.
2022-06-10 21:59:39 -04:00
Andrew Kelley
6bf529dc38 link/wasm: fix writing past the end of debug info buffer
The function `writeDbgInfoNopsBuffered` was based on the function
`pwriteDbgInfoNops`, originally written by me, and then modified to
write to a memory buffer instead of an open file. When writing to a
file, any extra bytes beyond the end of the file extend the size of
the file, and the function body of `pwriteDbgInfoNops` takes advantage
of this when `next_padding_bytes` causes the write to go beyond the
end of the file. However, when writing to a memory buffer, the
underlying array list must be expanded if the write would cause the
buffer to expand.
2022-06-10 17:55:17 -07:00
Jakub Konka
62023c60b4 stage2: correctly work out dirname for ar 2022-06-11 00:23:10 +02:00
Andrew Kelley
3c3bc5af29 Sema: introduce bitSizeAdvanced to recursively resolve types
Same pattern as abiSizeAdvanced.

Fixes compiler crash for nested packed structs.
2022-06-10 15:04:39 -07:00
Andrew Kelley
fcfeafe99a
Merge pull request #11819 from ziglang/std.debug.Trace
introduce std.debug.Trace and use it to debug a LazySrcLoc in stage2 that is set to a bogus value
2022-06-10 05:26:59 -04:00
Andrew Kelley
58bc562cb4 update packed struct behavior tests to new language semantics 2022-06-09 20:37:24 -07:00
Andrew Kelley
436aafd3e2 remove a stage2 test case that relies on a very deep compiler stack
It's causing Drone CI failure on aarch64. I will open a follow-up issue
to track this missing test case.
2022-06-09 20:00:59 -07:00
Andrew Kelley
2bf532fc23 stage2: use std.debug.Trace only when explicitly enabled
Because it bumps up the stack space requirements, which is making a test
case fail on aarch64 drone CI.
2022-06-09 20:00:59 -07:00
Andrew Kelley
32c90cb553 stage2: fix handling of aggregates with mixed comptime-only fields 2022-06-09 19:23:36 -07:00