* define std.crypto.sha2.Sha512224
* rename blunder
* add sha512-224 and sha512-256 tests
* fix Sha2x64 for variations that aren't a multiple of 64 bits
Also removes the LOCK namespace from std.c.wasi because wasi libc does
not have flock.
closes#19336
related to #19352
Co-authored-by: Ryan Liptak <squeek502@hotmail.com>
This is a partial revert of 105db13536.
As we learned from Void Linux packaging, these options are not actually
helpful since the distribution package manager may very well want to
cross-compile the packages that it is building.
So, let's not overcomplicate things. There are already the standard
options: -Dtarget, -Dcpu, and -Ddynamic-linker.
These options are generally provided when the project generates machine
code artifacts, however, there may be a project that does no such thing,
in which case it makes sense for these options to be missing. The Zig
Build System is a general-purpose build system, after all.
Certain types (notably, `std.ComptimeStringMap`) were resulting in excessively
long type names when instantiated, which in turn resulted in excessively long
symbol names. These are problematic for two reasons:
* Symbol names are sometimes read by humans -- they ought to be readable.
* Some other applications (looking at you, xcode) trip on very long symbol names.
To work around this for now, we cap the depth of value printing at 1, as opposed
to the normal 3. This doesn't guarantee anything -- there could still be, for
instance, an incredibly long aggregate -- but it works around the issue in
practice for the time being.
When building zig natively from source on an RK3588 SoC board, the
generated stage3 compiler will use unsupported 'sha256h.4s' instructions
due to mis-identified CPU features. This happens when trying to
compile a new project generated with "zig init"
$ ~/devel/zig/build/stage3/bin/zig build
thread 919 panic: Illegal instruction at address 0x1fdc0c4
/home/dliviu/devel/zig/lib/std/crypto/sha2.zig:223:29: 0x1fdc0c4 in round (zig)
asm volatile (
^
/home/dliviu/devel/zig/lib/std/crypto/sha2.zig:168:20: 0x1fdca87 in final (zig)
d.round(&d.buf);
^
/home/dliviu/devel/zig/lib/std/crypto/sha2.zig:180:20: 0x1c8bb33 in finalResult (zig)
d.final(&result);
^
/mnt/home/dliviu/devel/zig/src/Package/Fetch.zig:754:49: 0x1a3a8eb in relativePathDigest (zig)
return Manifest.hexDigest(hasher.finalResult());
^
/mnt/home/dliviu/devel/zig/src/main.zig:5128:53: 0x1a37413 in cmdBuild (zig)
Package.Fetch.relativePathDigest(build_mod.root, global_cache_directory),
^
???:?:?: 0xff09ffffffffffff in ??? (???)
Unwind information for `???:0xff09ffffffffffff` was not available, trace may be incomplete
???:?:?: 0x3f7f137 in ??? (???)
Aborted (core dumped)
system/linux.zig parses "/proc/cpuinfo" to determine the CPU features, but it
seems to generate the wrong set of features from:
$ cat /proc/cpuinfo
processor : 0
BogoMIPS : 48.00
Features : fp asimd evtstrm crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x2
CPU part : 0xd05
CPU revision : 0
.....
processor : 4
BogoMIPS : 48.00
Features : fp asimd evtstrm crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x4
CPU part : 0xd0b
CPU revision : 0
.....
To fix this, use the Linux kernel way of reading the feature registers as documented
here: https://www.kernel.org/doc/html/latest/arch/arm64/cpu-feature-registers.html
arm.zig already has the code to parse the feature register values, we just need to
collect them in an array and pass them for identification.
Signed-off-by: Liviu Dudau <liviu@dudau.co.uk>
The operation `undefined & 0` ought to result in the value `0`, and likewise for
zeroing only some bits. `std/packed_int_array.zig` tests were failing because
this behavior was not implemented -- this issue was previously masked by faulty
bitcast logic which turned `undefined` values into `0xAA` on pointer loads.
Ideally, we would like to be able to track the undefined bits at comptime.
This is related to #19634.
This commit reverts the handling of partially-undefined values in
bitcasting to transform these bits into an arbitrary numeric value,
like happens on `master` today.
As @andrewrk rightly points out, #19634 has unfortunate consequences
for the standard library, and likely requires more thought. To avoid
a major breaking change, it has been decided to revert this design
decision for now, and make a more informed decision further down the
line.
I have noted several latent bugs in the handling of packed memory on
big-endian targets, and one of them has been exposed by the previous
commit, triggering this test failure. I am opting to disable it for now
on the ground that the commit preceding this one helps far more than it
hurts.
We've got a big one here! This commit reworks how we represent pointers
in the InternPool, and rewrites the logic for loading and storing from
them at comptime.
Firstly, the pointer representation. Previously, pointers were
represented in a highly structured manner: pointers to fields, array
elements, etc, were explicitly represented. This works well for simple
cases, but is quite difficult to handle in the cases of unusual
reinterpretations, pointer casts, offsets, etc. Therefore, pointers are
now represented in a more "flat" manner. For types without well-defined
layouts -- such as comptime-only types, automatic-layout aggregates, and
so on -- we still use this "hierarchical" structure. However, for types
with well-defined layouts, we use a byte offset associated with the
pointer. This allows the comptime pointer access logic to deal with
reinterpreted pointers far more gracefully, because the "base address"
of a pointer -- for instance a `field` -- is a single value which
pointer accesses cannot exceed since the parent has undefined layout.
This strategy is also more useful to most backends -- see the updated
logic in `codegen.zig` and `codegen/llvm.zig`. For backends which do
prefer a chain of field and elements accesses for lowering pointer
values, such as SPIR-V, there is a helpful function in `Value` which
creates a strategy to derive a pointer value using ideally only field
and element accesses. This is actually more correct than the previous
logic, since it correctly handles pointer casts which, after the dust
has settled, end up referring exactly to an aggregate field or array
element.
In terms of the pointer access code, it has been rewritten from the
ground up. The old logic had become rather a mess of special cases being
added whenever bugs were hit, and was still riddled with bugs. The new
logic was written to handle the "difficult" cases correctly, the most
notable of which is restructuring of a comptime-only array (for
instance, converting a `[3][2]comptime_int` to a `[2][3]comptime_int`.
Currently, the logic for loading and storing work somewhat differently,
but a future change will likely improve the loading logic to bring it
more in line with the store strategy. As far as I can tell, the rewrite
has fixed all bugs exposed by #19414.
As a part of this, the comptime bitcast logic has also been rewritten.
Previously, bitcasts simply worked by serializing the entire value into
an in-memory buffer, then deserializing it. This strategy has two key
weaknesses: pointers, and undefined values. Representations of these
values at comptime cannot be easily serialized/deserialized whilst
preserving data, which means many bitcasts would become runtime-known if
pointers were involved, or would turn `undefined` values into `0xAA`.
The new logic works by "flattening" the datastructure to be cast into a
sequence of bit-packed atomic values, and then "unflattening" it; using
serialization when necessary, but with special handling for `undefined`
values and for pointers which align in virtual memory. The resulting
code is definitely slower -- more on this later -- but it is correct.
The pointer access and bitcast logic required some helper functions and
types which are not generally useful elsewhere, so I opted to split them
into separate files `Sema/comptime_ptr_access.zig` and
`Sema/bitcast.zig`, with simple re-exports in `Sema.zig` for their small
public APIs.
Whilst working on this branch, I caught various unrelated bugs with
transitive Sema errors, and with the handling of `undefined` values.
These bugs have been fixed, and corresponding behavior test added.
In terms of performance, I do anticipate that this commit will regress
performance somewhat, because the new pointer access and bitcast logic
is necessarily more complex. I have not yet taken performance
measurements, but will do shortly, and post the results in this PR. If
the performance regression is severe, I will do work to to optimize the
new logic before merge.
Resolves: #19452Resolves: #19460
* fix UB when Thread.spawn fails, and we try to join uninitialized
threads (through new `thread_count` variable).
* make sure that the tests pins down synchronization guarantees: in the
main thread, we can observe `1` due to synchronization from
`Thread.join()`. To make sure that once uses Acq/Rel, and not just
relaxed, we should also additionally check that each thread observes
1, regardless of whether it was the one to call once.
On Windows, the command line arguments of a program are a single WTF-16 encoded string and it's up to the program to split it into an array of strings. In C/C++, the entry point of the C runtime takes care of splitting the command line and passing argc/argv to the main function.
https://github.com/ziglang/zig/pull/18309 updated ArgIteratorWindows to match the behavior of CommandLineToArgvW, but it turns out that CommandLineToArgvW's behavior does not match the behavior of the C runtime post-2008. In 2008, the C runtime argv splitting changed how it handles consecutive double quotes within a quoted argument (it's now considered an escaped quote, e.g. `"foo""bar"` post-2008 would get parsed into `foo"bar`), and the rules around argv[0] were also changed.
This commit makes ArgIteratorWindows match the behavior of the post-2008 C runtime, and adds a standalone test that verifies the behavior matches both the MSVC and MinGW argv splitting exactly in all cases (it checks that randomly generated command line strings get split the same way).
The motivation here is roughly the same as when the same change was made in Rust (https://github.com/rust-lang/rust/pull/87580), that is (paraphrased):
- Consistent behavior between Zig and modern C/C++ programs
- Allows users to escape double quotes in a way that can be more straightforward
Additionally, the suggested mitigation for BatBadBut (https://flatt.tech/research/posts/batbadbut-you-cant-securely-execute-commands-on-windows/) relies on the post-2008 argv splitting behavior for roundtripping of the arguments given to `cmd.exe`. Note: it's not necessary for the suggested mitigation to work, but it is necessary for the suggested escaping to be parsed back into the intended argv by ArgIteratorWindows after being run through a `.bat` file.
signature/s:
Algorithm Before After
---------------+---------+-------
ecdsa-p256 3707 4396
ecdsa-p384 1067 1332
ecdsa-secp256k1 4490 5147
Add ECDSA to the benchmark by the way.
This was overlooked in #19458. Using the fully qualified name of each
module usually makes sense, but there is one module where it does not,
namely, the root module, since its name is `root`. The original Autodoc
tar creation logic used `comp.root_name` for the root module back when
it was the only module included in `sources.tar`, and that made sense.
Now, we get the best of both worlds, using the proper root name for the
root module while using the module name for the rest.
This makes the host http header have the port if and only if it differs
from the defaults based on the protocol.
This is an alternate implementation that closes#19624.
