Commit Graph

47 Commits

Author SHA1 Message Date
Andrew Kelley
766b315b38 std.GeneralPurposeAllocator: logging improvements
It now uses the log scope "gpa" instead of "std".

Additionally, there is a new config option `verbose_log` which enables
info log messages for every allocation. Can be useful when debugging.
This option is off by default.
2021-01-30 20:15:26 -07:00
Andrew Kelley
9698ea3173 std.Thread.Mutex: restore the "Held" API
so that std.Thread.Mutex.Dummy can be used as a drop in replacement.
2021-01-14 21:28:22 -07:00
Andrew Kelley
a9667b5a85 organize std lib concurrency primitives and add RwLock
* move concurrency primitives that always operate on kernel threads to
   the std.Thread namespace
 * remove std.SpinLock. Nobody should use this in a non-freestanding
   environment; the other primitives are always preferable. In
   freestanding, it will be necessary to put custom spin logic in there,
   so there are no use cases for a std lib version.
 * move some std lib files to the top level fields convention
 * add std.Thread.spinLoopHint
 * add std.Thread.Condition
 * add std.Thread.Semaphore
 * new implementation of std.Thread.Mutex for Windows and non-pthreads Linux
 * add std.Thread.RwLock

Implementations provided by @kprotty
2021-01-14 20:41:37 -07:00
Jay Petacat
1595ce273e Remove deprecated stream aliases 2021-01-08 16:54:56 -05:00
Jay Petacat
a9b505fa77 Reduce use of deprecated IO types
Related: #4917
2021-01-07 23:48:58 -08:00
LemonBoy
dd973fb365 std: Use {s} instead of {} when printing strings 2021-01-02 17:12:57 -07:00
Frank Denis
6c2e0c2046 Year++ 2020-12-31 15:45:24 -08:00
Vexu
7e30e83900
small fixes and zig fmt 2020-12-09 13:54:26 +02:00
Nathanaël Courant
81890e30e2 Fix general purpose allocator incorrectly modifying total_requested_bytes in some cases 2020-12-08 15:53:34 -05:00
Isaac Yonemoto
a2cd9dc3bd
Gpa mutex configurable (#7234)
* makes the mutex for the gpa configurable

* fixed logic, added test

* updates docstring; pushes logic to one place, better duck-type
2020-12-03 15:49:35 -05:00
xackus
5c8f7f81cd change debug.assert to testing.expect in tests 2020-11-06 22:48:54 +01:00
Jan Prudil
aadccc4206 Make std.meta.Int accept a signedness parameter 2020-10-17 14:09:59 +02:00
LemonBoy
bd9003ed5b std: ArenaAllocator tries to resize before allocating
Closes #5116
2020-09-25 10:58:07 +02:00
Mark Barbone
42b1b6be90 Add resize for arena allocator 2020-09-08 13:04:14 -04:00
Sahnvour
575fbd5e35 hash_map: rename to ArrayHashMap and add new HashMap implementation 2020-09-02 00:17:50 +02:00
LemonBoy
29de809a92 gpa: Don't leak memory when the upper bound is hit 2020-08-31 12:35:25 +02:00
LemonBoy
f20305d249 gpa: Fix bookkeeping logic
The backing allocator may return a block that's actually bigger than the
one required by the user, use the correct quantity when keeping track of
the allocation ceiling.

Closes #6049
2020-08-29 20:51:30 +02:00
Andrew Kelley
973e6c978c std: clean up GeneralPurposeAllocator memset code
The freeSlot function was only called once so I inlined the logic and
utilized some of the other locals that were in scope.
2020-08-25 19:48:39 -07:00
Andrew Kelley
6fb105fdd7 std: GeneralPurposeAllocator: set freed bytes to undefined
Helps catch use-after-free. Caught a couple issues in the self-hosted
compiler.
2020-08-25 13:36:40 -07:00
Andrew Kelley
4a69b11e74 add license header to all std lib files
add SPDX license identifier
copyright ownership is zig contributors
2020-08-20 16:07:04 -04:00
Andrew Kelley
583b843803 std.heap.GeneralPurposeAllocator: add never_unmap config option
This is a temporary debugging trick you can use to turn segfaults into more helpful
logged error messages with stack trace details. The downside is that every allocation
will be leaked!
2020-08-18 15:09:48 -07:00
heidezomp
16d118a8d9 update std and src-self-hosted for std.log breaking change 2020-08-13 17:14:15 +02:00
Andrew Kelley
60ce5edaf9 GeneralPurposeAllocator: default to store more stack frames in test mode
Now tests will by default store 8 stack frames per allocation rather
than the normal default of 4.
2020-08-11 14:05:28 -07:00
Andrew Kelley
20510d209b GeneralPurposeAllocator: use std.log instead of std.debug.print
`std.builtin.StackTrace` gains a `format` function.

GeneralPurposeAllocator uses `std.log.err` instead of directly printing
to stderr. Some errors are recoverable.

The test runner is modified to fail the test run if any log messages of
"err" or worse severity are encountered.

self-hosted is modified to always print log messages of "err" severity
or worse even if they have not been explicitly enabled.

This makes GeneralPurposeAllocator available on the freestanding target.
2020-08-11 02:01:32 -04:00
Andrew Kelley
f98cffc615 std: general purpose allocator: use AutoHashMap
As pointed out by Sahnvour, AutoHashMap is both more convenient and will
have better performance in this case.
2020-08-08 15:59:03 -07:00
Andrew Kelley
5b57e35ce0 fix general purpose allocator test cases on Windows
The tests are cleverly testing some alignment stuff, but were getting
thwarted by Windows choosing to allocate 64K aligned pages.
2020-08-08 13:46:18 -07:00
Andrew Kelley
4d0f83e23e GeneralPurposeAllocator: naming convention refactor 2020-08-08 13:05:04 -07:00
Andrew Kelley
cb9405cdbd don't collect stack trace frames in release safe mode by default
We don't pass no-omit-frame-pointer in release safe by default, so it
also makes sense to not try to collect stack trace frames by default in
release safe mode.
2020-08-08 12:04:19 -07:00
Andrew Kelley
051aadd781 std lib general purpose allocator: disable stack tracing on mips
Sadly, trying to collect stack frames goes into an infinite loop on
mips. This sets the default number of stack frames to collect to 0 on
mips.
2020-08-08 02:38:32 -07:00
Andrew Kelley
1b1921f0e2 stage1: deal with WebAssembly not supporting @returnAddress()
This makes `@returnAddress()` return 0 for WebAssembly (when not using
the Emscripten OS) and avoids trying to capture stack traces for the
general purpose allocator on that target.
2020-08-08 01:00:29 -07:00
Andrew Kelley
9f5a7d5922 utilize math.ceilPowerOfTwo 2020-08-08 00:39:46 -07:00
Andrew Kelley
cd6cdd0a75 std.mem.Allocator: add return_address to the interface
The high level Allocator interface API functions will now do a
`@returnAddress()` so that stack traces captured by allocator
implementations have a return address that does not include the
Allocator overhead functions. This makes `4` a more reasonable default
for how many stack frames to capture.
2020-08-08 00:34:13 -07:00
Andrew Kelley
0347df82e8 improvements & fixes for general purpose allocator integration
* std.Mutex API is improved to not have init() deinit(). This API is
   designed to support static initialization and does not require any
   resource cleanup. This also happens to work around some kind of
   stage1 behavior that wasn't letting the new allocator mutex code
   get compiled.
 * the general purpose allocator now returns a bool from deinit()
   which tells if there were any leaks. This value is used by the test
   runner to fail the tests if there are any.
 * self-hosted compiler is updated to use the general purpose allocator
   when not linking against libc.
2020-08-07 23:26:58 -07:00
Andrew Kelley
cc17f84ccc std: introduce GeneralPurposeAllocator
`std.GeneralPurposeAllocator` is now available. It is a function that
takes a configuration struct (with default field values) and returns an
allocator. There is a detailed description of this allocator in the
doc comments at the top of the new file.

The main feature of this allocator is that it is *safe*. It
prevents double-free, use-after-free, and detects leaks.

Some deprecation compile errors are removed.

The Allocator interface gains `old_align` as a new parameter to
`resizeFn`. This is useful to quickly look up allocations.

`std.heap.page_allocator` is improved to use mmap address hints to avoid
obtaining the same virtual address pages when unmapping and mapping
pages. The new general purpose allocator uses the page allocator as its
backing allocator by default.

`std.testing.allocator` is replaced with usage of this new allocator,
which does leak checking, and so the LeakCheckAllocator is retired.

stage1 is improved so that the `@typeInfo` of a pointer has a lazy value
for the alignment of the child type, to avoid false dependency loops
when dealing with pointers to async function frames.

The `std.mem.Allocator` interface is refactored to be in its own file.

`std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`.

This allocator is great for debug mode, however it needs some work to
have better performance in release modes. The next step will be setting
up a series of tests in ziglang/gotta-go-fast and then making
improvements to the implementation.
2020-08-07 22:45:45 -07:00
Vexu
e85fe13e44
run zig fmt on std lib and self hosted 2020-07-11 20:41:19 +03:00
Jonathan Marler
67e97a1f0f ArenaAllocator: use full capacity 2020-06-29 05:12:30 -04:00
Jonathan Marler
35e8876c23 Revert "arena_allocator: refactor and use full capacity"
This reverts commit e120b07a52.
2020-06-29 05:12:30 -04:00
Jonathan Marler
e120b07a52 arena_allocator: refactor and use full capacity 2020-06-28 18:40:15 -04:00
Jonathan Marler
a728436992 new allocator interface after Andrew Kelley review 2020-06-27 08:57:35 -06:00
Jonathan Marler
dc9648f868 new allocator interface 2020-06-26 13:34:48 -06:00
Andrew Kelley
ad4b591db1 small bump to ArenaAllocator minimum alloc size
self-hosted parser benchmark:
throughput: 69.4 MiB/s => 72.2 MiB/s
2020-05-20 16:30:28 -04:00
Andrew Kelley
b1bcdc96ca arena allocator: smaller minimum allocation size 2020-05-20 16:11:55 -04:00
Andrew Kelley
a32d3a85d2 rework self-hosted compiler for incremental builds
* introduce std.ArrayListUnmanaged for when you have the allocator
   stored elsewhere
 * move std.heap.ArenaAllocator implementation to its own file. extract
   the main state into std.heap.ArenaAllocator.State, which can be
   stored as an alternative to storing the entire ArenaAllocator, saving
   24 bytes per ArenaAllocator on 64 bit targets.
 * std.LinkedList.Node pointer field now defaults to being null
   initialized.
 * Rework self-hosted compiler Package API
 * Delete almost all the bitrotted self-hosted compiler code. The only bit
   rotted code left is in main.zig and compilation.zig
 * Add call instruction to ZIR
 * self-hosted compiler ir API and link API are reworked to support
   a long-running compiler that incrementally updates declarations
 * Introduce the concept of scopes to ZIR semantic analysis
 * ZIR text format supports referencing named decls that are declared
   later in the file
 * Figure out how memory management works for the long-running compiler
   and incremental compilation. The main roots are top level
   declarations. There is a table of decls. The key is a cryptographic
   hash of the fully qualified decl name. Each decl has an arena
   allocator where all of the memory related to that decl is stored.
   Each code block has its own arena allocator for the lifetime of
   the block. Values that want to survive when going out of scope in
   a block must get copied into the outer block. Finally, values must
   get copied into the Decl arena to be long-lived.
 * Delete the unused MemoryCell struct. Instead, comptime pointers are
   based on references to Decl structs.
 * Figure out how caching works. Each Decl will store a set of other
   Decls which must be recompiled when it changes.

This branch is still work-in-progress; this commit breaks the build.
2020-05-10 02:05:54 -04:00
Andrew Kelley
18f1fef142
update standard library to new I/O streams API 2020-03-10 18:44:30 -04:00
Benjamin Feng
46d84a1b63 Convert a few more page_allocator 2020-01-30 00:27:44 -06:00
daurnimator
d7333d8798 std: fix LoggingAllocator, add simple test 2020-01-09 13:34:46 -05:00
Andrew Kelley
ed36dbbd9c
mv std/ lib/
that's all this commit does. further commits will fix cli flags and
such.

see #2221
2019-09-25 23:35:41 -04:00