Commit Graph

53 Commits

Author SHA1 Message Date
Andrew Kelley
d56a65a8c4 std.http.Client: default to lazy root cert scanning
After this change, the system will be inspected for root certificates
only upon the first https request that actually occurs. This makes the
compiler no longer do SSL certificate scanning when running `zig build`
if no network requests are made.
2023-01-17 01:44:56 -05:00
Andrew Kelley
476cbe871a fix build failures on 32-bit arm due to u64/usize coercion 2023-01-11 15:39:49 -08:00
Andrew Kelley
c50f65304f std.http.Client: support the Reader interface 2023-01-11 15:39:48 -08:00
Andrew Kelley
aa87789c29 std.Uri: make scheme non-optional 2023-01-06 18:52:39 -07:00
Andrew Kelley
646a911c19 std.http.Client: update from old std.Url to new std.Uri 2023-01-06 18:05:37 -07:00
Andrew Kelley
3806091a10 std.http.Client: fix handling of \r\n before next chunk size 2023-01-06 17:53:06 -07:00
Andrew Kelley
a7a933d7ee std.http.Client: support transfer-encoding: chunked
closes #14204

In order to add tests for this I need to implement an HTTP server in the
standard library (#910) so that's probably the next thing I'll do.
2023-01-05 19:57:00 -07:00
Andrew Kelley
3055ab7f86 std.http.Client: fail header parsing under more conditions
* when HTTP header continuations are used
 * when content-type or location header occurs more than once
2023-01-05 13:39:17 -07:00
Andrew Kelley
ba1e53f116 avoid triggering LLVM bug on MIPS
See #13782
2023-01-05 00:03:59 -07:00
Andrew Kelley
8248fdbbdb std.http.Client: support HTTP redirects
* std.http.Status.Class: add a "nonstandard" enum tag. Instead of
   having `class` return an optional value, it can potentially return
   nonstandard.
 * extract out std.http.Client.Connection from std.http.Client.Request
   - this code abstracts over plain/TLS only
   - this is the type that will potentially be stored in a client's LRU
     connection map
 * introduce two-staged HTTP header parsing
   - API users can rely on a heap-allocated buffer with a maximum limit,
     which defaults to 16 KB, or they can provide a static buffer that
     is borrowed by the Request instance.
   - The entire HTTP header is buffered because there are strings in
     there and they must be accessed later, such as with the case of
     HTTP redirects.
   - When buffering the HTTP header, the parser only looks for the
     \r\n\r\n pattern. Further validation is done later.
   - After the full HTTP header is buffered, it is parsed into
     components such as Content-Length and Location.
 * HTTP redirects are handled, with a maximum redirect count option that
   defaults to 3.
   - Connection: close is always used for now; implementing keep-alive
     connections and an LRU connection pool in std.http.Client is a task
     for another day.

see #2007
2023-01-04 18:37:53 -07:00
Andrew Kelley
5d9429579d std.http.Headers.Parser: parse version and status 2023-01-04 18:37:53 -07:00
Andrew Kelley
2cdc0a8b50 std.http.Client: do not heap allocate for requests 2023-01-04 18:37:53 -07:00
Andrew Kelley
7178451d62 std.crypto.tls.Client: make close_notify optional
Although RFC 8446 states:

> Each party MUST send a "close_notify" alert before closing its write
> side of the connection

In practice many servers do not do this. Also in practice, truncation
attacks are thwarted at the application layer by comparing the amount of
bytes received with the amount expected via the HTTP headers.
2023-01-02 18:27:38 -07:00
Andrew Kelley
3127bd79fb std.http.Client: don't send TLS close_notify
It appears to be implemented incorrectly in the wild and causes the read
connection to be closed even though that is a direct violation of RFC
8446 Section 6.1.

The writeEnd function variants are still there, ready to be used.
2023-01-02 16:57:16 -07:00
Andrew Kelley
611a1fdd6d std.crypto.tls: add API for sending close_notify
This commit adds `writeEnd` and `writeAllEnd` in order to send data and
also notify the server that there will be no more data written.

Unfortunately, it seems most TLS implementations in the wild get this
wrong and immediately close the socket when they see a close_notify,
rather than only ending the data stream on the application layer.
2023-01-02 16:57:16 -07:00
Andrew Kelley
2d090f61be add std.http.Headers
This is a streaming HTTP header parser. All it currently does is detect
the end of headers. This will be a non-allocating parser where one can
bring supply their own buffer if they want to handle custom headers.

This commit also improves std.http.Client to not return the HTTP headers
with the read functions.
2023-01-02 16:57:16 -07:00
Andrew Kelley
940d368e7e std.crypto.tls.Client: fix the read function
The read function has been renamed to readAdvanced since it has slightly
different semantics than typical read functions, specifically regarding
the end-of-file. A higher level read function is implemented on top.

Now, API users may pass small buffers to the read function and
everything will work fine. This is done by re-decrypting the same
ciphertext record with each call to read() until the record is finished
being transmitted.

If the buffer supplied to read() is large enough, then any given
ciphertext record will only be decrypted once, since it decrypts
directly to the read() buffer and therefore does not need any memcpy. On
the other hand, if the buffer supplied to read() is small, then the
ciphertext is decrypted into a stack buffer, a subset is copied to the
read() buffer, and then the entire ciphertext record is saved for the
next call to read().
2023-01-02 16:57:16 -07:00
Andrew Kelley
477864dca5 std.crypto.tls.Client: fix truncation attack vulnerability 2023-01-02 16:57:16 -07:00
Andrew Kelley
5b8b5f2505 add url parsing to the std lib 2023-01-02 16:57:15 -07:00
Andrew Kelley
29475b4518 std.crypto.tls: validate previous certificate 2023-01-02 16:57:15 -07:00
Andrew Kelley
bbc074252c introduce std.crypto.CertificateBundle
for reading root certificate authority bundles from standard
installation locations on the file system. So far only Linux logic is
added.
2023-01-02 16:57:15 -07:00
Andrew Kelley
93ab8be8d8 extract std.crypto.tls.Client into separate namespace 2023-01-02 16:57:15 -07:00
Andrew Kelley
b97fc43baa std.crypto.Tls: client is working against some servers 2023-01-02 16:57:15 -07:00
Andrew Kelley
40a85506b2 std.crypto.Tls: add read/write methods 2023-01-02 16:57:15 -07:00
Andrew Kelley
d2f5d0b199 std.crypto.Tls: parse the ServerHello handshake 2023-01-02 16:57:15 -07:00
Andrew Kelley
ba44513c2f std.http reorg; introduce std.crypto.Tls
TLS is capable of sending a Client Hello
2023-01-02 16:57:15 -07:00
Ali Chraghi
11dce78944 std.http: fix typo 2022-10-06 21:22:20 +03:00
Meghan
3a64693db3
std: add http definitions for Method and Status (#10661) 2022-05-11 15:43:18 -04:00
Andrew Kelley
402f967ed5 move std.http to the standard library orphanage
I want to take the design of this in a different direction. I think this
abstraction is too high level. I want to start bottom-up.

std-lib-orphanage commit 179ae67d61455758d71037434704fd4a17a635a9
2020-09-29 17:40:37 -07:00
Sahnvour
575fbd5e35 hash_map: rename to ArrayHashMap and add new HashMap implementation 2020-09-02 00:17:50 +02:00
Andrew Kelley
4a69b11e74 add license header to all std lib files
add SPDX license identifier
copyright ownership is zig contributors
2020-08-20 16:07:04 -04:00
daurnimator
91235b9371
std: don't store allocator inside of std.http.HeaderEntry 2020-07-13 00:38:59 +10:00
daurnimator
f7e4014c82
std: use *Unmanaged data structures in http.Headers object 2020-07-13 00:34:58 +10:00
Vexu
e85fe13e44
run zig fmt on std lib and self hosted 2020-07-11 20:41:19 +03:00
Andrew Kelley
289eab9177
Merge pull request #5786 from ziglang/std-hash-map
reimplement std.HashMap
2020-07-05 21:12:20 +00:00
Andrew Kelley
632acffcbd update std lib to new hash map API 2020-07-05 21:11:42 +00:00
joachimschmidt557
0ae1157e45 std.mem.dupe is deprecated, move all references in std
Replaced all occurences of std.mem.dupe in stdlib with
Allocator.dupe/std.mem.dupeZ -> Allocator.dupeZ
2020-07-04 21:40:06 +03:00
Andrew Kelley
d4d954abd2 std.sort: give comparator functions a context parameter 2020-06-08 15:16:40 -04:00
xackus
dbc00e2424 ArrayList: remove old (before span) API 2020-04-11 20:40:34 -04:00
xackus
7a28c644aa new ArrayList API: fix everything else 2020-04-02 16:12:08 +02:00
Andrew Kelley
9e7ae06249
std lib API deprecations for the upcoming 0.6.0 release
See #3811
2020-03-30 14:23:22 -04:00
Benjamin Feng
4aae55b4cc Replace fmt with new fmtstream 2020-03-12 10:41:09 -05:00
Benjamin Feng
c11d1055b8 Integrated outstreams with new formatter 2020-03-12 10:26:28 -05:00
Benjamin Feng
699c50a375 Switch a bunch of FBA to use testing.allocator 2020-02-12 17:17:56 -06:00
daurnimator
65013d8599
std: fix bug in http.headers where .put captures user-held variable 2020-01-21 03:17:36 +11:00
Andrew Kelley
5575e2a168
std.mem.compare: breaking API changes
* `std.mem.Compare` is now `std.math.Order` and the enum tags
   renamed to follow new style convention.
 * `std.mem.compare` is renamed to `std.mem.order`.
 * new function `std.math.order`
2020-01-01 18:08:40 -05:00
Andrew Kelley
c3d8b1ffeb
remove iterator API from std.ArrayList
This is not a meaningful abstraction. Use a for loop on the result
of `toSlice` or `toSliceConst`.

An iterator can be implemented on top of ArrayList by applications which
want additional functionality, such as removing elements while
iterating.

Closes #3037.
2019-12-10 15:08:10 -05:00
Robin Voetter
4b4fbe3887
Replace @typeOf with @TypeOf in all zig source
This change was mostly made with `zig fmt` and this also modified some whitespace. Note that in some files, `zig fmt` produced incorrect code, so the change was made manually.
2019-12-10 11:09:41 -05:00
Andrew Kelley
8b2622cdd5
std.fmt.format: tuple parameter instead of var args 2019-12-08 22:53:51 -05:00
Andrew Kelley
bf3ac66150
remove type coercion from array values to references
* Implements #3768. This is a sweeping breaking change that requires
   many (trivial) edits to Zig source code. Array values no longer
   coerced to slices; however one may use `&` to obtain a reference to
   an array value, which may then be coerced to a slice.

 * Adds `IrInstruction::dump`, for debugging purposes. It's useful to
   call to inspect the instruction when debugging Zig IR.

 * Fixes bugs with result location semantics. See the new behavior test
   cases, and compile error test cases.

 * Fixes bugs with `@typeInfo` not properly resolving const values.

 * Behavior tests are passing but std lib tests are not yet. There
   is more work to do before merging this branch.
2019-11-27 03:37:50 -05:00