Commit Graph

1449 Commits

Author SHA1 Message Date
Mark Johnston
a61d2c7fbd dtrace tests: Diff expected and actual output upon a miscompare
This makes quick diagnosis of test failures easier.

MFC after:	1 week
2024-01-10 18:19:36 -05:00
Mark Johnston
ba10366037 dtrace tests: Catch up with ping(8) output changes
MFC after:	1 week
2024-01-10 18:19:00 -05:00
Mark Johnston
66bfb4b2ee dtrace tests: Require perl
A number of tests use it, so just require it globally as we do with
other tools.

MFC after:	1 week
2024-01-10 18:18:27 -05:00
Mark Johnston
9f5dd61e14 dtrace tests: Stop hard-coding an incorrect path to sleep(1)
MFC after:	1 week
2024-01-10 18:18:01 -05:00
Mark Johnston
e25922b34e dtrace tests: Run ksh with -p
In particular, avoid loading the user's .profile file, since that can
have undesirable side effects.  Most tests were already careful to do
this.

MFC after:	1 week
2024-01-10 18:17:59 -05:00
Domagoj Stolfa
93f27766a7 dtrace: Add the 'oformat' libdtrace option
This option can be used to specify a format to use in DTrace output.
The following formats are supported:
 - json
 - xml
 - html
 - none (default DTrace output)

This is implemented using libxo and integrated into libdtrace. Client
code only works with the following API:

 - dtrace_oformat_setup(dtrace_hdl_t *) -- to be called when output is starting.
 - dtrace_oformat_teardown(dtrace_hdl_t *) -- to be called when output is finished
 - dtrace_oformat(dtrace_hdl_t *) -- check if oformat is enabled.
 - dtrace_set_outfp(FILE *) -- sets the output file for oformat.
 - Ensure that oformat is correctly checked in the drop handler and record
   processing callbacks.

This commit also adds tests which check if the generated output is
valid (JSON, XML) and extends the dtrace(1) describing the structured output.

Reviewed by:	markj
Discussed with:	phil
MFC after:	2 months
Sponsored by:	Innovate UK
Differential Revision:	https://reviews.freebsd.org/D41745
2024-01-10 18:14:26 -05:00
Mark Johnston
78cd75393e ctfconvert: Handle DW_AT_data_bit_offset
This attribute is new in DWARF 4 and supersedes DW_AT_bit_offset.

PR:		276059
Reported by:	rscheff
Tested by:	rscheff
MFC after:	1 week
2024-01-01 23:12:29 -05:00
Mark Johnston
c3268c23de libdtrace: Fix TCP data offset handling in the tcpinfo_t translator
The header gives an offset in 32-bit words, and the translator is
supposed to convert that to a byte count.  But, the conversion was
incorrect.

Reviewed by:	tuexen, rscheff
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D43264
2024-01-01 18:03:54 -05:00
Richard Scheffenegger
38c63bdc46 libdtrace: decode all tcp header flags and add
decoding capability of TH_AE to dtrace, including
the example provided with tcpdebug.

MFC after:             1 week
Reviewed By:           markj, mav
Sponsored by:          NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D43243
2023-12-31 15:02:03 +01:00
Mark Johnston
ad3174ecea libdtrace: Fix line number reporting in error messages
MFC after:	1 week
2023-12-31 12:47:30 -05:00
Dimitry Andric
bc6051a060 libicp(_rescue)?: use asm versions for all 32-bit arm flavors
OpenZFS upstream refactored the asm versions of sha2 to be usable on all
32-bit arm flavors, so it is not necessary to limit this to armv6 and
armv7.

Suggested by:	jhb
2023-12-13 20:00:35 +01:00
Brooks Davis
8ccd0b876e libc: expose execvpe for Linux compat
We already implemented execvpe internally with an _ prefix in libc so
go ahead and expose it for compatibility with Linux.

This reverts c605eea952.

Bump __FreeBSD_version for the addition and add definitions to supress
compat shims in libzfs (zfs changes were merged from upstream).

PR:		275370 (request and exp-run (thanks antoine!))
Reviewed by:	kevans
Differential Revision:	https://reviews.freebsd.org/D42846
2023-12-11 19:24:53 +00:00
Dimitry Andric
1a4a9a5057 libicp_rescue: unbreak for armv6 after recent OpenZFS import
Similar to fb17dfa0c8, fix libicp_rescue to include asm versions of
sha2 on armv6, to unbreak the build of rescue.

Fixes:		3494f7c019
2023-12-09 12:51:50 +01:00
Dimitry Andric
fb17dfa0c8 libicp: unbreak for armv6 after recent OpenZFS import
The following upstream commit:
727497ccdf module/icp/asm-arm/sha2: enable non-SIMD asm kernels on armv5/6

does indeed enable sha2 asm for earlier arm CPUs, but since libicp's
Makefile was not updated, this leads to:

  ld: error: undefined reference due to --no-allow-shlib-undefined: zfs_sha256_block_armv7

Fix it by compiling sha256-armv7.S and sha512-armv7.S for armv6 too.

Fixes:		3494f7c019
2023-12-09 00:09:50 +01:00
Alan Somers
d565784a7e zfsd: fault disks that generate too many I/O delay events
If ZFS reports that a disk had at least 8 I/O operations over 60s that
were each delayed by at least 30s (implying a queue depth > 4 or I/O
aggregation, obviously), fault that disk.  Disks that respond this
slowly can degrade the entire system's performance.

MFC after:	2 weeks
Sponsored by:	Axcient
Reviewed by:	delphij
Differential Revision: https://reviews.freebsd.org/D42825
2023-11-29 07:51:11 -07:00
Alexander Motin
2ac9cecac6 Fix typo in previous d282baddb0, breaking DTrace. 2023-11-17 12:42:33 -05:00
Alexander Motin
d282baddb0 Add interface NVME to devstat
This allows to list only NVMe devices in systat, iostat, vmstat, etc.
Previously those were counted as OTHER.
2023-11-15 23:03:40 -05:00
Martin Matuska
e716630d4c zfs: merge openzfs/zfs@887a3c533
Notable upstream pull request merges:
 #15022 5caeef02f RAID-Z expansion feature
 #15457 887a3c533 Increase L2ARC write rate and headroom
 #15504 1c1be60fa Unbreak FreeBSD world build after 3bd4df384

Obtained from:	OpenZFS
OpenZFS commit:	887a3c533b
2023-11-09 13:19:17 +01:00
Stephen J. Kiernan
5e3190f700 dirdeps: Update Makefile.depend* files with empty contents
Some Makefile.depend* files were committed with no contents or empty
DIRDEPS list, but they should have DIRDEPS with some contents.
2023-10-29 17:01:04 -04:00
John Baldwin
f53355131f Trim various $FreeBSD$
Approved by:	markj (cddl/contrib changes)
Reviewed by:	imp, emaste
Differential Revision:	https://reviews.freebsd.org/D41961
2023-10-10 10:34:43 -07:00
Martin Matuska
abcdc1b9ae zfs: merge openzfs/zfs@2e2a46e0a
Notable upstream pull request merges:
 #15243 b53077a9e Add zfs_prepare_disk script for disk firmware install
 #15298 2e2a46e0a Invoke zdb by guid to avoid import errors
 #15301 e5d70f467 ZIL: Avoid dbuf_read() in ztest_get_data()

Obtained from:	OpenZFS
OpenZFS commit:	2e2a46e0a5
2023-09-24 23:58:34 +02:00
Martin Matuska
7a0e9e3f8f zfs: update share/zfs/compatibility.d to match current ZFS code
MFC after:	3 days
2023-09-22 01:37:38 +02:00
Alan Somers
0b294a386d Fix zfsd with the device_removal pool feature.
Previously zfsd would crash in the presence of a pool with a
top-level-vdev that had previously been removed.  The crash happened
because the configuration nvlist of such a TLV contains an empty
ZPOOL_CONFIG_CHILDREN array, which led to a pop_front from an empty
list, which has undefined behavior.

The crash only happened in stable/14 and later, probably do to
differences in libcxx, but the change should be MFCed anyway.

PR:		273663
Reported by:	Marek Zarychta <zarychtam@plan-b.pwste.edu.pl>
MFC after:	1 week
Sponsored by:	Axcient
Reviewed by:	mav
Differential Revision: https://reviews.freebsd.org/D41818
2023-09-12 08:46:12 -06:00
Martin Matuska
315ee00fa9 zfs: merge openzfs/zfs@804414aad
Notable upstream pull request merges:
  #15024 Add missed DMU_PROJECTUSED_OBJECT prefetch
  #15029 Do not request data L1 buffers on scan prefetch
  #15036 FreeBSD: catch up to __FreeBSD_version 1400093
  #15039 Fix raw receive with different indirect block size
  #15047 FreeBSD: Fix build on stable/13 after 1302506
  #15049 Fix the ZFS checksum error histograms with larger record sizes
  #15052 Reduce bloat in ereport.fs.zfs.checksum events
  #15056 Avoid extra snprintf() in dsl_deadlist_merge()
  #15061 Ignore pool ashift property during vdev attachment
  #15063 Don't panic if setting vdev properties is unsupported for this vdev type
  #15067 spa_min_alloc should be GCD, not min
  #15071 Add explicit prefetches to bpobj_iterate()
  #15072 Adjust prefetch parameters
  #15076 Refactor dmu_prefetch()
  #15079 set autotrim default to 'off' everywhere
  #15080 ZIL: Fix config lock deadlock
  #15088 metaslab: tuneable to better control force ganging
  #15096 Avoid waiting in dmu_sync_late_arrival()
  #15097 BRT should return EOPNOTSUPP
  #15103 Remove zl_issuer_lock from zil_suspend()
  #15107 Remove fastwrite mechanism
  #15113 libzfs: sendrecv: send_progress_thread: handle SIGINFO/SIGUSR1
  #15122 ZIL: Second attempt to reduce scope of zl_issuer_lock
  #15129 zpool_vdev_remove() should handle EALREADY error return
  #15132 ZIL: Replay blocks without next block pointer
  #15148 zfs_clone_range should return descriptive error codes
  #15153 ZIL: Avoid dbuf_read() before dmu_sync()
  #15172 copy_file_range: fix fallback when source create on same txg
  #15180 Update outdated assertion from zio_write_compress

Obtained from:	OpenZFS
OpenZFS commit:	804414aad2
2023-08-26 23:51:42 +02:00
Domagoj Stolfa
7a3af0a59b dtrace: Remove $FreeBSD$ from genmakefiles.sh.
$FreeBSD$ is gone in head, so this should be removed as well.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D41595
2023-08-25 14:55:59 -06:00
Warner Losh
fa9896e082 Remove $FreeBSD$: two-line nroff pattern
Remove /^\.\\"\n\.\\"\s*\$FreeBSD\$$\n/
2023-08-16 11:55:10 -06:00
Warner Losh
d0b2dbfa0e Remove $FreeBSD$: one-line sh pattern
Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/
2023-08-16 11:55:03 -06:00
Warner Losh
1d386b48a5 Remove $FreeBSD$: one-line .c pattern
Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/
2023-08-16 11:54:42 -06:00
Warner Losh
2a63c3be15 Remove $FreeBSD$: one-line .c comment pattern
Remove /^/[*/]\s*\$FreeBSD\$.*\n/
2023-08-16 11:54:29 -06:00
Warner Losh
42b388439b Remove $FreeBSD$: one-line .h pattern
Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/
2023-08-16 11:54:23 -06:00
Warner Losh
b3e7694832 Remove $FreeBSD$: two-line .h pattern
Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
2023-08-16 11:54:16 -06:00
Domagoj Stolfa
6de0222840 dtrace: Add WITH_DTRACE_ASAN
See commit 4ae6991228.  This version of
the commit avoids inadvertently changing SHLIBDIR for libdtrace.so.
2023-08-02 09:15:08 -04:00
Eric van Gyzen
1112883e19 dtrace: remove dead code for PR_REQUESTED
libproc's PR_REQUESTED is not implemented on FreeBSD.  Remove dead code
in dtrace that would handle it.

Reviewed by:	markj
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D41225
2023-08-01 12:28:50 -05:00
Eric van Gyzen
bcbce2a29b dtrace: do not overload libproc flags
dtrace stored its PR_RLC and PR_KLC flags in the proc_handle's flags,
where they collided with PATTACH_FORCE and PATTACH_RDONLY, respectively.
Thus, Psetflags(PR_KLC) effectively also set the PATTACH_RDONLY flag.

Since the flags are private to dtrace (at least on FreeBSD), store them in
dtrace's own dt_proc structure instead.

On FreeBSD, either PR_RLC or PR_KLC was always set, so remove code that
would handle the case where neither was set.

Reviewed by:	markj
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D41121
2023-08-01 12:28:50 -05:00
Eric van Gyzen
837f6ecd88 dtrace: remove illumos code from dt_proc.c
The illumos #ifdef's in this file make it harder to read and maintain.
There is little change upstream, and increasing changes for FreeBSD.
Remove the illumos code with `unifdef`.  The only manual changes here
are the #includes and #defines at the top, and removing blank lines.

Reviewed by:	markj
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D41173
2023-08-01 12:28:50 -05:00
Mark Johnston
5cdaac92e1 dtrace: Revert the addition of WITH_DTRACE_ASAN
The follow-up fix triggers a lib32 build failure, revert everything
until the problem is addressed.
2023-08-01 13:03:10 -04:00
Mark Johnston
848ff9bc1b libdtrace: Explicitly set SHLIBDIR and SHLIB_MAJOR
They were previously being defined by cddl/lib/Makefile.inc, and as of
commit 4ae6991228 were being overridden by defaults in bsd.own.mk,
which changed SHLIBDIR to /usr/lib.

Reported by:	Domagoj Stolfa <ds815@cam.ac.uk>
Fixes:		4ae6991228 ("dtrace: Add WITH_DTRACE_ASAN")
2023-08-01 10:11:42 -04:00
Domagoj Stolfa
4ae6991228 dtrace: Add WITH_DTRACE_ASAN
This option is a blanket for all the DTrace-related software. The option
when enabled passes in -fsanitize=address -fsanitize=undeifned, enabling
ASAN and UBSAN in the following components:

 - libdtrace
 - dtrace(1)
 - lockstat(1)
 - plockstat(1)

The option defaults to "no" and is intended as a developer aid.

Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D41157
2023-07-27 16:07:34 -04:00
Alan Somers
2a0c0aea42 Fix zfsd unittests after 92642bba4d
At the time we ensured that the more important functional tests were
working, but neglected to update the unit tests.

MFC after:	2 weeks
MFC with:	92642bba4d
Sponsored by:	Axcient
2023-07-14 15:57:40 -06:00
Eric van Gyzen
dad11f990e dtrace: prevent forked child from running after an error condition
The pid/killonerror test uses an invalid probe specifier to verify that
the child process is killed.  It occasionally fails because the "date"
command is allowed to run long enough to print the date.  This is harmless
in this case, but is clearly not ideal.

When the dt_proc_control thread is about to exit, and the dtrace command
forked the child, do not make the child runnable.

Reviewed by:	markj
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D40976
2023-07-12 10:33:28 -05:00
Alan Somers
92642bba4d zfsd: listen for sysevent.fs.zfs instead of misc.fs.zfs
At some point the names of these devd events changed.  Probably it
happened when importing OpenZFS.  Before that, FreeBSD's sysevent_alloc
method didn't create a "class" nvpair in the event, which led to
log_sysevent using the event's ev_subclass field as its type.

MFC after:	2 weeks
Sponsored by:	Axcient
Differential Revision: https://reviews.freebsd.org/D39437
2023-07-10 16:36:57 -06:00
Christos Margiolis
911f026039 dtrace: move kinst tests to common
Reviewed by:	markj
Approved by:	markj (mentor)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D40414
2023-07-04 18:46:59 +03:00
Michael Tuexen
2176c9ab71 dtrace: improve siftr probe
Improve consistency of the field names with tcpsinfo_t:
* Use mss instead of max_seg_size.
* Use lport and rport instead of tcp_localport and tcp_foreignport.

Use t_flags instead of flags to improve consistency with t_flags2.

Add laddr and raddr, since the addresses were missing when compared
to the output of siftr.

Reviewed by:		cc
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D40834
2023-07-02 03:08:51 +02:00
Michael Tuexen
f85b82daf1 dtrace: third update to the siftr probe
This adds the changes corresponding to
https://cgit.freebsd.org/src/commit/?id=60167184abd54ca12c6bf7ab60f2a08b41342f84
to the siftr probe.
This concludes the series up updates.

Reviewed by:		rscheff
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D40825
2023-07-01 09:30:34 +02:00
Michael Tuexen
643d067119 dtrace: another update of the siftr probe
This adds the changes corresponding to
https://cgit.freebsd.org/src/commit/?id=d090464ecd4af5cd400ef5cbbfe8409d019eac34
to the siftr probe.

Reviewed by:		rscheff
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D40824
2023-07-01 02:45:16 +02:00
Michael Tuexen
cade9a8e37 dtrace: update siftr probe
This adds the changes corresponding to
https://cgit.freebsd.org/src/commit/?id=1241e8e7aed5d709a6cf62024e9ae750981c03ae
to the siftr probe.

Reviewed by:		rscheff
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D40823
2023-07-01 02:26:52 +02:00
Michael Tuexen
f5541b85a5 dtrace: fix constants used by the siftr probe
The constants used in the C code were changed in
https://svnweb.freebsd.org/changeset/base/343646
but changing them also in the dtrace translator was missed.

Reviewed by:		cc, rscheff
MFC after:		3 days
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D40805
2023-06-30 22:15:47 +02:00
Mark Johnston
258a0d760a ctfconvert: Integer encoding types are unsigned
Before this change, encodings in the user-defined range were being
sign-extended.

MFC after:	1 week
Sponsored by:	Innovate UK
2023-06-28 16:29:49 -04:00
Martin Matuska
4e8d558c9d zfs: merge openzfs/zfs@feff9dfed
Notable upstream pull request merges:
  #14833 Update compatibility.d files
  #14841 ZIL: Reduce scope of per-dataset zl_issuer_lock
  #14863 zil: Add some more statistics
  #14866 btree: Implement faster binary search algorithm
  #14894 Fix inconsistent definition of zfs_scrub_error_blocks_per_txg
  #14892 Fix concurrent resilvers initiated at same time
  #14903 Fix NULL pointer dereference when doing concurrent 'send' operations
  #14910 ZIL: Allow to replay blocks of any size
  #14939 Fix the L2ARC write size calculating logic
  #14934 Introduce zfs_refcount_(add|remove)_few()
  #14946 Improve l2arc reporting in arc_summary
  #14953 Finally drop long disabled vdev cache
  #14954 Fix the L2ARC write size calculating logic (2)
  #14955 Use list_remove_head() where possible
  #14959 ZIL: Fix race introduced by f63811f072

Obtained from:	OpenZFS
OpenZFS commit:	feff9dfed3
2023-06-10 19:31:17 +02:00
Elyes Haouas
f6a119cf8d zfsd: Fix typo in comment
Signed-off-by: Elyes Haouas <ehaouas@noos.fr>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/653
2023-06-01 20:33:42 -06:00