Previously zfsd would crash in the presence of a pool with a
top-level-vdev that had previously been removed. The crash happened
because the configuration nvlist of such a TLV contains an empty
ZPOOL_CONFIG_CHILDREN array, which led to a pop_front from an empty
list, which has undefined behavior.
The crash only happened in stable/14 and later, probably do to
differences in libcxx, but the change should be MFCed anyway.
PR: 273663
Reported by: Marek Zarychta <zarychtam@plan-b.pwste.edu.pl>
MFC after: 1 week
Sponsored by: Axcient
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D41818
Notable upstream pull request merges:
#15024 Add missed DMU_PROJECTUSED_OBJECT prefetch
#15029 Do not request data L1 buffers on scan prefetch
#15036 FreeBSD: catch up to __FreeBSD_version 1400093
#15039 Fix raw receive with different indirect block size
#15047 FreeBSD: Fix build on stable/13 after 1302506
#15049 Fix the ZFS checksum error histograms with larger record sizes
#15052 Reduce bloat in ereport.fs.zfs.checksum events
#15056 Avoid extra snprintf() in dsl_deadlist_merge()
#15061 Ignore pool ashift property during vdev attachment
#15063 Don't panic if setting vdev properties is unsupported for this vdev type
#15067 spa_min_alloc should be GCD, not min
#15071 Add explicit prefetches to bpobj_iterate()
#15072 Adjust prefetch parameters
#15076 Refactor dmu_prefetch()
#15079 set autotrim default to 'off' everywhere
#15080 ZIL: Fix config lock deadlock
#15088 metaslab: tuneable to better control force ganging
#15096 Avoid waiting in dmu_sync_late_arrival()
#15097 BRT should return EOPNOTSUPP
#15103 Remove zl_issuer_lock from zil_suspend()
#15107 Remove fastwrite mechanism
#15113 libzfs: sendrecv: send_progress_thread: handle SIGINFO/SIGUSR1
#15122 ZIL: Second attempt to reduce scope of zl_issuer_lock
#15129 zpool_vdev_remove() should handle EALREADY error return
#15132 ZIL: Replay blocks without next block pointer
#15148 zfs_clone_range should return descriptive error codes
#15153 ZIL: Avoid dbuf_read() before dmu_sync()
#15172 copy_file_range: fix fallback when source create on same txg
#15180 Update outdated assertion from zio_write_compress
Obtained from: OpenZFS
OpenZFS commit: 804414aad2
libproc's PR_REQUESTED is not implemented on FreeBSD. Remove dead code
in dtrace that would handle it.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D41225
dtrace stored its PR_RLC and PR_KLC flags in the proc_handle's flags,
where they collided with PATTACH_FORCE and PATTACH_RDONLY, respectively.
Thus, Psetflags(PR_KLC) effectively also set the PATTACH_RDONLY flag.
Since the flags are private to dtrace (at least on FreeBSD), store them in
dtrace's own dt_proc structure instead.
On FreeBSD, either PR_RLC or PR_KLC was always set, so remove code that
would handle the case where neither was set.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D41121
The illumos #ifdef's in this file make it harder to read and maintain.
There is little change upstream, and increasing changes for FreeBSD.
Remove the illumos code with `unifdef`. The only manual changes here
are the #includes and #defines at the top, and removing blank lines.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D41173
They were previously being defined by cddl/lib/Makefile.inc, and as of
commit 4ae6991228 were being overridden by defaults in bsd.own.mk,
which changed SHLIBDIR to /usr/lib.
Reported by: Domagoj Stolfa <ds815@cam.ac.uk>
Fixes: 4ae6991228 ("dtrace: Add WITH_DTRACE_ASAN")
This option is a blanket for all the DTrace-related software. The option
when enabled passes in -fsanitize=address -fsanitize=undeifned, enabling
ASAN and UBSAN in the following components:
- libdtrace
- dtrace(1)
- lockstat(1)
- plockstat(1)
The option defaults to "no" and is intended as a developer aid.
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D41157
At the time we ensured that the more important functional tests were
working, but neglected to update the unit tests.
MFC after: 2 weeks
MFC with: 92642bba4d
Sponsored by: Axcient
The pid/killonerror test uses an invalid probe specifier to verify that
the child process is killed. It occasionally fails because the "date"
command is allowed to run long enough to print the date. This is harmless
in this case, but is clearly not ideal.
When the dt_proc_control thread is about to exit, and the dtrace command
forked the child, do not make the child runnable.
Reviewed by: markj
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D40976
At some point the names of these devd events changed. Probably it
happened when importing OpenZFS. Before that, FreeBSD's sysevent_alloc
method didn't create a "class" nvpair in the event, which led to
log_sysevent using the event's ev_subclass field as its type.
MFC after: 2 weeks
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D39437
Improve consistency of the field names with tcpsinfo_t:
* Use mss instead of max_seg_size.
* Use lport and rport instead of tcp_localport and tcp_foreignport.
Use t_flags instead of flags to improve consistency with t_flags2.
Add laddr and raddr, since the addresses were missing when compared
to the output of siftr.
Reviewed by: cc
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D40834
Tracing memcpy() would crash the kernel, because we'd also trace the
memcpy() calls from kinst_invop(). To fix this, introduce kinst_memcpy()
whose arguments are 'volatile', so that we avoid having the compiler
replace it with a regular memcpy().
Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40284
By specifying the -d flag, libdtrace will dump the D script after it has
applied syntactical sugar transformations (e.g if/else). This is useful
for both understanding what dt_sugar does, as well as debugging it.
Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38732
Because dt_module_update() is highly OS-specific, the ifdefs make it
hard to read and follow what is going on. Also handle dm_modid, and
remove handling of the ".filename" section, since we can easily fetch
the filename from the module's pathname (k_stat->pathname).
Reviewed by: markj
Approved by: markj (mentor)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39177
dtrace_instr_size() is needed by the forthcoming RISC-V port of kinst,
as well as by libdtrace in D38825 for both amd64 and RISC-V.
Reviewed by: markj, mhorne
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39489
If you remove an unused spare and then reinsert it, zfsd will now online
it in all pools.
Do not MFC without 2a58b312b6 (but it's ok to MFC that one without this
one).
Submitted by: Ameer Hamza <ahamza@ixsystems.com> (zfsd), Me (tests)
MFC after: 2 weeks
MFC with: 2a58b312b6
Sponsored by: iX Systems, Axcient
Pull Request: https://github.com/freebsd/freebsd-src/pull/697
The section will contain static relocations which do not need to be
preserved after linking, and moreover these relocations may reference
symbols that end up getting removed.
Do not set SHF_ALLOC and instead let the linker decide what needs to be
done.
PR: 258872
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Implementation of block cloning for ZFS
#13741 SHA2 reworking and API for iterating over multiple implementations
#14282 Sync thread should avoid holding the spa config write lock
when possible
#14283 txg_sync should handle write errors in ZIL
#14359 More adaptive ARC eviction
#14469 Fix NULL pointer dereference in zio_ready()
#14479 zfs redact fails when dnodesize=auto
#14496 improve error message of zfs redact
#14500 Skip memory allocation when compressing holes
#14501 FreeBSD: don't verify recycled vnode for zfs control directory
#14502 partially revert PR 14304 (eee9362a7)
#14509 Fix per-jail zfs.mount_snapshot setting
#14514 Fix data race between zil_commit() and zil_suspend()
#14516 System-wide speculative prefetch limit
#14517 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
#14519 Do not hold spa_config in ZIL while blocked on IO
#14523 Move dmu_buf_rele() after dsl_dataset_sync_done()
#14524 Ignore too large stack in case of dsl_deadlist_merge
#14526 Use .section .rodata instead of .rodata on FreeBSD
#14528 ICP: AES-GCM: Refactor gcm_clear_ctx()
#14529 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
#14532 Handle unexpected errors in zil_lwb_commit() without ASSERT()
#14544 icp: Prevent compilers from optimizing away memset()
in gcm_clear_ctx()
#14546 Revert zfeature_active() to static
#14556 Remove bad kmem_free() oversight from previous zfsdev_state_list
patch
#14563 Optimize the is_l2cacheable functions
#14565 FreeBSD: zfs_znode_alloc: lock the vnode earlier
#14566 FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
#14567 spl: Add cmn_err_once() to log a message only on the first call
#14568 Fix incremental receive silently failing for recursive sends
#14569 Restore ASMABI and other Unify work
#14576 Fix detection of IBM Power8 machines (ISA 2.07)
#14577 Better handling for future crypto parameters
#14600 zcommon: Refactor FPU state handling in fletcher4
#14603 Fix prefetching of indirect blocks while destroying
#14633 Fixes in persistent error log
#14639 FreeBSD: Remove extra arc_reduce_target_size() call
#14641 Additional limits on hole reporting
#14649 Drop lying to the compiler in the fletcher4 code
#14652 panic loop when removing slog device
#14653 Update vdev state for spare vdev
#14655 Fix cloning into already dirty dbufs
#14678 Revert "Do not hold spa_config in ZIL while blocked on IO"
Obtained from: OpenZFS
OpenZFS commit: 431083f75b
This drops function `getexecname()` redirection.
Historically `getexecname()` is a compatibility definition. Since
openzfs has its own implementation of function `getexecname()` in libspl
and has been merged into base, the compat header file stdlib.h is
no longer needed and should not be used.
Also without this fix libspl will end up an incompatible version of
`getprogname()` with libc. In particular, if zfs is enabled, programs
such as pgrep in /rescue can be wrongly statically linked with libspl
and will not function properly.
PR: 269738
Reviewed by: markj
Fixes: 9e5787d228 Merge OpenZFS support in to HEAD
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D38733
It contains the ctf tools that most users will never use.
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D38223
