Commit Graph

188 Commits

Author SHA1 Message Date
Alex Richardson
e17ebfd3b7 Remove -I flag for include path that doesn't exist
Found this while trying to get macOS bootstrap to work again after OpenZFS merge.

Reviewed By:	#zfs, freqlabs
Differential Revision: https://reviews.freebsd.org/D26192
2020-09-10 15:37:07 +00:00
Matt Macy
9e5787d228 Merge OpenZFS support in to HEAD.
The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
  project quotas, encrypted datasets,
  allocation classes, vectorized raidz,
  vectorized checksums, various command line
  improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D25872
2020-08-25 02:21:27 +00:00
Mark Johnston
d2d16e5663 Enable creation of static userspace probes in incremental builds.
To define USDT probes, dtrace -G makes use of relocations for undefined
symbols: the target address is overwritten with NOPs and the location is
recorded in the DOF section of the output object file.  To avoid link
errors, the original relocation is destroyed.  However, this means that
the same input object file cannot be processed multiple times, as
happens during incremental rebuilds.  Instead, only set the relocation
type to NONE, so that all information required to reconstruct USDT
probes is preserved.

Reported by:	bdrewery
MFC after:	3 weeks
Sponsored by:	The FreeBSD Foundation
2020-08-20 19:28:19 +00:00
Bryan Drewery
b155807e48 dtrace tests: Support globbing for excludes
Downstream this makes skipping tests like common/ip/tst.*sctp*.ksh simpler.

Reviewed by:	vangyzen, cem, markj
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D24608
2020-04-29 21:12:32 +00:00
Alan Somers
35f139130a zfsd.8: fix orphan .Xr
Though ZFS is a kernel module, it has no man page in section 4.

Reported by:	phk
MFC after:	2 weeks
2020-04-18 19:47:38 +00:00
Mariusz Zaborski
39d51a9400 DTrace: add a top level makefile to the new test suit
Pointed out by:	markj
2019-06-09 22:45:07 +00:00
Mariusz Zaborski
75ed05ef7d DTrace: create an amd64 test suit
Create two tests checking if we can read urgs registers and if the
rax register returns a correct number.

Reviewed by:	markj
Discussed with:	lwhsu
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D20364
2019-06-05 22:32:26 +00:00
Mark Johnston
f0e2814d34 Hook up the existing i386 DTrace tests to the build.
Now that it's relatively easy to do so, we might as well.

MFC after:	1 week
Event:		Waterloo Hackathon 2019
2019-05-22 03:42:03 +00:00
Mark Johnston
2df0edc13c Make it possible to generate makefiles for arch-dependent DTrace tests.
MFC after:	1 week
Event:		Waterloo Hackathon 2019
2019-05-22 03:10:23 +00:00
Edward Tomasz Napierala
b9d89f5e2f Drop -g from CFLAGS for zfsd(8). No idea why it was ever there.
Reviewed by:	kib, ngie, asomers
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D19915
2019-04-16 12:25:15 +00:00
Edward Tomasz Napierala
93a07b2278 Make zfsd(8) build obey CFLAGS.
Reviewed by:	imp
Obtained from:	CheriBSD
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D19865
2019-04-10 13:42:37 +00:00
Enji Cooper
a530b61063 Integrate cddl/usr.sbin/zfds/tests into the FreeBSD test suite
This change integrates the unit tests for zfsd into the test suite using the
integration method described in r345203.

This change removes the `LOCALBASE` includes added for the port version of
googlemock/googletest, as well as unnecessary `LIBADD`/`DPADD` and `CXXFLAGS`
defines, which are included in the `GTEST_CXXFLAGS` variable, as part of
r345203.

Reviewed by:	asomers
Approved by:	emaste (mentor)
MFC after:	2 months
MFC with:	r345203
Differential Revision: https://reviews.freebsd.org/D19552
2019-03-15 21:49:19 +00:00
Michael Tuexen
1e88cc8b59 Add support for send, receive and state-change DTrace providers for
SCTP. They are based on what is specified in the Solaris DTrace manual
for Solaris 11.4.

Reviewed by:		0mp, dteske, markj
Relnotes:		yes
Differential Revision:	https://reviews.freebsd.org/D16839
2018-08-22 21:23:32 +00:00
Matt Macy
d12e91d584 Make dnode definition uniform on !x86
gcc4 requires -fms-extensions to accept anonymous union members
2018-08-21 03:45:09 +00:00
Brad Davis
edb1df35b0 Fix the build by just installing systop since testing shows it works with:
dwatch -X systop

Reviewed by:	kp
Approved by:	allanjude (mentor)
2018-08-11 16:06:32 +00:00
Devin Teske
37b0d996dc dwatch(1): Add systop profile
Provides a top-like view of syscall consumers.

MFC after:	3 days
X-MFC-to:	stable/11
Sponsored by:	Smule, Inc.
2018-08-11 06:32:31 +00:00
Devin Teske
2282756519 dwatch(1): Fix syntax error in vop_readdir profile
Reported by:	Arne Ehrlich <ehrlich@consider-it.de>
MFC after:	3 days
X-MFC-to:	stable/11
Sponsored by:	Smule, Inc.
2018-08-11 06:13:11 +00:00
Alexander Motin
7fca1b93c4 Do not blindly include illumos kernel headers instead of user-space.
It is not needed now, and I doubt it much helped at all, creating more
confusions then good.
2018-08-02 18:55:55 +00:00
Michael Tuexen
7bda966394 Add a dtrace provider for UDP-Lite.
The dtrace provider for UDP-Lite is modeled after the UDP provider.
This fixes the bug that UDP-Lite packets were triggering the UDP
provider.
Thanks to dteske@ for providing the dwatch module.

Reviewed by:		dteske@, markj@, rrs@
Relnotes:		yes
Differential Revision:	https://reviews.freebsd.org/D16377
2018-07-31 22:56:03 +00:00
Michael Tuexen
53e0911116 Improve TCP related tests for dtrace.
Ensure that the TCP connections are terminated gracefully as expected
by the test. Use appropriate numbers for sent/received packets.
In addition, enable tst.localtcpstate.ksh, which should pass, but
doesn't until https://reviews.freebsd.org/D16369 is committed.

Reviewed by:		markj@
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D16288
2018-07-22 10:50:59 +00:00
Michael Tuexen
dc9f20b3f3 Fix the UDP tests for dtrace.
The code imported from opensolaris was depending on ping supporting
UDP for sending probes. Since this is not supported by ping on FreeBSD
use a perl script instead.
The remote test requires the usage of ksh93, so state that in the
sheband.
Enable the local test, but keep the remote test disabled, since it
requires a remote machine on the LAN.

Reviewed by:		markj@, gnn@
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D16268
2018-07-15 20:34:22 +00:00
Devin Teske
e52d921647 dwatch(1): Update manual to reference actual release
MFC after:	3 days
X-MFC-to:	stable/11
Sponsored by:	Smule, Inc.
2018-06-03 23:49:02 +00:00
Devin Teske
7d958cc9aa dwatch(1): Fix "-t test" for post-processing profiles
Profiles that perform post-processing of the DTrace output were
dropping the "-t test" option on the floor. Fix handling of this
option for said profiles.

X-MFC-to:	stable/11
X-MFC-with:	r334261-334262
Sponsored by:	Smule, Inc.
2018-05-29 22:36:37 +00:00
Devin Teske
1b114e50c9 dwatch(1): Eliminate ANSI dimming in developer mode
"Developer mode" (passing of "-dev" options), which enables debugging
features on compilation error, used to dim lines unrelated to error.

That proved distracting and feedback from testers also confirmed that
simply highlighting the line the compiler complains about is enough.

Sponsored by:	Smule, Inc.
2018-05-27 22:32:45 +00:00
Devin Teske
e08e9e9990 dwatch(1): Guard against error when given -t "*..."
dwatch allows you to customnize the predicate (condition) for when
information is displayed. The DTrace syntax for this is:

	probe[, ...] /predicate/ { [actions] }

But if predicate is something like "*args[1]!=NULL" to test that
the first pointer in an array is non-NULL, the syntax produced is:

	probe[, ...] /*arg1!=NULL/ { [actions] }

The issue being that "/*" is the beginning of a comment and thus
the following error is emitted:

	dtrace: failed to compile script /dev/stdin:
		line 535: /* encountered inside a comment

This patch adds whitespace around the argument given to -t,
producing:

	probe[, ...] / *arg1!=NULL / { [actions] }

Sponsored by:	Smule, Inc.
2018-05-27 22:27:47 +00:00
Devin Teske
1764b74809 dwatch(1): Refactor sendrecv profile
The profile for send(2)/recv(2) observation has been refactored to
eliminate alloca() in favor of translations available in HEAD.

Sponsored by:	Smule, Inc.
2018-05-12 06:23:30 +00:00
Devin Teske
f134539c02 dwatch(1): Expose process for ip/tcp/udp
Knowing the value of execname during these probes is of some value even
if it is commonly the interrupt kernel thread (intr[12]) -- quite often
it is not, but that depends on the probe.

Sponsored by:	Smule, Inc.
2018-05-12 06:18:15 +00:00
Devin Teske
a061d97027 dwatch(1): Allow `-E code' to override profile EVENT_DETAILS
This allows quick changes to the formatted output of a profile.

Sponsored by:	Smule, Inc.
2018-05-12 06:01:41 +00:00
Devin Teske
6a21b3cdf4 dwatch(1): Export ARGV to profiles loaded via load_profile()
A module that wishes to post-process the output needs to know which
arguments were passed in order to re-execute a child in a pipe-chain.
Further, the expansion of ARGV needs to be such that items are escaped
properly.

Sponsored by:	Smule, Inc.
2018-05-12 05:49:31 +00:00
Devin Teske
4a73674e32 dwatch(1): Simplify info message test
The info() function already tests $QUIET

Sponsored by:	Smule, Inc.
2018-05-12 05:43:47 +00:00
Devin Teske
5f2202d104 dwatch(1): Separate default values so `-[BK] num' don't affect usage
If you were to pass an invalid option after `-B num' or `-K num' you
would see that the usage statement would show the value you passed
instead of the actual default.

Moving the default values to separate variables that are unaffected
by the options-parsing allows the usage statement to correctly show
the hard-coded default values if no flags are used.

Sponsored by:	Smule, Inc.
2018-05-12 05:41:28 +00:00
Devin Teske
4a84c26cfc dwatch(1): Bugfix, usage displayed with `-1Q'
A return statement should have been an exit in list_profiles().
If the user passed `-Q' to list profiles and asked for one-line
per profile (`-1'), list_profiles() would not exit as should.

Sponsored by:	Smule, Inc.
2018-05-12 05:36:47 +00:00
Devin Teske
4e16c227ac dwatch(1): Remove the line used to demonstrate `-dev' option
In recently added sendrecv profile, there was a line purposefully
added to introduce a compilation error in which `-dev' is used to
debug the entry. Removing the entry.

Sponsored by:	Smule, Inc.
2018-04-22 02:40:21 +00:00
Devin Teske
f3cf700b57 dwatch(1): Add profile for send(2)/recv(2) syscalls
Sponsored by:	Smule, Inc.
2018-04-22 02:36:06 +00:00
Devin Teske
bcce9a2b33 dwatch(1): Add `-dev' option to aid debugging of profiles
The options `-d' (debug), `-e' (exit after compile), and `-v' (verbose)
when combined in any order (though best remembered as `-dev') will run
the conflated script through dtrace(1), test for error conditions, and
show the line that dtrace(1) failed at (with context).

If no errors are found, the output is the same as `-e[v]'.

When writing a new profile for dwatch(1), you can quickly test to
make sure it compiles by running `dwatch -devX profile_name' where
profiles live in /usr/libexec/dwatch or /usr/local/libexec/dwatch
(the latter being where profiles installed via ports should go).

Sponsored by:	Smule, Inc.
2018-04-22 02:20:17 +00:00
Devin Teske
76a4e1dc81 Fix display of wrong pid from dtrace_sched(4)
Fix a comment while here.

Sponsored by:	Smule, Inc.
2018-03-08 22:32:18 +00:00
Devin Teske
0cc4c55e45 Bump dwatch(1) internal version from 1.0-beta-91 to 1.0 2018-03-06 23:58:53 +00:00
Devin Teske
5bf5ca772c Introduce dwatch(1) as a tool for making DTrace more useful
Reviewed by:	markj, gnn, bdrewery (earlier version)
Relnotes:	yes
Sponsored by:	Smule, Inc.
Differential Revision:	https://reviews.freebsd.org/D10006
2018-03-06 23:44:19 +00:00
Alan Somers
cb4985fbf6 zdb: raise WARNS from 0 to 2
This has only been possible since r329694 and r329508

MFC after:	3 weeks
X-MFC-With:	329694
X-MFC-With:	329508
Sponsored by:	Spectra Logic
2018-02-21 15:51:48 +00:00
Alexander Motin
e5a4a83784 MFV r318941: 7446 zpool create should support efi system partition
illumos/illumos-gate@7855d95b30
7855d95b30

https://www.illumos.org/issues/7446
  Since we support whole-disk configuration for boot pool, we also will need
  whole disk support with UEFI boot and for this, zpool create should create efi-
  system partition.
  I have borrowed the idea from oracle solaris, and introducing zpool create -
  B switch to provide an way to specify that boot partition should be created.
  However, there is still an question, how big should the system partition be.
  For time being, I have set default size 256MB (thats minimum size for FAT32
  with 4k blocks). To support custom size, the set on creation "bootsize"
  property is created and so the custom size can be set as: zpool create B -
  o bootsize=34MB rpool c0t0d0
  After pool is created, the "bootsize" property is read only. When -B switch is
  not used, the bootsize defaults to 0 and is shown in zpool get output with
  value ''. Older zfs/zpool implementations are ignoring this property.
  https://www.illumos.org/rb/r/219/

Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
Reviewed by: Yuri Pankov <yuri.pankov@gmail.com>
Approved by: Dan McDonald <danmcd@kebe.com>
Author: Toomas Soome <tsoome@me.com>

This commit makes no sense for FreeBSD, that is why I blocked the option,
but it should be good to stay closer to upstream.
2018-02-21 00:18:57 +00:00
Alan Somers
d727d8b0b1 Optimize zfsd for the happy case
If there are no damaged pools, then ignore all GEOM events.  We only use
them to fix damaged pools.  However, still pay attention to ZFS events.

MFC after:	20 days
X-MFC-With:	329284
Sponsored by:	Spectra Logic Corp
2018-02-15 21:30:30 +00:00
Alan Somers
a07d59d1da zfsd: Allow zfsd to work on any type of GEOM provider
cddl/usr.sbin/zfsd/zfsd_event.cc
	Remove the check for da and ada devices.  This way zfsd can work on md,
	geli, glabel, gstripe, etc devices.  geli in particular is useful
	combined with ZFS.  gnop is also useful for simulating drive pulls in
	the ZFSD test suite.

	Also, eliminate the DevfsEvent class entirely.  Move its
	responsibilities into GeomEvent.  We can get everything we need to know
	just from listening to GEOM events.

lib/libdevdctl/event.cc
	Fix GeomEvent::DevName for CREATE events.  Oddly, the relevant field is
	named "cdev" for CREATE events but "devname" for disk events.

MFC after:	3 weeks
Relnotes:	Yes (probably worth mentioning the geli part)
Sponsored by:	Spectra Logic Corp
2018-02-14 23:52:39 +00:00
Alan Somers
af63bbd0f0 zfsd: Don't spare a vdev that's being replaced
If a zfs pool contains a replacing vdev (either created manually by "zpool
replace" or by zfsd(8) via autoreplace by physical path) and then new spares
get added to the pool, zfsd shouldn't use one to replace the drive that is
already being replaced.  That's a waste of resources that just slows down
the rebuild.

PR:		225547
MFC after:	3 weeks
Sponsored by:	Spectra Logic Corp
2018-01-30 21:25:43 +00:00
Mark Johnston
d678ce4b6b Remove tst.zonename.d from the list of expected failures.
X-MFC with:	r327888
2018-01-14 17:56:19 +00:00
Mark Johnston
224e0c2f61 Add "jid" and "jailname" variables to DTrace.
These return the jail ID and jail name for the traced process,
respectively, and are analogous to "zonename" on Solaris/illumos.
"zonename" is now aliased to "jailname".

Also add some stress tests for the new variables.

Submitted by:	Domagoj Stolfa <domagoj.stolfa@gmail.com>
Reviewed by:	dteske (previous version)
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D13877
2018-01-12 19:59:46 +00:00
Mark Johnston
60eddb209b Add a regression test for r327794.
MFC after:	2 weeks
2018-01-10 21:40:36 +00:00
Mark Johnston
7c3701904a Mark uctf/err.user64mode.ksh as EXFAIL for now.
MFC after:	1 week
2017-12-15 18:09:23 +00:00
Mark Johnston
97cb52fa9a Actually add the -x setenv test Makefile, missed in r326499.
X-MFC with:	r326499
2017-12-08 19:26:25 +00:00
Mark Johnston
04006780d9 Complete support for dtrace's -x setenv option.
This allows one to override the environment for processes created with
dtrace -c. By default, the environment is inherited.

This support was originally merged from illumos in r249367 but was lost
when the commit was later reverted and then brought back piecemeal.

Reported by:	Samuel Lepetit <slepetit@apple.com>
MFC after:	2 weeks
2017-12-03 16:57:28 +00:00
Mark Johnston
f4b90f5a5c Revert r326181 for now.
We can't link an executable using -m32 until the lib32 phase of a
buildworld, though the build works fine when executing make from
cddl/usr.sbin/dtrace/tests. Some other solution will need to be found.
2017-11-27 17:54:17 +00:00