Commit Graph

13199 Commits

Author SHA1 Message Date
Michael Meffie
dcf44ab5fc tests: do not resolve addresses in vos/vl test
The vos-t test adds a set of 10.* test addresses to a test vlserver and
runs vos to read them back.  When the test is run in an environment
where hosts have been assigned in the 10.* internal network, vos will resolve
the addresses to hostnames and the test fails.  Pass the -noresolve
option to vos for this test when checking for the expected list of
addresses.

Example test output before this commit:

    ./vos-t
    ...
    #   seen: 10.0.0.0
    10.0.0.1
    myhost.example.com
    10.0.0.3
    ...
    not ok 5 - vos output matches

Change-Id: Ief43fe180a0dfff211f28d5f47be6224270907a3
Reviewed-on: https://gerrit.openafs.org/14020
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2020-01-11 00:46:42 -05:00
Andrew Deason
37c5db3ce7 FBSD: Declare vnops/vfsops static
Declare our vnode and vfs operations as static functions, since they
are not referenced outside of osi_vfsops.c/osi_vnodeops.c. Shuffle
around the definitions in osi_vnodeops.c so that we don't need forward
declarations for the functions.

Change-Id: Idbbe05a8b248ac29c2795c365be6a4e99da536dd
Reviewed-on: https://gerrit.openafs.org/13973
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
2020-01-10 17:14:06 -05:00
Andrew Deason
a4e9365fff FBSD: Remove support for 8.x and 9.x
According to <https://www.freebsd.org/security/unsupported.html>,
FreeBSD 8.x EoL was on August 1, 2015, and FreeBSD 9.x EoL was on
December 31, 2016. Remove our support for these versions, since they
haven't been supported by FreeBSD itself for a while.

FreeBSD 10.x EoL was on October 31, 2018, which has passed, but was
less than a year ago. So keep 10.x in for now.

Adjust our preprocessor checks accordingly:

- In FBSD-specific dirs, assume AFS_FBSD100_ENV and lower is always
  true. Assume __FreeBSD_version is always at least 1000000.

- In non-FBSD dirs, convert AFS_FBSD100_ENV and lower to AFS_FBSD_ENV.

Change-Id: I965e65d3b95573bb374661217b24b686c7b68ed2
Reviewed-on: https://gerrit.openafs.org/13842
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2020-01-10 16:10:57 -05:00
Andrew Deason
eab0bb0af8 tests: Explicitly build target 'all' by default
Commit 68f40643 (Build tests by default) added new targets in our
top-level Makefile, that caused us to effectively run
'cd tests && make' as part of the default build. Since no explicit
target is provided, 'make' tries to build the first target in the
given Makefile. On some platforms (such as *BSD), 'make' finds the
first defined target as a pattern rule (%.c) from our included
makefiles, and tries to build the target %.c, which it cannot do. This
causes the build to fail with:

    cd tests && make
    make[3]: don't know how to make %.c. Stop

To fix this, just explicitly build the 'all' target when we build our
tests by default.

Change-Id: I319271482685ec35087c470d95fdcaec6e1d8c47
Reviewed-on: https://gerrit.openafs.org/13993
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
2020-01-10 16:10:49 -05:00
Andrew Deason
ce7a76a13e tests: Stop vlserver on errors
Currently, if we encounter an error and 'goto out' after starting the
test vlserver, we'll exit without stopping the test vlserver. This can
confuse the test harness, causing 'runtests' to hang forever.

To avoid this, move the afstest_StopServer() call to also run when
we're bailing out, but only if the server has actally started of
course.

Change-Id: Ice5a56c20bc8d2eac85b3e760850c4d85e4601a8
Reviewed-on: https://gerrit.openafs.org/13992
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
2020-01-10 11:31:31 -05:00
Andrew Deason
a21a2f8edb tests: Introduce afstest_GetProgname
Currently, in tests/volser/vos-t.c we call afs_com_err as
"authname-t", which is clearly a mistake during some code refactoring
(introduced in commit 2ce3fdc5, "tests: Abstract out code to produce a
Ubik client").

We could just change this to "vos-t", but instead of specifying
constant strings everywhere, change this to figure out what the
current command is called, and just use that. Put this code into a new
function, afstest_GetProgname, and convert existing tests to use that
instead of hard-coding the program name given to afs_com_err.

Change-Id: I3ed02c89f93798568783c7d717e8fb2e39dcce14
Reviewed-on: https://gerrit.openafs.org/13991
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2020-01-09 23:56:27 -05:00
Andrew Deason
48d181ca1f libtool: Serialize building libfoo.la and libfoo.a
We have a few libraries where we have separate targets to build
libfoo.la (to get libfoo.so) and libfoo.a. Currently, these targets
can be built in parallel, and both are built with libtool. This can
cause problems because of two behaviors with libtool:

- When running --mode=link for libfoo.a or libfoo.la, it effectively
  runs 'rm -rf .libs/libfoo.*' to clean up its work area.

- When running --mode=link for libfoo.a, libtool sets up some scratch
  space in .libs/libfoo.ax to unpack various static libs.

So when 'make libfoo.a' is running, libtool creates a .libs/libfoo.ax
dir, and unpacks various object files inside of it. If while that is
running, 'make libfoo.la' runs, it causes libtool to remove that
directory and all its contents. This causes 'make libfoo.a' to fail
with confusing messages like this (for libafsrpc.a):

    /bin/sh ../../libtool --quiet --mode=link --tag=CC  gcc -static   -O  -o libafsrpc.a [...]
    find: '.libs/libafsrpc.ax/libopr_pic.a': No such file or directory
    ar: .libs/libafsrpc.ax/libfsint_pic.a/afscbint.cs.o: No such file or directory
    make[3]: *** [Makefile:59: libafsrpc.a] Error

To avoid this, prevent building libfoo.la and libfoo.a at the same
time, by just making libfoo.la depend on libfoo.a. Do this for all of
the libraries we build in this way: libafshcrypto, libkopenafs,
libafsauthent, and libafsrpc.

Change-Id: I821768b3b4cd99cf5bf98605068773347ada0fb2
Reviewed-on: https://gerrit.openafs.org/14017
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2020-01-08 00:20:14 -05:00
Andrew Deason
057f848a9c ubik: Introduce ugen_secproc_func
We currently specify the signature of the 'secproc' function callback
in multiple places. Consolidate them into a single typedef.

Change-Id: Ic785f47fc726bff6c37f7fd826f1e2626d006776
Reviewed-on: https://gerrit.openafs.org/13986
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2020-01-08 00:16:18 -05:00
Andrew Deason
86170750dd doc: Document new rxgk options
Commit e5b1e6f1 (Add rxgk client options to vl and pt utilities) added
a couple of new command-line options related to rxgk, but didn't add
them to the relevant man pages.

Add a brief description of these new options to the manpages for pts,
vos, ptserver, and vlserver.

Change-Id: I2d9bfdeb0a31d396740ca2a4d42e14c025b6f79e
Reviewed-on: https://gerrit.openafs.org/13947
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Tested-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2020-01-08 00:15:42 -05:00
Cheyenne Wills
bebae936b4 afs: Fix EIO error when reading a 4G or larger file
When reading a file with a file length of >= 4G, the cache manager is
failing the read with an EIO error.

In afs_GetDCache, the call to IsDCacheSizeOK is passed a parameter that
contains only the lower 32bits of the file length (which requires a 64
bit value). This results in the EIO error if the length is over 2^32 -1.

The AFSFetchStatus.Length member needs to be combined with the
AFSFetchStatus.Length_hi to obtain the full 64bit file length.

Fix the calls to IsDCacheSizeOK to use the full 64bit file length.

Commit "afs: Check dcache size when checking DVs
7c60a0fba11dd24494a5f383df8bea5fdbabbdd7" - gerrit 13436 - added the
IsDCacheSizeOK function and the associated calls.

As a note, the AFSFetchStatus.DataVersion is the lower 32 bits of the
full 64bit version number, AFSFetchStatus.dataVersionHigh contains
the high order 32bits.  The function IsDCacheSizeOK is passed just the
32bit component, the only use of the parameter is in an error message.

Change-Id: Idbe6233bd6ef792ed2b92d9337aba334e23f1452
Reviewed-on: https://gerrit.openafs.org/14002
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
2020-01-03 12:41:04 -05:00
Marcio Barbosa
daf6616aab macos: add entry for afs into synthetic.conf
The root mount point is read-only as of macOS 10.15. As a result, /afs
cannot be created at this location. To workaround this restriction,
macOS 10.15 provides an alternative way to create mount points at the
root. To make it possible, an entry for the mount point in question must
be added to /etc/synthetic.conf. The synthetic entities described in
this file are not physically present on the disk. Instead, they are
synthesized by the kernel during system boot.

This commit adds an entry for afs into the file mentioned above. Knowing
that this change only takes effect after reboot, also provide directions
to the user during the installation process.

Change-Id: I7a05f4b9a48e443dbaa20a624a92b8b54c510000
Reviewed-on: https://gerrit.openafs.org/13928
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Yadavendra Yadav <yadayada@in.ibm.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-30 12:44:41 -05:00
Marcio Barbosa
0563642cc1 macos: add script to notarize OpenAFS
In order to integrate the notarization process into our existing build
scripts, this patch introduces a script to automatically notarize the
OpenAFS package.

Change-Id: Ia9743cd39485e68de540b79b165b9d92020ad187
Reviewed-on: https://gerrit.openafs.org/13671
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-30 12:42:54 -05:00
Andrew Deason
10d176afd2 Do not build shared-only libs for --disable-shared
Commit 0f1e54c4 (Pass -shared when linking some shared libraries)
changed some of our linking rules to pass -shared to libtool when
linking. When building with the --disable-shared configure option,
this causes those linker rules to fail, since shared libraries are
disabled. Before commit 0f1e54c4, we could build with --disable-shared
successfully.

To allow us to build again with --disable-shared, just don't build the
relevant shared-only libraries at all, when shared libraries are
disabled. To accomplish this, introduce a new substitution variable,
SHARED_ONLY, which allows certain lines in Makefiles to become
commented-out when shared libraries are disabled. Update all of the
shared-only libraries to be built conditionally based on this
variable.

Except for libuafs.la, which appears to be not referenced by anything.
Just remove the rules for that instead.

Change-Id: I82084a08d2f9c12ca438bd7b1626e1376159c975
Reviewed-on: https://gerrit.openafs.org/13927
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
2019-12-27 20:49:51 -05:00
Andrew Deason
d0941e81b2 pts: Use cmd_AddParmAtOffset for common parms
Update pts to use cmd_AddParmAtOffset and symbolic constants for our
common parameters, instead of using bare literals like '16'.

Change-Id: Ib8fe77983a6bba46c3182585774e067512449f0e
Reviewed-on: https://gerrit.openafs.org/13946
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
2019-12-22 20:24:51 -05:00
Andrew Deason
90726f837c tests: Check if vlserver died during startup
Currently, the volser/vos test starts a local vlserver to communicate
with. If the vlserver dies during startup, the spawned 'vos'
subprocesses take forever to run, since we need to wait for our Rx
calls to timeout for every operation.

To make it less annoying to detect and investigate errors that might
cause the vlserver to fail during startup, check if the vlserver dies
right away. We already sleep for 5 seconds when starting the vlserver,
so just check if the pid still exists after those 5 seconds.

Change-Id: I6c33059542fa975e4cb389b718f9da190cd13289
Reviewed-on: https://gerrit.openafs.org/13942
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-20 20:43:34 -05:00
Andrew Deason
94acb9f36b rx: Make rx_identity_free idempotent
rx_identity_free sets the given identity to NULL, but it
unconditionally derefs the given identity. Make it a no-op for NULL
identities, to make related cleanup code and destructors simpler.

Change-Id: I863c72be71fb4b3056a2cd8fc2bf19cfb2d5dfbb
Reviewed-on: https://gerrit.openafs.org/13945
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-20 20:41:25 -05:00
Andrew Deason
d3d2530691 rx: Make rx_opaque_free idempotent
Currently rx_opaque_free sets the given argument to NULL, a style that
helps prevent double-frees. However, it doesn't check if the given
buffer is already NULL, which makes potential callers that use a 'goto
done'-style cleanup block do something like:

 done:
    if (buf)
        rx_opaque_free(&buf);

To avoid the extra if(), make rx_opaque_free a no-op if it's given a
NULL buffer, similar to how free(NULL) is a no-op on most platforms.

Slightly refactor how we reference our argument as well, to limit the
number of layers of indirection the code needs to deal with.

Do the same for rx_opaque_zeroFree.

Note that there are currently no callers of
rx_opaque_free/rx_opaque_zeroFree, but future commits will add some.

Change-Id: Ic86a9c63903bebbddd311912cfbcb61198e3f0b0
Reviewed-on: https://gerrit.openafs.org/13944
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-20 11:59:18 -05:00
Andrew Deason
71bf9ac08c ptserver: Fix WhoIsThisWithName indentation
Many lines in this block in WhoIsThisWithName are oddly indented by 1
more space than usual. Fix them.

Change-Id: I5e3ec4974cebc694c7b02c1ea6e037d4ec335a12
Reviewed-on: https://gerrit.openafs.org/13943
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-20 11:47:03 -05:00
Andrew Deason
68f406436c Build tests by default
While it's not feasible to run all of our tests by default during the
build, we should be able to at least make sure the tests can build.
So, make the default build targets also build our tests, by making the
'finale' target build the tests.

Change-Id: Ieadd48ba2774526de8a13136e6cc8a50434ed2f5
Reviewed-on: https://gerrit.openafs.org/13941
Tested-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-20 11:32:22 -05:00
Andrew Deason
0b8b6683fb tests: Fix manpage tests for objdir builds
The manpage tests have a couple of problems when running for objdir
builds:

- We try to specify './tests-lib/perl5' as a directory to find our
  helper library. However, the cwd when we're running the tests is in
  an objdir build, where the helper library is in the srcdir. Fix this
  by using the SOURCE env var specified by the tests wrapper.

- All of these tests specify the directory in which to find the man
  pages in a subdir of BUILD, but our manpages are located in the src
  dir (since they are built by regen.sh, not by configure/make). Fix
  this by specifying a SOURCE-based directory instead.

To avoid needing to make the same change for each of these tests, also
refactor the manpage tests so each test only needs to specify the
subdirectory and command name, and get rid of some of the common
boilerplate.

Change-Id: I96be199b1dec8db0545ae3cf19d2595c4afe4cdd
Reviewed-on: https://gerrit.openafs.org/13940
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-20 11:29:45 -05:00
Marcio Barbosa
63fd13bf9e macos: prepare for notarization
With the public release of macOS 10.14.5, all new and updated kernel
extensions must be notarized by Apple. To be taken into consideration,
all executables must be signed and the Hardened Runtime capability must
be enabled.

This patch adds the missing prerequisites mentioned above.

Change-Id: I2d3ad66cb7ce062b91d0616955f3bc2b06ca5822
Reviewed-on: https://gerrit.openafs.org/13670
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Tested-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-20 10:49:45 -05:00
Marcio Barbosa
c7864b7360 macos: packaging support for MacOS X 10.15
This commit introduces the new set of changes / files required to
successfully create the dmg installer on OS X 10.15 "Catalina".

Change-Id: I628a3210fa42b2f34ff78030930f83e836775392
Reviewed-on: https://gerrit.openafs.org/13669
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
2019-12-20 10:48:25 -05:00
Marcio Barbosa
93815caabc macos: add support for MacOS 10.15
This commit introduces the new set of changes / files required to
successfully build the OpenAFS source code on OS X 10.15 "Catalina".

Change-Id: I849d4c837bf9ae36fe5c33356bc1c66a2fc513ac
Reviewed-on: https://gerrit.openafs.org/13668
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-20 10:47:06 -05:00
Marcio Barbosa
d4302d4214 macos: upgrade *.xib files
According to Xcode 11, the *.xib files updated by this commit use an
older format that is potentially insecure when decoded. To fix this
problem, Xcode automatically upgraded these files to the modern format.
These changes are required to build OpenAFS on Catalina (Xcode 11).

Change-Id: Ica8c464eff93496d87fc854b193bfb0dad07a3c2
Reviewed-on: https://gerrit.openafs.org/13935
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-20 10:46:10 -05:00
Marcio Barbosa
677b038814 macos: tell the compiler the system include path
In order to support multiple SDKs, macOS Catalina no longer has the
/usr/include directory. As a result, the compiler needs to know where
these headers can be found. To successfully build OpenAFS on OSX 10.15,
set KROOT so the compiler knows the correct location of these headers.

Change-Id: I5ef33b34b6a4e6111983a63a2d34326ca4af9d30
Reviewed-on: https://gerrit.openafs.org/13936
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-20 10:42:12 -05:00
Andrew Deason
f4ab3767b7 tests: Fix most tests for objdir builds
Fix a few miscellaneous issues with building and running our tests in
objdir builds:

- Our C tests use -I$(srcdir)/../.. in the CFLAGS, so we can #include
  <tests/tap/basic.h>. However, basic.h actually gets copied from
  src/external/c-tap-harness/tests/tap/ to tests/tap/ during the
  build, and so basic.h is available in the objdir, not srcdir. For
  objdir builds, this causes building the tests to fail with failing
  to find basic.h. Fix this to use TOP_OBJDIR as the include path
  instead.

- Our 'make check' in tests/ tries to run ./libwrap; but our cwd will
  be in the objdir for objdir builds, and libwrap is a script in our
  srcdir. Fix this to run libwrap from the srcdir path.

- In tests/opr/softsig-t, it tries to find the 'softsig-helper' binary
  in the same dir as 'softsig-t'. However, softsig-t is just a script
  in the srcdir, but softsig-helper is a binary built in the objdir.
  Fix this to use the BUILD env var provided by the tests wrapper, by
  default.

Change-Id: Iff642613bfc88d0d7e348660dc62f59e6fa8af75
Reviewed-on: https://gerrit.openafs.org/13939
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-13 00:03:21 -05:00
Andrew Deason
847b63af92 FBSD: Remove pre-8 code
Commit 123f0fb1 (config: remove support for old FreeBSD releases)
removed our support for FreeBSD releases before FreeBSD 8. However,
various areas of code still reference the symbols from those old
versions (e.g. AFS_FBSD53_ENV). Remove our ifdef logic for these old
symbols, according to the following rules:

- In FBSD-specific dirs, assume AFS_FBSD80_ENV is always true (as well
  as the symbols for earlier versions)

- In non-FBSD dirs, convert AFS_FBSD80_ENV to AFS_FBSD_ENV (and do the
  same for all earlier versions)

This allows us to remove code that was specific to older FreeBSD
versions, and simplify some ifdef conditionals.

Also remove the definitions for AFS_FBSD80_ENV and earlier versions in
our existing param.h files.

With this commit, the functions afs_start, afs_vop_lock,
afs_vop_unlock, and afs_vop_islocked are now always unreferenced, so
remove them.

Change-Id: Ia5a5ba5ee5b71a86cb4514305e20f1bb34487100
Reviewed-on: https://gerrit.openafs.org/13812
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Tim Creech <tcreech@tcreech.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-10 17:55:39 -05:00
Yadavendra Yadav
f9c716fca1 afs: Add ppc64le changes in osconf.m4 file.
If swig package is installed on a ppc64le system, build fails for
"libuafs" while running "shlib-build". "shlib-build" gets executed for
builing ukernel.so and this is triggered if "LIBUAFS_BUILD_PERL" is not
empty. Having "swig" package on system sets "LIBUAFS_BUILD_PERL" to
'LIBUAFS_BUILD_PERL' value. The reason for build failure was inside
"shlib-build", 'linker' was not set (it was empty). 'linker' value is
set based on SHLIB_LINKER, which was not defined in  osconf.m4 if build
system is ppc64le.

To fix this add ppc64le_linux26 case in osconf.m4 file.

Change-Id: I79d2f78b2af34207c81f4f5ab05fdc387404acad
Reviewed-on: https://gerrit.openafs.org/13980
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-09 11:11:54 -05:00
Cheyenne Wills
d79a8e13e5 util: Use a struct for afsUUID_to_string
Replace the use of a character array with a structure that contains
the size of the buffer that is needed.  This allows the C compiler to
perform a type check to ensure the correct sized buffer is used. In
addition, the size of the buffer is now specified in just one location.

Change the signature of the afsUUID_to_string function to return a
pointer to the start of a formatted UUID. This allows the use of
afsUUID_to_string in a way that is consistent with other object
formatting functions:

    struct uuid_fmtbuf uuidstr;
    printf("... %s ...",
             afsUUID_to_string(uuid, &uuidstr));

Update callers to use the new uuid_fmtbuf struct when calling
afsUUID_to_string.

Change-Id: I6d6f86ce6c058defc6256e8e88dee4449dd4f7e6
Reviewed-on: https://gerrit.openafs.org/13831
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-09 03:19:28 -05:00
Marcio Barbosa
f5f8b93369 viced: add opt to allow admin writes on RO servers
Add the new option -admin-write to allow write requests from superusers
on file servers running in readonly mode (-readonly). This lets sites
run fileservers in readonly mode for normal users, but allows members of
the system:administrators group to modify content.

Change-Id: Id8ed3513a748815c07cb98e426c1d21ac300b416
Reviewed-on: https://gerrit.openafs.org/13707
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Tested-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-12-05 23:10:48 -05:00
Andrew Deason
7cdf1a93cf afs: Skip checking chunkBytes sanity for RW files
Currently, the IsDCacheSizeOK check can trigger a false positive for a
dcache, if the data in the dcache was populated by a local write to a
file that was later extended with sparse data.

For example: say a client opens a new file, and writes 4 bytes to
offset 0, and then writes 4 bytes to offset 0x400000. After the first
write, the first chunk for the file will contain just 4 bytes, and
after the second write, the first chunk is unchanged (since we're
writing to a different area of the file), but the file is now 0x400004
bytes long. The sparse area of the file will be correctly filled with
zeroes for local reads and on the fileserver, but the 4-byte chunk
causes IsDCacheSizeOK to complain and mark the dcache as invalid.

Even though nothing is wrong, this causes the following scary messages
to potentially appear in the kernel log, and the relevant dcache to be
invalidated:

    afs: Detected corrupt dcache for file 1.536870913.2.2: chunk 0 (offset 0) has 4 bytes, but it should have 131072 bytes
    afs: (dcache 0xfffffdeadbeefb4d, file length 4194308, DV 1, dcache mtime 1575049956, index 996, dflags 0x2, mflags 0x0, states 0x4, vcache states 0x1)
    afs: Ignoring the dcache for now, but this may indicate corruption in the AFS cache, or a bug.

It's probably difficult or impossible to detect if this specific case
is happening, so to avoid this scenario, just avoid doing the size
check at all for RW data from the cache.

Change-Id: Ia40ec838c525d9abc13a03be39028e4ca04a9457
Reviewed-on: https://gerrit.openafs.org/13969
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
2019-11-29 14:27:33 -05:00
Marcio Barbosa
0593017177 viced: prevent writes on readonly fileservers
Currently, a fileserver can be initialized as readonly. In this mode,
writes on this server should not be allowed. Unfortunately, updates on
files stored by readonly fileservers are not completely prevented. In
some situations, the check for RO server is omitted (e.g. if the user is
the owner of the file to be updated). In other situations, the same
check is redundant.

To fix these problems, consolidate this check in one place.

Change-Id: Id53e15216404dfe691a87c7b4964ff08924c262c
Reviewed-on: https://gerrit.openafs.org/13934
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: Benjamin Kaduk <kaduk@mit.edu>
2019-11-29 11:51:41 -05:00
Marcio Barbosa
2ae2a15c9d sys: retry lsetpag if errno is EINTR
The variable errno might be set by some system calls to indicate the
reason why the system call in question did not work as expected. If the
setpag system call is interrupted by a signal, the value of errno will
be EINTR. This value means that setpag did not succeed because it was
interrupted.

If lsetpag did not succeed and errno is equal to EINTR, try again.

Change-Id: Ibf306d62fc8d2fa9ccb0692f9031c5aa659b2bfe
Reviewed-on: https://gerrit.openafs.org/12295
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-11-28 17:51:37 -05:00
Marcio Barbosa
9563807791 afs: afs_pag_wait() makes process unkillable
To enforce a maximum average rate of one PAG allocation per second,
afs_pag_wait(), called by afs_setpag*(), sleeps until the difference
between the current time and pag_epoch gets greater than pagCounter.
Unfortunately, this function ignores the code returned by afs_osi_Wait().
As a result, it is not possible to kill the process that requested the
new pag while afs_pag_wait() is sleeping.

To fix this problem, do not ignore the code returned by afs_osi_Wait().

Change-Id: I6be11a569edcafa6ecdf716e5315fc75f5a128e8
Reviewed-on: https://gerrit.openafs.org/12260
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-11-28 02:18:32 -05:00
Andrew Deason
9d08545475 afs: Ensure CDirty is set during afs_write loop
Currently, in afs_write(), we set CDirty on the given vcache, and then
write the given data into various dcaches. When writing to a dcache,
we call afs_DoPartialWrite, which may cause us to flush the dirty data
to the fileserver and clear the CDirty bit.

If we were given more than 1 chunk of data to write, we will then go
through another iteration of the loop, writing more dirty data into
dcaches, but CDirty will not be set. This can cause issues with, for
example, afs_SimpleVStat() or afs_ProcessFS(), which use CDirty to
determine whether or not to merge in FetchStatus info from the
fileserver into our local cache. This can cause our local cache to
incorrectly reflect the state of the file on the fileserver, instead
of the state of the locally-modified file in our cache.

A more detailed example is as follows. Consider a small C program that
copies a file, fchmod()ing the destination before closing it:

    void
    do_copy(char *src_name, char *dest_name)
    {
        /* error checking elided */
        src_fd = open(src_name, O_RDONLY);
        dest_fd = open(dest_name, O_WRONLY|O_CREAT|O_TRUNC, 0755);
        fstat(src_fd, &st);
        src_buf = mmap(NULL, st.st_size, PROT_READ, MAP_SHARED, src_fd, 0);
        write(dest_fd, src_buf, st.st_size);
        munmap(src_buf, st.st_size);
        close(src_fd);
        fchmod(dest_fd, 0100644);
        close(dest_fd);
    }

Currently, on FBSD, using this to copy a 7862648-byte file, using a
smallish cache (10000 blocks) will cause the destination to appear to
be truncated, because avc->f.m.Length will be incorrect, even though
all of the relevant data was written to the fileserver.

On most other platforms such as SOLARIS and LINUX, this is not a
problem, since currently they only write one page of data at a time to
afs_write(), and so they never hit multiple iterations of the while()
loop inside afs_write().

To fix this, just set CDirty on every iteration of the while() loop in
afs_write(). In general, we need to set CDirty after calling
afs_DoPartialStore() anywhere if the caller continues to write more
data. But all callers already do this, except for this one instance in
afs_write().

Thanks to tcreech@tcreech.com for helping find occurrences of the
relevant issue.

FIXES 135041

Change-Id: I0f7a324ea2d6987a576786292be2d06487359aa6
Reviewed-on: https://gerrit.openafs.org/13948
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
2019-11-22 00:52:11 -05:00
Andrew Deason
4a9078c6bb afs: Avoid giving wrong 'tf' to afs_InitVolSlot
Commit 75e3a589 (libafs: afs_InitVolSlot function) split out a bit of
our code that initializes a struct volume into the afs_InitVolSlot
function. However, it caused us to almost always pass a non-NULL 'tf'
to afs_InitVolSlot, even if the target volume was not found.

That is, before that commit, our code roughly did this:

    for (...; j != 0; j = tf->next) {
        ...;
        tf = &staticVolume;
        if (tf->volume == volid)
            break;
    }
    if (tf && j != 0) {
        use_tf_data();
    } else {
        use_blank_data();
    }

The reason for the extra 'j != 0' check after the loop is to see if we
hit the end of the volume hash chain, or if we actually found a
matching 'tf' in the loop.

And after that commit, the code did this:

    for (...; j != 0; j = tf->next) {
        ...;
        if (j != 0) {
            tf = &staticVolume;
            if (tf->volume == volid)
                break;
        }
    }
    if (tf) {
        use_tf_data();
    } else {
        use_blank_data();
    }

The check for 'j != 0' was moved to inside the for loop, but 'j' is
always nonzero in the loop (otherwise, the for() would exit the loop).
This means that if we didn't find a matching 'tf' in the loop, our
'tf' would be non-NULL anyway, and so we'd initialize our volume slot
from just the last entry in the hash chain.

This means that for volumes that are not found in the VolumeItems
file, our struct volume will probably be initialized with arbitrary
data from another volume, instead of being initialized to the normal
defaults (the 'else' clause in afs_InitVolSlot).

This means that the 'dotdot' entry for the volume may be wrong, and so
we may report the wrong parent dir for the root of a volume. However,
the 'dotdot' entry should be fixed when the volume root is accessed
via a mountpoint, so any such issue should be temporary. And of
course, on some platforms (LINUX) we don't ever use the 'dotdot'
information for a volume, and even on other platforms, often resolving
the '..' entry is handled by other means (e.g. shells often calculate
it themselves). But some 'pwd' calculations and other '..' corner
cases may be affected.

To fix this, change the relevant loop so that we only set 'tf' to
non-NULL when we actually find a matching entry.

Change-Id: I53118960462c0057725e749cbf588e98024217c3
Reviewed-on: https://gerrit.openafs.org/13933
Tested-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-11-08 10:54:13 -05:00
Andrew Deason
360b9d5d71 afs: Avoid -1 error for vreadUIO/vwriteUIO
Commit c6b61a45 (afs: Verify osi_UFSOpen worked) added various checks
to return an error if a given osi_UFSOpen failed. However, two of
these checks (in afs_UFSReadUIO and afs_UFSWriteUIO) result in us
returning -1 on error, in functions that otherwise return errno codes
(e.g. ENOSPC). An error code of -1 might get interpreted as
RX_CALL_DEAD, which would be rather confusing, so use EIO as a generic
error instead.

Change-Id: I23b9a73b82d999d8ee4670b5e7ec39b9d820fb0f
Reviewed-on: https://gerrit.openafs.org/13931
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Mark Vitale <mvitale@sinenomine.net>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-11-08 02:56:41 -05:00
Andrew Deason
b3b56d7965 doc: Fix realm capitalization
In this example, krbtgt.Example.COM clearly refers to the principal
name converted from krbtgt/Example.COM, and so by convention the realm
name would be in all caps. Fix this example to use the all-caps realm
name, for consistency.

This mistake was introduced by commit 1cc8feb6 (doc: replace hostnames
with IETF example hostnames), the realm was in all caps before that
commit.

Mistake spotted by Chas Williams.

Change-Id: Icaf4931868752064c4617c8ad778122e076ae3cb
Reviewed-on: https://gerrit.openafs.org/13930
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Mark Vitale <mvitale@sinenomine.net>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-11-08 02:47:56 -05:00
Andrew Deason
6ec46ba777 OPENAFS-SA-2019-003: ubik: Avoid unlocked ubik_currentTrans deref
Currently, SVOTE_Debug/SVOTE_DebugOld examine some ubik internal state
without any locks, because the speed of these functions is more
important than accuracy.

However, one of the pieces of data we examine is ubik_currentTrans,
which we dereference to get ubik_currentTrans->type. ubik_currentTrans
could be set to NULL while this code is running, so there is a small
chance of this code causing a segfault, if SVOTE_Debug() is running
when the current transaction ends.

We only ever initialize ubik_currentTrans as a write transation (via
SDISK_Begin), so this check is pointless anyway. Accordingly, skip the
type check, and always assume that any active transaction is a write
transaction.  This means we only ever access ubik_currentTrans once,
avoiding any risk of the value changing between accesses (and we no
longer need to dereference it, anyway).

Note that, since ubik_currentTrans is not marked as 'volatile', some C
compilers, with certain options, can and do assume that its value will
not change between accesses, and thus only fetch the pointer value once.
This avoids the risk of NULL dereference (and thus, crash, if pointer
stores/loads are atomic), but the value pointed to by ubik_currentTrans->type
would be incorrect when the transaction ends during the execution of
SVOTE_Debug().

Change-Id: Ia36c58e5906f5e8df59936f845ae11e886e8ec38
Reviewed-on: https://gerrit.openafs.org/13915
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: Benjamin Kaduk <kaduk@mit.edu>
2019-10-22 15:18:59 -04:00
Andrew Deason
93aee3cf40 OPENAFS-SA-2019-002: Zero all server RPC args
Currently, our server-side RPC argument-handling code generated from
rxgen initializes complex arguments like so (for example, in
_RXAFS_BulkStatus):

    AFSCBFids FidsArray;
    AFSBulkStats StatArray;
    AFSCBs CBArray;
    AFSVolSync Sync;

    FidsArray.AFSCBFids_val = 0;
    FidsArray.AFSCBFids_len = 0;
    CBArray.AFSCBs_val = 0;
    CBArray.AFSCBs_len = 0;
    StatArray.AFSBulkStats_val = 0;
    StatArray.AFSBulkStats_len = 0;

This is done for any input or output arguments, but only for types we
need to free afterwards (arrays, usually). We do not do this for
simple types, like single flat structs. In the above example, we do
this for the arrays FidsArray, StatArray, and CBArray, but 'Sync' is
not initialized to anything.

If some server RPC handlers never set a value for an output argument,
this means we'll send uninitialized stack memory to our peer.
Currently this can happen in, for example,
MRXSTATS_RetrieveProcessRPCStats if 'rxi_monitor_processStats' is
unset (specifically, the 'clock_sec' and 'clock_usec' arguments are
never set when rx_enableProcessRPCStats() has not been called).

To make sure we cannot send uninitialized data to our peer, change
rxgen to instead 'memset(&arg, 0, sizeof(arg));' for every single
parameter. Using memset in this way just makes this a little simpler
inside rxgen, since all we need to do this is the name of the
argument.

With this commit, the rxgen-generated code for the above example now
looks like this:

    AFSCBFids FidsArray;
    AFSBulkStats StatArray;
    AFSCBs CBArray;
    AFSVolSync Sync;

    memset(&FidsArray, 0, sizeof(FidsArray));
    memset(&CBArray, 0, sizeof(CBArray));
    memset(&StatArray, 0, sizeof(StatsArray));
    memset(&Sync, 0, sizeof(Sync));

Change-Id: Iedccc25e50ee32bd1144e652b951496cb7dde5d2
Reviewed-on: https://gerrit.openafs.org/13914
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
2019-10-22 15:18:50 -04:00
Andrew Deason
ea276e83e3 OPENAFS-SA-2019-001: Skip server OUT args on error
Currently, part of our server-side RPC argument-handling code that's
generated from rxgen looks like this (for example):

    z_result = SRXAFS_BulkStatus(z_call, &FidsArray, &StatArray, &CBArray, &Sync);
    z_xdrs->x_op = XDR_ENCODE;
    if ((!xdr_AFSBulkStats(z_xdrs, &StatArray))
         || (!xdr_AFSCBs(z_xdrs, &CBArray))
         || (!xdr_AFSVolSync(z_xdrs, &Sync)))
            z_result = RXGEN_SS_MARSHAL;
fail:
    [...]
    return z_result;

When the server routine for implementing the RPC results a non-zero
value into z_result, the call will be aborted. However, before we
abort the call, we still call the xdr_* routines with XDR_ENCODE for
all of our output arguments. If the call has not already been aborted
for other reasons, we'll serialize the output argument data into the
Rx call. If we push more data than can fit in a single Rx packet for
the call, then we'll also send that data to the client. Many server
routines for implementing RPCs do not initialize the memory inside
their output arguments during certain errors, and so the memory may be
leaked to the peer.

To avoid this, just jump to the 'fail' label when a nonzero 'z_result'
is returned. This means we skip sending the output argument data to
the peer, but we still free any argument data that needs freeing, and
record the stats for the call (if needed). This makes the above
example now look like this:

    z_result = SRXAFS_BulkStatus(z_call, &FidsArray, &StatArray, &CBArray, &Sync);
    if (z_result)
        goto fail;
    z_xdrs->x_op = XDR_ENCODE;
    if ((!xdr_AFSBulkStats(z_xdrs, &StatArray))
         || (!xdr_AFSCBs(z_xdrs, &CBArray))
         || (!xdr_AFSVolSync(z_xdrs, &Sync)))
            z_result = RXGEN_SS_MARSHAL;
fail:
    [...]
    return z_result;

Change-Id: I2bdea2e808bb215720492b0ba6ac1a88da61b954
Reviewed-on: https://gerrit.openafs.org/13913
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-10-22 15:18:40 -04:00
Cheyenne Wills
a455452d7e LINUX 5.3: Add comments for fallthrough switch cases
With commit 6e0f1c3b45 (LINUX: Honor
--enable-checking for libafs) building libafs against a linux 5.3
kernel compiles with errors due to fall through in case statements when
--enable-checking / --enable-warning is used.

e.g.
  src/opr/jhash.h:82:17: error: this statement may fall through
                                [-Werror=implicit-fallthrough=]
         case 3 : c+=k[2];
                  ~^~~~~~

The GCC compiler will disable the implicit-fallthrough check for case
statements that contain a "special" comment ( /* fall through */ ).

Add the 'fall through' comment to indicate where fall throughs are
acceptable.

This commit only adds comments and does not alter any executable code.

The -Wimplicit-fallthrough flag was enabled globally in the linux kernel
build in 5.3-rc2 (commit: a035d552a93bb9ef6048733bb9f2a0dc857ff869
Makefile: Globally enable fall-through warning)

Change-Id: Ie6ca425e04b53a22d07b415cb8afd172af7e8081
Reviewed-on: https://gerrit.openafs.org/13881
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-10-10 23:31:40 -04:00
Marcio Barbosa
747afb94aa afs: avoid extra VL_GetEntryByName for .readonly's
In the VLDB, there's only one logical entry for a volume and its
associated clones; there are not separate entries for the RW volume
"avol", the RO volume "avol.readonly", and the BK volume
"avol.backup".  And so, when looking up a volume in the VLDB by name,
the vlserver ignores any trailing ".readonly" or ".backup" in the
given name.  More concretely, the result of calling
VL_GetEntryByName*("avol") is identical to that from calling
VL_GetEntryByName*("avol.readonly").

Accordingly, if afs_GetVolumeByName(name) failed because the volume
was not found in the VLDB, afs_GetVolumeByName(name.readonly) will
fail as well (barring a change in external circumstances, such as the
volume being created or a network connection coming back up).
Therefore, the extra call in EvalMountData() is not necessary and can
be removed.

Remove the extra call, to slightly improve the response time of the
client if the volume in question does not exist, and to reduce
vlserver load when patched clients are looking up nonexistent volumes.

Change-Id: I4f2f668107281565ae72a563a263121bd9bb7e3c
Reviewed-on: https://gerrit.openafs.org/13334
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Marcio Brito Barbosa <mbarbosa@sinenomine.net>
Reviewed-by: Cheyenne Wills <cwills@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-10-10 14:51:42 -04:00
Michael Meffie
860cbec815 RedHat: package rxstat_* programs
Install libadmin rxstat_* sample programs with 'make install'/'make
dest'. Include these programs in the openafs rpm package.

Change-Id: I81b965cf440c869072cce0065a3c74c4c699b8b8
Reviewed-on: https://gerrit.openafs.org/13883
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-10-08 06:43:08 -04:00
Cheyenne Wills
b03f3e6101 RedHat: Update makesrpm.pl to use @PACKAGE_VERSION@ instead of @VERSION@
Commit 2f2c2ce62a
'Remove automake autoconf vars' replaced the automake variable @VERSION@
with the autoconf variable @PACKAGE_VERSION@. (Gerrit #13357)

The RedHat openafs.spec.in is not processed using autoconf, but
by 'makesrpm.pl', which was not updated to use @PACKAGE_VERSION@.

Update makesprm.pl to use @PACKAGE_VERSION@ instead of @VERSION@

Change-Id: I74d1d61e40e660459942ec68cfdedfe569a6abeb
Reviewed-on: https://gerrit.openafs.org/13887
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-10-04 10:58:06 -04:00
Andrew Deason
d9fc4890f0 rx: Fix test for end of call queue for LWP
Commit 6ad3d646 (rx: Correctly test for end of call queue) fixed a
broken end-of-queue check in rx_GetCall, but it only fixed the
RX_ENABLE_LOCKS version of rx_GetCall. The non-locks version (i.e. the
LWP version) still had this bug. Fix it for the LWP case, to avoid
some rare cases where an Rx call can get stuck in the incoming queue.

Also remove the comment added by commit 170dbb3c (rx: Use opr queues),
since we're fixing the mentioned problem.

Change-Id: I5b96d97d9aba7bc4b383133b2136f949f3ed22bc
Reviewed-on: https://gerrit.openafs.org/13880
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
2019-10-04 10:32:25 -04:00
Mark Vitale
aefc4c4f46 viced: consistently enforce host thread quota for ICBS(3)
From time to time, the fileserver may issue potentially long-running
RXAFSCB_* RPCs back to a host (client).  If these are holding h_Lock_r
(host->lock) while running, they may cause other service threads for the
same host (client) to block.

In order to prevent a given host from tying up too many service threads
in this way, the fileserver enforces a quota limiting how many threads
can be waiting for h_Lock_r on a particular host while waiting for one
of the following RPCs to complete:
- RXAFSCB_TellMeABoutYourself (TMAY)
- RXAFSCB_WhoAreYou
- RXAFSCB_ProbeUuid
- RXAFSCB_InitCallBackState (ICBS)
- RXAFSCB_InitCallBackState3 (ICBS3)

Note: Although some of these RPCs are relatively lightweight, they may
still experience network delays.

This quota is enforced by calling h_threadquota() in h_Lookup_r and
h_GetHost_r.  The quota check is enabled for a given host by turning on
host->hostFlags HWHO_INPROGRESS for the duration of the RXAFSCB_* RPC.
The quota check is only needed, and should only be enabled, when the RPC
is issued while h_Lock_r is held.

However, there are a few paths to ICBS(3) where h_Lock_r is held but
HWHO_INPROGRESS is not set.  A delay in those paths may allow a host to
consume an unlimited number of fileserver threads.  One such path
observed in a field report was SRXAFS_FetchStatus -> CallPreamble ->
BreakDelayedCallBacks_r -> RXAFSCB_ICBS3.

Instead, enable host thread quotas for all remaining unregulated ICBS(3)
RPCs.

Change-Id: I70b96055ff80d8650bdbaec0302b7d18a8f22d56
Reviewed-on: https://gerrit.openafs.org/13873
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
2019-09-27 11:10:28 -04:00
Cheyenne Wills
a133f1b1e7 Retire the AFS_PTR_FMT macro
Originally '%x' was commonly used as the printf specifier for formatting
pointer values.

Commit 37fc3b0144 introduced the
AFS_PTR_FMT macro to support platform-dependent printf format specifiers
for pointer representation. This macro defined the format specifier as
'%p' for Windows, and '%x' for non-Windows platforms.

Commit 2cf12c43c6 changed the printf
pointer format specifier from '%x' to '%p' on non-Windows platforms as
well, so at this point '%p' is the printf pointer format specifier for
all supported platforms.

Since the AFS_PRT_FMT macro is no longer platform-dependent, and all C89
compilers support the '%p' specifier, retire the macro to simplify the
printf format strings.

Change-Id: I0cb13cccbe6a8d0000edd162b623ddcdb74c1cf7
Reviewed-on: https://gerrit.openafs.org/13830
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Michael Meffie <mmeffie@sinenomine.net>
2019-09-27 11:01:45 -04:00
Andrew Deason
0f1e54c47c Pass -shared when linking some shared libraries
Currently, we use $(LT_LDLIB_shlib) to build most of our shared
libraries. This invokes libtool, passing our various flags like
PTH_LDFLAGS and PTH_CFLAGS (since all of our shared-library code is for
pthreads). Notably, we do NOT pass the -shared flag; the -shared flag
tells libtool to only build a shared library, and to not also build a
static library (on systems where libtool supports building shared and
static libraries simultaneously). Because of this, our LT_LDLIB_shlib
invocations build both, which is reasonably correct for our per-module
convenience libraries (that end up getting linked statically into the
binaries that we install), but is not entirely correct for the public
libraries that we install.  Specifically, for ABI compatibility
purposes, we must provide both shared and static libraries of the public
libraries that we install, and since libtool on AIX does not build (or
install) a static library at all with --mode-link unless -static is
passed, we have separate rules to build the shared and static libraries
for final installation.

This can cause install errors with parallel make (on non-AIX systems),
and possibly other errors, when we go to install the relevant library
into TOP_LIBDIR.  For example, in src/kopenafs, we have the following
rules:

    ${TOP_LIBDIR}/libkopenafs.${SHLIB_SUFFIX}: libkopenafs.la
            ${LT_INSTALL_DATA} libkopenafs.la ${TOP_LIBDIR}/libkopenafs.la
            ${RM} ${TOP_LIBDIR}/libkopenafs.la

    ${TOP_LIBDIR}/libkopenafs.a: libkopenafs.a
            ${INSTALL_DATA} libkopenafs.a $@

The rule to install libkopenafs.so will invoke libtool to do the
install, which will install libkopenafs.so, libkopenafs.so.X.Y, and
libkopenafs.a (from .libs/libkopenafs.a, not the libkopenafs.a we
built separately). If we are running the rule to install libkopenafs.a
in parallel, it may fail with an error like so:

    /usr/bin/install -c -m 644 libkopenafs.a /home/buildbot/openafs/fedora26-x86_64/build/lib/libkopenafs.a
    /usr/bin/install: cannot create regular file '/home/buildbot/openafs/fedora26-x86_64/build/lib/libkopenafs.a': File exists
    make[3]: *** [Makefile:35: /home/buildbot/openafs/fedora26-x86_64/build/lib/libkopenafs.a] Error 1

Even without that error, this confusion means that the libkopenafs.a
installed into TOP_LIBDIR may be the one from
src/kopenafs/libkopenafs.a, or the one from libtool's
src/kopenafs/.libs/libkopenafs.a; it depends on what order the rules
are run. If those libraries are different, that could potentially
cause all sorts of other problems.

To avoid this, we can pass -shared to libtool when building our shared
libraries. We used to pass -shared when building shared libraries,
since -shared is almost always one our SHLIB_LDFLAGS set in
src/osconf.m4. However, ever since commit 2c3a517e (Retire
Makefile.shared), SHD_CFLAGS, SHD_LDFLAGS, and SHD_CCRULE have all
been unused, and SHD_LDFLAGS was the only place where we used
SHLIB_LDFLAGS. As a result, we never use SHLIB_LDFLAGS anywhere, and
so we never pass -shared to anything.

However, we cannot pass -shared to libtool when building all of our
shared libraries, since we do need the static library for our per-module
convenience libraries. For example, liboafs_rx.la has no
separately-built static library (librx.a is for LWP, liboafs_rx.{so,a}
is for pthreads), but liboafs_rx needs to be linked statically into all
of our command-line tools.

So to fix this, introduce a new linking rule, called
LT_LDLIB_shlib_only, which causes the given library to be built only
as a shared library (by giving -shared to libtool), and not as a
static library. Update the build rules to use this new linking rule
for the libraries that need it, and leave the others alone. Since the
only use of LT_LDLIB_shlib_missing is also used for a public library
(afshcrypto), also pass -shared in that rule.

Also remove SHD_* and SHLIB_LDFLAGS variables, since they are unused.

Change-Id: Ia9e040afa3819f1ff70d050a400fecb9624bb9ba
Reviewed-on: https://gerrit.openafs.org/13786
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
2019-09-27 10:47:15 -04:00
Yadavendra Yadav
1de602aaad aklog: avoid infinite lifetime tokens by default
Currently we get tokens for infinite lifetime using aklog impersonate
feature. Based on inputs from Ben, this was done for server to server
tickets to be valid forever.  However on 1.8.x we have other
mechanisms that were usable for server-to-server authentication with
strong enctypes, so we do not need to provide user level akimpersonate
to generate tokens for infinite lifetime. For this we have added new
option -token-lifetime <hrs>, this can take values from 0 to 720
hours. If 0 is specified it means tokens will have infinite lifetime.
By default 10 hours will be token lifetime for akimpersonate tokens.

Change-Id: I8190be81771b34682cc000ac051888561dc63c2f
Reviewed-on: https://gerrit.openafs.org/13828
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Tested-by: Benjamin Kaduk <kaduk@mit.edu>
2019-09-23 17:13:36 -04:00