The background daemon worker pool is responsible for processing
background Store and Fetch operations. With the SMB interface
primary store and fetch operations are performed in the SMB worker
thread which makes sense since those operations must be synchronous
to the incoming request.
With the AFS redirector interface almost all of the work is performed
by the background daemon worker pool. It is therefore critical that
the workers not get stuck in a state that starves applications.
For example, copy of a file that is larger than the cache to \\AFS
will result in a background store request for each chunk size of
the file. If each worker thread grabs one to process, only one will
make progress and the rest will block. If a cleanup operation
(aka handle close) occurs the entire file will be flushed to the
server synchronously in the redirector worker thread. That thread
will cause of the background daemon threads to block.
Any subsequent fetch data requests that get queued behind the list
of stores will in turn block until they clear. This behavior is not
fair.
This patchset adds a new test to the cm_BkgDaemon() request
selection loop, cm_RequestWillBlock(). If a request will block it
is skipped. If there are no requests to process that would not have
blocked, the worker will sleep for 25ms instead of the usual 1s.
For BkgStore operations, the CM_SCACHEFLAG_DATASTORING flag is
used to indicating a blocking state.
For BkgFetch and PreFetch operations, the CM_BUF_WRITING and
CM_BUF_READING flags on the first cm_buf_t of the range is used
to indicate a blocking state.
Change-Id: I95d9d1f14dbe0c7d717e6a7253ccfb10a9fac851
Reviewed-on: http://gerrit.openafs.org/6056
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Failure to use interlocked operations in cm_GetSCache can
result in an under count
Change-Id: I370bcb9ac8b40f1c1b7ea3b7647fc76e3a554e2c
Reviewed-on: http://gerrit.openafs.org/6046
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
smb_ReceiveNTCreateX() calls cm_CheckNTOpen() which now
requires the smb_fid_t allocated fid value for use in share
mode locking. Move the allocation of the smb_fid earlier
in the function and apply necessary cleanup in error paths.
Change-Id: Iee0e39cdf393bfd245829bdfb42ddfba63908451
Reviewed-on: http://gerrit.openafs.org/6004
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
cm_GetSCache used to hold cm_scacheLock write-locked from
start to finish except that it didn't. There were several
places where cm_scacheLock was dropped and reacquired due
to lock ordering requirements. Unfortunately, this has
two problems. First, the function isn't very fast in the
most common case since cm_scacheLock is write-locked for
the search for an existing FID. Second, there is a race
that results when cm_GetNewSCache() drops the cm_scacheLock.
To make things faster, use a read-lock for the common case.
To avoid the race, if the FID cannot be located, call
cm_GetNewSCache() first and then obtain the cell and volume
information. Then perform a second lookup for the FID while
holding cm_scacheLock write-locked. If we lost the race or
there was an error obtaining the cell and volume info, put
the new cm_scache_t back onto the end of the LRU queue.
Change-Id: Idb94275d23c160ee0a2dc8fdcfd0f09506ecb2d3
Reviewed-on: http://gerrit.openafs.org/6003
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
If the cm_scache_t cannot be obtained, do not dereference it
while returning an error to the redirector.
Change-Id: Ia8aaa33ca4ab410dc1b0100ea5668d5660d90b75
Reviewed-on: http://gerrit.openafs.org/6002
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
It has been noticed that multiple RPCs can be active on
a cm_scache_t object at the same time. This is especially
true of directory objects with the redirector. Track the
number of active RPCs and use that number in cm_MergeStatus
when deciding whether or not to discard the cached data for
the object.
Change-Id: If291bb4c0e48d0ec087c3a7c2640e4598e7fd419
Reviewed-on: http://gerrit.openafs.org/6001
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
The smb username lock and the daemon global lock can be requested
while the scache dirlock is held if there are no free buffers
and the service is forced to claw back extents from the redirector.
Adjust the locking hierarchy accordingly.
Change-Id: I85387a16ca580d678af45f3931aa5e81fe0a0f2c
Reviewed-on: http://gerrit.openafs.org/6000
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
The array of srvAddr structs we use in afs_LoopServers have indices
unrelated to the indices of conns, rxconns, etc. Several places were
assuming that addr[i] corresponded to conn[i], which is not
necessarily true. So instead, do not use the separate addr array
(except when populating the conn and rxconn arrays), and just get the
srvAddr structure by going through the relevant conn[i].
Change-Id: Ib60126811f51943bfc81ab7c48d7f247f00f1cad
Reviewed-on: http://gerrit.openafs.org/5790
Reviewed-by: Marc Dionne <marc.c.dionne@gmail.com>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
Turn rxevent_Cancel into a function rather than a macro which modifies
its argument as a side effect. rxevent_Cancel now checks whether the
event being cancelled is already NULL, as well as NULLifying the event
when it is actually cancelled.
Update all of the callers to reflect this new API, and so they no
longer do unecessary work.
Change-Id: I75b68f1c8f1a3023edd6113600663fe2b60d6097
Reviewed-on: http://gerrit.openafs.org/5840
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
For a while now, we've had both new and old-style rx event callback
functions. Modify all of our event handlers, and the functions that
install them, to use only new style functions, and get rid of the
old-style function prototypes.
Change-Id: Ic7c568df9d191edb082fb41fb7705c54ca93cf48
Reviewed-on: http://gerrit.openafs.org/5839
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
Add an implementation of red/black trees to our runtime library.
This is originally derived from the FreeBSD macro-based rbtree
implementation, but is heavily reworked to not use macros, to improve
legibility, and to favour speed over structure compactness.
A test suite is provided in tests/opr/
Change-Id: I123209d3f89b5f8c1b85d1e5cd7d1d650ccc68ed
Reviewed-on: http://gerrit.openafs.org/5838
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
The code to schedule a new delayed ack event is distributed throughout
the RX code. Consolidate this into a single helper function.
Change-Id: If54e36b5648f8caffe64cc2203dc0041fd06b8b3
Reviewed-on: http://gerrit.openafs.org/5837
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
Add the opr_containerof macro, which can be used to find the base
address of a structure which contains a member whose location is known.
This formulation is heavily used throughout OpenAFS to determine the
base address of structures containing queue pointers - this provides
a central definition, rather than coding it from scratch each time.
Change-Id: I2a32a214252ad304238f740ad92579e5bb7287be
Reviewed-on: http://gerrit.openafs.org/5836
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
Make use of the INetFwPolicy2 COM interface to configure
the Vista/Win7 firewall for all network profile types.
Change-Id: Id96d98f6e166f3529a117997f5a3c35b0051d1a4
Reviewed-on: http://gerrit.openafs.org/5812
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Use file server locks to enforce file share access modes
via the afs redirector interface. The approach taken
integrates share mode enforcement with the file server
lock tracking code in the service. The share mode
enforcement mimics that of the SMB Server interface.
This patchset includes two functional changes to
the previous locking and share mode processing:
1. The cm_scache_t fsLockCount field is used to
determine if the desired lock can be granted
by the file server. If not, the RXAFS_SetLock()
request is skipped and the request is failed
locally.
2. cm_CheckNTOpen() now accepts the desired and
and share access modes. The share access mode
is used to determine if a test lock should be
obtained at all. If the share mode is FILE_SHARE_WRITE
then no lock is requested. This change permits
Microsoft Office applications to offer the user
the ability to open the file in read-only mode
and notify the user when the document can be
opened in read-write mode.
Developed with Peter Scott <pscott@kerneldrivers.com>
FIXES 130239
Change-Id: If9b4beb74586296d4feae52dd06c03f0b8b595cd
Reviewed-on: http://gerrit.openafs.org/5823
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Oracle Solaris 11 no longer supports x86 (amd64 is required). If we
try to build the x86 module, /usr/include/sys/kobj.h complains that
the ISA is unsupported, and refuses to go on. So, just remove
MODLOAD32 from the libafs directories to build on sunx86_511.
Change-Id: Ic18936d1a0f53ed290d885617d313347aa6ce87a
Reviewed-on: http://gerrit.openafs.org/5835
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
We just unlinked the file, so we know we won't be able to rmdir() the
same thing. Give a path one level higher to
namei_RemoveDataDirectories, so we start rmdir()ing at the parent dir.
Change-Id: Ifa54d1cc07ed2d277416d09eabc45aa1e1c3803a
Reviewed-on: http://gerrit.openafs.org/5833
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
A few places were specifying both O_EXCL and O_TRUNC to open().
O_TRUNC does not make any sense with O_EXCL, and doesn't do anything,
so remove O_TRUNC from these instances to make the code more clear.
Change-Id: Ie8d9d7b9912a96683ef34f762607e75468b5ac78
Reviewed-on: http://gerrit.openafs.org/5832
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
Since FSCTL_SET_REPARSE_POINT is not yet implemented
return STATUS_INVALID_PARAMETER to all requests that
do not fail validity tests.
Change-Id: I53d4b4fc9d957daa70b16d37a529778ed8bbfa5e
Reviewed-on: http://gerrit.openafs.org/5829
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
According to MS_FSCC 2.3.54 if the input buffer length is less than the size
of a REPARSE_DATA_BUFFER structure, or the input buffer length is greater
than 16,384, or a REPARSE_DATA_BUFFER structure has been specified for a
third party reparse tag, or the GUID specified for a third party reparse tag
does not match the GUID known by the operating system for this reparse
point, or the reparse tag is 0 or 1, then the return status shall be
STATUS_IO_REPARSE_DATA_INVALID.
Change-Id: I0072902afc8bc354af8f9b84b1021b37191dc23a
Reviewed-on: http://gerrit.openafs.org/5828
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Windows NT, Windows 2000, Windows XP, Windows Server 2003, Windows Vista,
Windows Server 2008, Windows 7, and Windows Server 2008 R2 support the
FSCTL_IS_PATHNAME_VALID Request (section 2.3.21) and return STATUS_SUCCESS
whenever this request is invoked. We will do the same.
Change-Id: I432c8a41e1102648faaa3d8c0f3b8c36ea89bfc8
Reviewed-on: http://gerrit.openafs.org/5827
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
When populating the directory entry information for the
redirector the service must set the reparse point attribute.
Change-Id: I26db0b822f850f0a19606eeed3c29e0e72562e2f
Reviewed-on: http://gerrit.openafs.org/5824
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
The file server will set the rx call status bit (0x1)
when the rpc is in process and all of the locks are held.
At this point it is not possible for another store data rpc
to begin on the vnode prior to the completion of the current
rpc. Once this status bit is detected as set, the exclusive
store data synchronization on the cm_scache_t can be dropped.
This permits the next store data rpc to perform its biod
construction.
Change-Id: Ic856769650781b4f5f4ab4ac86df4946496bd655
Reviewed-on: http://gerrit.openafs.org/5741
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Some of the routines during a volume restore may set needsSalvaged, if
an inconsistency is detected while writing the given volume data.
However, after the data is read, we set the volume header information
to what was found in the dump stream, ignoring any needsSalvaged that
may have been set.
To ensure that inconsistent volumes in this situation actually get
demand-salvaged (for DAFS) or offlined (non-DAFS), keep the value of
needsSalvaged in the header, if it was set.
Change-Id: I575d14d84120be0fbbeb185ed2726f1285e6f2ce
Reviewed-on: http://gerrit.openafs.org/5822
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
because fileptr's handle can be reset during CopyOnWrite
we need to SetDirHandle after that call
FIXES 130215
Change-Id: I8578d30da42bdd9be2c552cdac4d2c526c22ee45
Reviewed-on: http://gerrit.openafs.org/5818
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
Add the capability to do a "pretty" build, where we output something
like " CC /path/to/foo.o" to build foo.o, instead of the entire
compiler invocation, similarly to how the Linux kernel build appears.
Add the "pretty" building for CC and LD rules.
This also prints out some helpful information when a command fails,
which can sometimes otherwise be annoying to figure out post-mortem.
To enable the pretty building, make with V=0. To output everything
that is actually run with V=0, make with 'V=0 Q=' .
Note that this does not work with all makes, since not all makes will
propagate command-line-specified variables to sub-makes without -e.
Non-working makes include /usr/ccs/bin/make on HP-UX and Solaris.
However, GNU make will work, as will /usr/xpg4/bin/make on Solaris.
Change-Id: Idce0afffe7d5be88b0743ec3f926a52efb1a6a74
Reviewed-on: http://gerrit.openafs.org/4486
Reviewed-by: Russ Allbery <rra@stanford.edu>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
A few makefiles specify an old-style suffix rule, such as:
.c.o:
$(AFS_CCRULE) $<
Not all makes seem to interpret these rules correctly (such as Solaris
/usr/xpg4/bin/make). Since it is easy to do so, specify pattern-based
rules along with these, like so:
%.o: %.c
$(AFS_CCRULE) $<
Change-Id: I052f1156d1a7e29beedb0fb59390073c2521459e
Reviewed-on: http://gerrit.openafs.org/5819
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
Add the replacement setenv and unsetenv functions to roken, for use on
(at least) HP-UX and older Solaris.
Change-Id: Id05137b6e6c88eb6648bd6a6e649f9ef311e32a4
Reviewed-on: http://gerrit.openafs.org/5801
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
we don't assign hd before dereferencing. stop dereferencing
and just use the queue cursor.
Change-Id: I023fd959c5d49575dbdf89fd4dab77341c5c52d0
Reviewed-on: http://gerrit.openafs.org/5814
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
Instead of probing the DAFS-ness of the fileserver by probing which
FSSYNC opcodes it supports, detect DAFS-ness by looking at the SYNC
response header flags, which explicitly state whether or not the
endpoint is DAFS. This avoids unnecessary "protocol mismatch" log
messages when the endpoint is not DAFS.
Change-Id: Ie05a587951d6d5f69ae03cf3749f8c53e1eb6b62
Reviewed-on: http://gerrit.openafs.org/5800
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
the straight return from call_syscall isn't what we want. munge so
e.g. EDOM handling for ktc_GetToken does the right thing.
Change-Id: I3fbba799e0156bad29b27c74a1c7709ee8c50c34
Reviewed-on: http://gerrit.openafs.org/5807
Reviewed-by: Derrick Brashear <shadow@dementix.org>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
This workaround is no longer relevant. It is also error-prone and
there are far better ways to achieve a similar result, so get rid of
it.
Change-Id: If77f0cd95c8c608913d0c8717729a05c1ad67df9
Reviewed-on: http://gerrit.openafs.org/5811
Reviewed-by: Simon Wilkinson <sxw@inf.ed.ac.uk>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
While the -log option to volserver is supposed to print additional log
information, it shouldn't spam the log with useless data. Remove some
of the log lines that are really more "debug" information, so we log
the same amount of information as in the 1.4 series.
Change-Id: I1fd93344a7ebe11c6d5ad507dad87babe517785a
Reviewed-on: http://gerrit.openafs.org/5810
Reviewed-by: Derrick Brashear <shadow@dementix.org>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Resolve a buffer leak in afs_readdir that was visible on all non-Linux
platforms (Linux has an independent readdir implementation)
Change-Id: Ia5106f0e5c57430d3483b6add5b1629ff3fbc7b3
Reviewed-on: http://gerrit.openafs.org/5794
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
GetVolumeHeader assumes that headers on the LRU are not associated
with a volume in an exclusive state. This is known to not be true for
some cases when salvage requests are received over FSSYNC, and may be
true in other scenarios. It's easy to just skip such headers, so skip
them.
Change-Id: I343d2e4f7af1c753641767be322420f38873c217
Reviewed-on: http://gerrit.openafs.org/5808
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
The RemoteNameLength passed in the IOCTL_AFS_CANCEL_CONNECTION call
must not include the trailing NUL. NPGetConnection() returns the
size of the buffer used which does include the trailing NUL.
Change-Id: Ic5eae51aade91db4a799bea83d898af6fbc37504
Reviewed-on: http://gerrit.openafs.org/5806
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Track whether or not the pFcb->NPFcb->Resource is held so
it can be released in all error paths.
Change-Id: I8c8bfb912329e8c11ac7857086412d85aa64fea6
Reviewed-on: http://gerrit.openafs.org/5804
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
During the Heimdal transition that default ccache type
was changed to FILE. Restore the use of API: ccaches.
Change-Id: Idd8dfb46fc9167d8c1eb44466b76906a152eef9d
Reviewed-on: http://gerrit.openafs.org/5802
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Reduce the complexity of KFW_AFS_klog. Introduce
KFW_AFS_continue_aklog_processing_after_krb5_error() and
combine the input realm and realm_of_cell cases making
use of the RealmName variable.
Change-Id: I363544548652bc892f50dce00865e13e7d42de7f
Reviewed-on: http://gerrit.openafs.org/5799
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@secure-endpoints.com>
Tested-by: Jeffrey Altman <jaltman@secure-endpoints.com>
A user on the openafs-info mailing list noted that the Admin Guide is
unclear about creating read-only replicas on different partitions on
the same fileserver. Clarify the rules here.
Change-Id: I8229b4fba36c128772bb8198e6b8310a51206c03
Reviewed-on: http://gerrit.openafs.org/5745
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Derrick Brashear <shadow@dementix.org>
