remove SUPERGROUPS condition for compilation of pts commands:
Interactive, Quit, Source, Sleep
fix the assignment of 'source' to permit it to function
rename functions to avoid conflicts with Win32 API on Windows
replace bcopy and bzero with memcpy and memset to permit compilation
on Windows
replace ubik_Call(xxx) with ubik_xxx() (testpt.c)
add -DSUPERGROUPS and map.c to NTMakefile
New registry value "BlockSize" can be used to specify an alternative
block size. The default is 4K. A larger blocksize will be needed if
you want to support a 6TB cache.
Also extend the service startup timeout hint to two minutes to give
the AFS client service more time to startup successfully when the
cache is really large.
when storing dirty buffers only store the bytes that are dirty
increase the default chunksize from 128KB to 1MB
remove OVERLAPPED field from cm_buf_t. It was unused.
add a dummy pointer in order to ensure data structure compatibility
between the checked and release builds.
(1) fixes a bug that could cause a 'host' structure to not be removed
from the global host list if the 'host' did not possess an interface
list. This would happen with older AFS clients that do not support the
WhoAreYou family of RPCs. Windows clients older than 1.3.80 and old
Transarc UNIX clients.
(2) fixes a bug which could result in ViceLog being called with an
uninitialized 'hoststr' buffer as a parameter.
(3) ensures that only addresses known to belong to the 'host' are
added to the address hash table. The list of addresses provided by
the client are stored as alternates and are only used when searching
for a client that is no longer accessible on the primary address.
These addresses are not stored in the address hash table within
initInterfaceAddr_r().
The addresses provided by the client should not be added to the hash
table because they have not been verified as belonging to the 'host'
that provided them. The contents of the list may in fact be completely
unreliable. Consider the existing UNIX clients that generate the list
at startup and never alter it even after the client has migrated to a
different network. If two client's both claim the same address,
lookups by address may fail to find the correct one.
a. The client list might contain private address ranges which
are likely to be re-used by many clients allocated addresses
by a NAT.
b. The client list will not include any public addresses that
are hidden by a NAT.
c. Private address ranges that are exposed to the server will
be obtained from the rx connections that use them.
d. Lists provided by the client are not necessarily truthful.
Many existing clients (UNIX) do not refresh the IP address
list as the actual assigned addresses change. The end result
is that they report the initial address list for the lifetime
of the process. In other words, a client can report addresses
that they are in fact not using. Adding these addresses to
the host interface list without verification is not only
pointless, it is downright dangerous.
e. The reported addresses do not include port numbers and
guessing that the port number is 7001 does not work when
port mapping devices such as NATs or some VPNs are in
use.
(4) improves logging to ensure that all references to a 'host' structure
report both a memory address and the IP address/port. this will avoid
confusion *if* more than one 'host' structure is assigned the same
primary address.
(5) logs the UUID along with the client addresses when initializing the
host's interface list. (level 125)
(6) saves memory by using a smaller structure for the UUID hash table
FIXES 67355
MultiProbeAlternateAddress_r badly indexes the list of interfaces for
clients with multiple IP interfaces, resulting in peers with IP
address 0 port 0 to be created. This in turn results in rxi_sendmsg
errors (on systems where caught early, as on Linux, on others it may
pass unnoticed).
The windows cache manager has suffered from poor performance as a result
of Create, Rename, and Delete operations because they invalidate the
contents of the directory pages in the cache thereby forcing them to be
reloaded from the file server. As the directory size increases, the clock
time necessary to perform the reload increases.
This delta adds support for parsing and updating the AFS3 directory buffers
to cm_dir.c. It then uses that functionality to perform local updates to
the directory buffers whenever the following conditions are met:
1. the data version on the directory as a result of the change
was incremented by one.
2. all of the directory buffers required for the update are in
the cache.
If these conditions are not met, the directory is reloaded from the file
server.
FIXES 15855
In cm_NameI, keep track of what fids were crossed when evaluating the
path. If we discover a loop return a CM_ERROR_TOO_MANY_SYMLINKS error.
This resolves the complaint that when infinite loops are hit in the afs
name space the afsd_service.exe process uses 100% of the CPU.
The windows cache manager has suffered from poor performance as a result
of Create, Rename, and Delete operations because they invalidate the
contents of the directory pages in the cache thereby forcing them to be
reloaded from the file server. As the directory size increases, the clock
time necessary to perform the reload increases.
This delta adds support for parsing and updating the AFS3 directory buffers
to cm_dir.c. It then uses that functionality to perform local updates to
the directory buffers whenever the following conditions are met:
1. the data version on the directory as a result of the change
was incremented by one.
2. all of the directory buffers required for the update are in
the cache.
If these conditions are not met, the directory is reloaded from the file
server.
if all of the servers are down when a callback is due to expire
delay the expiration until at least one server is available.
this prevents some applications that are running when the CM
is off the network from failing if their pages are swapped out.
FIXES 66302
change text of error message to indicate that the user is not a member
of the AFS Administrators Group instead of not being a Windows administrator.
don't try so hard to give up all callbacks. If the server doesn't
respond in 10 seconds, too bad!
cleanup the server probe code a bit. reorganize the code so that we
can avoid unnecessary pointer evaluation. add a missing include file.
* Do not give back callbacks to down servers
* Output more cm_scache_t data in afsd_alloc.log
* call VolStatus_Service_Stopped after the service has stopped
This delta adds an interface to an optional volume status handler.
The handler (if provided) receives status updates when volumes
change state between online, offline, busy, and alldown.
enable afsdb records for get cellinfo lookup outside of afsd_service.exe
====================
This delta was composed from multiple commits as part of the CVS->Git migration.
The checkin message with each commit was inconsistent.
The following are the additional commit messages.
====================
do not perform afsdb lookup for Freelance.Local.Root cell
The UNIX client does not follow mount points or symlinks when evaluating
ioctl paths during commands such as "fs examine". The Windows client did
which was annoying when you wanted to know the FID of a mount point that
was not properly being evaluated.
Since the library creates its own background thread, the library must
load its own reference to itself to prevent the library from being
unloaded behind its back.
remove the conditionalized code used to give up callbacks in response
to stat cache recycling due to performance impacts described in the
commit for DELTA windows-give-up-callbacks-20070627
This large patch adds support for giving up callbacks in response to three
events:
1. power management suspend
2. power management shutdown
3. stat cache object recycling
The third item is submitted as a condition compilation if GIVE_UP_CALLBACKS
is defined. Properly handing callback give ups and the associated race
conditions with revokes and fetch status requests requires a great deal of
over head. The first attempt used one GiveUpCallBacks RPC for each callback
that was being dropped as the stat cache object was recycled. This resulted
in a 27% performance drop in the MIT stress test. The code that is being
committed maintains a callback give up list on each server object. The
callback is added to the list as the callbacks are dropped and then they
are sent to the server in bulk by the background daemon thread if the
server is known to be UP after a ping. Logic is added to the
EndCallbackRequest and CallbackRevoke operations to ensure that race
conditions are addressed. With all of this, there is a 17% performance drop
in the MIT stress test.
As a result, it is my conclusion that the client side costs associated with
optimizing the load on the server are simply too high. I am committing this
code to ensure that it is not lost. I will remove this support in the next
patch while leaving the support for giving up all callbacks in response
to suspend and shutdown events.
FIXES 63763
probe for something else for 2.4 and older
====================
This delta was composed from multiple commits as part of the CVS->Git migration.
The checkin message with each commit was inconsistent.
The following are the additional commit messages.
====================
i'll spare you
return an error when the cm_fid_t * is NULL since we can't look up
the volume to obtain a server list without knowing which volume we
should be looking up
if the fidp is known to be NULL, don't call cm_GetServerList()
Add name and ID hash tables for cell lookups. cell lookups occur on
every request. sometimes multiple times. removing the walking of the
cell list when there are dozens of cells decreases cpu utilization and
increases throughput.
there were two sets of registry values that could be used to configure
the daemon thread check intervals. keep the one that was documented
in the release notes and discard the other.
Add a registry value "daemonCheckOfflineVolInterval" to configure the
offline volume check interval.
Ensure that the cm_GetConn... functions initialized the output variables
to NULL on error.
When we are faking the status data we can use the vnode value to determine
if the object should be treated as a directory or file. even is a directory
and odd is a file. This works even when we have never successfully
obtained status data for the object.
If the we can match up the host address from which the revoke was received
with one of our cm_server_t objects, then we know which cell the revoke
has been received from. With that information we can ensure that we only
revoke the status of cm_scache_t objects belonging to that cell.
Reverse the order of the allCellsp list. Append new cells onto the end
of the list. This ensures that the workstation cell will always be the
first in the list. Adding additional cells will not degrade the performance
to the workstation cell.
No longer permit cm_GetCell() or cm_FindCellByID() to return NULL simply
because cm_UpdateCell() failed. The cm_cell_t object still exists and
is valid even if the vlServersp list is empty.
Modify the lock management in cm_GetCell_Gen() to ensure we drop all the
locks.
In cm_Analyze() update the volume status when one of the servers reports
VBUSY or VRESTARTING.
fix deadlock on cm_volumeLock introduced by last week's work
in cm_Analyze, make sure we get a cm_cell_t reference otherwise we
won't find the cm_volume_t we are searching for when ALLOFFLINE or
ALLBUSY.
VMWare adapters have proven unreliable replacements for the Microsoft
loopback adapter. Registering AFS often results in a name space collision.
Add cm_DumpCells() function and dump the cells as part of "fs memdump"
Dump all cm_scache_t and cm_volume_t regardless of reference counts
Fix cm_GetCell_Gen() to not allocate a new cm_cell_t when evaluating
mount points to aliases. Instead, after looking up the alias successfully
search the allCellsp list for the fullname of the cell. If found, use
the existing entry and cleanup the one we were about to allocate.
Use read locks whenever possible instead of write locks when searching
the allCellsp list.
Don't assume that WM_DESTROY is the final message received by a
window. Verify dialog data structures when handling messages and
reset the window data field when freeing the data structure.
Zero should be considered a valid credentials type identifier in
Network Identity Manager.
When checking if an identity is configured to obtain a token for a
specific cell, don't go through the list of cells if AFS tokens
are disabled for the identity.
Similarly, when removing a token for a specific cell from all
identities, don't bother modifying identities for whom AFS tokens
are disabled.
Keep track of whether a specific cell was added to the list of
cells to authenticate for an identity because it was listed in the
configuration or because a token for the cell already existed.
Correct an off-by-one error when calculating buffer sizes for
multi strings which failed to account for a double NULL
terminator.
Don't update the cell->identity mapping if a token for that cell
could not be obtained.
If the list of cell to authenticate for an identity is empty, we
still need to write the empty string to the configuration.
Otherwise, removing all the tokens from an identity will not
result in a configuration change reflecting that.
fix cm_IoctlPathAvailability to return the current volume state.
0, CM_ERROR_ALLBUSY, CM_ERROR_ALLDOWN, CM_ERROR_ALLOFFLINE
modify fs.c to generate messages when the errors are received.
When the system's IP address list changes we invalidate the existing
RX connections and probe all of the servers. A better algorithm is
to probe all vldb servers, invalidate the rx connections, and then
probe all file servers.
update the lwp version of rxi_sendmsg to return the same error, -1,
returned by the pthread version.
replace errno with WSAGetLastError() in the Windows blocks so that
the correct error value is checked.
FIXES 61906
2.6.21.1 introduces an additional .parent pointer in the middle of
the structure. As the OpenAFS code just initialises the structure
with a list, this causes it to assign the value intended
for .proc_handler to .parent
* re-write cm_Analyze to make better use of the known volume
status. VL_Server queries cannot result in CM_ERROR_ALLOFFLINE
messages.
* renamed cm_CheckBusyVolumes to cm_CheckOfflineVolumes.
busy volumes will be reset to srv_non_busy by the function
but there is no mechanism for querying the busy state other
than by attempting to access the resource.
* cm_Analyze will query the state of an offline volume before
deciding whether or not to retry when all volume instances
are offline.
FIXES 61767
1 - task_struct loses thread_info, which is now accessible through the
task_thread_info() macro. A configure test is added to deal with this.
2 - the SLAB_CTOR_VERIFY flag is gone
* changed the enum values for cm_serverRef_t state info to use a
private name space to avoid collisions (srv_)
* added a srv_deleted state for cm_serverRef_t objects. This
state is set when cm_FreeServerList() is called with the
CM_FREESERVERLIST_DELETE flag set. cm_FreeServerList() may
not always delete the cm_serverRef_t from the list if it is
still in use by another thread. the srv_deleted state means
the object's contents are no longer valid and it must be
skipped. It will be deleted the next time the object is
freed and the refcount hits zero.
* the srv_deleted state is also used when a file server reports
either VNOVOL or VMOVED instead of marking the cm_serverRef_t
as offline. This is done to prevent additional usage of the
stale vldb data while waiting for the update volume request
to complete.
* added a state field to the cm_volume_t object (enum volstate
vl_ name space) that maintains the state of the volume based
upon the states of all of the cm_serverRef_t and cm_server_t
objects.
* modified cm_UpdateVolume() to set the state of the cm_volume_t
RW, RO, and BK to either vl_alldown or vl_online. There can't
be any other states because cm_UpdateVolume() destroys any
previous knowledge we might have had regarding busy or offline
volume status
* modified cm_UpdateVolume() to update the volume name in the
cm_volume_t to the volume base name if the previous value was
a volume ID.
* modified cm_FollowMountPoint() to check to see if the volume
name is a volume ID and if so call cm_GetVolumeByID instead
of cm_GetVolumeByName. This ensures that volume IDs are always
looked up as numeric values. There is no longer a need to
maintain a separate cm_volume_t containing the string representation
of the ID value.
* Added a flags parameter to cm_GetVolumeByName() and cm_GetVolumeByID().
The first flag is a "CREATE" flag which is set by all existing
calls. The flag is not set by calls to cm_GetVolumeByID() from
the server probe code when volume status is being updated. We
do not want the server probe operation to result in additional
turnover in the cached volume data. The second flag is NO_LRU_UPDATE
which is set when the server probe code updates the volume status.
This flag will be used to prevent the server probe operation from
changing the order of the least recently used queue.
* Modified cm_GetVolumeByName to ensure that only one cm_volume_t is
allocated for a given set of normal, readonly, and backup volumes
regardless of whether or not the volume is accessed via name or
ID number. The cm_volume_t namep field is always the base name
of the volume.
* Added a new volume state, vl_unknown. This state is used as
the initial state for all cm_volume_t when the cache manager starts,
for each cm_volume_t at creation, and for each cm_volume_t when
recycling. The cache manager does not know the state of all
volumes in the world, only those that are in the cache and for
which it has queried the VLDB and hosting file servers.
* modified cm_GetVolumeByName() to initialize the state of a
volume to vl_unknown. The actual state will be set when a
cm_VolumeUpdate() call completes successfully.
* changed name of scache hash table variables to avoid ambiguity
when adding hash tables for volumes
* fix a buffer overrun in sys\pioctl_nt.c pioctl().
(thanks Asanka)
* modified cm_UpdateVolume() to handle the case in which there is
no RW volume but there is are RO volumes for a given base name.
This is done by querying for the ".readonly" volume name if the
base name does not exist in the VLDB. We never query for the
.backup name because under the current usage model a .backup
volume may only exist on the server that the read-write volume
is located. If there is no RW volume, there can be no .backup.
* Added four hash tables for cm_volume_t objects to improve the
search time of cm_GetVolumeByID(), cm_GetVolumeByName() and
cm_ForceUpdateVolume(). One each for Name, RWID, ROID, and
BKID. Three ID hash tables are necessary as long as it is
desireable to maintain a single cm_volume_t containing all
of the related RW, RO, and BK volume data. Having the RW and
RO volume data in the same object is necessary for the
implementation of cm_GetROVolumeID() which returns either the
RO or RW ID depending upon the existence of RO volume instances.
* Added a volume LRU queue so that volume reuse becomes fairer.
This does not replace the all Volumes list which is used when
it is desireable to walk a list of all the volumes whose order
is not going to change out from underneath you which makes it
safe to drop the cm_volumeLock.
* handles volume hash table updates where volume name to
volume ID number changes. The volume name remains
constant in the cm_volume_t. if a vos rename is performed,
the name of the volume will change and the volume IDs will be
updated. Subsequent access to the old volume ID will create a
new cm_volume_t with the new name.
* Added a daemon thread operation to query the state of volumes
listed as busy or offline. cm_CheckBusyVolumes() calls
RXAFS_GetVolumeStatus() for each volume ID that is marked vl_busy
or vl_offline. If the volume is now online, the status on the
volume is updated. The default period is 600 seconds. This can
be configured with the BusyVolumeCheckInterval registry value.
* Added prototype for smb_IoctlPrepareRead() which was missing a
return type in the function definition.
* Added volume id lists to the cm_server_t. These lists are
allocated in blocks of ~32 IDs. When a cm_PingServer()
detects a change in server state, the state of the cm_volume_t
is updated.
* Added volID to the cm_serverRef_t object. volID is used
to identify the volume for which the object is a referral.
cm_FreeServerList() uses the volID to remove the volume
from the cm_server_t.
* In cm_Analyze, when VNOVOL or VMOVED are received,
call cm_ForceVolumeUpdate() to force a refresh of the volume
location data.
* Added cm_VolumeStatusNotification() which is used at the moment
to log volume status changes to the trace log. It will also
be used as the access point to the File System Filter driver
notification engine.
* Added an all cm_scache_t list to cm_data. This replaces the use
of the stat cache LRU queue when we need to enumerate all
entries. The LRU list order is not static and when using it to
enumerate all entries it can result in items being missed or
items being processed more than once.
* Modified cm_Analyze(). Instead of reseting the busy or offline
state of a volume and forcing a retry of the operation
cm_Analyze will defer to the background daemon thread that will
update the state once every 600 seconds.
* Added the automatic generation of a Freelance ".root" read-write
mountpoint that refers to the root.afs volume of the workstation
cellname at the time the mountpoint is created.
In rxkad_CheckResponse, when checking the return value from tkt_CheckTimes,
the order is == 0, == -1, < -1, <= 0. The <= 0 case is extraneous so
remove it. Both < -1 and <= 0 returned RXKADBADTICKET.
same deal as purge. a transaction on the volume outstanding holds it. we don't need to preclude all access
====================
This delta was composed from multiple commits as part of the CVS->Git migration.
The checkin message with each commit was inconsistent.
The following are the additional commit messages.
====================
same deal as purge. a transaction on the volume outstanding holds it. we don't n
eed to preclude all access
====================
same deal as purge. a transaction on the volume outstanding holds it. we don't n
eed to preclude all access
FIXES 60809
Problems fixed with this patch:
/1/ supergroup bug: when updating an entry in prdb, the logic in
pt_mywrite is supposed to unset flagged & found bitmap entries.
This failed on little-endian architecture machines.
/2/ warnings; a few fixes to eliminate some compiler noise.
FIXES 60258
Do not return access denied when applying directory patches if the
user does not have read permission. This is the case we want to
fake the directory entries for. Also, make sure we set the directory
attribute on non-files so that the path can be accessed via the
Explorer Shell.
FIXES 60258
When the ACL on a directory is list only, attempts to read the status
of items in the directory will fail. Therefore, it is pointless to try.
Instead, when we know the user ACL does not have read permission, we
should immediately lie about the status info. That way we don't pound
the file server with requests that will produce an abort which in turn
will trigger force the file server to delay responses to the client.
This change has the added benefit that cached status info is no longer
leaked to callers that do not have appropriate permissions.
With this change the Explorer Shell is much more responsive.