5067 Commits

Author SHA1 Message Date
Jeffrey Altman
cd406ecc5c DEVEL15-windows-config-20080308
LICENSE MIT

raise the daemon count to 4 now that we have a means of avoiding
contention

raise the tracebufsize to 10000 entries


(cherry picked from commit f94e0d5688be0778d346711b6698e2ef567e1969)
2008-03-08 23:32:44 +00:00
Jeffrey Altman
efe393ab41 DEVEL15-windows-syscfg-getifinfo-20080308
LICENSE MIT

syscfg_GetIFInfo() obtains the current list of IP addresses.
Its a really expensive operation.  Call the function once
and cache the data until the next IP address change instead
of calling it everytime we receive a WhoAreYou or construct a
new server object.  Adds a new global rw lock, cm_syscfgLock


(cherry picked from commit 801b670734e254665576dcb71756ea64a123abc9)
2008-03-08 23:25:08 +00:00
Jeffrey Altman
94a08438eb DEVEL15-windows-scache-20080308
LICENSE MIT

do not discard data buffers we wish to keep


(cherry picked from commit 27008912ec2a0526e71f174b9e4e6d8dd1fc8f19)
2008-03-08 23:18:54 +00:00
Jeffrey Altman
cbb4af81fd DEVEL15-windows-performance-deadlock-20080308
LICENSE MIT

correct a deadlock when calling HaveCallback


(cherry picked from commit c32f5466f0cf7150d8bbe286fa51834108dce63f)
2008-03-08 23:16:34 +00:00
Jeffrey Altman
741e4c002d DEVEL15-windows-cm-callback-remove-call-mutex-hold-20080308
LICENSE MIT

Remove all of the code in #ifdef HOLD_CALL_MUTEX as we no longer
use it.


(cherry picked from commit 694202f29e5eafbcf0a07908ebdc2c67a2eecc3b)
2008-03-08 15:45:02 +00:00
Jeffrey Altman
6713c06942 DEVEL15-windows-eventlog-20080308
LICENSE MIT

we need set privilege not query privilege


(cherry picked from commit 526babc882364573be745b717c32fa1da29acaa2)
2008-03-08 14:17:32 +00:00
Jeffrey Altman
65d6a9bd97 DEVEL15-windows-cm-performance-tuning-200803007
LICENSE MIT

This is a first cut at a cache manager statistics monitor
that can be used to determine the necessary cache parameters
to support the working set.

Off by default the performance package can be activated
by setting "daemonPerformanceTuningInterval" in the service
Parameters key.  As with the other daemon interval values
the unit is in seconds.

At service start and each succeeding interval the cache
manager will write statistics to %TEMP%\afsd_performance.txt
showing the relative usage of cm_scache_t, cm_volume_t and
cm_buf_t objects.  The FID statistics keep track of all FIDs
seen by the cache manager during the service session whether
or not they are backed by any live object in the cache.

These statistics are not stored in the cache file.


(cherry picked from commit fdcf6bba5546f2a09e972323be9469a49ce04c6f)
2008-03-07 22:24:04 +00:00
Jeffrey Altman
ffe8b996b2 DEVEL15-windows-cm-conn-20080307
LICENSE MIT

Convert cm_conn_t reference counts to use Interlocked operations.
This permits several cm_serverLock holds to be converted to read
locks.

Add string translation for VL errors to cm_Analyze().

Permit RXKAD errors other than RXKADEXPIRED to be treated as
non-fatal.  Instead immediately retry an alternate server if
there is one.  This will permit the client to continue to
access replicated data from an alternate site if one of the
file servers is misconfigured.


(cherry picked from commit 1f4ca0f488d765d5d09de9edf8a8fc4fbe6b09b1)
2008-03-07 22:16:06 +00:00
Jeffrey Altman
b0b3e061f1 DEVEL15-windows-cm-server-interlocked-20080307
LICENSE MIT

Convert cm_server_t reference counts to use Interlocked
operations.  This permits almost all of the cm_serverLock
holds to be converted to read locks.

Add missing cm_PutServerNoLock() calls in the multi_Rx
version of cm_CheckServers().  (Thanks to Asanka)


(cherry picked from commit 0479c650c17f35a4cf0de523cfc036b8d21629ce)
2008-03-07 22:08:54 +00:00
Jeffrey Altman
de64ae4e8d DEVEL15-windows-vl-noent-response-20080308
LICENSE MIT

Prior to looking up a volume in the vldb the cm_volume_t
object is constructed.  If the response is VL_NOENT the
cm_volume_t object should be removed and place at the
head of the recycling list so that we do not recycle
something we might actually care about.

Convert more cm_volumeLock holds to read locks now
that we are using Interlocked operations to manipulate
the reference counts.


(cherry picked from commit 757ee6da75123653dd094a6e1fdd374edf0c55f1)
2008-03-07 22:04:09 +00:00
Jeffrey Altman
217e7d6889 DEVEL15-windows-scache-verification-20080307
LICENSE MIT

Add a hash value verification check for stat cache entries


(cherry picked from commit 45200a26985678ccad051385f50f94321eb98256)
2008-03-07 17:46:00 +00:00
Jeffrey Altman
8d4347a687 DEVEL15-windows-rxstats-default-20080307
LICENSE MIT

set the defaults for RxEnablePeerStats and RxEnableProcessStats back
to 1 (on) so admins are not surprised.


(cherry picked from commit 73c2fa385a6f61483189dc68c4a10a49d65c8bf5)
2008-03-07 17:42:37 +00:00
Simon Wilkinson
ccd454443f DEVEL15-fakestat-access-for-nautilus-20080307
LICENSE IPL10

in order that nautilus' .directory checks can work without stat()ing every damn
thing, do what we do for the mac, basically.


(cherry picked from commit f637b6292d3723cd2cf672ecaba6f179813ab64b)
2008-03-07 17:31:28 +00:00
Jeffrey Altman
6ad80c9afa DEVEL15-windows-smb-writedata-crash-20080307
LICENSE MIT
FIXES 88731

If an application opens a file for read only and then attempts to
write to it, we would attempt to release an scp mutex without having
obtained either the scp or the mutex within smb_WriteData

There was no check in smb_ReadData to ensure that the file was opened
for reading.  Add one.


(cherry picked from commit 5589875561e0062b2f06fc4ce0a8fad37a0d8eb5)
2008-03-07 15:37:26 +00:00
Jeffrey Altman
efcef7c17f DEVEL15-windows-interlocked-volume-refcount-20080306
LICENSE MIT

Switch cm_volume_t objects to InterlockedIncrement/InterlockedDecrement
for reference counting.

Remove protections against null pointers being passed into cm_GetVolume()
Instead, do not call cm_GetVolume() if the pointer is NULL.

Fix a buffer data version comparison that should be bad version number
instead of <= 0.


(cherry picked from commit fb154e60e3cb6cf9e934f9a75c5ca67473ac36a2)
2008-03-07 01:14:13 +00:00
Derrick Brashear
42c527deab DEVEL15-dafs-restarting-error-change-20080306
LICENSE IPL10

on VSALVAGE the client will correctly resort its list and fail over to
other replicas, but only for RO where there is more than one replica,
obviously; since in dafs we expect to only delay on salvage this is the most
useful course of action


(cherry picked from commit 2137631fb44d934d51979850ebfcb6a50076b095)
2008-03-06 16:55:46 +00:00
Derrick Brashear
87a3ff8b72 DEVEL15-pioctl-settoken-malloc-fail-check-20080206
LICENSE IPL10

avoid malloc failure panic (oops)


(cherry picked from commit f58e08baae0a5ee9be516e9e220a12b78e5bf414)
2008-03-06 16:44:10 +00:00
Jeffrey Altman
82904b8ace DEVEL15-windows-afsd-misc-20080306
LICENSE MIT

(1) an attempt to make better use of bandwidth from the BkgDaemon threads
    by preventing the thread from blocking on a vnode that is already
    storing data in another thread

(2) prevents CM_SCACHEFLAG_ASYNCSTORE from being reset on a write failure.

(3) fixes cm_EvaluateSysName to avoid accessing uninitialized memory

(4) prevents a lock leak if the symlink's mountpointstring is too long.
    (This could never actually happen but better to correct the code.)


(cherry picked from commit 6a631075c478794e998ad67d90e33d304521fbd3)
2008-03-06 14:34:29 +00:00
Derrick Brashear
8afb0e0b26 DEVEL15-large-partition-support-20080305
LICENSE IPL10
FIXES 88811

support partitions over 2tb


(cherry picked from commit b651ece9bdb2ea9dc19907a5686e9d084f3c03d6)
2008-03-05 21:53:25 +00:00
Jeffrey Altman
3ee503e234 DEVEL15-windows-scache-mx-to-rw-20080302
LICENSE MIT

Replace the cm_scache_t mutex with a rwlock permitting a small amount
of additional parallelization in places where it is safe to use read
locks instead of write locks.  All functions that eventually call
cm_SyncOp must use write locks.


(cherry picked from commit 69f9b46e69bb7c8c9356b062a8f4908fe38ad14b)
2008-03-03 04:25:39 +00:00
Jeffrey Altman
4d7b6b13b5 DEVEL15-windows-cm_buf-20080302
LICENSE MIT

remove unused constants, data structures and fields


(cherry picked from commit ba94313f4030184890194c84f8d5c2f3f69f52d1)
2008-03-02 21:56:18 +00:00
Jeffrey Altman
274878f55f DEVEL15-windows-asyncstore-again-20080301
LICENSE MIT

must write the entire asyncstoresize otherwise dirty buffers are left
for writing by buf_IncrSync thread.


(cherry picked from commit af2318cfde026f5dc6b479089689dba63f7b8abc)
2008-03-01 19:00:19 +00:00
Jeffrey Altman
41e6d32086 DEVEL15-windows-buf-more-clean-vnode-speedups-20080301
LICENSE MIT

do not call buf_CleanAsync if you know the buffer isn't dirty


(cherry picked from commit b47407ae72491a129fc9ed6990585e426fc2a223)
2008-03-01 18:57:24 +00:00
Jeffrey Altman
47801d13e0 DEVEL15-windows-freelance-20080301
LICENSE MIT

when support for multiple valid buffer data versions was added
forget to remove the force dv change from the freelance code.
this broken automatic additions of new mount points.


(cherry picked from commit 7083ceec5c21e500f23110152dcc994c7ccfab92)
2008-03-01 17:44:28 +00:00
Jeffrey Altman
da2bc8dc95 DEVEL15-windows-buf-mx-20080301
LICENSE MIT

further reductions in buffer mutex acquisition churn


(cherry picked from commit fb3f10f3297c08807c0a3f0b345d88cd19742726)
2008-03-01 16:30:02 +00:00
Jeffrey Altman
611d550a47 DEVEL15-windows-eventlog2-20080301
LICENSE MIT

set the registry values on each start.  otherwise, we will never update
the incorrect entries left over from previous installs.


(cherry picked from commit 3b5a1acef030ad9208111511813a0b69c6a0de99)
2008-03-01 16:00:29 +00:00
Jeffrey Altman
283b4a2954 DEVEL15-windows-eventlog-20080301
LICENSE MIT

somewhere along the way Microsoft stopped opening Event Message files
explicitly.  Instead they search the PATH environment variable for
the specified file.  I think this is broken but simply registering
"afsd_service.exe" instead of the fully qualified path works.


(cherry picked from commit e4df679d1bcc6fb7e3c3eb4169e489c172910628)
2008-03-01 15:38:34 +00:00
Jeffrey Altman
5cd9032376 DEVEL15-windows-smb-asyncstore-20080229
LICENSE MIT

fix it


(cherry picked from commit b8cacec96969b044c1cfdd1919ca9e2e36999243)
2008-03-01 04:30:38 +00:00
Jeffrey Altman
df13fd4cb0 DEVEL15-windows-cm-buf-misc-20080229
LICENSE MIT

(1) Add an undocumented store behind mode for use in testing.  Set
    EnableSMBAsyncStore to 2.  When set all smb_WriteData calls are
    background writes, all calls to cm_FSync are skipped and file close
    operations do not block for dirty buffers to be written.  This
    permits all writes to be performed in the buf_IncrSyncer thread.

(2) Do not use I64 in osi_Log() format strings as all parameters are
    converted to size_t which is 32-bit on 32-bit Windows.

(3) Reduce the number of times the cm_buf_t mutex is obtained, dropped,
    obtained, dropped in buf_IncrSyncer

(4) In buf_CleanAsyncLocked, request that a full chunk be written instead
    of just the current buffer.  cm_SetupStoreBIOD will stop at the
    first clean buffer.  This reduces the overall number of RPCs that
    must be performed.

(5) Define CM_BUF_VERSION_BAD and use it instead of -1.


(cherry picked from commit 114de458ca7717cf0bcda90df61f24495fe8f922)
2008-02-29 22:59:52 +00:00
Jeffrey Altman
e677ed9ff1 DEVEL15-windows-dnlc-20080229
LICENSE MIT

even if dnlc is not going to be used, the data structures must
still be initialized as they are in the cache file.


(cherry picked from commit fccf05a96d038a1a911a95d309ecb08090d2c6a1)
2008-02-29 15:58:53 +00:00
Kevin McBride
5c1af6ac6f DEVEL15-checkauth-unlock-before-return-20080228
LICENSE IPL10

only matters once you have pthreads, but...


(cherry picked from commit 8ffc274282fa40c6e2d35ba861b9dc932afa032d)
2008-02-29 15:13:06 +00:00
Simon Wilkinson
20136421a0 DEVEL15-configure-enable-warnings-20080228
LICENSE IPL10

add a switch to turn on warnings when using gcc


(cherry picked from commit 737c03a75c62b3cd0ec92b33aea84d33dc39c2e7)
2008-02-29 15:13:04 +00:00
Jeffrey Altman
47064e9243 DEVEL15-windows-libadmin-closesocket-20080228
LICENSE MIT

On Windows close() does not apply to sockets, use closesocket()
instead.


(cherry picked from commit 502aca9e8ccff36d0c10203c66200e526d2273b3)
2008-02-29 01:27:57 +00:00
Jeffrey Altman
7e6a650ce5 DEVEL15-windows-buf-deadlock-20080228
LICENSE MIT

avoid deadlock in buf_FlushCleanPages().
cannot obtain buffer mutex after a successful
Stabilize call because the scp will be be locked
and obtaining buffer mutex after scache mutex
is a lock order violation.


(cherry picked from commit 0546641c0da59289d03a9417984fe5f7e5636a75)
2008-02-28 18:22:58 +00:00
Jeffrey Altman
a709dbf9ef DEVEL15-windows-smb-lock-timeouts-20080228
LICENSE MIT

Attempts to open files which are already write-locked by another
client took forever to return a lock not granted error.  This
was because cm_Analyze() would retry the lock request for up to
the RDRtimeout in response to the EAGAIN error.  The problem
was that cm_IntSetLock() was not setting the CM_REQ_NORETRY flag.

While examining this issue, discovered two other things:

(1) the infinite wait logic on lock request processing was broken
(2) the cancel outstanding lock request logic wasn't implemented
(3) cm_Analyze() would put the thread to sleep even when retries
    were not permitted.

Also removed a number of compile time warnings.


(cherry picked from commit e9f7894e3576ef9e01234966f7869440f1823718)
2008-02-28 17:17:15 +00:00
Derrick Brashear
715cbd0765 DEVEL15-arm-lwp-adm-20080227
LICENSE IPL10

use ucontext-less lwp on arm linux


(cherry picked from commit 023d3f3bae273d7af16d52fd0c468fc05653abe5)
2008-02-27 22:31:11 +00:00
Jeffrey Altman
b6654a597f DEVEL15-windows-osi-convertRToW-and-logging-optimizations-20080227
LICENSE MIT

Add lock_convertRToW which permits a read-lock to be upgraded to a
write lock.  If the caller is the only reader it permits a fast
transition otherwise it adds the caller to the waiters queue.

In the osi_Log macros, check to see if the log is enabled before making
the function call.  This avoids significant function call overhead.

In the cache manager, make use of the above.


(cherry picked from commit 72687c02b6ee5ff3029937b0278733b252f14550)
2008-02-27 17:06:43 +00:00
Jeffrey Altman
92f03f92ce DEVEL15-windows-dnlc-20080226
LICENSE MIT

Remove race conditions in the statistics code by switching to
Interlocked functions.

Speed up cm_dnlcEnter() by searching first with a read lock and
then switching to a write lock if not found.

Add HKLM\SOFTWARE\OpenAFS\Client registry values "UseDNLC" and "Debug"DNLC".
I suspect with the B+tree code that the DNLC is unnecessary overhead but
leave it on by default for now.


(cherry picked from commit 107899cefaeebe693f8025e9c0d88784facd2236)
2008-02-27 04:55:16 +00:00
Derrick Brashear
bcbbb05645 DEVEL15-openafs-1-5-3-20080225
LICENSE IPL10

make 1.5.33
openafs-devel-1_5_33
2008-02-26 04:15:52 +00:00
Jeffrey Altman
4b5dff37e0 DEVEL15-viced-cleanup-old-addresses-as-they-become-invalid-20080225
LICENSE IPL10

otherwise we can end up with stale addresses when a client uses then leaves an
address never to return


(cherry picked from commit 428cac5d6dfc287452af51c08eba0f0fca276864)
2008-02-26 04:14:17 +00:00
Jeffrey Altman
c403d93b55 DEVEL15-windows-notes-20080225
LICENSE MIT

1.5.33 (more)
2008-02-26 01:08:19 +00:00
Jeffrey Altman
a703190112 DEVEL15-windows-version-20080225
LICENSE MIT

1.5.33 = 1.5.3300.0
2008-02-26 01:05:39 +00:00
Jeffrey Altman
0007e327a8 DEVEL15-windows-force-anon-vldb-20080225
LICENSE MIT

For organizations with krb5 aware file servers but transarc or old openafs
vldb servers, add force anonymous vldb lookup option

  HKLM\SYSTEM\CurrentControlSet\Services\TransarcAFSDaemon\Parameters
	DWORD	"ForceAnonVLDB"  (default is 0)


(cherry picked from commit dd72e06bf11fccd70cba32c4b6411a5e6ff3914c)
2008-02-25 22:21:24 +00:00
Jeffrey Hutzelman
1b4b3256fe DEVEL15-threadquota-enforcement-fixes-20080225
LICENSE IPL10
FIXES 87416

correctly set and unset HWHO_INPROGRESS as we pass through host probing package.

also, do not block on lock checking for HOSTDELETED. instead, return a null host
and VBUSY; restructure code to allow returning a null host with prejudice.


(cherry picked from commit f4f7ac048c52f859000a0d42640f0a6d43cc942a)
2008-02-25 20:38:34 +00:00
Jeffrey Altman
a8dcb5884c DEVEL15-windows-volume-logsavestring-20080225
LICENSE MIT

remember to save the volume name when logging


(cherry picked from commit d8c932ed73039777f5e21e70d6850325061f8241)
2008-02-25 19:40:13 +00:00
Jeffrey Altman
b7d2cffd94 DEVEL15-windows-smb-asyncstoresize-20080225
LICENSE MIT

Further testing on the 115KB AT&T Edge connection shows that 128KB
provides reasonably robust performance with six simultaneous copies of
multi-MB files to AFS.


(cherry picked from commit 704fddccd0f714c848b37ccee427f8eeded53bd6)
2008-02-25 15:34:31 +00:00
Jeffrey Altman
b48ead62cb DEVEL15-windows-smb-async-store-20080224
LICENSE MIT

Add two new configuration knobs to control the behavior of smb_WriteData.

HKLM\SOFTWARE\OpenAFS\Client
  DWORD EnableSMBAsyncStore  (default: 1)
  DWORD SMBAsyncStoreSize    (default: 32K)

Instead of tying the async store size to either the chunksize (too large)
or the buffer block size (too small) provide an intermediate value that
can be independently controlled.

In the future it would be desireable for the async store size to be
dynamically determined based upon measurable characteristics of the
network.  In the meantime, 32KB is an acceptable performance compromise
that should work well on 1Gbit networks and low-speed cellular networks.


(cherry picked from commit f95915a55ae579bab4052f24f6fedca21d114567)
2008-02-25 05:36:46 +00:00
Jeffrey Altman
945498236d DEVEL15-windows-release-notes-20080224
LICENSE MIT

1.5.33
2008-02-25 05:24:46 +00:00
Derrick Brashear
e1e3660336 DEVEL15-nulluuid-check-only-valid-input-20080224
LICENSE IPL10

as reported by jhutz@cmu.edu


(cherry picked from commit 27dc7b13aa6356ba76916e47c6ddb48aa4b99846)
2008-02-25 05:00:16 +00:00
Jeffrey Altman
a25822b409 DEVEL15-windows-notes-20080224
LICENSE MIT

1.5.33
2008-02-25 04:18:42 +00:00