3989
Comment: debugging hints
|
4357
more thoughts on policies
|
Deletions are marked like this. | Additions are marked like this. |
Line 60: | Line 60: |
* update: still makes no sense, as an "Abort``Always" rule can be embedded as a "Abort``If``Matched" rule without any predicates. Thus: ContinueOption:: Abort``If``Matched | Continue * implementation: this will hopefully translate to policies like "if ''<..>'' do ''x'', otherwise if ''<..>'' do ''y'', however if ''<..>'' do z, otherwise do ''default''" |
Status
Latest additions
- automatic pseudo-striping if
- file consists of no more than one object
network has high latency, i.e. an effective RTT of > 10ms
Test cell in Hamburg
Hartmut has built enhanced openafs-osd-bpc on dirac machines
- some of the most recent changes are not yet in the svn trunk
latest code is in /afs/osdtest/openafs (also take note of perftest there)
- as of last week, the dcache interface machine was not yet available
- dcache support requires some special handling of linktables (?)
Plans in Zeuthen
cell desytest is still going to be used for development
ifh.de database servers are going to be outfitted with OSDDB soon
- two initial fileservers are going to be upgraded to OSD support, for tentative use in a production environment
- initial RXOSDs will either share with fileservers or use dedicated hardware at first, with the option of later relocation of data
Policies
Database backend
creation of sensible data structures in osddb.xg
- saving in the same manner as all data in the OSDDB
- typed entries are already supported, so this will require very little work
Code structure
- from fileserver perspective, there are at least two places from where policy evaluation might make sense
in SAFSS_CreateFile, almost all relevant information is known already, however
size estimations can be made no sooner than in common_StoreData64
- the latter has a disadvantage, as e.g. the filename needs to be looked up again
all functions that make direct use of the policy data structures should live in the osddb subtree, specifically in osddbuser.c
functions that add behaviour to the fileserver should live in vol_osd.c
Desired features
- policies control whether files are stored in OSDs at all and if the should be mirrored/striped (and in which way)
- policies can use (any combination of(?))
- estimated file size
- file name (by suffix? prefix/suffix? wildcard? regexp?)
- account name of file owner
- each directory can have exactly one policy connected to it
the volume's osdFlag could be used to store a default policy for all directories
- it remains to be seen how to make it clear in which cases the default is (not) used
- Hartmut argues that allowing rules that use (e.g.) both file size and name will introduce implementation problems
- however, designing a user-friendly system with such limitations raises interaction (i.e. explanation) issues
Possible ways for expressing policy rules
- as a directory can use no more than one policy, it appears sensible to allow creating "meta policies" from existing ones
- possible grammars
- a very simple way would be
- Policy
Rule Rule ...
- Rule
Predicate Predicate ... Verdict
- Verdict
NoOSD | SimpleFile | StripingRules
- StripingRules
NumStripes StripeSize NumCopies
- in this model, the first rule with all predicates being matched by a given file will apply, return its verdict and evaluation ends
- a very simple way would be
- an alternative implementation of the above grammar could be: Don't ever abort evaluation. Each matching rule will override the previous verdict. Thus, later rules have priority.
- a combination of the two ideas would require an enhanced grammar:
- Rule
Predicate Predicate ... Verdict ContinueOption
- ContinueOption
AbortIfMatched | AbortAlways | Continue
this would allow to specify wether the volume global policy should be used as a fallback. That's the only conceivable use of the "AbortAlways" tag.
update: still makes no sense, as an "AbortAlways" rule can be embedded as a "AbortIfMatched" rule without any predicates. Thus:
- ContinueOption
AbortIfMatched | Continue
implementation: this will hopefully translate to policies like "if <..> do x, otherwise if <..> do y, however if <..> do z, otherwise do default"
Hints
bos install is quicker than make install and doesn't strip debugging symbols
remove all -O2 strings from src/config/Makefile.config to disable optimization, as debugging doesn't go along with that really well