Status

Latest additions

• automatic pseudo-striping if
  • file consists of no more than one object

  • network has high latency, i.e. an effective RTT of > 10ms

Test cell in Hamburg

• Hartmut has built enhanced openafs-osd-bpc on dirac machines

  • some of the most recent changes are not yet in the svn trunk

  • latest code is in /afs/osdtest/openafs (also take note of perftest there)

• as of last week, the dcache interface machine was not yet available
• dcache support requires some special handling of linktables (?)

Plans in Zeuthen

• cell desytest is still going to be used for development

• ifh.de database servers are going to be outfitted with OSDDB soon

• two initial fileservers are going to be upgraded to OSD support, for tentative use in a production environment
• initial RXOSDs will either share with fileservers or use dedicated hardware at first, with the option of later relocation of data

Policies

Database backend

• creation of sensible data structures in osddb.xg

• saving in the same manner as all data in the OSDDB
  • typed entries are already supported, so this will require very little work

Code structure

• from fileserver perspective, there are at least two places from where policy evaluation might make sense

  • in SAFSS_CreateFile, almost all relevant information is known already, however

  • size estimations can be made no sooner than in common_StoreData64

  • the latter has a disadvantage, as e.g. the filename needs to be looked up again

• all functions that make direct use of the policy data structures should live in the osddb subtree, specifically in osddbuser.c

• functions that add behaviour to the fileserver should live in vol_osd.c

Desired features

• policies control whether files are stored in OSDs at all and if the should be mirrored/striped (and in which way)
• policies can use (any combination of(?))
  • estimated file size
  • file name (by suffix? prefix/suffix? wildcard? regexp?)
  • account name of file owner
• each directory can have exactly one policy connected to it

• the volume's osdFlag could be used to store a default policy for all directories

  • it remains to be seen how to make it clear in which cases the default is (not) used
• Hartmut argues that allowing rules that use (e.g.) both file size and name will introduce implementation problems
  • however, designing a user-friendly system with such limitations raises interaction (i.e. explanation) issues

Possible ways for expressing policy rules

• as a directory can use no more than one policy, it appears sensible to allow creating "meta policies" from existing ones
• possible grammars
  • a very simple way would be

    Policy

    Rule Rule ...

    Rule

    Predicate Predicate ... Verdict

    Verdict

    NoOSD | SimpleFile | StripingRules

    StripingRules

    NumStripes StripeSize NumCopies

    in this model, the first rule with all predicates being matched by a given file will apply, return its verdict and evaluation ends

  • an alternative implementation of the above grammar could be: Don't ever abort evaluation. Each matching rule will override the previous verdict. Thus, later rules have priority.
  • a combination of the two ideas would require an enhanced grammar:

    Rule

    Predicate Predicate ... Verdict ContinueOption

    ContinueOption

    AbortIfMatched | AbortAlways | Continue

    this would allow to specify wether the volume global policy should be used as a fallback. That's the only conceivable use of the "AbortAlways" tag.

    • update: still makes no sense, as an "AbortAlways" rule can be embedded as a "AbortIfMatched" rule without any predicates. Thus:

    ContinueOption

    AbortIfMatched | Continue

    • implementation: this will hopefully translate to policies like "if <..> do x, otherwise if <..> do y, however if <..> do z, otherwise do default"

Hints

• bos install is quicker than make install and doesn't strip debugging symbols

• remove all -O2 strings from src/config/Makefile.config to disable optimization, as debugging doesn't go along with that really well