Differences between revisions 3 and 4
Revision 3 as of 2008-09-04 11:09:15
Size: 3751
Editor: FelixFrank
Comment: table of contents, some additions
Revision 4 as of 2008-09-04 14:29:17
Size: 3989
Editor: FelixFrank
Comment: debugging hints
Deletions are marked like this. Additions are marked like this.
Line 60: Line 60:

= Hints =
 * `bos install` is quicker than `make install` and doesn't strip debugging symbols
 * remove all -O2 strings from ''src/config/Makefile.config'' to disable optimization, as debugging doesn't go along with that really well

TableOfContents

Status

Latest additions

  • automatic pseudo-striping if
    • file consists of no more than one object
    • network has high latency, i.e. an effective RTT of > 10ms

Test cell in Hamburg

  • Hartmut has built enhanced openafs-osd-bpc on dirac machines

    • some of the most recent changes are not yet in the svn trunk
    • latest code is in /afs/osdtest/openafs (also take note of perftest there)

  • as of last week, the dcache interface machine was not yet available
  • dcache support requires some special handling of linktables (?)

Plans in Zeuthen

  • cell desytest is still going to be used for development

  • ifh.de database servers are going to be outfitted with OSDDB soon

  • two initial fileservers are going to be upgraded to OSD support, for tentative use in a production environment
  • initial RXOSDs will either share with fileservers or use dedicated hardware at first, with the option of later relocation of data

Policies

Database backend

  • creation of sensible data structures in osddb.xg

  • saving in the same manner as all data in the OSDDB
    • typed entries are already supported, so this will require very little work

Code structure

  • from fileserver perspective, there are at least two places from where policy evaluation might make sense
    • in SAFSS_CreateFile, almost all relevant information is known already, however

    • size estimations can be made no sooner than in common_StoreData64

    • the latter has a disadvantage, as e.g. the filename needs to be looked up again
  • all functions that make direct use of the policy data structures should live in the osddb subtree, specifically in osddbuser.c

  • functions that add behaviour to the fileserver should live in vol_osd.c

Desired features

  • policies control whether files are stored in OSDs at all and if the should be mirrored/striped (and in which way)
  • policies can use (any combination of(?))
    • estimated file size
    • file name (by suffix? prefix/suffix? wildcard? regexp?)
    • account name of file owner
  • each directory can have exactly one policy connected to it
  • the volume's osdFlag could be used to store a default policy for all directories

    • it remains to be seen how to make it clear in which cases the default is (not) used
  • Hartmut argues that allowing rules that use (e.g.) both file size and name will introduce implementation problems
    • however, designing a user-friendly system with such limitations raises interaction (i.e. explanation) issues

Possible ways for expressing policy rules

  • as a directory can use no more than one policy, it appears sensible to allow creating "meta policies" from existing ones
  • possible grammars
    • a very simple way would be
      Policy

      Rule Rule ...

      Rule

      Predicate Predicate ... Verdict

      Verdict

      NoOSD | SimpleFile | StripingRules

      StripingRules

      NumStripes StripeSize NumCopies

    • in this model, the first rule with all predicates being matched by a given file will apply, return its verdict and evaluation ends
    • an alternative implementation of the above grammar could be: Don't ever abort evaluation. Each matching rule will override the previous verdict. Thus, later rules have priority.
    • a combination of the two ideas would require an enhanced grammar:
      Rule

      Predicate Predicate ... Verdict ContinueOption

      ContinueOption

      AbortIfMatched | AbortAlways | Continue

    • this would allow to specify wether the volume global policy should be used as a fallback. That's the only conceivable use of the "AbortAlways" tag.

Hints

  • bos install is quicker than make install and doesn't strip debugging symbols

  • remove all -O2 strings from src/config/Makefile.config to disable optimization, as debugging doesn't go along with that really well

AfsOsd/Protocol-2008-09-03 (last edited 2008-10-30 11:40:14 by localhost)