General concepts
Working with an AFS+OSD cell
The following techniques are only useful when using OSD as a frontend for HSM:
- {{{vos listobj
}}} is useful for finding all objects stored on a given OSD
- {{{fs prefetch
}}} allows the user to schedule tape restore operations
it is useful to have bosserver run a script to wipe objects off the OSDs according to OSD fill rate etc.
These are general techniques:
- {{{osd addserver
}}} makes an AFS fileserver known to the OSDDB
this is required in order to make use of ownerships and locations
How to migrate data from an OSD
- set a low write priority to stop fileservers from storing data on the OSD in question {{{osd setosd -wrprior 0
}}}
- use {{{vos listobj
}}} to identify the files (by fid) that have data on the OSD
- use {{{fs replaceosd
}}} to move each file's data to another OSD
Priorities and choice of storing OSD
- OSDs are used in a Round Robin fashion
- priorities add weight to each OSD
- static priorities can be set by the administrator {{{osd setosd -wrprior ... -rdprior ...
}}}
- priorities are dynamic
- ownerships apply a modifier of +/- 10 to the static priority
- locations apply a modifier of +/- 20
Data held in volumes, DBs etc.
- all metadata belonging to files in a volume are stored in a designated volume special file
metadata references OSDs by id
OSD id is an index into the OSDDB
- these IDs are permanent and can never be reused after deletion of an OSDDB entry
- view the OSDDB using {{{osd l
}}}
a file's metadata is found using the metadataindex stored in the file's vnode
view the metadata using {{{fs osd <file>
}}}
Policies
- policies are going to decide 4 basic things:
- whether object storage is to be used for a given file
- whether the objects are to be mirrored and how often
- how many stripes the file will comprise
- the size of the stripes
- two pieces of information will be used to make decisions:
- file size
- file name (i.e. prefix and suffix)
Open questions
- will inheritance be supported for policy definitions?
how many levels of inheritance are permitted (1 or n)?
- in what way can policies be represented/stored?
Technical aspects
Performance
- client connections to OSDs are cheap because they are unauthorized
- clients do connect to each single OSD they have business with
osd psread acts as though reading stripes from multiple OSDs
- i.e. it opens several connections, in this case all targets are the same however
- each stripe has an altered offset (e.g., normally first n stripes start at offset 0 on each OSD, here it is 0+(i*stripesize) etc.)
- this is not impossible for access to AFS fileservers, but making connections is more costly
Notes on the code (changes)
- original linktable was insufficient, further fields were required
- link table v2 uses 30 bits instead of 15
- hashing is used for management of DB contents
choice of OSDs resides in osddbuser.c
Debugging techniques
- add trace* calls to the code
- CM_TRACE_WASHERE especially handy
use