|
Size: 12578
Comment: link to RPM build instructions
|
← Revision 50 as of 2009-08-17 16:11:48 ⇥
Size: 60
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 4: | Line 4: |
= Important notes = On ''iokaste'', /root/.zshenv now contains {{{ PATH=/usr/heimdal/bin:/usr/afs/bin:$PATH }}} = Status = Overview of installs ( /!\ this is how it's currently supposed to be / under way): || Machine || service || version || || iokaste || fileserver || manually from svn || || || ubik servers || manually from svn || || || afsd || manually from svn || || phobos || fileserver || manually from svn, for debugging only || || || afsd || manually from svn || || ssu12/13, zyklop19/20 || fileserver || OpenAFS+OSD RPM from svn (not zyklop20) || || || rxosd || OpenAFS+OSD RPM from svn || || || afsd || OpenAFS+OSD RPM from svn 1.4.7 or SL5 RPM || || deimos || ubik servers || OpenAFS+OSD RPM from svn || = Building OpenAFS-OSD = To build openafs, kernel headers are recommended to be installed (so configure can detect the local sysname). On SL5, as root do a {{{ yum install kernel-devel }}} and you're set (for the time being). If there's still no ''/usr/src/linux'': {{{ ln -s /usr/src/kernels/2.6.18-92.1.10.el5-i686 /usr/src/linux }}} ...or whatever directory the kernel source was installed to. In the openafs-''version'' directory: {{{ mkdir `sys` cd `sys` ../configure --prefix=/usr --libdir=/usr/lib --bindir=/usr/bin --sbindir=/usr/sbin --with-krb5-conf --enable-bitmap-later --enable-bos-restricted-mode --enable-fast-restart --enable-bos-new-config --enable-supergroups --enable-transarc-paths --enable-object-storage --enable-osd-policies }}} == Quick RPM creation == Package building is described in TestCellSetup/IncludingFileservers. Currently, this works only with the ''openafs.SLx.spec'' file, revision 70. Install any ''openafs-1.4.x.osd.*.src.rpm'' package to retrieve it. On the build machine: {{{ cd /usr/src/packages/SOURCES svn export https://svnsrv.desy.de/desy/openafs-osd/trunk/openafs openafs-1.4.x.osd.rXXX tar cjf openafs-1.4.x.osd.rXXX-src.tar.bz2 rm -r openafs-1.4.x.osd.rXXX vim ../SPECS/openafs.SLx.spec rpmbuild -ba ../SPECS/openafs.SLx.spec }}} * Change 1.4.x to the AFS version being built and rXXX to the revision number of the HEAD in the subversion repository (obtain using `svn info https://...`). * To build packages with policy support, substitute `trunk` in the URL by `branches/policies` and the version to 1.4.x.osd.pol.rXXX. = Enabeling the client = * From the source directory '''copy''' {{{ src/libafs/MODLOAD-2.6.18-8.1.15.el5-MP/libafs.ko }}} (where ''2.6.18-8.1.15.el5-MP'' is the kernel version) to {{{ /lib/modules/2.6.18-8.1.10.el5/kernel/fs/openafs/libafs.ko }}} * {{{ depmod }}} * reboot * launch ''afsd'' = Enabling OSD = {{{ [iokaste] ~ # vos setfields osdtestvol2 -osd 1 [iokaste] ~ # fs protocol -enable RXOSD }}} = Issues = == Missing token is not handled well == {{{ $ osd volumes io -lun 1 RX xdr error Cannot list volumes Request aborted. $ osd createv io 1 536870924 Createvolume failed: RXOSD_create_part ended with 13 Request aborted.}}} In general, missing privileges are an issue but there is no "friendly output" describing the cause of the problem. Some examples of unprivileged access attempts: || '''fs osd''' || Error 13 reading length of metadata || works with '''r''' privilege granted to system:anyuser || || '''osd read''' || RX xdr error \\ Cannot read the object \\ rx_End``Call returns: 13 \\ Request aborted. || requires token || || '''osd volumes''' || RX xdr error \\ Cannot list volumes \\ Request aborted. || requires token || || '''osd examine''' || RXOSD_examine failed with code 13 \\ Request aborted. || requires token || Database queries '''osd listosds''' and '''osd servers''' seem to work unauthenticated. = Archiving = == Manually creating archival copies == The most simple way is to tell the fileserver to create a file copy in an archival osd: {{{ # fs archive ascii-file ascii-file done }}} An alternative to this is the `fs fidarchive` subcommand. It is also possible to specify the desired archive: {{{ # osd l id name(loc) ---total space--- flag prior. own. server lun size range 1 local_disk wr rd (0kb-1mb) 13 ssu12 1781 gb 0.0 % up 55 64 ssu12.if 2 (1mb-64gb) 14 ssu13 1781 gb 0.0 % up 55 64 ssu13.if 2 (1mb-64gb) 15 z19mini 91 gb 2.3 % up hsm 64 64 zyklop19 0 (1mb-1gb) 16 z19maxi 458 gb 0.0 % up 55 64 zyklop19 2 (1mb-64gb) 17 arch1 458 gb 0.0 % up arch 55 64 zyklop20 0 (0kb-2tb) 18 arch2 458 gb 0.0 % up arch 55 64 zyklop20 1 (0kb-2tb) 19 z20online 391 gb 0.0 % up 55 64 zyklop20 2 (1mb-64gb) # fs archive ascii-file -osd 18 ascii-file done # fs osd archive-file ascii-file has 284 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=3, 2009-01-28 09:55:34, 1 segm, flags=0x0 segment: lng=44537893, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.0, osd=18, stripe=0 metadata: md5=a64535612df9ff989b4bd6d6a133fb5d as from 2009-01-28 09:56:36 }}} The archival copy is on osd 18 (zyklop20, vicepb = lun 1). /!\ There is no warning about existent copies. Repeat the command: {{{ [iokaste] /afs/desytest/field_test # fs arch ascii-file -osd 18 ascii-file done [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 284 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=3, 2009-01-28 09:55:34, 1 segm, flags=0x0 segment: lng=44537893, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.0, osd=18, stripe=0 metadata: md5=a64535612df9ff989b4bd6d6a133fb5d as from 2009-01-28 09:56:36 }}} The call is silently ignored. Change the data: {{{ [iokaste] /afs/desytest/field_test # echo More data >> ascii-file [iokaste] /afs/desytest/field_test # fs arch ascii-file -osd 18 ascii-file done [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 284 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=5, 2009-01-28 09:59:27, 1 segm, flags=0x0 segment: lng=44537903, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.1, osd=18, stripe=0 metadata: md5=1891d17de5eb67c0bd30d54952ea0325 as from 2009-01-28 09:59:43 }}} The copy is updated. Another update: {{{ [iokaste] /afs/desytest/field_test # echo More data yet >>| ascii-file [iokaste] /afs/desytest/field_test # fs arch ascii-file ascii-file done [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 284 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=7, 2009-01-28 10:00:46, 1 segm, flags=0x0 segment: lng=44537917, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.0, osd=18, stripe=0 metadata: md5=eec1410b6ae326e6c9173802122372c0 as from 2009-01-28 10:00:51 }}} Even without specifying the target osd, the system correctly assumes that the archive should be refreshed. Without updates: {{{ [iokaste] /afs/desytest/field_test # fs arch ascii-file ascii-file done [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 436 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=7, 2009-01-28 10:00:46, 1 segm, flags=0x0 segment: lng=44537917, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.0, osd=18, stripe=0 metadata: md5=eec1410b6ae326e6c9173802122372c0 as from 2009-01-28 10:00:51 Archive, dv=7, 2009-01-28 10:00:51, 1 segm, flags=0x0 segment: lng=44537917, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.0, osd=17, stripe=0 metadata: md5=eec1410b6ae326e6c9173802122372c0 as from 2009-01-28 10:02:07 }}} Another archived copy was created, same data (as per md5data). A last update: {{{ [iokaste] /afs/desytest/field_test # echo And even more data >> ascii-file [iokaste] /afs/desytest/field_test # fs arch ascii-file ascii-file done [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 284 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=9, 2009-01-28 10:02:54, 1 segm, flags=0x0 segment: lng=44537936, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.1, osd=17, stripe=0 metadata: md5=150103c2d7bcfc367b707df35801cdfa as from 2009-01-28 10:02:59 }}} The copy in osd 17 is updated, the "old" archive in osd 18 is discarded. == Wiping files == If the data is archived, it can be wiped from its actual OSD: {{{ [iokaste] /afs/desytest/field_test # fs wipe ascii-file done [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 160 bytes of osd metadata, v=3 Archive, dv=9, 2009-01-28 10:02:54, 1 segm, flags=0x2 segment: lng=44537936, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.1, osd=17, stripe=0 metadata: md5=150103c2d7bcfc367b707df35801cdfa as from 2009-01-28 10:02:59 }}} On an access, it is silently restored to an "normal" osd: {{{ [iokaste] /afs/desytest/field_test # head ascii-file [1177711200] LOG ROTATION: DAILY [1177711200] LOG VERSION: 2.0 [1177711200] CURRENT HOST STATE: a;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.13 ms [1177711200] CURRENT HOST STATE: amafs;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.14 ms [1177711200] CURRENT HOST STATE: aphrodite;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.54 ms [1177711200] CURRENT HOST STATE: aquila;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.25 ms [1177711200] CURRENT HOST STATE: ares;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.59 ms [1177711200] CURRENT HOST STATE: argus;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.16 ms [1177711200] CURRENT HOST STATE: artax;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.08 ms [1177711200] CURRENT HOST STATE: arwen;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.18 ms [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 284 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=9, 2009-01-28 10:02:54, 1 fetches, last: 2009-01-28, 1 segm, flags=0x2 segment: lng=44537936, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.1, osd=17, stripe=0 metadata: md5=150103c2d7bcfc367b707df35801cdfa as from 2009-01-28 10:02:59 }}} The osd is selected the same way as always - depending on priorities, usage percentage, owner etc. == Automatic archiving == The `archiver` script is included in the OpenAFS+OSD sources. It runs infinitely and searches unarchived files in short intervals. It runs `fs fidarchive` on them. The `wiper` script will traverse OSDs that have the "hsm" flag set using {{{ # osd setosd <id> -wipeable }}} On those, files that have not been accessed for a while are wiped, so that only their latest archival copy remains in storage. The scripts have yet to be ported to our specific requirements (as far as they differ from Garching's). |
Contents
Sub pages: