Revision 38 as of 2009-08-20 10:36:34

Clear message

Contents

  1. Installation
    1. Caveats
    2. Overview
    3. System Configuration in VAMOS
    4. Creating System Profiles
    5. Activating Private Key Distribution
    6. Booting the system into installation
    7. System/Boot Method Matrix
  2. Package Handling & Automatic Updates
    1. SL Standard & Errata Packages
    2. Addon Packages (Zeuthen)
      1. Selectable Addon Packages (Zeuthen)
    3. Additional Modules for Kernel Updates
      1. ALSA
      2. ESD CAN Module (for PITZ Radiation Monitor)
      3. Nvidia
        1. SL3
        2. SL5
      4. XFS
        1. SL4
        2. SL5
      5. ARECA RAID (SL4)
  3. Adding a new SL release
  4. Booting a rescue system
  5. Building Kernel Packages
    1. 32-bit SL3
      1. Building the 1.2.x kernel-module-openafs (and openafs) packages
      2. Building the NEW 1.4.x kernel-module-openafs packages
  6. References

Installation

Caveats

The following classes of systems need some extra attention for installation:

Overview

SL3/4/5 hosts are installed using kickstart

The repository is mirrored from the ftp server at FNAL and is located on the installation server, a.ifh.de, in /nfs1/a/SL. The host profiles for the kickstart install are kept in /project/linux/SL/profiles, and some files needed during the postinstallation, before the AFS client is available, in /nfs1/a/SL6/post and /project/linux/SL/firstboot (both accessible through the http server running on a). More files, and most utility scripts, are located in /project/linux/SL.

Installation takes the following steps:

System Configuration in VAMOS

Choose a default derived from sl3-def. Defaults starting with "sl3-" are 32bit, those starting with "sl3a-" are 64bit. These will mainly differ in the settings for OS_ARCH and AFS_SYSNAME (see the sl3a-mod modifier). 64bit capable systems can run the 32bit version as well.

OS_ARCH is read by several tools in the following steps to determine what to install. The same is true for CF_SL_release: This variable determines which minor SL release the system will use. Both OS_ARCH and CF_SL_release affect the choice of installation kernel & initrd, installation repository, and yum repositories for updating and installing additional packages.

It should now be safe to do this step without disabling sue on the system, since sue.bootstrap will no longer permit OS_ARCH to change.

Run the Workflow whenever a system changes from DLx to SL3 or back, since some tools (scout) can only consult the netgroups to decide how things should be done. This is wrongwrongwrong, but ...

Creating System Profiles

This is done with the tool CKS.pl which reads "host.cks" files and creates "host.ks" files from them, using additional information from VAMOS, the AMS directory, or the live system still running SL3/4/5, as well as pre/post script building blocks from /project/linux/SL/{pre|post}.

CKS.pl is located in /project/linux/SL/scripts, and is fully perldoc'd. There should be a link pointing to it in the profiles directory as well. A sample DEFAULT.cks with many comments is located in the same directory. This is exactly the same for SL4/5, and the only SL4/5-specific option (SELINUX) is documented (and the default is almost certainly right).

To create a profile:

Activating Private Key Distribution

If you followed the instructions above (read the CKS output), you already know what to do: 

ssh configsrv sudo prepare-ai <host>

This will activate the one-shot mechanism for giving the host (back) its private keys (root password, kerberos keyfile, vamos/ssh keys, ...). The init script retrieved during postinstall will start the script /products/ai/scripts/ai-start which will NFS-export a certain directory to mentor, put an SSL public key there, and ask triton to crypt the credentials with that key and copy them into the directory. If after the installation the host has its credentials, it worked and no other system can possibly have them as well. If it hasn't the keys are burned and have to be scrubbed. Hasn't happened yet, but who knows.

If ai-start fails, the system will retry after 5 minutes. Mails will be sent to linuxroot from both triton and the installing system, indicating that this happened. The reason is usually that this step was forgotten. Remember it has to be repeated before every reinstallation. The ai daemon writes a log /var/log/ai/ai_script.log.

Booting the system into installation

There are several options:

System/Boot Method Matrix

Package Handling & Automatic Updates

See the "aaru" feature for how all this (except kernels) is handled.

There are three distinct mechanisms for package handling on the client:

SL Standard & Errata Packages

<!> For yum on SL3, the command to create the necessary repository data is yum-arch <dir>
For SL4/5, it is createrepo <dir>

(!) Running the script /project/linux/SL/scripts/UpdateRepo.pl -x /packages/RPMS/<sys>/System[.bulk] on a will do the right things to update the yum repodata, the repoview data accessible from Local_Linux_Repositories, and release the right afs volumes.

Errata are synced to a with /project/linux/SL/scripts/sync-a.pl (still manually). Packages to be installed additionally by /sbin/yumsel or updated by /sbin/aaru.yum.boot and /sbin/aaru.yum.daily are NOT taken from the errata mirror created like this, but instead from "staged errata" directories created (also, still manually) by the script /project/linux/SL/scripts/stage-errata. The sync/stage scripts send mail to linuxroot@ifh.de unless in dryrun mode. The stage_errata script is fully perldoc'ed, the others are too simple.

Addon Packages (Zeuthen)

Most of these are found in /afs/ifh.de/packages/RPMS/@sys/System, with their (no)src rpms in /afs/ifh.de/packages/SRPMS and the source tarballs in /afs/ifh.de/packages/SOURCES. Some come from external sources like the dag repository (http://dag.wieers.com/home-made/), freshrpms (http://freshrpms.net/) or the SuSE 8.2/9.0 distributions. These latter ones are typically not accompanied by a src rpm.

After adding a package, make it available to yum like this: 

cd /afs/.ifh.de/packages/RPMS/@sys/System
yum-arch .   # SL3
createrepo . # SL4/5
arcx vos release $PWD

Selectable Addon Packages (Zeuthen)

There's a way to provide packages in selectable repositories. For example, this was used to install an openafs-1.2.13 update on selected systems while the default for SL3 was still 1.2.11, and we didn't want to have 1.2.13 on every system.

These packages reside in directories SL/<release>/<arch>_extra/<name> on the installation server. For example, the afs update packages for 3.0.4/i386 would be in /nfs/a/SL/304/i386_extra/afs1213 . To have clients access this repository, set any vamos variable starting with CF_YUM_extrarepos (CF_YUM_extrarepos or CF_YUM_extrarepos_host or ...) to a space separated list of subdirectories in <arch>_extra.

For example, CF_YUM_extrarepos='afs1213' will make aaru.yum.create add this repository (accessible via nf or http) to the host's yum configuration.

To make available packages in such a repository, you must yum-arch (SL3) or createrepo (SL4) the *sub*directory (not <arch>_extra).

Note that matching kernel modules must still reside in a directory searched by the update script (see below). This should generally not cause problems since these aren't updated by yum anyway.

Additional Modules for Kernel Updates

{i} Starting with SL5, KUSL3.pl is being replaced by KUSL.pl. As of May 2007, the new script is still being tested on SL3/4, but eventually should be used on all platforms.

Handled by the kernel feature, the script /usr/sbin/KUSL3.pl reads its information about which kernels to install from VAMOS variables Linux_kernel_version and a few others, and carries out whatever needs to be done in order to install new kernels and remove old ones. The script is perldoc'ed.

Basically, set Linux_kernel_version in VAMOS, and on the host (after a sue.bootstrap) run KUSL3.pl. Make sure you like what it would do, then run KUSL[3].pl -x.

Kernels and additional packages are found in the repository mirror including the errata directory (CF_SL_release is used to find those), and in /afs/ifh.de/packages/RPMS/@sys/System (and some subdirectories).

If the variable Linux_kernel_modules is set to a (whitespace separated) list of module names, KUSL(3) will install (and require the availability of) the corresponding kernel-module rpm. For example, if Linux_kernel_version is 2.4.21-20.0.1.EL 2.4.21-27.0.2.EL, and Linux_kernel_modules is foo bar, the mandatory modules are:

Generally speaking, kernel module packages must comply with the SL conventions. The new KUSL.pl will also handle packages complying with the kmod conventions introduced with RHEL5.

KUSL(3) will refuse to install a kernel if mandatory packages are not available. Non mandatory packages include kernel-source, sound modules, kernel-doc.

ALSA

This is not needed on SL4/5. For SL3 we no longer provide alsa modules as well.

ESD CAN Module (for PITZ Radiation Monitor)

This hasn't been done yet for SL4/5, and probably never will be.

This is similar to the ALSA modules, but;

Nvidia

These modules are still required for SL3 in particular for the PITZ control systems. Handling is somewhat different on SL5. We don't provide these for SL4.

SL3

Note that SL5-only changes went into the SRPM between nvidia-driver-1.0.9631-1.SLZ and -5.SLZ, hence only the former should be used.

Again, similar to alsa. Maybe a bit simpler since the spec will deal with the kernel sources correctly and without further attention (it makes a copy of the source directory and then does the right thing).

  1. install the right kernel-source package (there's a build requirement)
  2. install the srpm: 
    rpm -ivh /packages/SRPMS/System/nvidia-driver-1.0.7174-3.src.rpm
  3. build (on an SMP system, on a UP system the define changes accordingly): 
    rpmbuild -ba nvidia-driver-1.0.7174-3.spec
    • on i386, will build i386 userspace packages
    • on x86_64, will build userspace and kernel package for current kernel 
    rpmbuild -bb --target i686 nvidia-driver-1.0.7174-3.spec
    • on i386, will build kernel module for running kernel 
    rpmbuild -bb --target i686 --define 'kernel 2.4.21-27.0.2.EL' ...
    • on i686, will build kernel module for other kernels 
    rpmbuild -bb --define 'kernel ...' --define 'build_module 1' nvidia...
    • on x86_64, will build kernel module for other kernel
  4. copy the .rpms to /afs/.ifh.de/packages/@sys/System/nvidia, yum-arch and release

SL5

Since nvidia no longer supports all cards in a single driver, "generations" had to be introduced to support all our PCs. There is a single SRPM, nvidia-driver-gx to build all binary packages from. It will only work on SL5.

The g1 "legacy driver" is required on the Dell Precision 350 with an NVS 280 SD (AGP) board. The new g2 driver should work an all newer systems. Notice the nvidia driver doesn't work on Xen kernels.

  1. make sure the right kernel-devel RPMs are installed (there's a build requirement)
  2. rebuild the SRPM, on 32bit: 
    rpmbuild --rebuild --target i686 --define 'kernel 2.6.18-53.1.4.el5' --define 'nvgen 1' nvidia-driver-gx-...src.rpm
rpmbuild --rebuild --target i686 --define 'kernel 2.6.18-53.1.4.el5' --define 'nvgen 2' nvidia-driver-gx-...src.rpm
    64bit: 
    rpmbuild --rebuild --define 'build_module 1' --define 'kernel 2.6.18-53.1.4.el5' --define 'nvgen 2' nvidia-driver-gx-...src.rpm

For generation 1, we only need the 32bit packages because the Dell 350 doesn't support x86_64.

XFS

These modules only make sense on the x86_64 architecture since running xfs on 32bit EL is not considered a good idea.

SL4

There are two ways to use the XFS filesystem which is not supported by the standard SL kernels:

To do the latter for a specific kernel:

  1. make sure the right kernel[-smp]-devel package is installed

  2. rebuild the source rpm: 
    rpmbuild --rebuild --define 'kernel_topdir /lib/modules/2.6.9-22.0.1.ELsmp/build' \
        /packages/SRPMS/System/xfs/kernel-module-xfs-2.6.9-22.EL-0.1-1.src.rpm
    Note the version-release in the source rpm (here: 2.6.9-22.EL) need not match the one for which you want to build (here: 2.6.9-22.0.1.EL). Hence there's no need to create and store a new source rpm.
  3. copy the rpm to /afs/.ifh.de/packages/amd64_linux26/System/xfs

  4. on a, run /project/linux/SL/scripts/UpdateRepo.pl /packages/RPMS/amd64_linux26/System -x

SL5

We currently use an SRPM taken from dev.centos.org to build external modules according to the kmod convention (KUSL can handle this).

To do this for a specific kernel:

  1. make sure the right kernel[-xen]-devel packages are both installed

  2. rebuild the source rpm: 
    rpmbuild --rebuild --define 'kversion 2.6.18-8.1.4.el5' \
        /packages/SRPMS/System/xfs.SL5/xfs-kmod-0.4-1.2.6.18_8.1.1.el5.src.rpm
    Note the kernel version-release in the source rpm (here: 2.6.18_8.1.1.el5) need not match the one for which you want to build (here: 2.6.18-8.1.4.el5). Hence there's no need to create and store a new source rpm.
  3. copy the rpms to /afs/.ifh.de/packages/amd64_rhel50/System/xfs

  4. on a, run /project/linux/SL/scripts/UpdateRepo.pl /packages/RPMS/i586_rhel50/System -x

ARECA RAID (SL4)

As of kernel 2.6.9-67.0.1.EL (SL4) and 2.6.18-53.1.4.el5 (SL5), this driver is included in the distribution. The current SL4 SRPM just builds a dummy, and as soon as no older kernel are installed on any system with this controller (and sbox1 is the only one), this whole package will be dropped.

The module is called arcmsr. This is needed for "storbox" servers with the Areca controller, and so far only for SL4/x86_64. Other builds have not been tested, though they should work. To build the matching kernel-module-arcmsr-version package for a specific kernel:

  1. make sure the right kernel[-smp]-devel package is installed

  2. rebuild the source rpm: 
    rpmbuild --rebuild --define 'kernel 2.6.9-27.ELsmp' \
        /packages/SRPMS/System/areca/kernel-module-arcmsr-2.6.9-22.0.2.ELsmp-1.20.0X.12-1.SLZ.src.rpm
    Note the version-release in the source rpm (here: 2.6.9-22.0.2.ELsmp) need not match the one for which you want to build (here: 2.6.9-27.ELsmp). Hence there's no need to create and store a new source rpm.
  3. copy the rpm to /afs/.ifh.de/packages/@sys/System/areca, createrepo, and release

Adding a new SL release

There are quarterly releases of SL, following Red Hat's updates to RHEL. Each new release must be made available for installation and updates. The procedure is the same for SL3 and SL4. Just substitute filenames and paths as appropriate:

Step 1: Mirror the new subdirectory

Step 2: Create empty extra postinstall repositories

Note: as of SL 3.0.5, these are no longer used and no longer included in the yum.conf automatically created. 

mkdir /nfs1/a/SL/44/i386_post
cd /nfs1/a/SL/44/i386_post
#SL3:
yum-arch-dl5 .
#SL4:
createrepo .

mkdir /nfs1/a/SL/44/x86_64_post
cd /nfs1/a/SL/44/x86_64_post
#SL3:
yum-arch-dl5 .
#SL4:
createrepo .

If some packages are needed at this stage, of course put them there...

Step 3: Create staged errata directories

Modify /project/linux/SL/scripts/stage-errata.cf to include the new release. Note if you're trying 30rolling as a test for the release, you must configure 30rolling, not 304 (or whatever). The same for SL4. Now run stage-errata.

Step 4: Make the kernel/initrd available by TFTP for PXE boot

Run the script 

/project/linux/SL/scripts/tftp_add_sl_release 44 i386

and accordingly for other releases and architectures. This will copy the kernel and the initrd, and create a pxelinux configuration file. You may still want/have to add a few lines in /tftpboot/pxelinux.cfg/default (for example, for Tier2 installs).

Step 5: Make the release available in VAMOS

Fire up the GUI, select "vars" as the top object, go to CF_SL_release, choose the "values_host" tab, and add the new value to the available choices. Set it on some test host.

Step 6: test

Make sure this works and sets the right link: 

  /project/linux/SL/scripts/pxe <testhost>

Make sure this chooses the right directory: 

  cd /project/linux/SL/profiles
  ./CKS.pl <testhost>

Make sure SLU works correctly: 

  ssh <testhost>
  /project/linux/SL/scripts/SLU.pl yes please

Try an installation:

Try updating an existing installation:

Booting a rescue system

There are several ways to do this, including:

From CD1 of the distribution

Simply boot from CD1 of the distribution. At the boot prompt, type linux rescue.

Over the network with the unified installation CD

Just boot whatever entry you normally would for installation, but add the keyword rescue to the command line.

Building Kernel Packages

32-bit SL3

First install the kernel srpm (not kernel-source). Make your changes to the spec, add patches etc. 

rpmbuild --sign -ba kernel-2.4.spec

This will build 

rpmbuild --sign -ba --target i686 kernel-2.4.spec

This will build

Trying to turn off build of the hugemen kernel breaks the spec.

Additional modules for these are built as for any other SL kernel, with one exception of course:

Building the 1.2.x kernel-module-openafs (and openafs) packages

For ordinary SL kernels, this is done at FNAL, hence we needn't bother. But for our own kernels, or if we want to apply a change to the SRPM, we have to do this ourselves.

The kernel version must always be defined, and always without the "smp" suffix.

Otherwise, when building on an SMP system, the build will work but the modules will not.

For each kernel version you want to build modules for, install the kernel-source, kernel, and kernel-smp RPMs on the build system, they're all needed. Then:

Building the NEW 1.4.x kernel-module-openafs packages

As of December 2006, there exists a unified SRPM for OpenAFS 1.4.2, which doesn't have the build problems described above, and works in exactly the same way on SL3, SL4, and SL5. It's named openafs.SLx, but will create packages named openafs with SL3, SL4, SL5 in the release number. The SRPM can (and should) be rebuilt without being root. The steps are the same on every platform:

First install the right kernel-source (SL3) or matching kernel[-smp|-largesmp|-xen|...]-devel package for the target kernel.

Then run: 

rpmbuild --rebuild --sign --target  i686 --define 'kernel 2.4.21...' --define 'build_modules 1' openafs.SLx-...src.rpm

There's always just one module built per invocation. Building on SMP systems is ok (and really fast).

Supported targets include i686, athlon (SL3 only), ia32e (SL3 only, must build on 64bit system), x86_64 (must build on 64bit system), ia64 (untested).

Supported kernel flavours include smp (SL3/4), hugemem (SL3/4), largesmp (SL4), xen (SL5), PAE (SL5).

References

http://www.redhat.com/docs/manuals/enterprise/RHEL-3-Manual/sysadmin-guide/ch-kickstart2.html

http://linux.duke.edu/projects/yum/