Differences between revisions 34 and 35
⇤ ← Revision 34 as of 2008-11-05 10:23:55 →
Size: 42129
Editor: StephanWiesand
Comment:
← Revision 35 as of 2008-11-05 10:24:09 → ⇥
Size: 42177
Editor: StephanWiesand
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was renamed from SL3 Operations Manual

Contents

  1. Installation
    1. Caveats
    2. Overview
    3. System Configuration in VAMOS
    4. Creating System Profiles
    5. Activating Private Key Distribution
    6. Booting the system into installation
    7. System/Boot Method Matrix
  2. Package Handling & Automatic Updates
    1. SL Standard & Errata Packages
    2. Addon Packages (Zeuthen)
      1. Selectable Addon Packages (Zeuthen)
    3. Additional Modules for Kernel Updates
      1. ALSA
      2. ESD CAN Module (for PITZ Radiation Monitor)
      3. Nvidia
        1. SL3
        2. SL5
      4. XFS
        1. SL4
        2. SL5
      5. ARECA RAID (SL4)
  3. Adding a new SL release
  4. Booting a rescue system
  5. Building Kernel Packages
    1. 32-bit SL3
      1. Building the kernel-module-openafs packages
      2. Building the NEW kernel-module-openafs packages
  6. References

Installation

Caveats

The following classes of systems need some extra attention for installation:

  • systems running with eth1 but no eth0 (globes - SUN V65x)

    • there's a "globe" post script now to handle this

    • {OK} seems no longer to be a problem with SL4

  • systems running with eth0, but having additional interfaces recognized as eth0 by anaconda

    • picus1,2: there's a "e1000-no-e100" post script for those
    • fatmans: there's an "acenic-no-e100" post script for those

    • {OK} SL4 seems to give e1000 priority over e100

  • systems with recent nvidia graphics

    • Dell Precision 370/380/390: this requires the proprietary driver in order to run

      any mode except 1280x1024@60Hz (achieved by setting CF_yumsel_host="nvidia 90"), which is still slightly experimental

    • {OK} no longer an issue on SL5

  • Dell Precision 380: A BIOS bug in versions including A02 makes SL3 (not SL4/5) kill the mouse and keyboard unless some reordering is done in /etc/modules.conf, which can be achieved by adding pr380-usb to $cfg{POSTINSTALL_SCRIPTS} in the CKS file

  • systems with certain ethernet cards anaconda has a problem to get up the second time

    • {OK} this problem seems to have vanished with SL 3.0.5

    • Dell 2850: workaround is to put the ks.cfg file onto a floppy and boot with "ks=floppy", or to pack a special initrd with the ks.cfg in the root filesystem and boot with ks=file, or to use the -local option to SLU

Overview

SL3/4/5 hosts are installed using kickstart

The repository is mirrored from the ftp server at FNAL and is located on the installation server, a.ifh.de, in /nfs1/a/SL. The host profiles for the kickstart install are kept in /project/linux/SL/profiles, and some files needed during the postinstallation, before the AFS client is available, in /nfs1/a/SL6/post and /project/linux/SL/firstboot (both accessible through the http server running on a). More files, and most utility scripts, are located in /project/linux/SL.

Installation takes the following steps:

  • Configure the host in VAMOS This is important, because several variables must be set correctly since they are needed by the tools used in the following steps.

  • Create a system profile Using CKS, information from VAMOS and possibly from the AMS directory or the live host, a kickstart file is generated that will steer the installation process.

  • Activate private key distribution Only after this step, the host will be able to request its private keys and initial configuration cache from mentor.

  • Prepare system boot into installation Current options include PXE, CD-ROM, and hard disk. Other possible methods like USB stick, multiple floppies, or a tftp grub floppy are not yet available.

  • Boot the system into installation During boot, the system will load the kernel and initrd made available in step (d). Networking information comes from a DHCP server (possible

    with all methods) or is provided on the kernel command line (CD-ROM & hard disk methods only). The installation system locates the kickstart profile. Information is from a kernel command line provided by the tftp server (PXE method), manually (CD-ROM method), or the script preparing the hard disk boot. The kickstart profile contains all other information needed, including

    the repository location, partitioning & package selection, and a postinstall script that will do some very basic configuration and retrieve and install a one-time init script. After the first reboot, this init script (executing as the very last one) will retrieve the system's private keys and initial vamos configuration cache, and then bootstrap our site mechanisms for system maintenance.

System Configuration in VAMOS

Choose a default derived from sl3-def. Defaults starting with "sl3-" are 32bit, those starting with "sl3a-" are 64bit. These will mainly differ in the settings for OS_ARCH and AFS_SYSNAME (see the sl3a-mod modifier). 64bit capable systems can run the 32bit version as well.

OS_ARCH is read by several tools in the following steps to determine what to install. The same is true for CF_SL_release: This variable determines which minor SL release the system will use. Both OS_ARCH and CF_SL_release affect the choice of installation kernel & initrd, installation repository, and yum repositories for updating and installing additional packages.

It should now be safe to do this step without disabling sue on the system, since sue.bootstrap will no longer permit OS_ARCH to change.

Run the Workflow whenever a system changes from DLx to SL3 or back, since some tools (scout) can only consult the netgroups to decide how things should be done. This is wrongwrongwrong, but ...

Creating System Profiles

This is done with the tool CKS.pl which reads "host.cks" files and creates "host.ks" files from them, using additional information from VAMOS, the AMS directory, or the live system still running SL3/4/5, as well as pre/post script building blocks from /project/linux/SL/{pre|post}.

CKS.pl is located in /project/linux/SL/scripts, and is fully perldoc'd. There should be a link pointing to it in the profiles directory as well. A sample DEFAULT.cks with many comments is located in the same directory. This is exactly the same for SL4/5, and the only SL4/5-specific option (SELINUX) is documented (and the default is almost certainly right).

To create a profile:

  • You need
    • write access to /project/linux/SL/profiles
    • read access to VAMOS
    • ssh access to the system to install, if it is up and partitions to be kept
  • Go into /project/linux/SL/profiles.
  • Check whether a .cks file for your host exists.
    • If it does, and you find you have to modify the file, make sure it is not a link to some other file before you do so.
    • If it does not, create one by starting with a copy from a similar machine, or a copy of DEFAULT.cks.

    • (!) NO host.cks IS NEEDED AT ALL if you just want to upgrade or reinstall a normal system without changing disk partitioning, since DEFAULT.cks is always read and should cover this case completely.

  • Run CKS.pl, like this: ./CKS.pl host

    <!> Always rerun CKS, even if the existing .cks file looks fine.

    This is because the selection and the postinstall script are copied into the kickstart file, and the old file may no longer be correct.
  • Watch the output. Make sure you understand what the profile is going to do to the machine! If in doubt, read and understand the

    <host>.ks file before actually installing.

    /!\ In particular, make sure you understand the partitioning, and any clearpart statements.

    Other than for DL5, these may wipe the disks even in "interactive" installs!
    Also make sure the SL release and architecture are what you want.

Activating Private Key Distribution

If you followed the instructions above (read the CKS output), you already know what to do: 

ssh triton sudo activ-ai <host>

This will activate the one-shot mechanism for giving the host (back) its private keys (root password, kerberos keyfile, vamos/ssh keys, ...). The init script retrieved during postinstall will start the script /products/ai/scripts/ai-start which will NFS-export a certain directory to mentor, put an SSL public key there, and ask triton to crypt the credentials with that key and copy them into the directory. If after the installation the host has its credentials, it worked and no other system can possibly have them as well. If it hasn't the keys are burned and have to be scrubbed. Hasn't happened yet, but who knows.

If ai-start fails, the system will retry after 5 minutes. Mails will be sent to linuxroot from both triton and the installing system, indicating that this happened. The reason is usually that this step was forgotten. Remember it has to be repeated before every reinstallation. The ai daemon writes a log /var/log/ai/ai_script.log.

Booting the system into installation

There are several options:

  • Perl Script If the system is still running a working SL installation, this is the most convenient and reliable method: After logging on as root, run the script 
    /project/linux/SL/scripts/SLU.pl yes please
    and either let the script reboot the system after the countdown, or interrupt the countdown with ^C and reboot (or have the user reboot) later. The script will create an additional, default, boot loader entry to start the installation system. By default, all needed information is appended to the kernel command line, including networking information. Hence not even DHCP is needed. The script comes with full perldoc documentation. Some additional options are available or may even be necessary for certain hosts (see 1.0).
  • SL CD-ROM

    (!) It is much more convenient now to use the unified CD

    Images are /nfs/a/SL/<release>/<arch>/images/SL/boot.iso The release and arch have to match exactly the planned installation, or the installation system will refuse to work.

    • If the system has a valid DHCP entry (inluding the MAC address):

      At the boot prompt enter "linux ks=http://a.ifh.de/SL/profiles/" If the system has more than one network interface, add "ksdevice=link". If the system has more than one network interface *connected*, instead add "ksdevice=eth0" or whatever is appropriate.

    • If the system has no valid DHCP entry yet: You have to add parameters like "ip=141.34.x.y netmask=255.255.255.0 gateway=141.34.x.1 dns=141.34.1.16" Or watch the log on the DHCP server, wait for the unknown MAC to appear, create a working entry for the host in /etc/dhcpd.conf, and restart the dhcp server. If you're quick enough, the client will receive an answer when it retries. Otherwise it has to be booted again. Don't forget to remove the CD before the first reboot, or installation will start all over.

  • Grand Unified Boot CD This CD has a boot loader menu that allows installing SL3, and SL4. It also has preset kernel parameters that save you almost all typing: ks=..., ksdevice=link, netmask=..., dns=... are always set, but hidden, and there are visible, editable templates for ip=..., gateway=...). Menu entries are:

    • local This is the default, and will boot from the primary hard disk (whatever the BIOS thinks this is). Hence it's no problem if you forget to remove the CD during installation.
    • dl5-dhcp Installs DL5, getting network parameters by DHCP.
    • dl5-manual Installs DL5 with network parameters specified on the command line. When you select this entry, you'll see the tail of the preset options: 
      "textmode=0 hostip=141.34.x.y gateway=141.34.x.1"
      Simply replace "x" and "y" by the appropriate values for the host and hit Enter.
    • sl303_32-d Install SL3.0.3/i386 using dhcp.
    • sl303_32-m Install SL3.0.3/i386. Network parameters are given on the command line (replace "x" and "y" in "ip=141.34.x.y gateway=141.34.x.1").
    • sl303_64-d Install SL3.0.3/x86_64 using dhcp.
    • sl303_64-m Install SL3.0.3/x86_64. Network parameters are given on the command line (replace "x" and "y" in "ip=141.34.x.y gateway=141.34.x.1").
    The entries for SL may vary over time, but generally follow the pattern

    sl<release>_<bits>-<method>, where bits is "32" or "64", and method is "d" for dhcp or "m" for manual. They have to be this cryptic because there's a 10 character limit for the labels :-( The ISO image is /project/linux/SL/CD/InstallCD.iso, and the script next to it (it's in the CD's root directory as well) can be used to create modified images, for additional SL releases or different DL5 install kernels etc.: Simply edit the variables at the top (@SL_releases, $DL_kernel) and rerun the script on a system that has syslinux and mkisofs installed (tested on DL5 and SL3/32). The script will tell you the directory in /tmp where it writes the image.

  • PXE This requires entries on both the DHCP and TFTP servers. The client will receive IP, netmask, gateway etc from the DHCP server, plus the information that it should fetch "pxelinux.0" from the TFTP server (actually, a) and run it. Then, pxelinux.0 will request the host configuration file (IP address in hex notation) from the TFTP server (a link in /tftpboot/pxelinux.cfg/). This will in turn

    tell pxelinux.0 which kernel & initrd to retrieve from the TFTP server, and what parameters the kernel should receive on the command line.

    • TFTP & DHCP (script): As root on a, run 

      /project/linux/SL/scripts/pxe <host>
      This will add the right link for the system in /tftpboot/pxelinux.cfg and also attempt to update the DHCP configuration on the right server.
    • DHCP (manually): If the system has no valid DHCP entry yet, you have to use the last method given above in (b) to find out the MAC and create or complete the entry in the configuration file manually. In addition to IP and MAC, the following parameters have to be supplied: 
      next-server 141.34.32.16;
filename "pxelinux.0";
      If there was an incomplete entry for the system, these will already be present after running the "pxe" utility script.
    The changes on the DHCP server will be reverted the next time the dhcp feature runs, so no need for cleanup. To get rid of the link in /tftpboot/pxelinux.cfg, simply run (as root on z) 
    /project/linux/SL/scripts/unpxe <host>
    If the client boots via PXE afterwards, it will pick up the default configuration which tells it to boot from its local disk. Anyway, using PXE is not recommended for systems which have no "one time boot menu" or a "request network boot" key stroke.
  • GRUB Floppy As a last resort, one can try the grub floppy. This method will obtain the kernel and the initrd by tftp, hence a matching link has to exist in /tftpboot/pxelinux.cfg on the install server. If the host has a working dhcp entry, just boot the default entry on the floppy. If it doesn't select the other entry, hit 'e' to get into the editor, replace all "x" and "y" in the templates for IP addresses and netmasks, and finally hit "b" to boot the modified entry. The network drivers in GRUB only work for older cards. This is no problem because the more recent ones support PXE anyway. In particular, the 3C905 in PIII desktop PCs (najade or older PIII 750) is supported. The floppy image is located in /project/linux/SL/Floppy. To adapt it to a new SL release or install kernel, simply loop mount the image and make the obvious changes in boot/grub/menu.lst.

System/Boot Method Matrix

  		•

    Script

    CD

    Floppy

    PXE

    older systems

    yes

    maybe

    yes

    no

    PIII 750 desktop

    yes

    some(1)

    yes

    no(2)

    Najade (PIII 850)

    yes

    some(1)

    yes

    no(2)

    Nereide (P4 1700)

    yes

    yes

    ?

    ?

    Oceanide (P4 2400)

    yes

    yes

    ?

    yes

    all Dell desktops

    yes

    yes

    no

    yes

    ice (intel serverboard)

    yes

    yes

    yes

    yes

    Supermicro PIII

    yes

    yes

    no

    no

    Supermicro Xeon

    yes

    yes

    no

    no

    globe (SUN V65x)

    yes

    yes

    no

    yes

    heliade (SUN V20z)

    yes

    yes

    no

    yes

    galaxy (SUN X4100)

    yes

    yes

    no

    yes

    Dell 1850

    yes

    yes

    no

    yes

    Dell 2850

    yes(3)

    yes(3)

    no

    yes(3)

    Dell 9G Servers (1950...)

    yes

    yes

    no

    yes

    (1) This seems to depend on the motherboard and/or BIOS revision. In fact, some 850MHz models won't boot from CD while there are older 750MHz systems that will. (2) The PXE implementation in the MBA has a bug: It does not recognize the next-server argument and always tries to download the Loader from the same server that sent the DHCP offer. Hence these systems

    can be PXE-booted by setting up a special DHCP&TFTP server. (3) Anaconda up to and including at least SL 3.0.4 has a problem to get up the NIC a second time. Workarounds include putting the ks.cfg on a floppy or into the initrd, or using the -local switch to SL3U.pl. Fixed in recent SL3 releases (3.0.7 is ok, maybe 3.0.5 was as well).

Package Handling & Automatic Updates

See the "aaru" feature for how all this (except kernels) is handled.

There are three distinct mechanisms for package handling on the client:

  • aaru (package updates) Handled by the aaru feature, the scripts /sbin/aaru.yum.daily and /sbin/aaru.yum.boot run yum to update installed packages. Yum [2] is told to use specific repository descriptions for these tasks, which are created by /sbin/aaru.yum.create before, according to the values of VAMOS variables OS_ARCH, CF_SL_release, CF_YUM_extrarepos* and CF_DZPM_AGING.
  • yumsel (addition and removal of packages) Handled by the aaru feature, the script /sbin/yumsel installs additional packages or removes installed ones. Configuration files for this task are read from /etc/yumsel.d/, which is populated by /sbin/yumsel.populate before, according to the values of VAMOS variables CF_yumsel_*.
  • KUSL(3) (everything related to kernels) Handled by the kernel feature, this script deals with kernels and related packages (modules, source), according to the values of VAMOS variable Linux_kernel_version and a few others. On SL5, KUSL3 was replaced by KUSL, and this should happen eventually on SL3/4 as well. SL3/4 systems in update class "A" already use KUSL.pl.

SL Standard & Errata Packages

<!> For yum on SL3, the command to create the necessary repository data is yum-arch <dir>
For SL4/5, it is createrepo <dir>

(!) Running the script /project/linux/SL/scripts/UpdateRepo.pl -x /packages/RPMS/<sys>/System[.bulk] on a will do the right things to update the yum repodata, the repoview data accessible from Local_Linux_Repositories, and release the right afs volumes.

Errata are synced to a with /project/linux/SL/scripts/sync-a.pl (still manually). Packages to be installed additionally by /sbin/yumsel or updated by /sbin/aaru.yum.boot and /sbin/aaru.yum.daily are NOT taken from the errata mirror created like this, but instead from "staged errata" directories created (also, still manually) by the script /project/linux/SL/scripts/stage-errata. The sync/stage scripts send mail to linuxroot@ifh.de unless in dryrun mode. The stage_errata script is fully perldoc'ed, the others are too simple.

Addon Packages (Zeuthen)

Most of these are found in /afs/ifh.de/packages/RPMS/@sys/System, with their (no)src rpms in /afs/ifh.de/packages/SRPMS and the source tarballs in /afs/ifh.de/packages/SOURCES. Some come from external sources like the dag repository (http://dag.wieers.com/home-made/), freshrpms (http://freshrpms.net/) or the SuSE 8.2/9.0 distributions. These latter ones are typically not accompanied by a src rpm.

After adding a package, make it available to yum like this: 

cd /afs/.ifh.de/packages/RPMS/@sys/System
yum-arch .   # SL3
createrepo . # SL4/5
arcx vos release $PWD

Selectable Addon Packages (Zeuthen)

There's a way to provide packages in selectable repositories. For example, this was used to install an openafs-1.2.13 update on selected systems while the default for SL3 was still 1.2.11, and we didn't want to have 1.2.13 on every system.

These packages reside in directories SL/<release>/<arch>_extra/<name> on the installation server. For example, the afs update packages for 3.0.4/i386 would be in /nfs/a/SL/304/i386_extra/afs1213 . To have clients access this repository, set any vamos variable starting with CF_YUM_extrarepos (CF_YUM_extrarepos or CF_YUM_extrarepos_host or ...) to a space separated list of subdirectories in <arch>_extra.

For example, CF_YUM_extrarepos='afs1213' will make aaru.yum.create add this repository (accessible via nf or http) to the host's yum configuration.

To make available packages in such a repository, you must yum-arch (SL3) or createrepo (SL4) the *sub*directory (not <arch>_extra).

Note that matching kernel modules must still reside in a directory searched by the update script (see below). This should generally not cause problems since these aren't updated by yum anyway.

Additional Modules for Kernel Updates

{i} Starting with SL5, KUSL3.pl is being replaced by KUSL.pl. As of May 2007, the new script is still being tested on SL3/4, but eventually should be used on all platforms.

Handled by the kernel feature, the script /usr/sbin/KUSL3.pl reads its information about which kernels to install from VAMOS variables Linux_kernel_version and a few others, and carries out whatever needs to be done in order to install new kernels and remove old ones. The script is perldoc'ed.

Basically, set Linux_kernel_version in VAMOS, and on the host (after a sue.bootstrap) run KUSL3.pl. Make sure you like what it would do, then run KUSL[3].pl -x.

Kernels and additional packages are found in the repository mirror including the errata directory (CF_SL_release is used to find those), and in /afs/ifh.de/packages/RPMS/@sys/System (and some subdirectories).

If the variable Linux_kernel_modules is set to a (whitespace separated) list of module names, KUSL(3) will install (and require the availability of) the corresponding kernel-module rpm. For example, if Linux_kernel_version is 2.4.21-20.0.1.EL 2.4.21-27.0.2.EL, and Linux_kernel_modules is foo bar, the mandatory modules are:

  • name

    version

    release

    kernel-module-foo-2.4.21-20.0.1.EL

    latest

    latest

    kernel-module-bar-2.4.21-20.0.1.EL

    latest

    latest

    kernel-module-foo-2.4.21-27.0.2.EL

    latest

    latest

    kernel-module-bar-2.4.21-27.0.2.EL

    latest

    latest

Generally speaking, kernel module packages must comply with the SL conventions. The new KUSL.pl will also handle packages complying with the kmod conventions introduced with RHEL5.

KUSL(3) will refuse to install a kernel if mandatory packages are not available. Non mandatory packages include kernel-source, sound modules, kernel-doc.

ALSA

This is not needed on SL4/5. For SL3 we no longer provide alsa modules as well.

Matching kernel-module-alsa-`uname -r` packages are installed by KUSL3.pl if (a) they are available and (b) the package alsa-driver is installed (the latter should be the case on desktops after yumsel has run for the first time).

Both are created from the alsa-driver srpm found in /packages/SRPMS/System. Besides manual rebuilds, there is now the option to use the script /project/linux/SL3/modules/build-alsa-modules.pl. (which should be moved eventually)

Short instructions for building the kernel modules package manually (for an easier method, see below):

  • <!> First make sure you use the right compiler.

    which gcc should say /usr/bin/gcc, not something from /opt/products!
  • Install the kernel-source rpm for the target kernel. For example, this is kernel-source-2.4.21-20.EL for both kernels 2.4.21-20.EL and 2.4.21-20.ELsmp. KUSL3 will do this for you if Linux_kernel_source is set accordingly (if in doubt, set it to "all" in VAMOS, and on the build system sue.bootstrap and run KUSL3). You need not be running the target kernel in order to build the modules.
  • Clean the source directory 
    cd /usr/src/linux-2.4.21.....
make mrproper
  • Configure the source tree. First, find the right configuration:

    • either the file /boot/config-<kernelversion

    • or a matching file /usr/src/linux-2.4.21..../configs/kernel-.... 
    cp <config for target kernel> .config
sed -i 's/^\(EXTRAVERSION.*\)custom/\1/' Makefile
make oldconfig
make dep
make clean

    If the kernel release ends in smp (i686 and x86_64, but not ia32e SMP), append smp to the EXTRAVERSION in the Makefile. Otherwise, the modules built will still be ok, but there'll be complaints in the logs about them not matching the kernel.

  • Build the binary module package, like this: 
    cd /usr/src/packages/SPECS
rpm -ivh /packages/SRPMS/System/alsa-driver-1.0.7-1.nosrc.rpm
ln -s /packages/SOURCES/alsa/* ../SOURCES

rpmbuild -bb --target i686 --define "kernel 2.4.21-20.0.1.ELsmp" \
          alsa-driver-1.0.7-1.spec
  • Repeat steps (b,c,d) for every target kernel. Modify the target and kernel version according to what you need. We'll typically need i686 for both SMP and UP kernels. The 64bit modules have to be built on a 64bit system. Note the ia32e kernel actually *is* SMP although the name doesn't say so, and there is no UP kernel for this architecture at all. Here's a table of what you probably want to build:

    • target

      version

      needed

      i686

      2.4.21-20.0.1.ELsmp

      definitely

      i686

      2.4.21-20.0.1.EL

      definitely

      ia32e

      2.4.21-20.0.1.EL

      probably not

      x86_64

      2.4.21-20.0.1.ELsmp

      probably not

  • Copy the resulting kernel-module-alsa-<kernelversion> rpms to the right directory: 

    i686:         /afs/.ifh.de/packages/RPMS/i586_rhel30/System/alsa
ia32e/x86_64: /afs/.ifh.de/packages/RPMS/amd64_rhel30/System/alsa
    Then make them available to yum: 
    cd /afs/.ifh.de/..../System
yum-arch .
arcx vos release $PWD
    There is no need to copy the alsa-driver rpms generated, unless a new alsa version has been built, in which case one of the resulting packages should be copied and yum-arch'd per target directory. After step (f), KUSL3 will pick up the modules.

Scripted build of the kernel modules packages:

  • <!> First make sure you use the right compiler.

    which gcc should say /usr/bin/gcc, not something from /opt/products!
  • Make sure the kernel-source rpms are installed for all kernels you want to build for.

  • Make sure the whole kernel source tree(s) are owned by you, and that you can run ssh root@<buildhost>.

  • Run the script like this: 
    /project/linux/SL/scripts/build-sl3-alsa-modules.pl 1.0.8-1
    You'll be prompted for every kernel that the script can sensibly build modules for on this system. Pick the ones you want. Check the dryrun output, and once you like it: 
    /project/linux/SL/scripts/build-sl3-alsa-modules.pl -x 1.0.8-1
    This should build everything you need (after going through the prompting again).
  • Copy the output rpms into the repository (as described in step (f) for the manual build above). The script will print the commands that need to be executed.

ESD CAN Module (for PITZ Radiation Monitor)

This hasn't been done yet for SL4/5, an probably never will be.

This is similar to the ALSA modules, but;

  • The srpms are in /packages/SRPMS/esdcan (different ACL from others due to proprietary license of source).
  • There's no build script.
  • Builds should be done manually, on pitzrap itself, and always against

    a fresh kernel-source package:

    1. remove the kernel-source package(s)
    2. install the right kernel-source package for the kernel you want to build the module for
    3. configure it: 
      cd /usr/src/linux-2.4.....
cp configs/kernel-2.4.21-i686.config .config
sed -i 's/^\(EXTRAVERSION.*\)custom/\1/' Makefile
make dep
make clean
    4. install the srpm (it doesn't matter from which kernel build it is): 
      rpm -ivh /packages/SRPMS/esdcan/kernel-module-esdcan-...3.3.3-1.src.rpm
    5. build: 
      rpmbuild -ba [--define 'kernel 2.4.21....'] --target i686 kernel-module-esdcan.spec
    6. copy the .i686.rpms to /afs/.ifh.de/packages/i586_rhel30/System/esdcan, yum-arch and release

Nvidia

These modules are still required for SL3 in particular for the PITZ control systems. Handling is somewhat different on SL5. We don't provide these for SL4.

SL3

Note that SL5-only changes went into the SRPM between nvidia-driver-1.0.9631-1.SLZ and -5.SLZ, hence only the former should be used.

Again, similar to alsa. Maybe a bit simpler since the spec will deal with the kernel sources correctly and without further attention (it makes a copy of the source directory and then does the right thing).

  1. install the right kernel-source package (there's a build requirement)
  2. install the srpm: 
    rpm -ivh /packages/SRPMS/System/nvidia-driver-1.0.7174-3.src.rpm
  3. build (on an SMP system, on a UP system the define changes accordingly): 
    rpmbuild -ba nvidia-driver-1.0.7174-3.spec
    • on i386, will build i386 userspace packages
    • on x86_64, will build userspace and kernel package for current kernel 
    rpmbuild -bb --target i686 nvidia-driver-1.0.7174-3.spec
    • on i386, will build kernel module for running kernel 
    rpmbuild -bb --target i686 --define 'kernel 2.4.21-27.0.2.EL' ...
    • on i686, will build kernel module for other kernels 
    rpmbuild -bb --define 'kernel ...' --define 'build_module 1' nvidia...
    • on x86_64, will build kernel module for other kernel
  4. copy the .rpms to /afs/.ifh.de/packages/@sys/System/nvidia, yum-arch and release

SL5

Since nvidia no longer supports all cards in a single driver, "generations" had to be introduced to support all our PCs. There is a single SRPM, nvidia-driver-gx to build all binary packages from. It will only work on SL5.

The g1 "legacy driver" is required on the Dell Precision 350 with an NVS 280 SD (AGP) board. The new g2 driver should work an all newer systems. Notice the nvidia driver doesn't work on Xen kernels.

  1. make sure the right kernel-devel RPMs are installed (there's a build requirement)
  2. rebuild the SRPM, on 32bit: 
    rpmbuild --rebuild --target i686 --define 'kernel 2.6.18-53.1.4.el5' --define 'nvgen 1' nvidia-driver-gx-...src.rpm
rpmbuild --rebuild --target i686 --define 'kernel 2.6.18-53.1.4.el5' --define 'nvgen 2' nvidia-driver-gx-...src.rpm
    64bit: 
    rpmbuild --rebuild --define 'build_module 1' --define 'kernel 2.6.18-53.1.4.el5' --define 'nvgen 2' nvidia-driver-gx-...src.rpm

For generation 1, we only need the 32bit packages because the Dell 350 doesn't support x86_64.

XFS

These modules only make sense on the x86_64 architecture since running xfs on 32bit EL is not considered a good idea.

SL4

There are two ways to use the XFS filesystem which is not supported by the standard SL kernels:

  • run a modified kernel (provided for example in the SL contrib area, or CentOSplus)

  • build kernel-module-xfs packages for the standard kernel (possible on SL4 only)

To do the latter for a specific kernel:

  1. make sure the right kernel[-smp]-devel package is installed

  2. rebuild the source rpm: 
    rpmbuild --rebuild --define 'kernel_topdir /lib/modules/2.6.9-22.0.1.ELsmp/build' \
        /packages/SRPMS/System/xfs/kernel-module-xfs-2.6.9-22.EL-0.1-1.src.rpm
    Note the version-release in the source rpm (here: 2.6.9-22.EL) need not match the one for which you want to build (here: 2.6.9-22.0.1.EL). Hence there's no need to create and store a new source rpm.
  3. copy the rpm to /afs/.ifh.de/packages/amd64_linux26/System/xfs

  4. on a, run /project/linux/SL/scripts/UpdateRepo.pl /packages/RPMS/amd64_linux26/System -x

SL5

We currently use an SRPM taken from dev.centos.org to build external modules according to the kmod convention (KUSL can handle this).

To do this for a specific kernel:

  1. make sure the right kernel[-xen]-devel packages are both installed

  2. rebuild the source rpm: 
    rpmbuild --rebuild --define 'kversion 2.6.18-8.1.4.el5' \
        /packages/SRPMS/System/xfs.SL5/xfs-kmod-0.4-1.2.6.18_8.1.1.el5.src.rpm
    Note the kernel version-release in the source rpm (here: 2.6.18_8.1.1.el5) need not match the one for which you want to build (here: 2.6.18-8.1.4.el5). Hence there's no need to create and store a new source rpm.
  3. copy the rpms to /afs/.ifh.de/packages/amd64_rhel50/System/xfs

  4. on a, run /project/linux/SL/scripts/UpdateRepo.pl /packages/RPMS/i586_rhel50/System -x

ARECA RAID (SL4)

As of kernel 2.6.9-67.0.1.EL (SL4) and 2.6.18-53.1.4.el5 (SL5), this driver is included in the distribution. The current SL4 SRPM just builds a dummy, and as soon as no older kernel are installed on any system with this controller (and sbox1 is the only one), this whole package will be dropped.

The module is called arcmsr. This is needed for "storbox" servers with the Areca controller, and so far only for SL4/x86_64. Other builds have not been tested, though they should work. To build the matching kernel-module-arcmsr-version package for a specific kernel:

  1. make sure the right kernel[-smp]-devel package is installed

  2. rebuild the source rpm: 
    rpmbuild --rebuild --define 'kernel 2.6.9-27.ELsmp' \
        /packages/SRPMS/System/areca/kernel-module-arcmsr-2.6.9-22.0.2.ELsmp-1.20.0X.12-1.SLZ.src.rpm
    Note the version-release in the source rpm (here: 2.6.9-22.0.2.ELsmp) need not match the one for which you want to build (here: 2.6.9-27.ELsmp). Hence there's no need to create and store a new source rpm.
  3. copy the rpm to /afs/.ifh.de/packages/@sys/System/areca, createrepo, and release

Adding a new SL release

There are quarterly releases of SL, following Red Hat's updates to RHEL. Each new release must be made available for installation and updates. The procedure is the same for SL3 and SL4. Just substitute filenames and paths as appropriate:

Step 1: Mirror the new subdirectory

  • Create a new logical volume on a: 
    lvcreate -L 30G -n SL44 vg00

  • Add an according line in /etc/fstab (mount with the acl option)

  • Add an according line to ~TARCH/exports/a/etc/exports, then sue.update exports
  • Create the directory, mount the volume, and make sure permissions and security context are right: 
    chgrp sysprog /nfs1/a/SL/44
chmod g+w /nfs1/a/SL/44
setfacl -m g:sysprog:rwx /nfs1/a/SL/44
setfacl -m d:g:sysprog:rwx /nfs1/a/SL/44
chcon system_u:object_r:httpd_sys_content_t nfs1/a/SL/44
    The last command makes it possible to access the directory through apache. The chgrp and chmod are actually redundant if ACLs are used.
  • Modify sync-a.pl to include the new release. Now sync-a.pl (do a dryrun first, and look wether additional subdirectories should be excluded).
  • If you're using x0rolling for testing, make a link like this: 
    /nfs1/a/SL/44 -> 40rolling
  • Check access through NFS and http

Step 2: Create empty extra postinstall repositories

Note: as of SL 3.0.5, these are no longer used and no longer included in the yum.conf automatically created. 

mkdir /nfs1/a/SL/44/i386_post
cd /nfs1/a/SL/44/i386_post
#SL3:
yum-arch-dl5 .
#SL4:
createrepo .

mkdir /nfs1/a/SL/44/x86_64_post
cd /nfs1/a/SL/44/x86_64_post
#SL3:
yum-arch-dl5 .
#SL4:
createrepo .

If some packages are needed at this stage, of course put them there...

Step 3: Create staged errata directories

Modify /project/linux/SL/scripts/stage-errata.cf to include the new release. Note if you're trying 30rolling as a test for the release, you must configure 30rolling, not 304 (or whatever). The same for SL4. Now run stage-errata.

Step 4: Make the kernel/initrd available by TFTP for PXE boot

Run the script 

/project/linux/SL/scripts/tftp_add_sl_release 44 i386

and accordingly for other releases and architectures. This will copy the kernel and the initrd, and create a pxelinux configuration file. You may still want/have to add a few lines in /tftpboot/pxelinux.cfg/default (for example, for Tier2 installs).

Step 5: Make the release available in VAMOS

Fire up the GUI, select "vars" as the top object, go to CF_SL_release, choose the "values_host" tab, and add the new value to the available choices. Set it on some test host.

Step 6: test

Make sure this works and sets the right link: 

  /project/linux/SL/scripts/pxe <testhost>

Make sure this chooses the right directory: 

  cd /project/linux/SL/profiles
  ./CKS.pl <testhost>

Make sure SLU works correctly: 

  ssh <testhost>
  /project/linux/SL/scripts/SLU.pl yes please

Try an installation:

  • activ-ai <testhost>

  • then boot it

Try updating an existing installation:

  • set CF_SL_release for the host in VAMOS

  • sue.bootstrap

  • sue.update aaru

  • have a look into /var/log/yum.log, and check everything still works

Booting a rescue system

There are several ways to do this, including:

From CD1 of the distribution

Simply boot from CD1 of the distribution. At the boot prompt, type linux rescue.

Over the network with the unified installation CD

Just boot whatever entry you normally would for installation, but add the keyword rescue to the command line.

Building Kernel Packages

32-bit SL3

First install the kernel srpm (not kernel-source). Make your changes to the spec, add patches etc. 

rpmbuild --sign -ba kernel-2.4.spec

This will build

  • kernel.src.rpm
  • kernel-source.i386.rpm
  • kernel-doc.i386.rpm
  • kernel-BOOT.i386.rpm 

rpmbuild --sign -ba --target i686 kernel-2.4.spec

This will build

  • kernel.i686.rpm
  • kernel-smp.i686.rpm
  • kernel-hugeme.i686.rpm
  • kernel-unsupported.i686.rpm
  • kernel-smp-unsupport.i686.rpm
  • kernel-hugemem-unsupported.i686.rpm

Trying to turn off build of the hugemen kernel breaks the spec.

Additional modules for these are built as for any other SL kernel, with one exception of course:

Building the kernel-module-openafs packages

For ordinary SL kernels, this is done at FNAL or CERN, hence we needn't bother. But for our own kernels we have to do this ourselves.

/!\ This only works correctly on non-SMP build systems.

On an SMP system, the build will work but the modules will not.

Install the kernel-source, kernel, and kernel-smp RPMs on the build system, they're all needed.

Then run: 

PATH=/usr/kerberos/bin:$PATH rpmbuild --rebuild --sign --target i686 --define 'kernel 2.4.21...' openafs-...src.rpm

Building the NEW kernel-module-openafs packages

As of December 2006, there exists a unified SRPM for OpenAFS 1.4.2, which doesn't have the build problems described above, and works in exactly the same way on SL3, SL4, and SL5. It's named openafs.SLx, but will create packages named openafs with SL3, SL4, SL5 in the release number. The SRPM can (and should) be rebuilt without being root. The steps are the same on every platform:

First install the right kernel-source (SL3) or matching kernel[-smp|-largesmp|-xen|...]-devel package for the target kernel.

Then run: 

rpmbuild --rebuild --sign --target  i686 --define 'kernel 2.4.21...' --define 'build_modules 1' openafs.SLx-...src.rpm

There's always just one module built per invocation. Building on SMP systems is ok (and really fast).

Supported targets include i686, athlon (SL3 only), ia32e (SL3 only, must build on 64bit system), x86_64 (must build on 64bit system), ia64 (untested).

Supported kernel flavours include smp (SL3/4), hugemem (SL3/4), largesmp (SL4), xen (SL5), PAE (SL5).

References

http://www.redhat.com/docs/manuals/enterprise/RHEL-3-Manual/sysadmin-guide/ch-kickstart2.html

http://linux.duke.edu/projects/yum/

SL Operations Manual (last edited 2017-01-16 10:23:13 by TimmEssigke)