WebMO Home Page
   Features
   Pro Features
   Enterprise Features
   Compare Features
   System Requirements
   Screen Shots
   Working Demo
   Support & Tutorials
   Support Forum
   Testimonials
   Revision History
   Mailing List
   Educational Resources
   Workshops
   License Info
   Pricing
   Buy WebMO Pro/Ent
   Obtain Free License
   Download
WebMO - Computational chemistry on the WWW
Recent news

WebMO 18.0 is now available for free download!

WebMO 18.0 Pro and Enterprise have a variety of additional features and is available for purchase.

The WebMO app is now available for iOS and Android.

January 23, 2018
Creating a Redhat Apt Repository

Although many publically available apt repositories exist, creating one's own repository can result in better access and more control over the repository.

Overview

The basic steps to creating a repository are first to create a mirror of the Redhat distribution and then to create an apt repository from the mirror. After the repository is created, it should be made available through a server so that clients can access it. Finally, the repository should be kept up-to-date by regularly refreshing the mirror and regenerating the apt index files. These tasks involve precisely specifying lengthy directory locations, so they are best accomplished by scripts in order to eliminate typographical errors.

Create a Redhat mirror

There are several means to mirror a directory structure on another server, including lwp-rget, lftp, and rsynch. Some factors to consider when creating a mirror are

  • Mirror source: The redhat server itself, ftp://ftp.redhat.com, is often unavailable due to too many connections. Instead, use one of the secondary mirrors listed at http://www.redhat.com/download/mirror.html.
  • Directory structure: It is important to preserve the original directory structure of the Redhat distribution. Note that all of the original distribution rpms (i386, i686, noarch) are all combined in a single directory, e.g. /pub/redhat/linux/9/en/os/i386/RedHat/RPMS/, whereas the updated rpms are separated by architecture, e.g., /pub/redhat/linux/updates/9/en/os/[athlon|i386|i586|i686|noarch].
  • Since each distribution can be over 1GB, only mirror the distributions that you wish to serve. Also, if source files are not necessary, then consider mirroring only RPM's, and not SRPM's.
  • Create your local mirror directory on a partition on which you create the mirror is large enough to accomodate all the distributions and updates.
A script that uses lftp to create a mirror follows. It would intially be invoked as 'make-redhat-mirror full 9' and subsequently invoked as 'make-redhat-mirror quick 9'. Of course, modify the remote and local mirror locations appropriately, and use the desired redhat release, e.g., '7.3' or '8.0' in place of '9' as desired.

#!/bin/sh

# make-redhat-mirror
# create local subdirectory structure and download a mirror

# absolute directory for local redhat mirror
LOCALMIRRORDIR=/home/mirror/redhat/pub
# remote mirror location
REMOTEMIRROR=http://mirror.phy.bnl.gov
# remote mirror directory
REMOTEMIRRORDIR=redhat-pub

if [ $# -ne 2 -o \( "$1" != "full" -a "$1" != "all" -a "$1" != "quick" -a "$1" != "test" -a "$1" != "clean" \) ]; then
        echo "Usage: $0  "
        echo "   is one of:"
        echo "        \"full\": create mirror of os and updates"
        echo "        \"quick\": update mirror of updates"
        echo "        \"test\": output directory locations; does nothing"
        echo "   gives the Redhat version in canonical form"
        echo "        (eg: \"8.0\", \"9\")"
        exit 1
fi

MODE=$1
VERSION=$2

if [ "$MODE" = "test" ]; then
        echo "LOCAL OS DIR:"
        echo "$LOCALMIRRORDIR/redhat/linux/$VERSION/en/os/i386"
        echo "LOCAL UPDATES DIR:"
        echo "$LOCALMIRRORDIR/redhat/linux/updates/$VERSION/en/os"
        echo "REMOTE OS MIRROR SITE:"
        echo "$REMOTEMIRROR/$REMOTEMIRRORDIR/redhat/linux/$VERSION/en/os/i386"
        echo "REMOTE UPDATES MIRROR SITE:"
        echo "$REMOTEMIRROR/$REMOTEMIRRORDIR/redhat/linux/updates/$VERSION/en/os"
        exit 0
fi


# create local directory structure
if [ "$MODE" = "full" ]; then
mkdir -p $LOCALMIRRORDIR/redhat/linux/$VERSION/en/os/i386
mkdir -p $LOCALMIRRORDIR/redhat/linux/updates/$VERSION/en/os
fi

# mirror operating system
if [ "$MODE" = "full" ]; then
cd $LOCALMIRRORDIR/redhat/linux/$VERSION/en/os/i386
lftp -e "open $REMOTEMIRROR/$REMOTEMIRRORDIR/redhat/linux/$VERSION/en/os/i386 && \
mirror --delete --verbose -X *.src.rpm && \
quit"
fi

# mirror updates
cd $LOCALMIRRORDIR/redhat/linux/updates/$VERSION/en/os
lftp -e "open $REMOTEMIRROR/$REMOTEMIRRORDIR/redhat/linux/updates/$VERSION/en/os && \
mirror --delete --verbose -X *.src.rpm && \
quit"
Download and install apt rpm

In order to create an apt repository, the apt rpm must be installed on the server in order to access the indexing programs (genbasedir, genpkglist, gensrclist). Download the version of apt appropriate for your architecture

apt-0.5.4cnc7 for Redhat 6.2
apt-0.5.5cnc6 for Redhat 7.0
apt-0.5.4cnc9 for Redhat 7.1
apt-0.3.19cnc55 for Redhat 7.2 (this is an outdated version; try downloading download 7.3 instead)
apt-0.5.5cnc5 for Redhat 7.3
apt-0.5.5cnc6 for Redhat 8.0
apt-0.5.5cnc6 for Redhat 9

and install the rpm.

Create apt repository

The two steps to creating an apt repository are to provide the rpms in an organized directory structure and then to index the rpms.

The problem with providing the operating system and update rpms is that they are organized differently in the redhat distribution. The originally distributed operating system is distributed in a single directory with i386, i686, and noarch rpm's, whereas the updates are distributed in separate directories for different architectures. In contrast, the preferred organization for apt repositories is for both the operating system and the updates to each be organized into a single directory for each distribution. The solution is to use links, so that the operating system directory becomes the "os" apt directory, and the various redhat update rpms are combined into a single "update" apt directory. Finally, an "extras" apt directory can be created for third party rpm's.

After an appropriate apt repository structure is created, the rpm's (or more accurately, linked rpm's) must be indexed to create the apt database. First, a "base" directory is made with in the the apt repository. Then release files are written to this directory. Finally, the genbasedir tool is used to index the rpms and create the apt database.

A script to create the apt repository and index it follows. It would intially be invoked as 'make-apt full 9' and subsequently invoked as 'make-apt quick 9'. Of course, modify the local mirror and apt repository locations appropriately, and use the desired redhat release, e.g., '7.3' or '8.0' in place of '9' as desired.

#! /bin/sh

# make-apt
# Create apt databases from a redhat mirror

# Based on the following scripts and information:
# http://freshrpms.net/apt/server/
# ftp://apt-rpm.tuxfamily.org/pub/ftp.tuxfamily.org/apt-rpm/scripts/
# http://mirror.phy.bnl.gov/apt-rpm/scripts/makeapt.sh

# Absolute directory for top of redhat mirror
MIRRORDIR=/home/mirror/redhat/pub
# Absolute directory for top of apt repository
APTDIR=/var/www/html/apt
# URL of repository
ORIGINURL=myserver.domain.edu


# The idea of full/all/quick is:
#       * the first time you create the apt repository, choose "full".
#       * each night (when updates are downloaded), relaunch with "quick"
#       * if extra packages are put in 'extra' repository, relaunch with "quick"
#       * if the main os packages change (should not), use "all"

if [ $# -ne 2 -o \( "$1" != "full" -a "$1" != "all" -a "$1" != "quick" -a "$1" != "test" -a "$1" != "clean" \) ]; then
        echo "Usage: $0  "
        echo "   is one of:"
        echo "        \"full\": full apt repository creation and index generation"
        echo "        \"all\": no apt repository creation, but full indexing"
        echo "        \"quick\": no apt repository creation, only updates & extra indexing"
        echo "        \"test\": output directory locations; does nothing"
        echo "   gives the Redhat version in canonical form"
        echo "        (eg: \"8.0\", \"9\")"
        exit 1
fi

MODE=$1
VERSION=$2

if [ "$MODE" = "test" ]; then
        echo "APTDIR:"
        echo "$APTDIR/redhat/${VERSION}/en/i386"
        echo "OS MIRRORDIR:"
        echo "$MIRRORDIR/redhat/linux/${VERSION}/en/os/i386/RedHat/RPMS"
        echo "OS RPM:"
        find $MIRRORDIR/redhat/linux/${VERSION}/en/os/i386/RedHat/RPMS -maxdepth 1 -type f ! -name "*.src.rpm" -name "*.rpm" | head -1
        echo "UPDATES MIRRORDIR:"
        echo "$MIRRORDIR/redhat/linux/updates/${VERSION}/en/os/"
        echo "UPDATE RPM:"
        find $MIRRORDIR/redhat/linux/updates/${VERSION}/en/os/ -type f ! -name "*.src.rpm" -name "*.rpm" | head -1
        exit 0
fi

if [ "$MODE" = "clean" ]; then
# Caution: Wipes out all extra rpms's; therefore, commented out and undocumented!!!
#rm -rf $APTDIR/redhat/${VERSION}
exit 0
fi 

if [ "$MODE" = "full" ]; then
# Create directory
mkdir -p $APTDIR/redhat/${VERSION}/en/i386/base
# Create the main release file
cat >$APTDIR/redhat/${VERSION}/en/i386/base/release <<EOF
Origin: $ORIGINURL
Label: Red Hat $VERSION
Suite: Red Hat $VERSION
Architectures: i386
Components: os updates extra
Description: apt repository of Red Hat $VERSION os + updates + extra RPM's
EOF
fi

###########################################################################
#### REDHAT I386 OS
###########################################################################

if [ "$MODE" = "full" ]; then
# Create the release file
cat >$APTDIR/redhat/${VERSION}/en/i386/base/release.os <<EOF
Archive: stable
Component: os
Version: $VERSION
Origin: $ORIGINURL
Label: Red Hat $VERSION
Architecture: i386
EOF
fi

if [ "$MODE" = "full" -o "$MODE" = "all" ]; then
# Remove old symlink to rpm directory
rm -f $APTDIR/redhat/${VERSION}/en/i386/RPMS.os
# Create new symlink to rpm directory
ln -s $MIRRORDIR/redhat/linux/${VERSION}/en/os/i386/RedHat/RPMS $APTDIR/redhat/${VERSION}/en/i386/RPMS.os

# Generate apt index
genbasedir --flat --bloat $APTDIR/redhat/${VERSION}/en/i386 os

# Generate yum index
#echo "yumifying $APTDIR/redhat/${VERSION}/en/i386/RPMS.os"
#rm -rf $APTDIR/redhat/${VERSION}/en/i386/RPMS.os/headers
#yum-arch -lq $APTDIR/redhat/${VERSION}/en/i386/RPMS.os
fi

###########################################################################
#### REDHAT UPDATES
###########################################################################

if [ "$MODE" = "full" ]; then
# Create directory
mkdir -p $APTDIR/redhat/${VERSION}/en/i386/RPMS.updates
# Create release file
cat >$APTDIR/redhat/${VERSION}/en/i386/base/release.updates <<EOF
Archive: stable
Component: updates
Version: $VERSION
Origin: $ORIGINURL
Label: Red Hat $VERSION
Architecture: i386
EOF
fi

# Remove old symlinks to rpms
rm -f $APTDIR/redhat/${VERSION}/en/i386/RPMS.updates/*.rpm
# Create new symlinks for all updates (in multiple directories)
cd $APTDIR/redhat/${VERSION}/en/i386/RPMS.updates/
find $MIRRORDIR/redhat/linux/updates/${VERSION}/en/os/ -type f ! -name "*.src.rpm" -name "*.rpm" -exec ln -sf {} \;

# Regenerate apt index
genbasedir --flat --bloat $APTDIR/redhat/${VERSION}/en/i386 updates

# Regenerate yum headers
#echo "yumifying $APTDIR/redhat/${VERSION}/en/i386/RPMS.updates"
#rm -rf $APTDIR/redhat/${VERSION}/en/i386/RPMS.updates/headers
#yum-arch -lq $APTDIR/redhat/${VERSION}/en/i386/RPMS.updates

###########################################################################
#### REDHAT EXTRA PACKAGES
###########################################################################

if [ "$MODE" = "full" ]; then
# Create directory
mkdir -p $APTDIR/redhat/${VERSION}/en/i386/RPMS.extra
# Create release file
cat >$APTDIR/redhat/${VERSION}/en/i386/base/release.extra <<EOF
Archive: stable
Component: extra
Version: $VERSION
Origin: $ORIGINURL
Label: Red Hat $VERSION
Architecture: i386
EOF
fi

# Regenerate apt index
genbasedir --flat --bloat $APTDIR/redhat/${VERSION}/en/i386 extra

# Regenerate yum headers
#echo "yumifying $APTDIR/redhat/${VERSION}/en/i386/RPMS.extra"
#rm -rf $APTDIR/redhat/${VERSION}/en/i386/RPMS.extra/headers
#yum-arch -lq $APTDIR/redhat/${VERSION}/en/i386/RPMS.extra

###########################################################################
########### REMAKE HASH
###########################################################################

genbasedir --hashonly $APTDIR/redhat/${VERSION}/en/i386 os updates extra
Serve up apt repository

Once the repository is constructed, it must be made available by http, ftp, or rsync. If the repository is made in the root document tree of your web server, e.g., /var/www/html/apt, the following entry should be added to /etc/httpd/conf/httpd.conf:

<Directory /var/www/html/apt>
Options +Indexes
</Directory>

in order to allow the directories to be listed. Reload the httpd configuration file with

# /sbin/service httpd reload

The apt repsotiory can now be accessed at http://myserver.domain.edu/apt

Update apt repository

The apt repository should be regularly updated via a daily or weekly cron job. The two commands that must be executed are 'make-redhat-mirror quick 9' and 'make-apt quick 9', the desired redhat release, e.g., '7.3' or '8.0' in place of '9' as desired. This pair of commands needs to executed for each Redhat version being maintained in the repository.

Add to apt repository

The os directory contains the original redhat distribution rpms, and the updates directory contains Redhat update rpms. The extras directory is intended for one to place additional rpms.

Instruct clients to use repository

Clients can use the repository by placing the following entry into their /etc/apt/sources.list file

rpm http://myserver.domain.edu/apt redhat/9/en/i386 os updates extra

The version number "9" should be replaced with "7.3", "8.0", etc. as appropriate.

Clients can then retreive the apt database with
# apt-get update
check their dependeny tree structure with
# apt-get check
install packages with
# apt-get install
and perform a system upgrade with
# apt-get upgrade

Cautious clients may prefer first to simulate an upgrade with
# apt-get -s upgrade
and to downloadthe rpms without installing them
# apt-get --download-only upgrade

The downloaded rpms are cached in /var/cache/apt/archives. To verify whether the correct architectures of the rpms were downloaded, one can check for the presence of i686 installed rpms with
# rpm -qa -qf '%{NAME}-->%{ARCH}\n' | grep 686
and compare these the cached rpms
#ls -alF /var/cache/apt/archives/*686*

References

http://freshrpms.net/apt/server/
ftp://apt-rpm.tuxfamily.org/pub/ftp.tuxfamily.org/apt-rpm/scripts/apt-rpm.html
http://mirror.phy.bnl.gov/apt-rpm/README.html
http://apt4rpm.sourceforge.net/
https://moin.conectiva.com.br/AptRpm
http://www.dragonsdawn.net/~gordon/red-hat-apt-repository-howto/
http://www.jukie.net/~bart/apt-rpm/

Copyright © 2003, WebMO, LLC, all rights reserved.