Page History

Note: When downloading software to install in these instructions, always

download the source code, and avoid binary installers.

  Binary installers tend

to make inaccurate assumptions about what libraries you have installed on your

system, as well as other problems.

Note: Java/JMX depends upon RPC/Portmap to manage its connections.  This means that iptable firewalls need to take this into account.  Port 8900-8999 should be set to only allow connections from localhost, and all high numbered ports should only accept connections from the local classful network (18.0.0.0/8 in the case of MIT).

# Setup local directory structure and users.

## Setup the www user.

{code

}
groupadd www
useradd -g www -d /home/www -m -r -s /bin/bash www

{code}
## Add the logs user.
{code}
groupadd logs
useradd -g logs -d /home/logs -m -r -s /bin/bash logs

{code}
##* Set the www user directory structure and copy the config files into place.
{code}
mkdir /home/www/weblib
mkdir /home/www/etc
mkdir /home/www/etc/krb
cp -a /home/root/weblib /home/www/weblib
scp -r root@trogdor.mit.edu:${REPOS_HOME}/System-configs/www-user/etc /home/www/etc
cp /root/map-dev-ws1-new-keytab /home/www/etc/krb/daemon.map-dev-ws1.krb5.keytab
chown -R www:www /home/www/etc
chmod a-w,o-rx,u+r /home/www/etc/krb/daemon.map-dev-ws1.krb5.keytab

{code}
##* Make certain that the library files are put into their correct locations.
{code}
scp -r root@trogdor.mit.edu:${REPOS_HOME}/System-configs/www-user/weblib /home/www/weblib/

{code}
## If needed (if this system is going to hold an Alfresco instance, or something similar), setup the repository user.

{code

}
groupadd repos
useradd -g repos -d /home/repos -m -r -s /bin/bash repos

{code}
## If needed, setup the database user.

{code

}
groupadd db
useradd -g db -d /home/db -m -r -s /bin/bash db

{code}
##* The logs user is intended to allow developers access to log and configuration files without granting root access.  Change the group ownership of any file that developers will need non-root access to to be owned by the logs user, and chmod those files to be group readable.
# Then, check the version of openssl already installed with the OS.
{code}
openssl version
{code}
## If the version is lower then 0.9.8a, or if it is not installed, download openssl 0.9.8a source from [http://www.openssl.org/source/]. follow the instruction in the INSTALL document, compile and install the binaries. the default location is /usr/local/ssl, but we need to use /home/www/ssl. If you want to change it, run config like this:

{code

}
cd /home/www/tmp
tar -xzvf /home/www/tmp/openssl-0.9.8a.tar.gz
cd openssl-0.9.8a
./config --prefix=/home/www/ssl --openssldir=/home/www/ssl
make
make install

{code}
# Set up certificates.  If using the local version of SSL, put them into /tmp/ssl, and copy them to the Apache conf directory during the config:

## Create the SSL directories.

{code

}
mkdir /home/www/ssl
mkdir /home/www/ssl/certs
mkdir /home/www/ssl/private
chown -R www:www /home/www/ssl

{code}
## get the mitca at [http://ca.mit.edu/mitClient.crt] and save it as /home/www/ssl/certs/mitClient.crt

## convert mitCA.crt to pem format:

{code

}
openssl x509 -in /home/www/ssl/certs/mitClient.crt -inform DER -outform \
    PEM -out /home/www/ssl/certs/mitCA.pem

{code}
## Generate rsa key
##* This simply generates some random stuff:
{code}
ps > /tmp/foo
ps -elf >> /tmp/foo
cd /home/www/ssl
openssl genrsa -rand /tmp/foo 1024 >/home/www/ssl/private/`hostname`-key.

pem
{code}
## Generate request for a certificate
{code}
cd /home/www/ssl/
openssl req -key /home/www/ssl/private/`hostname`-key.pem -new \
   >/home/www/ssl/certs/`hostname`-req.pem
{code}

\##*send the file /home/www/ssl/certs/`hostname`-req.pem to mitcert@mit.edu,

{code

}
Please be aware, the organization (O) is:
Massachusetts Institute of Technology
and the common name (CN) is the name of the server or
service, including the domain name (.mit.edu).  Also, some servers, such
as Thalia servers, can represent an entire subdomain.  These servers
will need certificates issued with a wildcard in the domain name, such
as \*.isda-thalia-1.mit.edu.

Remember, if the server is a Thalia server, it will need a wildcard
certificate and DNS record for *.`hostname`, and if it is doing any type of
authentication, it will need a joint client/server certificate to be able
to connect to the Shibboleth server (and have end users connect to it as
well).

{code}
## To generate a self signed temporary certificate, add the x509 and nodes options to the openssl command line.
{code}
cd /home/www/ssl
openssl req -key /home/www/ssl/private/`hostname`-key.pem -new \
     -x509 -nodes >/home/www/ssl/certs/`hostname`-temp-cert.pem

{code}
## When you receive a certificate from MIT Certificates, save it as /home/www/ssl/certs/`hostname`-cert.pem

## Commands to view cert information:

##* to look at a request:

{code

}
openssl req -in /home/www/ssl/certs/`hostname`-req.pem -text

{code}
##* to look at the private key:
{code}
openssl rsa -in /home/www/ssl/private/`hostname`-key.pem -text

{code}
##* to look at the server certificate:
{code}
openssl x509 -in /home/www/ssl/certs/`hostname`-cert.pem -text

{code}
# set up apache-ssl

## download Apache 2.2.4 from apache archive site at [http://archive.apache.org/dist/httpd/

]
## Unpack apache 2.2.4 (tar \-xzvf) and do "cd httpd-2.2.4"

{code

}
cd /home/www/tmp
tar -xzvf /home/www/tmp/httpd-2.2.4.tar.gz
cd httpd-2.2.

4
{code}
## compile apache following the instruction in the INSTALL file.  To enable the SSL, do the following:
{code}
./configure --prefix=/home/www/apache-2.2.4 --enable-ssl \
  --enable-modules="most mod_rewrite"
make
make install
ln -s /home/www/apache-2.2.4 /home/www/apache

{code}
# set up mod-jk

## download mod-jk 1.2.21 source (previous versions have a security hole that could allow a remote attacker to execute arbitary code) from [http://tomcat.apache.org/connectors-doc/].

{code

}
cd /home/www/tmp
tar -xzvf /home/www/tmp/tomcat-connectors-1.2.21-src.tar.gz
cd tomcat-connectors-1.2.21-src

{code}
## build and install binaries according to BUILD.txt. apxs is at /home/www/apache/bin/apxs. mod_jk.so will be put at /home/www/apache/modules

{code

}
cd native
./configure --with-apxs=/home/www/apache/bin/apxs --enable-ssl
make
make install

{code}
# install jdk 1.6 which is required by tomcat 5.5.25

## download jdk 1.6 binary at [http://java.sun.com/javase/downloads/index.jsp

]
## You may need to set the binary file to be executable:

{code

}
chmod u+x,u-w jdk-6-linux-i586.bin

{code}
## execute the binary installer as root.  If it produces a rpm file, use rpm \-ivh to install it.  If you downloaded the straight binary installer, move to a directory with installed software, such as /usr/local.  Also, you will need to page through a licensing agreement and type yes to accept it.

{code

}
cd /usr/local
/home/root/jdk-6-linux-i586.bin

{code}
##* or
{code}
./jdk-6-linux-i586-rpm.bin
rpm -ivh jdk-6-linux-i586

{code}
## create a file in /etc/profile.d named java_home.sh.

  It should contain a line exporting a variable pointing to the Java home directory.  Then make this file world executable.:

{code

}
cat > /etc/profile.d/java_home.sh
export JAVA_HOME=/usr/local/jdk1.6.0
^C
chmod a+xr,a-w /etc/profile.d/java_home.sh

{code}
# install the SASH Server.  NOTE:  These instructions are written assuming version 2.0.4-BETA of the SASH Server, but this version is not officially supported.  We are waiting on version 2.0.4 to become officially supported.

## If this is a server ops installed and managed RHEL 4 VM, request that they install the 32 bit NPTL threading libraries, and reboot the VM.  If this is a RHEL 5 VM, check the the threading libraries are 32 bit compatible.  If this is a physical server, this step can be skipped.

## download sash-server-2.0.4-BETA.tar.gz from: [https://portal.sourcelabs.com/?module=download] You will need to register and login to the web site to be able to download the SASH Server.  Also, copy the sashsrv.conf mkInstance, modify-instance-file.sh, monitor.war and mit-worker-consolidater.sh out of the ISDA repository on Trogdor and onto the machine.

{code

}
scp -r root@trogdor:/opt/software-repository-tmp/SASH_Server/mit-worker-consolidater.sh  \
  root@trogdor:/opt/software-repository-tmp/SASH_Server/mkInstance \
  root@trogdor:/opt/software-repository-tmp/SASH_Server/monitor.war \
  root@trogdor:/opt/software-repository-tmp/SASH_Server/sash-2.0-flattened-jars.zip \
  root@trogdor:/opt/software-repository-tmp/SASH_Server/sashctl \
  root@trogdor:/opt/software-repository-tmp/SASH_Server/sash_home.sh \
  root@trogdor:/opt/software-repository-tmp/SASH_Server/sash-server-2.0.4-BETA.tar.gz \
  root@trogdor:/opt/software-repository-tmp/SASH_Server/sashsrv.conf \
  root@trogdor:/opt/software-repository-tmp/SASH_Server/server.xml \
  root@trogdor:/opt/software-repository-tmp/SASH_Server/template \
  root@trogdor:/opt/software-repository-tmp/SASH_Server/web \
  root@trogdor:/opt/software-repository-tmp/SASH_Server/consolidator.conf  \
  root@trogdor:/opt/software-repository-tmp/common-init/apache_home.sh  \
  /home/root

{code}
## unzip and untar (gunzip , tar \-xvf) into your working directory, such as /home

{code

}
cd /home/www
tar -xzvf /home/root/sash-server-2.0.4-BETA.tar.gz
ln -s sash-server-2.0.4-BETA sash-server
cd sash-server-2.0.4-BETA

{code}
## Copy sash_home.sh and apache_home.sh into /etc/profile.d.

{code

}
cp /home/root/sash_home.sh /home/root/apache_home.sh /etc/profile.d
chmod a+rx /etc/profile.d/sash_home.sh /etc/profile.d/apache_home.sh
source /etc/profile.d/sash_home.sh
source /etc/profile.d/apache_home.sh

{code}
## Copy the custom sashctl file and the MIT worker consolidator file into the SASH server bin directory.
{code}
cp -a --reply=yes /home/root/template/* $SASHSRV_HOME/conf/template
cp --reply=yes /home/root/mkInstance $SASHSRV_HOME/bin
cp --reply=yes /home/root/sashctl $SASHSRV_HOME/bin
cp --reply=yes /home/root/mit-worker-consolidater.sh $SASHSRV_HOME/bin
cp --reply=yes /home/root/monitor.war $SASHSRV_HOME/conf/template/webapps
cp --reply=yes /home/root/consolidator.conf /home/root/sashsrv.conf $SASHSRV_HOME/conf
chmod ug+rx,a-w $SASHSRV_HOME/bin/mit-worker-consolidater.sh $SASHSRV_HOME/bin/mkInstance

{code}
## Copy the sash-2.0-flattened-jars.zip file onto the server and update the commons-io jar file.

{code

}
cd ${SASHSRV_HOME}/core/server/webapps/manager/WEB-INF/lib
unzip -j /home/root/sash-2.0-flattened-jars.zip  sash-2.0-flattened-jars/commons-io-1.3.1-1.jar

{code}
# Do the configuration:
## SASH Server part: cd into the tomcat home directory
{code}
cd $SASHSRV_HOME/conf/

template
{code}
### enter the conf directory and create a jk directory
{code}
cd conf
mkdir jk
cd jk

{code}
### copy the sashsrv.conf template into the template conf directory.

{code

}
cp /home/root/template/conf/sashsrv.conf /home/www/sash-server-2.0.4-BETA/conf/template/conf

{code}
### If this need HTTP connections to the Tomcat connectors, enable direct connections to the SASH Server, by removing the '<\!\--' and '-\->' from around the http port connector block in server.xml.  Also works for https:
{code}
<Connector port="%%HTTP_PORT%%"
        maxHttpHeaderSize="8192"
        maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
        enableLookups="false" redirectPort="8443" acceptCount="100"
        connectionTimeout="20000" disableUploadTimeout="true" />

{code}
### edit tomcat_users.xml, and add the following user definition just above the '</tomcat-users>' line:

{code

}
<role rolename="manager"/>
       <user username="tomcat" password="*****" roles="tomcat,manager"/>

{code}
###* be certain to change the password to be the password for the team the server is providing services to.  Check with your groups manager to see what the password standards for your group are.  Do not cut and paste this password into place, do not use "*****" as a password, etc.

### Update the www user.

{code

}
chown -R www:www /home/www

{code}
###* And edit $SASHSRV_HOME/bin/setvars, $SASHSRV_HOME/bin/sash-server.init, and $SASHSRV_HOME/bin/sash-server-CS2.init, and change the SASHSRV_USER.

{code

}
cd $SASHSRV_HOME/bin
sed -i s:SASHSRV_USER=\"sashsrv\":SASHSRV_USER=\"www\":g $SASHSRV_HOME/bin/setvars
sed -i s:SASHSRV_USER=\"sashsrv\":SASHSRV_USER=\"www\":g $SASHSRV_HOME/bin/sash-server.init
sed -i s:SASHSRV_USER=\"sashsrv\":SASHSRV_USER=\"www\":g $SASHSRV_HOME/bin/sash-server-CS2.init

{code}
### To progate the changes made to the config, delete the default server container, and create a new one.
{code}
cd $SASHSRV_HOME/servers
rm -rf default
cd $SASHSRV_HOME/bin
./mkInstance -N default
cd $SASHSRV_HOME/servers/default

{code}
###* Also create any other server containers that may be needed.  Some of the common ones are mitid, roles, moira, sis, and misc.
### Set up the additional directories and files needed for JMX and the standard ISDA/MAP configuration scripts.
{code}
cd /home/www
mkdir /home/www/etc
mkdir /home/www/etc/containers
mkdir /home/www/etc/krb
mkdir /home/www/etc/jmx
touch /home/www/etc/jmx/jmxremote.password
chmod a-wx,u+r,go-r /home/www/etc/jmx/jmxremote.password
chown -R www:www /home/www

{code}
### Run the java_home.sh script and start the SASH Server

{code

}
source /etc/profile.d/java_home.sh
$SASHSRV_HOME/bin/sashctl all start

{code}
### Tomcat creates a mod_jk.conf file in ./core/conf/auto/ directory the first time is runs.  Correct it to point to where mod_jk.so resides.

###* change
{code}
LoadModule jk_module "/home/www/apache/libexec/mod_jk.so"

{code}
###* to
{code}
LoadModule jk_module "/home/www/apache/modules/mod_jk.so"

{code}
## Apache side:
### edit /home/www/apache/conf/httpd.conf

###* edit the following directives:

{code

}
ServerRoot "/home/www/apache"          # change to apache home directory
User www                               # change from daemon
Group www                              # change from daemon
ServerAdmin map-support@mit.edu        # change to our group email address
Include conf/extra/httpd-vhosts.conf   # Uncomment
Include conf/extra/httpd-ssl.conf      # Uncomment

{code}
###* add to /home/www/apache/conf/httpd.conf, and the bottom of the other includes:

{code

}
# SASH Server/mod_jk includes
Include /home/www/sash-server-2.0.4-BETA/core/conf/auto/mod_jk.conf

<IfModule \!mod_rewrite.c>
    LoadModule rewrite_module "/home/www/apache/modules/mod_rewrite.so"
</IfModule>

{code}
### edit /home/www/apache/conf/extra/httpd-vhosts.conf to have ONLY one of the following VirtualHost blocks:

###* Thalia:

{code

}
NameVirtualHost *:80

<VirtualHost *:80>
     ServerName *.isda-thalia2.mit.edu

     RewriteEngine On

     RewriteCond %{HTTP_HOST}    !^isda-thalia2\.mit\.edu [NC]
     RewriteCond %{HTTP_HOST}    !^test\.isda-thalia2\.mit\.edu [NC]
     RewriteCond %{HTTP_HOST}    !^demo\.isda-thalia2\.mit\.edu [NC]
     RewriteCond %{HTTP_HOST}    !^hst\.isda-thalia2\.mit\.edu [NC]
     RewriteCond %{HTTP_HOST}    !^ap\.isda-thalia2\.mit\.edu [NC]
     RewriteRule ^/(.*)         http://isda-thalia2.mit.edu/$1 [L,R]
</VirtualHost>

{code}
###* Web Services - edit for correct server name:
{code}
<VirtualHost *:80>
     RewriteEngine On

     RewriteRule ^/(.*)         https://finniganfen.mit.edu/$1 [L,R]

</VirtualHost>

{code}
###* To prevent some web applications (such as WarehouseServices) from being redirected to https, add an escape clause between "RewriteEngine On" and the RewriteRule:
{code}
RewriteCond %{REQUEST_URI}       !/WarehouseService

{code}
### edit /home/www/apache/conf/extra/httpd-ssl.conf and alter the following directives:

{code

}
# points to directory for static html files
DocumentRoot "/home/www/apache/htdocs"
# the servername of the server
ServerName gybe.mit.edu:443
# the admins of this server
ServerAdmin map-support@mit.edu
# error log file
ErrorLog /home/www/apache/logs/error_log
# access log file
TransferLog /home/www/apache/logs/access_log
# public server certificate
SSLCertificateFile /home/www/ssl/certs/gybe.mit.edu.pem
# private server certificate
SSLCertificateKeyFile /home/www/ssl/private/https-key.pem
#certificate path
SSLCACertificatePath /home/www/ssl/certs
# certificate authority key
SSLCACertificateFile /home/www/ssl/certs/mitCA.pem

SSLVerifyClient require
SSLVerifyDepth 10

{code}
### add the following after the '<Directory "/home/www/apache/cgi-bin">' block in /home/www/apache/conf/extras/httpd-ssl.conf

{code

}
SSLOptions +StdEnvVars +ExportCertData

{code}
# copy the following files to the noted locations.  They should be in the ISDA software repository.
## MitIdService.jar moves to $SASHSRV_HOME/core/shared/lib

{code

}
cp /home/root/MitIdService.jar $SASHSRV_HOME/core/shared/lib

{code}
## rolesApplicationContext.xml  moves to $SASHSRV_HOME/core/shared/classes

{code

}
cp /home/root/rolesApplicationContext.xml $SASHSRV_HOME/core/shared/classes/

{code}
# install the web init script into /etc/init.d, and place starter links into the /etc/rc.d/ runlevel directories.  It is in the Software Repository on Trogdor.

## edit the variables in the top section of the web file to use the directories and binaries correct for this system

## be certain to check if apache is using a httpdctl or apachectl starter program, usually contained in /home/www/apache/bin, and set the apachectl variable accordingly

## set web to be executable

{code

}
chmod a+rx,a-w /etc/init.d/web

{code}
## link startweb and stopweb to the web program, from wherever it is located, and link start scripts in /etc/init.d:

{code

}
ln -s /etc/init.d/web /root/startweb
ln -s /etc/init.d/web /root/stopweb
ln -s /etc/init.d/web /etc/rc.d/rc1.d/K15web
ln -s /etc/init.d/web /etc/rc.d/rc2.d/K15web
ln -s /etc/init.d/web /etc/rc.d/rc3.d/K15web
ln -s /etc/init.d/web /etc/rc.d/rc4.d/K15web
ln -s /etc/init.d/web /etc/rc.d/rc5.d/K15web
ln -s /etc/init.d/web /etc/rc.d/rc6.d/K15web
ln -s /etc/init.d/web /etc/rc.d/rc2.d/S15web
ln -s /etc/init.d/web /etc/rc.d/rc3.d/S15web
ln -s /etc/init.d/web /etc/rc.d/rc4.d/S15web
ln -s /etc/init.d/web /etc/rc.d/rc5.d/S15web

{code}
# update paths in /etc/profile, by adding the following line in the path manipulation code block (you can find it by searching for /usr/local/sbin)

{code

}
pathmunge /usr/local/bin
        pathmunge /usr/kerberos/bin

{code}
# If this is an upgrade on a server that had previously had a tomcat on it, there are additional steps to move necessay files and code to the new directories.
## copy the webapps from the old deploy of tomcat to the new one.  Be certain to restart the server if it was running previously.
{code}
cd /home/www/sash-server-2.0.4-BETA/servers/$SERVER/webapps
cp -a geows\* mapws\* mitidws\* uaws\* testcert\* TestRemoteAlfresco\* \
     /home/www/sash-server-2.0.4-BETA/servers/$SERVER/webapps

{code}
##* to see the applications deployed in a container that are not part of the default tomcat install, get a listing of the directory:
{code}
ls -1 --hide=balancer --hide ROOT --hide=jsp-examples \
    --hide=servlets-examples --hide=tomcat-docs --hide=webdav

{code}
## Copy the libMitIdNativeClient.so into /home/www/weblib

{code

}
cp /root/libMitIdNativeClient.so /home/www/weblib

{code}
## Restart web services and tomcat
{code}
/etc/init.d/web restart

{code}
# Set up kerberos daemon principle and service ACLs for the system.
## Send email to accounts@mit.edu and ask for a kerberos principle in the format of daemon.\[hostname\].krb5.keytab.  When you receive it, install it in /home/www/etc/krb as daemon.\[hostname\].krb5.keytab.

## Send email to Mark Silis  or Jim Repa and request that the kerberos principle above (daemon/\[hostname\].mit.edu@ATHENA.MIT.EDU) be added to the access control list for the databases on Illinois that the web service will be using (often mitid or roles).

# If this is a Web Services system, install an AFS client, or check that a client is installed.

  Thalia systems do not need this.

## Check if an AFS client is installed by looking at the root directory.  If a client is installed, the afs directory will be near the top.

{code

}
ls -l /

{code}
## If an AFS client is not installed, download these packages from the MIT Athena or Thalia software lockers:
{code}
mit-openafs-setup-1.2-3.noarch.rpm
mit-krb-config-1.0-3.noarch.rpm
mit-openafs-package.patch

{code}
##* Unless the server is a virtual server.  If this is the case, email server ops to have the virtualized AFS kernel module installed.
## Use rpm to install these packages, installing the Kerberos configuration package first.
{code}
rpm -ivh mit-krb-config-1.0-3.noarch.rpm
rpm -ivh mit-openafs-setup-1.2-3.noarch.rpm

{code}
##* Please note: There are no paths in these commands.  Store them in a conveinent install directory, and cd to it first.

## Go to the OpenAFS client binary directory and execute the setup script. It will ask if you want the AFS client to be started at boot time.  Type yes.

{code

}
cd /opt/mit-openafs-setup/bin
./setup

{code}
##* If system is a SMP (multiprocessor) machine, apply the SMP patch before compiling.

{code

}
cd /opt/mit-openafs-setup/bin
patch < /root/mit-openafs-package.patch
./setup

{code}
# If this is a Web Services system, install version of moira that uses Kerberos 5.  Thalia systems do not need this.
## upload moira-rhel4-clients.tar.gz onto the server, and untar to /usr/local

{code

}
cd /usr/local
tar -xzvf /root/moira-rhel4-clients.tar.gz

{code}
# To start and stop tomcat and apache, use the initialization scripts in /etc/init.d.  Be certain to leave them running when you are finished.

#* starting
{code}
/etc/init.d/web start

{code}
#* stopping
{code}
/etc/init.d/web stop
{code}