Installation of the firehose pipeline is a multi-step process.  It has been tested on the Mac OSX platform (Leopard and Snow Leopard) and also some Linux implementations.  We believe that it is stable in these configurations, but some users have reported difficulties installing aspects of the xidl dependency tree.  Users are encouraged to examine (and update if appropriate!) the world-editable bugs wiki for tips on how to improve installation of xidl for use with FIRE.

System Requirements

  1. Unix installation and license for IDL.  Preferably v6.0 or higher.

Installation Procedure

NEW!

The xidl distribution is now available via anonymous svn, and several of the fixes outlined below for xidl are now included in its repository.  So, new users are advised to download in this manner rather than via the tarball.

  1. Download and install the XIDL package of subroutines, provided by J.X. Prochaska.  Documentation on installation procedures is provided on the linked website.  XIDL is now distributed via anonymous SVN.  The XIDL package depends in turn on the SDSS idlspec2d and idlutils packages, so you'll need to install those as well.  Note that the instructions on the princeton website about installation are helpful but very out of date.  It is probably advisable to get the idlspec2d code from the svn repository at sdss3.org rather than the Princeton tarball.  If you are having trouble installing xidl, especially compiling the externally linked libraries with "evilmake," please refer to the BUGS page on this wiki for help on different operating systems. 
  2. There are 3 minor changes you'll need to make to xidl, to add FIRE as an instrument and also to plug a known memory leak. Please refer to this page for detailed instructions.  (The previous sentences are obsolete for users obtaining xidl via svn) 
  3. Download the latest release of the firehose code base via anonymous svn as described here.  This usually goes into the directory pointed to by the environment variable $IDL_LOCAL.
  4. Set a new environment variable $FIRE_DIR in your shell (e.g. via a .cshrc or similar configuration file) to point to the directory where you unpacked the FIREHOSE code.
  5. Start up an idl session.  At the command prompt, type " which, 'x_specplot' ".  If IDL returns the path of this procedure properly, your xidl environment variables are set up correctly.
  6. From the IDL command prompt, type "firehose."  This should bring up a GUI used for the reduction procedure and indicates that your paths are set up correctly.

Summary of Installation Commands

unix> cd $IDL_LOCAL

unix> svn export http://www.sdss3.org/svn/repo/idlutils/tags/v5_4_20 idlutils [Downloads idlutils]

unix> svn export http://www.sdss3.org/svn/repo/idlspec2d/tags/v5_4_30 idlspec2d [Downloads idlspec2d]

   [Follow instructions on SDSS3 website for compiling idlutils and idlspec2d]

   [NOTE: some FIRE users have reported problems compiling idlutils on 64 bit machines.  See the top post of the known bugs thread for a workaround]

unix> svn co https://svn.ucolick.org/xidl/trunk xidl  [NOTE: Downloads xidl]

unix> svn co svn://pele.mit.edu//FIRE FIRE [Downloads FIRE]

unix> setenv FIRE_DIR $IDL_LOCAL/FIREHOSE/   [NOTE: this should probably be put in your .cshrc or .bashrc file]

unix> idl
IDL> which, "x_specplot"
% Compiled module: WHICH.
% Compiled module: STRSPLIT.
Module X_SPECPLOT Not Compiled.

Other Files Containing Module X_SPECPLOT in IDL !path:
/Users/Rob/idl/........./x_specplot.pro

IDL> firehose

<GUI appears if installed correctly>


  • No labels

13 Comments

  1. I hit an error during the extraction step because XIDL wasn't properly compiled. I fixed this problem by following the instructions for installing XIDL found here: http://www.ucolick.org/~xavier/IDL/xidl_install.html.

  2. Unknown User (jane.rigby_1@touchstonenetwork.net)

    Deleted.  Long story short, make sure you use absolute paths in your XIDL_ENV: 

    XIDL_DIR=/Users/jrrigby1/Idl/xidl   NOT  XIDL_DIR=~/Idl/xidl

    XIDL_DIR=/Idl/xidXIDL_DIR=/Idl/xidl

  3. Unknown User (k.allers_1@touchstonenetwork.net)

    Just had an xidl install problem.  When running bin/evilmake I ran into the following error:

    -------------

    ld -bundle /usr/lib/bundle1.o -L/usr/lib  -L/usr/lib/gcc/powerpc-apple-darwin8/4.0.0 -lc -lgcc -flat_namespace -undefined suppress -o /Users/kna003/Work/IDL/xidl//lib/libxmath.dylib arclincorr.o arrmedsigclip.o arrmedsclipwgt.o arrmedsigmask.o rectify.o vector_median.o rebin2dspec.o invert_arc.o x_gsmooth.o nrecipes.o nrutil.o
    ld: library not found for -lgcc
    make[1]: *** [/Users/kna003/Work/IDL/xidl//lib/libxmath.dylib] Error 1
    ld -bundle /usr/lib/bundle1.o -L/usr/lib  -L/usr/lib/gcc/powerpc-apple-darwin8/4.0.0 -lc -lgcc -flat_namespace -undefined suppress -o /Users/kna003/Work/IDL/xidl//lib/libxmath.dylib arclincorr.o arrmedsigclip.o arrmedsclipwgt.o arrmedsigmask.o rectify.o vector_median.o rebin2dspec.o invert_arc.o x_gsmooth.o nrecipes.o nrutil.o

    ld: library not found for -lgcc

    make[1]: *** [/Users/kna003/Work/IDL/xidl//lib/libxmath.dylib] Error 1

    -------------

    I tracked the error down to how evilmake determine the idl arch.  Because my .idlstartup was being called (includes initialize and paths etc.), it was reading "executed .idlstartup" as the architecture.  Easily remedied by commenting out the line defining IDL_STARTUP in my .bashrc.

  4. Unknown User (phil.massey_1@touchstonenetwork.net)

    I had a problem compiling the idlutils on both a Macbook Air running Yosemite using HPC's gfortran 5.0.  The error I get is:

    ld -bundle /usr/lib/bundle1.o -L/usr/lib -lc -lcc_dynamic -flat_namespace -undefined suppress -o /users/massey/idlstuff/xidl/lib/libdimage.dylib idl_dmedsmooth.o dmedsmooth.o dselip.o dfind.o idl_dsmooth.o dsmooth.o dsigma.o idl_dobjects_multi.o dobjects_multi.o 

    ld: library not found for -lcc_dynamic

    make[2]: *** [/users/massey/idlstuff/xidl/lib/libdimage.dylib] Error 1

     

     

    I removed the idlutils and idlspec2d subroutines and reloaded them, now taking the most recent versions:

    unix> svn export http://www.sdss3.org/svn/repo/idlutils/tags/v5_5_17 idlutils
     
    unix> svn export http://www.sdss3.org/svn/repo/idlspec2d/tags/v5_7_6 idlspec2d
    evilmake then worked properly.  How well these versions play with the current firehose is yet to be seen.
     
  5. Unknown User (phil.massey_1@touchstonenetwork.net)

    There's a small inconsistency in the instructions which caused a problem when I blindly followed them:

    unix> svn co svn://pele.mit.edu//FIRE FIRE [Downloads FIRE]
     
    unix> setenv FIRE_DIR $IDL_LOCAL/FIREHOSE/
    If the first command is used, then the environmental variable needs to be set to $IDL_LOCAL/FIRE, not /FIREHOSE.
  6. Unknown User (chakrabarti_1@touchstonenetwork.net)

    I'm having trouble installing xidl.  I see that idlutils, xidl, idlspec2d each have a different makefile (evilmake).  Should

     they all be the same?  I did have to make a change to the evilmake in /bin/idlutils 

    to get it to compile (g77 is no longer available on macports so I forced it to use 

    macports).

     

    Does anyone know if these should be the same or should we use the different evilmake's?

    1. I think when I last compiled I used gfortran in place of g77, but that was some time back.  I can say however that you should use the evilmake included with idlutils to make both that package and idlspec2d.  Then, use the version of evilmake included in the xidl distribution to make that package.  Usually a make clean and make install together are appropriate.  

  7. Unknown User (alexji@mit.edu)

    I was unable to find xidl on the SVN site, but it is also available here: https://github.com/profxj/xidl

  8. Unknown User (sdewit_1@touchstonenetwork.net)

    For me the svn export of idlutils and idlspec2d and the link to xidl did not work! I dont know if this is "as from date x" or just for me in particular, but I'll be posting the new links anyway:

    The correct commands for me were:

    svn export http://svn.sdss.org/public/repo/sdss/idlutils/tags/v5_4_20 idlutils    #for idlutils v5_4_20

    and

    svn export http://svn.sdss.org/public/repo/eboss/idlspec2d/tags/v5_4_30 idlspec2d      #for idlspec2d v5_4_30

    and for xidl you could clone this github page (if git is not installed: sudo apt install git):

    git clone https://github.com/profxj/xidl.git .



  9. Unknown User (sdewit_1@touchstonenetwork.net)

    Futhermore I want to add, that for the compilation, you should add these lines to your .bashrc. To edit, use gedit ~/.bashrc. After adding the lines, use . ~/.bashrc to activate them. I don't know if it's necessary to also add these lines to ~/.bash_profile. After this step, evilmake all should work when triggered from the $IDLUTILS_DIR. Use cd $IDLUTILS_DIR to check if your paths are constructed correctly.

    Necessary lines:

    #for IDLUTILS

    export IDL_PATH=+/usr/local/itt:$IDL_PATH                   # Should NOT be the usr directory in your ~ directory, so do: cd /usr  to get there. Just add the path to where idl is stored here.

    export IDLUTILS_DIR=/somewhere/lib/idlutils              # somewhere = anywhere you like. Just make a directory in your ~ directory called 'lib' (as an example) and store idlutils and idlspec2d there!

    export PATH=$IDLUTILS_DIR/bin:$PATH                      

    export IDL_PATH=+$IDLUTILS_DIR/goddard/pro:$IDL_PATH

    export IDL_PATH=+$IDLUTILS_DIR/pro:$IDL_PATH

    export IDL_DIR=/usr/local/itt/idl706                               # MUST be the directory in which the external directory is located. This is necessary to find the export.h file!

     

    #for IDLSPEC

    export IDLSPEC2D_DIR=/somewhere/lib/idlspec2d

    export PATH=$IDLSPEC2D_DIR/:$PATH

    export IDL_PATH=+$IDLSPEC2D_DIR/:$IDL_PATH

     

    #for FIREHOSE

    export FIRE_DIR=$IDL_DIR/FIRE/

  10. Unknown User (rbeaton_1@touchstonenetwork.net)

    A note that I followed (basically) the instructions above to get the software – not sure what is wrong with the instructions on sdss.org about the SVN.

    My additions to my .cshrc file look like this – I already had a fair bit of IDL stuff set up and I use a different shell – so I thought this might be useful:

    setenv IDLUTILS_DIR "/Users/rachaelbeaton/IDL_TOOL/idlutils/"

    setenv PATH "$IDLUTILS_DIR/bin:$PATH"

    setenv IDL_PATH "$IDLUTILS_DIR/pro:$IDL_PATH"

     --

    setenv IDLSPEC2D_DIR "/Users/rachaelbeaton/IDL_TOOL/idlspec2d"

    setenv PATH "$IDLSPEC2D_DIR/:$PATH"

    setenv IDL_PATH "$IDLSPEC2D_DIR/:$IDL_PATH"

    -- 

    setenv FIRE_DIR "/Users/rachaelbeaton/IDL_TOOL/FIRE/" 

    setenv IDL_DIR "/Applications/exelis/idl85/bin/idl"


    Then i used these directions (http://www.ucolick.org/~xavier/IDL/xidl_install.log) – 

    2) IDLUTILS 
    
    cd idlutils
    bin/evilmake clean   ; Won't work if you didn't set IDLUTILS_DIR
    bin/evilmake 
    
    
    3) IDLSPEC2D
    
    cd idlspec2d
    $IDLUTILS_DIR/bin/evilmake clean
    $IDLUTILS_DIR/bin/evilmake 
    
    
    4) XIDL
    
    cd xidl
    bin/evilmake clean
    bin/evilmake 

    And lastly, the instruction from above:

    IDL> which, "x_specplot"
    % Compiled module: WHICH.

    /<IDL_TOOL PATH>/xidl/Spec/General/x_specplot.pro

     
    IDL> firehose

    which brought up the GUI. 

  11. Unknown User (phil.massey_1@touchstonenetwork.net)

    On Mac OSX with relatively recent X11, there's some missing library stuff.  If you go to startup FIREHOSE and get the following:

    Error: attempt to add non-widget child "dsm" to parent "idl" which supports only widgets

    exit IDL and do the following (this required some googling!):

    sudo mv /opt/X11/lib/libXt.6.dylib{,.bak}

    sudo cp /opt/X11/lib{/flat_namespace,}/libXt.6.dylib


    This has made everything work again.



  12. Unknown User (phil.massey_1@touchstonenetwork.net)

    Has anyone gotten the FIRE pipeline successfully installed under the Catalina OS on a Mac?  As you now, apple dropped support for 32-bit applications.  As far as I can tell, this is a complete show-stopper.  When I evilmake idlspec2 it no longer makes lib/libspec2d.so, only lib/libspec2d.dylib.   When I go to run a trace it attempts to load libspec2d.so, and so fails. I'm sure this won't be the only issue.  Any guidance would be appreciated.   Thanks!