version 27 AUG 2014 by Jen Winters and John Subasavage

version 17 MAY 2016 by Todd Henry

You can find here details about how to:

 1. Move data around at CTIO from the Torrent
machine to the ctio36 storage machine. 

 2. Retrieve data from the 0.9m at CTIO and
to do basic bias subtraction and

*****   MOVE DATA FROM new-ctioa4 (Torrent computer) TO ctio36   *****


1. Type "huygens" to ensure that you are on the machine huygens, which
   is the only ATL computer cleared for entry to computers at CTIO.

2. Type "ssh -l observer" and use the
   new-ctioa4 password to get into the Torrent computer (known as
   new-ctioa4).  observer@new-ctioa4 will be displayed on the screen.

3. You will enter in the /home/observer directory.  The data you want
   are typically found in the /home/data directory, so cd to there,
   find your data, and note the path, typically /home/data/YYYYMMDD-09

4. leave new-ctioa4 by typing exit --- you are now back in ATL

5. Type "ssh" and use the v12 password to get
   in.  v12% will be displayed on the screen.

AT CTIO ON ctio36:

6. Create a directory named /u363/v12/ctiopi/YYYYMMDD for the new
   data, and cd into it.
7. ftp over to the new-ctioa4 machine by 
   /usr/bin/sftp -o User=observer
   and use the new-ctioa4 password to get in.

8. Move to where the new data are and copy the files to ctio36 with
   mget *.fits .

*****                 SLURP DATA FROM CTIO TO ATL                *****

1. cd /nfs/recons2/incoming.0.9m

2. mkdir YYYYMMDD corresponding to the observation date and cd into
   that new directory

3. Type "huygens" to ensure that you are on the machine huygens, which
   is the only one cleared for entry to computers at CTIO.

4. Type "ssh" and use the v12 password to get
   in.  v12% will be displayed on the screen.

5. The data you want are typically found in the /u363/v12/ctiopi/
   directory.  Type "ls" to see where you are and verify that the data
   you are looking for are there.  If you don't find it there check


6. cd into the directory for the night you want by typing cd YYYYMMDD

7. Type "ls" to see what's in that directory; there should be files
   ending in ".fits" and two files of headers.

8. To see how many entries you will slurp, do a word count by typing
   ls | wc

9. Type "rsync -avz YYYY*"
   i.e. "rsync -avz 2016*"

   The * will ensure that the headers get transferred, as well as the
   .fits data files.

   When asked, use your ATL password.

10. When done slurping, type "exit" to come back to ATL.

11. Type "ls -l" to get directory listing.  Check the file sizes to
    make sure they are all the same.

12. Type "ls * | wc" to make sure that all the files were successfully

*****                 PROCESS THE DATA IN ATL                    *****

A) cd into the file that you created for the slurping process
   (i.e. /nfs/recons3/SMARTS/20030327) type in "xgterm -sb &". In
   xgterm, and type "cl". Then load the 'redpi' package. Log on to the
   recons machine, as this will speed up the process significantly.

   A1) Sometimes the observer will name the .fits files with *.CTIOPI.*
       instead of *.09.*. If this is the case, you will need to rename
       the image files.

   A2) To do this, type 'ls *.fits >> old'. This will create a file
       called 'old' that contains all the slurped image files. copy the
       'old' file into another file called 'new'. Then edit the 'new'
       file by replacing CTIOPI with 09.

   A3) Use 'imrename' in iraf to change all image files names: 'imrename
       @old @new'.

B) type "fixnameall"; it will ask you for the root name; the root name is
   the string of eight characters that is the date you are working on
   (ie, 20070303).  This step will generate a list; wait until it's
   done to go to next step.

C) type "gofall"  

D) type "procall". You will then be asked to enter the bias section; trim
   section, ccd readout noise, and ccd gain in turn; the numbers
   should be provided. (The numbers in each column are identical all
   the way down).
   Make sure that the numbers given for you to enter match the ones in
   the column.  If they do, hit enter; otherwise, enter the numbers
   manually in the form specified (including the brackets), and then
   hit enter.
   Then it will say "now reducing V band star images", "now reducing R
   band star images", "now reducing I band star images". Once the
   whole process is finished, you will see a "cl>" appear; this lets
   you know that you're done with this set of data.

E) "ccdlist *.fits" to make sure that the filter is correct and

*****                SORT DATA INTO DIRECTORIES                  *****

PHOTOMETRY DIRECTORIES  (found at: /nfs/recons3/phot.0.9m/)

A) If there is a photometry night, please make sure that you create a
   directory for that night and then COPY all frames into that
   directory BEFORE you sort the frames into the parallax directories.

A) If there are new objects and the frames are photometry frames,and
   you don't know what the parallax filter is, just copy them to the
   photometry directory (no need to create a new parallax directory
   for them. A parallax filter probably has not yet been selected for

PARALLAX DIRECTORIES (found at: /nfs/recons4/CTIOPI/regions/)

A) hselect *.o.fits 'ra, $I, object' yes >> sort.file

B) If there are objects that are not ours (they are not on the
   observing lists): EXO planet data go into exoplanet.dir under
   special targets; BIN data go into the binary directory under
   special targets.

C) Then, after you have sorted all the non-parallax frames where they
   are supposed to go, type 'astrometry.processor sort.file'.  This
   will sort all the frames into the parallax directories.
D) If the directory does not exist, the code will ask you if you want
   to create the directory and then will sort it into that newly
   created directory if you reply 'yes'. If you reply 'no', it will
   ask you for an alternative directory destination.  If you simply
   hit enter, it will do nothing.

E) The code might also complain if the name of the directory contains
   capital letters. If this is the case, simply rename the directory
   so that all letters are lower case.

Once the frames have been sorted, we move the entire directory (with
the raw frames only -- no .o.fits files should be left in it, unless a
photometry night was begun and then scrapped) to

*****                UPDATE THE REDUCTION LOG                    *****


*****           SMARTS SPECIFIC STEPS (somewhat obsolete)        *****

7) Check to see that the data are in the expected directory.  This
   step can usually be omitted as observers typically store the data
   here, but if there is ever a problem with the next step, come back
   to this one.

   cd /ua45/v12/"YYYYMMDD"-"PI NAME"/  (i.e., 20090402-Ciardi)

4) Return to original directory and run smartsproc script.


   ./smartsproc (the './' is not explicitly necessary)

5) Answer questions using all lower case.

   PI name: ciardi
   Date (YYYYMMDD): 20090402

6) If it doesn't barf, all is well.  Log into IRAF from the current directory.


7) Change directory to where the input and output file names were
   written (and where the fits data will be stored).

   cd /u363/v12/SMARTS/"PI NAME"/"YYYYMMDD"/   
      (i.e., /u363.etc/ciardi/20090402/)

   wfits @infile @outfile

8) Once all data are written to fits, generate an info file

   hselect *.fits $I,object,filters yes > "YYYYMMDD"."PI NAME".inf
	 (i.e., 20090402.ciardi.inf).

9) Exit IRAF, go into directory data are stored, and ftp back to GSU server

   cd /u363/v12/SMARTS/"PI NAME"/"YYYYMMDD"/ 

   rsync -avz 2009* "your

   (i.e., rsync -avz 2009*

   (NOTE: if the PI name does not exist on the GSU server, this will
   barf.  You'll need to log onto joy and change directory to
   /ftp/pub/SMARTS/ and add the PI name in that directory).

Check the slurping process every few minutes as you will undoubtedly
lose the link and have to kill the process and up arrow to restart the
rsync command.

That's it!