.TH X2SYS_INIT l "15 May 2006" "GMT4.1.2" "GMT4.1.2"
.SH NAME
x2sys_init \- Initialize data base for track data files
.SH SYNOPSIS
\fBx2sys_init\fP \fITAG\fP \fB\-D\fP\fIdeffile\fP [ \fB\-E\fP\fIsuffix\fP ] 
[ \fB\-F\fP ] [ \fB\-Gd\fP|\fBg\fP ] [ \fB\-I\fP\fIdx\fP[/\fIdy\fP] ] [ \fB\-M\fP[\fBi\fP|\fBo\fP][\fIflag\fP] ] 
[ \fB\-R\fP\fIwest\fP/\fIeast\fP/\fIsouth\fP/\fInorth\fP[\fBr\fP] ] [ \fB\-V\fP ] [ \fB\-Wt\fP|\fBd\fP\fIgap\fP ]
.SH DESCRIPTION
\fBx2sys_init\fP initializes a set of data bases that are particular to one kind
of track data.  These data, their associated data bases, and key parameters are given a short-hand
notation called a system TAG.   The TAG remembers settings such as file format,
whether the data are geographic or not, and the binning resolution for track indexes.
Running \fBx2sys_init\fP is a prerequisite to running \fBx2sys_binlist\fP which will
create a crude representation of where data track go and which observations are
available; this information serves as input to
\fBx2sys_put\fP which updates the track data base. Then, \fBx2sys_get\fP can
be used to find which tracks and data are available inside a given region.
.TP
\fITAG\fP
The unique name of this data type system.
.TP
.B \-D
Definition file for this data set.
.SH OPTIONS
No space between the option flag and the associated arguments.
.TP
.B \-E
Specifies the file extension (suffix) for these data files.  If not given we use the
definition file prefix as the suffix (see \fB\-D\fP).
.TP
.B \-F
Force creating new files if old ones are present [Default will abort if old files are found].
.TP
.B \-G
Selects geographical coordinates.  Append \fBd\fP for discontinuity at the
Dateline (makes longitude go from -180 to + 180) or \fBg\fP for discontinuity
at Greenwhich (makes longitude go from 0 to 360 [Default]).
.TP
.B \-I
\fIx_inc\fP [and optionally \fIy_inc\fP] is the grid spacing. Append \fBm\fP to
indicate minutes or \fBc\fP to indicate seconds.
.TP
.B \-M
Multiple segment file(s).  Segments are separated by a special record.
For ASCII files the first character must be \fIflag\fP [Default is '>'].
For binary files all fields must be NaN.
.TP
.B \-R
\fIwest, east, south,\fP and \fInorth\fP specify the Region of interest, and you may specify them
in decimal degrees or in [+-]dd:mm[:ss.xxx][W|E|S|N] format.  Append \fBr\fP if lower left and upper right
map coordinates are given instead of wesn.  The two shorthands \fB\-Rg \-Rd\fP stand for global domain
(0/360 or -180/+180 in longitude respectively, with -90/+90 in latitude).
.TP
.B \-V
Selects verbose mode, which will send progress reports to stderr [Default runs "silently"].
.TP
.B \-W
Give \fBt\fP or \fBd\fP and append the corresponding
maximum time gap (in user units; this is typically seconds [Infinity]), or distance gap
(in user units; this is typically km [Infinity] ) allowed between
the two points immediately on either side of a crossover.
.SH DEFINITION FILES
The *.def files contain information about the data file format.  The *.def files
have two sections: (1) header information and (2) column information.  All
header information starts with the character # in the first column, immediately
followed by an upper-case directive.  If the directive takes an argument it is
separated by white-space.  You may append a trailing # comments.
Five directives are recognized:
.br
.sp
\fBASCII\fP states that the data files are in ASCII format.
.br
\fBBINARY\fP states that the data files are binary files.
.br
\fBSKIP\fP takes an integer argument which is either the number of lines to
skip (when reading ASCII files) or the number of bytes to skip (when reading
binary files).
.br
\fBGEO\fP indicates that these are geographic data set, with periodicities in
the \fIx\fP-coordinate (longitudes).
.br
\fBMULTISEG\fP means each track consists of multiple segments separated
by a \fBGMT\fP multisegment header (alternatively,
use \fB\-M\fP when defining the system TAG).
.br
.sp
The column information consists of one line per column in the order the
columns appear in the data file.  For each column you must provide  seven
attributes:
.sp
.br
\fIname type NaN NaN-proxy scale offset oformat\fP
.br
.sp
\fIname\fP is the name of the column variable.  It is expected that you will use
the special names \fIlon\fP (or \fIx\fP if Cartesian) and \fIlat\fP (or \fIy\fP)
for the two required coordinate columns, and \fItime\fP when optional time data
are present.
.br
\fItype\fP is \fBa\fP for ASCII representations of numbers, whereas for
binary files you may choose from
\fBc\fP for signed 1-byte character (-127,+128), \fBu\fP for unsigned
byte (0-255), \fBh\fP for signed 2-byte integers (-32768,+32767),
\fBi\fP for  signed 4-byte integers (-2,147,483,648,+2,147,483,647),
\fBf\fP for 4-byte floating points and \fBd\fP for 8-byte double
precision floating points.
.br
\fINaN\fP is Y if certain values are to be replaced by NAN, and N otherwise.
.br
\fINaN-proxy\fP is that special value.
.br
\fIscale\fP is used to multiply the data after reading.
.br
\fIoffset\fP is used to add to the scaled data.
.br
\fIoformat\fP is a C-style format string used to print values from this column.
.br
.sp
For ASCII data the \fItype\fP format is always \fBa\fP. \fINaN\fP should be Y
if there is a certain value (e.g., -9999) that is used to represent
missing data (NaNs).  If so, the \fINaN-proxy\fP value should be set
accordingly (otherwise it is ignored).  The \fIscale\fP and \fIoffset\fP are
applied to the data values after they are read (first scaled, then offset), 
whereas the \fIoformat\fP is used by some programs for formatted output.
If you give - as the \fIoformat\fP then \fBGMT\fP's formatting machinery will be used\"'
instead (i.e., \fBPLOT_DEGREE_FORMAT\fP, \fBPLOT_DATE_FORMAT\fP, \fBPLOT_CLOCK_FORMAT\fP).
Some file formats already have definition files premade.  These include
mgd77 (for plain ASCII MGD77 data files), mgd77+ (for enhanced MGD77+
netCDF files), gmt (for old mgg supplement binary files), xy (for plain ASCII
x, y tables), xyz (same, with z-column), geo (for plain ASCII longitude,
latitude files), and geoz (same, with z-column).
.SH EXAMPLES
If you have a large set of track data files you can organize them using the
x2sys tools.  Here we will outline the steps.  Let us assume that your track
data file format consist of 2 header records with text information followed
by any number of identically formatted data records with 6 columns (lat,
lon, time, obs1, obs2, obs3) and that files are called *.trk. We will call
this the "line" format.  First, we create the line.def file:
.br
.sp
\fB\fP# Define file for the line format
.br
\fB\fP#ASCII		# File is ASCII
.br
\fB\fP#SKIP 2		# Skip 2 header records
.br
\fB\fP#name	type 	NaN	NaN-proxy	scale offset oformat
.br
lon	a	N	0	1	0	%10.5f
.br
lat	a	N	0	1	0	%9.5f
.br
time	a	N	0	1	0	%7.1f
.br
obs1	a	N	0	1	0	%7.2f
.br
obs2	a	N	0	1	0	%7.2f
.br
obs3	a	N	0	1	0	%7.2f
.br
.sp
.br
Next we create the TAG and the databases for these line track files.
Assuming these contain geographic data and that we want to keep track of the
data distribution at a 1 x 1 degree resolution, we may run
.br
.sp
\fBx2sys_init\fP LINE \fB\-V \-G \-D\fPline \fB\-R\fP0/360/-90/90 \fB\-I\fP1/1 \fB\-E\fPtrk
.br
.sp
where we have used LINE as our system tag.  When x2sys tools try to read your
line data files they will first look in the current directory and second look
in the file \fITAG\fP_paths.txt for a list of additional directories to examine.
Therefore, create such a file (here LINE_paths.txt) and stick the full paths to
your data directories there.  All TAG-related files (definition files, tag files,
and track data bases built) will be expected to be in the directory pointed to
by \fB$X2SYS_HOME\fP.
.br
.TP
.B Create tbf file(s):
Once the (empty) TAG database have been initialized we go through a
two-step process to populate them.  First we run \fBx2sys_binlist\fP on
all our track files to create one (or more) multi-segment track bin-index
files (tbf).  These contain information on which 1 x 1 degree bins (or any
other blocksize; see \fB\-I\fP) each
track has visited and which observations (in your case obs1, obs2, obs3)
were actually observed (not all tracks may have all three kinds of
observations everywhere).  For instance, if your tracks are listed in
the file tracks.lis we may run this command:
.br
.sp
\fBx2sys_binlist\fP \fB\-V \-T\fPLINE `cat tracks.lis` > tracks.tbf
.TP
.B Update index data base:
Next, the track bin-index files are fed to \fBx2sys_put\fP
which will insert the information into the TAG databases:
.br
.sp
\fBx2sys_put\fP \fB\-V \-T\fPLINE tracks.tbf
.TP
.B Search for data:
You may now use \fBx2sys_get\fP to find all the tracks within a certain
sub-region, and optionally limit the search to those tracks that have
a particular combination of observables.  E.g., to find all the tracks
which has both obs1 and obs3 inside the specified region, run
.br
.sp
\fBx2sys_get\fP \fB\-V \-T\fPLINE \fB\-R\fP20/40/-40/-20 \fB\-F\fPobs1,obs3 > tracks.tbf
.TP
.B MGD77[+] or GMT:
Definition files already exist for MGD77 files (both standard ASCII and
enhanced netCDF-based MGD77+ files), the old *.gmt files manipulated by the
mgg supplements, and plain x,y[,z] and lon,lat[,z] tracks.
To initiate new track databases to be used with MGD77 data from NGDC, try
.br
.sp
\fBx2sys_init\fP MGD77 \fB\-V \-D\fPmgd77 \fB\-E\fPmgd77 \fB\-Rd \-Gd \-I\fP1/1 \fB\-Wt\fP900  \fB\-Wd\fP5
.br
.sp
where we have chosen a 15 minute (900 sec) or 5 km threshold to indicate a data gap;
the other steps are similar.
.TP
.B Binary files:
Let us pretend that your line files actually are binary files with a
128-byte header structure (to be skipped) followed by the data records
and where \fIlon\fP, \fIlat\fP, \fItime\fP are double precision numbers while the three
observations are 2-byte integers which must be multiplied by 0.1.
Finally, the first two observations may be -32768 which means there is
no data available.
All that is needed is a different line.def file:
.br
.sp
\fB\fP# Define file for the binary line format
.br
\fB\fP#BINARY	# File is now binary
.br
\fB\fP#SKIP 128	# Skip 128 bytes
.br
\fB\fP#name	type	 NaN?	NaN-proxy	scale offset oformat
.br
lon	d	N	0	1	0	%10.5f
.br
lat	d	N	0	1	0	%9.5f
.br
time	d	N	0	1	0	%7.1f
.br
obs1	h	Y	-32768	0.1	0	%6.1f
.br
obs2	h	Y	-32768	0.1	0	%6.1f
.br
obs3	h	N	0	0.1	0	%6.1f
.br
.sp
The rest of the steps are identical.  
.SH "SEE ALSO"
.IR x2sys_binlist (l)
.IR x2sys_get (l)
.IR x2sys_put (l)
