[ViewVC] Contents of: JSOC/proj/cookbook/ReadMe

This directory contains a "cookbook" collection of sample DRMS modules,
along with a simplified make system that can be used to build comparable
applications modules outside of the NetDRMS/JSOC distribution trees.


Table of Contents

        Directories
CVS/            CVS repository information
Makevars/       files containing system/architecture-specific make variable
        definitions for use in a generic Makefile

        Regular Program
smpl_00.c       does (almost) nothing at all - like hello_world, but exhibits
        use of the command-line parsing used by JSOC module drivers

        DRMS Modules
smpl_01.c       same as smpl_00, but written as a module
smpl_02.c       echoes its arguments, of a wide variety of types
smpl_03.c       prints a list of the data series known to DRMS - a simple
        version of show_series
smpl_04.c       prints the number of unique records in a selected data series
        and the number of data segments per record - a simple version of
        show_info
smpl_05.c       inserts data records from noaa_ar.dat into series created with
        noaa_ar.jsd
smpl_06.c       updates data records in series built with smpl_05 with new
        keyword values
smpl_07.c       inserts image data in record segments in a data series created
        with images.jsd
smpl_08.c       calculates and reports primary statistics of record segments
        such as those generated with smpl_07

        Other
Makefile        The generic Makefile
ReadMe          (this file)
images.jsd      A JSD file that can be used to build a (keywordless) DRMS data
        series for storing data segments
noaa_ar.dat     A summary listing of data from daily NOAA Active Region reports
        for regions with spots from 1996 onward; can be used as input for a
        DRMS data series
noaa_ar.jsd     A JSD file that can be used to build a (segmentless) DRMS data
        series for the data from noaa_ar.dat

********************************************************************************

Recipe 0: How to write and build a program in this directory, using features
of the DRMS library

The program smpl_00.c is technically not a DRMS module, just a standard C
program similar to the familiar "Hello, world" one. The only real difference
is that it is linked against the DRMS API to use its command-line argument
parsing features. The message that the program prints when executed can be
changed from the default, "done", by running the command:

        smpl_00 print=something
or
        smpl_00 print= "Hello, world!"

(White space after the '=' sign is optional; quotes are necessary if the
argument string has embedded white space.)

The program can be built, like all of the modules in this directory as well,
by typing

        make smpl_00

(alternatively, you can simply type make and all the modules in the target
list MODS in the Makefile will be compiled.) The Makefile is generic: it is
designed to work on different hardware and operating system platforms by
accessing an appropriate set of make variables definitions from the various
files in the subdirectory makevars. To add a new module that does not require
linking with any additional libraries beyond those required for DRMS, all you
need to do (besides writing the module!) is to append its name (without the
filename extension) to the list of MODS in the Makefile.

Note that the Makefile requires the external definition of the environment
variable JPLAT to the appropriate name for your build platform. If, when you
run the make, you get a message like:

        Error: no appropriate Makevars; is $JPLAT defined? won't compile...

Either you have not defined the system-appropriate environment variable,
or there is no file Makevars_$JPLAT.mk in the Makevars directory. It should
be straightforward to create an appropriate one by replicating one of the
existing examples and substituting appropriate command paths.  Likewise,
if you get a message like:

        Makefile:1: Makevars/Makevars_XXX.mk: No such file or directory
        make: *** No rule to make target `Makevars/Makevars_XXX.mk'.  Stop.

it means that you have defined the environment variable JPLAT (as XXX), but
there is no corresponding Makevars_XXX.mk in the directory. Again, you can
create the appropriate one by replicating one of the existing examples and
substituting appropriate command paths.

(Alternatively, if you will only be building and running on one platform
ever, you could just add a platform definition line like:

        JPLAT = linux_x86_64

or whatever before the first line in the Makefile.)

The architecture-dependent Makevars_*.mk files define absolute paths to
essential commands, especially compilers. If you get messages like:

        make: /usr/local/bin/icc: Command not found

You must suitably redefine the relevant Make variables (e.g. CC or ICC in
this example) in the file being included. If you have to change from icc
to gcc, be sure to change the appropriate CFLAGS and LDFLAGS variables as
well, removing the -xW option.

The Makefile also requires external definition of the environment variable
DRMS to be the root of your NetDRMS distribution as described in the
installation instructions. If you get an error message like:

error: could not open source file "cmdparams.h"

it is probably because you have not correctly set the DRMS environment variable.
(Again, if you wish you could define DRMS in the Makefile rather than as an
environment variable.)

The Makefile assumes that all third-party libraries, in particular those for
cfitsio and postgres that are essential for DRMS, reside in one of the default
search locations, /usr/lib or /usr/local/lib. If they are elsewehere, then
you will need to add the base path to the LIBD definition line in the Makefile.

In order to build a debug version of any of the modules in the project (with
symbols and without optimization), you can set the environment variable JPLAT
to debug. The debug "platform" variables are identical to those in the
linux_x86_64 file, except for the C compiler options. Since it is very likely
in debugging that you may need symbols in the DRMS library as well, you should
build a debug platform version of that library by running (in the DRMS root
directory):

        setenv JSOC_DEBUG 1
        make MACH=debug

********************************************************************************

Recipe 1: How to write and build a minimal DRMS module

The program smpl_01.c is a true DRMS module; it exhibits the minimum structure
necessary for a C program that will be able to interact with the DRMS API to
fetch and store date in the DRMS and SUMS. This module in fact does neither,
as it is identical in functionality to smpl_00.c; but it nevertheless opens
the required connection to the DRMS, and will fail if that is not possible for
some reason, such as the database server being unavailable or the user lacking
proper authorization to access it. It can therefore serve as a useful test for
minimal connectivity to DRMS.

Note that this program does not have a main() entry, it is instead called
DoIt(). All DRMS modules are called DoIt() and link against another main()
program in the DRMS library that opens a connection to DRMS and SUMS for
them, calls DoIt(), and then closes the connection, saving or discarding
the results according to the return status of the module.

In addition to the "print" argument, this module also features two "flag"
arguments. If you run
        smpl_01 -v
it will print out a little more information about what it is up to (this is
a common flag), and if you run
        smpl_01 -a
it will force an abort, that is, returns a non-zero value to the main program.
You can also run
        smpl_01 -av
or      smpl_01 -a -v

You can get a list of the arguments, and default values (if any), by typing
        smpl_01 -H
This is a feature of all DRMS modules, as it is handled by the (hidden) main
program.  Likewise, the command
        smpl_01 -V
results in some extra verbose information from the main program itself before
and after the module is called.

********************************************************************************

Recipe 2: Command-line Argument Parsing

Module smpl_02 also has no interaction with the DRMS database. It shows the
full range of command-line argument types and how they are to be processed
in the module. Type
        smpl_02 -H
to see the full argument list and then fool around with various values for the
arguments. Most of the argument types should be self explanatory. ARG_INTS and
ARG_FLOATS can be used to specify arrays of arbitrary length (including zero)
of integers or floats, comma separated and enclosed (if there are more than one)
within matching pairs of either brackets, braces, or parentheses. ARG_NUME is
used for an enumerated list of possible strings, returning the order of the
selected string in the enumerated list, similar to an enum declarator in C.

********************************************************************************

Recipe 3: Communicating with the DRMS Database - show_series

Module smpl_03 shows how direct communication with the database can be done
through a module. It is basically a simple version of the show_series
application.  The module still bypasses most of the API. Normally you should
not need to communicate with the database at such a low level in a module, but
this shows how to do it if you are familiar with SQL and need more power and
flexibility than is provided by the record management API. You can use this
module, like show_series, to see what data series are already in your DRMS
database.

********************************************************************************

Recipe 4: Beginning to Use the API - show_info

Module smpl_04 is the first one to actually use the DRMS record management
API. It is a (very) simplified version of the show_info application. You can
use this module to begin to explore the various data series in your DRMS.

********************************************************************************

Recipe 5: Adding Records to a Data Series

Up to this point, all of the cookbook modules operated on data series that
already existed. At some point, you need to be able to create and populate
data series yourself. Module smpl_05 is an example of a module that can be
used to add (or update) records in a data series, in this case one containing
records correspnding to the daily NOAA reports of individual solar active
regions with spots. The data from those reports has been assembled in a
simple ASCII table text form in the file noaa_ar.dat. The module smpl_05
reads data from that table and inserts the appropriate records in a data
data series. In order to exercise it, you will first need to create a data
series with the appropriate structure that you can write to. To do so, run
        create_series noaa_ar.jsd
You will probably have to change the Seriesname from drms.NOAA_ar to
something in a different namespace from drms; each user should have a
personal namespace. Then you can run the module with the appropriate value
for the "ds" argument.

Any time you want to remove this series (to try out different series
modifications in the JSD for example), you can delete it by typing
        delete_series drms.NOAA_ar
(or whatever its series name is) and answering yes to the safeguard prompts.
If you do not change the series name in the JSD, you can also force its
deletion and recreation by typing
        create_series -f noaa_ar.jsd
If you give it a new series name of course it will not remove the old one.

Once you have created the data series, you can run
        smpl_05
The default value for the ds parameter is drms.noaa_ar, so you will have to
run the module with a different series name for the "ds" argument the series
you created has a diffeent name. The default value for the "data" argument
is noaa_ar.dat, a file included in the cookbook.

********************************************************************************

Recipe 6: Modifying Records in a Data Series

If you inspect the last few records in the data series just created, for
example with
        show_info -a ds= drms.noaa_ar n= -5
you will discover that the penultimate record, that for the observation of
AR 11024 on 2009.07.10, has a very strange and probably erroneous value
for its latitude, -9 deg, when the latitude on the preceding and following
days was -25 deg. This is the result of what was presumably a typographical
error in the NOAA/USAF report for that day (SRS Number 191 Issued at 0030Z
on 10 Jul 2009). Module smpl_06 is an example of one that could be used to
correct or update the values of selected records in a data series. It shows
how runtime parameters (and internally computed variables) can be used in
place of record set specifiers to restrict the recordset, and how key values
in selected records can be "changed".  In order to fix the relevant record,
you could run either
        smpl_06 date= 2009.07.10 ar= 11024 key= lat value= -25
or
        smpl_06 ds= "drms.noaa_ar[2009.07.10][11024]" key= lat value= -25
and then reinspect the last few records. The record in question should have
been updated with a new value for its latitude.  In reality, the DRMS API
does not allow for actual value changes to records in the database, which is
why the function drms_clone_record() is used in this update module. If you
type
        show_info -ar ds= "drms.noaa_ar[\!Region=11024\!]"
you will see that there are really two entries in the database with the same
values for the two prime keys Date=2009.07.10 and Region=11024. The one with
the higher recnum is the only one that will show up in ordinary queries.

********************************************************************************

Recipe 7: Creating Records with Data Segments

The data series worked with in the previous two examples had no associated
data segments, repositories for bulk data such as images or other data arrays.
The JSD file images.jsd describes a series with the opposite structure. It
contains no keywords at all in the database (except for the hidden keywords
like recnum, so that all records are unique). However, it does allow for a
segment, which can be any 2-dimensional array of short ints (which can
represent scaled floats), stored internally as Rice-compressed FITS binary
tables.  As in the preparations for Recipe 5, run
        create_series images.jsd

You can then run smpl_07 to create records (one at a time) in this series.
Each will have a data segment of specified size and shape, with the values
filled randomly using a somewhat Byzantine procedure to create a semblance
of large-scale structure. You might want to fool around with different values
of the parameters dist and seed.  In order to view the data segments, run for
example
        show_info -pq ds= drms.images"[]"
and then run a viewer, e.g. ds9, on the fits files in the displayed SUMS
directories.

Note a few interesting features of the JSD file you used to create this data
series. First the Archive flag is a negative number. Normally the Archive flag
is set to 1 if the data segments are to be archived to tape before they age
off the SUMS disks (in 10 days in this case), 0 otherwise. However, if the
data segments are allowed to disappear without having been archived to tape,
the data records in the DRMS will still remain. Sometimes that is useful, but
in this case, with a series that contains no ancillary data at all, that would
be pointless. The value of -1 will force the DRMS to actually remove any records
when their corresponding data segments age off the disk.

Also, note that the module automatically created FITS files named v.fits
with binary table row compression. That is because of the parameters specified
for the series Segment "Data" in the JSD file. You might want to experiment
with different values for the various fields in the Segment descriptor, "name",
scope, "type", "naxis" (which would cause the module to fail if changed),
"axis_n" (which must be specified if scope is variable rather than vardim and
might cause the module to fail if inappropriate values of parameters are
chosen, "protocol", the compression and scaling parameters. In order to
conduct such experiments you must either create a new series with a different
series name by specifying a different value for Seriesname in the JSD, or, if
you wish to keep the series name, recreate the series afresh, since its
essential structure is being changed. To do the latter, run:
        delete_series drms.images
(or whatever name you gave the series) and then
        create_series images.jsd
Alternatively, you can just run:
        create_series -f images.jsd
which will force deletion of the existing series if it has the same Seriesname.

********************************************************************************

Recipe 8: Analyzing Data in Record Data Segments

Now that you have created a data series with records containing "real" data,
you can begin to explore the functions that fetch data from SUMS segments into
memory for processing.

smpl_08.c is a simple program that calculates and prints the primary statistics
(count, mean, standard deviation) of each segment (or a selected segment) of
a selected data set. It takes as arguments the input data set specification and
optionally a segment name as well. The default record set specified is the
last record inserted into the data series by smpl_07. (If you gave that series
a different name from drms.images you will either have to change the default
value for the "ds" argument in smpl_08.c or provide the correct series name
as the value of the argument ds when you run smpl_08.
Revision:	1.11
Committed:	Fri Nov 4 00:57:19 2011 UTC (11 years, 10 months ago) by rick
Branch:	MAIN
CVS Tags:	NetDRMS_Ver_6-4, NetDRMS_Ver_8-0, NetDRMS_Ver_6-2, NetDRMS_Ver_6-3, NetDRMS_Ver_6-0, NetDRMS_Ver_6-1, Ver_6-0, Ver_6-1, Ver_6-2, Ver_6-3, Ver_6-4, NetDRMS_Ver_8-8, Ver_8-5, NetDRMS_Ver_7-0, NetDRMS_Ver_8-1, Ver_7-0, Ver_LATEST, NetDRMS_Ver_LATEST, NetDRMS_Ver_8-12, NetDRMS_Ver_8-10, NetDRMS_Ver_8-11, NetDRMS_Ver_9-1, NetDRMS_Ver_9-0, NetDRMS_Ver_9-3, NetDRMS_Ver_9-2, NetDRMS_Ver_9-5, NetDRMS_Ver_9-4, NetDRMS_Ver_8-2, NetDRMS_Ver_8-3, NetDRMS_Ver_9-41, Ver_9-41, Ver_DRMSLATEST, NetDRMS_Ver_8-4, NetDRMS_Ver_8-5, NetDRMS_Ver_8-6, Ver_8-8, NetDRMS_Ver_8-7, Ver_8-2, Ver_9-3, Ver_8-0, Ver_8-1, Ver_8-6, Ver_8-7, Ver_8-4, Ver_8-11, Ver_7-1, Ver_9-1, Ver_8-3, NetDRMS_Ver_7-1, Ver_9-5, Ver_9-4, Ver_8-10, Ver_9-2, Ver_8-12, Ver_9-0, HEAD
Changes since 1.10:	+11 -0 lines
Log Message:	added instructions for building debug versions of modules
#	Content
1	This directory contains a "cookbook" collection of sample DRMS modules,
2	along with a simplified make system that can be used to build comparable
3	applications modules outside of the NetDRMS/JSOC distribution trees.
4
5
6	Table of Contents
7
8	Directories
9	CVS/ CVS repository information
10	Makevars/ files containing system/architecture-specific make variable
11	definitions for use in a generic Makefile
12
13	Regular Program
14	smpl_00.c does (almost) nothing at all - like hello_world, but exhibits
15	use of the command-line parsing used by JSOC module drivers
16
17	DRMS Modules
18	smpl_01.c same as smpl_00, but written as a module
19	smpl_02.c echoes its arguments, of a wide variety of types
20	smpl_03.c prints a list of the data series known to DRMS - a simple
21	version of show_series
22	smpl_04.c prints the number of unique records in a selected data series
23	and the number of data segments per record - a simple version of
24	show_info
25	smpl_05.c inserts data records from noaa_ar.dat into series created with
26	noaa_ar.jsd
27	smpl_06.c updates data records in series built with smpl_05 with new
28	keyword values
29	smpl_07.c inserts image data in record segments in a data series created
30	with images.jsd
31	smpl_08.c calculates and reports primary statistics of record segments
32	such as those generated with smpl_07
33
34	Other
35	Makefile The generic Makefile
36	ReadMe (this file)
37	images.jsd A JSD file that can be used to build a (keywordless) DRMS data
38	series for storing data segments
39	noaa_ar.dat A summary listing of data from daily NOAA Active Region reports
40	for regions with spots from 1996 onward; can be used as input for a
41	DRMS data series
42	noaa_ar.jsd A JSD file that can be used to build a (segmentless) DRMS data
43	series for the data from noaa_ar.dat
44
45	********************************************************************************
46
47	Recipe 0: How to write and build a program in this directory, using features
48	of the DRMS library
49
50	The program smpl_00.c is technically not a DRMS module, just a standard C
51	program similar to the familiar "Hello, world" one. The only real difference
52	is that it is linked against the DRMS API to use its command-line argument
53	parsing features. The message that the program prints when executed can be
54	changed from the default, "done", by running the command:
55
56	smpl_00 print=something
57	or
58	smpl_00 print= "Hello, world!"
59
60	(White space after the '=' sign is optional; quotes are necessary if the
61	argument string has embedded white space.)
62
63	The program can be built, like all of the modules in this directory as well,
64	by typing
65
66	make smpl_00
67
68	(alternatively, you can simply type make and all the modules in the target
69	list MODS in the Makefile will be compiled.) The Makefile is generic: it is
70	designed to work on different hardware and operating system platforms by
71	accessing an appropriate set of make variables definitions from the various
72	files in the subdirectory makevars. To add a new module that does not require
73	linking with any additional libraries beyond those required for DRMS, all you
74	need to do (besides writing the module!) is to append its name (without the
75	filename extension) to the list of MODS in the Makefile.
76
77	Note that the Makefile requires the external definition of the environment
78	variable JPLAT to the appropriate name for your build platform. If, when you
79	run the make, you get a message like:
80
81	Error: no appropriate Makevars; is $JPLAT defined? won't compile...
82
83	Either you have not defined the system-appropriate environment variable,
84	or there is no file Makevars_$JPLAT.mk in the Makevars directory. It should
85	be straightforward to create an appropriate one by replicating one of the
86	existing examples and substituting appropriate command paths. Likewise,
87	if you get a message like:
88
89	Makefile:1: Makevars/Makevars_XXX.mk: No such file or directory
90	make: *** No rule to make target `Makevars/Makevars_XXX.mk'. Stop.
91
92	it means that you have defined the environment variable JPLAT (as XXX), but
93	there is no corresponding Makevars_XXX.mk in the directory. Again, you can
94	create the appropriate one by replicating one of the existing examples and
95	substituting appropriate command paths.
96
97	(Alternatively, if you will only be building and running on one platform
98	ever, you could just add a platform definition line like:
99
100	JPLAT = linux_x86_64
101
102	or whatever before the first line in the Makefile.)
103
104	The architecture-dependent Makevars_*.mk files define absolute paths to
105	essential commands, especially compilers. If you get messages like:
106
107	make: /usr/local/bin/icc: Command not found
108
109	You must suitably redefine the relevant Make variables (e.g. CC or ICC in
110	this example) in the file being included. If you have to change from icc
111	to gcc, be sure to change the appropriate CFLAGS and LDFLAGS variables as
112	well, removing the -xW option.
113
114	The Makefile also requires external definition of the environment variable
115	DRMS to be the root of your NetDRMS distribution as described in the
116	installation instructions. If you get an error message like:
117
118	error: could not open source file "cmdparams.h"
119
120	it is probably because you have not correctly set the DRMS environment variable.
121	(Again, if you wish you could define DRMS in the Makefile rather than as an
122	environment variable.)
123
124	The Makefile assumes that all third-party libraries, in particular those for
125	cfitsio and postgres that are essential for DRMS, reside in one of the default
126	search locations, /usr/lib or /usr/local/lib. If they are elsewehere, then
127	you will need to add the base path to the LIBD definition line in the Makefile.
128
129	In order to build a debug version of any of the modules in the project (with
130	symbols and without optimization), you can set the environment variable JPLAT
131	to debug. The debug "platform" variables are identical to those in the
132	linux_x86_64 file, except for the C compiler options. Since it is very likely
133	in debugging that you may need symbols in the DRMS library as well, you should
134	build a debug platform version of that library by running (in the DRMS root
135	directory):
136
137	setenv JSOC_DEBUG 1
138	make MACH=debug
139
140	********************************************************************************
141
142	Recipe 1: How to write and build a minimal DRMS module
143
144	The program smpl_01.c is a true DRMS module; it exhibits the minimum structure
145	necessary for a C program that will be able to interact with the DRMS API to
146	fetch and store date in the DRMS and SUMS. This module in fact does neither,
147	as it is identical in functionality to smpl_00.c; but it nevertheless opens
148	the required connection to the DRMS, and will fail if that is not possible for
149	some reason, such as the database server being unavailable or the user lacking
150	proper authorization to access it. It can therefore serve as a useful test for
151	minimal connectivity to DRMS.
152
153	Note that this program does not have a main() entry, it is instead called
154	DoIt(). All DRMS modules are called DoIt() and link against another main()
155	program in the DRMS library that opens a connection to DRMS and SUMS for
156	them, calls DoIt(), and then closes the connection, saving or discarding
157	the results according to the return status of the module.
158
159	In addition to the "print" argument, this module also features two "flag"
160	arguments. If you run
161	smpl_01 -v
162	it will print out a little more information about what it is up to (this is
163	a common flag), and if you run
164	smpl_01 -a
165	it will force an abort, that is, returns a non-zero value to the main program.
166	You can also run
167	smpl_01 -av
168	or smpl_01 -a -v
169
170	You can get a list of the arguments, and default values (if any), by typing
171	smpl_01 -H
172	This is a feature of all DRMS modules, as it is handled by the (hidden) main
173	program. Likewise, the command
174	smpl_01 -V
175	results in some extra verbose information from the main program itself before
176	and after the module is called.
177
178	********************************************************************************
179
180	Recipe 2: Command-line Argument Parsing
181
182	Module smpl_02 also has no interaction with the DRMS database. It shows the
183	full range of command-line argument types and how they are to be processed
184	in the module. Type
185	smpl_02 -H
186	to see the full argument list and then fool around with various values for the
187	arguments. Most of the argument types should be self explanatory. ARG_INTS and
188	ARG_FLOATS can be used to specify arrays of arbitrary length (including zero)
189	of integers or floats, comma separated and enclosed (if there are more than one)
190	within matching pairs of either brackets, braces, or parentheses. ARG_NUME is
191	used for an enumerated list of possible strings, returning the order of the
192	selected string in the enumerated list, similar to an enum declarator in C.
193
194	********************************************************************************
195
196	Recipe 3: Communicating with the DRMS Database - show_series
197
198	Module smpl_03 shows how direct communication with the database can be done
199	through a module. It is basically a simple version of the show_series
200	application. The module still bypasses most of the API. Normally you should
201	not need to communicate with the database at such a low level in a module, but
202	this shows how to do it if you are familiar with SQL and need more power and
203	flexibility than is provided by the record management API. You can use this
204	module, like show_series, to see what data series are already in your DRMS
205	database.
206
207	********************************************************************************
208
209	Recipe 4: Beginning to Use the API - show_info
210
211	Module smpl_04 is the first one to actually use the DRMS record management
212	API. It is a (very) simplified version of the show_info application. You can
213	use this module to begin to explore the various data series in your DRMS.
214
215	********************************************************************************
216
217	Recipe 5: Adding Records to a Data Series
218
219	Up to this point, all of the cookbook modules operated on data series that
220	already existed. At some point, you need to be able to create and populate
221	data series yourself. Module smpl_05 is an example of a module that can be
222	used to add (or update) records in a data series, in this case one containing
223	records correspnding to the daily NOAA reports of individual solar active
224	regions with spots. The data from those reports has been assembled in a
225	simple ASCII table text form in the file noaa_ar.dat. The module smpl_05
226	reads data from that table and inserts the appropriate records in a data
227	data series. In order to exercise it, you will first need to create a data
228	series with the appropriate structure that you can write to. To do so, run
229	create_series noaa_ar.jsd
230	You will probably have to change the Seriesname from drms.NOAA_ar to
231	something in a different namespace from drms; each user should have a
232	personal namespace. Then you can run the module with the appropriate value
233	for the "ds" argument.
234
235	Any time you want to remove this series (to try out different series
236	modifications in the JSD for example), you can delete it by typing
237	delete_series drms.NOAA_ar
238	(or whatever its series name is) and answering yes to the safeguard prompts.
239	If you do not change the series name in the JSD, you can also force its
240	deletion and recreation by typing
241	create_series -f noaa_ar.jsd
242	If you give it a new series name of course it will not remove the old one.
243
244	Once you have created the data series, you can run
245	smpl_05
246	The default value for the ds parameter is drms.noaa_ar, so you will have to
247	run the module with a different series name for the "ds" argument the series
248	you created has a diffeent name. The default value for the "data" argument
249	is noaa_ar.dat, a file included in the cookbook.
250
251	********************************************************************************
252
253	Recipe 6: Modifying Records in a Data Series
254
255	If you inspect the last few records in the data series just created, for
256	example with
257	show_info -a ds= drms.noaa_ar n= -5
258	you will discover that the penultimate record, that for the observation of
259	AR 11024 on 2009.07.10, has a very strange and probably erroneous value
260	for its latitude, -9 deg, when the latitude on the preceding and following
261	days was -25 deg. This is the result of what was presumably a typographical
262	error in the NOAA/USAF report for that day (SRS Number 191 Issued at 0030Z
263	on 10 Jul 2009). Module smpl_06 is an example of one that could be used to
264	correct or update the values of selected records in a data series. It shows
265	how runtime parameters (and internally computed variables) can be used in
266	place of record set specifiers to restrict the recordset, and how key values
267	in selected records can be "changed". In order to fix the relevant record,
268	you could run either
269	smpl_06 date= 2009.07.10 ar= 11024 key= lat value= -25
270	or
271	smpl_06 ds= "drms.noaa_ar[2009.07.10][11024]" key= lat value= -25
272	and then reinspect the last few records. The record in question should have
273	been updated with a new value for its latitude. In reality, the DRMS API
274	does not allow for actual value changes to records in the database, which is
275	why the function drms_clone_record() is used in this update module. If you
276	type
277	show_info -ar ds= "drms.noaa_ar[\!Region=11024\!]"
278	you will see that there are really two entries in the database with the same
279	values for the two prime keys Date=2009.07.10 and Region=11024. The one with
280	the higher recnum is the only one that will show up in ordinary queries.
281
282	********************************************************************************
283
284	Recipe 7: Creating Records with Data Segments
285
286	The data series worked with in the previous two examples had no associated
287	data segments, repositories for bulk data such as images or other data arrays.
288	The JSD file images.jsd describes a series with the opposite structure. It
289	contains no keywords at all in the database (except for the hidden keywords
290	like recnum, so that all records are unique). However, it does allow for a
291	segment, which can be any 2-dimensional array of short ints (which can
292	represent scaled floats), stored internally as Rice-compressed FITS binary
293	tables. As in the preparations for Recipe 5, run
294	create_series images.jsd
295
296	You can then run smpl_07 to create records (one at a time) in this series.
297	Each will have a data segment of specified size and shape, with the values
298	filled randomly using a somewhat Byzantine procedure to create a semblance
299	of large-scale structure. You might want to fool around with different values
300	of the parameters dist and seed. In order to view the data segments, run for
301	example
302	show_info -pq ds= drms.images"[]"
303	and then run a viewer, e.g. ds9, on the fits files in the displayed SUMS
304	directories.
305
306	Note a few interesting features of the JSD file you used to create this data
307	series. First the Archive flag is a negative number. Normally the Archive flag
308	is set to 1 if the data segments are to be archived to tape before they age
309	off the SUMS disks (in 10 days in this case), 0 otherwise. However, if the
310	data segments are allowed to disappear without having been archived to tape,
311	the data records in the DRMS will still remain. Sometimes that is useful, but
312	in this case, with a series that contains no ancillary data at all, that would
313	be pointless. The value of -1 will force the DRMS to actually remove any records
314	when their corresponding data segments age off the disk.
315
316	Also, note that the module automatically created FITS files named v.fits
317	with binary table row compression. That is because of the parameters specified
318	for the series Segment "Data" in the JSD file. You might want to experiment
319	with different values for the various fields in the Segment descriptor, "name",
320	scope, "type", "naxis" (which would cause the module to fail if changed),
321	"axis_n" (which must be specified if scope is variable rather than vardim and
322	might cause the module to fail if inappropriate values of parameters are
323	chosen, "protocol", the compression and scaling parameters. In order to
324	conduct such experiments you must either create a new series with a different
325	series name by specifying a different value for Seriesname in the JSD, or, if
326	you wish to keep the series name, recreate the series afresh, since its
327	essential structure is being changed. To do the latter, run:
328	delete_series drms.images
329	(or whatever name you gave the series) and then
330	create_series images.jsd
331	Alternatively, you can just run:
332	create_series -f images.jsd
333	which will force deletion of the existing series if it has the same Seriesname.
334
335	********************************************************************************
336
337	Recipe 8: Analyzing Data in Record Data Segments
338
339	Now that you have created a data series with records containing "real" data,
340	you can begin to explore the functions that fetch data from SUMS segments into
341	memory for processing.
342
343	smpl_08.c is a simple program that calculates and prints the primary statistics
344	(count, mean, standard deviation) of each segment (or a selected segment) of
345	a selected data set. It takes as arguments the input data set specification and
346	optionally a segment name as well. The default record set specified is the
347	last record inserted into the data series by smpl_07. (If you gave that series
348	a different name from drms.images you will either have to change the default
349	value for the "ds" argument in smpl_08.c or provide the correct series name
350	as the value of the argument ds when you run smpl_08.