CM/V5.12/release.notes

                       Release Notes JSOC V5.12        15DEC2010
                       ------------------------        ---------


A release is a set of files, each having a specific version.  And a release typcially
has a version number because over time you have newer and newer releases of the 
same product.  For example, a hypothetical 1.3 release may contain fileA#1.8, 
fileB#1.2, fileC#2.2 and a 1.4 release may contain fileA#2.5, fileB#2.1, fileC#2.9. 
JSOC releases are similarly versioned and contain a set of such files.  JSOC release
code is guaranteed to compile on cluster nodes (eg., n00, n02).  The resulting binaries
have been minimally tested.  At the time of the creation of the release, the
release versions of each file will be the most recent.  But as time passes, newer versions 
of some files will be made, and there is no guarantee that these changes will
not destabilize JSOC (ie., they may cause JSOC to no longer compile or execute
properly).  

There are several ways to use this release.  If you wish to simply use pre-built
binaries, you can simply use the production binaries, which are located at 
/home/production/cvs/JSOC.  Every time a release is created, the binaries in
this location get updated.  Only the production user can update these binaries.
So, you could run /home/production/cvs/JSOC/bin/linux_x86_64/show_keys, for example.
If instead you want to work with stable source files, then you must have a sandbox,
which is a local copy (in your home directory) of the files in the cvs depot.  
You would probably want to work with a sandbox if you plan on making eventual 
changes to the depot files.  Changes you make to your sandbox files are not visible 
to other users until you "commit" those changes back to the cvs depot.  Please see
"If You Don't Have a Sandbox" below for more information on how to create a sandbox.  
There is also a "working" release which resides in in /home/jsoc/cvs/JSOC.  New 
files may be placed here and existing files may be edited for common use before the 
next official release.  Each time a release gets created, the source and binaries of 
the working release get updated.  WARNING: the files you see here may not be stable 
since by the time you see them, another user may have edited them. Only the production 
release is guaranteed to be stable and unchanged between releases.

Obtaining the Release
---------------------
To update your working directory to this release, or to check-out this release anew, 
please visit http://jsoc.stanford.edu/jsocwiki/CvsInit. Please keep in mind that
users may have modified files since the release was created, so use of the 
scripts documented in the web page may result in a working directory whose
content is not identical to the release.  If updating, you can supply 
the flag "-R" to the jsoc_update.pl and jsoc_sync.pl scripts to download the
latest release.  This will ensure that your working directory has the exact, latest
release versions of the files (eg., jsoc_sync.csh -R). If checking-out, 
you can supply the argument "-r Ver_LATEST" to the "cvs checkout" command
to achieve the analogous result, but for a fresh checkout.  WARNING: if you use 
the "-R" or "-r" flags, please use only jsoc_update.pl or jsoc_sync.pl to update 
your sources thereafter.  Use of "-R" or "-r Ver_LATEST" will result in a cvs
"sticky flag" being set.  jsoc_update.pl and jsoc_sync.pl clear this sticky flag.

Additional Info
---------------
If you are unfamiliar with the use of cvs see the file:
JSOC/CM/working_with_sandbox.txt.

There's a linux4 cvs gui at xim:/usr/bin/lincvs
Also on our jsoc web page:

http://jsoc.stanford.edu/cvs/JSOC/

Use the Apache cvs gui to see the diffs. For example, go to
http://jsoc.stanford.edu/cvs/JSOC/base/drms/
and click on the name in the File column and then click on
"diffs to previous #" to see the diffs.

Changes since previous release (V5.11 - Oct 6, 2010)
--------------------------------------------------------

NEW FEATURES:
- Moved all FITS-export code out of lib DRMS and into a new library in
JSOC/base/export/libs/exportDRMS. 
- Added code to check for user permissions to read from the
_jsoc.sl_table. This table is accessed when delete_series is called to
determine if the series being deleted is being replicated (and we
disallow deletion of slony-replicated series).
- The COMMENT and HISTORY keywords are now exported into the FITS header
when a record being exported contains those keywords.
- Added 'RECNUM' to set of FITS keywords exported during export of FITS
files. 
- Modified the jsd parser to print a warning if provided external
keyword name (in the keyword description) is RECNUM, since this will
collide with the auto-export of RECNUM.
- New module set_info - this is virtually identical to set_keys,
except that it will also copy FITS files into series that contain
FITS-protocol DRMS data segments.
- Added file-locking syncronization to publish_series.sh to disallow
multiple instances of publish_series.sh to run.
- Updated the flatfield code to a newer version.
- Added support for APID 44 to housekeeping-processing code.
- Added DATE keyword to housekeeping-processing code.
- Ticket #303 - Created editloggertabs.pl, a script to
create/delete/modify configuration tables used by monitoring/logging
code.
- Created newdrmsuser.pl, a script that facilitates new user creation;
it creates the default/appropriate entries in the db and it runs
masterlists.
- Modified the DRMS library code that interfaces with SUMS to attempt
re-connection in the event that the existing SUMS connection
vaporizes. 
- Added a jsoc_main parameter, "--loomain", that causes modules to
loop attempting to connect to SUMS when the initial SUMS-connection
attempt failes. By default, if the initial SUMS-connection fails, then
modules do not attempt to connect again.
- Add DRMS support for SUMS' storeset 1. If you create a series with a
tapegroup greater than 9999, then the storeset will be greater than 0
(storeset = tapegroup % 10000). The storeset controls which block of
/SUMXX gets used for that series. Currently, only storeset 0 and 1 are
used (1 means store on /SUM100, which is at LMSAL). 0 means store on
any of the other SUMS partitions.
- Add DRMSPGPORT to serverdefs.h. This is now a localizable define
that specifies the port to use on the PostgreSQL db used for DRMS.
- In the series publication code, print out the query that finds
long-running transaction in the email sent out warning about
long-running transactions blocking publication.
- Added perl module to the CVS tree - this module provides code that 
formats log output for use with anomaly-monitoring code.
- New SUMS API function - SUM_nop(). This can be called to check if
SUMS is running.
- Added "-T" flag for show_info. This causes the archive tape name and
file number to be printed for each record printed.
- Added support for additional housekeeping APIDs.

DEFECTS FIXED:
- Export all symbols from the lib DRMS library linked into DRMS
modules so that libdsds.so (a Stanford-specific library that
interfaces with DSDS code) can call into a couple of lib DRMS
functions. Run-time code using libdsds.so had started failing.
- Remove the trailing 'Z' in the DATE- and DATE__OBS-keyword values
when these keywords are exported. The trailing 'Z' is not allowed
according to the FITS-file standard.
- Ticket #319 - Modified drms_link_getpidx(DRMS_Record_t *rec) to
return a status code that is used by drms_insert_series(). If
drms_link_getpidx() fails - in the reported case, because the linked
series did not exist - then drms_insert_series() fails with an error.
- Fixed segmentation fault in jsoc_info that could happen if a 'bad
segment' link exists.
- Removed some memory leaks and memory corruption problems with one of
the export-utility library source files.
- Ticket #306 - Added case statements for ARG_DOUBLE and ARG_DOUBLES
to the function that returns the name (string) for each argument data
type.
- Fixed a problem with SUMLIB_Main_Update.pgc - there was an extra
declaration for SUM_Main_Update() which differed from the declaration
in sum_rpc.h. Removed the declaration in SUMLIB_Main_Update.pgc
- Fixed a problem in show_coverage where a failure occurred when the
user specified a key to use.
- Added missing return value from DoIt() function in show_series.
- Fixed problems with confidence and cleaned-up code in some
magnetic-pipeline modules.
- In delete_series, moved the call to delete SUs before the code that
deletes db tables. This way the db tables are not touched if there is
a failure deleting the SUs (and then the user can try delete_series
again).
- Improved handling of negative pixel values in the level1 despike
code.
- AIA level1 - Added code to set temperature and throughput (response)
related keywords.
- 

EXISTING DEFECTS:
- Please see http://jsoc.stanford.edu/trac/report/1 for a list and
description of most known bugs.
Revision:	1.1
Committed:	Wed Dec 15 19:42:18 2010 UTC (12 years, 9 months ago) by arta
Branch:	MAIN
CVS Tags:	Ver_6-0, Ver_6-1, Ver_6-2, Ver_6-3, Ver_6-4, NetDRMS_Ver_8-8, Ver_8-5, Ver_5-14, Ver_5-13, Ver_5-12, Ver_LATEST, NetDRMS_Ver_LATEST, NetDRMS_Ver_8-12, NetDRMS_Ver_8-10, NetDRMS_Ver_8-11, NetDRMS_Ver_9-1, NetDRMS_Ver_9-0, NetDRMS_Ver_9-3, NetDRMS_Ver_9-2, NetDRMS_Ver_9-5, NetDRMS_Ver_9-4, Ver_7-0, NetDRMS_Ver_9-41, Ver_9-41, NetDRMS_Ver_8-4, NetDRMS_Ver_8-5, NetDRMS_Ver_8-6, Ver_8-8, NetDRMS_Ver_8-7, Ver_8-2, Ver_9-3, Ver_8-0, Ver_8-1, Ver_8-6, Ver_8-7, Ver_8-4, Ver_8-11, Ver_7-1, Ver_9-1, Ver_8-3, Ver_9-5, Ver_9-4, Ver_8-10, Ver_9-2, Ver_8-12, Ver_9-0, HEAD
Log Message:	Release notes for JSOC release 5.12
#	Content
1	Release Notes JSOC V5.12 15DEC2010
2	------------------------ ---------
3
4
5	A release is a set of files, each having a specific version. And a release typcially
6	has a version number because over time you have newer and newer releases of the
7	same product. For example, a hypothetical 1.3 release may contain fileA#1.8,
8	fileB#1.2, fileC#2.2 and a 1.4 release may contain fileA#2.5, fileB#2.1, fileC#2.9.
9	JSOC releases are similarly versioned and contain a set of such files. JSOC release
10	code is guaranteed to compile on cluster nodes (eg., n00, n02). The resulting binaries
11	have been minimally tested. At the time of the creation of the release, the
12	release versions of each file will be the most recent. But as time passes, newer versions
13	of some files will be made, and there is no guarantee that these changes will
14	not destabilize JSOC (ie., they may cause JSOC to no longer compile or execute
15	properly).
16
17	There are several ways to use this release. If you wish to simply use pre-built
18	binaries, you can simply use the production binaries, which are located at
19	/home/production/cvs/JSOC. Every time a release is created, the binaries in
20	this location get updated. Only the production user can update these binaries.
21	So, you could run /home/production/cvs/JSOC/bin/linux_x86_64/show_keys, for example.
22	If instead you want to work with stable source files, then you must have a sandbox,
23	which is a local copy (in your home directory) of the files in the cvs depot.
24	You would probably want to work with a sandbox if you plan on making eventual
25	changes to the depot files. Changes you make to your sandbox files are not visible
26	to other users until you "commit" those changes back to the cvs depot. Please see
27	"If You Don't Have a Sandbox" below for more information on how to create a sandbox.
28	There is also a "working" release which resides in in /home/jsoc/cvs/JSOC. New
29	files may be placed here and existing files may be edited for common use before the
30	next official release. Each time a release gets created, the source and binaries of
31	the working release get updated. WARNING: the files you see here may not be stable
32	since by the time you see them, another user may have edited them. Only the production
33	release is guaranteed to be stable and unchanged between releases.
34
35	Obtaining the Release
36	---------------------
37	To update your working directory to this release, or to check-out this release anew,
38	please visit http://jsoc.stanford.edu/jsocwiki/CvsInit. Please keep in mind that
39	users may have modified files since the release was created, so use of the
40	scripts documented in the web page may result in a working directory whose
41	content is not identical to the release. If updating, you can supply
42	the flag "-R" to the jsoc_update.pl and jsoc_sync.pl scripts to download the
43	latest release. This will ensure that your working directory has the exact, latest
44	release versions of the files (eg., jsoc_sync.csh -R). If checking-out,
45	you can supply the argument "-r Ver_LATEST" to the "cvs checkout" command
46	to achieve the analogous result, but for a fresh checkout. WARNING: if you use
47	the "-R" or "-r" flags, please use only jsoc_update.pl or jsoc_sync.pl to update
48	your sources thereafter. Use of "-R" or "-r Ver_LATEST" will result in a cvs
49	"sticky flag" being set. jsoc_update.pl and jsoc_sync.pl clear this sticky flag.
50
51	Additional Info
52	---------------
53	If you are unfamiliar with the use of cvs see the file:
54	JSOC/CM/working_with_sandbox.txt.
55
56	There's a linux4 cvs gui at xim:/usr/bin/lincvs
57	Also on our jsoc web page:
58
59	http://jsoc.stanford.edu/cvs/JSOC/
60
61	Use the Apache cvs gui to see the diffs. For example, go to
62	http://jsoc.stanford.edu/cvs/JSOC/base/drms/
63	and click on the name in the File column and then click on
64	"diffs to previous #" to see the diffs.
65
66	Changes since previous release (V5.11 - Oct 6, 2010)
67	--------------------------------------------------------
68
69	NEW FEATURES:
70	- Moved all FITS-export code out of lib DRMS and into a new library in
71	JSOC/base/export/libs/exportDRMS.
72	- Added code to check for user permissions to read from the
73	_jsoc.sl_table. This table is accessed when delete_series is called to
74	determine if the series being deleted is being replicated (and we
75	disallow deletion of slony-replicated series).
76	- The COMMENT and HISTORY keywords are now exported into the FITS header
77	when a record being exported contains those keywords.
78	- Added 'RECNUM' to set of FITS keywords exported during export of FITS
79	files.
80	- Modified the jsd parser to print a warning if provided external
81	keyword name (in the keyword description) is RECNUM, since this will
82	collide with the auto-export of RECNUM.
83	- New module set_info - this is virtually identical to set_keys,
84	except that it will also copy FITS files into series that contain
85	FITS-protocol DRMS data segments.
86	- Added file-locking syncronization to publish_series.sh to disallow
87	multiple instances of publish_series.sh to run.
88	- Updated the flatfield code to a newer version.
89	- Added support for APID 44 to housekeeping-processing code.
90	- Added DATE keyword to housekeeping-processing code.
91	- Ticket #303 - Created editloggertabs.pl, a script to
92	create/delete/modify configuration tables used by monitoring/logging
93	code.
94	- Created newdrmsuser.pl, a script that facilitates new user creation;
95	it creates the default/appropriate entries in the db and it runs
96	masterlists.
97	- Modified the DRMS library code that interfaces with SUMS to attempt
98	re-connection in the event that the existing SUMS connection
99	vaporizes.
100	- Added a jsoc_main parameter, "--loomain", that causes modules to
101	loop attempting to connect to SUMS when the initial SUMS-connection
102	attempt failes. By default, if the initial SUMS-connection fails, then
103	modules do not attempt to connect again.
104	- Add DRMS support for SUMS' storeset 1. If you create a series with a
105	tapegroup greater than 9999, then the storeset will be greater than 0
106	(storeset = tapegroup % 10000). The storeset controls which block of
107	/SUMXX gets used for that series. Currently, only storeset 0 and 1 are
108	used (1 means store on /SUM100, which is at LMSAL). 0 means store on
109	any of the other SUMS partitions.
110	- Add DRMSPGPORT to serverdefs.h. This is now a localizable define
111	that specifies the port to use on the PostgreSQL db used for DRMS.
112	- In the series publication code, print out the query that finds
113	long-running transaction in the email sent out warning about
114	long-running transactions blocking publication.
115	- Added perl module to the CVS tree - this module provides code that
116	formats log output for use with anomaly-monitoring code.
117	- New SUMS API function - SUM_nop(). This can be called to check if
118	SUMS is running.
119	- Added "-T" flag for show_info. This causes the archive tape name and
120	file number to be printed for each record printed.
121	- Added support for additional housekeeping APIDs.
122
123	DEFECTS FIXED:
124	- Export all symbols from the lib DRMS library linked into DRMS
125	modules so that libdsds.so (a Stanford-specific library that
126	interfaces with DSDS code) can call into a couple of lib DRMS
127	functions. Run-time code using libdsds.so had started failing.
128	- Remove the trailing 'Z' in the DATE- and DATE__OBS-keyword values
129	when these keywords are exported. The trailing 'Z' is not allowed
130	according to the FITS-file standard.
131	- Ticket #319 - Modified drms_link_getpidx(DRMS_Record_t *rec) to
132	return a status code that is used by drms_insert_series(). If
133	drms_link_getpidx() fails - in the reported case, because the linked
134	series did not exist - then drms_insert_series() fails with an error.
135	- Fixed segmentation fault in jsoc_info that could happen if a 'bad
136	segment' link exists.
137	- Removed some memory leaks and memory corruption problems with one of
138	the export-utility library source files.
139	- Ticket #306 - Added case statements for ARG_DOUBLE and ARG_DOUBLES
140	to the function that returns the name (string) for each argument data
141	type.
142	- Fixed a problem with SUMLIB_Main_Update.pgc - there was an extra
143	declaration for SUM_Main_Update() which differed from the declaration
144	in sum_rpc.h. Removed the declaration in SUMLIB_Main_Update.pgc
145	- Fixed a problem in show_coverage where a failure occurred when the
146	user specified a key to use.
147	- Added missing return value from DoIt() function in show_series.
148	- Fixed problems with confidence and cleaned-up code in some
149	magnetic-pipeline modules.
150	- In delete_series, moved the call to delete SUs before the code that
151	deletes db tables. This way the db tables are not touched if there is
152	a failure deleting the SUs (and then the user can try delete_series
153	again).
154	- Improved handling of negative pixel values in the level1 despike
155	code.
156	- AIA level1 - Added code to set temperature and throughput (response)
157	related keywords.
158	-
159
160	EXISTING DEFECTS:
161	- Please see http://jsoc.stanford.edu/trac/report/1 for a list and
162	description of most known bugs.