| 1 |
Release Notes JSOC V4.4 9Jun2008 |
| 2 |
----------------------- -------- |
| 3 |
|
| 4 |
A release is a set of files, each having a specific version. And a release typcially |
| 5 |
has a version number because over time you have newer and newer releases of the |
| 6 |
same product. For example, a hypothetical 1.3 release may contain fileA#1.8, |
| 7 |
fileB#1.2, fileC#2.2 and a 1.4 release may contain fileA#2.5, fileB#2.1, fileC#2.9. |
| 8 |
JSOC releases are similarly versioned and contain a set of such files. JSOC release |
| 9 |
code is guaranteed to compile on cluster nodes (eg., n00, n02). The resulting binaries |
| 10 |
have been minimally tested. At the time of the creation of the release, the |
| 11 |
release versions of each file will be the most recent. But as time passes, newer versions |
| 12 |
of some files will be made, and there is no guarantee that these changes will |
| 13 |
not destabilize JSOC (ie., they may cause JSOC to no longer compile or execute |
| 14 |
properly). |
| 15 |
|
| 16 |
There are several ways to use this release. If you wish to simply use pre-built |
| 17 |
binaries, you can simply use the production binaries, which are located at |
| 18 |
/home/production/cvs/JSOC. Every time a release is created, the binaries in |
| 19 |
this location get updated. Only the production user can update these binaries. |
| 20 |
So, you could run /home/production/cvs/JSOC/bin/linux_x86_64/show_keys, for example. |
| 21 |
If instead you want to work with stable source files, then you must have a sandbox, |
| 22 |
which is a local copy (in your home directory) of the files in the cvs depot. |
| 23 |
You would probably want to work with a sandbox if you plan on making eventual |
| 24 |
changes to the depot files. Changes you make to your sandbox files are not visible |
| 25 |
to other users until you "commit" those changes back to the cvs depot. Please see |
| 26 |
"If You Don't Have a Sandbox" below for more information on how to create a sandbox. |
| 27 |
There is also a "working" release which resides in in /home/jsoc/cvs/JSOC. New |
| 28 |
files may be placed here and existing files may be edited for common use before the |
| 29 |
next official release. Each time a release gets created, the source and binaries of |
| 30 |
the working release get updated. WARNING: the files you see here may not be stable |
| 31 |
since by the time you see them, another user may have edited them. Only the production |
| 32 |
release is guaranteed to be stable and unchanged between releases. |
| 33 |
|
| 34 |
Obtaining the Release |
| 35 |
--------------------- |
| 36 |
To update your working directory to this release, or to check-out this release anew, |
| 37 |
please visit http://jsoc.stanford.edu/jsocwiki/CvsInit. Please keep in mind that |
| 38 |
users may have modified files since the release was created, so use of the |
| 39 |
scripts documented in the web page may result in a working directory whose |
| 40 |
content is not identical to the release. If updating, you can supply |
| 41 |
the flag "-R" to the jsoc_update.pl and jsoc_sync.pl scripts to download the |
| 42 |
latest release. This will ensure that your working directory has the exact, latest |
| 43 |
release versions of the files (eg., jsoc_sync.csh -R). If checking-out, |
| 44 |
you can supply the argument "-r Ver_LATEST" to the "cvs checkout" command |
| 45 |
to achieve the analogous result, but for a fresh checkout. WARNING: if you use |
| 46 |
the "-R" or "-r" flags, please use only jsoc_update.pl or jsoc_sync.pl to update |
| 47 |
your sources thereafter. Use of "-R" or "-r Ver_LATEST" will result in a cvs |
| 48 |
"sticky flag" being set. jsoc_update.pl and jsoc_sync.pl clear this sticky flag. |
| 49 |
|
| 50 |
Additional Info |
| 51 |
--------------- |
| 52 |
If you are unfamiliar with the use of cvs see the file: |
| 53 |
JSOC/CM/working_with_sandbox.txt. |
| 54 |
|
| 55 |
There's a linux4 cvs gui at xim:/usr/bin/lincvs |
| 56 |
Also on our jsoc web page: |
| 57 |
|
| 58 |
http://jsoc.stanford.edu/cvs/JSOC/ |
| 59 |
|
| 60 |
Use the Apache cvs gui to see the diffs. For example, go to |
| 61 |
http://jsoc.stanford.edu/cvs/JSOC/base/drms/ |
| 62 |
and click on the name in the File column and then click on |
| 63 |
"diffs to previous #" to see the diffs. |
| 64 |
|
| 65 |
Changes since previous release (V4.3 - May 26, 2008) |
| 66 |
----------------------------------------------------- |
| 67 |
New Features: |
| 68 |
|
| 69 |
* TS_SLOT implementation. In this incarnation of slotted keywords, |
| 70 |
there is a new keyword, *_round, which determines where in slot 0 the |
| 71 |
epoch resides. If *_round is X seconds, then the epoch is X/2 seconds |
| 72 |
after the beginning of slot 0. The allows the user to refer to the |
| 73 |
first observtion of multiple series of differing cadence by the same |
| 74 |
date string. |
| 75 |
* When parsing time strings within record-set queries, if no time zone |
| 76 |
is specified, or if an invalid time zone is specified, the time zone |
| 77 |
in the keyword->info->unit field is assumed (unit is specified in the |
| 78 |
series' jsd). Also, support time strings that have the format |
| 79 |
YYYY.MM.DD_<time zone>. |
| 80 |
* Add code to verify that keyword format fields in the .jsd are |
| 81 |
compatible with the data type of the keyword. If an incompatibility |
| 82 |
is detected, a warning is printed, but the module will continue to run |
| 83 |
to completion. |
| 84 |
* Add drms_run script (from old jsoc source tree) to new JSOC source |
| 85 |
tree (drms_run was not copied over during the migration for unknown |
| 86 |
reasons). |
| 87 |
* In jsoc_info, added units to series_struct keyword table. |
| 88 |
* In show_info, added "-i" flag (input query?) to cause record query to be |
| 89 |
printed for each record. |
| 90 |
* In lookdata.html, add sections 7 and 8 for exporting data and minor fixes sections 1-6. |
| 91 |
* In level0 housekeeping configuration file, changed the location to |
| 92 |
which dayfiles are written to |
| 93 |
/tmp21/production/lev0_60d/hk_hsb_dayfile. |
| 94 |
* In level0 housekeeping configuration file, added new environment |
| 95 |
variables to handle writing to HK By APID data series for AIA, HMI, |
| 96 |
and SDO data series. |
| 97 |
* In Housekeeping decoding code, updated SDO_to_DRMS_time function to |
| 98 |
a static function and made this match Art's function in |
| 99 |
hmi_time_setting.c. Removed SDO_to_DRMS_time function from |
| 100 |
decode_hk_vcdu.c |
| 101 |
* The FDS cron script (dlfds.pl) now calls the orbit vector module |
| 102 |
(extract_fds_statev) after downloding and ingesting the FDS data |
| 103 |
files. |
| 104 |
* Don't add new data to fds_orbit_vectors series if the new data are |
| 105 |
equivalent to the existing data. Add code to create |
| 106 |
fds_orbit_ingesthist series if it doesn't exist. |
| 107 |
* In Level0 image decoding, allow compid = 128 (N,K,R=16,0,0 raw mode). |
| 108 |
* In level 0 ingestion code, print out status on system() error. |
| 109 |
* In write_hk_to_drms.c, added code to write to different HK by APID |
| 110 |
data series for HMI,AIA, and SDO. |
| 111 |
* In write_hk_to_drms.h, added defines for ranges of APIDs for HMI,AIA |
| 112 |
and SDO packets.These values are used by code to determine which HK by |
| 113 |
APID data series to write hk keywords to. |
| 114 |
* aia.lev0.jsd: Draft of aia level 0 data series to switch to after 60 day test. |
| 115 |
* hmi.lev0.jsd: Draft of jsd version to switch to after 60 day test. |
| 116 |
* In df_apid_ds_list_for_egsefm, added values to tell |
| 117 |
ingest_dayfile.pl script which dayfile data series to write dayfiles |
| 118 |
from egsefm. |
| 119 |
* In df_apid_ds_list_for_hsb, updated values for production. Used to |
| 120 |
tell ingest_dayfile.pl script which dayfile data series to write. |
| 121 |
* In df_apid_ds_list_for_moc, checked in mapping file which tells |
| 122 |
ingest_dayfile script which dayfile data series to save dayfile. |
| 123 |
* In df_apid_list_day_file_egsefm, list of apid to tell |
| 124 |
ingest_dayfile.pl script which to ingest into DRMS dayfile series for |
| 125 |
egsefm dayfiles. |
| 126 |
* In df_apid_list_day_file_moc, list of apids tells ingest_dayfile.pl |
| 127 |
script which dayfiles to load to drms dayfile data series. |
| 128 |
* In gdfdrms.pl, ingest_LM_dayfile.pl, ingest_hsb_dayfile.pl, added |
| 129 |
$ENV{'SUMSERVER'}="d02.Stanford.EDU". |
| 130 |
* In ingest_dayfile.pl: Updated script to ingest xml file and dayfile |
| 131 |
in data series. Currently does this for xml files from moc production |
| 132 |
server. |
| 133 |
* In ingest_dayfile.pl, updated script to write to DATE value in |
| 134 |
dayfile data series from string value(2008.05.29) to time value(i.e., |
| 135 |
2008.05.29_00:00:00.000_TAI) |
| 136 |
* In ingest_dayfile.pl, merge changes from ingest_LM_dayfile.pl and |
| 137 |
ingest_hsb_dayfile.pl into ingest_dayfile.pl. |
| 138 |
* Finalize FDS download scripts - put JSOC_DBNAME and JSOC_DBUSER into |
| 139 |
env var set code into top-level script, add seriesname parameter to |
| 140 |
fdsIngest.pl. |
| 141 |
* Added dlfds.pl, the new FDS cron script that reads a config file |
| 142 |
that specifies 'development' or 'ground' or 'live'. |
| 143 |
* Add jsd for sdo_ground.fds, and download config file for sdo ground moc products. |
| 144 |
* Allow the fds ingest script to accept a series name parameter, and make the default sdo.fds |
| 145 |
* Makefile and CVS modules file changes to accommodate new "cookbook" |
| 146 |
project which contains example modules. |
| 147 |
|
| 148 |
Bug Fixes |
| 149 |
* Remove memory leaks in drms_names.c |
| 150 |
* If a user specifies a TS_EQ keyword query that has a duration and |
| 151 |
that duration is less than one slot-width, then round up to one |
| 152 |
slot-width and print a warning. If the duration is not a multiple of |
| 153 |
the slot-width, then print a warning saying the duration is rounded. |
| 154 |
* Add libfitsrw.a to makefile rules for the meta-libraries. |
| 155 |
* Fix an error in the record-set query parsing code that invalidated |
| 156 |
queries that contained more than one sql query. |
| 157 |
* In modules that ingests DSDS series, fixed sign error in calculation of MJD. |
| 158 |
* In save_packet_to_dayfile.c, added fix in check_dfd_file() function |
| 159 |
where the file descriptor was never closed with closedir() and added |
| 160 |
closedir(dir_p) to load_dfd_node(). |
| 161 |
* In aia.lev0_60d.jsd, with new updates to short keywords, the |
| 162 |
create_series failed because of - in short keyword names. Fix missing |
| 163 |
comma on keyword line. |
| 164 |
* Add the script path prefix to the FDS download specification file, |
| 165 |
and remove the newline that gets added to the JSOC_MACHINE env |
| 166 |
variable value. |
| 167 |
|
| 168 |
Miscellaneous SUMS changes: |
| 169 |
* dont' set pgport for datacapture machines |
| 170 |
* add pgport setenv |
| 171 |
* add msg "Tape %s not in T50" |
| 172 |
* add msg Check tape_svc log |
| 173 |
* don't set pgport for datacapture machines |
| 174 |
* This has been replace by jsoc_sum db |
| 175 |
* define SUMSERVER on d02 for everybody |
| 176 |
* add getmsgimmed() for SUM_close() to clean up fd |
| 177 |
* remove OFFSITEHOST |
| 178 |
* add NC_PaRequest_AP_60d() |
| 179 |
* dont' scp md5 file to other machine. will be on local dcs machine only. |
| 180 |
* limit 2000 |
| 181 |
* new limit and no order by group_id |
| 182 |
* initial |
| 183 |
* take out pgport stuff. now set in sum_svc.c |
| 184 |
* add CLOSED != -2 for unassigned tape too |
| 185 |
* add != -2 to exclude cleaning tapes |
| 186 |
* change ds list window size |
| 187 |
|
| 188 |
Miscellaneous Data Capture changes: |
| 189 |
* update processing of .parc file |
| 190 |
* add -f to mv |
| 191 |
* change TOTAL_TLM_VCDU to TOTAL_TLM_IM_PDU |
| 192 |
* retry on .qac failure too |
| 193 |
* remove file afte success on retry |
| 194 |
* more msgs on retry |
| 195 |
* add retry and output stdout and stderr to file |
| 196 |
* use /usr/bin/scp |
| 197 |
* don't copy .qac file if scp of .tlm file fails |
| 198 |
* add printout at end of file |
| 199 |
* fix vcdu count |
| 200 |
* update for summary only |
| 201 |
|
| 202 |
Miscellaneous Level 0 changes: |
| 203 |
* add test0 |
| 204 |
* use sigalrmflg and execute code in main loop and not signal handler |
| 205 |
* In level 0 ingestion code, update series names. |
| 206 |
* In level 0 ingestion code, fix set_HMI_mech_values() call. |
| 207 |
* In level 0 ingestion code, different ds for hmi and aia and make fsn 30 bits. |
