The processing of

USAF_MWL files

Conversion of Zürich/McIntosh classification as observed by the USAF observatories


On the 20th October 1997 Helen E Coffey of NOAA/NGDC/EC2 wrote me an e-mail telling me she would like to see the CV derived by the CV-Network compared to a CV calculated from the older files from USAF_MWL or from the SGD-tables.  They were kindly put on the server for me to download, and though the files were dated back to 1969, the first day containing the full three-letter code according to Patrick S. McIntosh was the 27th November 1971 and is then the first date in this tabulation.

The challenge was taken immediately, and the processing is explained below.

One year's file is opened by Microsoft Script Wizard in Microsoft Excel (version 5.0 in this case).  By text-functions the proper data like

            Observatory

            Date

             Zürich/McIntosh-classification

            Region number

proc1.gif (13622 bytes)

 

are pulled from the textcolumns and sorted by date (files originally sorted by region-number. This new Excel-file is saved as a new spreadsheet, after first having been sorted monthly into columns; i.e. four columns x 12 months.

 

The next macro pulls out each observatory's name by IAU-codes and pastes these into new columns divided into their appropriate IAU-coded columns.  They are saved as temporary monthly files.

proc2.gif (4939 bytes)

 

The third macro collects data from each observatory for each month of the year.   Some years there have been up to 7 observatories submitting their observations.   This may end up in 84 files plus the 12 monthly files, taking up upto 56 megabytes of disk space!  This is however temporary, as they are automatically deleted as soon as the annual results have been calculated and saved as a yearfile containing only the calculated CV-USAF numbers.  This again, is even saved in a deformulated file taking up only about 60K of disk space per year!

proc3.gif (4697 bytes)

proc4.gif (13766 bytes)

ERRORS

While processing these data, a large amount of non-existing classifications according to the system occurred.  They had to be replaced by the nearest logical classification in order to give the correct picture.  The replacements are as follows:

 

CV-USAF PROGRAMMING; REPLACED FORMULAS: CV-USAF PROJECT
Action
Found Classification '0RX',was replaced by Correct Classification 'HRX'
Found Classification '0XX',was replaced by Correct Classification 'AXX'
Found Classification 'A',was replaced by Correct Classification 'AXX'
Found Classification 'AAX',was replaced by Correct Classification 'AXX'
Found Classification 'ABX',was replaced by Correct Classification 'AXX'
Found Classification 'ACA',was replaced by Correct Classification 'AXX'
Found Classification 'ACR',was replaced by Correct Classification 'AXX'
Found Classification 'ACS',was replaced by Correct Classification 'AXX'
Found Classification 'ADA',was replaced by Correct Classification 'AXX'
Found Classification 'AHA',was replaced by Correct Classification 'AXX'
Found Classification 'AHR',was replaced by Correct Classification 'AXX'
Found Classification 'AHS',was replaced by Correct Classification 'AXX'
Found Classification 'AHX',was replaced by Correct Classification 'AXX'
Found Classification 'ARX',was replaced by Correct Classification 'HRX'
Found Classification 'AX',was replaced by Correct Classification 'AXX'
Found Classification 'AXI',was replaced by Correct Classification 'AXX'
Found Classification 'AXO',was replaced by Correct Classification 'AXX'
Found Classification 'AXO',was replaced by Correct Classification 'BXO'
Found Classification 'B',was replaced by Correct Classification 'BXO'
Found Classification 'BAI',was replaced by Correct Classification 'BXI'
Found Classification 'BAX',was replaced by Correct Classification 'AXX'
Found Classification 'BBX',was replaced by Correct Classification 'AXX'
Found Classification 'BGO',was replaced by Correct Classification 'BXO'
Found Classification 'BR',was replaced by Correct Classification 'CRO'
Found Classification 'BRI',was replaced by Correct Classification 'CRI'
Found Classification 'BRO',was replaced by Correct Classification 'CRO'
Found Classification 'BSI',was replaced by Correct Classification 'BXI'
Found Classification 'BSO',was replaced by Correct Classification 'BXO'
Found Classification 'BX',was replaced by Correct Classification 'BXO'
Found Classification 'BXC',was replaced by Correct Classification 'BXI'
Found Classification 'BXX',was replaced by Correct Classification 'AXX'
Found Classification 'C',was replaced by Correct Classification 'CAO'
Found Classification 'CA',was replaced by Correct Classification 'CAO'
Found Classification 'CA0',was replaced by Correct Classification 'CAO'
Found Classification 'CAC',was replaced by Correct Classification 'CAI'
Found Classification 'CAX',was replaced by Correct Classification 'AXX'
Found Classification 'CBX',was replaced by Correct Classification 'AXX'
Found Classification 'CR',was replaced by Correct Classification 'CRO'
Found Classification 'CRX',was replaced by Correct Classification 'HRX'
Found Classification 'CS',was replaced by Correct Classification 'CSO'
Found Classification 'CSX',was replaced by Correct Classification 'CSO'
Found Classification 'CXO',was replaced by Correct Classification 'BXO'
Found Classification 'DAX',was replaced by Correct Classification 'AXX'
Found Classification 'DHX',was replaced by Correct Classification 'HHX'
Found Classification 'DRC',was replaced by Correct Classification 'DRI'
Found Classification 'DS0',was replaced by Correct Classification 'DSO'
Found Classification 'DXO',was replaced by Correct Classification 'BXO'
Found Classification 'DXX',was replaced by Correct Classification 'AXX'
Found Classification 'EA',was replaced by Correct Classification 'EAO'
Found Classification 'EK',was replaced by Correct Classification 'EKO'
Found Classification 'EXX',was replaced by Correct Classification 'AXX'
Found Classification 'FK',was replaced by Correct Classification 'FKI'
Found Classification 'HA',was replaced by Correct Classification 'HAX'
Found Classification 'HAO',was replaced by Correct Classification 'CAO'
Found Classification 'HH',was replaced by Correct Classification 'HHX'
Found Classification 'HHC',was replaced by Correct Classification 'DHC'
Found Classification 'HHO',was replaced by Correct Classification 'HHX'
Found Classification 'HK',was replaced by Correct Classification 'HKX'
Found Classification 'HKO',was replaced by Correct Classification 'DKO'
Found Classification 'HR',was replaced by Correct Classification 'HRX'
Found Classification 'HRO',was replaced by Correct Classification 'CRO'
Found Classification 'HS',was replaced by Correct Classification 'HSX'
Found Classification 'HSC',was replaced by Correct Classification 'DSC'
Found Classification 'HSI',was replaced by Correct Classification 'CSI'
Found Classification 'HSO',was replaced by Correct Classification 'CSO'
Found Classification 'HX',was replaced by Correct Classification 'HAX'
Found Classification 'HXO',was replaced by Correct Classification 'BXO'
Found Classification 'HXX',was replaced by Correct Classification 'AXX'
Found Classification 'SO',was replaced by Correct Classification 'DSO'

During updating of CV-USAF files for 1997, 1st Quarter 1998 on the 4thAug98,
the following Replacements were added:

Found Classification 'ASO',was replaced by Correct Classification 'AXX'
Found Classification 'CXO',was replaced by Correct Classification 'CRO'
Found Classification 'DCX',was replaced by Correct Classification 'DRO'
Found Classification 'EAX',was replaced by Correct Classification 'EAO'
Found Classification 'CH',was replaced by Correct Classification 'CHO'
Found Classification 'AHX',was replaced by Correct Classification 'AXX'
Found Classification 'DA',was replaced by Correct Classification 'DAO'
Found Classification 'HRO',was replaced by Correct Classification 'HRX'
Found Classification 'ADR',was replaced by Correct Classification 'AXX'
Found Classification 'BCS',was replaced by Correct Classification 'BXO'
Found Classification 'DBX',was replaced by Correct Classification 'AXX'
Found Classification 'HKI',was replaced by Correct Classification 'HKX'
Found Classification 'HHO',was replaced by Correct Classification 'HHX'

The processing amount of files and bytes for the computer

FILE-PROCESSING USAF-PROJECT 1997: 
Converting USAF-Region Classifications
to CV after Malde	

DOWNLOADING		
FILE-TYPE				FILES	TOTAL BYTES
TEXT-FILES DOWNLOADED   SGD-TABLE	14	5 655 404
TEXT-FILES DOWNLOADED   USAF_MWL	18	13 907 201
		
T O T A L				32	19 562 605

THE PROCESSING		
FILE-TYPE				FILES	TOTAL BYTES
USAF-FILES				17	14 477 115
SREG-FILES + MACRO			186	565 297 152
SREG-MAIN MACRO				1	218 096
XXXX9601 MONTHLY FILE + MACRO		1	2 355 712
MONTHLY OBS. FILES			1331	337 328 640
TAB7181-FILE				1	1 181 184
CVTOT-FILE MACRO + SHEET		17	22 108 670
		
T O T A L				1554	942 966 569

FILES STORED		
FILE-TYPE				FILES	TOTAL BYTES
YEARLY TOTAL FILES			27	1 648 640
SOLAR CYCLE FILES			3	245 760
DAILY AVERAGES + 24wkSmCV		3	1 941 504
		
T O T A L				33	3 835 904

Processing Chart

zmcvusaf.gif (14068 bytes)

NOTE! THE PROCESSING HAS CHANGED CREATING CV-USAF FOR 1998
BY USING ONE EXCEL-FILE ONLY, MAX. APPROX. 4,5 MB, FINISHED DATA
ONLY 52K.
MORE LATER IN 1999...

CV-HN, K.I.Malde 1999-03-26

Project period: 07Nov97 - 15Dec97


Created: 26.03.99     İKjell Inge Malde    CV-Helios Network   e-mail: cv.helios@rl.telia.no