CTD Processing Information

CalCOFI processes our CTD casts using the guidelines recommended by Seabird for our 911+ sensor array and depths. Please refer to the CTD Data Processing Protocol web page for the Seasoft modules applied. This page describes the in-house software used to merge CTD sensor data with bottle data, derive bottle corrections to apply to sensor data, & CTD data flagging. Also explained is the organization of data and metadata files distributed online as preliminary & final data products.
Files referenced: '.asc' & '.hdr' files are Seasoft-generated CTD data files. All other files referenced are SIO-CalCOFI data files generated at-sea and ashore from data acquistion & analysis software and SIO-CalCOFI-developed in-house software. SIO-CalCOFI's data processing methods, file formats, and metadata descriptions are available on the Data Formats & Methods web pages.

1A) BTLvsCTD¹ Step 1: (files required: sta.csvs with all bottle data columns populated; corrected YYMMcasts.csv, Seasoft-processed non-binavg CTD.asc & .hdr files). Preliminary bottle data ('YYMM###.csv' sta.csvs) are merged with 4-sec-ave (prior-to-bottle-closure) CTD .asc data, creating a single YY-YYMMSS_CTDCSV###-###.csv that is imported into Excel.

1B) Bottle vs CTD sensor data are plotted: primary CTD oxygen vs bottle oxygen ml/L & umol/kg; secondary CTD oxygen vs bottle ml/L & umol/Kg; FlV (usually V1) vs Chl-a; ISUSV (usually V6) vs NO3; CTD Salts > 350m are compared to bottle salts.

Three duplicate spreadsheets are pasted as new sheets to allow sorting and deletion of specific columns (or you can cut-paste specific columns to new sheets, if you prefer)

• 'Salts > 350m' - all data are sorted by Pr or Depth, Largest to smallest and all rows less than 350m deleted. Two columns are added: bottle salt - CTD Salt00 & bottle salt - CTD Salt 11. Mean, SD, Min, Max, & n are calculated at the bottom of the column. Any values above 0.02 are excluded. The mean values of the bottle minus primary & secondary salts become the offsets.
•  'Oxygen umol/kg' - for some reason leaving blank fields in the umol/Kg O2 columns can generate "wavy" scatter plots so deleting rows missing bottle oxygen data and missing CTD oxygen sensor data (unusual) is advised. "Fliers" are removed - either bad or questionable bottle or sensor data. Four plots total - Ox1ML vs OxB, Ox2ML vs OxB, Ox1UM vs OxB, Ox2UM vs OxB.
•  'ISUS vs NO3' - remove missing or 0 ISUS voltages rows and missing bottle NO3 rows then plot & add a trendline.
•  'Fluorometer voltage vs chl-a' plot can be generated from the main spreadsheet without issue. Or you can create a 'Chl-a' spreadsheet sorted by depth and delete all depths > 200m.

On all plots - all bottle-data fliers, such as mistrips or misdraws, are removed before trendlines added. PDF plots of all regressions are generated plus 400px & 1024px jpgs for posting online. Preliminary regressions and plots are posted since it can take a year for final data to become available. Once final, regression coefficients and plots are updated with new data.
1C) Seasoft 'Split' and 'Binavg' modules are applied to CTD.asc files, generating 1m binavg up & downcast asc files. It is important to verify any mislabeled CTD .hdr files have been corrected before step 2A to insure the matching sta.csv merges properly with CTD.asc. When there's a mismatch, it's usually caused by the cast number (###; aka 'Proc #' or 'Order Occupied') or CalCOFI Line Sta (eg 93.3 120.0) values being mis-entered when the cast starts. It is usually corrected at-sea but the .hdr correction may be lost if the .hex file has not been corrected. It requires the cast's .hex file be corrected using a binary editor (HxD Hex Editor) to stop the error from re-appearing when processed by Seasoft. If corrected, the original CTD cast .hex & .hdr files should be updated into the cast.zip (YYMM###.zip) & the cruise's YYMMSS_CTDCast.zip distributed online, & documented on the 'CTD Updates' web page.

1D) Inspect and correct YYMMcasts.csv & YYMMevents.csv. YYMMevents.csv will require "erase previous event" editing and Line & Sta verification. After YYMMcasts.csv and YYMMevents.csv have been manually corrected, run the DECODR² 'Clean Event Log' module to compare casts.csv order occupied numbering to the eventlog. (JLW usually checks & corrects the YYMMcasts.csv so coordinate with her to avoid duplicating effort).

Read more ...

CalCOFI deploys a 24-10L bottle CTD-Rosette on our quarterly cruises. The ability of CalCOFI to collect seawater samples at the same time as electronic sensor data allows CalCOFI to use bottle data to correct & calibrate CTD data and vice-versa. CalCOFI's hydrographic time-series is historically based on bottle sampling since 1949. But since 1993, the hydrographic time-series data has integrated CTD sensor data - all temperatures are typically from the primary CTD temperature sensor. Secondary temperature data may be used if the primary temperature data are problematic ie biofouled. Some bottle salinities, typically ISLs (interpolated standard levels) or missing bottle data, may be replaced by CTD sensor salinities. Most bottle oxygen ISL data or missing bottle data reported are CTD oxygen sensor data. Footnotes in the hydrographic data reports usually specify the source.

Our Seabird 911+ is configured with dual Temperature, Conductivity, and Oxygen sensors on separately plumbed & pumped sensor arrays. These are the only redundant sensors on our CTD; data from these sensors are labeled Temp1, Salt1, Ox1, & Temp2, Salt2, Ox2 respectively. Other single sensors include: a fluorometer (Wetlabs currently or Seapoint, Chelsea on older cruises), transmissometer (660nm, 25cm Wetlabs C-Star currently), Satlantic ISUS v3 Nitrate sensor, remote PAR & surface PAR (Biospherical), pH (Seabird SBE18), and altimeter (Benthos/Datasonics).

CTD sensor data that are bottle-corrected are: primary (Salt1) and secondary (Salt2) salinities; primary and secondary oxygens in both ml/L ((Ox1, Ox2) and uM/Kg (Ox1uM, Ox2uM); fluorometer estimated chlorophyll-a (EstChl); and ISUS estimated nitrate (EstNO3).

On the upcast, we collect seawater samples at ~20 depths and use the bottle values to calibrate certain sensors. CalCOFI typically deploys the CTD-Rosette to 515m, depth-permitting, or if shallower, to within ~10m of bottom. An altimeter is installed and used to determine height off the bottom. Weather-permitting, the wireout speed is 30m/min for the first 100m then 60m/min to terminal depth. The CTD-Rosette descends without stopping to the terminal depth, referenced as the downcast and annotated "D" in final data. Based primarily on the downcast fluorometer profile and mixed layer depth, the upcast will stop a specific depths to collect seawater samples. Ten meter bottle spacing is centered around the fluorometer (chlorophyll-a) peak. Other depths are based on historical "standard' level depths such as 10m, 50m, and 100m. Please note that there are two basin stations, Santa Monica Basin & Santa Barbarba Basin, where the CTD-Rosette is deployed deeper than 515m, to within 10m of bottom. Winter & Spring CalCOFI cruise CTD casts on Line 66.7 (MBARI's SECRET line) are often to 1000m.

Bottle-correcting Salinities: A salinity offset is calculated by comparing bottle salinities from depths greater than 350m, where the slope is near-vertical, to CTD primary (Salt1) and secondary (Salt2) salinities values. The offsets are applied to all the CTD salts and reported as Salt1_Corr & Salt2_Corr. The SaltAve_Corr is the average of these two columns. Non-corrected salinities are included in the CTD data csvs as Salt1 & Salt2.

Bottle-correcting Oxygens: Seabird SBE43 oxygens sensors often over-estimate the oxygen surface value near-surface during the downcast. Soaking the CTD near-surface for several minutes improves this issue but using bottle vs CTD oxygen regression coefficients work well. CalCOFI applies two bottle corrections to CTD primary and secondary oxygen sensor data:

1. Cruise-corrected: four-second averages of CTD oxygen data are plotted versus the oxygen bottle data for the entire cruise. "Four-seconds-prior-to-bottle-closure" average CTD oxygen data  is calculated using 96 records (24 records/sec x 4 secs) of oxygen data before the bottle closure confirmation scan number. Seasoft collects CTD data at 24 scans/sec and each scan (data record) is numbered chronologically, starting at 1. When a bottle is tripped and the deck unit receives confirmation the bottle has fired and closed successfully, the current scan number is recorded in the data files. Two scan values are recorded - one when the bottle trip is initiated and one when the bottle closure confirmation is received. These are tabulated for every bottle closure in the .bl file for each cast as well as embedded in the CTD data file (.hex or .dat).
   SIO-CalCOFI developed software (BtlVsCTD.exe) parses the bottle confirmation scan number for each bottle from within the .bl file. Then averages 96 (4-secs) scans of data prior to the bottle closure and merges it into a csv with the matching depth bottle values. Note - the "4-sec ave prior to bottle closure" criteria for sensor data is SIO-CalCOFI's method of comparing a consistent number of records to bottle data. The number of records average in 1m binavg data is variable. Depending on the total time for a CTD-rosette stop and bottle closure, averaging 200 to 2000+ records.
2. Station-corrected: sensor behavior can vary from station to station particularly if the seawater temperature changes significantly. Although cruise-corrected CTD oxygen sensor data are very good, station-corrected CTD oxygens are consider the best since it accounts for variability in sensor behavior. Our software program (BtlVsCTD), compares the bottle oxygen data from the individual cast to 4-sec averaged CTD oxygen data. A regression is generated from this comparison and correction coefficients are applied dynamically to bottle-correct the CTD sensor oxygens. "Dynamically' means each station will have slightly different correction coefficients, which are tabulated in a metadata file.
   The correction coefficients are applied to all 1m binavg CTD oxygen data and the bottle vs continuous data are plotted for both primary & secondary sensors. These plots are available in the CTD data files, in the csvs-plots sub-directory. All regression coefficients including the dynamic regressions derived for each cast for both primary and secondary oxygens are tabulated in the metadata coefficients file.

Estimated Chlorophyll: quantifying chlorophyll from fluorometer data is challenging due to variability in the state of the phytoplankton. The primary use of the CTD fluorometer is to identify the depth of the chlorophyll maximum to target bottle seawater sampling depths. Since CalCOFI typically collects and analyzes over 1000 chlorophyll samples per cruise. We compare discreet chlorophyll measurements vs fluorometer voltages (4-sec averages prior to bottle closure). The regression coefficients generated are then applied to fluorometer voltage to estimate chlorophyll. As with CTD oxygens, both cruise-corrected and station-corrected values are calculated with station-corrected estimated chlorophyll-a considered the best.

Estimated Nitrate: the Satlantic MBARI-ISUS nitrate sensor is deploy on the CTD-rosette in vertical orientation and battery powered. As with chlorophyll, since CalCOFI typically collects and analyzes over 1000 nutrient samples per cruise. We compare discreet nitrate measurements vs ISUS voltages (4-sec averages prior to bottle closure). The regression coefficients calculated are applied to ISUS voltage to estimate nitrate. Both cruise-corrected and station-corrected values are calculated with station-corrected considered the best.
Note: it was due to the "burn-in" drift caused by a new ISUS lamp that necessitated the development of the station-corrected regression method. Originally, only the cruise-correction coefficients were derived and applied to ISUS voltages. But when an ISUS with new lamp was deployed, the ISUS estimated-nitrate was (initially) severly underestimated. As the lamp "burned-in" over the first 10 casts, the NO3 vs voltage relationship changed. It took ~8 hours of "burn-in" for the ISUS voltage vs NO3 relationship to stabilize.Addressing station-to-station variability necessitated the implementation of station-corrected regression methodology. This bottle-correction has been applied to oxygen and fluorometer sensors as well. The station-correction method only works on CTD stations where bottle samples are taken. On some cruises, stations north of Pt Conception may have a reduced number or no bottle samples taken so station-corrected CTD data will not be available. Bottle-corrected CTD data, using cruise-average coefficients derived from southern stations, are usually available.

Transmissometer: a variety of transmissometers have been deployed on our CTD-rosette since 1993. For the last decade, a Wetlabs C-Star 660nm 25cm transmissometer has been our primary unit. Standard practice is to calculate updated M & B coefficients by doing a deck test once the CTD is terminated. A dark voltage and in-air voltage are measured then M & B coefficient derived using a transmissometer Excel spreadsheet template. M & B are entered into the Seabird .xmlcon file before the first cast. 1m binavg beam attenuation coefficient and % transmission data are derived using standard Seabird processing modules. These data are included in the CTD data csvs but no additional corrections are applied or further processing performed.

pH: although CalCOFI has deployed a Seabird SBE18 pH sensor on our CTD-rosette since 2009, no bottle-corrections have been applied. DIC-pH samples have been collected but a cross-check or bottle-sample calibration routine has not been implemented yet.

Data available as cruise-corrected and station-corrected are: primary & secondary oxygen data in ml/L & uM/Kg, ISUS estimated nitrate data, and fluorometer estimated chlorophyll. Seasoft-processed, non-bottled corrected CTD data are also reported along with other CTD physical measurements such as density, potential temperature, specific-volume anomaly, dynamic height, and other nutrients - silicate, nitrite, phosphate, and ammonia.