Plotting EUVS Irradiance and Quality Flags - Short Example

This script demonstrates reading, analyzing and plotting the GOES-R EUVS irradiance and data quality flags. This example, using the GOES-16 EUVS-B 121 nm 1-minute average data from September 18, 2024, shows how to read a NetCDF file, parse the irradiance and data quality flag variables, and display both the irradiance and flags in a single plot.

First, several Python software libraries must be imported. These are used to read the EUVS NetCDF data files and manipulate the data in the files. Some syntax in this notebook is version-dependent. To view the version of each individual software package, run the following in a code cell: pip list

import netCDF4 as nc
import ncflag
# Import libraries to read the NetCDF data files and the data quality flags.

import numpy as np
# Import numpy.

import cftime
import datetime
# Import libraries used to read and plot the data timestamps.

import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import matplotlib.ticker as tck
# Import matplotlib and specific libraries used for formatting the tick marks on the x-axis of the plots.

import requests
import os
# Import libraries to download the data file.

Next, download the data file that is used in this code example. The file is downloaded to the current working directory; this is the same directory out of which this code is currently running.

local_directory = './'
euvs_filename = 'sci_euvs-l2-avg1m_g16_d20240918_v1-0-5.nc'
# Name of the EUVS file that is used in this notebook.

if os.path.exists(local_directory + euvs_filename):
    print('EUVS data file exists in working directory')
if not os.path.exists(local_directory + euvs_filename):
    print('EUVS data file does not exist in working directory; downloading file from the GOES-R data website')
    with open(local_directory + euvs_filename, "wb") as f:
        euvs_data_website = 'https://data.ngdc.noaa.gov/platforms/solar-space-observing-satellites/goes/' + \
                            'goes16/l2/data/euvs-l2-avg1m_science/2024/09/'
        r = requests.get(euvs_data_website + euvs_filename)
        f.write(r.content)
# Check if the EUVS data file exists in the local working directory. If the file does not exist in the local
# working directory, it will be downloaded to the local working directory from the GOES-R space weather data
# website.

EUVS data file does not exist in working directory; downloading file from the GOES-R data website

Finally, read in the EUVS data file. The file header and variable data descriptions can be printed out. These printouts contain important information about the contents of the data variables such as array size, values, types, etc.

euvs_data = nc.Dataset(local_directory + euvs_filename)
# Read in the EUVS data file.

print(euvs_data)

<class 'netCDF4.Dataset'>
root group (NETCDF4 data model, file format HDF5):
    Conventions: ACDD-1.3, Spase v2.2.6
    title: L2 EUVS 1 minute averages
    summary: The EXIS Extreme Ultraviolet Sensor (EUVS) measures seven line irradiances at extreme and far ultraviolet wavelengths as well as the Magnesium II index. This file contains measurements with a 1 minute average and associated flags.
    keywords: NumericalData.MeasurementType.Irradiance
    keywords_vocabulary: SPASE: Space Physics Archive Search and Extract Data Model version 2.2.6, GCMD: NASA Global Change Master Directory (GCMD) Earth Science Keywords version 8.5
    naming_authority: gov.nesdis.noaa
    history: See algorithm information.
    source: GOES-R EXIS/EUVS L1b
    processing_level: Level 2
    processing_level_description: Derived products
    license: These data may be redistributed and used without restriction.
    metadata_link:
    creator_name: Janet Machol
    creator_type: person
    creator_institution: DOC/NOAA/NESDIS/NCEI/OGSSD/STP
    creator_email: goesr.exis@noaa.gov
    creator_url: https://www.ncei.noaa.gov/
    institution: DOC/NOAA/NESDIS
    publisher_name: National Centers for Environmental Information
    publisher_type: institution
    publisher_institution: DOC/NOAA/NESDIS/NCEI
    publisher_email: goesr.exis@noaa.gov
    publisher_url: https://www.ncei.noaa.gov/
    references: GOES-R EUVS.03 Algorithm Theoretical Basis Document
    instrument: GOES R Series Extreme Ultraviolet Sensor and X-ray Irradiance Sensors (EXIS)
    instrument_id:
    program: Geostationary Operational Environmental Satellite, R-Series (GOES-R)
    project: Geostationary Operational Environmental Satellite R-Series (GOES-R) Core Ground Segment, Satellite Product Analysis and Distribution System (SPADES)
    time_coverage_resolution: PT1M
    processing_parameters_file:
    algorithm_parameters: None
    id: sci_euvs-l2-avg1m_g16_d20240918_v1-0-5.nc
    date_created: 2024-09-25T06:55:57.304Z
    orbital_slot: GOES-East
    platform: g16
    L1b_system_environment: SCI
    L1b_production_site: NCEI Boulder
    time_coverage_start: 2024-09-18T00:00:00.000Z
    time_coverage_end: 2024-09-19T00:00:00.000Z
    algorithm: cli-euvs03_avg
    algorithm_version: [1 4]
    algorithm_date: 2022-10-12
    input_system_environments: sci
    input_files_first: sci_exis-l1b-sfeu_g16_d20240918_v1-0-3.nc
    input_files_last: sci_exis-l1b-sfeu_g16_d20240918_v1-0-3.nc
    input_files_total: 1
    git_commit_hash: c35a9f2b69a9314c0b167013f11ad16f2bd27fcf
    L1b_LUT_Filenames: EUVSA_Cal_INR(fm1_CDRL79revU_tdep).h5 EUVSB_Cal_INR(fm1_CDRL79revU_tdep).h5 EUVSC_Cal_INR_(fm1_CDRL79revM_tdep).h5 EUVSPEC_Cal_INR_(fm1_CDRL79revJ_tdep).h5 Yearly_1AU_Correction_Table(2024)-757339200.0.h5
    dimensions(sizes): time(1440), lines(7), bounds(2), wavelength_bin(23)
    variables(dimensions): float64 time(time), float32 irr_256(time), float32 irr_284(time), float32 irr_304(time), float32 irr_1175(time), float32 irr_1216(time), float32 irr_1335(time), float32 irr_1405(time), float32 MgII_EXIS(time), float32 MgII_standard(time), uint8 irr_256_flag(time), uint8 irr_284_flag(time), uint8 irr_304_flag(time), uint8 irr_1175_flag(time), uint8 irr_1216_flag(time), uint8 irr_1335_flag(time), uint8 irr_1405_flag(time), uint8 MgII_flag(time), uint8 irr_256_flag_excluded(time), uint8 irr_284_flag_excluded(time), uint8 irr_304_flag_excluded(time), uint8 irr_1175_flag_excluded(time), uint8 irr_1216_flag_excluded(time), uint8 irr_1335_flag_excluded(time), uint8 irr_1405_flag_excluded(time), uint8 MgII_flag_excluded(time), uint8 irr_256_num(time), uint8 irr_284_num(time), uint8 irr_304_num(time), uint8 irr_1175_num(time), uint8 irr_1216_num(time), uint8 irr_1335_num(time), uint8 irr_1405_num(time), uint8 MgII_num(time), uint8 EUVS_C_active_channel(time), float32 au_factor(time), uint8 geocorona_flag(time), float32 wavelength_bounds(lines, bounds), float32 wavelength_lines(lines), float32 irr_284_1nm(time), float32 irr_304_1nm(time), float32 irr_1216_1nm(time), uint8 yaw_flip_flag(time), float32 model_wavelength_bounds(wavelength_bin, bounds), float32 model_irradiance_spectrum(time, wavelength_bin), uint8 model_case_flag(time)
    groups:

This example shows the irradiance and quality flags for the EUVS-B 121.6 nm (Lyman-Alpha) irradiance. This is the brightest line measured by the EUVS instruments.

The variables in the NetCDF file are accessed with the following syntax: file_variable[“variable_name”].

print(euvs_data["irr_1216"])
print('')
print(euvs_data["irr_1216_flag"])
print('')
print(euvs_data["geocorona_flag"])
print('')
print(euvs_data["time"])

<class 'netCDF4.Variable'>
float32 irr_1216(time)
    long_name: Average irradiance for 121.6-nm line.
    comments: Flare detection is currently not performed while the geocorona flag is set.
    units: W/m2
    _FillValue: -9999.0
    valid_min: 0.004
    valid_max: 0.2
unlimited dimensions: time
current shape = (1440,)
filling off

<class 'netCDF4.Variable'>
uint8 irr_1216_flag(time)
    long_name: Flag for irr_1216.
    _FillValue: 255
    flag_masks: [7 1 2 4]
    flag_values: [0 1 2 4]
    flag_meanings: good_quality eclipse lunar_transit bad_or_no_data
unlimited dimensions: time
current shape = (1440,)
filling off

<class 'netCDF4.Variable'>
uint8 geocorona_flag(time)
    long_name: Geocoronal flag.
    comments: Flag set if any point in the average is indicated as majorly impacted by the geocorona.
    _FillValue: 255
    flag_masks: [1 1]
    flag_values: [0 1]
    flag_meanings: none degraded_due_to_geocorona
unlimited dimensions: time
current shape = (1440,)
filling off

<class 'netCDF4.Variable'>
float64 time(time)
    long_name: Record start time. Neglects leap seconds since 2000-01-01.
    comments: Time[UTC] =1 Jan 2000 12:00:00[UTC] + time[secs] + n[secs] where n = {0/number of leap secs since 1 Jan 2000} for a conversion function that {ignores/includes} leap secs.
    units: seconds since 2000-01-01 12:00:00 UTC
    _FillValue: -9999.0
unlimited dimensions: time
current shape = (1440,)
filling off

The timestamps in the data file are in units of seconds since 12:00:00 UTC on January 1 2000. The Python function num2pydate, part of the cftime library, converts the timestamps from these units to datetime objects that are in YYYY-MoMo-DD-HH-MinMin-SS format.

euvs_timestamps = cftime.num2pydate(euvs_data["time"][:], euvs_data["time"].units)
# Convert timestamps to datetime objects.

The data quality flags are extracted using the ncflag library (https://github.com/5tefan/ncflag).

The quality flags are read from the “irr_1216_flag” and “geocorona_flag” NetCDF variables using ncflag.FlagWrap. The “geocorona_flag” variable indicates when the 121 nm irradiance is affected by geocoronal absoprtion. This wavelength is absorbed by the uppermost layers of Earth’s atmosphere. This absorption appears as a dip in the irradiance time series, similar to the irradiance during the eclipse.

euvsb_flags = ncflag.FlagWrap.init_from_netcdf(euvs_data["irr_1216_flag"])
geocorona_flag = ncflag.FlagWrap.init_from_netcdf(euvs_data["geocorona_flag"])
# Parse the quality flag variables using ncflag.

print('Number of indices at which the EUVSB quality flags are set:')
for i in euvsb_flags.flag_meanings:
# Loop over all indices in the flag_meanings array of the euvsb_flags object.

    flag_set = euvsb_flags.get_flag(i)
    # Determine indices where the current flag is and is not set. Every index is assigned a Boolean value:
    # True means the flag is set, False means the flag is not set.

    flag_frequency = np.count_nonzero(flag_set)
    print(f"{i:<14}: {flag_frequency}")
    # Print the number of indices at which each flag is set.

print('')
print('Number of indices at which the geocorona quality flag is set:')
for j in geocorona_flag.flag_meanings:
# Loop over all the indicies in the flag_meanings array of the geocorona_flag object.

    flag_set = geocorona_flag.get_flag(j)
    # Determine indices where the current flag is and is not set. Every index is assigned a Boolean value:
    # True means the flag is set, False means the flag is not set.

    flag_frequency = np.count_nonzero(flag_set)
    print(f"{j:<25}: {flag_frequency}")
    # Print the number of indices at which each flag is set.

Number of indices at which the EUVSB quality flags are set:
good_quality  : 1327
eclipse       : 20
lunar_transit : 0
bad_or_no_data: 93

Number of indices at which the geocorona quality flag is set:
none                     : 1079
degraded_due_to_geocorona: 361

There is no lunar transit on this day, but the ‘eclipse’ and ‘bad_or_no_data’ flags indicate there is an eclipse on this day. The ‘degraded_due_to_geocorona’ flag also indicates there is a period affected by geocoronal absorption.

The geostationary orbit of the GOES satellites means they experience 2 eclipse seasons, centered on the spring and fall equinoxes, each year. The spring eclipse season starts at the end of February, peaks in the middle of March and ends in early April. The fall eclipse season starts at the end of August, peaks in the middle of September and ends in early October. As seen by the satellite, the Earth eclipses the Sun each day during the eclipse season. The longest eclipse is 70 minutes.

The first and last days of the eclipse season can cause penumbra-only eclipses. During these events, the satellite is in the penumbra shadow of the eclipse, but not the umbra. The Sun is still partially visible, causing a dip in the irradiance as a function of time.

The ncflag.get_flag() function parses the flag bit values at each index to determine which flags are set.

geocorona_flag_set = geocorona_flag.get_flag('degraded_due_to_geocorona')
good_flag_set = euvsb_flags.get_flag('good_quality')
# The 'geocorona_flag_set' and 'good_flag_set' arrays contain Boolean values indicating if each flag is or is not
# set at a particular index.

Now that the quality flags and timestamps are defined, plots of the irradiance can be made.

euvsb_121_irrad = np.ma.filled(euvs_data["irr_1216"][:], np.nan)
# Define an array of the EUVS-B 121 nm irradiance values and replace the missing and fill values in the irradiance
# with NaN values. The missing and fill values in the NetCDF data array are masked indices. The np.ma.filled()
# code is a simple way to replace the masked values with NaNs, which excludes them from plotting and numerical
# operations.

plt.figure(figsize = (30, 30))
plt.plot(euvs_timestamps[good_flag_set], euvsb_121_irrad[good_flag_set], color = 'black', label = \
         r'$\lambda$ = 121 nm: good_quality_flag set', marker = '.', markersize = 16, linestyle = 'none')
plt.plot(euvs_timestamps[~geocorona_flag_set], euvsb_121_irrad[~geocorona_flag_set], color = 'dodgerblue', \
         label = r'$\lambda$ = 121 nm: geocorona_flag excluded', marker = '.', markersize = 10, linestyle = \
         'none')
plt.rc('font', size = 25)
plt.xlabel('Hour', fontsize = 30)
plt.ylabel('Irradiance (W/m$^2$)', fontsize = 30)
plt.gca().xaxis.set_major_locator(mdates.HourLocator())
plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%H'))
plt.gca().xaxis.set_minor_locator(mdates.MinuteLocator([15, 30, 45]))
plt.tick_params(axis = 'x', which = 'major', length = 18, width = 2, direction = 'in')
plt.tick_params(axis = 'x', which = 'minor', length = 8, width = 2, direction = 'in')
plt.tick_params(axis = 'y', which = 'major', length = 18, width = 2, direction = 'in')
plt.tick_params(axis = 'y', which = 'minor', length = 8, width = 2, direction = 'in')
plt.xticks(fontsize = 25, ha = 'center')
plt.yticks(fontsize = 25)
lg = plt.legend(loc = 'center right', fontsize = 30, numpoints = 1)
#lg.legendHandles[0]._legmarker.set_markersize(30) #If matplotlib version is < 3.7, use this line.
#lg.legendHandles[1]._legmarker.set_markersize(30) #If matplotlib version is < 3.7, use this line.
lg.legend_handles[0]._markersize  = 30 #If matplotlib version is 3.7 or newer, use this line.
lg.legend_handles[1]._markersize = 30 #If matplotlib version is 3.7 or newer, use this line.
plt.margins(0.02, 0.05)
plt.title('GOES-16 EUVS-B 121 nm Irradiance: ' + str(euvs_timestamps[0].year) + '-' + \
          str(euvs_timestamps[0].month) + '-' + str(euvs_timestamps[0].day), fontsize = 40, y = 1.02)
# Plot EUVS-B irradiance.

plt.twinx()
# Create a second x/y-axis pair to show the quality flags on a new y-axis on the right side of the plot.
# The x-axis is the same for both sets of data shown on the y-axes.

for i in euvsb_flags.flag_meanings:
# Loop over all EUVS-B quality flags.

    flag_set = euvsb_flags.get_flag(i)
    # Determine indices where current flag is and is not set.

    if np.any(flag_set):
        plt.plot(euvs_timestamps[flag_set], [i for _ in range(len(euvs_timestamps[flag_set]))], marker = 'x', \
                 linestyle = 'none', markersize = 10)
        plt.ylabel('Data Quality Flag ("x")', fontsize = 30)
        plt.yticks(fontsize = 20)
        plt.tick_params(axis = 'y', which = 'major', length = 12, width = 2, direction = 'in')
        plt.gca().xaxis.set_minor_locator(mdates.MinuteLocator([15, 30, 45]))
        plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%H'))
        plt.gca().xaxis.set_major_locator(mdates.HourLocator())
        # If the current flag is set anywhere, plot the flag at all timestamps where it is set.
plt.plot(euvs_timestamps[geocorona_flag_set], ['degraded_due_to_geocorona' for _ in \
         range(len(euvs_timestamps[geocorona_flag_set]))], marker = 'x', linestyle = 'none', markersize = 10)
# Plot quality flags on the right y-axis.

[<matplotlib.lines.Line2D object at 0x7f033136d550>]

This plot shows the following:

• The black dots are the irradiance at the indices that have the ‘good_quality’ flag set. This excludes the eclipse at 04:13-05:37.

• The blue dots are the irradiance at the indices that do NOT have the ‘degraded_due_to_geocorona’ flag set. This excludes the geocorona dip at 02:00-08:00. The plotting code uses the Boolean ‘~’ syntax to plot the non-geocorona irradiance, even though only the time period at which the geocorona flag is set is defined as an array.

• The colored ‘x’ on the right y-axis plot the data quality flags as a time series. The eclipse flag is set during the penumbra before and after the eclipse umbra. The no_data flag is set during the eclipse umbra, when EUVS cannot see the Sun. The geocorona flag is set during the entire geocorona absorption period, which overlaps with the eclipse.

• The exact timestamps at which each flag is set can be found by indexing the timestamp array with the flag arrays: good_timestamps = euvs_timestamps[good_quality_flag].

euvs_data.close()
# Close the open NetCDF file.