Basics of netCDF files
% Visit a link, any suggestions?
Locating a data file on the internet
To locate a netCDF data file, browse to an OPeNDAP website, as e.g. the OpenEarth website:
Find the Jarkus netCDF file by clicking on the following links: HYRAX ==> Rijkswaterstaat ==> JarKus ==> transects. Click on the link to the file and extract the direct link to the JarKus netCDF data file.
url = 'http://OPeNDAP .deltares.nl:8080/OPeNDAP/rijkswaterstaat/jarkus/profiles/transect.nc';
For some frequently used filetypes a dedicated funtion exists. This has the advantage that it returns the link to the netCDF file on the Deltares network if it is available. This is faster than accessing data over the internet.
url = jarkus_url;
Interacting with a netCDF file
To let matlab talk with a netCDF file, the snc toolbox has been developed. See what is in there:
help snctools
SNCTOOLS (<a href="http://mexcdf.sourceforge.net/">http://mexcdf.sourceforge.net/</a>).
SNCTOOLS is a package of simple tools for manipulating netCDF files, or
at least it started out that way. The focus used to be just on reading
and writing netCDF files, but can also handle the following formats under
certain conditions.
HDF4: can read and write datasets via MATLAB's HDFSD interface.
netCDF4, GRIB2, OPeNDAP: can read datasets via netcdf-java. For GRIB2
please check the README.
Note that by default, SNCTOOLS will transpose data ordering and
dimensions for the sake of backwards compatibility. This behavior can
be controlled with the PRESERVE_FVD preference.
-------------------------------------------------------------------------
NC_ADDHIST - Either appends or constructs a history attribute.
NC_ADDDIM - Adds a dimension to an existing netcdf file
NC_ADDRECS - Adds records to unlimited-dimension file.
NC_ADDNEWRECS - Adds monotonically increasing records to file.
NC_ADDVAR - Adds a variable to a NetCDF file.
NC_ATTGET - Get the values of a NetCDF attribute.
NC_ATTPUT - Put an attribute into a netcdf file.
NC_CAT_A - Concatentates set of files into ascending order.
NC_CREATE_EMPTY - Creates an empty netCDF file.
NC_DATATYPE_STRING - Convert enumerated datatype to string.
NC_DIFF - Determines if two NetCDF files contain same data.
NC_DUMP - Matlab counterpart to the NetCDF utility 'ncdump'.
NC_GETALL - Read the entire netCDF contents into structure
NC_GETDIMINFO - Returns metadata about a specific netCDF dimension.
NC_GETLAST - Get last few datums from unlimited NetCDF variable.
NC_GETVARINFO - Returns metadata about a specific NetCDF variable.
NC_INFO - Information about a netCDF file.
NC_ISCOORDVAR - Yes if the given variable is a coordinate variable.
NC_ISUNLIMITEDVAR - Yes if the given variable has a record dimension.
NC_ISATT - Yes if the given attribute is in the file.
NC_ISVAR - Yes if the given variable is in the netcdf file.
NC_PADHEADER - Pads the metadata header of a netcdf file.
NC_VARGET - Retrieve data from a NetCDF variable.
NC_VARPUT - Write data into a NetCDF file.
NC_VARRENAME - Renames a NetCDF variable.
SNC2MAT - Saves netcdf file to *.mat format
Important functions are nc_varget and nc_dump
help nc_dump
NC_DUMP: a Matlab counterpart to the netCDF utility 'ncdump'.
NC_DUMP(NCFILE) prints metadata about the netCDF file NCFILE.
NC_DUMP(NCFILE,VARNAME) prints metadata about just the one netCDF
variable named VARNAME.
NC_DUMP(NCFILE,<VARNAME>,fid) prints output to file opened
with fid = fopen(...) instead of to screen (default fid=1: screen).
NC_DUMP(NCFILE,VARNAME,<'fname'>) prints output to new file 'fname'.
If the preference 'USE_JAVA' is set to true and netcdf-java is on the
javaclasspath, NC_DUMP can also display metadata for GRIB2 files and
OPeNDAP URLS files as if they were netCDF files.
Setting the preference 'PRESERVE_FVD' to true will compel MATLAB to
display the dimensions in the opposite order from what the C utility
ncdump displays.
Example: This example file is shipped with R2008b.
nc_dump('example.nc');
Example: Display metadata for an OPeNDAP URL. This requires the
netcdf-java backend.
url = 'http://coast-enviro.er.usgs.gov/models/share/balop.nc';
nc_dump(url);
See also nc_info.
help nc_varget
NC_VARGET: Retrieve data from a netCDF variable.
DATA = NC_VARGET(NCFILE,VARNAME) retrieves all the data from the
variable VARNAME in the netCDF file NCFILE.
DATA = NC_VARGET(NCFILE,VARNAME,START,COUNT) retrieves the contiguous
portion of the variable specified by the index vectors START and
COUNT. Remember that SNCTOOLS indexing is zero-based, not
one-based. Specifying a -1 in COUNT means to retrieve everything
along that dimension from the START coordinate.
DATA = NC_VARGET(NCFILE,VARNAME,START,COUNT,STRIDE) retrieves
a non-contiguous portion of the dataset. The amount of
skipping along each dimension is given through the STRIDE vector.
A '_FillValue' attribute is honored by flagging those datums as NaN.
A 'missing_value' attribute is honored by flagging those datums as
NaN. The data is returned as double precision.
If the named NetCDF variable has valid scale_factor and add_offset
attributes, then the data is scaled accordingly. The data is returned
as double precision.
Preferences
-----------
PRESERVE_FVD - Setting the preference 'PRESERVE_FVD' to true will
compel NC_VARGET to preserve the fastest varying
dimension, i.e. NC_VARGET will not transpose the data.
This flips the order of the dimension IDs from what
one would see by using the ncdump C utility. This may
result in a performance boost when working with large
data.
USE_STD_HDF4_SCALING
- The CF conventions differ from HDF4 in how scale
factors are to be applied; check their respective
references for details. By default, HDF4 datasets are
scaled according to CF conventions, but setting this
preference to true forces NC_VARGET to use standard
HDF4 conventions.
EXAMPLE: This example file is shipped with R2008b.
data = nc_varget('example.nc','peaks',[0 0], [20 30]);
EXAMPLE: Retrieve data from a URL. This requires the netcdf-java
backend.
url = 'http://coast-enviro.er.usgs.gov/models/share/balop.nc';
data = nc_varget(url,'lat_rho');
Example: Retrieve data from the example HDF4 file.
data = nc_varget('example.hdf','Example SDS');
See also: nc_varput, nc_attget, nc_attput, nc_dump.
View metadata
We can get data from this file using the function nc_varget. But first, let's see what is in the file using nc_dump. nc_dump shows all the metadata in the file. In the case of the JarKus file this is a lot.
nc_dump(url)
netCDF p:\mcdata\opendap\rijkswaterstaat\jarkus\profiles\transect.nc {
dimensions:
time = UNLIMITED ; (44 currently)
alongshore = 2178 ;
cross_shore = 1925 ;
stringsize = 100 ;
variables:
// Preference 'PRESERVE_FVD': false,
// dimensions consistent with ncBrowse, not with native MATLAB netcdf package.
int32 id(alongshore), shape = [2178]
id:long_name = "identifier"
id:comment = "sum of area code (x1000000) and alongshore coordinate"
int32 areacode(alongshore), shape = [2178]
areacode:long_name = "area code"
areacode:comment = "codes for the 15 coastal areas as defined by rijkswaterstaat"
char areaname(alongshore,stringsize), shape = [2178 100]
areaname:long_name = "area name"
areaname:comment = "names for the 15 coastal areas as defined by rijkswaterstaat"
double alongshore(alongshore), shape = [2178]
alongshore:long_name = "alongshore coordinate"
alongshore:units = "m"
alongshore:comment = "alongshore coordinate within the 15 coastal areas as defined by rijkswaterstaat"
double cross_shore(cross_shore), shape = [1925]
cross_shore:long_name = "cross-shore coordinate"
cross_shore:units = "m"
cross_shore:comment = "cross-shore coordinate relative to the rsp (rijks strand paal)"
double time(time), shape = [44]
time:standard_name = "time"
time:axis = "T"
time:units = "days since 1970-01-01 00:00 +1:00"
time:comment = "measurement year see bathy and time_topo for more details"
int32 crs([]), shape = [1]
crs:grid_mapping_name = "Oblique Stereographic"
crs:semi_major_axis = 6.3774e+006
crs:semi_minor_axis = ""
crs:inverse_flattening = 299.153
crs:latitude_of_projection_origin = 52.1562
crs:longitude_of_projection_origin = 5.38764
crs:false_easting = 155000
crs:false_northing = 463000
crs:scale_factor_at_projection_origin = 0.999908
double x(alongshore,cross_shore), shape = [2178 1925]
x:standard_name = "projection_x_coordinate"
x:units = "m"
x:axis = "X"
double y(alongshore,cross_shore), shape = [2178 1925]
y:standard_name = "projection_y_coordinate"
y:units = "m"
y:axis = "Y"
double lat(alongshore,cross_shore), shape = [2178 1925]
lat:standard_name = "latitude"
lat:units = "degree_north"
lat:axis = "X"
double lon(alongshore,cross_shore), shape = [2178 1925]
lon:standard_name = "longitude"
lon:units = "degree_east"
lon:axis = "Y"
double angle(alongshore), shape = [2178]
angle:long_name = "angle of transect"
angle:units = "mradian"
angle:comment = "positive counter clockwise 0 east"
double mean_high_water(alongshore), shape = [2178]
mean_high_water:long_name = "mean high water"
mean_high_water:units = "m"
mean_high_water:comment = "mean high water relative to nap"
double mean_low_water(alongshore), shape = [2178]
mean_low_water:long_name = "mean low water"
mean_low_water:units = "m"
mean_low_water:comment = "mean low water relative to nap"
int32 max_cross_shore_measurement(time,alongshore), shape = [44 2178]
max_cross_shore_measurement:long_name = "Maximum cross shore measurement index"
max_cross_shore_measurement:comment = "Index of the cross shore measurement (0 based)"
int32 min_cross_shore_measurement(time,alongshore), shape = [44 2178]
min_cross_shore_measurement:long_name = "Minimum cross shore measurement index"
min_cross_shore_measurement:comment = "Index of the cross shore measurement (0 based)"
double rsp_x(alongshore), shape = [2178]
rsp_x:long_name = "location for beach pole"
rsp_x:units = "m"
rsp_x:axis = "X"
rsp_x:comment = "Location of the beach pole (Rijks strand paal)"
double rsp_y(alongshore), shape = [2178]
rsp_y:long_name = "location for beach pole"
rsp_y:units = "m"
rsp_y:axis = "Y"
rsp_y:comment = "Location of the beach pole (Rijks strand paal)"
double rsp_lat(alongshore), shape = [2178]
rsp_lat:long_name = "location for beach pole"
rsp_lat:units = "degrees_north"
rsp_lat:comment = "Location of the beach pole (Rijks strand paal)"
double rsp_lon(alongshore), shape = [2178]
rsp_lon:long_name = "location for beach pole"
rsp_lon:units = "degrees_east"
rsp_lon:comment = "Location of the beach pole (Rijks strand paal)"
double time_topo(time,alongshore), shape = [44 2178]
time_topo:long_name = "measurement date of topography"
time_topo:units = "days since 1970-01-01 00:00 +1:00"
time_topo:comment = "Measurement date of the topography"
double time_bathy(time,alongshore), shape = [44 2178]
time_bathy:long_name = "measurement date of bathymetry"
time_bathy:units = "days since 1970-01-01 00:00 +1:00"
time_bathy:comment = "Measurement date of the bathymetry"
int16 origin(time,alongshore,cross_shore), shape = [44 2178 1925]
origin:long_name = "measurement method"
origin:comment = "Measurement method {1:"..", 3:"...", 5:"...."} used short for space considerations"
double altitude(time,alongshore,cross_shore), shape = [44 2178 1925]
altitude:standard_name = "surface_altitude"
altitude:units = "m"
altitude:comment = "altitude above geoid (NAP)"
altitude:coordinates = "lat lon"
altitude:grid_mapping = "crs"
altitude:_FillValue = -9999
//global attributes:
:title = "Jarkus Data"
:institution = "Rijkswaterstaat"
:source = "on shore and off shore measurements"
:history = "Data received from Rijkswaterstaat, converted to netCDF on 23-Jul-2009"
:references = "Original source: http://www.watermarkt.nl/kustenzeebodem/Deltares storage: https://repos.deltares.nl/repos/mcdata/trunk/jarkus/Converted with script with $Id: jarkus_grid2netcdf.m 699 2009-07-17 07:54:10Z boer_g $"
:Conventions = "CF-1.4"
:spatial_ref = "COMPD_CS["Amersfoort / RD New + NAP",PROJCS["Amersfoort / RD New",GEOGCS["Amersfoort",DATUM["Amersfoort",SPHEROID["Bessel 1841",6377397.155,299.1528128,AUTHORITY["EPSG","7004"]],TOWGS84[565.04,49.91,465.84,-0.40939438743923684,-0.35970519561431136,1.868491000350572,0.8409828680306614],AUTHORITY["EPSG","6289"]],PRIMEM["Greenwich",0.0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.017453292519943295],AXIS["Geodetic latitude",NORTH],AXIS["Geodetic longitude",EAST],AUTHORITY["EPSG","4289"]],PROJECTION["Oblique Stereographic",AUTHORITY["EPSG","9809"]],PARAMETER["central_meridian",5.387638888888891],PARAMETER["latitude_of_origin",52.15616055555556],PARAMETER["scale_factor",0.9999079],PARAMETER["false_easting",155000.0],PARAMETER["false_northing",463000.0],UNIT["m",1.0],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","28992"]],VERT_CS["Normaal Amsterdams Peil",VERT_DATUM["Normaal Amsterdams Peil",2005,AUTHORITY["EPSG","5109"]],UNIT["m",1.0],AXIS["Gravity-related height",UP],AUTHORITY["EPSG","5709"]],AUTHORITY["EPSG","7415"]]"
}
From the metadata we can see that there is a field 'id'. To get this data, use nc_varget.
id = nc_varget(url,'id');
Now id contains the id's:
id(1:10)
ans =
2000100
2000101
2000102
2000103
2000104
2000105
2000106
2000120
2000140
2000160
Zero based indexing
TODO
Stride
TODO
netCDF viewers
TODO