polyadcirc.pyGriddata package¶

Submodules¶

polyadcirc.pyGriddata.file_management module¶

This module handles the setting up of landuse classfication folders, bash scripts, and the cleaning of these folders after Griddata_v1.32.F90 has been run in each of the folders.

polyadcirc.pyGriddata.file_management.copy(src, dst)¶

If file !exists then copy src into dest

Parameters:	src (string) – source file name dst (string) – destination file name

polyadcirc.pyGriddata.file_management.mkdir(path)¶

If path !exists then os.makedirs()

Parameters:	path (string) – path of directory to create

polyadcirc.pyGriddata.file_management.remove(files)¶

Remover one or more files or directories

Parameters:	files (string) – Path of files or directories to remove

Created on Wed Jan 15 02:44:47 2014

@author: pkjain

polyadcirc.pyGriddata.file_management.rename13(dirs=None, basis_dir=None)¶

Renames all *.13 files in dirs to fort.13

Parameters:	dirs (list) – list of directory names

polyadcirc.pyGriddata.file_management.symlink(src, dst, overwrite=1)¶

Creates a symbolic link pointing to src named dst. Overwrites existing links by default.

Parameters:	src (string) – source dst (string) – destination overwrite (binary) – 0 - do not overwrite, 1 - overwrite links, 2 - overwrite files and links, 3 - overwrite directories, files and links

polyadcirc.pyGriddata.grid_management module¶

This module contains the class definition for grid_mamangement.

polyadcirc.pyGriddata.grid_management.compare(basis_dir=None, default=0.012)¶

Create a set of diagnostic plots in basis_dir/figs

Parameters:	basis_dir (string) – directory containing the test folder and landuse folders default (float) – default Manning’s n

class polyadcirc.pyGriddata.grid_management.gridInfo(basis_dir, grid_dir, gap_data_list, flag=1, file_name='fort.14', table_folder=None, executable_dir=None)¶

Bases: polyadcirc.pyADCIRC.basic.pickleable

This class contains references to the fort.14, *.asc, and *.table files specific to a particular grid.

cleanup_landuse_folder(folder_name=None)¶

Removes all files in folder_name(or current directory) except .table and the most recent ``.14``.

Parameters:	folder_name (string) – path to folder to clean up relative to self.base_dir

cleanup_landuse_folders()¶

Removes all files except *.table and the most recent *.14 in all landuse folders in the current directory.

convert(folder_name, keep_flags=0)¶

convert() where source is the final *.14 file produced by the bash script associated with self in folder_name

Parameters:	folder_name (string) – name of the folder containing the *.14 to be converted

See convert()

create_bash_script(folder_name)¶

Creates bash script called grid_file.sh in self.basis_dir or cwd if folder_name==None to run Griddata in order on all the *.in file associated with this grid

Parameters:	folder_name (string) – folder for which to create bash scripts

create_griddata_input_files(folder_name)¶

Creates a series of *.in files named in order to be run by ./Griddata_parallel -I griddata_##.in

Parameters:	folder_name (string) – folder for which to create input files

get_dict_of_unique_tables()¶

Return type:	dict
Returns:	a dict of all the unqiue *.table files required by grid_all_*.sh where unique_tables.keys() = table_file_name, and unique_tables.values() = tableInfo object

get_landclasses()¶

Return type:	list
Returns:	landclasses[i] = (class_num, table_file_name)

prep_all(removeBinaries=False, class_nums=None, condense=True, TOL=None)¶

Assumes that all the necessary input files are in self.basis_dir. This function generates a landuse_## folder in self.basis_dir for every land classification number containing a fort.13 file specific to that land classification number.

Todo

Update so that landuse folders can be prepped n at a time and so that this could be run on a HPC system

Currently, the parallel option preps the first folder and then all the remaining folders at once.

Parameters:

parallel (binary) – Flag whether or not to simultaneously prep landuse folders. removeBinarues (binary) – Flag whether or not to remove *.asc.binary files when completed. class_nums (list) – List of integers indicating which classes to prep. This assumes all the *.asc.binary files are already in existence. condense (bool) – Flag whether or not to condense fort.13 to only non-zero values within a tolerance. TOL (double) – Tolerance below which to consider a Manning’s n value to be zero if condense == True

prep_test(removeBinaries=False)¶

Assumes prep_all() has been run first. Prepares a fort.13 file for testing purposes.

Parameters:	removeBinaries (binary) – flag wheter or not to remove *.asc.binary files

setup_folder(folder_name='temp')¶

Set up a single folder with name folder_name

Parameters:	folder_name (string) – folder name relative to self.base_dir
Return type:	string
Returns:	file name of bash script for this land class

setup_landuse_folder(class_num, manningsn_value=1, folder_name=None)¶

Set up a single landuse with name landuse_class_num

Parameters:	class_num (int) – land classification number for this folder manningsn_value (int) – Manning’s n value for this land classification folder_name (string) – folder name relative to self.base_dir
Return type:	string
Returns:	file name of bash script for this land class

setup_landuse_folders(create_all=True)¶

Set up landuse folders by copying all necessary files to run grid_all_data into the landuse folder. One folder is created for each landuse classification.

If create_all is True then sets up num_landuse folders. Else sets up num_landuse-1 folders.

Parameters:	create_all (bool) – flag to not skip first folder
Return type:	list
Returns:	list of file names of bash scripts for each land class

setup_tables(folder_name)¶

Creates a *.table in folder_name for each unqiue *.table file required by grid_all_*.sh

Parameters:	folder_name (string) – name of the folder to create table in

setup_tables_single_value(class_num, manningsn_value, folder_name)¶

Creates a *.table in folder_name for t_name. The land classification with t_class_number in t_name is assigned a value of manningsn_value and all other landuse classifications are assigned a manningsn_value of 0

Parameters:	class_num (int) – land classification number manningsn_value (float) – Manning’s n value folder_name (string) – name of the folder to create table in

polyadcirc.pyGriddata.manufacture_gap module¶

This module contains functions to generate GAP formatted *.asc files as a form of simulated data to be used by Griddata_v1.32.F90. This module requires Numpy version 1.7.0 or above.

polyadcirc.pyGriddata.manufacture_gap.random(xl, xr, yl, yu, landclasses, cellsize=30, p=None, path=None)¶

Generates a random matrix that covers the area defined by xl, xr, yl, yu where the land classification numbers are chosen from landclasses.

See also

Numpy documenation

Parameters:	xl (float) – leftmost value in meters xr (float) – rightmost value in meters yl (float) – lower value in meters yu (float) – upper value in meters landclasses (1D array-like or int) – list of land classification numbers cellsize (int) – size of the cell in meters p (1D array-like of size num_land_classes) – probabilities associated with each land classification path (string) – folder to write out probability structure to as p_struct.txt
Return type:	int or numpy.ndarray
Returns:	See numpy.random.choice

polyadcirc.pyGriddata.manufacture_gap.random_horizontal(y_points, xl, xr, landclasses, cellsize=30, p_sections=None, path=None)¶

Generates a random matrix that covers the area defined by xl, xr, y_points where the land classification numbers are chosen from landclasses.

See also

Numpy documenation

Parameters:	y_points (ordered 1D array-like must be of size num_sections+1) – y values in meters that divde the grid into vertical intervals (y_min, ...., y_max) xl (float) – lower value in meters xr (float) – upper value in meters landclasses (1D array-like or int) – list of land classification numbers cellsize (int) – size of the cell in meters p_sections (list of size num_sections of 1D array-like or int) – list of probabilities for each section associated with each land classification path (string) – folder to write out probability structure to as p_struct.txt
Return type:	int or numpy.ndarray
Returns:	See numpy.random.choice

polyadcirc.pyGriddata.manufacture_gap.random_patches(x_points, y_points, landclasses, cellsize=30, p_sections=None, path=None)¶

Generates a random matrix that covers the area defined by x_points, y_points where the land classification numbers are chosen from landclasses.

See also

Numpy documenation

Parameters:	x_points (ordered 1D array-like must be of size num_sections+1) – x values in meters that divde the grid into vertical intervals (x_min, ...., x_max) y_points (ordered 1D array-like must be of size num_sections+1) – y values in meters that divde the grid into vertical intervals (y_min, ...., y_max) landclasses (1D array-like or int) – list of land classification numbers cellsize (int) – size of the cell in meters p_sections (list of size num_sections of 1D array-like or int) – list of probabilities for each section associated with each land classification sections are numbered n = j + n_x_sections * i where i and j are the ith and jth x and y section respectively path (string) – folder to write out probability structure to as p_struct.txt
Return type:	int or numpy.ndarray
Returns:	See numpy.random.choice

polyadcirc.pyGriddata.manufacture_gap.random_vertical(x_points, yl, yu, landclasses, cellsize=30, p_sections=None, path=None)¶

Generates a random matrix that covers the area defined by x_points, yl, yu where the land classification numbers are chosen from landclasses.

See also

Numpy documenation

Parameters:	x_points (ordered 1D array-like must be of size num_sections+1) – x values in meters that divde the grid into vertical intervals (x_min, ...., x_max) yl (float) – lower value in meters yu (float) – upper value in meters landclasses (1D array-like or int) – list of land classification numbers cellsize (int) – size of the cell in meters p_sections (list of size num_sections of 1D array-like or int) – list of probabilities for each section associated with each land classification path (string) – folder to write out probability structure to as p_struct.txt
Return type:	int or numpy.ndarray
Returns:	See numpy.random.choice

polyadcirc.pyGriddata.manufacture_gap.write_gapfile(gap_data, xllcorner, yllcorner, file_name='gap_data.asc', cellsize=30, NODATA_value=-9999)¶

Writes out a GAP formatted *.asc file to file_name.

Parameters:	file_name (string) – full path to file_name gap_data (numpy.ndarray) – nrows by ncols 2D array xllcorner (float) – x UTM coordinate of lower left corner (SW) in meters yllcorner (float) – y UTM coordinate of lower left corner (SW) in meters cellsize (int or 30) – size of cells(square grid) in meters, pixel resolution NODATA_value (int) – land classification number for no data

polyadcirc.pyGriddata.table_management module¶

This modules controls the management and creation of *.table files

polyadcirc.pyGriddata.table_management.create_gap_list(table, gap_files)¶

Create a list() of gapInfo objects from a list of files.

Parameters:	gap_files (list) – file names of gap formatted files
Return type:	list
Returns:	list of gapInfo objects

polyadcirc.pyGriddata.table_management.create_gap_list_from_folder(table, folder_name)¶

Create a list() of gapInfo objects from the files in folder.

Parameters:	folder_name (string) – folder containing gap formatted files
Return type:	list
Returns:	list of gapInfo objects

polyadcirc.pyGriddata.table_management.create_table(landuse_table, folder_name=None)¶

Create table_name.table in folder_name where the landuse classification numbered landuse_table.keys() is assigned a manningsn_value of landuse_table['key']

Parameters:	landuse_table (tableInfo) – table to create folder_name (string) – folder to create the table in

polyadcirc.pyGriddata.table_management.create_table_single_value(class_num, landuse_table, manningsn_value, folder_name=None)¶

Create a *.table in folder_name where the landuse classification numbered class_num is assigned a value of manningsn_value and all other landuse classifications are assigned a manningsn_value of 0

Parameters:	class_num (int) – land classification number landuse_table (tableInfo) – table to base the single value table off of manningsn_value (float) – Manningn’s n value for class_num folder_name (string) – folder to create the table in

class polyadcirc.pyGriddata.table_management.gapInfo(file_name, table, horizontal_sys=None, UTM_zone=None)¶

Bases: polyadcirc.pyADCIRC.basic.pickleable

This object stores information specific to a GAP dataset, methods for creating it’s portion of the *.in file, and methods for creating the table(s) needed for this GAP dataset.

create_table(folder_name=None)¶

Create self.table_name.table in`` folder_name`` where the landuse classification numbered landuse_table.keys() is assigned a manningsn_value of landuse_table['key'].

Parameters:	folder_name (string) – folder to create the table in

create_table_single_value(class_num, manningsn_value, folder_name=None)¶

Create a *.table in folder_name where the landuse classification numbered class_num is assigned a value of manningsn_value and all other landuse classifications are assigned a manningsn_value of 0.

Parameters:	class_num (int) – land classification number manningsn_value (float) – Manningn’s n value for class_num folder_name (string) – folder to create the table in

local_str(basis_dir, folder_name=None)¶

Parameters:	basis_dir (string) – the folder containing the *.asc files and the directory folder_name folder_name (string) – name of folder to create *.in for
Return type:	string
Returns:	text that matches relevant lines of *.in file and uses basis_dir for *.asc files

read_table(folder_name=None)¶

Read in self.table.file_name in folder_name

Parameters:	folder_name (string) – folder to read the table from
Return type:	tableInfo
Returns:	an object with all of the information in that table

polyadcirc.pyGriddata.table_management.read_table(table_file_name, folder_name=None)¶

Read in table_file_name in folder_name

Parameters:	table_file_name (string) – local file name of table folder_name (string) – folder to read the table from
Return type:	tableInfo
Returns:	an object with all of the information in that table

polyadcirc.pyGriddata.table_management.read_tables(folder_name=None)¶

Read in all *.table files in folder_name and return a list of tableInfo objects

Parameters:	folder_name (string) – folder to read the table(s) from
Return type:	list of tableInfo
Returns:	list of objects with all of the information in that table

class polyadcirc.pyGriddata.table_management.tableInfo(file_name, land_classes)¶

Bases: polyadcirc.pyADCIRC.basic.pickleable

This class stores the relation between Manning’s n values and land class numbers.

create_table(folder_name=None)¶

Create table_name.table in`` folder_name`` where the landuse classification numbered landuse_table.keys() is assigned a manningsn_value of landuse_table['key'].

Parameters:	folder_name (string) – folder to create the table in

create_table_single_value(class_num, manningsn_value, folder_name=None)¶

Create a *.table in folder_name where the landuse classification numbered class_num is assigned a value of manningsn_value and all other landuse classifications are assigned a manningsn_value of 0.

Parameters:	class_num (int) – land classification number manningsn_value (float) – Manningn’s n value for class_num folder_name (string) – folder to create the table in

get_landclasses()¶

Return type:	list
Returns:	list of land_classes (integers)

get_num_landclasses()¶

Rtype int:
Returns:	total number of land_classes

read_table(folder_name=None)¶

Read in self.file_name in folder_name .

Parameters:	folder_name (string) – folder to read the table from
Return type:	tableInfo
Returns:	an object with all of the information in that table

polyadcirc.pyGriddata.table_to_mesh_map module¶

This module given a *.table (and *.table and *.13 files from prep_table_to_mesh_map) produces a *.13 file of Manning’s n values for that *.table, or dict, or array of these values.

exception polyadcirc.pyGriddata.table_to_mesh_map.Error¶

Bases: exceptions.Exception

Base class for exceptions in this module.

exception polyadcirc.pyGriddata.table_to_mesh_map.LenError(expr, msg)¶

Bases: polyadcirc.pyGriddata.table_to_mesh_map.Error

Exception raised for errors in dimension or length.

polyadcirc.pyGriddata.table_to_mesh_map.add_dict(dict_list, weights)¶

Adds a list of dict together.

Parameters:	dict_list – list of dicts weights (list) – list of weights
Return type:	dict
Returns:	a dict[k] = weights[0]*dict_list[0] + ... + weights[-1]*dict_list[-1]

polyadcirc.pyGriddata.table_to_mesh_map.add_dict_pair(dict1, dict2)¶

Adds two dict together.

Parameters:	dict1 (dict) – first dict dict2 (dict) – second dict
Return type:	dict
Returns:	a dict[k] = dict1[k] + dict2[k]

polyadcirc.pyGriddata.table_to_mesh_map.combine_basis_vectors(weights, vectors, default_value=None, node_num=None)¶

Combine basis vectors using weights as the Manning’s n value for each basis vector. If a default_value is set then all nodes with out data are set to the default_value.

Parameters:	weights (numpy.ndarray) – array of size (num_of_basis_vec, 1) vectors (list of dicts OR numpy.ndarray of size (node_num, num_of_basis_vec)) – basis vectors
Returns:	an array of size (node_num, 1) containing the manningsn value at all nodes in numerical order or a dictionary

polyadcirc.pyGriddata.table_to_mesh_map.combine_bv_array(weights, array)¶

Combine basis vector arrays using weights as the Manning’s n value for each basis vector array.

Parameters:	weights (numpy.ndarray) – array of size (num_of_basis_vec, 1) array (numpy.ndarray of size (node_num, num_of_basis_vec)) – array of basis vectors
Returns:	an array of size (node_num, 1) containing the manningsn value at all nodes in numerical order

polyadcirc.pyGriddata.table_to_mesh_map.condense_bv_dict(mann_dict, TOL=None)¶

Condenses the mann_dict land classificaiton mesh by removing values that are below TOL.

Parameters:	mann_dict (dict) – a dictionary created from a fort.13 formatted file or a dictionary of Manning’s n values TOL (double) – Tolerance close to zero, default is 1e-7
Return type:	dict
Returns:	basis vector of values

polyadcirc.pyGriddata.table_to_mesh_map.condense_lcm_folder(basis_folder, TOL=None)¶

Condenses the fort.13 lanudse classification mesh files in landuse_* folders in basis_dir by removing values taht are below TOL.

Parameters:	basis_dir (string) – the path to directory containing the landuse_## folders TOL (double) – Tolerance close to zero, default is 1e-7

polyadcirc.pyGriddata.table_to_mesh_map.create_from_fort13(domain, mann_dict, vectors)¶

Creates a basis vector where the value at default nodes is the value in the mann_dict created from reading in a fort.13 formatted file. If the node is default in both mann_dict and vectors then it remains a default node.

Parameters:	domain (domain) – a computational domain for a physical domain mann_dict (dict) – a dictionary created from a fort.13 formatted file or a dictionary of Manning’s n values vectors (dict) – basis vectors
Return type:	dict
Returns:	basis vector of values

polyadcirc.pyGriddata.table_to_mesh_map.create_shelf(domain, shelf_bathymetry, vectors)¶

Creates a contitnetal shelf basis vector where the value at default nodes between user defined bathymetric bounds are 1 and the other default nodes are untouched. This basis vector can now be used to create a fort.13 file. Remember bathymetry is positive in the down direction.

Parameters:	domain (domain) – a computational domain for a physical domain shelf_bathymetry (numpy.ndarray) – the bathymetric limits of the continental shelf [min, max] vectors (dict) – basis vectors
Return type:	dict
Returns:	basis vector that represents the continental shelf

polyadcirc.pyGriddata.table_to_mesh_map.determine_types(domain, vectors)¶

Determine the dominant land classification types for a particular mesh and the corresponding percentages of those land classification types.

Parameters:	domain (domain) – a computational domain for a physical domain vectors (list) – basis vectors
Return type:	numpy.ndarray
Returns:	sorted list of ranking, land classification number, and percentage of land classification type present as a (len, 3) array

polyadcirc.pyGriddata.table_to_mesh_map.dict_to_array(data, default_value, node_num)¶

Given a dictonary, default_value, and number of nodes converts a dictornary to an array of size (node_num, 1) and fills in the missing entires with the default_value.

Parameters:	data – dict default_value (float) – default node_num (int) – total number of nodes in the mesh
Return type:	numpy.ndarray
Returns:	array version of the dict

polyadcirc.pyGriddata.table_to_mesh_map.get_basis_vec_array(path=None, node_num=None)¶

NOTE: this impementation currently assumes that there are no default nodes this will need to be updated later..

Parameters:	path (string) – folder containing the landuse folders node_num (int) – number of nodes in the mesh
Return type:	numpy.ndarray
Returns:	an array of size(node_num, num_basis_vec)

polyadcirc.pyGriddata.table_to_mesh_map.get_basis_vectors(path=None)¶

Each dict is structured as follows: keys – node number values – weighting for spatial averaging currently assumes only 1 folder per landuse classification with the name landuse_*/

Parameters:	path (string) – folder containing the landuse folders
Return type:	list
Returns:	list of dicts for each landuse classification.

polyadcirc.pyGriddata.table_to_mesh_map.get_default_nodes(domain, vectors=None)¶

Given a set of basis vectors and a domain returns a list of default nodes.

Parameters:	domain (domain) – a computational domain for a physical domain vectors (dict) – basis vectors
Return type:	list
Returns:	list of default nodes

polyadcirc.pyGriddata.table_to_mesh_map.merge_with_fort13(domain, mann_dict, factor, land_class_num, vectors)¶

Creates a basis vector where the value at default nodes and pure nodes in the land classification basis vector with land_class_num are replaced set to the the scaled value in the mann_dict which is created from reading in a fort.13 formatted file. If the node is default in both mann_dict and vectors then it remains a default node. Returns a new list of land classification vectors with:

vectors[land_class_num] = mixed_vector #original values
vectors[max(vectors.keys())+1] = new_vector #scaled merged values

Parameters:	mann_dict (dict) – a dictionary created from a fort.13 formatted file or a dictionary of Manning’s n values factor (float) – the factor by which to divide the values in mann_dict land_class_num (int) – land classification to split and merge vectors (list) – basis vectors
Return type:	list
Returns:	modified list of basis vectors

polyadcirc.pyGriddata.table_to_mesh_map.split_bv_nodes(land_class_num, vectors)¶

Given a set of basis vectors and a land classification class number splits a land classification basis vector dictionary into nodes that are purely that land classification and nodes that are only paritially that land classification. Returns a new list of land classification vectors with:

vectors[land_class_num] = mixed_vector
vectors[len(vectors)+1] = pure_vector

Parameters:	land_class_num (int) – land classification to split vectors (list) – basis vectors
Return type:	list
Returns:	modified list of basis vectors

Module contents¶

This package contains a set of modules mainly for the mapping of nodal data to an ADCIRC mesh using landuse data and the Fortran90 program Griddata_v1.32.F90. GridData is a FORTRAN program originally developed by Seizo Tanaka (ST3) and C.H.Lab., University of Notre Dame.

This package contains the modules

• file_management
• table_management
• table_to_mesh_map
• grid_management
• manufacture_gap

grid_management prepares *.table and *.13 files needed for table_to_mesh_map and creates the n x m matrix of multiplier factors where n = # nodes, and m = # land classification values, assuming there are no surprises with the spatial averaging.

additional needed files:

• compiled version of Gridata_v1.32.F90 named Griddata_parallel.out
• .asc files
• *.14 files
• fort.13 file to use as a template for convert()

table_to_mesh_map given a **_manning.table (and data file(s) from above) produces a fort.13 ready for use by ADCIRC