What Des HaD t Py D? HaD t Py takes the utput file fr a HEC-RAS simulatin and creates plts which can quickly be viewed spatially fr calibratin r validatin. The script is available nline at https://github.cm/latmkvic/hadt-py. The script and its dependent files are free t the public. Everything was written by Lily Ann Tmkvic, a graduate student researcher at the University f Califrnia Davis, wrking at the Center fr Watershed Sciences. What Des HaD t Py Need? T use HaD t Py in the way it was intended, the user shuld have the fllwing: HEC-RAS 5.0 r earlier HEC-DSSVue 2.0 r earlier Pythn (written in 2.7) Several Pythn Mdules listed later HaD t Py Users Manual Tl t quickly harvest and stre bserved versus cmputed data fr HEC- RAS 1D/2D mdel results Further instructin is fund within the Users Manual. Authr: Lily Ann Tmkvic Graduate Student Researcher UC Davis, Center fr Watershed Sciences latmkvic@ucdavis.edu
TABLE OF CONTENTS Intrductin... 2 Requirements... 2 Dwnlad Package Cntents and Descriptin... 3 Instructins... 4 Cncept... 4 File Requirements... 4 Pythn Mdules Required... 4 Jythn Files fr HEC-DSS... 4 bs_path Text File... 5 ne_dim_cmp_paths Text File... 5 tw_dim_crds Text File... 6 DSS_data_utput Text File... 6 ras_2d_cells Text File... 6 Main Cde... 7 hdf_filename... 7 bs_dss... 7 twd_dss... 7 plt_dir... 7 Tutrial Example... 8 Open RAS Prject... 8 Run RAS Prject... 8 Display Gage Shapefile... 8 Enter tw_dim_crds.txt Data... 9 Enter bs_paths.txt Data... 9 Enter ne_dim_cmp_paths.txt Data... 10 Mdify py_hdf_dss.py Preamble... 11 hdf_filename... 11 bs_dss... 11 twd_dss... 11 plt_dir... 11 Run the Cde!... 12 Viewing the Plts Interactively with GIS... 12 1 P a g e
INTRODUCTION HaD t Py takes the utput file fr a HEC-RAS simulatin and creates plts which can quickly be viewed spatially fr calibratin r validatin. The script is available nline at https://github.cm/latmkvic/had-t-py. The script and its dependent files are free t the public. Everything was written by Lily Ann Tmkvic, a graduate student researcher at the University f Califrnia Davis, wrking at the Center fr Watershed Sciences. Several peple helped alng the way t make this tl pssible, including but nt limited t: Bill Fleenr at UCD, and Jan Klipsch, Gary Brunner, Bill Charley, Mike Perryman, and Tm Evans at USACE-HEC. Fr help understanding hw t use the cde prperly yu can cntact Lily Ann Tmkvic at latmkvic@ucdavis.edu REQUIREMENTS T use HaD t Py in the way it was intended, the user shuld have the fllwing installed n their machine: HEC-RAS 5.0 r earlier HEC-DSSVue 2.0 r earlier Pythn Several Pythn Mdules listed later The user will als need t designate a flder r wrking directry with all f the files included in the dwnlad package. 2 P a g e
DOWNLOAD PACKAGE CONTENTS AND DESCRIPTION FILE pyhdf_dss.py stredssdata.py getdssdata.py bs_paths.txt ne_dim_cmp_paths tw_dim_crds.txt temp_files DSS_data_utput.txt ras_2d_cells.txt Tutrial txtfiles bs_paths ne_dim_cmp_paths tw_dim_crds BaldCreek2D.dss BaldCreekObservedData.dss BaldEagleGages (.cpg,.dbf,.sbn,.sbx,.shp,.shx) DESCRIPTION The main cde which the user will run in rder t get the plts/data A Jythn cde which will be peripherally run by HEC-DSSVue frm within the main cde It can stre recrds t a DSS file path A Jythn cde which will be peripherally run by HEC-DSSVue frm within the main cde It accesses DSS recrds and utputs the values and times t a temprary txt file fund in the temp_files flder A required txt file fr the cde t run. See bs_path Text File sectin fr crrect frmatting and mre infrmatin n its use A required txt file fr the cde t run. See ne_dim_cmp_paths Text File sectin fr crrect frmatting and mre infrmatin n its use A required txt file fr the cde t run. See tw_dim_crds Text File sectin fr crrect frmatting and mre infrmatin n its use A flder cntaining temprary txt files The temprary utput file fr getting DSS data A temprary file which cntains pertinent infrmatin n the gages fund in 2D flw area The flder which cntains all f the files necessary fr the Tutrial Example Where the cmpleted txt files are lcated fr the tutrial The bserved data DSS pathnames The cmputed 1D RAS DSS paths The spatial crdinates f the gages within the 2D areas An empty DSS file where the cde can stre 2D data that it harvests A DSS file which has (made-up) bserved data t cmpare t the gages A shapefile which has (made-up) gages fr the tutrial 3 P a g e
INSTRUCTIONS CONCEPT The main idea is pretty simple, there are bserved data entries in a DSS file, and there are the ne-dimensinal and tw-dimensinal cmputed data entries in a separate DSS file and an HDF file, respectively. HaD t Py takes the data frm the HDF and DSS files and handles them in Pythn. Currently, HaD t Py 1.0 nly handles water surface elevatin plts, but it wuld be rather simple t mdify the cde and allw ther plts t be shwn. ITEM TO PLOT Observed Data 1D Cmputed Pint 2D Cmputed Pint LOCATION An Observed Data DSS File The HEC-RAS Output DSS File The HEC-RAS Output HDF File FILE REQUIREMENTS Befre yu start using the cde, yu need t be sure that the required files and mdules are dwnladed/imprted and in the right places. It is best t take all f the files fund in the Dwnlad Package Cntents and Descriptin sectin f the Intrductin int the wrking directry flder, r the main flder where yu intend t perate this cde. This will becme mre clear in the File Requirements sectin. PYTHON MODULES REQUIRED imprt s imprt numpy as np frm scipy imprt spatial frm Tkinter imprt * imprt tkmessagebx imprt subprcess imprt h5py imprt matpltlib.pyplt as plt All f the abve mdules need t be dwnladed/available t yur Pythn interpreter. JYTHON FILES FOR HEC-DSS There are tw files which HEC-DSSVue will use when the main Pythn cde passes them thrugh the cmmand line. One file retrieves DSS data and the ther writes a DSS recrd t a file. The ne that writes the recrd t a file is currently nt used, but wuld be easy fr the user t implement. Unless yu knw what yu re ding, it is nt recmmended that yu change r delete these files. The files are: stredssdata.py getdssdata.py 4 P a g e
OBS_PATH TEXT FILE The bs_path text file cntains a simple list f the bserved data paths that the cde will lk up t get data frm the simulatin perid. The fllwing text bx demnstrates what the file shuld lk like: Gage: SUT Path: /SUTTER SLOUGH AT COURTLAND/SUT/STAGE/31DEC2005/IR-DAY/USGS/ Gage: VON Path: /SACRAMENTO RIVER AT VERONA/VON/STAGE/01FEB2005/1HOUR/USGS DWR/ Gage: YBY Path: /YOLO BYPASS NEAR WOODLAND/YBY/STAGE/28FEB2005/IR-DAY/USGS/ The imprtant pints f this file are highlighted with the bxes and arrws: 1. The wrds Gage: and Path: are nly used t indicate the gage name and pathname. a. i.e. the pathname shuldn t cntain Gage: r Path: as they are used as delimiters t parse the data 2. There are n line breaks between entries 3. There is ne space at the end f the file a. Again, the cde uses a line break t delimit the pathnames ONE_DIM_COMP_PATHS TEXT FILE The ne_dim_cmp_paths text file has a similar appearance t the bs_path file, but has a small distinctin. If there are n 1D gages, then this file shuld be cmpletely empty, but the file shuld still exist. Gage: SSS Path: /STEAMBOAT SLOUGH SAC TO SUTTER/16027.84/STAGE/%s/%s/%s/ Gage: SUT Path: /SUTTER SLOUGH SAC TO ELK/10665.02/STAGE/%s/%s/%s/ Gage: VON Path: /SACRAMENTO NCC TO AMERICAN/106605.4/STAGE/%s/%s/%s/ The imprtant pints are the same as befre. The distinctin between this and bs_path is that the last 3 fields f the pathname have %s instead f any value. This is because these three values will change based n the utput f the RAS simulatin. Fr instance, if the simulatin time perid is frm 14FEB2009 10:00 23FEB2009 23:00, and the hydrgraph interval was set t 15MIN, and the shrt ID f the plan was TESTING, then the last three fields will lk like: /01FEB2009/15MIN/TESTING/ The main cde will handle the access/ naming f the pathnames, what the user needs t d here is t crrelate the bserved gage lcatin t a RAS 1D Crss sectin. 5 P a g e
TWO_DIM_COORDS TEXT FILE This file is necessary fr the cde t run. This cntains the latitude and lngitude (in meters) fr each gage that is fund in a 2D area. Currently, the cde can find the cells that crrelate t the gage crdinates and will then stre thse cell indices fr each versin f the gemetry yu use. This is particularly helpful fr times where yu are changing the cell size r arrangements f cells in a 2D area. Belw is an example f what this file needs t lk like: Imprtant pints n this file are: Gage: SBP Lat: 616878.502335 Ln: 4293808.60825 Gage: YBY Lat: 617940.666062 Ln: 4281881.68027 Gage: LIS Lat: 623108.260000 Ln: 4256436.96000 1. There shuld be three fields and they are Gage:, Lat:, and Ln: 2. Lat and Ln shuld be in meters, nt decimal degrees, r any ther crdinate system DSS_DATA_OUTPUT TEXT FILE This is a temprary file. It des nt need t cntain anything, but it ideally exists. The Jythn scripts that cmmunicate with HEC-DSSVue use this text file t relay the data back t the main Pythn cde. RAS_2D_CELLS TEXT FILE This file is what the main cde will utput nce it finds the clsest (spatially) cell index t the gage pint, the distance frm the cells center t the gage, and the 2D Flw Area where the cell resides. The first line will indicate which.p##.hdf file the indices were pulled frm, and the next lines will crrelate t each gage lcatin. If the cde has already run fr that hdf file, it will use the already stred lcatins, therwise it will re-find the cell indices. 6 P a g e
MAIN CODE The main cde is in the py_hdf_dss.py file fund in the dwnlad package. The user needs nly t change 4 lines fr their purpses: hdf_filename = C:\...\RAS_Prject\PrjectName.p01.hdf bs_dss = r'c\...\observeddatafile.dss' twd_dss = r C:\...\Example2D.dss plt_dir = r'c:\users\lt\dcuments\pythn\plts ' HDF_FILENAME This needs t be the hdf file crrespnding t the plan yu want t evaluate, and it shuld be lcated within the RAS prject flder. i.e. dn t mve this file t anther lcatin t run the script. OBS_DSS This is the lcatin f the DSS file where the bserved data resides. It can be named anything (it desn t need t be named ObservedDataFile.dss, in case yu were wrried). TWOD_DSS This is the lcatin f the DSS file where the cde will stre data derived frm the 2D crdinates, r the gages. This can be a blank file, r a file which already has sme entries. PLOT_DIR The plt_dir variable describes where the plt png s will g. The script allcates a special flder fr each plan shrtid and stres all f the graphs within the plt directry flder. Fr example, if the gage is HEY and the plan is TEST PLAN then the plt fr HERE will be lcated in: plt_dir\test PLAN\HEY.png 7 P a g e
TUTORIAL EXAMPLE This tutrial example uses a prject fund in the example prjects flder f HEC-RAS. If the prjects haven t already been installed, yu can g t: http://www.hec.usace.army.mil/sftware/hec-ras/dwnlads/example_prjects.exe t dwnlad them. The first tutrial uses the Bald Eagle Creek example prject fund in Example Prjects\2D Unsteady Flw Hydraulics\BaldEagleCrkMulti2D. Specifically it uses the 1d-2D Dambreak Refined Grid plan (.p15). This dcument will walk yu thrugh hw t enter the data int the txt files, but if yu want t just see the cde wrk, the crrect files are lcated in the Tutrial/txtFiles flder. Just take thse files and cpy them ver the files in the wrking directry. OPEN RAS PROJECT T start, yu ll need t pen and run the plan in RAS. Open the prject by ging t the flder listed abve frm within RAS. RUN RAS PROJECT Select the Unsteady Flw Analysis Editr and chse File > Open 1d-2D Dambreak Refined Grid G t Optins > Stage and Flw Output Lcatins and select Bald Eagle Cr. Lck Haven 103189 and add t the list using the arrw This creates an utput hydrgraph at the lcatin at ne f ur gages Press Ok Cmpute the plan by pressing the Cmpute buttn DISPLAY GAGE SHAPEFILE (Optinal) While the simulatin is running yu can see the lcatin f the gages by adding the gage shapefile t RAS Mapper Frm the main RAS windw, select RAS Mapper Right click n Map Layers Select Add map data layers Navigate t the BaldEagleGages.shp file fund in the dwnlad package under the Tutrial flder (extra bnus) T add labels: Duble-click n BaldEagleGages t pen the Layer Prperties Select Label Features with Attribute Clumn(s) Click Edit Chse StatinID in the drpdwn under Attribute Text Nw yu can see where the gages are in the gemetry 8 P a g e
ENTER TWO_DIM_COORDS.TXT DATA Yu can see in RAS Mapper that 4 gages are in the 2D Flw Area and 1 is n a 1D Crss Sectin. In rder t get the data frm the 2D Flw Area cell pints, we need t enter gage crdinates. Frm the wrking directry (where the dwnlad files are fund, and where this file is fund!) pen the tw_dim_crds.txt file Fr a descriptin f this file, g t the tw_dim_crds Text File sectin. The crdinates are Latitude and Lngitude in meters. This is hw RAS finds pints. There are 2 ways t get the XY crdinates frm the shapefile pints: Ppulate the XY crdinates f the shapefile in ArcGIS using the Add XY Crdinates tl Using RAS Mapper: Zm in pretty tight n a gage Select the Measuring tl Duble-click with (left-click) n the gage pint Select Cpy Crdinates t Clipbard Paste them smewhere and yu can chse either rw (they shuld be abut the same) Enter a Latitude and Lngitude fr the 4 gages fund in the 2D Flw Area using the frmat: Gage: SBP Lat: 616878.502335 Ln: 4293808.60825 * Make sure t leave an extra blank line at the end! Fr this tutrial the Crdinates are listed here: Gage Latitude Lngitude BDL 2012154.86014 326091.04372 FAR 2082421.16365 365397.77488 NES 2071093.97205 359607.937664 TRB 2044957.56598 347579.767861 ENTER OBS_PATHS.TXT DATA Fr mre infrmatin n the text file g t the bs_path Text File sectin. Nw yu need t tell the cde where t lk fr the bserved data. Each gage needs an bserved data path, and currently, all the bserved data needs t be in the same DSS file. The Observed DSS data file can be fund in the wrking directry in the Tutrial flder, it is called BaldCreekObservedData.dss Yu can use the DSS file t fill ut the bs_paths.txt file fund in the wrking directry. Enter the pathnames in the fllwing frmat: Gage: SUT Path: /SUTTER SLOUGH AT COURTLAND/SUT/STAGE/31DEC2005/IR-DAY/USGS/ * Again, make sure t leave an extra blank line at the end! 9 P a g e
Fr the tutrial, the paths are listed here: Gage Pathname BDL /BDL/BELOW THE DAM/STAGE/01JAN1999/1HOUR/EXAMPLE DATA/ FAR /FAR/FARTHEST POINT/STAGE/01JAN1999/1HOUR/EXAMPLE DATA/ NES /NES/NEAR THE STREAM/STAGE/01JAN1999/1HOUR/EXAMPLE DATA/ TRB /TRB/TRIBUTARY/STAGE/01JAN1999/1HOUR/EXAMPLE DATA/ XSG /XSG/IN-RESERVOIR/STAGE/01JAN1999/1HOUR/EXAMPLE DATA/ ENTER ONE_DIM_COMP_PATHS.TXT DATA This is the last text file yu need t fill ut. This tells the cde where t lk fr the cmputed data in a 1D reach. Fr mre infrmatin g t the ne_dim_cmp_paths Text File sectin. Yu can find which crss sectin yu shuld select by fllwing these steps: In RAS Mapper, expand the Results tree fr the 1D-2D Refined Grid Plan Expand Gemetry Select XS Left Click n XS Zm in n the XSG gage (n the XS) Select the pinter tl Right click the Crss Sectin which is n the gage A pp-up will appear and has the name f the XS nde Then, yu ll need t find the DSS pathname, which yu can d by selecting the DSS buttn windw, then navigating t the crrect path. in the main RAS Use the fllwing frmat t enter the pathname: Gage: SSS Path: /STEAMBOAT SLOUGH SAC TO SUTTER/16027.84/STAGE/%s/%s/%s/ * Again, make sure t leave an extra blank line at the end! The path fr this tutrial is: /BALD EAGLE CR. LOCK HAVEN/103189/STAGE/%s/%s/%s/ The %s fr the last three allw the cde t enter the apprpriate path parts which crrespnd t the simulatin ptins (time windw, hydrgraph utput interval, and plan shrt ID) 10 P a g e
MODIFY PY_HDF_DSS.PY PREAMBLE The descriptin f the different parts f the preamble are fund in the Main Cde sectin. Open the pyhdf_dss.py pythn script in the wrking directry. Nw yu ll need t change 5 lines in the preamble under # User Input HDF_FILENAME This will crrespnd t the plan file which we re getting data frm. In this case, it s the 1D-2D Refined Grid plan which is.p15 If yur example prjects flder is lcated in the Dcuments flder then this line will lk like this: hdf_filename = r C:/Users/lt/Dcuments /Example Prjects/2D Unsteady Flw Hydraulics/BaldEagleCrkMulti2D/BaldEagleDamBrk.p15.hdf Be sure t use an r befre the qute, this avids escape characters in the filename. OBS_DSS This is the bserved data DSS file which will be used t make the bserved v. cmputed plts. Fr this tutrial, it will be lcated in the wrking directry (where the dwnlad package was installed) in the Tutrial flder. If yur wrking directry is C:/Dcuments/HaDtPy then the bs_dss line will lk like this: bs_dss = r'c:/users/lt/dcuments/hadtpy/tutrial/baldcreekobserveddata.dss' TWOD_DSS The twd_dss.dss file is where the cde will stre the 2D results as a DSS element. Fr this tutrial, it will be lcated in the wrking directry in the Tutrial flder, as well. Similar t abve, it will lk like: twd_dss = r C:/Users/lt/Dcuments/HaDtPy/Tutrial/BaldCreek2D.dss PLOT_DIR The plt directry, r plt_dir, is the flder where yu wuld like the resulting figures t g. Each time the cde is ran, it will create a flder with the RAS plan s shrt ID as the flder name, then put png s with the gage name as the image name within that flder. Yu can put this flder wherever yu d like. This is an example f where this culd g: plt_dir = r'c:/users/lt/dcuments/hadtpy/figures' Fr this tutrial, if yu were t use the abve path (with yur username instead f lt ) the plt fr XSG will be in this path: C:/Users/lt/Dcuments/HaDtPy/Figures/1D-2D Refined Grid/XSG.png 11 P a g e
RUN THE CODE! Once yu run the pyhdf_dss.py script, it will prduce 5 plts in whichever directry yu chse. After the script has run, yu can view the plts by navigating t the plt directry in yur file explrer. VIEWING THE PLOTS INTERACTIVELY WITH GIS (Optinal) This is a quick walk-thrugh f hw ne wuld g abut making the plts appear spatially in an interactive fashin. The screenshts in this sectin have shapefiles that are nt included in the HaD t Py dwnlad, but are easily exprted frm RAS Mapper (i.e. the 2D Area and the 1D Reach and Crss Sectins). This sectin is shwn in ArcMap but the same task can be accmplished in QGIS, a free and pen-surce GIS platfrm. Abve, yu can see the layut f the gages, and the RAS gemetry. If yu pen the attribute table f the Tutrial Gages (the BaldEagleGages shapefile fund in the Tutrial flder within the wrking directry), yu can see the Images field which has the html cde fr the gage plt. 12 P a g e
In rder t change these field attributes, begin editing by Selecting Editr> Start Editing. This allws yu t edit each attributes Image field individually. Fr each image, yu ll want t mdify the highlighted text in the cde belw t match the file lcatin f yur plt: <img src='c:\users\lelekew\dcuments\pythn\figures\1d-2d Refined Grid\XSG.png' width='500' /> Once all f the fields are changed, just select Stp Editing and Save Edits. Nw t get the ppups! Right-click n the BaldEagleGages layer in the Table f Cntents (TOC) and select Prperties Navigate t the Fields tab, and uncheck everything but StatinID and Image (this makes the pp-up less cluttered). Navigate t the HTML Ppup tab, and check the Shw cntent fr this layer using the HTML Ppup tl bx, and activate the As a table f the visible fields radi buttn, and check the Hide field name clumn bx. 13 P a g e
Press Ok and return t the regular GIS screen. Select the HTML Ppup tl: Nw, when yu click n the pints in the BaldEagleGages shapefile, a ppup will appear which has the statinid and the plt. Belw is an example f what a cuple f pen ppups wuld lk like. 14 P a g e