HYCOM Model Development, Validation, and Documentation Alan J. Wallcraft Naval Research Laboratory 8th Hybrid Coordinate Ocean Model Workshop August 21, 2003
HYCOM 2.1.03 (I) ffl First public release of HYCOM 2.1 ffi September, 2002 ffl Maintain all features of HYCOM 2.0 ffi Requires Fortran 90 ffi Can use HYCOM for pure iso-pycnal cases ffi KPP or Kraus-Turner mixed-layer ffi Energy-Loan (passive) ice model ffi High frequency atmospheric forcing ffi New I/O scheme (.a and.b files) ffi Scalability via OpenMP or MPI or both Λ Bit-for-bit multi-cpu reproducibility
HYCOM 2.1.03 (II) ffl Orthogonal curvilinear grids ffl Fully global domains ffi Near-global Mercator already in HYCOM 2.0 ffi Pan-Am grid with closed Bering Strait Λ Open Bering Strait requires a special halo exchange (still not available) ffl Multiple tracers ffi Passive ffi Simple biology (NPZ or NPZD) Λ Most of infrastructure in place Λ More work needed on biology and boundary conditions ffl Rivers as bogused surface precipitation ffi Annual or monthly climatology
HYCOM 2.1.03 (III) ffl Off-line one-way nesting ffi Similar to MICOM Λ Exact b.c. for depth averaged component Λ Relaxation for 3-D T/S/pressure/velocity Based on relaxation to climatology But with addition of velocity relaxation ffi Interpolate to target domain off-line Λ Source domain to target domain archive files Λ Nested model does not know about enclosing domain Λ Can discard deep iso-pycnal layers Λ Can remap to new vertical coordinate ffi Get boundary data from input archive files Λ Simplifies scalability Λ At the cost of more I/O and bigger files ffl Same resolution nesting unexpectedly useful ffi No need to rerun large domain ffi Change atmospheric forcing (e.g. use MM5) ffi Change vertical structure ffi Tracer studies (e.g. add biology)
HYCOM 2.1.03 (IV) ffl More general fixed (sigma/z) vertical coordinate ffi Emulate Z or Sigma or Sigma-Z models ffi HYCOM has partial cells Λ Emulate full cell Z models with the appropriate bathymetry ffl Alternative mixed-layer models ffi Mellor-Yamada 2.5 ffi Price-Weller-Pinkel ffl Explicit support for 1-D and 2-D domains ffi Tiling allows periodic domains ffi 1-D is 2x2 doubly periodic domain Λ Only read/write.b files ffi 2-D is Nx2 periodic domain Λ Infinite f-plane
HYCOM 2.1 (V) ffl Region-independent setup and diagnostics ffi Dynamic memory allocation ffi Compile once for all domains ffi Actual model code still domain-dependent ffl Restart processing ffi HYCOM from MICOM ffi HYCOM from archive file ffi HYCOM from coarser resolution HYCOM ffi Change land/sea boundary ffl Archive processing ffi Add/subtract/merge layers ffi Remap to new vertical coordinate ffi Mean and variability ffi Off-line sampling Λ Vertical profiles Λ Transport sections ffi To netcdf and other file formats Λ All x-y hycomproc fields Λ Interpolated to z-space
HYCOM FILE FORMAT ffl MICOM s PAKK I/O not efficient or accurate ffl HYCOM 2.1 reads/writes.a and.b files ffi.a is a raw IEEE REAL*4 array file (Fortran direct access) ffi.b is a plain-text header file (Fortran formatted) ffl This I/O is simple and portable ffl It can easily be parallelized ffi Have the N-th processor read/write every N-th 2-D array record ffl Convert to netcdf off-line ffi Climate and Forecast (CF) conventions Λ Extension of COARDS conventions Supports curvilinear grids Λ Most existing software is for COARDS only ffi Unlimited time dimension Λ Simplifies time concatenation ffi Safest to have single field per file Λ Avoids all 2GB limits
EQUAL AREA DOMAIN DECOMPOSITION ffl Simplest domain decomposition is equal-sized rectangular tiles ffi Each tile has four neighbors Λ Eight neighbors including halo corners ffl Overall speed controlled by slowest tile ffi Probably have an all ocean tile Λ no advantage to avoiding land within a tile ffl Discard tiles that are entirely over land ffi Simple to implement ffi Does not discard all land ffi P-MICOM probably first ocean model to do this ffi Should be in all MPI-based OGCM s
IMPROVED DOMAIN DECOMPOSITION ffl HYCOM 2.0 and 2.1 allows: ffi Discarded tiles ffi Variable tile size ffi Many North-South neighbors ffl Equal-ocean tiling ffi First distribute one axis, then the other ffi Near perfect load balance ffi Some tiles require more memory than others ffi Aspect ratio of rectangle can be large ffi More expensive halo exchange ffl Modified equal-area tiling ffi Discard all-land tiles ffi Shift tiles to fit coastline ffi Double-up tiles if less than half ocean ffi Compared to equal-area tiling: Λ Up to 2x the memory requirement Λ More expensive halo exchange Λ Often significantly fewer tiles
MODIFIED EQUAL AREA TILING 16x16 = 256 Tiles but only 143 Active 12% fewer than equal area tiling
Improvements since 2.1.03 ffl Vertical remapping uses PLM for fixed coordinate layers ffl Thin deep iso-pycnal layers ffl Stability from locally referenced potential density ffl GISS mixed layer model ffl Black-body correction to longwave flux ffl Option to relax to observed SST ffl Spatially varying iso-pycnal layer target densities ffl Nesting no longer requires co-located grids ffi General archive to archive horizontal interpolation ffl Hybrid to fixed vertical grid remapper ffi Allows fixed-coordinate nests inside hybrid coordinate outer domains Λ HYCOM to (fixed-grid) HYCOM Λ HYCOM to NCOM
Vertical Remapping ffl Vertical remapping has two phases ffi Locating the (new, iso-pycnal) layers ffi Interpolating from old to new layers ffl These are not completely separable ffi Can t locate layers without allowing for interpolation scheme ffl Finite Volume approach allows partial separation ffi Define a profile across original layers ffi Use profile in deciding where to put layers ffi Integrate this profile to get new layer averages ffl In HYCOM (hybgen) ffi Highly non-uniform layer thicknesses ffi Same number of layers ffi Most (iso-pycnal) layers don t change ffi Layers don t move more than one grid length(?) ffi Use PCM (Donor Cell) advection Λ Profile is constant across each layer Λ May be only practical scheme for hybgen
Off-line Vertical Remapping ffl Off-line remapping is harder ffi Highly non-uniform layer thicknesses ffi Different number of layers ffi Many layers change location ffi Layers can move a long way ffl Desirable properties ffi Simple to compute ffi Overall conservation ffi No change if layer is the same Λ Excludes linear between cell centers ffi No new extrema ffl PCM ffi Too diffusive ffi Lowest order of a family (PLM/PQM) ffl Piecewise Quadratic Method ffi Quadratic across cell ffi Very popular advection scheme ffi Tested it, but probably overkill
Off-line Vertical Remapping (II) ffl Piecewise Linear Method (Van Leer) ffi Linear profile, mean at cell center ffi Discontinuous at cell interfaces ffi Several choices for the slope ffi Tim Campbell developed interpolation variant Λ Non-uniform layer thickness Λ Maximize smoothness across interfaces Local optimum, global too much work ffl Archive to archive remapper ffi Only for fixed target coordinate Λ Hybrid to sigma-z ffl Archive to data3z ffi Returns fields at fixed depths ffi PCM or linear between cell centers ffl Single profile remapper ffi fixed depth version (data3z) ffi fixed cell version (PCM or PLM)
0 Salinity Profile, Year 2000, Day 03, i=251, j=092 PLM with Eq(21) PPM (mono, no steep) 50 depth (m) 100 150 200 35 35.2 35.4 35.6 35.8 36 psu
0 V-Velocity Profile, Year 2000, Day 03, i=251, j=092 200 400 depth (m) 600 800 PLM with Eq(21) PPM (mono, no steep) 1000-30 -25-20 -15-10 -5 0 5 10 cm/s
PLM Remapping of Fixed Coordinate Layers ffl In HYCOM 2.1.03, hybgen uses PCM ffl Perfect for iso-pycnal layers ffi Most (iso-pycnal) layers don t change Λ No remapping, no diffusion ffi Detrainment (thinning) does not change density Λ Unique to PCM Λ HYCOM might otherwise be impractical ffl Not optimal for fixed coordinates ffi Fixed layers always move ffi PCM is very diffusive ffl PCM is a special case of PLM ffi PLM with zero slope ffl Use PLM for fixed layers and PCM for iso-pycnals ffi Currently using same scheme for all variables ffi Could use PCM+PLM for density only, or density and salinity only
UP/DOWN-WELLING TEST - SIGMA-Z PCM (top) vs PLM
UP/DOWN-WELLING TEST - HYBRID PCM (top) vs PLM+PCM
BLACK SEA SST STATISTICS: PCM 09.1 vs Pathfinder : SST Mean Error (deg C) 09.1 vs Pathfinder : SST RMS Error (deg C) BLKa0.04 46N BLKa0.04 46N ci 0.03 0.96 to 5.13 ci 0.04-4.15 to 0.10 45N 45N 44N 44N 43N 43N 42N 42N 41N 41N 28E 29E 30E 31E 32E 33E 34E 35E 36E 37E 38E 39E 40E 41E 3. 09.1 vs Pathfinder : SST Correlation Coeff. BLKa0.04 46N 2.5 2. 1.5 1. 0.5 0. 1. 0.5 0. -.5-1. -1.5-2. -2.5-3. 09.1 vs Pathfinder : SST Skill Score 28E 29E 30E 31E 32E 33E 34E 35E 36E 37E 38E 39E 40E 41E BLKa0.04 46N ci 0.003 0.882 to 0.998 ci 0.003 0.347 to 0.975 45N 45N 44N 44N 43N 43N 42N 42N 41N 41N 1. 0.95 0.9 0.85 0.8 0.75 0.7 1. 0.95 0.9 0.85 0.7 0.8 28E 29E 30E 31E 32E 33E 34E 35E 36E 37E 38E 39E 40E 41E 0.75 28E 29E 30E 31E 32E 33E 34E 35E 36E 37E 38E 39E 40E 41E
BLACK SEA SST STATISTICS: PLM+PCM 09.5 vs Pathfinder : SST Mean Error (deg C) 09.5 vs Pathfinder : SST RMS Error (deg C) BLKa0.04 46N BLKa0.04 46N ci 0.03 0.77 to 4.52 ci 0.04-3.74 to 0.83 45N 45N 44N 44N 43N 43N 42N 42N 41N 41N 28E 29E 30E 31E 32E 33E 34E 35E 36E 37E 38E 39E 40E 41E 3. 09.5 vs Pathfinder : SST Correlation Coeff. BLKa0.04 46N 2.5 2. 1.5 1. 0.5 0. 1. 0.5 0. -.5-1. -1.5-2. -2.5-3. 09.5 vs Pathfinder : SST Skill Score 28E 29E 30E 31E 32E 33E 34E 35E 36E 37E 38E 39E 40E 41E BLKa0.04 46N ci 0.003 0.938 to 0.999 ci 0.003 0.503 to 0.986 45N 45N 44N 44N 43N 43N 42N 42N 41N 41N 1. 0.95 0.9 0.85 0.8 0.75 0.7 1. 0.95 0.9 0.85 0.7 0.8 28E 29E 30E 31E 32E 33E 34E 35E 36E 37E 38E 39E 40E 41E 0.75 28E 29E 30E 31E 32E 33E 34E 35E 36E 37E 38E 39E 40E 41E
UP/DOWN-WELLING TEST THIN DEEP ISO-PYCNAL LAYERS
OTHER NEW FEATURES OF NEXT RELEASE (SEPTEMBER?) ffl Arctic bi-polar patch with open Bering Strait ffi For 0.24 degree global domain ffl Bottom boundary layer in KPP ffl Diagnostics within HYCOM: ffi Time-averaged fields (archive files) ffi Drifters ffi Moored buoy sampling(?) ffi Transport section sampling(?) ffl Is anything else needed immediately?
WISHLIST FOR HYCOM (I) ffl Fully region-independent ffi Compile once, run on any region and any number of processors ffl Tidal forcing ffl Even more mixed-layer models ffl Support for data assimilation ffl Comprehensive tracer support ffi Within HYCOM and off-line ffi In z-space? ffl NCAR s CCSM Coupler ffi CICE sea-ice model ffi Coupled air-ocean-ice ffl ESMF compatibility ffi Earth System Modeling Framework http://www.esmf.ucar.edu/ ffi Alternative to CCSM ffi Much broader support ffi Might allow asynchronous I/O ffi Use ESMF for parallelization(?)
WISHLIST FOR HYCOM (II) ffl Improved/Alternative hybrid remapping (hybgen)? ffl Improved/Alternative advection (tsadvc)? ffl Improved/Alternative split-explicit time scheme? ffl Alternative free surface formulation? ffi Currently assume free surface is a small fraction of the total depth Λ Does this work for coastal domains? Λ Includes steric effects, but does not exactly conserve either mass or volume ffl Atmospheric pressure forcing? ffl Rivers (and surface E-P) as mass exchange? ffl Balance E-P via land precip runoff?