Design, Dev and Deploymnt of WI DNR Mapping Services

Design, Development, &
Deployment of Wisconsin DNR
Map Services

Jeff Shaw, John Laedlein, Jerry Sullivan
Wisconsin Department of Natural Resources
Bureau of Technology Services
GIS Services Section

Wisconsin Land Information Association - 2011 Annual Conference
February 16-18, Madison, WI

Design, Development & Deployment
of Wisconsin DNR Map Services
Presentation Content
This presentation shares experiences using ArcGIS Server
9.3.1 and 10 to build cached basemap services within an
applications development framework.

Initially, four types of cached services are being rolled out:
terrain services (NED hillshades),
imagery services (Landsat ETM+, NAIP 2008);
feature basemap services (ESRI Streetmap, DNR, DOT layers);
and a public lands layers service (WDNR, PADUS CBI, ESRI).

This talk will explore guiding concepts, technical design
issues, map rendering and labeling, caching considerations
and tips, and integration within an overall applications
framework.

AGS Map Service Design
& Caching Practices

Jeff Shaw
DNR, GIS Services

Cached Map Service Structure
L0: 1:9,000,000 L1: 1:4,500,000
C00000000 C00000001 C..0 C..1 C..2 C..3

R..0

R00000000

R..1

R..2

R00000001

R..3

Cache Folder Structure (Exploded format)

DNR Tiling Scheme

Coordinate System: Wisconsin Transverse Mercator (WTM)
Image Format: PNG32 or jpeg, 256x256
Scales: 16 levels* ( 1:8,110,080 to 1:990 )
495 *Based on scales useful for mapping (WGS-based services can’t do that,
990 because their scale varies with latitude)
1,980
3,960
7,920
15,840 1” = .25 mi (4” = 1 mi)
23,760 *
31,680 1”= .5 mi
47,520 *

* Origin
63,360 1”= 1 mi
95,040 *
126,720 1” = 2 mi
253,440
506,880 1” = 8 mi
1,013,760
2,027,520 1” = 32 mi
4,055,040
6,082,560*
8,110,080

Why Use a Custom Tiling Scheme?

Issues with WGS 84 Web Mercator-based tiling scheme
Scale changes with latitude (also distorts area, preserves shape)
7% difference in scale from N. Wisconsin to S. Wisconsin
31% difference from equator to central Wisconsin
WGS 1:72K scale is actually 1:53K in S. WI and 1:49K in N. WI
Services projected to WTM shrink 31% - text becomes too small
NOTE - Google & Bing don’t use ellipsoid and have .33% scale distortion in Y
direction “since used only for map display, not displaying coordinates”
Finer-grained scales desired
Potential desktop use
ArcMap assumes fixed scale when measuring !!! (v9.3.1) – will be 30% off w/WGS
Hundreds of layers stored in WTM Coordinate System

Projection on-the-fly performance – Threshold TBD
Note – You can edit WTM data w/in a web app displayed in WGS 84 provided you use the
Geometry service to project measurements..

Which Scales to Use?

If you might adopt WGS, and want to avoid designing around entirely
new scale thresholds…

WGS at Equator WGS at WI Lats Optimal WGS DNR Scales* Mapping units
9,244,667.36 6,536,967 7,168,000 8,110,080 1 inch = 128.0 miles
5,376,000 6,082,560 1 inch = 96.00 miles
4,622,333.68 3,268,483 3,584,000 4,055,040 1 inch = 64.0 miles
2,311,166.84 1,634,242 1,792,000 2,027,520 1 inch = 32.0 miles
1,155,583.42 817,121 896,000 1,013,760 1 inch = 16.0 miles
577,791.71 408,560 448,000 506,880 1 inch = 8.0 miles
288,895.85 204,280 224,000 253,440 1 inch = 4.0 miles
144,447.93 102,140 112,000 126,720 1 inch = 2.0 miles
72,223.96 51,070 56,000 63,360 1 inch = 1.0 miles
36,111.98 25,535 28,000 31,680 1 inch = .50 miles
18,055.99 12,768 14,000 15,840 1 inch = .25 miles
9,028.00 6,384 7,000 7,920 1 inch = .125 miles
4,514.00 3,192 3,500 3,960 1 inch = .063 miles
2,257.00 1,596 1,750 1,980 1 inch = .031 miles
1,128.50 798 875 990 1 inch = .016 miles
564.25 399 438 495 1 inch = .008 miles
*not including 1:95K, 1:47K & 1:23K
The cached images you might think would be displayed in ArcMap when projected, aren’t

Modifying a Tiling Scheme
for existing services

Adding new scale levels
New larger scales, not a problem
New intermediate scales
Rename levels
Add cached files for new levels
Update service config file and restart server
Need to clear cached files on client – or create a parallel
service that points at the same cache folder
Changing origin, image size, or coordinate system
– then need to regenerate the cache

Map Services

Cached map services:
Detailed Feature Basemap ”Dark” – lines & labels
Public Lands Basemap – polygon fill
Image Basemap – Hillshaded air photos & satellite imagery
Reference basemaps
Detailed Feature Basemap ”Light” – lines & labels
Basic Terrain Basemap ”Light” – tan hillshade

Basic Feature Basemap ”Light” – for use with light background Mashup basemaps

Dynamic map services:
Query Layers (PLSS, MCDs, GNIS, etc)
Air photos at 1:495 [ Future ??? ]

Future Cached Services:
Topo Map - simulates scanned topo-quads
Basic Feature Basemap “Dark”
Color Infra-Red (IR) Image Basemap
False Color Satellite

Move to and or have some services in WGS 84 Web Mercator-based scheme

Basemap Series
“Light” detailed basemap grp “Dark” detailed basemap group
Reference basemaps

“Light” simple basemap “Dark” simple basemap
Mashup Basemap

[ Future ]

“Light” Colored Basemap Group

Detailed “Dark” Basemap Group

Map Design Tips

Text halos
Light text with dark halo for dark background
Dark text with light halo for light background
Typical Maplex settings:
Repeat line labels & now polygons also (v10)
Abbreviation Dictionary
MSD-based anti-aliasing helps a lot!
Can use semi-transparent symbols (v10)
Labels on top
avoids placement conflicts & they don’t get covered up

Shared Cache Folder

All cache folders stored on the same Storage Area Network (SAN)
Dev, UAT and Production services all share the same
configuration file and cached files
Allows R&D, Dev, UAT, Prod and workstations to all read and
update the same cache folders.
Need less storage space
More horsepower for caching
No need to copy cached files

ags_cache
agsoutput
arcgiscache
MyCachedMapService
Layers
conf.xml
_alllayers

Shared Map Resource Folder

Store map resources on a file share
Multiple SOMS serving parallel services can use the
same map resource
Makes promoting changes easier
MyFileShare
Intranet
Caching_workstation
Development
UAT
Prod_staging
Production
Cached_Map
My_Resource.MSD
My_Resource.MXD
Dynamic_Map
Public
...

Staging Map Service

Used to generate the cached files
Exists on same disk drive as the published service
arcgisoutput
Arcgiscache
MyCachedService
MyCachedService-stage

Workflow
1. Populate the "staging” service levels
2. Move (drag) old levels folders into “trash” folder
3. Drag levels from “staging” service into the published one
4. Delete the “trash” files when convenient

You never need to copy files – would take 6 days for a statewide
image service (exploded format). ~6 hrs for compact format – still
too long for production

Proxy Map Resource

One “Staging” MXD/MSD used to generate the cache
Over 100 themes to cover all scales in “detailed” basemap
Use whatever name is useful to map author
(ex. 24k dissolved DNR_MGS_LNDS, 150K-300K, no labels)
Layers don’t need to be kept on-line
Can change/replace layers with less impact on service layer id
references
Technically, don’t need any layers – high availability

A different, “Proxy” MSD used for the published service
Fewer layers, less for AGS to manage
Can use an MSD, even if a MXD needed for caching
Better, simpler legend & user-friendly layer names
Select fields turned off or user-friendly alias assigned
Used for identify/find tasks
If using cache-on-demand at large scale(s), design for that

Cache File Management Tips

Turn off disk indexing

Turn off Windows Explorer thumbnail db
(can create locks)

Don’t use ArcMap to evaluate quality (v9.3.1)
(often resamples even when it shouldn’t)

Use RoboCopy to copy files, DOS command line to
delete files
(restarts automatically after network disconnection)

Cache Management

Log these per scale (level) per caching job:
# of Rows (folders)
# of Tiles (files)
Total size
Total size on disk
Total Duration

Cached to 1:1,980 scale
• Image: 4 days, 125 GB
• Feature: 5 days, 55 GB

Calculate size and processing time per image then use to estimate
processing time and disk storage space for entire service.

Problems/Bugs

Status bar is not reliable – rely on your log
Incomplete tiles – run the caching process twice (v 10)
To track number of tiles – count files to determine
completeness
Repeating labels can crash process (with AGS 10, SP 1)
Might generate incomplete images – don’t assume that
if it draws in ArcMap (even MSD) that it will work in
AGS.

Layer locking issues are resolved in ArcGIS 10!
(Use custom config parameter)

Sequential Layer ID References

DNR Service Registry - sample entry
<EN_Detailed_Features_DkBk>

<ID>EN_Detailed_Features_DkBk</ID><TYPE>STATIC</TYPE>
<URL>http://apwmad1a0229/ArcGIS/rest/services/DW_Map_Cached/EN_Detailed_Feature_Basem
ap_DkBk_WTM_Ext/MapServer</URL>
<SOAPURL>http://apwmad1a0229/ArcGIS/services/DW_Map_Cached/EN_Basic_Feature_Basema
p_DkBk_WTM_Ext/MapServer</SOAPURL>

<DNROFFICE id="2" shape="POINT"/>
<MCD id="7" shape="POLYGON"/>
<OPENWATER id="28" shape="POLYGON"/>
<COUNTY id="30" shape="POLYGON"/>

</EN_Detailed_Features_DkBk>

Issues

Outdated files cached on client
(need to clear client cache occasionally)
We might try:
• Saving a browser cookie that stores the date accessed
• Then, have application compare service update date with
last accessed date. Prompt if necessary.
■ Slow Cache-on-Demand (v9.3.1)
Conveying Positional Accuracy - disclaimers
Acknowledgements for layers used in public services

DNR Image Basemap Caches
John Laedlein – DNR, GIS Services

• External Image Basemap Cache
• Internal Image Basemap Cache
• Future…

External Image Basemap
Cache

• Currently available in Production
• Presented in Public DNR-Managed
Lands application
• Landsat ETM+ Imagery
• 10-meter NED Hillshade
• 2008 NAIP Orthos

Cache
Landsat,10m NED, 2008 NAIP
• Downloaded from USGS GloVis site
(Bands 7, 4 & 2)
• 14-meter resolution, extent WI &
adjacent states
• Cached at small scales
• Rendered as false natural color
• Data available on DNR FTP site
(~900mb) ftp://dnrftp01.wi.gov/geodata

Cache
• Acquired by DNR/FR in July 2009
• Extent buffered WI
• Cached at intermediate to large
scales as backdrop to semi-
transparent imagery
• Data available on DNR FTP site
(~130mb) ftp://dnrftp01.wi.gov/geodata

Cache
• Public domain imagery
• 1-meter, leaf-on, natural color
• Extent USGS Q-Q-Quads covering
WI
scales
• Data available from WisconsinView
site http://www.wisconsinview.org/

Internal Image Basemap
Cache
• Landsat ETM+ and 10m NED
Hillshade at small scales
• Most current “high-resolution” orthos
at intermediate to large scales
• Image metadata issues

Cache
Most current available high-res
orthos
• Mostly leaf-off
• Mostly includes orthos restricted to
internal DNR use
• Mostly a mix of 18-inch, 1-foot & 6-
inch orthos
scales, mainly from MrSID county
mosaics

Cache
Conditioning of imagery for caching
• Need to minimize MrSID “map collar”
artifacts
• 2008 NAIP orthos used where no
high-res imagery available
• Edited County boundary feature class
included for metadata purposes

Cache
Minimize MrSID “map collar”
artifacts
• Attempt to render map collars
transparent
• Adjust drawing order in staging mxd
• “Despeckle” black collars
• “Patch” most serious areas of white
collar overdrawing

Future Image Basemap
Caches
• 2010 NAIP Orthos
• 2010 WROC 18-inch statewide public
domain product
• Topo Basemap
– Including DRGs & elevation contours
– Alternatives being evaluated
• Color Infrared Basemap
– Including Landsat ETM+ CIR
– 2010 NAIP CIR
– DNR/FR CIR imagery

Choosing Vector Data for
Caching: Hydrography, Roads,
Public Lands
Caching vector basemaps requires
evaluating many alternative geodata
sources …

and often adding themes many, many
times to the Staging.mxd,

to set rendering and labelling properties
appropriate to the chosen caching
scales.

Public Lands
Hydrography – Basically, 2 choices
WDNR 1:24K Hydro, or USGS
National Hydrography Dataset (NHD) 2.0

An overriding consideration is
completeness of Stream Order, to
facilitate generalization;

And completeness of Flow Direction, for
large scale views.

NHD 2.0 is still not very ready …

Blue – Stream level coded

Pink – No stream level coded !

Public Lands
Roads required more mixing:
• Interstate & US Highways (3)
• Major Roads (4)
• DOT WISLR County Roads (5)
• Other Major Roads (6)
• Local Streets (7)
• Additional Street Classes (8)

Public Lands
• WDNR Managed Lands – many views
• US Forest Service – from WI office
• US Fish & Wildlife Service – cadastral
• National Park Service – gateway
• Other Federal Lands –
• Tribal Lands – TIGER ? PAD ? GLIFWC ?
• County Forests – WDNR or PAD ?
• Protected Areas Database – CBI (not USGS)
• County & Municipal Parks – ESRI

Public Lands
• Public Lands (0)
• County Forests - 80K out (1)
• State Natural Areas - non-sensitive (2)
• DNR Managed Lands (3)
– DNR Managed Lands (68K-150K) (4)
– DNR Managed Lands (150K - 500K) (5)
– DNR Managed Lands (dissolved) -
solid - 500K-2M (6)
– DNR Managed Lands (dissolved) -
solid - 2M Out (7)

Public Lands
• Local Parks (8)
– Local Parks (1:150-800k) (9)
– Local Parks (40K-150k) (10)
– Local Parks (40k in) (11)
• Federal Lands (12)
– Native American Lands - PAD - all scales (13)
– National Forest Boundaries - 150K in - USFS (14)
– Nat’l Forest Boundaries - 150K - 1.3M USFS (15)
– National Forest Boundaries - 1.3M-3M ESRI (16)
– Natl. Forest Boundaries - 3M out Streetmap (17)
– Other Federal Lands – PADUS-CBI - 150K in (18)
– Other Federal Lands – PADUS-CBI - 150K out (19)

Design, Dev and Deploymnt of WI DNR Mapping Services

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (7)

Ähnlich wie Design, Dev and Deploymnt of WI DNR Mapping Services

Ähnlich wie Design, Dev and Deploymnt of WI DNR Mapping Services (20)

Mehr von Wisconsin Land Information Association

Mehr von Wisconsin Land Information Association (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Design, Dev and Deploymnt of WI DNR Mapping Services