TerraServer RIP

“Two roads diverged in a yellow wood”

Fig 1 MapSavvy WMS referenced in AutoCAD Map 3D

A Brief History of WMS

Open Geospatial Consortium, OGC, has been an influential geospatial standards body since the late 90’s. OGC standards are at the root of SQL spatial databases, commercial GIS tools, as well as most open source mapping projects found at OSGeo. OGC standards have been adopted at all levels from Microsoft SQL Server, Oracle Spatial, ESRI, and AutoCAD to PostGIS, GeoServer, and Quantum GIS. OGC standards are part of the DNA of modern GIS and internet mapping.

One of the more popular OGC standards, first published at the turn of the millennium, is Web Mapping Service. WMS is a map distribution standard for showing arbitrary BBox map areas in common internet browsers.

One of the early significant WMS services was a partnership between Microsoft Research and the USGS called Microsoft TerraServer, which was in continuous operation from 1998 until today, 5/1/2012. More recently the name was changed to MSR Maps. Microsoft TerraServer pioneered free availability of geospatial data in an OGC compliant service. At the same time, TerraServer research established the viability of quad tile storage, but then merged tiles into arbitrary BBox chunks for WMS GetMap requests.

TerraServer was grandfather to two forks of web mapping, quad tile pyramids and WMS open data services. May 1st 2012 marks the end of this pioneering service.

requiescat in pace

How WMS works

WMS defines a standard method of publishing capabilities to the web and furnishing images of requested areas. A WMS client first asks what layers are available, in what formats, styles, and projections. Some decisions are made depending on need and availability. These decisions along with a specific bounding box are sent back to the WMS server. The WMS server builds the requested image and returns it as a rectangular png, jpeg, tiff etc. Behind the WMS server is some type of data repository, either vector data or imagery, but the WMS server handles all requests for specific parts of this data resource.

Fig2 - OGC Web Mapping Service

This was an ambitious scheme in 2000, attempting to smooth the distribution of a multitude of proprietary formats and display methods. The concept was simply to make access to spatial data a “write once, use anywhere architecture.” Momentum built slowly at first, but by this time, May of 2012, there are many choices for WMS servers, and WMS clients are built into a multitude of mapping tools and websites.

KaBoom! Two roads

In 2005 this whole approach was upended with the introduction of web maps and a new tile serving slippy map architecture. The problem WMS ran into was the limit of CPU resources attempting to build individual map requests for millions of users. Servers eventually bogged down trying to fulfill requests for arbitrary WMS viewports. The new slippy map architecture doesn’t build anything. Tiles flow to clients simply as requested, directly from storage or cache. The browser client is responsible for rendering, so web mapping suddenly became massively parallel. Map pan and zoom went from request, pause, display, to continuous uninterrupted motion, quite magical at its first introduction.

This approach was a recapitulation of TerraServer storage but dropped the WMS at the end of the server pipe. Tile pyramids became just static image stores streaming into a map client as needed by pan and zoom.

WMS was fine for the technical folks and enterprise focus on proprietary assets, but it just didn’t scale into the internet consumer realm. Web mapping forked at this point. WMS continued to grow as a basic function of every desktop map tool and internet client, while web mapping adopted vast tile pyramids prebuilt from vector features and aerial imagery. Microsoft Bing Maps, MapQuest, and Google Maps are the glamorous handmaidens of search, serving up map tiles to the masses. OGC services haul water behind the scenes, with the pocket protector crowd.

Between Two Roads

The two architectures coexist in the internet realm. OGC services are work horses in engineering cubicles, but the mobile generation is all about slippy tiles. Many web apps merge one of the world scale base map tile pyramids with layers of OGC services on top. This approach marshals world level extent together with locally maintained assets. Example Fig 3

Fig 3 – Bing Maps behind Census Bureau TIGER WMS/WFS layers

Against the Stream

WMS layers can live on top of popular web maps like Bing, but what about the opposite direction? Can WMS clients make use of worldwide tile pyramids?

“A dead thing can go with the stream, but only a living thing can go against it.”
G.K. Chesterton

OnTerra MapSavvy

Far down stream from Jim Gray and Tom Barclay, OnTerra created a simple WMS service to bridge the fork in the road between OGC and Bing Maps tiles, but in the opposite direction. MapSavvy is a WMS service that simply exposes Bing Maps Aerial and Road imagery to existing users of WMS clients. Subscribe to the service and imagery for the entire world is available inside your mapping tool of choice.

There are thousands of man years of development tied up in tools that perform very specific business functions. These tools are not especially public facing. They perform the required task, but the task is of no particular interest to the rest of us. Companies have to pay people to use tools like this. In cases of business focused WMS tools, wouldn’t it be nice to access the vast resources of tile imagery abounding on the internet?

For example, Microsoft is just finishing a program to capture most of the US and Western Europe at 30cm pixel resolution and serving this in Bing Maps Aerial. This kind of resolution at national scales is a very useful resource. Designers, engineers, defense agencies, and GIS analysts can always use higher resolution imagery available for large parts of the planet. Lower resolution imagery fills in context for the remainder of the world. Ref: Bing Maps resolution chart

Bing Maps 30cm Aerial: “The project is scheduled for completion by June 2012 at which point a refresh cycle will begin to update much of what will already have been collected, with priority placed on locations that have been more subject to change (as opposed to, say, the Mojave Dessert).” See Fig 4

Fig 4 – Global Mapper WMS client using MapSavvy - Mojave Desert 0.30 m per pixel

Here is a short list from the multitude of WMS clients:

• AutoCAD Map 3D – Fig 1
• Bentley MicroStation V8i
• Gaia
• Global Mapper – Fig 4
• Tableau
• Smallworld GeoSpatial
• gvSig – Fig 5
• Quantum GIS – Fig 6
• MapInfo
• TatukGIS Viewer
• TopoFusion

Fig 5 – gvSIG WMS client using MapSavvy - Coors Field in Denver

Fig 6 – QGIS WMS client - Detail of Coors Field Blake Street entrance - 0.30 m per pixel

Some Details

Bing Maps, like all web mapping tile services, leverages the symmetry of a Mercator world projection, which is technically defined as epsg:3857. In order to support the widest range of WMS clients MapSavvy publishes both epsg:3857 and epsg:4326. The second projection has less value, as it distorts the imagery, but, in cases where a WMS client cannot make use of the more accurate epsg:3857, epsg:4326 is helpful. This is especially useful, at higher resolutions where the distortion is reduced. MapSavvy helps resolve some of the distortion issues of epsg:4326 by using transformed images for zoom levels at the top of the tile pyramid.

Unlike Google Maps, which explicitly excludes use of Google tiles in a WMS service, Bing Maps wants customers to use their tile servers. OnTerra licenses Microsoft Bing Map tiles for use in a WMS. MapSavvy subscriptions include a restricted Microsoft sub-license for use of Bing road and imagery tiles in the client, which simplifies life for small business users.

Some TerraServer Alternatives

TerraServer has historical interest but in the past five years there hasn’t been much demand for DOQ, Urban Ortho, or DRG Topo maps. The demise of TerraServer may go largely unnoticed. For those left orphaned, National Atlas publishes several WMS endpoints. However, Bing Maps Aerial is a much improved replacement for USGS DOQ and USGS Urban Ortho. MapSavvy WMS can replace TerraServer DOQ and Urban Ortho layers directly in existing WMS clients.

If you’re looking for alternative USGS Topo contours, Ahlzen’s TopOSM is a good place to start.

TopOSM Contours

Fig 7 TopOSM contours over Bing Maps


Microsoft TerraServer was the starting point for a lot of what we take for granted in 2012. Its passing is understandable but leaves a twinge of regret.

Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;

- Robert Frost

“When you arrive at a fork in the road, take it.”
- Yogi Berra

Leaflet and Census TIGERweb

Fig 1 – Sample Leaflet interface to the US Census Bureau’s new TIGERweb WMS service with Bing Maps background

Ref: http://silverlight.onterrasys.com/MapSavvyLeaflet/

US Bureau of the Census – some background

1990 Census
The 1990 Census line data made a revolutionary leap from GBF-DIME format to TIGER, Topologically Integrated Geographic Encoding and Referencing, and also adopted CD-ROM media distribution. In the early 90′s I wrote a few TIGER translation programs for AutoCAD users and users of a popular (now defunct) desktop GIS package called Atlas*GIS. TIGER was useful for its linear data, but was more often used for the boundary polygons that could be tied to Census population demographic data. The line features of the 1990 TIGER were based on the older 1980 GBF-DIME enhanced with USGS 1:100,000 DLG features. GBF DIME was notable for the tell tale zigzag streets in urban areas.

2000 Census
By the mid 90’s Open GIS Consortium (OGC) was actively working on OpenGIS standards. As the internet continued ramping up, OGC released the OpenGIS Web Map Server (WMS) Specification in 2000. Even though Census data releases are a year or so after the actual census, it was several more years before the Census Bureau started investigating alternative WMS services for distributing data. In the meantime boundaries were available for internet and ftp download in ESRI shp and E00 formats. This was convenient, but requires self hosting web solutions.

2010 Census
Ever since the 2000 WMS specification there were hopes for a service distribution of census TIGER/Line. Not until the end of 2011 was this belatedly realized as TIGERweb. But of course by the mid 2000s internet mapping began changing radically with the popularity of Google Maps, GPS, Bing Maps, Open Street Maps, and now ubiquitous mobile devices. The Census Bureau will need to continue evolving to meet the challenges of these rapid changes.

TIGERweb 2011
So why not use the TIGERweb for an experimental look at some more recent web tools? TIGERweb Viewer UI is based on the ESRI Silverlight APIs. Silverlight is a powerful UI technology, but it is destined to be supplanted by html5. As browser market fragmentation and mobile devices force adoption of standards based technologies, proprietary plugins are just not making the transition, at least for public facing web apps.

Fig 2 – TIGERweb Viewer based on ESRI Silverlight API

TIGERweb Viewer seems oddly designed. Rather than the clean, spare look popular today, TIGERweb Viewer seems to be a throwback to an earlier internet. Unusual matrice animations with rotating panels and creepy crawling boundary selectors seem a bit out of place in the modern web. Even though technically possible, and simple to implement in Silverlight, such distractions do nothing to make the UI endearing. After the first blink most users will inevitably recall early internet flashing text and PowerPoint decks filled with animated gifs. Anyone remember the irritation of Clippy?

However, TIGERweb WMS Service is a wide open door for standards based javascript libraries. One of the more elegant available tools is LeafletJS. This is a “lightweight” js map library that works well with jQuery and abstracts away some of the complexity of supporting a fragmenting mobile browser world.

Leaflet is Open Source BSD-licensed with an active contributing community and available on github. It uses newer html5 technologies available in A-grade browsers, but includes fall back for older browsers. It’s nice to have touch events and the geolocation api.

The Leaflet api provides a convenient L.TileLayer.WMS object that transforms a WMS endpoint into a slippy tile map. Performance isn’t as good as a static tile pyramid, but it isn’t bad and simplifies life immensely.

  var censusTracts = new L.TileLayer.WMS("http://tigerweb.geo.census.gov/ArcGIS/services/tigerWMS/MapServer/WMSServer",
    layers: 'Census Tracts',
    format: 'image/png',
    transparent: true,
    attribution: TigerAttribution

Bing Maps
Unlike Google Maps, Microsoft Bing allows licensed access to tile servers. For development, non-profit, and education uses, this is just a matter of registering for a Bing key. Adding a Bing Maps layer with Leaflet is possible using https://gist.github.com/1221998. Once this Leaflet Bing extension javascript is referenced into a page, adding Bing Map styles is simply a matter of adding L.TileLayer.Bing.

var BingAerialWithLabels = new L.TileLayer.Bing(bingkey, "AerialWithLabels",
  maxZoom: 21,
  attribution: BingAttribution

Fig 3 – Leaflet Census TIGER WMS with Bing Maps Road style

TIGER/Line 2010 accuracy has been upgraded substantially, but you can occasionally see some discrepancies against Bing Aerial.

Fig 4 – Leaflet Census TIGER WMS with Bing Maps Aerial with Labels style

Once all of the WMS and Bing L.TileLayers are instantiated, a new L.Map with the desired L.TileLayer set is declared this way:

            map = new L.Map('mapDiv',
                        center: new L.LatLng(39.748, -104.981),
                        zoom: 8,
                        layers: [censusStates, censusCounties, censusTracts, censusBlockGrps, censusBlocks]

Leaflet also provides a simple L.Control.Layers for a user layer control panel:

            var baseMaps = {
                "BingRoads": BingRoad,
                "BingAerial": BingAerial,
                "BingAerialWithLabels": BingAerialWithLabels

            var overlayMaps = {
                "States": censusStates,
                "Counties": censusCounties,
                "Tracts": censusTracts,
                "BlockGroups": censusBlockGrps,
                "Blocks": censusBlocks,
                "Lakes": censusArealHydrography,
                "Rivers Streams": censusLinearHydrography,
                "LocalRoads": censusLocalRoads,
                "SecondaryRoads": censusSecondaryRoads,
                "PrimaryRoads": censusPrimaryRoads,
                "Nexrad": nexrad

            var layersControl = new L.Control.Layers(baseMaps, overlayMaps,
                collapsed: !L.Browser.touch

Fig 5 – Leaflet L.Control.Layers

TIGERweb WMS GetFeatureInfo is available for queries. For example:


<?xml version="1.0"?>
<FeatureInfoResponse xmlns="http://www.esri.com/wms" xmlns:esri_wms="http://www.esri.com/wms">
  HU100="111214" --HousingUnits
  NAME="Marin County"

jQuery makes XMLHttpRequest communication easy, but TIGERweb WMS doesn’t expose an open ClientAccessPolicy.xml, so a self hosted proxy is required.

                type: "GET",
                url: getInfoURL,
                dataType: "xml",
                success: GetFeatureInfo

Fig 6 – Leaflet Popup panel with the results of TIGERweb WMS GetFeatureInfo request

Some Caveats

1) The TIGERweb WMS furnishes images for all the TIGER boundary and line features. However, there doesn’t seem to be a WMS layer for labels like those on the TIGERweb Viewer. There is an undocumented label service that might be useful.


2) WMS GetFeatureInfo requests can provide information from the polygon stack, but the lack of a cross-domain policy file on the TIGERweb WMS server means that a pass through proxy service needs to be hosted on the same server as the Leaflet web app. This proxy service simply provides the same GetFeatureInfo result to the client while avoiding cross-domain issues.

1) Bing javascript
The Bing javascript extension seemed to run into a problem with intersect at this line:
if (zoom = coverage.zoomMin && coverage.bbox.intersects(bounds))

Removing the intersect(bounds) clause seemed to get things working without noticeable problems:
if (zoom = coverage.zoomMin )

2) Leaflet seems to have problems with TileLayer ordering. Bing TileLayers can mask other features if directly added to the map layer set.

3) There are some odd behaviors on Amazon Kindle Fire, possibly due to the Silk web browser’s EC2 caching.

4) Leaflet L.Control.Layers doesn’t work consistently in the mobile browsers I tested. The collapsed layer control icon never opened for iPhone or Kindle Fire. Adding the suggested option collapsed: !L.Browser.touch keeps the layer control open instead of closing when untouched. This wouldn’t be a problem except that mobile devices don’t have the screen space for extra panels.

5) The default attribution for Bing gets quite lengthy, especially when displaying Aerial vendor meta data. Turning this off and deferring to the Bing Maps Logo frees up more screen space which is at a premium on mobile devices.


Census Bureau’s new TIGERweb is a great step forward for users of TIGER data. Hopefully the momentum will continue forward to include demographic data. SF1 data joined to TIGER polygons would be a more useful service. Adding an OGC WFS service would be the ideal, but WFS always has problems with potentially very large result sets, so I’m not sure this is practical on a public OGC service.

Since the first time I saw Leaflet I’ve been intrigued with its elegance and the potential for abstracting away some of the issues surrounding web apps targeted at mobile and touch clients. In spite of a few minor issues Leaflet is one of the best javascript map apis I’ve used. I believe the issues I ran into are on a bug list and could be resolved by next release.

This bare bones CensusTest project took only a few hours over the weekend to write using Leaflet.

Fig 7 – Lower Manhattan

Storm Runoff modelling and MRLC

The Multi-Resolution Land Characteristics Consortium, MRLC, is a consortium of agencies at the federal level that produces the National Land Cover Database, NLCD 2001. The dataset was developed using a national coverage set of 8 band Landsat-7 Imagery along with 30m DEM. The imagery is processed from three separate dates to give a seasonal average land cover classification. The resolution is a bit coarse at 30m square, but it is a valuable resource because of its consistent national coverage.

More detailed information on MRLC:

In addition to the NLCD coverage there are two derivative layers:

  • NLCD 2001 impervious surface: The impervious surface data classifies each pixel into 101 possible values (0% – 100%). The data show the detailed urban fabric in which many of us reside. Low percentage impervious is in light gray with increasing values depicted in darker gray and the highest value in pink and red. White areas have no impervious surface.
  • NLCD 2001 canopy density: Like the impervious surface data, the canopy density database element classifies each pixel into 101 possible values (0% – 100%). The canopy density estimate apply to the forest only. These data can be combined with the land cover to estimate canopy density by forest type (deciduous, evergreen, mixed, woody wetland)

The data is available for public download. There is also one of those vintage ESRI viewers that qualifies for James Fee’s “Cash for Geo Clunkers” proposal. These Ancien Régime viewers are littered all over the federal landscape. It will take years for newer technology to replace this legacy of ArcIMS. Fortunately there is an exposed WMS service ( See GetCapabilities ), which permits access to MRLC layers without going through the “Viewer” nonsense. This WMS service proved very useful on a recent web project for Storm Runoff Mitigation.

I am no hydrologist, but once I was provided with the appropriate calculation approach the Silverlight UI was fairly straightforward. Basically there are a number of potential mitigation practices that play into a runoff calculation. Two fairly significant factors are Impervious Surface Area and Canopy Area, which are both available through MRLC’s WMS service. One simplified calculation model in use is called the TR-55 method.

By making use of the MRLC derived layers for Impervious Surface and Canopy at least approximations for these two factors can be derived for any given area of the Continental US. The method I used was to provide a GetMap request to the WMS service which then returned a pixel shaded image of the impervious surface density. Most of the hard work has already been done by MRLC. All I need to do is extract the value density for the returned png image.

Impervious Surface layer
Fig 3 – Impervious Surface shaded density red
Impervious Surface layer
Fig 4 – Canopy shaded density green

The density values are relative to gray. I at first tried a simple density calculation from the color encoded pixels by subtracting the base gray from the variable green: Green – Red = factor. The sum of these factors divided by the total pixel area of the image times the max 255 byte value is a rough calculation of the percentage canopy over the viewport. However, after pursuing the USGS for a few days I managed to get the actual percentile RGB tables and improve the density calculation accuracy. This average density percentile is then used in TR-55 as An*CNn with the Canopy CN value of 70.

The process of extracting density from pixels looks like this:

  HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(new Uri(getlayerurl));
  using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
    if (response.StatusDescription.Equals("OK"))
      using (Stream stream = response.GetResponseStream())
        byte[] data = ReadFully(stream, response.ContentLength);
        Bitmap bmp = (Bitmap)Bitmap.FromStream(new MemoryStream(data), false);

        UnsafeBitmap fast_bitmap = new UnsafeBitmap(bmp);
        PixelData pixel;
        string key = "";
        double value = 0;
        for (int x = 0; x < bmp.Width; x++)
          for (int y = 0; y < bmp.Height; y++)
            pixel = fast_bitmap.GetPixel(x, y);
            key = pixel.red + " " + pixel.green + " " + pixel.blue;
            if (imperviousRGB.Contains(key))
              value += Array.IndexOf(imperviousSurfaceRGB, key) * 0.01;

        double total = (bmp.Height * bmp.Width);
        double ratio = value / total;
        return ratio.ToString();

C#, unlike Java, allows pointer arithmetic in compilation marked unsafe. The advantage of using this approach here is a tremendous speed increase. The array of imperviousRGB strings to percentiles was supplied by the USGS. This process is applied in a WCF service to both the Canopy and the Impervious Surface layers and the result passed back to the TR-55 calculations.

Possible Extensions:

There are several extensions beyond the scope of this project that could prove interesting.

  1. First the NLCD uses a color classifications scheme. A similar color processing algorithm could be used to provide rough percentages of each of these classificcations for a viewport area. These could be helpful for various research and reporting requirements.
  2. However, beyond simple rectangular viewports, a nice extension would be the ability to supply arbitrary polygonal area of interests. This is fairly easy to do in Silverlight. The draw command is just a series of point clicks that are added to a Path geometry as line segments. The resulting polygon is then used as a clip mask when passing through the GetMap image. Probably a very simple point in polygon check either coded manually or using one of the C# ports of JTS would provide reasonable performance.
MRLC NLCD 2001 Colour Classification
Fig 3 - MRLC NLCD 2001 Colour Classification

What about resolution?

It is tempting to think a little bit about resolution. Looking at the MRLC image results, especially over a map base, it is obvious that at 100 ft resolution even the best of calculations are far from the fine grained detail necessary for accurate neighborhood calculations.

It is also obvious that Impervious Surface can be enhanced directly by applying some additional lookup from a road database. Using pavement estimates from a road network could improve resolution quality quite a bit in urbanized areas. But even that can be improved if there is some way to collect other common urban impervious surfaces such as rooftops, walkways, driveways, and parking areas.

NAIP 1m GSD 4 band imagery has fewer bands but higher resolution. NAIP is a resource that has been applied to unsupervised impervious surface extraction. However, the 4 band aquisition is still not consistently available for the entire US.

Now that more LiDAR data is coming on line at higher resolutions, why not use LiDAR classifications to enhance impervious surface?

Lidar All Elevation
LidarAll Classification
Lidar All Intensity
Lidar All Return

Just looking at the different style choices on the LidarServer WMS for instance, it appears that there are ways to get roof top and canopy out of LiDAR data. LiDAR at 1m resolution for metro areas could increase resolution for Canopy as well as rooftop contribution to Impervious Surface estimates.

In fact the folks at QCoherent have developed tools for classification and extraction of features like roof polygons. This extraction tool appled over a metro area could result in a useful rooftop polygon set. Once available in a spatial database these polygons can be furnished as an additional color filled tile pyramid layer. Availability of this layer would also let the Runoff calculation apply rooftop area estimates to roof drain disconnect factors.

Additionally improved accuracy of impervious surface calculations can be achieved by using a merging version of the simple color scan process. In a merging process the scan loop over the MRLC image does a lookup in the corresponding rooftop image. Any pixel positive for rooftop is promoted to the highest impervious surface classification. This estimate only applies so long as roof top green gardens remain insignificant.

Ultimately the MRLC will be looking at 1m GSD collection for NLCD with some combination of imagery like NAIP and DEM from LiDAR. However, it could be a few years before these high resolution resources are available consistently across the US.


The utility of WMS resources continues to grow as services become better known and tools for web applications improve. Other OWS services like WFS and WCS are following along behind, but show significant promise as well. The exposure of public data resource in some kind of OGC service should be mandatory at all government levels. The cost is not that significant compared to the cost effectiveness of cross agency, even cross domain, access to valuable data resources.

By using appropriate WMS tools like Geoserver and Geowebcache, vastly more efficient tile pyramids can become a part of any published WMS service layer. It takes a lot more storage so the improved performance may not be feasible for larger national and worldwide extents. However, in this Runoff Mitigation project, where view performance is less important, the OGC standard WMS GetMap requests proved to be quite useful for the TR-55 calculations and performance adequate.

My EPSG:54004 mystery solved!

EPSG:54004 problem fig 1
Fig 1 – DIA Looks a Lot Better!

With a helpful comment from SharpGIS I was able to finally pin down my baffling problem with EPSG:54004.

The problem is in the datums.



As Morten pointed out the 54004 datum includes a flattening, 298.257223563 :

So 54004 should be treated as an Ellipsoid rather than a Sphere.

There is a subtle difference in 900913. If you notice 900913 also includes a flattening:


PROJCS["Google Mercator",
    GEOGCS["WGS 84",
        DATUM["World Geodetic System 1984",
            SPHEROID["WGS 84",6378137.0,298.257223563,
        AXIS["Geodetic latitude",NORTH],
        AXIS["Geodetic longitude",EAST],

However, you might not notice in addition it includes an explicit minor axis parameter.
And this minor axis is identical to the major axis. The axis definition overrides the flattening in the Datum and is probably technically incorrect, but the idea was just to get a spherical mercator into a definition that people could use to match Google Maps. I’ve seen this definition in PostGIS, GeoServer, and OpenLayers.

I had already noticed this and played with a MercatorEllipsoid function to see if that would fix my problem. However, sadly, I made an error and miscalculated eccentricity. The real equation for e goes something like this:
double f = 1 / 298.257223563;
e = Math.Sqrt(2*f – Math.Pow(f,2));

resulting in e = 0.081819190842621486;

Once I made the correction for the proper eccentricity in MercatorEllipsoid, ESRI:54004 lines up with the EPSG:3857. DIA is back in its rightful place.

My MercatorEllipsoid function now calculates correct BBOX parameters for GetMap requests, but only for com.esri.wms.Esrimap services. Looks like ESRI is the expert and correctly produces ESRI:54004 with Datum as defined. However, not so with GeoServer.

GeoServer seems to ignore the flattening 298.257223563 or else assume it is like the 900913 where flattening is overridden by a minor axis parameter:
semi_minor axis = major.PARAMETER[semi_minor,6378137.0]

This leads to some problems. My WMS client now has to decide which service correctly interprets the DATUM on 54004. For now I just check for “com.esri.wms.Esrimap” in the WMS url and change datums accordingly. This will undoubtedly lead to problems with other services since I don’t yet know how MapServer or others treat 54004.


  1. ESRI is right again!
  2. Always check your math one more time
  3. Community responses produce answers

Thanks everyone!

Neo vs Paleo Geography

Jurassic Dinosaur skeleton
Fig 1 – Paleo, Neo – what about Jurassic Geography?

I gather that there is some twittering about neo versus paleo geography. See Peter Batty’s blog entry or James Fee’s blog. I myself don’t Twitter, but in general I’m happy for Peter’s paleo accomodation of the non twitterers, repeating the conversation in a blog entry. Peter has also updated comments with a new post questioning, “Are we now in a post neogeography era?” The dreaded paradigm shifts are coming fast and furiously.

I am not really able to comment on neo vs paleo as I myself fall even further back into “Jurassic Geography.” Looking at connotations we have this accounting:

····neo - 1990 – present, new, recent, different, Obama, Keynesian, Apple, Google Earth, Cloud, Java C# RubyRails Twitter

····paleo - as in paleolithic 2.8m – 10k old, prehistoric, ancient, early, primitive, Nixon, Supply Side, Microsoft, Windows Desktop, ESRI Arc???, C C++ Javascript telephone

Obviously the “paleo” label is not carried with quite the honor of “neo.” It’s reminiscent of the Galen / Myers-Brigg personality typology characterized as Lion, Otter, Beaver, and Golden Retriever. What do you want to be? Obviously not the beaver, but there has to be a significant part of the world in that category, like it or not. After all what would lions eat for dinner without a few of us beavers? Likewise there is a significant branch of paleo in the GIS kingdom.

However, in the pre-paleolithic era there are still a few of us left, falling into the “long tail” of the Jurassic. So carrying on down the connotation stream here is the Jurassic geography equivalent:

····jurassic – 206m-144m dinosaurs, fossils, pre paleolithic, Hoover, laissez faire, IBM Big Iron, Assembly Cobol, open source

Wait “Open Source” – Jurassic Geography? How did that get in there? The notoriously frugal days of Hoover never made it into the paleolithic era’s “Supply Side” economy. It’s Keynesian economics all over the neo world, so Jurassic geography is the frugal end of the spectrum and how can you get more frugal than free! Obviously Open Source is as Jurassic as they come in Geography circles.

As I’ve recently been in a gig hunting mode, I’ve been having quite a few in depth conversations about GIS stacks. As a small businessman outside the corporate halls of paleo geography, I’ve had few occasions to get an in depth education on the corporate pricing world. So I spent the past couple of days looking into it.

Let’s start at the bottom of the stack. Here is some retail pricing on a few popular GIS databases:

  • Oracle Standard Spatial $17,500 + $3850 annual
  • Oracle Enterprise Locator $47,500 + $10,450 annual
  • SQL Server 2008 Web edition ~ $3500
  • PostgreSQL/PostGIS $0.00

If you’re a Jurassic geographer which do you choose? Probably not Oracle Enterprise Locator. If your Paleo you look at that and think, “Man, I am the negotiator! We don’t pay any of that retail stuff for the masses.” Neo? – well how would I know how a neo thinks?

Next take a look at the middle tier:

  • ESRI ArcGIS Server standard workgroup license
    ····Minimum $5000 2cores + $1250 2core annual
    ····Additional cores $2500/core + $625/core annual
  • ESRI ArcGIS hosted application server license
    ····Minimum $40,000 4 cores + $10,000 4 core annual
    ····Additional cores $10,000/core + $2500/core annual
  • OWS GeoServer or MapServer minimum $0 + annual $0
    But, this is GIS isn’t it? We want some real analytic tools not just a few hundred spatial functions in JTS Topology suite. OK, better throw in a few QGIS or GRASS installs and add a few $0s to the desktop production. Oh, and cores, we need some, “make that a 16core 64 bit please” – plus $0.

I think you catch the Jurassic drift here. How about client side.

  • ESRI Silverlight free, well sort of , if you’re a developer, NGO, educational, or non-profit otherwise take a look at that ArcGIS license back a few lines.
  • Google API it’s Neo isn’t it? $10k per annum for a commercial use, maybe its Paleo after all.
  • Virtual / Bing Maps api $8k per annum transaction based and in typical license obfuscation fashion impossible to predict what the final cost will be. Paleo, “Just send me the invoice.”
  • OpenLayers is a javascript api client layer too, just solidly Jurassic at $0
  • Silverlight well it can be Jurassic, try DeepEarth over at codeplex or MapControl from Microsoft with the Bing imageservice turned off, OSM on.

It’s been an interesting education. Here is the ideal Jurassic GIS stack:
Amazon EC2 Windows instance + PostGIS database + GeoServer OWS + IIS Silverlight MapControl client
The cost: starts at $100/mo(1 processor 1.7Gb 32bit) up to $800/mo(4 processor 15Gb 64bit)

So what does a Jurassic geographer get in this stack?

Amazon Cloud based virtual server, S3 Backup, AMI image replication, Elastic IP, AWS console, choice of OS, cores, memory, and drive space. Ability to scale in a big way with Elastic load balancing, auto scaling, and CloudWatch monitoring. Performance options like CloudFront edge service or something experimental like Elastic MapReduce Hadoop clusters.

PostgreSQL/PostGIS – Standards compliant SQL server with GIST spatial indexing on OGC “Simple Features for SQL” specification compliant geometry with extended support for 3DZ, 3DM and 4D coordinates. A full set of roughly 175 geometry, management, and spatial functions. It supports almost all projections. All this and performance? maybe a little vague but not shabby:

“PostGIS users have compared performance with proprietary databases on massive spatial data sets and PostGIS comes out on top.”

Middle Tier:
Geoserver – standards compliant OWS service for WMS, WFS, WCS.
Data sources: Shapefile, Shapefile Directory, PostGIS, external WFS, ArcSDE, GML, MySQL, Oracle, Oracle NG, SQL Server, VPF
Export formats: WFS GML, KML, SVG, PDF, GeoRSS, Png, Jpeg, Geotiff, OGR Output – MapInfo Tab and MID/MIF, Shp, CSV, GeoJSON …
OGC standard SLD styling, built in gwc tile caching – seeded or as needed, managed connection pools, RESTful configuration api, and ACEGI integrated security.

WCS adds :

  1. ArcGrid – Arc Grid Coverage Format
  2. ImageMosaic – Image mosaicking plugin
  3. WorldImage – A raster file accompanied by a spatial data file
  4. Gtopo30 – Gtopo30 Coverage Format
  5. GeoTIFF – Tagged Image File Format with Geographic information
“GeoServer is the reference implementation of the Open Geospatial Consortium (OGC) Web Feature Service (WFS) and Web Coverage Service (WCS) standards, as well as a high performance certified compliant Web Map Service (WMS). “

Browser client viewer:
Take your pick here’s a few:

Well in these economic times Jurassic may in fact meet Neo. The GIS world isn’t flat and Jurassic going right eventually meets Neo going left, sorry Paleos. Will Obama economics turn out to be Hooverian in the end? Who knows, but here’s a proposition for the Paleos:

Let me do a GIS distribution audit. If I can make a Jurassic GIS Stack do what the Paleo stack is currently providing, you get to keep those annual Paleo fees from here to recovery. How about it?

rkgeorge @cadmaps.com

WMS 3D with Silverlight and WPF

Silverlight view of DRCOG WMS
Fig 1 – Silverlight view of DRCOG WMS Layers

WPF 3D view of DRCOG WMS Layers
Fig 2 – WPF 3D view of DRCOG WMS Layers

Silverlight 2.0 MapControl CTP makes it easy to create a WMS viewer. However, Silverlight exposes only a 2D subset of the full WPF xaml specification. In my experiments so far with LiDAR I can see the various TINS, contours, and point clouds as a 2D view, but what I would really like to do is view data in 3D. Recalling some work I did a few years ago using WPF GeometryModel3D elements I decided to look at a WPF loose xaml approach. This is not restricted to use with LiDAR WMS services. Here in Colorado relief is major part of life and it would be helpful to visualize any number of WMS services using a 3D terrain model. Google and VE(Bing) already do this for us using their proprietary DEM, but eventually I would like to use different DEM resolutions, anticipating Open Terrain sources or in house LiDAR. DRCOG’s WMS service is conveniently available and lies right on the edge of the Rocky Mountain uplift so it is a nice WMS service to highlight relief visualization.

First navigating to the WPF service using Silverlight is a bit different from html. The familiar anchor element is gone. One Silverlight approach is to add a HyperlinkButton:<HyperlinkButton Content=”3D” TargetName=”_blank” NavigateUri=”"/> and change the NavigateUri as needed. Another approach is to use Html.Page.Window.Navigate() or Html.Page.Window.NavigateToBookmark() allowing a click handler to setup the navigate:

        private void ClickTest_Button(object sender, RoutedEventArgs e)
            Uri testwpf = new Uri("http://www.cadmaps.com");
            HtmlPage.Window.Navigate(testwpf, "_blank");

Using one of these Navigate methods the Uri can be pointed at the service used to create the 3D loose xaml.

The next item is getting 3D terrain to populate the xaml mesh. Eventually LiDAR Server will provide 3D surface data, but in the meantime JPL has a nice trick for their srtm and us_ned WMS service. One of the selectable styles is “feet_short_int”. This style will produce a grayscale image with elevation coded into the int16 pixels. Using these elevations I can then create the xaml mesh along with some primitive manipulation tools.

Ref: JPL GetCapabilities

    <Layer queryable="0">
      <Title>United States elevation, 30m</Title>
        Continental United States elevation, produced from the USGS National Elevation.
        The default style is scaled to 8 bit from the orginal floating point data.
      <LatLonBoundingBox minx="-125" miny="24" maxx="-66" maxy="50"/>
      <Style><Name>default</Name> <Title>
Default Elevation</Title> </Style>
      <Style><Name>short_int</Name> <Title>
short int signed elevation values when format is image/png or tiff
</Title> </Style>
      <Style><Name>feet_short_int</Name> <Title>
short int elevation values in feet when format is image/png or image/tiff
</Title> </Style>
      <Style><Name>real</Name> <Title>
DEM real numbers, in floating point format, meters, when used with image/tiff
</Title> </Style>
      <Style><Name>feet_real</Name> <Title>
DEM in real numbers, in floating point format, feet, when used with image/tiff
</Title> </Style>
      <ScaleHint min="20" max="10000" /> <MinScaleDenominator>24000</MinScaleDenominator>

JPL us_ned 30m
Fig 3 – JPL us_ned 30m can be returned as feet_short_int

My first attempt used a .ashx service handler. I could then obtain the Bitmap like this:

     WebClient client = new WebClient();
     byte[] bytes = client.DownloadData(new Uri(terrainUrl));
     Bitmap bmp = (Bitmap)Bitmap.FromStream(new MemoryStream(bytes), true);

Unfortunately the returned pixel format of the Bitmap is incorrect. Instead of PixelFormat.Format16bppGrayScale the Bitmap returned PixelFormat.Format32bppArgb. This is not exactly helpful, quantizing the elevations. Instead of an elevation at each pixel GetPixel returns an RGB like 2A 2A 2A meaning e = p.B * 256 ^ 2 + p.G * 256 ^ 1 + p.R * 256 ^ 0; is incorrect. It appears that the returned value is quantized on the R value since to be gray each value of RGB must be equal. I checked the image returned by JPL using GdalInfo. This shows 2 band Uint16 values. Since Bitmap reads in a 4 byte value (2 Uint16s) it apparently defaults to Format32bppArgb and is then useless for actual elevation.

Driver: PNG/Portable Network Graphics
Files: C:\Users\rkgeorge\Pictures\wms.png
Size is 500, 351
Coordinate System is `'
Image Structure Metadata:
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  351.0)
Upper Right (  500.0,    0.0)
Lower Right (  500.0,  351.0)
Center      (  250.0,  175.5)
Band 1 Block=500x1 Type=UInt16, ColorInterp=Gray
Band 2 Block=500x1 Type=UInt16, ColorInterp=Alpha

color quantized example
Fig 4 – Example of a color quantized elevation from forced 32bppArgb

There is probably some type of pointer approach to pass through the raw bytes correctly. However, services are a nice way to decouple a view from the backend and it is just as easy to use a Java servlet which can take advantage of JAI, so in this case I switched to a Java servlet:

URL u = new URL(terrainUrl);
image = JAI.create(“url”, u);
SampleModel sm = image.getSampleModel();
int nbands = sm.getNumBands();
Raster inputRaster = image.getData();
int[] pixels = new int[nbands*width*height];

The resulting pixels array is int16[] so the correct 30m us_ned elevations are available from the JPL WMS service. Using the resulting elevations I can fill in a Positions attribute of my WPF MeshGeometry3D element. Since this is an array it is also easy to add a TriangleIndices attribute and a TextureCoordinates attribute for the draped image. The final loose xaml will eventually show up in the browser like this:

WPF view of DRCOG layers draped on JPL terrain
Fig 5 - WPF view of DRCOG layers draped on JPL terrain

Because I am simplifying life by using loose xaml there is no code behind for control of the views. However, we can use binding to attach various control values to camera positions and geometry transforms that allow a user to move around the terrain even without a full blown code behind xbap. For example:

    Position="0 0 " + dimw + ""
    FieldOfView="{Binding ElementName=zoomlensslider, Path=Value}">
          OffsetX="{Binding ElementName=hscroll, Path=Value}"
          OffsetY="{Binding ElementName=vscroll, Path=Value}"

In this case the camera FieldOfView is bound to a slider value and the camera position transform offsetx and offsety are bound to scrollbar values. Binding values to controls in the same xaml file is a powerful concept that provides dynamic capability without any code behind.

Code behind with an xbap would actually make more useful navigation tools possible, but this is a good start. The controls allow the user to zoom the camera lense, change camera x,y position, tilt and rotate the viewport as well as exaggerate the relief scale. Changing to an xbap that handles the terrain image translation with code behind would allow the use of mouse wheel and click hits for pulling up item data fields similar to the Silverlight WMS viewer.

As great as WPF is, it would still be a difficult problem to turn it into a tiled interface like the SIlverlight MapControl. I imagine the good folks at Microsoft are working on it. Basically the mesh elements would be tiled into a pyramid just like imagery tiles. Then these tiles would be loaded as needed. It is easy to imagine an algorithm that would work with the backing pyramid as a 3D view curve. In other words distance from the camera would translate to tiles at higher pyramid levels of lower resolution. A view field would pull tiles from the mesh pyramid using a distance curve so that lower resolution would fill in further ranges and higher resolution at closer ranges. Some kind of 3D will eventually be available but is beyond the scope of my experiment.

Some Problems

1) JPL WMS has some difficulty keeping up with GetMap requests as noted on their website, which means GetMap requests are unreliable. Sometimes there is no issue and other days the load is heavier and there are few times when a GetMap request is possible. Here is their explanation:
"Due to server overloading, client applications are strongly advised to use the existing tile datasets wherever possible, as described in the Tiled WMS or Google Earth KML support

Frequent and repetitive requests for non-cached, small WMS tiles require an excessive amount of server resources and will be blocked in order to preserve server functionality. The OnEarth server is an experimental technology demonstrator and does not have enough resources to support these requests.
An alternative solution already exists in the form of tiled WMS
While sets of tiles with different sizes and alignment can be added when needed, for large datasets the duplication of storage, processing and data management resources is prohibitive."

2) Silverlight works pretty well cross browser at least on my Windows development systems. My tests with Mozilla Firefox managed Silverlight and WPF loose xaml just fine. However, Google Chrome has problems with WPF loose xaml reporting this error:
"This application has failed to start because xpcom.dll was not found"

A quick Google search came up with this:
Chromium does not support XPCOM. It is a Mozilla specific API.

More details here:

3) Another baffling issue arose with the WPF loose xaml. It turns out that each click of the LinkButton or access of the url for the WPF service caused the service to be accessed three times in succession. This was true of both the .ashx and the servlet versions. The User Agent was not distinguishable so I wasn't able to short circuit the first two GET accesses. I only found this true for context.Response.ContentType = "application/xaml+xml"; not text/html etc. So far I haven't seen anything that would solve my problem. It is too bad since the computation for 15Mb WPF files is significant and three times for each result is a bit unnecessary. It may be something quite simple in the response header. I will keep an eye out. There was a similar problem with IE6 years ago, but in that case the service could watch for a Request.UserAgent notation and skip the unecessary duplicate calls.

4) WPF is verbose and 15Mb terrain views take a bit of patience especially considering the above problem. I need to add an auto xaml gzip to my Apache Tomcat. I'm sure there is something similar in IIS if I get around to a pointer solution to the grayscale.


Silverlight is a natural for interface development and integrates well with WPF ClickOnce for the 3D views that are still not available in Silverlight. My hope is that someday there will be a SIlverlight version capable of 3D scene graph development.

The LiDAR WMS Server roadmap also includes a style for grayscale elevations at some point. That would enable viewing much more detailed high resolution DEM directly from LAS files. I believe this will be a useful advance for use of online LiDAR viewing.

Also a thanks to DRCOG. I'm always glad to see agencies willing to support the wider GIS community by publishing data as OGC services and then further supporting it with adequate server performance. I just wish JPL had a bit more budget to do the same.

Silverlight view of LiDAR Server TIN
Fig 6 - Silverlight view of LiDAR Server TIN

WPF 3D view of LiDAR Server TIN
Fig 7 - WPF 3D view of LiDAR Server TIN

Open up that data, Cloud Data

 James Fee looks at AWS data and here is the Tiger .shp snapshot James mentions: Amazon TIGER snapshot
More details here: Tom MacWright

Too bad it is only Linux/Unix since I’d prefer to attach to a Windows EC2. TIGER is there as raw data files ready to attach to your choice of Linux EC2. As is Census data galore.

But why not look further?  It’s interesting to think about other spatial data out in the Cloud.

Jeffrey Johnson adds a comment to spatially adjusted about OSM with the question – what form a pg_dump or a pg database? This moves a little beyond raw Amazon public data sets.

Would it be possible to provide an EBS volume with data already preloaded to PostGIS? A user could then attach the EBS ready to use. Adding a middle tier WMS/WFS like GeoServer or MapServer can tie together multiple PG sources, assuming you want to add other pg databases.

Jeffrey mentions one caveat about the 5GB S3 limit. Does this mark the high end of a snapshot requiring modularized splitting of OSM data? Doesn’t sound like S3 will be much help in the long run if OSM continues expansion.

What about OpenAerial? Got to have more room for OpenAerial and someday OpenTerrain(LiDAR)!
EBS – volumes from 1 GB to 1 TB. Do you need the snapshot (only 5GB) to start a new EBS? Can this accommodate OpenAerial tiles, OpenLiDAR X3D GeoElevationGrid LOD. Of course we want mix and match deployment in the Cloud.

Would it be posible for Amazon to just host the whole shebang? What do you think Werner?

Put it out there as an example of an Auto Scaling, Elastic Load Balancing OSM, OpenAerial tile pyramids as CloudFront Cache, OpenTerrain X3D GeoElevationGrid LOD stacks. OSM servers are small potatoes in comparison. I don’t think Amazon wants to be the Open Source Google, but with Google and Microsoft pushing into the Cloud game maybe Amazon could push back a little in the map end.

I can see GeoServer sitting in the middle of all this data delight handing out OSM to a tile client where it is stacked on OpenAerial, and draped onto OpenTerrain. Go Cloud, Go!

Silverlight WMS Viewer

Silverlight MapControl OWS Viewer
Fig 1 – Silverlight MapControl CTP used as a Simple WMS Viewer

In the past couple of blogs I’ve looked at using SQL Server and PostGIS overlays on top of the Silverlight VE MapControl CTP. I’ve also looked at using PostGIS mediated by Geoserver as cached tile sources with GeoWebCache. As these investigations have shown, map layers can be provided using basic shape overlays directly from spatial database tables or as image tile sources such as GeoWebCache.

There is another possibility: using an Image element with its Source pointed at a WMS service. Although this approach is not as interactive as vector shape overlays or as efficient as a tile source, it does have one big advantage. It can be used to access public WMS services. Currently the most common OWS services in the public realm are WMS. WFS has never really seemed to take off as an OWS service and WCS is still a bit early to tell. Using a public WMS means conforming to the service as exposed by someone else. Many, probably most, WMS services do not have a tile caching capability, which means access to a public WMS in Silverlight MapControl requires a Silverlight Image element like this:

     <Image x:Name="wmsImage" Source="the WMS Source"></Image>

where ‘the WMS Source’ looks similar to this:
    &BBOX=-84.2103683719356, 24.8162723396119,
      -78.1019699344356, 28.2556356649882

If we use a Silverlight Image element to display a WMS GetMap request we have a basic WMS Viewer. However, there are some hurdles to overcome.

WMS servers provide several url encoded requests, GetCapabilities, GetMap, and for layers that are queryable GetFeatureInfo. The workflow is generally to look at a GetCapabilities result to discover the exposed layers and information about their extent, style, and query capabilities. In addition there is usually information about supported formats, coordinates systems, and possibly even scale ranges. After examining the GetCapabilities result, the next step is configuring a GetMap request, which is then placed in the Silverlight xaml hierarchy as an Image element. Once that is accomplished an optional third step is to use the map layer for GetFeatureInfo requests to look at attribute data for a selected feature. There is another request, DescribeLayer, which is not needed for this project.


The first hurdle in creating a Silverlight WMS viewer is accessing the XML returned from a GetCapabilities request and processing it into a useable Silverlight control. First there is a cross domain issue since our Silverlight viewer wants to show WMS map views from a different server. As in previous projects this is done by providing a WCF proxy service on the Silverlight server to provide access to different WMS services at the server rather than incurring the wrath of a client cross domain access.

Here is an example of a client initialization:
    svc = GetServiceClient();
    svc.GetCapabilitiesCompleted += svc_GetCapabilitiesCompleted;

and here is WMSService.svc GetCapabilities, which simply takes a GetCapabilities url encoded query and feeds back out to the svc_GetCapabilitiesCompleted the result as an XML string.

public string GetCapabilities(string url)
  HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
  using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
    StreamReader reader = new StreamReader(response.GetResponseStream());
    return (reader.ReadToEnd());

On the xaml client side we then pick up the resulting xml using XLinq to Parse into an XDocument.

private void svc_GetCapabilitiesCompleted(object sender, GetCapabilitiesCompletedEventArgs e)
  if (e.Error == null)
    XDocument document = XDocument.Parse(e.Result, LoadOptions.None);
    foreach (XElement element in document.Element

XLinq is very easy to use for manipulating XML, much easier in C# than using DOM commands. The intent of XLinq XML is similar to Java JDOM in providing an access syntax more appropriate to a language like C#. For example, document.Element(“WMT_MS_Capabilities”).Element(“Capability”).Element(“Layer”).Descendants(“Layer”), provides a collection of all the layer elements descending from the head layer in our GetCapabilities result. Using the layer sub elements we can build a simple selection menu from CheckBox and ComboBox elements. The main sub elements of interest are:



<LatLonBoundingBox minx=”-124.731422″ miny=”24.955967″ maxx=”-66.969849″ maxy=”49.371735″/>


Here is an example of the simplicity of XLinq’s query capability:

  XElement layer = document.Element("WMT_MS_Capabilities").Element("Capability").Element("Layer");
  var testsrs = from srs in layer.Elements("SRS") select (string)srs;
  if (testsrs.Contains("EPSG:900913")) EPSG900913.IsEnabled = true;
  else EPSG900913.IsEnabled = false;

XLinq is a pleasure to use combining aspects of JDOM, JAXB, and SQL, and making life easier for developers.

The final selection menu looks like this:

Silverlight MapControl Selection menu
Fig 1 – Silverlight Layer Selection Menu

An opacity slider is easy to add and lets users see through opaque layers. Not all WMS services support ‘Transparent=True’ parameters.

private void Slider_ValueChanged(object sender, RoutedPropertyChangedEventArgs e)
    OpacityText.Text = string.Format("{0:F2}", e.NewValue);

    foreach (UIElement item in layers.Children)
        if (item is CheckBox)
            if ((bool)(item as CheckBox).IsChecked)
                MapLayer lyr = VEMap.FindName((item as CheckBox).Content.ToString()) as MapLayer;
                lyr.Opacity = e.NewValue;


Now come a few more hurdles for GetMap requests. A layer selection or view change triggers a call to SetImage(MapLayer lyr, string style)

  private void SetImage(MapLayer lyr, string style)
    if (lyr.Children.Count > 0) lyr.Children.Clear();// clear previous layer
    int aw = (int)VEMap.ActualWidth;
    int ah = (int)VEMap.ActualHeight;
    LocationRect bounds = VEMap.GetBoundingRectangle();
    Point sw = new Point(bounds.Southwest.Longitude, bounds.Southwest.Latitude);
    Point ne = new Point(bounds.Northeast.Longitude, bounds.Northeast.Latitude);
    string wmsurl = wmsServer + "?SERVICE=WMS&Version=1.1.1&REQUEST=GetMap&LAYERS=" +
    lyr.Name + "&STYLES=" + style + "&TRANSPARENT=TRUE&SRS=EPSG:4326&BBOX=" +
    sw.X + "," + sw.Y + "," + ne.X + "," + ne.Y +
    "&FORMAT=image/png&EXCEPTIONS=text/xml&WIDTH=" + aw + "&HEIGHT=" + ah;
    Image image = new Image();
    image.Source = new BitmapImage(new Uri(wmsurl, UriKind.RelativeOrAbsolute));
    lyr.AddChild(image, new Location(bounds.North, bounds.West));
    image.MouseLeftButtonDown += Image_MouseClick;

Quite simple, however, there is a problem with this, which can be seen below.

Coordinate system problem
Fig 3 – Coordinate system problem

EPSG:4326 is versatile, since it’s supported by every WMS service I’ve run across, but this geographic coordinate projection system does not match Virtual Earth base maps, or Google, Yahoo, and OSM for that matter. All of the web mapping services use a tiled spherical mercator projection. Fortunately they all use the same one. At this point there is a relatively new official EPSG designated 3785. And, here is an interesting discussion including some commentary on EPSG politics: EPSG and 3785.

However, for awhile now Geoserver has included a pseudo EPSG number, “900913,” to do the same thing and it is already incorporated into my PostGIS spatial_ref_sys tables and the Geoserver version I am using. Unfortunately MapServer chose a different pseudo EPSG, “54004,” which illustrates the problem of an existing standards body getting miffed and dragging their feet. <SRS>EPSG:900913</SRS>, <SRS>EPSG:54004</SRS>, and EPSG:3785 are similar mathematically speaking, but it will take a few iterations of existing software before EPSG:3785 is distributed widely. In the meantime this is the definition distributed with Geoserver:

    900913=PROJCS["WGS84 / Simple Mercator", GEOGCS["WGS 84",
         DATUM["WGS_1984", SPHEROID["WGS_1984", 6378137.0, 298.257223563]],
         PRIMEM["Greenwich", 0.0], UNIT["degree", 0.017453292519943295],
         AXIS["Longitude", EAST], AXIS["Latitude", NORTH]],
         PARAMETER["latitude_of_origin", 0.0], PARAMETER["central_meridian", 0.0],
         PARAMETER["scale_factor", 1.0], PARAMETER["false_easting", 0.0],
         PARAMETER["false_northing", 0.0], UNIT["m", 1.0], AXIS["x", EAST],
         AXIS["y", NORTH], AUTHORITY["EPSG","900913"]]

In order to make a GetMap Image match a spherical Mercator we make a couple of modifications. First change “EPSG:4326″ to “EPSG:900913″ or “EPSG:54004.” Then provide a conversion like this for the BBox parameters (more details -Mercator projection)

  private Point Mercator(double lon, double lat)
    /* spherical mercator for Google, VE, Yahoo etc
     * epsg:900913 R= 6378137
     * x = long
     * y= R*ln(tan(pi/4 +lat/2)
    double x = 6378137.0 * Math.PI / 180 * lon;
    double y = 6378137.0 * Math.Log(Math.Tan(Math.PI/180*(45+lat/2.0)));
    return new Point(x,y);

Note that VEMap.GetBoundingRectangle(); guarantees a bounds that will never reach outside the 85,-85 latitude limits. (more precisely -85.05112877980659, 85.05112877980659) Using the above to calculate a BBox in spherical coordinates results in:

  sw = Mercator(bounds.Southwest.Longitude, bounds.Southwest.Latitude);
  ne = Mercator(bounds.Northeast.Longitude, bounds.Northeast.Latitude);

This makes the necessary correction, but not all WMS services publish support for EPSG:900913, EPSG:54004, or EPSG:3785. How to work around this dilemma? Unfortunately I have not yet found a solution to this problem, short of doing an image warp with each view change. Since some WMS services do not support any of the spherical Mercator projections, I’m left with a default EPSG:4326 that will not overlay correctly until zoomed in beyond ZoomLevel 7.

higher zoom
Fig 4 – Coordinate system problem disappears at higher zoom

One more hurdle in GetMap is the problem with a dateline wrap or antimeridian spanning. This occurs when a view box spans the longitude -180/180 break. After checking around a bit, the best solution I could come up with is splitting the image overlay into west and east parts.
if (bounds.West > bounds.East) I know I will need more than one tile:

  LocationRect boundsE = new LocationRect(bounds.North, bounds.West, bounds.South, 180.0);
  LocationRect boundsW = new LocationRect(bounds.North, -180.0, bounds.South, bounds.East);

Although this works for a 360 degree viewport, zoom levels > 2.5 are actually greater than 360 degrees and I need to work out some sort of Modular Arithmetic for more than two parts. ZoomLevel 1 will require 4 separate tiles. My problem currently is that detecting the actual number of degrees in a viewport is problematic. VEMap.GetBoundingRectangle() will return East and West Extents but this will not indicate width in actual degrees for a clock arithmetic display that repeats itself. For now I’m taking the simplest expedient of turning off the overlay for ZoomLevels less than 2.5.

Multiple earths at ZoomLevel 1
Fig 5 – Virtual Earth ZoomLevel 1 with viewport > 360deg


GetFeatureInfo requests are used to send a position back to the server to look up a list of table attributes for feature(s) intersecting that position. Not all WMS layers expose GetFeatureInfo as indicated in the GetCapabilities <Layer queryable=”0″>.

The basic approach is to handle a click event which then builds the necessary WMS GetFeatureInfo Request. There is a mild complication since UIElements do not possess MouseClick events. Substituting a MouseLeftButtonDown event is a work around to this lack: image.MouseLeftButtonDown += Image_MouseClick; There are other possibilities such as using SharpGIS.MouseExtensions. However, the Silverlight MapControl seems to mask these extensions. Until I can work out that issue I defaulted to the simple MouseLeftButtonDown event.

   private void Image_MouseClick(object sender, MouseButtonEventArgs e)
      Image img = sender as Image;
      Point pt = e.GetPosition(img);
      var location = VEMap.ViewportPointToLocation(e.GetPosition(VEMap));
      BitmapImage bi = img.Source as BitmapImage;
      string info = bi.UriSource.AbsoluteUri.ToString();
      int beg = info.IndexOf("LAYERS=")+7;
      int end = info.IndexOf('&',beg);
      string layer = info.Substring(beg, end-beg);
      info = info.Replace("GetMap", "GetFeatureInfo");
      info += "&QUERY_LAYERS=" + layer + "&INFO_FORMAT=application/vnd.ogc.gml
      &X=" + pt.X + "&Y=" + pt.Y;

      InfoPanel.Visibility = Visibility.Collapsed;
      InfoPanel.SetValue(MapLayer.MapPositionProperty, location);
      MapLayer.SetMapPositionOffset(InfoPanel, new Point(10, -150));

By attaching the event to each image element, I’m able to make use of bi.UriSource, just modifying the request parameter and adding the additional GetFeatureInfo parameters. There are some variations in the types of Info Formats exposed by different WMSs. I selected the gml which is supported by GeoServer and text/xml for use with USGS National Atlas.

private void svc_GetFeatureInfoCompleted(object sender, GetFeatureInfoCompletedEventArgs e)
  if (e.Error == null)
    XDocument document = XDocument.Parse(e.Result, LoadOptions.None);
    XNamespace wfs = "http://www.opengis.net/wfs";
    XNamespace gml = "http://www.opengis.net/gml";
    XNamespace geo = "http://www.web-demographics.com/geo";
    if (document.Element("ServiceExceptionReport") == null)
      if (document.Element("FeatureInfoResponse") == null)
        XElement feature = document.Element(wfs + "FeatureCollection").Element(gml + "featureMember");
        if (feature != null)
          XElement layer = (XElement)feature.FirstNode;
          if (layer != null)
            TextBlock tb = new TextBlock();
            tb.Foreground = new SolidColorBrush(Colors.White);
            tb.Text = layer.Name.ToString().Substring(layer.Name.ToString().IndexOf('}') + 1);
            InfoPanel.Visibility = Visibility.Visible;
            IEnumerable de = layer.Descendants();
            foreach (XElement element in de)
              string name = element.Name.ToString();
              name = name.Substring(name.IndexOf('}') + 1);
              XNamespace ns = element.Name.Namespace;
              string prefix = element.GetPrefixOfNamespace(ns);
              if (!prefix.Equals("gml"))
                tb = new TextBlock();
                tb.Foreground = new SolidColorBrush(Colors.White);
                tb.Text = name + ": " + element.Value.ToString();


There are some minor problems using Silverlight VE MapControl as a generic OWS Viewer.

  1. Lack of general support for EPSG:3785 in many public WMS servers
  2. Detecting viewport extents for ZoomLevels displaying more than 360 degrees
  3. Missing MouseClick and double click events for UIElements

None of these are show stoppers and given a couple more days I’d probably work out solutions for 2 and 3. The biggest hurdle is lack of support for spherical Mercator coordinate systems. As long as you have control of the WMS service this can be addressed, but it will be some time before EPSG:3785 propagates into all the WMS services out in the wild.

On the plus side, Silverlight’s sophistication makes adding nice design and interaction features relatively easy. Again having C# code behind makes life a whole lot easier than dusting off javascript. I’m also really liking XLinq for manipulating small xml sources.

Next on the project list will be adding some nicer animation to the information and menu panels. Further in the future will be adding WFS view capability.

Silverlight MapControl OWS Viewer
Fig 1 – Silverlight MapControl CTP WMS Viewer showing USGS Atlas

A Noob in Oracle Land

The announcement that Oracle is supporting AMIs in the Amazon cloud came as a surprise to me. I had heard that there was a teaser version of Oracle out there for developers, but had not expected Oracle to jump on the cloud side, especially after Larry Ellison’s recent diatribe against cloud computing.

   “It’s complete gibberish. It’s insane. When is this idiocy going to stop?”

Just curious about this oracle of gibberish, I went on a tour of Oracle Land, the Kingdom of Ellison. This is no small undertaking for an enterprise as ambitious as Oracle. There are endless products and sub-products. The base of the pyramid is the database server, but after buying 50 or more companies in the last year or so, the borders of the empire extend way beyond RDBMS.

The venerable RDBMS has come a long way since IBMs E.F. Codd introduced the concept back in the 70s. I vaguely remember Oracle breaking into the PC world shortly after Turbo Pascal. There was a single DB product for the DOS IBM PC, and documentation consisted of a couple of grayish paperback manuals. Shortly after this, late 80s, a small vendor introduced GeoSQL to hook AutoCAD to the GIS world through Oracle. This was my first introduction to the potential of spatial databases and Oracle. The empire of Ellison has grown since then, and now documentation would fill a library as well as Ellisons bank account.

As an aside, we live in an interesting age at the dusk of the great technology innovators. The infamous industrialists of the previous era now exist only as shadowy figures in history texts, but the business innovators of technology are still walking among us, Larry Ellison, Bill Gates, Steve Jobs. The multi-billion personal fortunes are just now entering the charitable fund phase where our grand children will know their names in some impersonally institutional mode such as the Gates Foundation.

First stop in Oracle Land was a download of the free, as in free beer, teaser version, OracleXE.

  • Total data stored in XE is limited to 4GB
  • XE is limited to 1GB of RAM
  • XE is limited to 1 processor

Since my entire interest in Oracle is the spatial side, my next stop was Justin Lokitz’s helpful article on integration with Geoserver. Leading to this:

Fig 1 -http://localhost:80/geoserver/wms?service=WMS&request=GetMap&format=

Fig 2 – http://localhost:80/geoserver/wms/kml_reflect?layers=COUNTIES

Not a bad start. The Geoserver layer abstracts away the spatial guts of OracleXE. However, curiosity leads on. I found that OracleXE has some spatial components labelled ‘Locator’ as opposed to ‘Spatial’. Though only a subset of the extensive enterprise spatial version, geometry queries are possible. It took me a bit to find my way around.

Interestingly the open source world is generally more helpful in this respect. Although extensive, the forums of commercial software vendors are less friendly. For instance Paul Ramsey of Refraction fame is regularly present on the PostGIS forums, and Frank Warmerdam is always available to give a helping hand at the immensely useful www.gdal.org. But I doubt that I will ever run across a Larry Ellison post on the OracleXE forum. Many posts to commercial forums appear to languish unanswered, which is seldom the case in the OpenSource project forums I monitor.

It is worth noting that gdal’s ogr2ogr can be built with Oracle support on systems with Oracle Client libraries installed.

Oracle’s SDO_Geometry is present in a useful form letting users run geographic join queries like this:

   select c.COUNTY, c.STATE_ABRV, c.TOTPOP, c.POPPSQMI from states s, counties c where s.state = ‘California’ and sdo_anyinteract (c.geom, s.geom) = ‘TRUE’;

My next step was to look at SDO_Geometry in JDBC. Unfortunately Oracle’s JGeometry spatial library is not available for OracleXE, but the LGPL open source JTS library provides helpful OraReader and OraWriter classes. These encapsulate the SDO_GEOMETRY Struct translation to/from jts.geom.Geometry, where the rest of the JTS api can be applied.

logger.info(rsmd.getColumnName(i)+": "+rsmd.getColumnType(i));
st = (oracle.sql.STRUCT) rs.getObject(1);
//convert STRUCT into JGeometry not available in OracleXE
//JGeometry j_geom = JGeometry.load(st);

//JTS to the rescue
OraReader reader = new OraReader();
Geometry geom = reader.read(st);
Coordinate[] coords = geom.getCoordinates();

Next stop, Amazon AWS EC2. Here is a list of the public Oracle AMIs offered::
   Oracle Database 11g Release 1 Enterprise Edition – 64 Bit
   Oracle Database 11g Release 1 Enterprise Edition – 32 Bit
   Oracle Database 11g Release 1 Standard Edition/Standard Edition One – 32 Bit
   Oracle Database 10g Release 2 Express Edition – 32 Bit

The last in the list, OracleXE edition, is the one to experiment with, unless you have a spare Oracle license floating around.

Time to try it:
  C:\>ec2-run-instances ami-7acb2f13 -k gsg-keypair
  C:\>ec2-describe-instances i-??????

and login:

Use of this machine requires acceptance of
the following license agreements.
 1. Oracle Enterprise Linux


 2. Oracle Technology Developer License Terms


 Please enter the above URLs into your browser and review them.
To accept the agreements, enter 'y', otherwise enter 'n'.
Do you accept? [y/n]: y
Thank you.

You may now use this machine.
Welcome to Oracle Database on EC2!
This is the first time this EC2 instance has been started.

Please set the oracle operating system password.
Please specify the passwords for the following database administrative accounts:

SYS (Database Administrative Account) Password:

Now for the link to Apex on the new OracleXE instance:

Fig 3 – Oracle Apex running from an EC2 OracleXE instance

Looks like we have it.

Oracle is the Big Daddy of spatial GIS. It is also the “Mother of all DBA complexity.” Running a spatial app with oracle in the background is not trivial, but it is getting easier. The EC2 OracleXE AMI makes starting an Oracle server instance a matter of minutes. Although lacking some of the capability of its free and open source competition, OracleXE can be useful for the garden variety web enabled spatial app. For the developer with lots of experience in Oracle, OracleXE provides a low cost entry onto the performance/price escalator.

Next on the agenda is adding SDO_GEOMETRY data along with some kind of real spatial rendering, which means in my case getting a tomcat server running with Geoserver on the same OracleXE instance. Alternatively it might be worth a try at installing the OracleXE .rpm on an AMI with a GIS stack already available. And, it will be useful to recompile ogr with oracle db support.

Of course the real mix and match challenge will be OracleXE on an EC2 (real soon now) Windows instance with Java, Tomcat, Geoserver serving a Google Map control coupled to Google Earth, OpenLayers, VirtualEarth. But really EC2 Windows will probably come preconfigured with the new MS SQL Server 2008 and all the promised geospatial goodies including Linq potential.

After just a short trip into the Ellison Empire, I must admit I still like the no frills PostgreSQL/PostGIS better.

TatukGIS – Generic ESRI with a Bit Extra

Fig1 basic TatukGIS Internet Server view element and legend/layer element

TatukGIS is a commercial product that is basically a generic brand for building GIS interfaces including web interfaces. It is developed in Gdynia Poland:

The core product is a Developer Kernel, DK, which provides basic building blocks for GIS applications in a variety of Microsoft flavors including:

  • DK-ActiveX – An ActiveX® (OCX) control supporting Visual Basic, VB.NET, C#, Visual C++
  • DK.NET – A manageable .NET WinForms component supporting C# and VB.NET
  • DK-CF – A manageable .NET Compact Framework 2.0/3.5 component – Pocket PC 2002 and 2003, Windows Mobile 5 and 6, Windows CE.NET 4.2, Windows CE 5 and 6
  • DK-VCL – A native Borland®/CodeGear® Delphi™/C++ Builder™

These core components have been leveraged for some additional products to make life a good deal easier for web and PDA developers. A TatukGIS Internet Server single server deployment license starts at $590 for the Lite Edition or $2000 per deployment server for the full edition in a web environment. I guess this is a good deal compared to ESRI/Oracle licenses, but not especially appealing to the open source integrators among us. There is support for the whole gamut of CAD, GIS, and raster formats as well as project file support for ESRI and MapInfo. This is a very complete toolkit.

The TatukGIS Internet Server license supports database access to all the usual DBs: "MSSQL Server, MySQL, Interbase, DB2, Oracle, Firebird, Advantage, PostgreSQL… " However, support for spatial formats are currently only available for Oracle Spatial/Locator and ArcSDE. Support for PostGIS and MS SQL Server spatial extensions are slated for release with TatukGIS IS 9.0.

I wanted to experiment a bit with the Internet Server, so I downloaded a trial version(free)..

Documentation was somewhat sparse, but this was a trial download. I found the most help looking in the sample subdirectories. Unfortunately these were all VB and it took a bit of experimental playing to translate into C#. The DK trial download did include a pdf document that was also somewhat helpful. Perhaps a real development license and/or server deployment license would provide better C# .NET documentation. I gather the historical precedence of VB is still evident in the current doc files.

The ESRI influence is obvious. From layer control to project serialization, it seems to follow the ESRI look and feel. This can be a plus or a minus. Although very familiar to a large audience of users, I am afraid the ESRI influence is not aesthetically pleasing or very smooth. I was able to improve over the typically clunky ArcIMS type zoom and wait interface by switching to the included Flash wrapper (simply a matter of setting Flash="true").

The ubiquitous flash plugin lets the user experience a somewhat slippy map interface familiar to users of Virtual Earth and Google Maps. We are still not talking a DeepZoom or Google Earth type interface, but a very functional viewer for a private data source. I was very pleased to find how easy it was to build the required functionality including vector and .sid overlays with layer/legend manipulation.

This is a very simple to use toolkit. If you have had any experience with Google Map API or Virtual Earth it is quite similar. Once a view element is added to your aspx the basic map interface is added server side:

<ttkGIS:XGIS_ViewerIS id="GIS" onclick=”GIS_Click" runat="server" OnPaint="GIS_Paint" Width="800px" Height="600px" OnLoad="GIS_Load" BorderColor="Black" BorderWidth="1px" ImageType="PNG24" Flash="True"></ttkGIS:XGIS_ViewerIS>

The balance of the functionality is a matter of adding event code to the XGIS_ViewerIS element. For example :

    protected void GIS_Load(object sender, EventArgs e)
       GIS.Open( Page.MapPath( "data/lasanimas1.ttkgp" ) );
       GIS.SetParameters("btnFullExtent.Pos", "(10,10)");
       GIS.SetParameters("btnZoom.Pos", "(40,10)");
       GIS.SetParameters("btnZoomEx.Pos", "(70,10)");
       GIS.SetParameters("btnDrag.Pos", "(100,10)");
       GIS.SetParameters("btnSelect.Pos", "(130,10)");

       addresslayer = (XGIS_LayerVector)GIS.API.Get("addpoints19");

The ttkgp project support allows addition of a full legend/layer menu with a single element, an amazing time saver:

<ttkGIS:XGIS_LegendIS id="Legend" runat="server" Width="150px" Height="600px" ImageType="PNG24" BackColor="LightYellow" OnLoad="Legend_Load" AllowMove="True" BorderWidth="1px"></ttkGIS:XGIS_LegendIS>

The result is a simple functional project viewer available over the internet, complete with zoom, pan, and layer manipulation. The real power of the TatukGIS is in the multitude of functions that can be used to extend these basics. I added a simple address finder and PDF print function, but there are numerous functions for routing, buffering, geocoding, projection, geometry relations etc. I was barely able to scratch the surface with my experiments.

Fig2 – TatukGIS Internet Server browser view with .sid imagery and vector overlays

The Bit Extra:
As a bit of a plus the resulting aspx is quite responsive. Because the library is not built with the MS MFC it has a performance advantage over the ESRI products it replaces. The TatukGIS website claims include the following:

"DK runs some operations run even 5 – 50 times faster than the leading GIS development products"

I wasn’t able to verify this, but I was pleased with the responsiveness of the interface, especially in light of the ease of development. I believe clients with proprietary data layers who need a quick website would be very willing to license the TatukGIS Internet Server. Even though an open source stack such as PostGIS, Geoserver, OpenLayers could do many of the same things, the additional cost of development would pretty much offset the TatukGIS license cost.

The one very noticeable restriction is that development is a Windows only affair. You will need an ASP IIS server to make use of the TatukGIS for development and deployment. Of course clients can use any of the popular browsers from any of the common OS platforms. Cloud clusters in Amazon’s AWS will not support TatukGIS IS very easily, but now that GoGrid offers Virtual Windows servers there are options.

Fig3 – TatukGIS Internet Server browser view with DRG imagery and vector overlays

Fig4 – TatukGIS Internet Server browser result from a find address function

Summary: TatukGIS Internet Server is a good toolkit for custom development, especially for clients with ESRI resources. The license is quite reasonable.