Mirror Land and the Last Foot


Fig 1 – Bing Maps Streetside

I know 2010 started yesterday but I slept in. I’m just a day late.

Even a day late perhaps it’s profitable to step back and muse over larger technology trends. I’ve worked through several technology tides in the past 35 years. I regretfully admit that I never successfully absorbed the “Gang of Four” Design Patterns. My penchant for the abstract is relatively low. I learn by doing concrete projects, and probably fall into the amateur programming category often dismissed by the “professional” programming cognoscenti. However, having lived through a bit of history already, I believe I can recognize an occasional technology trend without benefit of a Harvard degree or even a “Professional GIS certificate.”

What has been striking me of late is the growth of mirror realities. I’m not talking about bizarre multiverse theories popular in modern metaphysical cosmology, nor parallel universes of the many worlds quantum mechanics interpretation, or even virtual world phenoms such as Second Life or The Sims. I’m just looking at the mundane evolution of internet mapping.


Fig 2 – Google Maps Street View

One of my first mapping projects, back in the late 80′s, was converting the very sparse CIA world boundary file, WDBI, into an AutoCAD 3D Globe (WDBI came on a data tape reel). At the time it was novel enough, especially in the CAD world, to warrant a full color front cover of Cadence Magazine. I had lots of fun creating some simple AutoLisp scripts to spin the world view and add vector point and line features. I bring it up because at that point in history, prior to the big internet boom, mapping was a coarse affair at global scales. This was only a primitive wire frame, ethereal and transparent, yet even then quite beautiful, at least to map nerds.


Fig 3 – Antique AutoCAD Globe WDBI

Of course, at that time Scientists and GIS people were already playing with multi million dollar image aquisitions, but generally in fairly small areas. Landsat had been launched more than a decade earlier, but few people had the computing resources to play in that arena. Then too, US military was the main driving force with DARPA technology undreamed by the rest of us. A very large gap existed between Global and Local scales, at least for consumer masses. This access gap continued really until Keyhole’s aquisition by Google. There were regional initiatives like USGS DLG/DEM, Ordnance Survey, and Census TIGER. However, computer earth models were fragmented affairs, evolving relatively slowly down from satellite and up from aerial, until suddenly the entire gap was filled by Google and the repercussions are still very much evident.

Internet Map coverage is now both global and local, and everything in between, a mirror land. The full spectrum of coverage is complete. Or is it? A friend remarked recently that they feel like recent talk in mobile LiDAR echos earlier discussions of “Last Mile” when the Baby Bells and Cable Comms were competing for market share of internet connectivity. You can glimpse the same echo as Microsoft and Google jocky for market share of local street resolution, StreetView vs Streetside. The trend is from a global coarse model to a full scale local model, a trend now pushing out into the “Last Foot.” Alternate map models of the real world are diving into human dimension, feet and inches not miles, the detail of the street, my local personal world.

LiDAR contributes to this mirror land by adding a partial 3rd dimension to the flat photo world of street side capture. LiDAR backing can provide the swivel effects and the icon switching surface intelligence found in StreetView and Streetside. LiDAR capture is capable of much more, but internet UIs are still playing catchup in the 3rd dimension.

The question arises whether GIS or AEC will be the driver in this new human dimension “mirror land.” Traditionally AEC held the cards at feet and inches while GIS aerial platforms held sway in miles. MAC, Mobile Asset Collection, adds a middle way with inch level resolution capability available for miles.


Fig 4 – Video Synched to Map Route

Whoever, gets the dollars for capture of the last foot, in the end it all winds up inside an internet mirror land.

We are glimpsing a view of an alternate mirror reality that is not a Matrix sci-fi fantasy, but an ordinary part of internet connected life. Streetside and Street View push this mirror land down to the sidewalk.

On another vector, cell phone locations are adding the first primitive time dimension with life tracks now possible for millions. Realtime point location is a first step, but life track video stitched on the fly into photosynth streams lends credence to street side contingency.

The Location hype is really about linking those massive market demographic archives to a virtual world and then back connecting this information to a local personal world. As Sean Gillies in “Utopia or Dystopia” pointed out recently there are pros and cons. But, when have a few “cons” with axes ever really made a difference to the utopian future of technology?

With that thought in mind why not push a little on the future and look where the “Last Millimeter” takes us?
    BCI Brain Computer Interface
    Neuronal Prosthetics


Fig 5 – Brain Computer Interface

Eye tracking HUD (not housing and urban development exactly)


Fig 6- HUD phone?

I’m afraid the “Last Millimeter” is not a pretty thought, but at least an interesting one.

Summary

Just a few technology trends to keep an eye on. When they get out the drill for that last millimeter perhaps it’s time to pick up an ax or two.

WMS 3D with Silverlight and WPF


Silverlight view of DRCOG WMS
Fig 1 – Silverlight view of DRCOG WMS Layers

WPF 3D view of DRCOG WMS Layers
Fig 2 – WPF 3D view of DRCOG WMS Layers

Silverlight 2.0 MapControl CTP makes it easy to create a WMS viewer. However, Silverlight exposes only a 2D subset of the full WPF xaml specification. In my experiments so far with LiDAR I can see the various TINS, contours, and point clouds as a 2D view, but what I would really like to do is view data in 3D. Recalling some work I did a few years ago using WPF GeometryModel3D elements I decided to look at a WPF loose xaml approach. This is not restricted to use with LiDAR WMS services. Here in Colorado relief is major part of life and it would be helpful to visualize any number of WMS services using a 3D terrain model. Google and VE(Bing) already do this for us using their proprietary DEM, but eventually I would like to use different DEM resolutions, anticipating Open Terrain sources or in house LiDAR. DRCOG’s WMS service is conveniently available and lies right on the edge of the Rocky Mountain uplift so it is a nice WMS service to highlight relief visualization.

First navigating to the WPF service using Silverlight is a bit different from html. The familiar anchor element is gone. One Silverlight approach is to add a HyperlinkButton:<HyperlinkButton Content=”3D” TargetName=”_blank” NavigateUri=”"/> and change the NavigateUri as needed. Another approach is to use Html.Page.Window.Navigate() or Html.Page.Window.NavigateToBookmark() allowing a click handler to setup the navigate:

        private void ClickTest_Button(object sender, RoutedEventArgs e)
        {
            Uri testwpf = new Uri("http://www.cadmaps.com");
            HtmlPage.Window.Navigate(testwpf, "_blank");
        }

Using one of these Navigate methods the Uri can be pointed at the service used to create the 3D loose xaml.

The next item is getting 3D terrain to populate the xaml mesh. Eventually LiDAR Server will provide 3D surface data, but in the meantime JPL has a nice trick for their srtm and us_ned WMS service. One of the selectable styles is “feet_short_int”. This style will produce a grayscale image with elevation coded into the int16 pixels. Using these elevations I can then create the xaml mesh along with some primitive manipulation tools.

Ref: JPL GetCapabilities

    <Layer queryable="0">
      <Name>us_ned</Name>
      <Title>United States elevation, 30m</Title>
      <Abstract>
        Continental United States elevation, produced from the USGS National Elevation.
        The default style is scaled to 8 bit from the orginal floating point data.
      </Abstract>
      <LatLonBoundingBox minx="-125" miny="24" maxx="-66" maxy="50"/>
      <Style><Name>default</Name> <Title>
Default Elevation</Title> </Style>
      <Style><Name>short_int</Name> <Title>
short int signed elevation values when format is image/png or tiff
</Title> </Style>
      <Style><Name>feet_short_int</Name> <Title>
short int elevation values in feet when format is image/png or image/tiff
</Title> </Style>
      <Style><Name>real</Name> <Title>
DEM real numbers, in floating point format, meters, when used with image/tiff
</Title> </Style>
      <Style><Name>feet_real</Name> <Title>
DEM in real numbers, in floating point format, feet, when used with image/tiff
</Title> </Style>
      <ScaleHint min="20" max="10000" /> <MinScaleDenominator>24000</MinScaleDenominator>
    </Layer>


JPL us_ned 30m
Fig 3 – JPL us_ned 30m can be returned as feet_short_int

My first attempt used a .ashx service handler. I could then obtain the Bitmap like this:

     WebClient client = new WebClient();
     byte[] bytes = client.DownloadData(new Uri(terrainUrl));
     Bitmap bmp = (Bitmap)Bitmap.FromStream(new MemoryStream(bytes), true);

Unfortunately the returned pixel format of the Bitmap is incorrect. Instead of PixelFormat.Format16bppGrayScale the Bitmap returned PixelFormat.Format32bppArgb. This is not exactly helpful, quantizing the elevations. Instead of an elevation at each pixel GetPixel returns an RGB like 2A 2A 2A meaning e = p.B * 256 ^ 2 + p.G * 256 ^ 1 + p.R * 256 ^ 0; is incorrect. It appears that the returned value is quantized on the R value since to be gray each value of RGB must be equal. I checked the image returned by JPL using GdalInfo. This shows 2 band Uint16 values. Since Bitmap reads in a 4 byte value (2 Uint16s) it apparently defaults to Format32bppArgb and is then useless for actual elevation.

GDALINFO
Driver: PNG/Portable Network Graphics
Files: C:\Users\rkgeorge\Pictures\wms.png
Size is 500, 351
Coordinate System is `'
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  351.0)
Upper Right (  500.0,    0.0)
Lower Right (  500.0,  351.0)
Center      (  250.0,  175.5)
Band 1 Block=500x1 Type=UInt16, ColorInterp=Gray
Band 2 Block=500x1 Type=UInt16, ColorInterp=Alpha

color quantized example
Fig 4 – Example of a color quantized elevation from forced 32bppArgb

There is probably some type of pointer approach to pass through the raw bytes correctly. However, services are a nice way to decouple a view from the backend and it is just as easy to use a Java servlet which can take advantage of JAI, so in this case I switched to a Java servlet:

URL u = new URL(terrainUrl);
image = JAI.create(“url”, u);
SampleModel sm = image.getSampleModel();
int nbands = sm.getNumBands();
Raster inputRaster = image.getData();
int[] pixels = new int[nbands*width*height];
inputRaster.getPixels(0,0,width,height,pixels);

The resulting pixels array is int16[] so the correct 30m us_ned elevations are available from the JPL WMS service. Using the resulting elevations I can fill in a Positions attribute of my WPF MeshGeometry3D element. Since this is an array it is also easy to add a TriangleIndices attribute and a TextureCoordinates attribute for the draped image. The final loose xaml will eventually show up in the browser like this:


WPF view of DRCOG layers draped on JPL terrain
Fig 5 - WPF view of DRCOG layers draped on JPL terrain

Because I am simplifying life by using loose xaml there is no code behind for control of the views. However, we can use binding to attach various control values to camera positions and geometry transforms that allow a user to move around the terrain even without a full blown code behind xbap. For example:

<Viewport3D.Camera>
  <PerspectiveCamera
    Position="0 0 " + dimw + ""
    LookDirection="0,0,-1"
    FieldOfView="{Binding ElementName=zoomlensslider, Path=Value}">
    <PerspectiveCamera.Transform>
      <Transform3DGroup>
        <TranslateTransform3D
          OffsetX="{Binding ElementName=hscroll, Path=Value}"
          OffsetY="{Binding ElementName=vscroll, Path=Value}"
        />
      </Transform3DGroup>
    </PerspectiveCamera.Transform>
  </PerspectiveCamera>
</Viewport3D.Camera>

In this case the camera FieldOfView is bound to a slider value and the camera position transform offsetx and offsety are bound to scrollbar values. Binding values to controls in the same xaml file is a powerful concept that provides dynamic capability without any code behind.

Code behind with an xbap would actually make more useful navigation tools possible, but this is a good start. The controls allow the user to zoom the camera lense, change camera x,y position, tilt and rotate the viewport as well as exaggerate the relief scale. Changing to an xbap that handles the terrain image translation with code behind would allow the use of mouse wheel and click hits for pulling up item data fields similar to the Silverlight WMS viewer.

As great as WPF is, it would still be a difficult problem to turn it into a tiled interface like the SIlverlight MapControl. I imagine the good folks at Microsoft are working on it. Basically the mesh elements would be tiled into a pyramid just like imagery tiles. Then these tiles would be loaded as needed. It is easy to imagine an algorithm that would work with the backing pyramid as a 3D view curve. In other words distance from the camera would translate to tiles at higher pyramid levels of lower resolution. A view field would pull tiles from the mesh pyramid using a distance curve so that lower resolution would fill in further ranges and higher resolution at closer ranges. Some kind of 3D will eventually be available but is beyond the scope of my experiment.

Some Problems

1) JPL WMS has some difficulty keeping up with GetMap requests as noted on their website, which means GetMap requests are unreliable. Sometimes there is no issue and other days the load is heavier and there are few times when a GetMap request is possible. Here is their explanation:
"Due to server overloading, client applications are strongly advised to use the existing tile datasets wherever possible, as described in the Tiled WMS or Google Earth KML support

Frequent and repetitive requests for non-cached, small WMS tiles require an excessive amount of server resources and will be blocked in order to preserve server functionality. The OnEarth server is an experimental technology demonstrator and does not have enough resources to support these requests.
An alternative solution already exists in the form of tiled WMS
While sets of tiles with different sizes and alignment can be added when needed, for large datasets the duplication of storage, processing and data management resources is prohibitive."

2) Silverlight works pretty well cross browser at least on my Windows development systems. My tests with Mozilla Firefox managed Silverlight and WPF loose xaml just fine. However, Google Chrome has problems with WPF loose xaml reporting this error:
"This application has failed to start because xpcom.dll was not found"

A quick Google search came up with this:
Chromium does not support XPCOM. It is a Mozilla specific API.
https://developer.mozilla.org/en/XPCOM

More details here:
http://code.google.com/p/chromium/issues/detail?id=4051

3) Another baffling issue arose with the WPF loose xaml. It turns out that each click of the LinkButton or access of the url for the WPF service caused the service to be accessed three times in succession. This was true of both the .ashx and the servlet versions. The User Agent was not distinguishable so I wasn't able to short circuit the first two GET accesses. I only found this true for context.Response.ContentType = "application/xaml+xml"; not text/html etc. So far I haven't seen anything that would solve my problem. It is too bad since the computation for 15Mb WPF files is significant and three times for each result is a bit unnecessary. It may be something quite simple in the response header. I will keep an eye out. There was a similar problem with IE6 years ago, but in that case the service could watch for a Request.UserAgent notation and skip the unecessary duplicate calls.

4) WPF is verbose and 15Mb terrain views take a bit of patience especially considering the above problem. I need to add an auto xaml gzip to my Apache Tomcat. I'm sure there is something similar in IIS if I get around to a pointer solution to the grayscale.

Summary

Silverlight is a natural for interface development and integrates well with WPF ClickOnce for the 3D views that are still not available in Silverlight. My hope is that someday there will be a SIlverlight version capable of 3D scene graph development.

The LiDAR WMS Server roadmap also includes a style for grayscale elevations at some point. That would enable viewing much more detailed high resolution DEM directly from LAS files. I believe this will be a useful advance for use of online LiDAR viewing.

Also a thanks to DRCOG. I'm always glad to see agencies willing to support the wider GIS community by publishing data as OGC services and then further supporting it with adequate server performance. I just wish JPL had a bit more budget to do the same.

Silverlight view of LiDAR Server TIN
Fig 6 - Silverlight view of LiDAR Server TIN

WPF 3D view of LiDAR Server TIN
Fig 7 - WPF 3D view of LiDAR Server TIN