Demographic Landscapes – X3D/Census

Fig 1 – X3DOM view of tabblock census polygons demographics P0040003 density

PostGIS 2.2

PostGIS 2.2 is due for release sometime in August of 2015. Among other things, PostGIS 2.2 adds some interesting 3D functions via SFCGAL. ST_Extrude in tandem with ST_AsX3D offers a simple way to view a census polygon map in 3D. With these functions built into PostGIS, queries returning x3d text are possible.

Sample PostGIS 2.2 SQL Query:

SELECT ST_AsX3D(ST_Extrude(ST_SimplifyPreserveTopology(poly.GEOM, 0.00001),0,0, float4(sf1.P0040003)/19352),10) as geom, ST_Area(geography(poly.geom)) * 0.0000003861 as area, sf1.P0040003 as demographic
FROM Tract poly
JOIN sf1geo geo ON geo.geoid = poly.geoid
JOIN sf1_00003 sf1 ON geo.stlogrecno = sf1.stlogrecno
WHERE geo.geocomp='00' AND geo.sumlev = '140'
AND ST_Intersects(poly.GEOM, ST_GeometryFromText('POLYGON((-105.188236232418 39.6533019504969,-105.051581442071 39.6533019504969,-105.051581442071 39.7349599496294,-105.188236232418 39.7349599496294,-105.188236232418 39.6533019504969))', 4269))

Sample x3d result:

<IndexedFaceSet  coordIndex='0 1 2 3 -1 4 5 6 7 -1 8 9 10 11 -1 12 13 14 15 -1 16 17 18 19 -1 20 21 22 23'>
<Coordinate point='-105.05323 39.735185 0 -105.053212 39.74398 0 -105.039308 39.743953 0 -105.0393 39.734344 0 -105.05323 39.735185 0.1139417115 -105.0393 39.734344 0.1139417115 -105.039308 39.743953 0.1139417115 -105.053212 39.74398 0.1139417115 -105.05323 39.735185 0 -105.05323 39.735185 0.1139417115 -105.053212 39.74398 0.1139417115 -105.053212 39.74398 0 -105.053212 39.74398 0 -105.053212 39.74398 0.1139417115 -105.039308 39.743953 0.1139417115 -105.039308 39.743953 0 -105.039308 39.743953 0 -105.039308 39.743953 0.1139417115 -105.0393 39.734344 0.1139417115 -105.0393 39.734344 0 -105.0393 39.734344 0 -105.0393 39.734344 0.1139417115 -105.05323 39.735185 0.1139417115 -105.05323 39.735185 0'/;>
</IndexedFaceSet>
				.
				.

x3d format is not directly visible in a browser, but it can be packaged into x3dom for use in any WebGL enabled browser. Packaging x3d into an x3dom container allows return of an .x3d mime type model/x3d+xml, for an x3dom inline content html.

x3dom: http://www.x3dom.org/

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE X3D PUBLIC "ISO//Web3D//DTD X3D 3.1//EN" "http://www.web3d.org/specifications/x3d-3.1.dtd">
<X3D  profile="Immersive" version="3.1" xsd:noNamespaceSchemaLocation="http://www.web3d.org/specifications/x3d-3.1.xsd" xmlns:xsd="http://www.w3.org/2001/XMLSchema-instance">
    <scene>
				<viewpoint def="cam"
				centerofrotation="-105.0314177	39.73108357 0"
				orientation="-105.0314177,39.73108357,0.025, -0.25"
				position="-105.0314177	39.73108357 0.025"
				zfar="-1" znear="-1"
				fieldOfView="0.785398">
			</viewpoint>
        <group>
<shape>
<appearance>
<material DEF='color' diffuseColor='0.6 0 0' specularColor='1 1 1'></material>
</appearance>
<IndexedFaceSet  coordIndex='0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 -1 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 -1 38 39 40 41 -1 42 43 44 45 -1 46 47 48 49 -1 50 51 52 53 -1 54 55 56 57 -1 58 59 60 61 -1 62 63 64 65 -1 66 67 68 69 -1 70 71 72 73 -1 74 75 76 77 -1 78 79 80 81 -1 82 83 84 85 -1 86 87 88 89 -1 90 91 92 93 -1 94 95 96 97 -1 98 99 100 101 -1 102 103 104 105 -1 106 107 108 109 -1 110 111 112 113'>
<Coordinate point='-105.036957 39.733962 0 -105.036965 39.733879 0 -105.036797 39.734039 0 -105.036545 39.734181 0 -105.036326 39.734265 0 -105.036075 39.734319 0 -105.035671 39.734368 0 -105.035528 39.734449 0 -105.035493 39.73447 0 -105.035538 39.734786 0 -105.035679 39.73499 0 -105.035703 39.734927 0 -105.035731 39.734902 0 -105.035956 39.734777 0 -105.03643 39.734623 0/>
</IndexedFaceSet>

</shape>
        </group>
    </scene>
</X3D>

Inline x3dom packaged into html:

<html>
<head>
    <meta http-equiv="X-UA-Compatible" content="IE=edge"/>
    <title>TabBlock Pop X3D</title>
	<script src="http://ajax.aspnetcdn.com/ajax/jQuery/jquery-2.1.4.min.js"></script>
    <script type='text/javascript' src='http://www.x3dom.org/download/x3dom.js'> </script>
    <link rel='stylesheet' type='text/css' href='http://www.x3dom.org/download/x3dom.css'/>
</head>
<body>
<h2 id='testTxt'>TabBlock Population</h2>

<div id="content">
<x3d width='1000px' height='600px' showStat="true" showlog="false">
    <scene>
        <navigationInfo id="head" headlight='true' type='"EXAMINE"'></navigationInfo>
        <background  skyColor="0 0 0"></background>
		<directionallight id="directional"
				intensity="0.5" on="TRUE" direction="0 -1 0" color="1,1,1" zNear="-1" zFar="-1" >
        </directionallight>

		<Inline id="x3dContent" nameSpaceName="blockpop" mapDEFToID="true"
url="http://{server}/WMSService.svc/{token}/NAD83/USA/SF1QP/WMSLatLon?REQUEST=GetX3D&LAYERS=Tract&STYLES=P0040003&CRS=EPSG:4326&BBOX=39.6533019504969,-105.188236232418,39.7349599496294,-105.051581442071"></Inline>

    </scene>
</x3d>

</div>
</body>
</html>

In this case I added a non-standard WMS service adapted to add a new kind of request, GetX3D.

x3d is an open xml standards for leveraging immediate mode WebGL graphics in the browser. x3dom is an open source javascript library for translating x3d xml into WebGL in the browser.

“X3DOM (pronounced X-Freedom) is an open-source framework and runtime for 3D graphics on the Web. It can be freely used for non-commercial and commercial purposes, and is dual-licensed under MIT and GPL license.”



Why X3D?

I’ll admit it’s fun, but novelty may not always be helpful. Adding elevation does show demographic values in finer detail than the coarse classification used by a thematic color range. This experiment did not delve into the bivariate world, but multiple value modelling is possible using color and elevation with perhaps less interpretive misgivings than a bivariate thematic color scheme.

However, pondering the visionary, why should analysts be left out of the upcoming immersive world? If Occulus Rift, HoloLens, or Google Cardboard are part of our future, analysts will want to wander through population landscapes exploring avenues of millennials and valleys of the aged. My primitive experiments show only a bit of demographic landscape but eventually demographic terrain streams will be layer choices available to web users for exploration.

Demographic landscapes like the census are still grounded, tethered to real objects. The towering polygon on the left recapitulates the geophysical apartment highrise, a looming block of 18-22 year olds reflect a military base. But models potentially float away from geophysical grounding. Facebook networks are less about physical location than network relation. Abstracted models of relationship are also subject to helpful visualization in 3D. Unfortunately we have only a paltry few dimensions to work with, ruling out value landscapes of higher dimensions.

Fig 3 – P0010001 Jenks Population

Some problems

For some reason IE11 always reverts to software rendering instead of using the system’s GPU. Chrome provides hardware rendering with consequent smoother user experience. Obviously the level of GPU support available on the client directly correlates to maximum x3dom complexity and user experience.

IE11 x3dom logs show this error:

ERROR: WebGL version WebGL 0.94 lacks important WebGL/GLSL features needed for shadows, special vertex attribute types, etc.!
INFO: experimental-webgl context found
Vendor: Microsoft Microsoft, Renderer: Internet Explorer Intel(R) HD Graphics 3000, Version: WebGL 0.94, ShadingLangV.:
WebGL GLSL ES 0.94, MSAA samples: 4
Extensions: WEBGL_compressed_texture_s3tc, OES_texture_float, OES_texture_float_linear, EXT_texture_filter_anisotropic,
      OES_standard_derivatives, ANGLE_instanced_arrays, OES_element_index_uint, WEBGL_debug_renderer_info
INFO: Initializing X3DCanvas for [x3dom-1434130193467-canvas]
INFO: Creating canvas for (X)3D element...
INFO: Found 1 X3D and 0 (experimental) WebSG nodes...
INFO: X3DOM version 1.6.2, Revison 8f5655cec1951042e852ee9def292c9e0194186b, Date Sat Dec 20 00:03:52 2014 +0100

In some cases the ST_Extrude result is rendered to odd surfaces with multiple artifacts. Here is an example with low population in eastern Colorado. Perhaps the extrusion surface breaks down due to tessellation issues on complex polygons with zero or near zero height. This warrants further experimentation.

Fig 2 – rendering artifacts at near zero elevations

The performance complexity curve on a typical client is fairly low. It’s tricky to keep the model sizes small enough for acceptable performance. IE11 is especially vulnerable to collapse due to software rendering. In this experiment the x3d view is limited to the intersections with extents of the selected territory using turf.js.

var extent = turf.extent(app.territory);

In addition making use of PostGIS ST_SimplifyPreserveTopology helps reduce polygon complexity.

Xml formats like x3d tend to be verbose and newer lightweight frameworks prefer a json interchange. Json for WebGL is not officially standardized but there are a few resources available.

Lighthouse – http://www.lighthouse3d.com/2013/07/webgl-importing-a-json-formatted-3d-model/
Three.js – http://threejs.org/ JSONLoader BabylonLoader
Babylon.js – http://www.babylonjs.com/ SceneLoader BABYLON.SceneLoader.ImportMesh

Summary

PostGIS 2.2 SFCGAL functions offer a new window into data landscapes. X3d/x3dom is an easy way to leverage WebGL in the browser.

A future project may look at converting the x3d output of PostGIS into a json mesh. This would enable use of other client libraries like Babylon.js

Fig 4 – P0040003 Quantile Density

Territorial Turf

Fig 1 – Territories from Census Zip Code Tabulation Areas (ZCTAs) on Bing Maps

An interesting GIS development over the years has been the evolution from monolithic applications to multiple open source plug and play tools. Considering GIS as a three tier system with back end storage, a controller in the middle, and a UI display out front, more and more of the middle tier is migrating to either end.

SQL DBs, such as SQL Server, Oracle Spatial, and especially PostGIS, now implement a multitude of GIS functions originally considered middle tier domain. On the other end, the good folks at turf.js and proj4js continue to push atomic GIS functions out to javascript, where they can fit into the client side UI. The middle tier is getting thinner and thinner as time goes on. Generally the middle is what costs money, so rolling your own middle with less work is a good thing. As a desirable side effect, instead of kitchen sink GIS, very specific user tools can be cobbled together as needed.

Looking for an excuse to experiment with turf.js, I decided to create a Territory Builder utilizing turf.js on the client and some of the US Census Bureau Services on the backend.

US Census Bureau does expose some “useful” services at TigerWeb. I tend to agree with Brian Timoney that .gov should stick to generating data exposed in useful ways. Apps and presentation are fast evolving targets and historically .gov can’t really hope to keep up. Although you can use the census custom TigerWeb applications for some needs, there are many other occasions when you would like to build something less generic. For example a Territory builder over Bing Maps.

Here is the simplified POC work flow:

1. Use Census WMS GetMap to show polygons on a canvas over Bing Maps.
2. Use Census WMS GetFeatureInfo to return a GeoID of selected polygon(s).
3. Use Census REST service and GeoID to return selected polygon vertices.
4. Use Turf.js merge function to merge selected polygons into a single Territory polygon
5. Display territory polygon using Bing Maps Ajax v7 Microsoft.Maps.AdvancedShapes polygon

Fig 2 – Territory polygon with a demographic overlay on top of Bing Maps

Note: Because there doesn’t appear to be an efficient way to join demographic data to geography with current TigerWeb service APIs, the demographic tab of this app uses a custom WMS PostGIS backend, hooking SF1 to TIGER polygons.

1. Census WMS GetMap over Bing Maps

WMS GetMap requests simply return an image. In order to overlay the image on a Bing Map this Territory app uses an html5 canvas and context app.context.drawImage(imageObj, 0, 0); The trick is to stack the canvas in between the Bing Map and the Bing Map navigation, scale, and selector tools. TigerWeb WMS conveniently exposes epsg:3857 which correctly aligns with Bing Map tiles.

    function showCensus() {
        if (app.canvas == null) {
            app.canvas = document.createElement('canvas');
            app.canvas.id = 'censuscanvas';
            app.canvas.style.position = 'relative';
            var mapDiv = app.map.getRootElement();
            //position this canvas under Bing navigation and over map tiles
            mapDiv.insertBefore(app.canvas, mapDiv.childNodes[3]);
            app.context = app.canvas.getContext("2d");
        }
        //match canvas size to map size
        app.canvas.height = app.map.getHeight();
        app.canvas.width = app.map.getWidth();

        var imageObj = new Image();
        //image onload function draws image on context
        imageObj.onload = function () {
            app.canvas.style.opacity = app.alpha / 100;
            app.context.drawImage(imageObj, 0, 0);
        };
        //prepare a TigerWeb WMS GetMap request
        //use proj4.js to transform ll to mercator bbox
        var b = app.map.getBounds();
        var epsg3857 = "+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext  +no_defs";
        var sw = proj4(epsg3857).forward([b.getWest(), b.getSouth()]);
        var ne = proj4(epsg3857).forward([b.getEast(), b.getNorth()]);
        bbox = sw[0] + "," + sw[1] + "," + ne[0] + "," + ne[1];

        var url = "";
        if (tab == "Territory") {
            app.alpha = 100;
            // GetMap request
            url = "http://tigerweb.geo.census.gov/arcgis/services/TIGERweb/tigerWMS_Current/MapServer/WmsServer?REQUEST=GetMap&SERVICE=WMS&VERSION=1.3.0&LAYERS=" + app.censusPolygonType + "&STYLES=&FORMAT=image/png&BGCOLOR=0xFFFFFF&TRANSPARENT=TRUE&CRS=EPSG:3857&BBOX=" + bbox + "&WIDTH=" + app.canvas.width + "&HEIGHT=" + app.canvas.height;
        }
        imageObj.src = url;
    }

2. Census WMS GetFeatureInfo for obtaining GeoID

Unfortunately the WMS GetMap request has no IDs available for polygons, even if requested format is image/svg+xml. SVG is an xml format and could easily contain associated GeoID values for joining with other data resources, but this is contrary to the spirit of OGC WMS service specs. Naturally we must obey OGC Law, which is too bad. Adding a GeoID attribute would allow options such as choropleth fills directly on existing svg paths. For example adding id = “80132″ would allow fill colors by zipcode with a bit of javascript.

http://tigerweb.geo.census.gov/arcgis/services/TIGERweb/tigerWMS_Current/MapServer/WmsServer?REQUEST=GetMap&SERVICE=WMS&VERSION=1.3.0 &LAYERS=2010 Census ZIP Code Tabulation Areas&STYLES=&FORMAT=image/svg+xml&BGCOLOR=0xFFFFFF&TRANSPARENT=TRUE&CRS=EPSG:3857 &BBOX=-11679625.942909468,4709198.547476525,-11645573.246808422,4737900.651597611&WIDTH=891&HEIGHT=751

<g transform="matrix(1.3333333 0 0 -1.3333333 445.5 375.5)">
<path id="80132" d="M300.50809 -258.78592 L302.54407 -257.91829 L303.17977 -257.55608 L304.45554 -256.98889
                                .
				.
L280.30116 -268.55133 L280.7184 -268.34637 L298.33448 -259.95117 L300.50809 -258.78592 "
  stroke="#99454A" fill="none" stroke-opacity="1" stroke-width="1.5"
 stroke-linecap="round" stroke-linejoin="round" stroke-miterlimit="10" />

Instead TigerWeb WMS exposes GetFeatureInfo requests. Using a click event we calculate a lat,lon location and send a GetFeatureInfo request to find a GeoID for the enclosing polygon.

Request:
http://tigerweb.geo.census.gov/arcgis/services/TIGERweb/tigerWMS_Current/MapServer/WmsServer?REQUEST=GetFeatureInfo&SERVICE=WMS&VERSION=1.3.0&CRS=EPSG:4326 &BBOX=13.6972399985862,-128.010213138288,52.2800368298775,-53.444917258507&WIDTH=1061&HEIGHT=549 &INFO_FORMAT=text/xml&QUERY_LAYERS=2010 Census ZIP Code Tabulation Areas&X=388&Y=190

Response:
<?xml version=”1.0″ encoding=”UTF-8″?>
<FeatureInfoResponse xmlns=”http://www.esri.com/wms” xmlns:esri_wms=”http://www.esri.com/wms”>
<FIELDS INTPTLON=”-101.2099848″ INTPTLAT=”+38.9263824″ CENTLON=”-101.2064010″ CENTLAT=”+38.9226272″ STGEOMETRY=”Polygon” OBJECTID=”16553″ AREAWATER=”163771″ AREALAND=”1327709219″ FUNCSTAT=”S” ZCTA5CC=”B5″ MTFCC=”G6350″ NAME=”ZCTA5 67764″ LSADC=”Z5″ BASENAME=”67764″ GEOID=”67764″ ZCTA5=”67764″ OID=”221404258331221″/>
</FeatureInfoResponse>

Since cross browser restrictions come into play it’s necessary to add a bit of server side proxy code to actually return the xml.

Some MVC controller magic:

 function GetCensusFeature(e) {
    var x = e.pageX - $("#map").offset().left;
    var y = e.pageY - $("#map").offset().top;
    var FeatureUrl = {
        url: "http://tigerweb.geo.census.gov/arcgis/services/TIGERweb/tigerWMS_Current/MapServer/WmsServer?REQUEST=GetFeatureInfo&SERVICE=WMS&VERSION=1.3.0&LAYERS=" + app.censusPolygonType + "&STYLES=&FORMAT=image/png&BGCOLOR=0xFFFFFF&TRANSPARENT=TRUE&CRS=EPSG:3857&BBOX=" + bbox + "&WIDTH=" + app.canvas.width + "&HEIGHT=" + app.canvas.height + "&INFO_FORMAT=text/xml&QUERY_LAYERS=" + app.censusPolygonType + "&X=" + x + "&Y=" + y
    }
        app.postit('/api/Operation/GetFeatureInfo', {
            data: JSON.stringify(FeatureUrl),
            success: function (response) {
                if (response.length > 0) {
                    var parser = new DOMParser();
                    var xmlDoc = parser.parseFromString(response, "text/xml");
				          .
				          .

Eventally gets here to a C# Controller which can proxy the GetFeatureInfo request:

        /// <summary>
        /// GetFeatureInfo
        /// </summary>
        /// <param name="FeatureUrl"></param>
        /// <returns>xml doc</returns>
        [HttpPost]
        public async Task<IHttpActionResult> GetFeatureInfo(Models.FeatureUrl feature)
        {
            var wc = new HttpClient();
            var getFeatureInfoRequest = new Uri(feature.url);
            var response = await wc.GetAsync(getFeatureInfoRequest);
            response.EnsureSuccessStatusCode();
            var content = await response.Content.ReadAsStreamAsync();
            var result = "";
            using (var reader = new StreamReader(content))
            {
                while (!reader.EndOfStream)
                {
                    result += reader.ReadLine();
                }
            }
            return Ok(result);
        }

Finally, on success, javascript can parse the xml for GeoID and add a canvas context box with some text information:

     success: function (response) {
              if (response.length > 0) {
                    var parser = new DOMParser();
                    var xmlDoc = parser.parseFromString(response, "text/xml");
                    if (tab == "Territory" && xmlDoc.getElementsByTagName("FeatureInfoResponse")[0].hasChildNodes()) {
                        app.context.beginPath();
                        app.context.rect(x, y, 125, 25);
                        app.context.fillStyle = 'white';
                        app.context.fill();
                        app.context.lineWidth = 1;
                        app.context.strokeStyle = 'black';
                        app.context.stroke();
                        app.context.fillStyle = 'black';
                        app.context.font = '11pt Calibri';
                        var fields = xmlDoc.getElementsByTagName("FIELDS")[0];
                        app.context.fillText(fields.getAttribute('NAME'), x + 5, y + 15);
                        var geoid= fields.getAttribute('GEOID')
					            .
					            .

Fig 3 – building territories from Census Block Groups on Bing Maps

3. Census REST to obtain vertices

The GEOID retrieved from our proxied GetFeatureInfo request allows us to grab vertices with another TigerWeb service, Census REST.

This spec is a little more proprietary and requires some detective work to unravel.
FeatureUrl.url = “http://tigerweb.geo.census.gov/arcgis/rest/services/TIGERweb/PUMA_TAD_TAZ_UGA_ZCTA/MapServer/1/query?where=GEOID%3D” + geoid + “&geometryPrecision=6&outSR=4326&f=pjson”;

There are different endpoint urls for the various polygon types. In this case Zip Codes are found in PUMA_TAD_TAZ_UGA_ZCTA. We don’t need more than 1 meter resolution so precision 6 is good enough and we would like the results in epsg:4326 to avoid a proj4 transform on the client.

This follows the same process as previously sending the REST request through a proxy controller. You can see the geojson geometry result with this sample request:
http://tigerweb.geo.census.gov/arcgis/rest/services/TIGERweb/PUMA_TAD_TAZ_UGA_ZCTA/MapServer/1/query?where=GEOID%3D80004&geometryPrecision=6&outSR=4326&f=pjson

Census REST doesn’t appear to offer a simplify parameter so the coordinates returned are at the highest resolution. Highly detailed polygons can easily return several thousand vertices, which is a problem for performance, but the trade-off is eliminating hosting data ourselves.

4. turf.js merge function to Build Territory

Finally the interesting part, where we get to use turf.js to handle merging multiple polygons into a single territory polygon.

var FeatureUrl = {
url: "http://tigerweb.geo.census.gov/arcgis/rest/services/TIGERweb/PUMA_TAD_TAZ_UGA_ZCTA/MapServer/1/query?where=GEOID%3D" + geoid + "&geometryPrecision=6&outSR=4326&f=pjson";
}
app.postit('/api/Operation/GetFeatureGeo', {
    data: JSON.stringify(FeatureUrl),
    success: function (response) {
        var polygon = JSON.parse(response);
        if (polygon.features.length == 0 || polygon.features[0].geometry.rings.length == 0) {
            $('#loading').addClass('hide');
            alert("No Geo Features returned.")
            return;
        }
        if (app.territoryFeatures == null) {
            app.territoryFeatures = new Array();
            app.territory = turf.polygon(polygon.features[0].geometry.rings);
            app.territoryFeatures.push(app.territory);
        }
        else {
            var tpolygon = turf.polygon(polygon.features[0].geometry.rings);
            app.territoryFeatures.push(tpolygon);
            var fc = turf.featurecollection(app.territoryFeatures);
            try {
                app.territory = turf.merge(fc);
            }
            catch (err) {
                $('#loading').addClass('hide');
                alert("turf.js error: " + err.message);
                return;
            }
        }
        if (app.territory.geometry.coordinates.length > 0) {
            displayTerritory();
            $('#totalDivID').removeClass('hide');
        }

        $('#loading').addClass('hide');
    },
    error: function (response) {
        alert("TigerWeb error:" + response);
    }
});

5. Display Territory Bing Maps Ajax polygon

The final rendering just uses Bing Maps Ajax v7 Microsoft.Maps.AdvancedShapes to add the new territory to the map.

function displayTerritory() {
    app.territoryLayer.clear();
    var coordLen = app.territory.geometry.coordinates.length;
    if (app.territory.geometry.type == 'Polygon') {
        var rings = new Array();
        for (var i = 0; i < coordLen; i++) {
            var vertices = new Array();
            for (var j = 0; j < app.territory.geometry.coordinates[i].length; j++) {
                vertices.push(new Microsoft.Maps.Location(app.territory.geometry.coordinates[i][j][1], app.territory.geometry.coordinates[i][j][0]));
            }
            rings.push(vertices);
        }
        var polygon = new Microsoft.Maps.Polygon(rings, { fillColor: new Microsoft.Maps.Color(100, 100, 0, 100) });
        app.territoryLayer.push(polygon);
    }
    else if (app.territory.geometry.type == 'MultiPolygon') {
        var multi = new Array();
        for (var i = 0; i < coordLen; i++) {
            var ringslen = app.territory.geometry.coordinates[i].length;
            for (var j = 0; j < ringslen; j++) {
                var vertices = new Array();
                for (var k = 0; k < app.territory.geometry.coordinates[i][j].length; k++) {
                    vertices.push(new Microsoft.Maps.Location(app.territory.geometry.coordinates[i][j][k][1], app.territory.geometry.coordinates[i][j][k][0]));
                }
                multi.push(vertices)
            }
        }
        var polygon = new Microsoft.Maps.Polygon(multi, { fillColor: new Microsoft.Maps.Color(100, 100, 0, 100) });
        app.territoryLayer.push(polygon);
    }
    else {
        $('#loading').addClass('hide');
        alert("geometry type is " + app.territory.geometry.type + ". Territory requires a Polygon or MultiPolygon.")
        return;
    }
}

Fig 4 – Territory Total for P0040003 Latino Origin – Map: Quantile population classifier Census Block Groups on Bing Map

Summary

TigerWeb offers some useful data access. With TigerWeb WMS and REST api, developers can customize apps without hosting and maintaining a backend SQL store. However, there are some drawbacks.

Some potential improvements:
1. Adding an svg id=GeoID would really improve the usefulness of TigerWeb WMS image/svg+xml, possibly eliminating steps 2 and 3 of the workflow.

2. Technically it’s possible to use the TigerWeb REST api to query geojson by area, but practically speaking the results are too detailed for useful performance. A helpful option for TigerWeb REST would be a parameter to request simplified polygons and avoid lengthy vertice transfers.

turf.js is a great tool box, however, occasionally the merge function had trouble with complex polygons from TigerWeb.

Fig 6 - turf merge had some rare difficulties with complex polygons from TigerWeb

In the end, territories are useful for query aggregates of SF1 demographic data.

Fig 5 – Territory Total for P0120025 Male 85 Years and older – Map: Jenks population classifier Census Block Group on Bing Map

2020 The Last Census?

Fig 1 - SF1QP Quantile Population County P0010001 P1.TOTAL POPULATION Universe: Total population

Preparation for US 2020 Census is underway at this mid-decennial point and we’ll see activity ramping up over the next few years. Will 2020 be the last meaningful decennial demographic data dump? US Census has been a data resource since 1790. It took a couple centuries for Census data to migrate into the digital age, but by Census 2000, data started trickling into the internet community. At first this was simply a primitive ftp data dump, ftp2.census.gov, still very useful for developers, and finally after 2011 exposed as OGC WMS, TigerWeb UI, and ESRI REST.

However, static data in general, and decennial static data in particular, is fast becoming anachronistic in the modern era. Surely the NSA data tree looks something like phone number JOIN Facebook account JOIN Twitter account JOIN social security id JOIN bank records JOIN IRS records JOIN medical records JOIN DNA sequence….. Why should this data access be limited to a few black budget bureaus? Once the data tree is altered a bit to include mobile devices, static demographics are a thing of the past. Queries in 2030 may well ask “how many 34 year old male Hispanic heads of households with greater than 3 dependents with a genetic predisposition to diabetes are in downtown Denver Wed at 10:38AM, at 10:00PM?” For that matter let’s run the location animation at 10 minute intervals for Tuesday and then compare with Sat.

“Privacy? We don’t need no stinking privacy!”

I suppose Men and Black may find location aware DNA queries useful for weeding out hostile alien grays, but shouldn’t local cancer support groups also be able to ping potential members as they wander by Star Bucks? Why not allow soda vending machines to check for your diabetic potential and credit before offering appropriate selections? BTW How’s that veggie smoothie?

Back to Old School

Fig 2 - SF1QD Quantile Density Census Block Group P0050008 P5.HISPANIC OR LATINO ORIGIN BY RACE Universe: Total population

By late 2011 census OGC services began to appear along with some front end data web UIs, and ESRI REST interfaces. [The ESRI connection is a tightly coupled symbiotic relationship as the Census Bureau, like many government bureaucracies, relies on ESRI products for both publishing and consuming data. From the outside ESRI could pass as an agency of the federal government. For better or worse “Arc this and that” are deeply rooted in the .gov GIS community.]

For mapping purposes there are two pillars of Census data, spatial and demographic. The spatial data largely resides as TIGER data while the demographic data is scattered across a large range of products and data formats. In the basement, a primary demographic resource is the SF1, Summary File 1, population data.

“Summary File 1 (SF 1) contains the data compiled from the questions asked of all people and about every housing unit. Population items include sex, age, race, Hispanic or Latino origin, household relationship, household type, household size, family type, family size, and group quarters. Housing items include occupancy status, vacancy status, and tenure (whether a housing unit is owner-occupied or renter-occupied).”

The intersection of SF1 and TIGER is the base level concern of census demographic mapping. There are a variety of rendering options, but the venerable color themed choropleth map is still the most widely recognized. This consists of assigning a value class to a color range and rendering polygons with their associated value color. This then is the root visualization of Census demographics, TIGER polygons colored by SF1 population classification ranges.

Unfortunately, access to this basic visualization is not part of the 2010 TigerWeb UI.

There are likely a few reasons for this, even aside from the glacially slow adoption of technology at the Bureau of the Census. A couple of obvious reasons are the sheer size of this data resource and the range of the statistics gathered. A PostGIS database with 5 level primary spatial hierarchy, all 48 SF1 population value files, appropriate indices, plus a few helpful functions consumes a reasonable 302.445 GB of a generic Amazon EC2 SSD elastic block storage. But, contained in those 48 SF1 tables are 8912 demographic values which you are welcome to peruse here. A problem for any UI is how to make 8912 plus 5 spatial levels usable.

Fig 3 – 47 SF1 tables plus sf1geo geography join file

Filling a gap

Since the Census Bureau budget did not include public visualization of TIGER/Demographics what does it take to fill in the gap? Census 2010 contains a large number of geographic polygons. The core hierarchy for useful demographic visualization is state, county, tract, block group, and block.

Fig 4 – Census polygon hierarchy

Loading the data into PostGIS affords low cost access to data for SF1 Polygon value queries such as this:

-- block tabblock
SELECT poly.GEOM, geo.stusab, geo.sumlev, geo.geocomp, geo.state, geo.county, geo.tract, geo.blkgrp, geo.block, poly.geoid10, sf1.P0010001, geo.stlogrecno
FROM tabblock poly
JOIN sf1geo geo ON geo.geoid10 = poly.geoid10
JOIN sf1_00001 sf1 ON geo.stlogrecno = sf1.stlogrecno
WHERE geo.geocomp='00' AND geo.sumlev = '101' AND ST_Intersects(poly.GEOM, ST_GeometryFromText('POLYGON ((-104.878035974004 38.9515291859429,-104.721023973742 38.9515291859429,-104.721023973742 39.063158980149,-104.878035974004 39.063158980149,-104.878035974004 38.9515291859429))', 4269))
ORDER BY geo.state, geo.county, geo.tract, geo.blkgrp, geo.block

Returning 1571 polygons in 1466 ms. Not too bad, but surely there’s room for improvement. Where is Paul Ramsey when you need him?

Fig 5 - PostgreSQL PostGIS Explain Query

Really Old School – WMS

Some considerations on the data:

A. Queries become unwieldy for larger extents with large numbers of polygons

Polygon Counts
county 3,233
tract 74,133
blockgroup 220,740
tabblock 11,166,336

These polygon counts rule out visualizations of the entire USA, or even moderate regions, at tract+ levels of the hierarchy. Vector mapping is not optimal here.

B. The number of possible image tile pyramids for 8912 values over 5 polygon levels is 5 * 8192 = 44,560. This rules out tile pyramids of any substantial depth without some deep Google like pockets for storage. Tile pyramids are not optimal either.

C. Even though vector grid pyramids would help with these 44,560 demographic variations, they suffer from the same restrictions as A. above.

One possible compromise of performance/visualization is to use an old fashioned OGC WMS GetMap request scheme that treats polygon types as layer parameters and demographic types as style parameters. With appropriate use of WMS <MinScaleDenominator> <MaxScaleDenominator> the layers are only rendered at sufficient zoom to reasonably limit the number of polygons. Using this scheme puts rendering computation right next to the DB on the same EC2 instance, while network latency is reduced to simple jpeg/png image download. Scaling access to public consumption is still problematic, but for in-house it does work.

Fig 6 – Scale dependent layer rendering for SF1JP - SF1 Jenks P0010001 (not density)

Fig 7 - a few of 8912 demographic style choices

There are still issues with a scale rendering approach. Since population is not very homogenous over US coverage extent, scale dependent rendering asks to be variable as well. This is easily visible over population centers. Without some type of pre-calculated density grid, the query is already completed prior to knowledge of the ideal scale dependency. Consequently, static rendering scales have to be tuned to high population urban regions. Since “fly over” US is generally less interesting to analysts, we can likely live with this compromise.

Fig 8 - SF1QD SF1 Quantile Density Census Tract P0010001/geographic Area



Classification schemes

Dividing a value curve to display adequately over a viewport range can be accomplished in a few different ways: equal intervals, equal quantile, jenks natural break optimization, K-means clustering, or “other.” Leaning toward the simpler, I chose a default quantile (guarantees some color) with a ten class single hue progression which of course is not recommended by color brewer. However 10 seems an appropriate number for decennial data. I also included a jenks classifier option which is considered a better representation. The classifier is based only on visible polygons rather than the entire polygon population. This means comparisons region to region are deceptive, but after all this is visualization of statistics.

“There are three kinds of lies: lies, damned lies, and statistics.” Mark Twain

Fig 9 – SF1JP SF1 Jenks Census Tract P0010001 (not density)

In order to manage Census data on a personal budget these compromises are involved:

1. Only expose SF1 demographic data for 2010 i.e. 8912 population value types
2. Only provide primary level polygon hierarchy – state, county, tract, blockgroup, block
3. Code a custom OGC WMS service – rendering GetMap image on the server
4. Resolution scale rendering to limit polygon counts down the polygon hierarchy
5. Provide only quantile and Jenks classifier options
6. Classifier applied only to viewport polygon selection

This is a workable map service for a small number of users. Exposing as an OGC WMS service offers some advantages. First there are already a ton of WMS clients available to actually see the results. Second, the Query, geometry parsing, and image computation (including any required re-projection) are all server side on the same instance reducing network traffic. Unfortunately the downside is that the computation cost is significant and discouraging for a public facing service.

Scaling could be accomplished in a few ways:

1. Vertical scaling to a high memory EC2 R3 instance(s) and a memory tuned PostGIS
2. Horizontal auto scaling to multiple instances with a load balancer
3. Storage scaling with pre-populated S3 tile pyramids for upper extents

Because this data happens to be read only for ten years, scaling is not too hard, as long as there is a budget. It would also be interesting to try some reconfiguration of data into NoSQL type key/value documents with perhaps each polygon document containing the 8912 values embedded along with the geometry. This would cost a bit in storage size but could decrease query times. NoSQL also offers some advantages for horizontal scaling.

Summary

The Census Bureau and its census are obviously not going away. The census is a bureaucracy with a curious inertial life stretching back to the founding of our country (United States Constitution Article 1, section 2). Although static aggregate data is not going to disappear, dynamic real time data has already arrived on stage in numerous and sundry ways from big data portals like Google, to marketing juggernauts like Coca Cola and the Democratic Party, to even more sinister black budget control regimes like the NSA.

Census data won’t disappear. It will simply be superseded.

The real issue for 2020 and beyond is, how to actually intelligently use the data. Already data overwhelms analytic capabilities. By 2030, will emerging AI manage floods of real time data replacing human analysts? If Wall Street still exists, will HFT algos lock in dynamic data pipelines at unheard of scale with no human intervention? Even with the help of tools like R Project perhaps the human end of data analysis will pass into anachronism along with the decennial Census.

Fig 10 - SF1JP SF1 Jenks Census Blocks P0010001

Visualizing Large Data Sets with Bing Maps Web Apps

Fig 1 - MapD Twitter Map 80M tweets Oct 19 - Oct 30

Visualizing large data sets with maps is an ongoing concern these days. Just ask the NSA, or note this federal vehicle tracking initiative reported at the LA Times. Or, this SPD mesh network for tracking any MAC address wandering by.

“There was of course no way of knowing whether you were being watched at any given moment. How often, or on what system, the Thought Police plugged in on any individual wire was guesswork. It was even conceivable that they watched everybody all of the time. But at any rate they could plug in your wire whenever they wanted to.”

George Orwell, 1984

On a less intrusive note, large data visualization is also of interest to anyone dealing with BI or just fascinated with massive public data sets such as twitter universe. Web maps are the way to go for public distribution and all web apps face the same set of issues when dealing with large data sets.

1. Latency of data storage queries, typically SQL
2. Latency of services for mediating queries and data between the UI and storage.
3. Latency of the internet
4. Latency of client side rendering

All web map javascript APIs have these same issues whether it’s Google, MapQuest, Nokia Here, or Bing Maps. This is a Bing Maps centric perspective on large data mapping, because Bing Maps has been the focus of my experience for the last year or two.

Web Mapping Limitation

Bing Maps Ajax v7 is Microsoft’s javascript API for web mapping applications. It offers typical Point (Pushpin), Polyline, and Polygon vector rendering in the client over three tile base map styles, Road, Aerial, and AerialWithLabels. Additional overlay extensions are also available such as traffic. Like all the major web map apis, vector advantages include client side event functions at the shape object level.

Although this is a powerful mapping API rendering performance degrades with the number of vector entities in an overlay. Zoom and pan navigation performs smoothly on a typical client up to a couple of thousand points or a few hundred complex polylines and polygons. Beyond these limits other approaches are needed for visualizing geographic data sets. This client side limit is necessarily fuzzy as there is a wide variety of client hardware out there in user land, from older desktops and mobile phones to powerful gaming rigs.

Large data Visualization Approaches

1) Tile Pyramid – The Bing Maps Ajax v7 API offers a tileLayer resource that handles overlays of tile pyramids using a quadkey nomenclature. Data resources are precompiled into sets of small images called a tile pyramid which can then be used in the client map as a slippy tile overlay. This is the same slippy tile approach used for serving base Road, Aerial, and AerialWithLabels maps, similar to all web map brands.

Fig 2 - example of quadkey png image names for a tile pyramid

Pro:· Fast performance

  • Server side latency is eliminated by pre-processing tile pyramids
  • Internet streaming is reduced to a limited set of png or jpg tile images
  • Client side rendering is reduced to a small set of images in the overlay

Con: Static data – tile pyramids are pre-processed

  • data cannot be real time
  • Permutations limited – storage and time limitations apply to queries that have large numbers of permutations
  • Storage capacity – tile pyramids require large storage resources when provided for worldwide extents and full 20 zoom level depth

2) Dynamic tiles – this is a variation of the tile pyramid that creates tiles on demand at the service layer. A common approach is to provide dynamic tile creation with SQL or file based caching. Once a tile has been requested it is then available for subsequent queries directly as an image. This allows lower levels of the tile pyramid to be populated only on demand reducing the amount of storage required.

Pro:

  • Can handle larger number of query permutations
  • Server side latency is reduced by caching tile pyramid images (only the first request requires generating the image)
  • Internet streaming is reduced to a limited set of png tile images
  • Client side rendering is reduced to a small set of images in the overlay

Con:

  • Static data – dynamic data must still be refreshed in the cache
  • Tile creation performance is limited by server capability and can be a problem with public facing high usage websites.

3) Hybrid - This approach splits the zoom level depth into at least two sections. The lowest levels with the largest extent contain the majority of a data set’s features and is provided as a static tile pyramid. The higher zoom levels comprising smaller extents with fewer points can utilize the data as vectors. A variation of the hybrid approach is a middle level populated by a dynamic tile service.

Fig 3 – Hybrid architecture

Pro:

  • Fast performance – although not as fast as a pure static tile pyramid it offers good performance through the entire zoom depth.
  • Allows fully event driven vectors at higher zoom levels on the bottom end of the pyramid.

Con:

  • Static data at larger extents and lower zoom levels
  • Event driven objects are only available at the bottom end of the pyramid

Example:
sample site and demo video

tile layer sample

Fig 4 - Example of a tileLayer view - point data for earthquakes and Mile Markers

Fig 5 - Example of same data at a higher zoom using vector data display

4) Heatmap
Heatmaps refer to the use of color gradient or opacity overlays to display data density. The advantage of heat maps is the data reduction in the aggregating algorithm. To determine the color/opacity of a data set at a location the data is first aggregated by either a polygon or a grid cell. The sum of the data in a given grid cell is then applied to the color gradient dot for that cell. If heatmaps are rendered client side they have good performance only up to the latency limits of service side queries, internet bandwidth, and local rendering.

Fig 6 - Example of heatmap canvas over Bing Maps rendered client side

Grid Pyramids – Server side gridding
Hybrid server side gridding offers significant performance advantages when coupled with pre-processed grid cells. One technique of gridding processes a SQL data resource into a quadkey structure. Each grid cell is identified by its unique quadkey and contains the data aggregate at that grid cell. A grid quadkey sort by length identifies all of the grid aggregates at a specific quadtree level. This allows the client to efficiently download the grid data aggregates at each zoom level and render locally on the client in an html5 canvas over the top of a Bing Maps view. Since all grid levels are precompiled, resolution of cells can be adjusted by Zoom Level.

Pro:

  • Efficient display of very large data sets at wide extents
  • Can be coupled with vector displays at higher zoom levels for event driven objects

Con: gridding is pre-processed

  • real time data cannot be displayed
  • storage and time limitations apply to queries that have large numbers of permutations

Fig 7 – Grid Pyramid screen shot of UI showing opacity heatmap of Botnet infected computers

5) Thematic
Thematic maps use spatial regions such as states or zipcodes to aggregate data into color coded polygons. Data is aggregated for each region and color coded to show value. A hierarchy of polygons allows zoom levels to switch to more detailed regions at closer zooms. An example hierarchy might be Country, State, County, Sales territory, Zipcode, Census Block.

Pro:

  • Large data resources are aggregated into meaningful geographic regions.
  • Analysis is often easier using color ranges for symbolizing data variation

Con:

  • Rendering client side is limited to a few hundred polygons
  • Very large data sets require pre-processing data aggregates by region

Fig 8 - thematic map displaying data aggregated over 210 DMA regions using a quantized percentile range

6) Future trends
Big Data visualization is an important topic as the web continues to generate massive amounts of data useful for analysis. There are a couple of technologies on the horizon that help visualization of very large data resources.

A. Leverage of client side GPU

Here is an example WebGL http://www.web-maps.com/WebGLTest using CanvasLayer. ( only Firefox, chrome, IE11 *** Cannot be viewed in IE10 ***)

This sample shows speed of pan zoom rendering of 30,000 random points which would overwhelm typical js shape rendering. Data performance is good up to about 500,000 points per Brendan Kenny. Complex shapes need to be built up from triangle primitives. Tessellation rates for polygon generation approaches 1,000,000 triangles per 1000ms using libtess. Once tessellated the immediate mode graphic pipeline can navigate at up to 60fps. Sample code is available on github.

This performance is achieved by leveraging the client GPU. Because immediate mode graphics is a powerful animation engine, time animations can be used to uncover data patterns and anomalies as well as making some really impressive dynamic maps like this Uber sample. Unfortunately all the upstream latency remains: collecting the data from storage and sending it across the wire. Since we’re talking about larger sets of data this latency is more pronounced. Once data initialization finishes, client side performance is amazing. Just don’t go back to the server for new data very often.

Pro:

  • Good client side navigation performance up to about 500,000 points

Con:

  • requires a webgl enabled browser
  • requires GPU on the client hardware
  • subject to latency issues of server query and internet streaming
  • WebGL tessellation triangle primitives make display of polylines and polygons complex

Fig 9 – test webGL 30,000 random generated points (requires WebGL enabled browser – Firefox, Chrome, IE11)

Note: IE11 added WebGL capability which is a big boost for the web. There are still some glitches, however, and gl_PointSize in shader is broken for simple points like this sample.

Fig 10 – Very interesting WebGL animations of shipping GPS tracks using WebGL Canvas –courtesy Brendan Kenny

B. Leverage of server side GPU
MapD – Todd Mostak has developed a GPU based spatial query system called MapD (Massively Parallel Database)

MapD Synopsis:
  • MapD is a new database in development at MIT, created by Todd Mostak.
  • MapD stands for “massively parallel database.”
  • The system uses graphics processing units (GPUs) to parallelize computations. Some statistical algorithms run 70 times faster compared to CPU-based systems like MapReduce.
  • A MapD server costs around $5,000 and runs on the same power as five light bulbs.
  • MapD runs at between 1.4 and 1.5 teraflops, roughly equal to the fastest supercomputer in 2000.
  • uses SQL to query data.
  • Mostak intends to take the system open source sometime in the next year.
  • Bing Test: http://onterrawms.blob.core.windows.net/bingmapd/index.htm

    Bing Test shows an example of tweet points over Bing Maps and illustrates the performance boost from the MapD query engine. Each zoom or pan results in a GetMap request to the MapD engine that queries millions of tweet point records (81 million tweets Oct 19 – Oct 30), generating a viewport png image for display over Bing Map. The server side query latency is amazing considering the population size of the data. Here are a couple of screen capture videos to give you the idea of the higher fps rates:

    MapDBingTestAerialYellow50ms.wmv
    MapDBingHeatTest.wmv

    Interestingly, IE and FireFox handle cache in such a way that animations up to 100fps are possible. I can set a low play interval of 10ms and the player appears to do nothing. However, 24hr x12 days = 288 images are all being downloaded in just a few seconds. Consequently the next time through the play range the images come from cache and animation is very smooth. Chrome handles local cache differently in Windows 8 and it won’t grab from cache the second time. In the demo case the sample runs at 500ms or 2fps which is kind of jumpy but at least it works in Windows 8 Chrome with an ordinary internet download speed of 8Mbps

    Demo site for MapD: http://mapd.csail.mit.edu/

    Pro:

    • Server side performance up to 70x
    • Internet stream latency reduced to just the viewport image overlay
    • Client side rendering as a single image overlay is fast

    Con:

    • Source code not released, and there may be proprietary license restrictions
    • Most web servers do not include GPU or GPU clusters – especially cloud instances

    Note: Amazon AWS offers GPU Clusters but not cheap.

    Cluster GPU Quadruple Extra Large 22 GiB memory, 33.5 EC2 Compute Units, 2 x NVIDIA Tesla “Fermi” M2050 GPUs, 1690 GB of local instance storage, 64-bit platform, 10 Gigabit Ethernet( $2.100 per Hour)

    NVidia Tesla M2050 – 448 CUDA Cores per GPU and up to 515 Gigaflops of double-precision peak performance in each GPU!

    Fig 11 - Demo displaying public MapD engine tweet data over Bing Maps

    C. Spatial Hadoophttp://spatialhadoop.cs.umn.edu/
    Spatial Hadoop applies the parallelism of Hadoop clusters to spatial problems using the MapReduce technique made famous by Google. In the Hadoop world a problem space is distributed across multiple CPUs or servers. Spatial Hadoop adds a nice collection of spatial objects and indices. Although Azure Hadoop supports .NET, there doesn’t seem to be a spatial Hadoop in the works for .NET projects. Apparently MapD as open source would leap frog Hadoop clusters at least for performance per dollar.

    D. In Memory database (SQL Server 2014 Hekatron in preview release) – Microsoft plans to enhance the next version of SQL Server with in-memory options. SQL server 2014 in-memory options allows high speed queries for very large data sets when deployed to high memory capacity servers.

    Current SQL Server In-Memory OLTP CTP2

    Creating Tables
    Specifying that the table is a memory-optimized table is done using the MEMORY_OPTIMIZED = ON clause. A memory-optimized table can only have columns of these supported datatypes:

    • Bit
    • All integer types: tinyint, smallint, int, bigint
    • All money types: money, smallmoney
    • All floating types: float, real
    • date/time types: datetime, smalldatetime, datetime2, date, time
    • numeric and decimal types
    • All non-LOB string types: char(n), varchar(n), nchar(n), nvarchar(n), sysname
    • Non-LOB binary types: binary(n), varbinary(n)
    • Uniqueidentifier”

    Since geometry and geography data types are not supported with the next SQL Server 2014 in-memory release, spatial data queries will be limited to point (lat,lon) float/real data columns. It has been previously noted that for point data, float/real columns have equivalent or even better search performance than points in a geography or geometry form. In-memory optimizations would then apply primarily to spatial point sets rather than polygon sets.

    Natively Compiled Stored Procedures The best execution performance is obtained when using natively compiled stored procedures with memory-optimized tables. However, there are limitations on the Transact-SQL language constructs that are allowed inside a natively compiled stored procedure, compared to the rich feature set available with interpreted code. In addition, natively compiled stored procedures can only access memory-optimized tables and cannot reference disk-based tables.”

    SQL Server 2014 natively compiled stored procedures will not include any spatial functions. This means optimizations at this level will also be limited to float/real lat,lon column data sets.

    For fully spatialized in-memory capability we’ll probably have to wait for SQL Server 2015 or 2016.

    Pro:

    • Reduce server side latency for spatial queries
    • Enhances performance of image based server side techniques
      • Dynamic Tile pyramids
      • images (similar to MapD)
      • Heatmap grid clustering
      • Thematic aggregation

    Con:

    • Requires special high memory capacity servers
    • It’s still unclear what performance enhancements can be expected from spatially enabled tables

    D. Hybrids

    The trends point to a hybrid solution in the future which addresses the server side query bottleneck as well as client side navigation rendering bottleneck.

    Server side –
    a. In-Memory spatial DB
    b. Or GPU based parallelized queries

    Client side – GPU enhanced with some version of WebGL type functionality that can makes use of client GPU

    Summary

    Techniques are available today that can accommodate large date resources in Bing Maps. Trends indicate that near future technology can really increase performance and flexibility. Perhaps the sweet spot for Big Data map visualization over the next few years will look like a MapD or a GPU Hadoop engine on the server communicating to a WebGL UI over 1 gbps fiber internet.

    Orwell feared that we would become a captive audience. Huxley feared the truth would be drowned in a sea of irrelevance.

    Amusing Ourselves to Death, Neil Postman

    Of course, in America, we have to have the best of both worlds. Here’s my small contribution to irrelevance:

    Fig 12 - Heatmap animation of Twitter from MapD over Bing Maps (100fps)

    Kind of 3D with D3

    Fig 1 – NASA Near Earth Observation - Cloud Optical Thickness using D3js.org Orthographic projection

    Charts and Graphs

    Growing up in the days before graphing calculators (actually before calculators at all), I’ve had a lingering antipathy toward graph and chart art. Painful evenings plotting interminable polynomials one dot at a time across blue lined graph paper left its inevitable scars. In my experience, maps are a quite different thing altogether. Growing up in Ohio at the very edge of Appalachia, I have quite pleasant memories of perusing road atlases, planning escapes to a dimly grasped world somewhere/anywhere else.

    The “I’m outa here” syndrome of a mid-century youth, landed me in Colorado where I quickly found the compelling connection between USGS Topo maps and pleasant weekends climbing in Colorado’s extensive National Forests. I still fondly recall the USGS map desk at Denver’s Lakewood Federal Center where rows of flat metal cabinet drawers slid out to expose deep piles of 1:24000 scale topography maps with their rich linear contour features.

    I bore you with this personal recollection to set the stage for my amazement at discovering d3.js. My enduring prejudice of charts and graphs had already begun to crack a bit after seeing Hans Rosling’s entertaining Ted lectures from 2006. (D3 version of Wealth and Health of Nations). Perhaps my pentient for the concrete posed a barrier to more abstract demands of statistical modeling. At any rate Hans Rosling’s engaging enthusiasm was able to make a dent in my prejudice, and I found myself wondering how his graphs were coded. Apart from the explicit optimism conveyed by Rosling’s energy, the visual effect of primary colored balloons rising in celebratory fashion is quite hopefully contradictory of the harsh Hobbesian dictum.

    “the life of man, solitary, poor, nasty, brutish, and short.”

    The revelation that my childhood experience of the 50s is recapitulated by the modern economic experience of children in Trinidad and Tobago was something of an eye opener. Although no maps are in sight, the dynamic visualization of normally depressing socio-economic statistics lent visual energy to a changing landscape of global poverty.

    Fig 2 – Hans Rosling’s Wealth & Health of Nations implemented in D3

    The application of technology to the human condition seems so inspiring that it’s worth the reminder that technology has its flaws. Can it really be that the computational power in hand on the streets of Lagos Nigeria now exceeds that of Redmond or Palo Alto? Will ready access to MIT courseware on Probability and Statistics actually mitigate the desperation of Nairobi’s slums? A dash of Realpolitik is in order, but the graphics are none the less beautiful and inspiring.

    Fig 3 – Mobile phones transform Lagos

    D3.Geo

    d3js.org is an open source javascript library for data visualization. In d3, data enters, performs, and exits with a novel select pattern geared to dynamic rendering. The houselights dim, the stage is lit, data enters stage left, the data performance is exquisite, data exits stage right. This is a time oriented framework with data on the move. The dynamism of data is the key to compelling statistics and d3 examples illustrate this time after time.

    With d3.js Economics may remain the Dismal Science, but its charts and graphs can now be a thing of beauty. D3 charts are visually interesting, but more than that, chart smarts are leaking into the spatial geography domain. Mike Bostock of NYT fame and Jason Davies are prolific contributors to d3js.org.

    D3 version 3.0, released just a couple of weeks ago, added a wide range of projective geometry to d3.geo. Paths, Projections, and Streams combine to realize a rich html5 rendering library capable of animation effects rarely seen in GIS or on the web.

    Visually these are not your grandfathers’ charts and maps.

    TopoJSON

    There’s more. Sean Gillies recently remarked on the advent of TopoJSON, apparently a minor side project of mbostock, fully supported by D3.Geo.Paths. In the GIS world ESRI has long held the high ground on topological data (the Arc of GIS), and now some blokes from NYT charts and graphs have squashed it into an easy to use javascript library. Wow!

    TopoJSON is still more or less a compressed transport mechanism between OGC Simple Features in a server DB and client side SVG, but I imagine there will shortly be server side conversions for PostGIS and SQL Server to take advantage of major low latency possibilities. Client side simplification using the Visvalingam’s algorithm are virtually instantaneous, so zoom reduction can occur client side quite nicely.

    I think you get the idea. D3.js is powerful stuff and lots of fun.

    Some Experiments

    It’s the New Year’s holiday with spare time to fire up a favorite data source, NASA Neo, and try out a few things. The orthographic projection is an azimuthal projection with the advantage of reproducing the visual perspective of earth from a distant vantage point.

    Simple as:

    var projection = d3.geo.orthographic()
        .scale(245)
        .clipAngle(90);
    

    Fig 4 – Natural earth world data in d3js.org orthographic projection

    In addition to a graticule and circle edge paths, this takes advantage of TopoJSON formatted natural earth world data published on GitHUB by, you guessed it, mbostock.

            d3.json("topojson/world-110m.json", function (error, world) {
                nasa.svg.append("path")
                    .datum(topojson.object(world, world.objects.land))
                    .attr("class", "boundary")
                    .attr("d", nasa.path);
            });
    

    Add some mouse event handlers to whirl the globe:

    mousedown: function () {
            nasa.p0 = [d3.event.pageX, d3.event.pageY];
            nasa.context.clearRect(0, 0, nasa.width, nasa.height);
            d3.event.preventDefault();
        },
        mousemove: function () {
            if (nasa.p0) {
                nasa.p0 = [d3.event.pageX, d3.event.pageY];
                nasa.projection.rotate([nasa.λ(nasa.p0[0]), nasa.φ(nasa.p0[1])]);
                nasa.svg.selectAll("path").attr("d", nasa.path);
            }
        },
        mouseup: function () {
            if (nasa.p0) {
                nasa.mousemove();
                nasa.context.clearRect(0, 0, nasa.width, nasa.height);
                nasa.getOverlay(nasa.overlay);
                nasa.p0 = null;
            }
        },
    

    Interactive orthographic from mbostock has a nice set of handlers adapted for this experiment. Note there are still a few quirks regarding transposing mousedown event locations to be worked out in my experimental adaptation. (My holiday free time is running out so some things have to make do.)

    With mouse action, d3’s orthographic projection becomes a globe. It responds much more smoothly in Chrome than IE, apparently due to a javascript performance advantage in Chrome.

    Fig 5 – javascript performance testing

    This ortho globe feels 3D all because D3 affords a fast refresh through a projection on the vector continent and graticule paths.

    nasa.svg.selectAll("path").attr("d", nasa.path); 

    Vectors are fast, but for deeper information content I turn to NASA’s Near Earth Observation data, available as imagery from this public WMS service. The beauty of this imagery is still compelling after all these years.

    Fig 6 – NASA Land night temperature overlay

    However, geodetic imagery needs to be transformed to orthographic as well. D3 has all the necessary plumbing. All I added was the NASA WMS proxy with a .net WCF service.

    getOverlay: function (overlay) {
            if (overlay != null && overlay.length>0) {
    
                nasa.Showloading();
                nasa.image = new Image;
                nasa.image.onload = nasa.onload;
                nasa.image.src = Constants.ServiceUrl + "/GetMap?url=http://neowms.sci.gsfc.nasa.gov/wms/wms?Service=WMS%26version=1.1.1%26Request=GetMap%26Layers=" + overlay + "%26SRS=EPSG:4326%26BBOX=-180.0,-90,180,90%26width="+nasa.width+"%26height="+nasa.height+"%26format=image/jpeg%26Exceptions=text/xml";
            }
        },
    
        onload: function () {
    
            var dx = nasa.image.width,
                dy = nasa.image.height;
    
            nasa.context.drawImage(nasa.image, 0, 0, dx, dy);
    
            var sourceData = nasa.context.getImageData(0, 0, dx, dy).data,
                target = nasa.context.createImageData(nasa.width, nasa.height),
                targetData = target.data;
    
            for (var y = 0, i = -1; y < nasa.height; ++y) {
                for (var x = 0; x < nasa.width; ++x) {
                    var p = nasa.projection.invert([x, y]), λ = p[0], φ = p[1];
                    if (λ > 180 || λ < -180 || φ > 90 || φ < -90) { i += 4; continue; }
                    var q = ((90 - φ) / 180 * dy | 0) * dx + ((180 + λ) / 360 * dx | 0) << 2;
                    targetData[++i] = sourceData[q];
                    targetData[++i] = sourceData[++q];
                    targetData[++i] = sourceData[++q];
                    targetData[++i] = 255;
                }
            }
            nasa.context.clearRect(0, 0, nasa.width, nasa.height);
            nasa.context.putImageData(target, 0, 0);
            nasa.Hideloading();
        },
    

    Looping through 1,182,720 pixels in javascript is not the fastest, but just to be able to do it at all with only a dozen lines of javascript is very satisfying. There may be some server side options to improve performance and PostGIS Raster is worthy of investigation. However, html5 browsers with enhanced GPU access should eventually supply higher performance raster transforms.

    Fig 7 – NASA Land night temperatures transformed with D3 Orthographic projection.invert

    Selection JsTree

    For this experiment I also made use of JsTree for the layer selections out of the NASA WMS GetCapabilities. Layer choices are extensive and even with a tree expander approach the options overflow the available div space of IE’s jQuery UI draggable accordion panel. Scroll bars work poorly with draggable divs. A future enhancement could be manually allocated multiple expanders following NASA’s Ocean, Atmosphere, Energy, Land, and Life categories. Unfortunately this would invalidate the nice GetCapabilities layer extraction feature of the proxy service. NASA’s WMS also provides LegendURL elements for better understanding of the color ranges, which should be added to the selection panel.

    MouseWheel
    Since d3.geo offers projection.scale(), mousewheel events are a nice to have option that is easily bound to a browser window with jquery-mousewheel plugin.

    $(window).mousewheel(function (event, delta, deltaX, deltaY) {
                var s = nasa.projection.scale();
                if (delta > 0) {
                    nasa.projection.scale(s * 1.1);
                }
                else {
                    nasa.projection.scale(s * 0.9);
                }
                nasa.svg.selectAll("path").attr("d", nasa.path);
                nasa.context.clearRect(0, 0, nasa.width, nasa.height);
                nasa.getOverlay(nasa.overlay);
            });
    

    Tilted Perspective
    Even though NEO data resolution doesn’t really warrant it, using tilted perspective d3.geo.satellite projection affords a space station view point. Incorporating a steerable camera is a more complex project than time allows.

    var projection = d3.geo.satellite()
        .distance(1.1)
        .scale(5500)
        .rotate([76.00, -34.50, 32.12])
        .center([-2, 5])
        .tilt(25)
        .clipAngle(25);
    

    Fig 8 – Tilted perspective d3.geo.satellite projection of NASA BlueMarbleNG-TB

    Possible Extensions:
    Time is the next target, since NASA Neo exposes Time extension features.
    <Extent default=”2012-12-11″ name=”time”>

    Example:
    http://neowms.sci.gsfc.nasa.gov/wms/wms?Service=WMS&version=1.1.1&Request=GetMap&Layers=MOD_LSTAD_D&time=2000-03-06/2000-04-26/P1D&SRS=EPSG:4326&BBOX=-180.0,-85,180,85&width=1000&height=500&format=image/jpeg&Exceptions=text/xml

    One approach would be a time series download as an array of image objects from the proxy service, with a step through display bound to a slider on the UI. Once downloaded and projected the image stack could be serially displayed as an animation sequence. NASA WMS Time domain is somewhat abbreviated with only about 12-14 steps available. For a longer animation affect some type of caching worker role might store images as a continuous set of PostGIS raster data types with an ongoing monthly update. PostGIS has the additional advantage of pre-projected imagery fetching for more efficient time lapse performance.

    Warning:
    IIS requires adding a .json application/json mime type to make use of TopoJSON formatted data.

    Summary:

    Mike Bostock and Jason Davies have done us all a great favor making the power of d3 freely available on GitHUB under BSD License. For sheer prodigality http://bl.ocks.org/mbostock and http://bl.ocks.org/jasondavies examples can hardly be matched. Anyone need a Guyou Hemisphere projection or perhaps a Peirce Quincuncial tessellation? d3.geo has it ready to serve up in web friendly fashion.

    James Fee sums up d3 quite succinctly in his recent post “Why D3 Will Change How We Publish Maps”. I agree. We’re all joining the blokes over in charts and graphs.

    links:
    http://www.jasondavies.com/maps
    Interrupted Bonne
    World Tour
    Rotating EquiRect

    Extraterrestrial Map Kinections

    image

    Fig 1 – LRO Color Shaded Relief map of moon – Silverlight 5 XNA with Kinect interface

     

    Silverlight 5 was released after a short delay, at the end of last week.
    Just prior to exiting stage left, Silverlight, along with all plugins, shares a last aria. The spotlight now shifts abruptly to a new diva, mobile html5. Backstage the enterprise awaits with a bouquet of roses. Their concourse will linger long into the late evening of 2021.

    The Last Hurrah?

    Kinect devices continue to generate a lot of hacking interest. With the release of an official Microsoft Kinect beta SDK for Windows, things get even more interesting. Unfortunately, Kinect and the web aren’t exactly ideal partners. It’s not that web browsers wouldn’t benefit by moving beyond the venerable mouse/keyboard events. After all, look at the way mobile touch, voice, inertia, gyro, accelerometer, gps . . . have all suddenly become base features in mobile browsing. The reason Kinect isn’t part of the sensor event farmyard may be just a lack of portability and an ‘i’ prefix. Shrinking a Kinect doesn’t work too well as stereoscopic imagery needs a degree of separation in a Newtonian world.

    [The promised advent of NearMode (50cm range) offers some tantalizing visions of 3D voxel UIs. Future mobile devices could potentially take advantage of the human body’s bi-lateral symmetry. Simply cut the device in two and mount one half on each shoulder, but that isn’t the state of hardware at present. ]

    clip_image001

    Fig 2 – a not so subtle fashion statement OmniTouch

     

    For the present, experimenting with Kinect control of a Silverlight web app requires a relatively static configuration and a three-step process: the Kinect out there, beyond the stage lights, and the web app over here, close at hand, with a software piece in the middle. The Kinect SDK, which roughly corresponds to our visual and auditory cortex, amplifies and simplifies a flood of raw sensory input to extract bits of “actionable meaning.” The beta Kinect SDK gives us device drivers and APIs in managed code. However, as these APIs have not been compiled for use with Silverlight runtime, a Silverlight client will by necessity be one step further removed.

    Microsoft includes some rich sample code as part of the Kinect SDK download. In addition there are a couple of very helpful blog posts by David Catuhe and a codeplex project, kinect toolbox.

    Step 1:

    The approach for using Kinect for this experimental map interface is to use the GestureViewer code from Kinect Toolbox to capture some primitive commands arising from sensory input. The command repertoire is minimal including four compass direction swipes, and two circular gestures for zooming, circle clockwise zoom in, and circle counter clockwise zoom out. Voice commands are pretty much a freebie, so I’ve added a few to the mix. Since GestureViewer toolbox includes a learning template based gesture module, you can capture just about any gesture desired. I’m choosing to keep this simple.

    Step 2:

    Once gesture recognition for these 6 commands is available, step 2 is handing commands off to a Silverlight client. In this project I used a socket service running on a separate thread. As gestures are detected they are pushed out to local port 4530 on a tcp socket service. There are other approaches that may be better with final release of Silverlight 5.

    Step 3:

    The Silverlight client listens on port 4530, reading command strings that show up. Once read, the command can then be translated into appropriate actions for our Map Controller.

    clip_image003

    Fig 3 – Kinect to Silverlight architecture

    Full Moon Rising

     

    But first, instead of the mundane, let’s look at something a bit extraterrestrial, a more fitting client for such “extraordinary” UI talents. NASA has been very busy collecting large amounts of fascinating data on our nearby planetary neighbors. One data set that was recently released by ASU, stitches together a comprehensive lunar relief map with beautiful color shading. Wow what if the moon really looked like this!

    clip_image008

    Fig 4 – ASU LRO Color Shaded Relief map of moon

    In addition to our ASU moon USGS has published a set of imagery for Mars, Venus, Mercury, as well as some Saturn and Jupiter moons. Finally, JPL thoughtfully shares a couple of WMS services and some imagery of the other planets:
    http://onmars.jpl.nasa.gov/wms.cgi?version=1.1.1&request=GetCapabilities
    http://onmoon.jpl.nasa.gov/wms.cgi?version=1.1.1&request=GetCapabilities

    This type of data wants to be 3D so I’ve brushed off code from a previous post, NASA Neo 3D XNA, and adapted it for planetary data, minus the population bump map. However, bump maps for depicting terrain relief are still a must have. A useful tool for generating bump or normal imagery from color relief is SSBump Generator v5.3 . The result using this tool is an image that encodes relative elevation of the moon’s surface. This is added to the XNA rendering pipeline to combine a surface texture with the color relief imagery, where it can then be applied to a simplified spherical model.

    clip_image004

    Fig 5 – part of normal map from ASU Moon Color Relief imagery

    The result is seen in the MoonViewer client with the added benefit of immediate mode GPU rendering that allows smooth rotation and zoom.

    The other planets and moons have somewhat less data available, but still benefit from the XNA treatment. Only Earth, Moon, Mars, Ganymede, and Io have data affording bump map relief.

    I also added a quick WMS 2D viewer html using OpenLayers against the JPL WMS servers to take a look at lunar landing sites. Default OpenLayers isn’t especially pretty, but it takes less than 20 lines of js to get a zoomable viewer with landing locations. I would have preferred the elegance of Leaflet.js, but EPSG:4326 isn’t supported in L.TileLayer.WMS(). MapProxy promises a way to proxy in the planet data as EPSG:3857 tiles for Leaflet consumption, but OpenLayers offers a simpler path.

    clip_image006

    Fig 6 – OpenLayer WMS viewer showing lunar landing sites

    Now that the Viewer is in place it’s time to take a test drive. Here is a ClickOnce installer for GestureViewer modified to work with the Silverlight Socket service: http://107.22.247.211/MoonKinect/

    Recall that this is a Beta SDK, so in addition to a Kinect prerequisite, there are some additional runtime installs required:

    Using the Kinect SDK Beta

    Download Kinect SDK Beta 2:
    http://www.kinectforwindows.org/download/

    Be sure to look at the system requirements and the installation instructions further down the page. This is Beta still, and requires a few pieces. The release SDK is rumored to be available the first part of 2012.

    You may have to download some additional software as well as the Kinect SDK:

    Finally, we are making use of port 4530 for the Socket Service. It is likely that you will need to open this port in your local firewall.

    As you can see this is not exactly user friendly installation, but the reward is seeing Kinect control of a mapping environment. If you are hesitant to go through all of this install trouble, here is a video link that will give you an idea of the results.

    YouTube video demonstration of Kinect Gestures

     

    Voice commands using the Kinect are very simple to add so this version adds a few.

    Here is the listing of available commands:

           public void SocketCommand(string current)
            {
                switch (command)
                {
                        // Kinect voice commands
                    case "mercury-on": { MercuryRB.IsChecked = true; break; }
                    case "venus-on": { VenusRB.IsChecked = true; break; }
                    case "earth-on": { EarthRB.IsChecked = true; break; }
                    case "moon-on": { MoonRB.IsChecked = true; break;}
                    case "mars-on": { MarsRB.IsChecked = true; break;}
                    case "marsrelief-on": { MarsreliefRB.IsChecked = true; break; }
                    case "jupiter-on": { JupiterRB.IsChecked = true; break; }
                    case "saturn-on": { SaturnRB.IsChecked = true; break; }
                    case "uranus-on": { UranusRB.IsChecked = true; break; }
                    case "neptune-on": { NeptuneRB.IsChecked = true; break; }
                    case "pluto-on": { PlutoRB.IsChecked = true; break; }
    
                    case "callisto-on": { CallistoRB.IsChecked = true; break; }
                    case "io-on": { IoRB.IsChecked = true;break;}
                    case "europa-on": {EuropaRB.IsChecked = true; break;}
                    case "ganymede-on": { GanymedeRB.IsChecked = true; break;}
                    case "cassini-on": { CassiniRB.IsChecked = true; break; }
                    case "dione-on":  {  DioneRB.IsChecked = true; break; }
                    case "enceladus-on": { EnceladusRB.IsChecked = true; break; }
                    case "iapetus-on": { IapetusRB.IsChecked = true;  break; }
                    case "tethys-on": { TethysRB.IsChecked = true; break; }
                    case "moon-2d":
                        {
                            MoonRB.IsChecked = true;
                            Uri uri = Application.Current.Host.Source;
                            System.Windows.Browser.HtmlPage.Window.Navigate(new Uri(uri.Scheme + "://" + uri.DnsSafeHost + ":" + uri.Port + "/MoonViewer/Moon.html"), "_blank");
                            break;
                        }
                    case "mars-2d":
                        {
                            MarsRB.IsChecked = true;
                            Uri uri = Application.Current.Host.Source;
                            System.Windows.Browser.HtmlPage.Window.Navigate(new Uri(uri.Scheme + "://" + uri.DnsSafeHost + ":" + uri.Port + "/MoonViewer/Mars.html"), "_blank");
                            break;
                        }
                    case "nasaneo":
                        {
                            EarthRB.IsChecked = true;
                            System.Windows.Browser.HtmlPage.Window.Navigate(new Uri("http://107.22.247.211/NASANeo/"), "_blank"); break;
                        }
                    case "rotate-east": {
                            RotationSpeedSlider.Value += 1.0;
                            tbMessage.Text = "rotate east";
                            break;
                        }
                    case "rotate-west":
                        {
                            RotationSpeedSlider.Value -= 1.0;
                            tbMessage.Text = "rotate west";
                            break;
                        }
                    case "rotate-off":
                        {
                            RotationSpeedSlider.Value = 0.0;
                            tbMessage.Text = "rotate off";
                            break;
                        }
                    case "reset":
                        {
                            RotationSpeedSlider.Value = 0.0;
                            orbitX = 0;
                            orbitY = 0;
                            tbMessage.Text = "reset view";
                            break;
                        }
    
                    //Kinect Swipe algorithmic commands
                    case "swipetoleft":
                        {
                            orbitY += Microsoft.Xna.Framework.MathHelper.ToRadians(15);
                            tbMessage.Text = "orbit left";
                            break;
                        }
                    case "swipetoright":
                        {
                            orbitY -= Microsoft.Xna.Framework.MathHelper.ToRadians(15);
                            tbMessage.Text = "orbit right";
                            break;
                        }
                    case "swipeup":
                        {
                            orbitX += Microsoft.Xna.Framework.MathHelper.ToRadians(15);
                            tbMessage.Text = "orbit up";
                            break;
                        }
                    case "swipedown":
                        {
                            orbitX -= Microsoft.Xna.Framework.MathHelper.ToRadians(15);
                            tbMessage.Text = "orbit down";
                            break;
                        }
    
                    //Kinect gesture template commands
                    case "circle":
                        {
    
                            if (scene.Camera.Position.Z > 0.75f)
                            {
                                scene.Camera.Position += zoomInVector * 5;
                            }
                            tbMessage.Text = "zoomin";
                            break;
                        }
                    case "circle2":
                        {
                            scene.Camera.Position += zoomOutVector * 5;
                            tbMessage.Text = "zoomout";
                            break;
                        }
                }
            }

    Possible Extensions

    After posting this code, I added an experimental stretch vector control for zooming and 2 axis twisting of planets. These are activated by voice: ‘vector twist’, ‘vector zoom’, and ‘vector off.’ The Map control side of gesture commands could also benefit from some easing function animations. Another avenue of investigation would be some type of pointer intersection using a ray to indicate planet surface locations for events.

    Summary

    Even though Kinect browser control is not prime time material yet, it is a lot of experimental fun! The MoonViewer control experiment is relatively primitive. Cursor movement and click using posture detection and hand tracking is also feasible, but fine movement is still a challenge. Two hand vector controlling for 3D scenes is also promising and integrates very well with SL5 XNA immediate mode graphics.

    Kinect 2.0 and NearMode will offer additional granularity. Instead of large swipe gestures, finger level manipulation should be possible. Think of 3D voxel space manipulation of subsurface geology, or thumb and forefinger vector3 twisting of LiDAR objects, and you get an idea where this could go.

    The merger of TV and internet holds promise for both whole body and NearMode Kinect interfaces. Researchers are also adapting Kinect technology for mobile as illustrated by OmniTouch.

    . . . and naturally, lip reading ought to boost the Karaoke crowd (could help lip synching pop singers and politicians as well).

    clip_image008

    Fig 7 – Jupiter Moon Io

    FOSS4G 2011

    I was privileged to spend a few days at FOSS4G in Denver. It’s quite the opportunity to have an international convergence of GIS developers handily just a bus ride up the highway. I could walk among the current luminaries of OsGeo, Arnulf Christl, Paul Ramsey, Frank Warmerdam, Simone Giannecchini, and so on and on, and trade a handshake with the hard working front range folks, Peter Batty, Brian Timoney (way to go Brian!), and Matt Kruzemark too. The ideas are percolating in every corner, but it strikes me in an odd way that this world has changed irrevocably. The very technology that makes Geo FOSS possible is also making it, of all things, mundane.

    Open Standards

    It wasn’t that long ago that OGC standards were brand new. Each release anticipated a new implementation for discovery. Now implementations proliferate helter skelter. OGC services? Take your pick. Or, slice, dice, and merge with MapProxy. Map Tiling engines line up like yogurt brands at Whole Foods. GeoFOSS has progressed a long way from naive amazement over OGC WMS connections. WFS and WCS have already passed their respective moments of glorious novelty.

    This is the year of WPS, Web Processing Service, and the hope of constructing webs of analysis chains from disparate nodes. Each node adds a piece toward a final solution. Anyone with some experience using FME Transformers has some idea of what this looks like on the desktop. WPS moves analytic workflows into open standards and onto the web. Ref: 52north.org and GeoServer Ext

    FME Workbench
    Fig2 – example of FME Workbench processing chain

    On the floor at FOSS4G 2011

    In a different vein, the next release of PostGIS will push raster analysis right into SQL. This is new territory for PostGIS and elevates a raster datatype to the same level as geometry. This PostGIS Raster Functions page gives some flavor for the raster functions soon to be available when 2.0 releases. Ever needed a raster reprojection?

    raster ST_Transform(raster rast, integer srid, double precision scalex, double precision scaley, text algorithm=NearestNeighbor, double precision maxerr=0.125);

    Algorithm options are: ‘NearestNeighbor’, ‘Bilinear’, ‘Cubic’, ‘CubicSpline’, and ‘Lanczos.’

    Wow ST_MapAlgebra! And on the roadmap MapAlgebra raster on raster! MySQL and SQL Server have quite the feature curve to follow as PostGIS forges ahead.

    (See even the slightly jaded can catch some excitement at FOSS4G.)

    Numerous workshops dealt with javascript map frameworks OpenLayers, MapQuery, GeoExt, Leaflet. The html5 trend is underlined by news flashes from the concurrent Microsoft Build Conference.

    “For the web to move forward and for consumers to get the most out of touch-first browsing, the Metro style browser in Windows 8 is as HTML5-only as possible, and plug-in free. The experience that plug-ins provide today is not a good match with Metro style browsing and the modern HTML5 web.” Steven Sinofsky

    CouchDB/GeoCouch was present although “eventually consistent” NoSQL seems less pressing.

    Although Oracle is trying hard to break Java, it is still a popular platform among FOSS projects. The Javascript platform is a growth industry with the help of backend tools like Node.js, html5 WebSockets, and Asynchronous WebWorkers. C/C++ takes away wins in the popular performance shootout.

    3D WebGL made an appearance with WebGLEarth, while Nasa World Wind still has an occasional adherent.

    Open Software

    With over 900 attendees, the small world of FOSS seems all of a sudden crowded, jostling through accelerating growth. The Ordnance Survey was represented at a plenary session, “We have to learn to embrace open source. We want to use OS. We want to encourage OS.” Who’d of ever thought? FCC.gov opens their kimono in a big way with data transparency and OS APIs. Work shop topics include such things as: “Open Source Geospatial Software Powering Department of Defense Installation and Environment Business Systems,” or catch this, “The National Geospatial-Intelligence Agency OSS Challenge.” Is this the new Face of FOSS?

    Open Bureaucracy?

    Such bureaucracies know little of agile. The memorable phrase “embrace, extend, extinguish” comes to mind. But who embraces whom? Is the idea of FOSS and its larger www parent, a trend that eventually extinguishes bureaucracy or does the ancient ground of bureaucratic organization trump connection? Byzantine bureaucracy has been around since … well the Byzantine Empire, and given human nature is likely enduring. But, what does a “flat” Byzantine Bureaucracy look like? How about crowd sourced Byzantia? Would an aging Modernity mourn the loss of “Kafkaesque” as an adjective?

    Assuming growth continues, will success reduce the camaraderie of community as a motivation? Just this year we hear of OSM’s Steve Coast sidling into Microsoft, followed a little later by GDAL’s Frank Warmerdam beaming up into the Google mother ship. The corporate penalty, of course, is the loss of personal intellectual property. In other words, will Steve Coast’s imaginative ideas now fall under the rubric, “all your base are belong to us,” and Frank’s enduring legacy recede into the patent portfolio of “Do no evil?” FOSS4G halls are still filled with independent consultants and academics, but a significant corporate representation is evident by this apparent oxymoron, a presentation by ESRI, “Open Source GIS Solutions.”

    Some Perspective:

    The GIS nervous system grows connections around the world faster than a three year old’s brain. OK, maybe I exaggerate, after all, “During the first three years, your child’s brain establishes about 1,000 trillion nerve connections!” Really, how many connections are there in the World Wide Web. For all its impact on life, our beloved internet encompasses the equivalent of what, a few cubic centimeters of a single infant’s anatomy.

    Open Geospatial and FOSS are just a small part of this minor universe, but it’s easy to forget that even ten years ago this all hardly existed. “The first Interoperability Program testbed (Web Mapping Testbed) appeared in 1999.” About the same time Frank Warmerdam started GDAL and the GFOSS engines started.

    I still recall the wonder of downloading Sol Katz utilities from BLM’s ftp. The novelty of all this data free to use from the USGS was still fresh and amazing. Sol sadly died before seeing his legacy, but what a legacy. The Sol Katz award this year went to Java Topology Suite’s well deserving Martin Davis.

    Alice in Mirrorland – Silverlight 5 Beta and XNA

    “In another moment Alice was through the glass, and had jumped lightly down into the Looking-glass room”

    Silverlight 5 Beta was released into the wild at MIX 11 a couple of weeks ago. This is a big step for mirror land. Among many new features is the long anticipated 3D capability. Silverlight 5 took the XNA route to 3D instead of the WPF 3D XAML route. XNA is closer to the GPU with the time tested graphics rendering pipeline familiar to Direct3D/OpenGL developers, but not so familiar to XAML developers.

    The older WPF 3D XAML aligns better with X3D, the ISO sanctioned XML 3D graphics standard, while XNA aligns with the competing WebGL javascript wrapper for OpenGL. Eventually XML 3D representations also boil down to a rendering pipeline, but the core difference is that XNA is immediate mode while XML 3D is kind of stuck with retained mode. Although you pick up recursive control rendering with XML 3D, you lose out when it comes to moving through a scene in the usual avatar game sense.

    From a Silverlight XAML perspective, mirror land is largely a static machine with infrequent events triggered by users. In between events, the machine is silent. XAML’s retained mode graphics lacks a sense of time’s flow. In contrast, enter XNA through Alice’s DrawingSurface, and the machine whirs on and on. Users occasionally throw events into the machine and off it goes in a new direction, but there is no stopping. Frames are clicking by apace.

    Thus time enters mirror land in frames per second. Admittedly this is crude relative to our world. Time is measured out in the proximate range of 1/20th to 1/60th a second per frame. Nothing like the cusp of the moment here, and certainly no need for the nuance of Dedekind’s cut. Time may be chunky in mirror land, but with immediate mode XNA it does move, clicking through the present moment one frame at a time.

    Once Silverlight 5 is released there will be a continuous XNA API across Microsoft’s entire spectrum: Windows 7 desktops, Windows 7 phones, XBox game consoles, and now the browser. Silverlight 5 and WP7 implementations are a subset of the full XNA game framework available to desktop and XBox developers. Both SL5 and WP7 will soon have merged Silverlight XNA capabilities. For symmetry sake XBox should have Silverlight as apparently announced here. It would be nice for a web browsing XBox TV console.

    WP7 developers will need to wait until the future WP7 Mango release before merging XNA and Silverlight into a single app. It’s currently an either/or proposition for the mobile branch of XNA/SL.

    At any rate, with SL5 Beta, Silverlight and 3D XNA now coexist. The border lies at the <DrawingSurface> element:

    <DrawingSurface Draw="OnDraw" SizeChanged="DrawingSurface_SizeChanged" />

    North of the border lies XML and recursive hierarchies, a largely language world populated with “semantics” and “ontologies.” South of the border lies a lush XNA jungle with drums throbbing in the night. Yes, there are tropical white sands by an azure sea, but the heart of darkness presses in on the mind.

    XAML touches the academic world. XNA intersects Hollywood. It strikes me as one of those outmoded Freudian landscapes so popular in the 50’s, the raw power of XNA boiling beneath XAML’s super-ego. I might also note there are bugs in paradise, but after all this is beta.

    Merging these two worlds causes a bit of schizophrenia. Above is Silverlight XAML with the beauty of recursive hierarchies and below is all XNA with its rendering pipeline plumbing. Alice steps into the DrawingSurface confronting a very different world indeed. No more recursive controls beyond this border. Halt! Only immediate mode allowed. The learning curve south of the border is not insignificant, but beauty awaits.

    XNA involves tessellated models, rendering pipelines, vertex shaders, pixel shaders, and a high level shading language, HLSL, accompanied by the usual linear algebra suspects. Anytime you run across register references you know this is getting closer to hardware.

    …a cry that was no more than a breath: “The horror! The horror!”

    sampler2D CloudSampler : register(s0);
    static const float3 AmbientColor = float3(0.5f, 0.75f, 1.0f);
    static const float3 LampColor = float3(1.0f, 1.0f, 1.0f);
    static const float AmbientIntensity = 0.1f;
    static const float DiffuseIntensity = 1.2f;
    static const float SpecularIntensity = 0.05f;
    static const float SpecularPower = 10.0f;
    			.
    			.
    

    Here is an overview of the pipeline from Aaron Oneal’s MIX talk:

    So now that we have XNA it’s time to take a spin. The best way to get started is to borrow from the experts. Aaron Oneal has been very kind to post some nice samples including a game engine called Babylon written by David Catuhe.

    The Silverlight 5 beta version of Babylon uses Silverlight to set some options and SL5 DrawingSurface to host scenes. Using mouse and arrow keys allows the camera/avatar to move through the virtual environment colliding with walls etc. For those wishing to get an idea of what XNA is all about this webcafe model in Babylon is a good start.

    The models are apparently produced in AutoCAD 3DS and are probably difficult to build. Perhaps 3D point clouds will someday help, but you can see the potential for navigable high risk complex facility modeling. This model has over 60,000 faces, but I can still walk through exploring the environment without any difficulty and all I’m using is an older NVidia motherboard GPU.

    Apparently, SL5 XNA can make a compelling interactive museum, refinery, nuclear facility, or WalMart browser. This is not a stitched pano or photosynth interior, but a full blown 3D model.

    You’ve gotta love that late afternoon shadow affect. Notice the camera is evidently held by a vampire. I checked carefully and it casts no shadow!

    But what about mapping?

    From a mapping perspective the fun begins with this solar wind sample. It features all the necessary models, and shaders for earth, complete with terrain, multi altitude atmosphere clouds, and lighting. It also has examples of basic mouse and arrow key camera control.

    Solar Wind Globe
    Fig 4 – Solar Wind SL5 XNA sample

    This is my starting point. Solar Wind illustrates generating a tessellated sphere model with applied textures for various layers. It even illustrates the use of a normal (bump) map for 3D effects on the surface without needing a tessellated surface terrain model. Especially interesting is the use of bump maps to show a population density image as 3D.

    My simple project is to extend this solar wind sample slightly by adding layers from NASA Neo. NASA Neo conveniently publishes 45 categories and 129 layers of a variety of global data collected on a regular basis. The first task is to read the Neo GetCapabilities XML and produce the TreeView control to manage such a wealth of data. The TreeView control comes from the Silverlight Toolkit project. Populating this is a matter of reading through the Layer elements of the returned XML and adding layers to a collection which is then bound to the tree view’s ItemsSource property.

        private void CreateCapabilities111(XDocument document)
        {
            //WMS 1.1.1
            XElement GetMap = document.Element("WMT_MS_Capabilities").Element("Capability")
                .Element("Request").Element("GetMap").Element("DCPType")
                .Element("HTTP").Element("Get").Element("OnlineResource");
            XNamespace xlink = "http://www.w3.org/1999/xlink";
            getMapUrl = GetMap.Attribute(xlink + "href").Value;
            if (getMapUrl.IndexOf("?") != -1) getMapUrl =
                      getMapUrl.Substring(0, getMapUrl.IndexOf("?"));
    
            ObservableCollection layers = new ObservableCollection();
            foreach (XElement element in
    document.Element("WMT_MS_Capabilities").Element("Capability")
                    .Element("Layer").Descendants("Layer"))
            {
                if (element.Descendants("Layer").Count() > 0)
                {
                    WMSLayer lyr0 = new WMSLayer();
                    lyr0.Title = (string)element.Element("Title");
                    lyr0.Name = "header";
                    foreach (XElement element1 in element.Descendants("Layer"))
                    {
                        WMSLayer lyr1 = new WMSLayer();
                        lyr1.Title = (string)element1.Element("Title");
                        lyr1.Name = (string)element1.Element("Name");
                        lyr0.sublayers.Add(lyr1);
                    }
    
                    layers.Add(lyr0);
                }
            }
            LayerTree.ItemsSource = layers;
        }
    

    Once the tree is populated, OnSelectedItemChanged events provide the trigger for a GetMap request to NASA Neo returning a new png image. I wrote a proxy WCF service to grab the image and then write it to png even if the source is jpeg. It’s nice to have an alpha channel for some types of visualization.

    The difficulty for an XNA novice like myself is understanding the hlsl files and coming to terms with the rendering pipeline. Changing the source image for a Texture2D shader requires dropping the whole model, changing the image source, and finally reloading the scene model and pipeline once again. It sounds like an expensive operation but surprisingly this re-instantiation seems to take less time than receiving the GetMap request from the WMS service. In WPF it was always interesting to put a Video element over the scene model, but I doubt that will work here in XNA.

    The result is often a beautiful rendering of the earth displaying real satellite data at a global level.

    Some project extensions:

    • I need to revisit lighting which resides in the cloud shader hlsl. Since the original cloud model is not real cloud coverage, it is usually not an asset to NASA Neo data. I will need to replace the cloud pixel image with something benign to take advantage of the proper lighting setup for daytime.
    • Next on the list is exploring collision. WPF 3D provided a convenient RayMeshGeometry3DHitTestResult. In XNA it seems getting a point on the earth to trigger a location event requires some manner of collision or Ray.Intersects(Plane). If that can be worked out the logical next step is grabbing DEM data from USGS for generating ground level terrain models.
    • There is a lot of public LiDAR data out there as well. Thanks to companies like QCoherent, some of it is available as WMS/WFS. So next on the agenda is moving 3D LiDAR online.
    • The bump map approach to displaying variable geographic density as relief is a useful concept. There ought to be lots of global epidemiology data that can be transformed to a color density map for display as a relief bump map.

    Lots of ideas, little time or money, but Silverlight 5 will make possible a lot of very interesting web apps.

    Helpful links:
    Silverlight 5 Beta: http://www.silverlight.net/getstarted/silverlight-5-beta/
    Runtime: http://go.microsoft.com/fwlink/?LinkId=213904

    Silverlight 5 features:
    http://i1.silverlight.net/content/downloads/silverlight_5_beta_features.pdf?cdn_id=1
    “Silverlight 5 now has built-in XNA 3D graphics API”

    XNA: http://msdn.microsoft.com/en-us/aa937791.aspx
    Overview

    NASA Neo: http://localhost/NASA-Neo/publish.htm

    Babylon Scenes: Michel Rousseau, courtesy of Bewise.fr

    Babylon Engine: David Catuhe / Microsoft France / DPE

    Summary:

    “I am real!” said Alice, and began to cry.

    “You won’t make yourself a bit realler by crying,” Tweedledee remarked: “there’s nothing to cry about.”

    “If I wasn’t real,” Alice said – half-laughing through her tears, it all seemed so ridiculous – “I shouldn’t be able to cry.”

    Map Clipping with Silverlight



    Fig 1 – Clip Map Demo

    Bing Maps Silverlight Control has a lot of power, power that is a lot of fun to play with even when not especially practical. This weekend I was presented with a challenge to find a way to show web maps, but restricted to a subset of states, a sub-region. I think the person making the request had more in mind the ability to cut out an arbitrary region and print it for reporting. However, I began to think why be restricted to just the one level of the pyramid. With all of this map data available we should be able to provide something as primitive as coupon clipping, but with a dynamic twist.

    Silverlight affords a Clip masking mechanism and it should be simple to use.

    1. Region boundary:

    The first need is to compile the arbitrary regional boundary. This would be a challenge to code from scratch, but SQL Server spatial already has a function called “STUnion.” PostGIS has had an even more powerful Union function for some time, and Paul Ramsey has pointed out the power of fast cascaded unions. Since I’m interested in seeing how I can use SQL Serrver, though, I reverted to the first pass SQL Server approach. But, as I was looking at STUnion it was quickly apparent that this is a simple geo1.STUnion(geo2) function and what is needed is an aggregate union. The goal is to union more than just two geography elements at a time, preferably the result of an arbitrary query.

    Fortunately there is a codeplex project, SQL Server Tools, which includes the very thing needed, along with some other interesting functions. GeographyAggregateUnion is the function I need, Project/Unproject and AffineTransform:: will have to await another day. This spatial tool kit consists of a dll and a register.sql script that is used to import the functions to an existing DB. Once in place the functions can be used like this:

    SELECT dbo.GeographyUnionAggregate(wkb_geometry)) as Boundary
    FROM [Census2009].[dbo].[states]
    WHERE NAME = ‘Colorado’ OR NAME = ‘Utah’ or NAME = ‘Wyoming’

    Ignoring my confusing choice of geography column name, “wkb_geometry,” this function takes a “geography” column result and provides the spatial union:

    Or in my case:


    Fig 3 – GeographyUnionAggregate result in SQL Server

    Noting that CO, WY, and UT are fairly simple polygons but the result is 1092 nodes I tacked on a .Reduce() function.
    dbo.GeographyUnionAggregate(wkb_geometry).Reduce(10) provides 538 points
    dbo.GeographyUnionAggregate(wkb_geometry).Reduce(100) provides 94 points
    dbo.GeographyUnionAggregate(wkb_geometry).Reduce(100) provides 19 points

    Since I don’t need much resolution I went with the 19 points resulting from applying the Douglas-Peuker thinning with a tolerance factor of 1000.

    2. Adding the boundary

    The next step is adding this union boundary outlining my three states to my Silverlight Control. In Silverlight there are many ways to accomplish this, but by far the easiest is to leverage the builtin MapPolygon control and add it to a MapLayer inside the Map hierarchy:

    <m:MapLayer>
      <m:MapPolygon x:Name=”region”
      Stroke=”Blue” StrokeThickness=”5″
        Locations=”37.0003960382868,-114.05060006067 37.000669,-112.540368
        36.997997,-110.47019 36.998906,-108.954404
             .
            .
            .
        41.996568,-112.173352 41.99372,-114.041723
         37.0003960382868,-114.05060006067 “/>
    </m:MapLayer>


    Now I have a map with a regional boundary for the combined states, CO, WY, and UT.

    3. The third step is to do some clipping with the boundary:

    UIElement.Clip is available for every UIElement, however, it is restricted to Geometry clipping elements. Since MapPolygon is not a geometry it must be converted to a geometry to be used as a clip element. Furthermore PathGeometry is very different from something as straight forward as MapPolygon, whose shape is defined by a simple LocationCollection of points.

    PathGeometry in XAML:


    <Canvas.Clip>
      <PathGeometry>
        <PathFigureCollection>
          <PathFigure StartPoint=”1,1″>
            <PathSegmentCollection>
              <LineSegment Point=”1,2″/>
              <LineSegment Point=”2,2″/>
              <LineSegment Point=”2,1″/>
              <LineSegment Point=”1,1″/>
            </PathSegmentCollection>
          </PathFigure>
        </PathFigureCollection>
      </PathGeometry>
    </Canvas.Clip>


    The easiest thing then is to take the region MapPolygon boundary and generate the necessary Clip PathGeometry in code behind:

      private void DrawClipFigure()
      {
        if (!(MainMap.Clip == null))
        {
         &nbspMainMap.ClearValue(Map.ClipProperty);
        }
        PathFigure clipPathFigure = new PathFigure();
        LocationCollection locs = region.Locations;
        PathSegmentCollection clipPathSegmentCollection = new PathSegmentCollection();
        bool start = true;
        foreach (Location loc in locs)
        {
          Point p = MainMap.LocationToViewportPoint(loc);
          if (start)
          {
           clipPathFigure.StartPoint = p;
           start = false;
         }
         else
         {
          LineSegment clipLineSegment = new LineSegment();
          clipLineSegment.Point = p;
          clipPathSegmentCollection.Add(clipLineSegment);
         }
        }
        clipPathFigure.Segments = clipPathSegmentCollection;
        PathFigureCollection clipPathFigureCollection = new PathFigureCollection();
        clipPathFigureCollection.Add(clipPathFigure);

        PathGeometry clipPathGeometry = new PathGeometry();
        clipPathGeometry.Figures = clipPathFigureCollection;
        MainMap.Clip = clipPathGeometry;
      }

    This Clip PathGeometry can be applied to the m:Map named MainMap to mask the underlying Map. This is easily done with a Button Click event. But when navigating with pan and zoom, the clip PathGeometry is not automatically updated. It can be redrawn with each ViewChangeEnd:
    private void MainMap_ViewChangeEnd(object sender, MapEventArgs e)
    {
      if (MainMap != null)
      {
        if ((bool)ShowBoundary.IsChecked) DrawBoundary();
        if ((bool)ClipBoundary.IsChecked) DrawClipFigure();
      }
    }


    This will change the clip to match a new position, but only after the fact. The better way is to add the redraw clip to the ViewChangeOnFrame:

    MainMap.ViewChangeOnFrame += new EventHandler<MapEventArgs>(MainMap_ViewChangeOnFrame);

    private void MainMap_ViewChangeOnFrame(object sender, MapEventArgs e)
    {
      if (MainMap != null)
      {
        if ((bool)ShowBoundary.IsChecked) DrawBoundary();
        if ((bool)ClipBoundary.IsChecked) DrawClipFigure();
      }
    }


    In spite of the constant clip redraw with each frame of the navigation animation, navigation is smooth and not appreciably degraded.

    Summary:

    Clipping a map is not terrifically useful, but it is supported with Silverlight Control and provides another tool in the webapp mapping arsenal. What is very useful, are the additional functions found in SQL Server Tools. Since SQL Server spatial is in the very first stage of its life, several useful functions are not found natively in this release. It is nice to have a superset of tools like GeographyAggregateUnion, Project/Unproject,
    and AffineTransform::.

    The more generalized approach would be to allow a user to click on the states he wishes to include in a region, and then have a SQL Server query produce the boundary for the clip action from the resulting state set. This wouldn’t be a difficult extension. If anyone thinks it would be useful, pass me an email and I’ll try a click select option.



    Fig 4 – Clip Map Demo

    Mirror Land and the Last Foot


    Fig 1 – Bing Maps Streetside

    I know 2010 started yesterday but I slept in. I’m just a day late.

    Even a day late perhaps it’s profitable to step back and muse over larger technology trends. I’ve worked through several technology tides in the past 35 years. I regretfully admit that I never successfully absorbed the “Gang of Four” Design Patterns. My penchant for the abstract is relatively low. I learn by doing concrete projects, and probably fall into the amateur programming category often dismissed by the “professional” programming cognoscenti. However, having lived through a bit of history already, I believe I can recognize an occasional technology trend without benefit of a Harvard degree or even a “Professional GIS certificate.”

    What has been striking me of late is the growth of mirror realities. I’m not talking about bizarre multiverse theories popular in modern metaphysical cosmology, nor parallel universes of the many worlds quantum mechanics interpretation, or even virtual world phenoms such as Second Life or The Sims. I’m just looking at the mundane evolution of internet mapping.


    Fig 2 – Google Maps Street View

    One of my first mapping projects, back in the late 80′s, was converting the very sparse CIA world boundary file, WDBI, into an AutoCAD 3D Globe (WDBI came on a data tape reel). At the time it was novel enough, especially in the CAD world, to warrant a full color front cover of Cadence Magazine. I had lots of fun creating some simple AutoLisp scripts to spin the world view and add vector point and line features. I bring it up because at that point in history, prior to the big internet boom, mapping was a coarse affair at global scales. This was only a primitive wire frame, ethereal and transparent, yet even then quite beautiful, at least to map nerds.


    Fig 3 – Antique AutoCAD Globe WDBI

    Of course, at that time Scientists and GIS people were already playing with multi million dollar image aquisitions, but generally in fairly small areas. Landsat had been launched more than a decade earlier, but few people had the computing resources to play in that arena. Then too, US military was the main driving force with DARPA technology undreamed by the rest of us. A very large gap existed between Global and Local scales, at least for consumer masses. This access gap continued really until Keyhole’s aquisition by Google. There were regional initiatives like USGS DLG/DEM, Ordnance Survey, and Census TIGER. However, computer earth models were fragmented affairs, evolving relatively slowly down from satellite and up from aerial, until suddenly the entire gap was filled by Google and the repercussions are still very much evident.

    Internet Map coverage is now both global and local, and everything in between, a mirror land. The full spectrum of coverage is complete. Or is it? A friend remarked recently that they feel like recent talk in mobile LiDAR echos earlier discussions of “Last Mile” when the Baby Bells and Cable Comms were competing for market share of internet connectivity. You can glimpse the same echo as Microsoft and Google jocky for market share of local street resolution, StreetView vs Streetside. The trend is from a global coarse model to a full scale local model, a trend now pushing out into the “Last Foot.” Alternate map models of the real world are diving into human dimension, feet and inches not miles, the detail of the street, my local personal world.

    LiDAR contributes to this mirror land by adding a partial 3rd dimension to the flat photo world of street side capture. LiDAR backing can provide the swivel effects and the icon switching surface intelligence found in StreetView and Streetside. LiDAR capture is capable of much more, but internet UIs are still playing catchup in the 3rd dimension.

    The question arises whether GIS or AEC will be the driver in this new human dimension “mirror land.” Traditionally AEC held the cards at feet and inches while GIS aerial platforms held sway in miles. MAC, Mobile Asset Collection, adds a middle way with inch level resolution capability available for miles.


    Fig 4 – Video Synched to Map Route

    Whoever, gets the dollars for capture of the last foot, in the end it all winds up inside an internet mirror land.

    We are glimpsing a view of an alternate mirror reality that is not a Matrix sci-fi fantasy, but an ordinary part of internet connected life. Streetside and Street View push this mirror land down to the sidewalk.

    On another vector, cell phone locations are adding the first primitive time dimension with life tracks now possible for millions. Realtime point location is a first step, but life track video stitched on the fly into photosynth streams lends credence to street side contingency.

    The Location hype is really about linking those massive market demographic archives to a virtual world and then back connecting this information to a local personal world. As Sean Gillies in “Utopia or Dystopia” pointed out recently there are pros and cons. But, when have a few “cons” with axes ever really made a difference to the utopian future of technology?

    With that thought in mind why not push a little on the future and look where the “Last Millimeter” takes us?
        BCI Brain Computer Interface
        Neuronal Prosthetics


    Fig 5 – Brain Computer Interface

    Eye tracking HUD (not housing and urban development exactly)


    Fig 6- HUD phone?

    I’m afraid the “Last Millimeter” is not a pretty thought, but at least an interesting one.

    Summary

    Just a few technology trends to keep an eye on. When they get out the drill for that last millimeter perhaps it’s time to pick up an ax or two.