Monthly Archives: March 2008

Deep Zoom Easter Eggs?

Posted on by
Reply

Easter rolls around early this year and the MIX08 Deep Zoom announcement made it just in time to try some experiments in the Easter Egg tradition. Using Deep Zoom Composer, images can be inserted at various locations within the zoom pyramid. This means full images of documents, html pages, schematics, data records, photos … can be inserted down at lower levels of a main image’s zoom. This was pointed out in Laurence Moroney’s recent blog posting, which showed an image inserted into the pupil of an eye in a framing picture. Looking at potential for this type of DeepZoom, here is an experiment from a geospatial perspective.



Fig 1 Denver USGS UrbanArea 0.25m zoomed in to make Coors Field visible

The SparseImageSceneGraph.xml generated by Deep Zoom Composer shows how position and insert scale are part of a SceneNode Element:

<SceneGraph version=”1″>
<AspectRatio>0.999999999999994</AspectRatio>
<SceneNode>
<FileName>C:\temp\Denver\denver5\source images\den8x8.jpg</FileName>
<x>0</x>
<y>0</y>
<Width>1</Width>
<Height>1</Height>
<ZOrder>1</ZOrder>
</SceneNode>
.
.
.
<SceneNode>
<FileName>C:\temp\Denver\denver5\source images\taylorbuchholz.jpg</FileName>
<x>0.59990935133569</x>
<y>0.584386453325495</y>
<Width>0.000326534539695684</Width>
<Height>0.000457282456000806</Height>
<ZOrder>8</ZOrder>
</SceneNode>
<SceneNode>
<FileName>C:\temp\Denver\denver5\source images\jeffbaker.jpg</FileName>
<x>0.590298544487025</x>
<y>0.565998027596921</y>
<Width>0.000436734620380339</Width>
<Height>0.000624543890228259</Height>
<ZOrder>9</ZOrder>
</SceneNode>
.
.
.
</SceneGraph>

This brings to mind the various easter egg items available in well known software packages. Interestingly aerial imagery lends itself to this type of feature since the spatial dimension provides a handy reference for hanging additional information.

As an experiment I’ve updated a Denver USGS Urban Area imagery tile with some easter egg information in the Deep Zoom approach. In this case zooming into Coors Field you will notice some small patches scattered around the field. Behind home plate are a couple of informational items from the Colorado Rockies Home page, including a roster list. Then panning around the field you will see various player cards at their field positions.

In each of these cases the image is available at sufficient scale to read the text. The scene graph will allow multiple depth image insertion, so I could go further hiding some detail of a player life inside a dot on an individual player card.


Fig 2 – Zoomed in to see the location of additional SceneNode images


Fig 3 – Zooming further you can read the text and view additional information embedded in the map


Fig 4 – Finally zoomed to an individual player card at short stop.

So far I can see this as little more than a novelty, but perhaps it would have some uses. For example a sitemap builder could be enhanced to output a Deep Zoom image to allow moving around a multipage graph showing every page in a site. This type of tool would be a great way to QC a static html site (are there any of these still around?). Perhaps facility management would benefit as well, since personnel records could be associated with a desk inside an office, inside a building floor, inside an office campus giving a secondary spatial way to review office space. Adding some papers to the desktop and then sending a deepzoom link with the apropriate zoom factor might be a novel approach to document workflow management:
public void ZoomAboutLogicalPoint( double zoomIncrementFactor, double zoomCenterLogicalX, double zoomCenterLogicalY )

In telecom, outside plant items such as transformers and switches could contain schematics embedded within a snapshot of the item, which is itself embedded in a wide area schematic or aerial imagery showing a city wide sector.
The next trick is to find a way to back track through a Deep Zoom Scene graph to a real world coordinate. This seems possible using the pair of methods shown in the MultiScaleImage class:

MultiScaleImage..::.LogicalToElementPoint Method
MultiScaleImage..::.ElementToLogicalPoint Method

Once a coordinate has been reversed into its outer root element coordinates, it should be possible to assign a real world georeference, which would be used for a nearest query on a PostGIS table for obtaining additional attributes or accessing live data.

Which brings up the big problem I see to date on the DeepZoom technology: it is static. The processing of large image sets into pyramids is not fast. Although there are some tantalizing command line hints in the ImageTool.exe and SparseImageTool.exe packaged with Deep Zoom Composer there is no real documentation:

usage: ImageTool [-v] options:
-v enable verbose outputcommands:
convert

[outdir ] [basename ] [explode]
[large[tif|jpg|wdp|png] [quality ] [tilesize

]
Converts an image or a dir with files to a Seadragon stored image.
The optional outdir parameter specifies the output directory path.
The optional basename parameter allows overriding the output filesename
which defaults to the basename of the input file.
The default form is single-file (not exploded).
Specify large to use a slower algorithm, but one that can process very large images.
The default format is HDPhoto (wdp).
The quality value should be a number in the interval [0.0-1.0]. Default is 0.8
A quality value of 1.0 will result in lossless compression if possible in the chosen format.
If not specified, tilesize values default to size of 255 and a physical size of 256.
Legal values for tilesize values are >= 64 and <= 2048.dump [levels] [index]
Dumps info about the specified Seadragon stored image file.

extract [] []
Extract level image into specified output directory in
TIFF format. If level is not specified, all are output.
If the output directory is not specified, files are written to
the current working directory.

The possibility of adding smaller items to an already existing large pyramid holds promise for more dynamic real time updating from a content management system.

In looking at the pyramid of jpg files it seems possible to add a simple place holder of the right dimension and then change content by overwriting the already existing pyramid jpgs with updated content. In this way easter egg content with relatively small jpg sets could be updated dynamically out of a content management system on each refresh. If Microsoft SeaDragon adds MultiScaleImage elements to WPF xaml some of the static issues surrounding DeepZoom viewing might be easier to handle.

Twitt

Deep Zoom a TerraServer UrbanArea on EC2

Posted on by
Reply


Fig 1 – Silverlight MultiScaleImage of a high resolution Denver image – 200.6Mb .png

Just to show that I can serve a compiled Deep Zoom Silverlight app from various Apache servers I loaded this Denver example on a Windows 2003 Apache Tomcat here: http://www.web-demographics.com/Denver, and then a duplicate on a Linux Ubuntu7.10 running as an instance in the Amazon EC2, this time using Apache httpd not Tomcat: http://www.gis-ows.com/Denver Remember these are using beta technology and will requires updating to Silverlight 2.0. The Silverlight install is only about 4.5Mb so the install is relatively painless on a normal bandwidth connection.

Continuing the exploration of Deep Zoom, I’ve had a crash course in Silverlight. Silverlight is theoretically cross browser compatible (at least for IE, Safari, and FireFox), and it’s also cross server. The trick for compiled Silverlight is to use Visual Studio 2008 with .NET 3.5 updates. Under the list of new project templates is a template called ‘Silverlight application’. Using this template sets up a project that can be published directly to the webapp folder of my Apache Server. I have not tried a DeepZoom MultiScaleImage on Linux FireFox or Mac Safari clients. However, I can view this on a Windows XP FireFox updated to Silverlight 2.0Beta as well as Silverlight updated IE7 and IE8 beta.

Creating a project called Denver and borrowing liberally from a few published examples, I was able to add a ClientBin folder under my Denver_Web project folder. Into this folder goes the pyramid I generate using Deep Zoom Composer. Once the pyramid is copied into place I can reference this source from my MultiScaleImage element source. Now the pyramid is viewable.

To make the MultiScaleImage element useful, I added a couple of additional .cs touches for mousewheel and drag events. Thanks to the published work of Lutz Gerhard, Peter Blois, and Scott Hanselman this was just a matter of including a MouseWheelHelper.cs in the project namespace and adding a few delegate functions to the main Page initialization code behind file. Pan and Zoom .cs

Now I need to backtrack a bit. How do I get some reasonable Denver imagery for testing this Deep Zoom technology? Well I don’t belong to DRCOG which I understand is planning on collecting 6″ aerials. There are other imagery sets floating around Denver, as well, I believe down to 3″ pixel resolution. However, the cost of aerial capture precludes any free and open source type of use. However, there is some nice aerial data available from the USGS. The USGS Urban Area imagery is available for a number of metropolitan areas, including Denver.


Fig 2 – Same high resolution Denver image zoomed in to show detail

USGS Urban Area imagery is a color orthorectified image set captured at approximately 1ft pixel resolution. The data is made available to the public through the TerraServer WMS. Looking over the TerraServer UrbanArea GetCapabilities layer I see that I can ‘GetMap’ this layer in EPSG:26913 (UTM83-13m). The best possible pixel resolution through the TerraServer WMS is 0.25m per pixel. To achieve this level of resolution I can use the max pixel Height and Width of 2000 over a metric bounding box of 500m x 500m. http://gisdata.usgs.net/IADD/factsheets/fact.html

For example:
http://terraservice.net/ogcmap.ashx?version=1.1.1&service=WMS&ServiceName=WMS&request=GetMap&layers=UrbanArea&srs=EPSG:26913&bbox=511172,4399768,511672,4400268&WIDTH=2000&HEIGHT=2000

This is nice data but I want to get the max resolution for a larger area and mosaic the imagery into a single large image that I will then feed into the Deep Zoom Composer tool for building the MultiScaleImage pyramid. Java is the best tool I have to make a simple program to connect to the WMS and pull down my images one at a time into the tiff format.
try {
File OutFile = new File(dir+imageFileName);
URL u = new URL(url);
HttpURLConnection geocon = (HttpURLConnection)u.openConnection();
geocon.setAllowUserInteraction(false);
geocon.setRequestMethod(“GET”);
geocon.setDoOutput(true);
geocon.setDoInput(true);
geocon.setUseCaches(false);
BufferedImage image = ImageIO.read(geocon.getInputStream());
ImageIO.write(image,”TIFF”,OutFile);
geocon.disconnect();
System.out.println(“download completed to “+dir+imageFileName+” “+bbox);
}

Looping this over my desired area creates a directory of 11.7Mb tif images. In my present experiment I grabbed a set of 6×6 tiles, or 36 tiff files at a total of 412Mb. The next step is to collect all of these tif tiles into a single mosaic. The Java JAI package contains a nice tool for this called mosaic:
mosaic = JAI.create(“mosaic”, pbMosaic, new RenderingHints(JAI.KEY_IMAGE_LAYOUT, imageLayout));

Iterating pbMosaic.addSource(translated); over my set of TerraServer tif files and then using PNGImageEncoder, I am able to create a single png file of about 200Mb. Now I have a sufficiently large image to drop into the Deep Zoom Composer for testing. The resulting pyramid of jpg files is then copied into my ClientBin subdirectory of the Denver VS2008 project. From there it is published to the Apache webapp. Now I can open my Denver webapp for viewing the image pyramid. On this client system with a good GPU and dual core cpu the image zoom and pan is quite smooth and replicates a nice local application viewing program with smooth transitions around real time zoom pan space. On an older Windows XP running FireFox the pan and zoom is very similar. This is on a system with no GPU so I am impressed.

Peeking into the pyramid I see that the bottom level 14 contains 2304 images for a 200Mb png pyramid. Each image stays at 256×256 and the compression ranges from 10kb to 20kb per tile. Processing into the jpg pyramid compresses from the original 412Mb tif set => 200.5Mb png mosaic => 45.7Mb 3084 file jpg pyramid. Evidently there is a bit of lossy compression, but the end effect is that the individual tiles are small enough to stream into the browser at a decent speed. Connected with high bandwidth the result is very smooth pan and zoom. This is basically a Google Earth or Virtual Earth user experience all under my control!

Now that I have a workflow and a set of tools, I wanted to see what limits I ran into. The next step was to increment my tile set to an 8×8 for 64 tifs to see if my mosaic tool would endure the larger size as well as the DeepZoom Composer. My JAI mosaic will be the sticking point on a maximum image size since the source images are built in memory which on this machine is 3Gb. Taking into account Vista’s footprint I can actually only get about 1.5Gb. One possible workaround to that bottleneck is to create several mosaics and then attempt to splice them in the Deep Zoom Composer by manually positioning them before exporting to a pyramid.

First I modified my mosaic program to write a Jpeg output with jpgParams.setQuality(1.0f); This results in a faster mosaic and a smaller export. The JAI PNG encoder is much slower than JPEG. With this modification I was able to export a couple of 3000m x 3000m mosaics as jpg files. I then used Deep Zoom Composer to position the two images horizontally and exported as a single collection. In the end the image pyramid is 6000m x 3000m and 152Mb of jpg tiles. It looks like I might be able to scale this up to cover a large part of the Denver metro UrbanArea imagery.

The largest mosaic I was able to get Deep Zoom Composer to accept was 8×8 or 16000px x 16000px which is just 4000m x 4000m on the ground. Feeding this 143Mb mosaic through Composer resulted in a pyramid consists of 5344 jpg files at 82.3Mb. However, scaling to a 5000m x 5000m set of 100 tif, the 221Mb mosaic, failed on import to Deep Zoom Composer. I say failed, but in this prerelease version the import finishes with a blank image shown on the right. Export works in the usual quirky fashion in that the export progress bar generally never stops, but in this case the pyramid also remains empty. Another quirky item to note is that each use of Deep Zoom Composer starts a SparseImageTool.exe process which continues consuming about 25% of cpu even after the Deep Zoom Composer is closed. After working awhile you will need to go into task manager and close down these processes manually. Apparently this is “pre-release.”


Fig 3 – Same high resolution Denver image zoomed in to show detail of Coors Field players are visible

Deep Zoom is an exciting technology. It allows map hackers access to real time zoom and pan of large images. In spite of some current size limitations on the Composer tool the actual pyramid serving appears to have no real limit. I verified on a few clients and was impressed that this magic works in IE and FireFox although I don’t have a Linux or Mac client to test. The compiled code serves easily from Apache and Tomcat with no additional tweaking required. My next project will be adapting these Deep Zoom pyramids into a tile system. I plan to use either an OWS front end or a Live Maps with a grid overlay. The deep zoom tiles can then be accessed by clicking on a tile to open a Silverlight MultiScaleImage. This approach seems like a simple method for expanding coverage over a larger metropolitan area while still using the somewhat limiting Deep Zoom Composer pre release.

Twitt

BMNG SilverLight 2.0 Beta Deep Zoom

Posted on by
Reply

Well somebody had to do it! Im willing to give it a try.

What is it?

One of the many announcements at the MIX08 conference is the availability of Deep Zoom technology for Silverlight 2.0 Beta 1. This results from an R&D program in Microsoft called Sea Dragon. Sea Dragon was evidently a Microsoft acquisition awhile back. Reminiscent of Keyhole(now google earth), Sea Dragon is a tool for smooth viewing of very large image resources. The novelty is to have it useable inside a SilverLight browser view. http://labs.live.com/Seadragon.aspx

For those so inclined to load beta stuff there is a demo available here with a nice code behind zoom pan capability: http://visitmix.com/blogs/Joshua/Hard-Rock-Cafe/

The idea is that behind the viewport is a large image in PNG, JPEG, TIFF, BMP format which is fed to the screen resolution using a MultiScaleImage element in a SilverLight 2.0 page. Actually just jpeg as it turns out.

Well of course this is extremely interesting to geospatial people since all of the WMS, WCS, WFS, KML, tile caching, image pyramids etc are aimed at exactly this functionality.

How is the user experience?

Holy Mackerel, finally a reason to use Vista! But wait this is SilverLight it should work in FireFox and Safari too. By the way,if any Sea Dragon big Tuna comes trolling by this post I already have my map hacker wishlist for 2.0 at the end of this entry.

OK, let’s try it.

First I grabbed a full resolution BMNG image off of BitTorrent: http://www.geotorrent.org/torrents/61.torrent

Chris Holmes has detailed instructions on this part here: http://docs.codehaus.org/display/GEOSDOC/Load+NASA+Blue+Marble+Data

The eventual result will be twelve large(267Mb) ecw files, “world-topo-bathy-200408-3x86400x43200.ecw,” one for each month of 2004. Ecw is a wavelet compression format for imagery http://en.wikipedia.org/wiki/ECW_(file_format), but to use the image we need it in a different format, Gdal to the rescue. The easiest approach is to take advantage of the FWtools bin directory to run a command line translation like this:

“C:\Program Files\FWTools2.1.0\bin\gdal_translate” -of GTiff world-topo-bathy-200401-3x86400x43200.ecw BMNG.tiff

After a few minutes the result is a tiff image of 86400×43200 of about 11Gb. Now it is time to use the Deep Zoom Composer (actually a decomposer) to process this into a MultiScaleImage info.bin
http://blogs.msdn.com/expression/archive/2008/03/05/download-the-preview-of-the-deep-zoom-composer.aspx

When I attempted an import of this 11Gb tiff into Deep Zoom Composer, the Mermaid.exe choked after a few minutes. I guess we aren’t ready for geospatial scale exactly yet. Note to self: do this with -o Tiff, since mermaids may not like GTiff.

So I went back to a smaller downsample to start my experiment. This time I chose a 3600×1800 png at 4.154Mb. This was rather instant success. Now there is a BMNG test pyramid on my hard drive. The pyramid is 13 levels and each subdirectory contains the necessary images in jpg. Deep Zoom Composer rather handily chunks the tiles in each level, even calculating non square tiling.


Fig 1 Example of the a pyramid level resulting from Deep Zoom Image Composer

After playing around a bit the export spits out what we need for a Silverlight MultiScaleImage. Remember this is an element type introduced with SilverLight2.0 Beta so you can’t really see this unless you want to do a beta Silverlight 2.0 install.

Here are some links on other neat things in R&D over at Microsoft labs.live: http://labs.live.com/photosynth/whatis/

SilverLight works a lot better with an IIS server, but I am using an Apache server, so I created a javascript Silverlight project. Using the default project, I modified the Scene.xaml and associated scene.js to make use of the new MultiScaleImage element:

<MultiScaleImage      
x:Name=“msi“      
ViewportWidth=“1.0“      
ViewportOrigin=“0,0“      
Source=“/BMNGZoom/BMNG/info.bin“>

This worked pretty well to get the image object in the browser with a sort of spring loaded entre. Perhaps the springy annoyance can be turned off by setting UseSpringsProperty=”off.” However, adding zoom and pan are bit more problematic. I am brand new to Silverlight but oddly there seem to be very few events available:
MouseMove, MouseEnter, MouseLeave, MouseLeftButtonDown, MouseLeftButtonUp

If you want MouseRight, Keyboard, MouseWheel etc, you need to have some code behind. Since I didn’t really have time to figure out all of the code behind tricks for getting this to serve from Apache, I took a primitive approach. By attaching an event to MouseLeftButtonUp I can simulate a click event. Then connecting this click event to the MultiScaleImage ViewportWidth *= 0.9; I could make a one way zoom in without too much effort. Not very useful, but good enough to get a feel for the interaction, which by the way is very impressive. The zooming is familiar to anyone used to VE or GE type of continuous zoom. Pretty nifty for a browser interface.

There is even an ‘old view’ to ‘new view’ animation effect, which cleverly distracts the user while the new tiles are streaming in over the old tiles. Local tile cache makes revisiting very smooth. I will have to try this on an older tier 0 GPU system so I can watch the tiles move slowly into place.

http://www.web-maps.com/BMNGZoom/ >


Fig 2- primitive test of a Deep Zoom BMNG

Now that I had this working on a relatively small 4.1 Mb image, my next step was to step up the size and see what the effect would be. I already knew 11Gb Gtiff was not going to work. Dividing the full Blue Marble into 8 tiles and using PNG output seemed like a possibility. This gives 8 files at 256Mb each.

However, I noticed that the pyramid files are jpeg so why not start with 8 jpeg files instead:
“C:\Program Files\FWTools2.1.0\bin\gdal_translate” -of JPEG
-projwin -180 90 -90 0 world-topo-bathy-200401-3x86400x43200.ecw BMNG1.jpg
Input file size is 86400, 43200
Computed -srcwin 0 0 21600 21600 from projected window.
0…10.

After a few minutes I could open the Deep Zoom Image Composer again and do an import on the full eight tile collection. The composer did not show anything in the view window with these larger jpg images so I was working blind on composition. I did the export anyway out of curiosity.
I’ll post the results next week since it will take a good bit of uploading time. The result of this bit of experiment was quite surprising. The pyramid building goes fairly smoothly in the Deep Zoom Composer and is painless compared to manually building pyramids in Java. Actually Geotools ImagePyramid has some advantages like iteration over multiple images and command line capability. But the resulting tile pyramid doesn’t have the client side streaming.The MultiScaleImage element hides the complexity of an ajax slippy map interface in a single element. On the down side adding functionality seems to be aimed at IIS ASP type servers. I imagine with a bit of time I can work out the details of a pan and MouseWheel zoom. SilverLight includes RenderTransform matrix capability, it just requires code behind to make it useful with mouse and keyboard functions.

The question is “how does this work?” Of course the answer is “I dont know,” but that doesn’t stop some speculation. The pyramid is obvious. The fact that it works on both a linux or a windows box eliminates a stub socket on the server side. It appears to be an object downloaded to the client which orchestrates things in an ajax mode. Of course clr won’t work with Firefox on Linux or Safari so there must be a plugin object which can be duplicated cross platform.

Wishlist for Deep Zoom 2.0 from a map hacker

1. Can we scale this puppy to use extremely large image sets? I imagine DCOG is going to want to see their 6″ aerials for the Denver metro area in Deep Zoom. Why should they have to come up with a pyramid tile set of MultiScaleImage elements?

2. How about MultiScaleImage elements for WPF xaml/xbap? I would like to use it with a more comprehensive set of code behind tools as an xbap.

3. Once it’s in WPF how about using some OO magic and add MultiScaleImageBrush for draping on a 3D mesh?

Lets extrapolate a couple more steps

4. Why stop at images? How about extending to MultiScale3DMesh. Then my spare time project for an AJAX LiDAR viewer won’t require much work.

5. Don’t stop there, lets have MultiScaleImageBrush on MultiScale3DMesh.

Now sharpen your patent pen

6. Why not MultiScaleVideo? Sooner or later all of the bifocaled baby boomers will be downloading movies to their iPhone, oops ZunePhone. How else are we going to see anything on those miniscule screens. Besides talk about “immersive,” movies could really be interactive. Imax resolution on a phone, why not!

Twitt

Wide Area HVAC controller using WPF and ZigBee Sensor grid

Posted on by
Reply

One project I’ve been working on recently revolves around an online controller for a wide area HVAC system. HVAC systems can sometimes be optimized for higher efficiency by monitoring performance in conjunction with environment parameters. Local rules can be established for individual systems based on various temperatures, humidity, and duct configurations. Briefly, a set of HVAC functions consisting of on/off relay switches and thermistors, can be observed from an online monitoring interface. Conversely state changes can be initiated online by issuing a command to a queue. These sensors and relays might be scattered over a relatively large geographic area and in multiple locations inside a commercial building.

It is interesting to connect a macro geospatial world with a micro world, drilling down through a local facility to a single thermistor chip. In the end its all spatial.

Using a simple map view allows drill down from a wide area to a building, a device inside a building, a switch bank, and individual relay or analog channel for monitoring or controlling. The geospatial aspect of this project is somewhat limited, however, the zoom and pan tools used in the map location also happen to work well in the facilities and graphing views.

The interface can be divided into three parts:
1) The onsite system – local base system and Zigbee devices
2) The online server system – standard Apache Tomcat
3) The online client interface – WPF xbap, although svg would also work with a bit more work

Onsite System

The electronically impaired, like myself, may find the details of controller PIC chip sets, relays, and thermistor spec sheets baffling, but really they look more hacky than they are:

Fig 0 – Left: Zigbee usb antenna ; Center: thermistor chip MCP9701A; Right: ProXR Zigbee relay controller

The onsite system is made up of sensors and controller boards. The controller boards include a Zigbee antenna along with a single bank of 8 relays and an addition set of 8 analog inputs. The sensors are wired to the controller board in this development mode. However, Zigbee enabled temperature sensors are also a possibility, just more expensive. See SunSpot for example: http://www.sunspotworld.com/ (Open Source hardware? )

ZigBee is a wifi type communications protocol based on IEEE 802.15.4. It allows meshes of devices to talk to each other via RF as long as they are within about 100-300 ft of another node on the mesh. Extender repeaters are also available. ZigBee enabled devices can be scattered around a facility and communicate back to a base system by relaying messages node to node through an ad hoc mesh network.

The onsite system has a local pc acting as the base server. The local onsite server communicates with an external connection via a internet router and monitors the Zigbee network. ASUS EeePCs look like a good candidate for this type of application. Messages originating from outside are communicated down to the individual relay through the ZigBee mesh, while state changes and analog readings originating from a controller relay or sensor channel are communicated up the ZigBee network and then passed to the outside from the local onsite server.

The local server must have a small program polling the ZigBee devices and handling communications to the outside world via an internet connection. The PC is equipped with a usb ZigBee antenna to communicate with the other ZigBee devices in the network. This polling software was written in Java even though that may not the best language for serial USB com control in Windows. The target system will be Linux based. The ZigBee devices we selected came with the USB driver that treats a USB port like a simple COM port.

Since this was a Java project the next step was finding a comm api. The sun JavaComm has discontinued support of Windows, although it is available for Linux. Our final onsite system will likely be Linux for cost reasons, so this is only a problem with the R&D system which is Windows based. I ended using a RXTX library, RXTXcomm.jar, at http://www.jcontrol.org/download/rxtx_en.html

Commands for our ProXR controller device are a series of numeric codes, for example<254;140;3;1>This series of commands puts the controller in command mode 254, a set bank status command 140, a byte indicating relays 0 and 1 on 3, and bank address 1. The result is relays 0 and 1 are switched to the on position. The commands are issued similarly for reading relay state and analog channels. <254;166;1> for example reads all 8 analog I/O channels as a set of 8 bytes.

Going in prototype mode we picked up a batch of three wire MCP9701A thermistor chips for a few dollars. The trick is to pick the right resistance to get voltage readings in to the mid range of the 8bit or 10bit analog channel read. Using 8 bit output lets us poll for temperature with around .5 degree F resolution.

The polling program issues commands and reads results on separate threads. If state is changed locally it is communicated back to the online server on the next polling message, while commands from the online command queue are written to the local controller boards with the return. In the meantime every polling interval sends an analog channel record back to the server.

Online Server

The online server is an Apache Tomcat service with a set of servlets to process communications from the onsite servers. Polled analog readings are stored in a PostgreSQL database with building:device: bank:channel addresses as well as a timestamp. The command queue is another PostgreSQL table which is checked at each poll interval for commands addressed to the building address which initiated the poll. Any pending commands are returned to the polling onsite server where they will be sent out to the proper device:bank:relay over the ZigBee network.

Two other tables simply provide locations of buildings as longitude, latitude in the wide area HVAC control system. Locations of devices insidebuildings are stored in a building table as floor and x,y coordinates. These are available for the client interface.

Client Interface

The client interface was developed using WPF xbap to take advantage of xaml controls and a WMS mapping interface. Initially the client presents a tabbed menu with a map view. The map view indicates the wide area HVAC extents with a background WMS image for reference. Zooming in to the building location of interest allows the user to select a building to show a floor plan with device locations indicated.

Fig 1 HVAC wide area map view

Once a building is selected the building floor plans are displayed. Selecting an individual device determines the building:device address.

Fig 2 Building:device address selection from facilities floor plan map

Finally individual relays can be selected from the device bank by pushing on/off buttons. Once the desired switch configuration is set in the panel, it can be sent to the command queue as a building:device:bank address command. Current onsite state is also double checked by the next polling return from the onsite server.

Fig 3 Relay switch panel for selected building:device:bank address.

The analog IO channels are updated to the server table at the set polling interval. A selection of the analog tab displays a set of graph areas for each of the 8 channels. The on/off panel initiates a server request for the latest set of 60 polled readings which are displayed graphically. It won’t be much effort to extend this analog graph to a bidirectional interface with user selectable ranges set by dragging floor and ceiling lines that trigger messages or events when a line is crossed.

Fig 4 Analog IO channel graphs

This prototype incorporates several technologies using a Java based Tomcat service online and a Java RXTXcomm Api for the local Zigbee polling. The client interface is also served out of Apache Tomcat as WPF xaml to take advantage of easier gui control building. In addition OGC WMS is used for the map views. The facilities plan views will be DXF translations to WPF xaml. Simple graphic selection events are used to build addresses to individual relays and channels. The server provides historical command queues and channel readings by storing time stamped records. PostgreSQL also has the advantage of handling record locking on the command queue when multiple clients are accessing the system.

This system is in the prototype stage but illustrates the direction of control systems. A single operator can maintain and monitor systems from any locations accessible to the internet, which is nearly anywhere these days. XML rendering graphics grammars for browsers like svg and xaml enable sophisticated interfacesthat are relatively simple to build.

There are several OGC specifications oriented toward sensor grids, http://www.opengeospatial.org/projects/groups/sensorweb. The state of art is still in flux but by virtue of the need for spatial management of sensor grids, there will be a geospatial component in an “ubiquitous sensor” world.

Twitt

LiDAR processing with Hadoop on EC2

Posted on by
Reply

I was inspired yesterday by a FRUGOS talk from a Lidar production manager showing how to use some OpenSource tools to make processing LiDAR easier. LiDAR stands for LIght Detection And Ranging. It’s a popular technology these days for developing detailed terrain models and ground classification. The USGS has some information here: USGS CLick and the state of PA has an ambitious plan to fly the entire state every three years at a 1.4 ft resolution. PAMAP LiDAR

My first stop was to download some .las data from the PAMAP site. This PA LiDAR has been tiled into 10000x10000ft sections which are roughly 75Mb each. First I wrote a .las translator so I could look at the data. The LAS 1.1 specification is about to be replaced by LAS 2.0 but in the meantime most available las data is still using the older spec. The spec contains a header with a space for variable length attributes and then the point data. Each point record contains 20bytes (at least in the PAMAP .las)  of information including x,y,z, as long values and a classification and reflectance value. The xyz data is scaled and offset by values in the header to obtain double values. The LiDAR sensor mounted on an airplane flies tracks across the are of interest while the LiDAR pulses at a set frequency. The location of the reading can then be determined by post processing against the GPS location of the instrument. The pulse scans result in tracks that angle across the flight path.

Since the PAMAP data is available in PA83-SF coordinates, despite the metric unit shown in the meta data, I also added a conversion to UTM83-17. This was a convenience for displaying over Terraserver backgrounds using the available EPSG:26917. Given the hi res aerial data available from PAMAP this step may be unnecessary in a later iteration.

Actually PAMAP has done this step already. Tiff grids are available for download. However, I’m interested in the gridding algorithm and possible implementation in a hadoop cluster so PAMAP is a convenient test set with a solution already available for comparison. It is also nice to think about going directly from raw .las data to my end goal of a set of image and mesh pyramids for the 3D viewer I am exploring for gmti data.


Fig 1 – LiDAR scan tracks in xaml over USGS DOQ

Although each section is only about 3.5 square miles the scans generate more than a million points per square mile. The pulse rate achieves roughly 1.4ft spacing for the PAMAP data collection. My translator produces a xaml file of approximately 85Mb which is much too large to show in a browser. I just pulled some scans from the beginning and the end to show location and get an idea of scale.

The interesting aspect of LiDAR is the large amount of data collected which is accessible to the public. In order to really make use of this in a web viewer I will need to subtile and build sets of image pyramids. The scans are not orthogonal so the next task will be to grid the raw data into an orthoganal cell set. Actually there will be three grids, one for the elevation, one for the classification, and one for the reflectance value. With gridded data sets, I will be able to build xaml 3D meshes overlaid with the reflectance or classification. The reflectance and classification will be translated to some type of image file, probably png.

Replacing the PA83-SF with a metric UTM base gives close to 3048mx3048m per PAMAP section.If the base cell starts at 1 meter, and using a 50cellx50cell patch, the pyramid will look like this:
dim cells/patch size  approx faces/section
1m 50x50  50mx50m  9,290,304
2m 50x50  100mx100m 2,322,576
4m 50x50  200mx200m   580,644
8m 50×50   400mx400m   145,161
16m 50×50   800mx800m 36,290
32m 50x50  1600mx1600m 9072
64m 50x50  3200mx3200m 2268
128m 50x50  6400mx6400m 567
256m 50x50  12800mx12800m 141

There will end up being two image pyramids and a 3D zaml mesh pyramid with the viewer able to move from one level to the next based on zoom scale. PAMAP also has high resolution aerial imagery which could be added as an additional overlay.

Again this is a large amount of data, so pursuing the AWS cluster thoughts from my previous posting it will be interesting to build a hadoop hdfs cluster. I did not realize that there is actually a hadoop public ami available with ec2 specific scripts that can initialize and terminate sets of EC2 instances. The hadoop cluster tools are roughly similar to the Google GFS with a chunker master and as many slaves as desired. Hadoop is an Apache project started by the Lucene team inspired by the Google GFS. The approach is useful in processing data sets larger than available disk space, or 160Gb in a small EC2 instance. The default instance limit for EC2 users is currently at 20. With permission this can be increased but going above 100 seems to be problematic at this stage. Using a hadoop cluster of 20 EC2 instances still provides a lot of horsepower for processing data sets like LiDAR. The cost works out to 20x$0.10 per hour = $2.00/hr which is quite reasonable for finite workflows. hadoop EC2 ami

This seems to be a feasible approach to doing gridding and pyramid building on large data resources like PAMAP. As PAMAP paves the way, other state and federal programs will likely follow with comprehensive sub meter elevation and classification available for large parts of the USA. Working out the browser viewing requires a large static set of pre-tiled images and xaml meshes. The interesting part of this adventure is building and utilizing a supercomputer on demand. Hadoop makes it easier but this is still speculative at this point.


Fig 1 – LiDAR scan tracks in xaml over USGS DRG

I still have not abandoned the pipline approach to WPS either. 52North has an early implementation of a WPS available written in Java. Since it can be installed as a war, it should be simple to startup a pool of EC2 instances using an ami with this WPS war installed. The challenge, in this case, is to make a front end in WPF that would allow the user to wire up the instance pool in different configurations. Each WPS would be configured with an atomistic service which is then fed by a previous service on up the chain to the data source. Conceptually this would be similar to the pipleline approach used in JAI which builds a process chain, but leaves it empty until a process request is initialized. At that point the data flows down through the process nodes and out at the other end. Conceptually this is quite simple, one AMI per WPS process and a master instance to push the data down the chain. The challenge will be a configuration interface to select a WPS atom at each node and wire the processes together.

I am open to suggestions on a reasonably useful WPS chain to use in experimentations.

AWS is an exciting technology and promises a lot of fun for computer junkies who have always had a hankering to play with supercomputing, but didn’t quite make the grade for a Google position.

Twitt