Neo vs Paleo Geography

Jurassic Dinosaur skeleton
Fig 1 – Paleo, Neo – what about Jurassic Geography?

I gather that there is some twittering about neo versus paleo geography. See Peter Batty’s blog entry or James Fee’s blog. I myself don’t Twitter, but in general I’m happy for Peter’s paleo accomodation of the non twitterers, repeating the conversation in a blog entry. Peter has also updated comments with a new post questioning, “Are we now in a post neogeography era?” The dreaded paradigm shifts are coming fast and furiously.

I am not really able to comment on neo vs paleo as I myself fall even further back into “Jurassic Geography.” Looking at connotations we have this accounting:

····neo - 1990 – present, new, recent, different, Obama, Keynesian, Apple, Google Earth, Cloud, Java C# RubyRails Twitter

····paleo - as in paleolithic 2.8m – 10k old, prehistoric, ancient, early, primitive, Nixon, Supply Side, Microsoft, Windows Desktop, ESRI Arc???, C C++ Javascript telephone

Obviously the “paleo” label is not carried with quite the honor of “neo.” It’s reminiscent of the Galen / Myers-Brigg personality typology characterized as Lion, Otter, Beaver, and Golden Retriever. What do you want to be? Obviously not the beaver, but there has to be a significant part of the world in that category, like it or not. After all what would lions eat for dinner without a few of us beavers? Likewise there is a significant branch of paleo in the GIS kingdom.

However, in the pre-paleolithic era there are still a few of us left, falling into the “long tail” of the Jurassic. So carrying on down the connotation stream here is the Jurassic geography equivalent:

····jurassic – 206m-144m dinosaurs, fossils, pre paleolithic, Hoover, laissez faire, IBM Big Iron, Assembly Cobol, open source

Wait “Open Source” – Jurassic Geography? How did that get in there? The notoriously frugal days of Hoover never made it into the paleolithic era’s “Supply Side” economy. It’s Keynesian economics all over the neo world, so Jurassic geography is the frugal end of the spectrum and how can you get more frugal than free! Obviously Open Source is as Jurassic as they come in Geography circles.

As I’ve recently been in a gig hunting mode, I’ve been having quite a few in depth conversations about GIS stacks. As a small businessman outside the corporate halls of paleo geography, I’ve had few occasions to get an in depth education on the corporate pricing world. So I spent the past couple of days looking into it.

Let’s start at the bottom of the stack. Here is some retail pricing on a few popular GIS databases:

  • Oracle Standard Spatial $17,500 + $3850 annual
  • Oracle Enterprise Locator $47,500 + $10,450 annual
  • SQL Server 2008 Web edition ~ $3500
  • PostgreSQL/PostGIS $0.00

If you’re a Jurassic geographer which do you choose? Probably not Oracle Enterprise Locator. If your Paleo you look at that and think, “Man, I am the negotiator! We don’t pay any of that retail stuff for the masses.” Neo? – well how would I know how a neo thinks?

Next take a look at the middle tier:

  • ESRI ArcGIS Server standard workgroup license
    ····Minimum $5000 2cores + $1250 2core annual
    ····Additional cores $2500/core + $625/core annual
  • ESRI ArcGIS hosted application server license
    ····Minimum $40,000 4 cores + $10,000 4 core annual
    ····Additional cores $10,000/core + $2500/core annual
  • OWS GeoServer or MapServer minimum $0 + annual $0
    But, this is GIS isn’t it? We want some real analytic tools not just a few hundred spatial functions in JTS Topology suite. OK, better throw in a few QGIS or GRASS installs and add a few $0s to the desktop production. Oh, and cores, we need some, “make that a 16core 64 bit please” – plus $0.

I think you catch the Jurassic drift here. How about client side.

  • ESRI Silverlight free, well sort of , if you’re a developer, NGO, educational, or non-profit otherwise take a look at that ArcGIS license back a few lines.
  • Google API it’s Neo isn’t it? $10k per annum for a commercial use, maybe its Paleo after all.
  • Virtual / Bing Maps api $8k per annum transaction based and in typical license obfuscation fashion impossible to predict what the final cost will be. Paleo, “Just send me the invoice.”
  • OpenLayers is a javascript api client layer too, just solidly Jurassic at $0
  • Silverlight well it can be Jurassic, try DeepEarth over at codeplex or MapControl from Microsoft with the Bing imageservice turned off, OSM on.

It’s been an interesting education. Here is the ideal Jurassic GIS stack:
Amazon EC2 Windows instance + PostGIS database + GeoServer OWS + IIS Silverlight MapControl client
The cost: starts at $100/mo(1 processor 1.7Gb 32bit) up to $800/mo(4 processor 15Gb 64bit)

So what does a Jurassic geographer get in this stack?

Hardware:
Amazon Cloud based virtual server, S3 Backup, AMI image replication, Elastic IP, AWS console, choice of OS, cores, memory, and drive space. Ability to scale in a big way with Elastic load balancing, auto scaling, and CloudWatch monitoring. Performance options like CloudFront edge service or something experimental like Elastic MapReduce Hadoop clusters.

Database:
PostgreSQL/PostGIS – Standards compliant SQL server with GIST spatial indexing on OGC “Simple Features for SQL” specification compliant geometry with extended support for 3DZ, 3DM and 4D coordinates. A full set of roughly 175 geometry, management, and spatial functions. It supports almost all projections. All this and performance? maybe a little vague but not shabby:

“PostGIS users have compared performance with proprietary databases on massive spatial data sets and PostGIS comes out on top.”

Middle Tier:
Geoserver – standards compliant OWS service for WMS, WFS, WCS.
Data sources: Shapefile, Shapefile Directory, PostGIS, external WFS, ArcSDE, GML, MySQL, Oracle, Oracle NG, SQL Server, VPF
Export formats: WFS GML, KML, SVG, PDF, GeoRSS, Png, Jpeg, Geotiff, OGR Output – MapInfo Tab and MID/MIF, Shp, CSV, GeoJSON …
OGC standard SLD styling, built in gwc tile caching – seeded or as needed, managed connection pools, RESTful configuration api, and ACEGI integrated security.

WCS adds :

  1. ArcGrid – Arc Grid Coverage Format
  2. ImageMosaic – Image mosaicking plugin
  3. WorldImage – A raster file accompanied by a spatial data file
  4. Gtopo30 – Gtopo30 Coverage Format
  5. GeoTIFF – Tagged Image File Format with Geographic information
“GeoServer is the reference implementation of the Open Geospatial Consortium (OGC) Web Feature Service (WFS) and Web Coverage Service (WCS) standards, as well as a high performance certified compliant Web Map Service (WMS). “

Browser client viewer:
Take your pick here’s a few:

Summary:
Well in these economic times Jurassic may in fact meet Neo. The GIS world isn’t flat and Jurassic going right eventually meets Neo going left, sorry Paleos. Will Obama economics turn out to be Hooverian in the end? Who knows, but here’s a proposition for the Paleos:

Let me do a GIS distribution audit. If I can make a Jurassic GIS Stack do what the Paleo stack is currently providing, you get to keep those annual Paleo fees from here to recovery. How about it?

rkgeorge @cadmaps.com

A Noob in Oracle Land

The announcement that Oracle is supporting AMIs in the Amazon cloud came as a surprise to me. I had heard that there was a teaser version of Oracle out there for developers, but had not expected Oracle to jump on the cloud side, especially after Larry Ellison’s recent diatribe against cloud computing.

   “It’s complete gibberish. It’s insane. When is this idiocy going to stop?”

Just curious about this oracle of gibberish, I went on a tour of Oracle Land, the Kingdom of Ellison. This is no small undertaking for an enterprise as ambitious as Oracle. There are endless products and sub-products. The base of the pyramid is the database server, but after buying 50 or more companies in the last year or so, the borders of the empire extend way beyond RDBMS.

The venerable RDBMS has come a long way since IBMs E.F. Codd introduced the concept back in the 70s. I vaguely remember Oracle breaking into the PC world shortly after Turbo Pascal. There was a single DB product for the DOS IBM PC, and documentation consisted of a couple of grayish paperback manuals. Shortly after this, late 80s, a small vendor introduced GeoSQL to hook AutoCAD to the GIS world through Oracle. This was my first introduction to the potential of spatial databases and Oracle. The empire of Ellison has grown since then, and now documentation would fill a library as well as Ellisons bank account.

As an aside, we live in an interesting age at the dusk of the great technology innovators. The infamous industrialists of the previous era now exist only as shadowy figures in history texts, but the business innovators of technology are still walking among us, Larry Ellison, Bill Gates, Steve Jobs. The multi-billion personal fortunes are just now entering the charitable fund phase where our grand children will know their names in some impersonally institutional mode such as the Gates Foundation.

First stop in Oracle Land was a download of the free, as in free beer, teaser version, OracleXE.

  • Total data stored in XE is limited to 4GB
  • XE is limited to 1GB of RAM
  • XE is limited to 1 processor

Since my entire interest in Oracle is the spatial side, my next stop was Justin Lokitz’s helpful article on integration with Geoserver. Leading to this:


Fig 1 -http://localhost:80/geoserver/wms?service=WMS&request=GetMap&format=
image/png&width=800&height=600&srs=EPSG:4326&layers=topp:COUNTIES
&styles=countypopdensity&bbox=-177.1,13.71,-61.48,76.63

Fig 2 – http://localhost:80/geoserver/wms/kml_reflect?layers=COUNTIES

Not a bad start. The Geoserver layer abstracts away the spatial guts of OracleXE. However, curiosity leads on. I found that OracleXE has some spatial components labelled ‘Locator’ as opposed to ‘Spatial’. Though only a subset of the extensive enterprise spatial version, geometry queries are possible. It took me a bit to find my way around.

Interestingly the open source world is generally more helpful in this respect. Although extensive, the forums of commercial software vendors are less friendly. For instance Paul Ramsey of Refraction fame is regularly present on the PostGIS forums, and Frank Warmerdam is always available to give a helping hand at the immensely useful www.gdal.org. But I doubt that I will ever run across a Larry Ellison post on the OracleXE forum. Many posts to commercial forums appear to languish unanswered, which is seldom the case in the OpenSource project forums I monitor.

It is worth noting that gdal’s ogr2ogr can be built with Oracle support on systems with Oracle Client libraries installed.

Oracle’s SDO_Geometry is present in a useful form letting users run geographic join queries like this:

   select c.COUNTY, c.STATE_ABRV, c.TOTPOP, c.POPPSQMI from states s, counties c where s.state = ‘California’ and sdo_anyinteract (c.geom, s.geom) = ‘TRUE’;

My next step was to look at SDO_Geometry in JDBC. Unfortunately Oracle’s JGeometry spatial library is not available for OracleXE, but the LGPL open source JTS library provides helpful OraReader and OraWriter classes. These encapsulate the SDO_GEOMETRY Struct translation to/from jts.geom.Geometry, where the rest of the JTS api can be applied.

logger.info(rsmd.getColumnName(i)+": "+rsmd.getColumnType(i));
st = (oracle.sql.STRUCT) rs.getObject(1);
//convert STRUCT into JGeometry not available in OracleXE
//JGeometry j_geom = JGeometry.load(st);

//JTS to the rescue
OraReader reader = new OraReader();
Geometry geom = reader.read(st);
Coordinate[] coords = geom.getCoordinates();
		.
		.

Next stop, Amazon AWS EC2. Here is a list of the public Oracle AMIs offered::
   Oracle Database 11g Release 1 Enterprise Edition – 64 Bit
   Oracle Database 11g Release 1 Enterprise Edition – 32 Bit
   Oracle Database 11g Release 1 Standard Edition/Standard Edition One – 32 Bit
   Oracle Database 10g Release 2 Express Edition – 32 Bit

The last in the list, OracleXE edition, is the one to experiment with, unless you have a spare Oracle license floating around.

Time to try it:
  C:\>ec2-run-instances ami-7acb2f13 -k gsg-keypair
  C:\>ec2-describe-instances i-??????

and login:

Use of this machine requires acceptance of
the following license agreements.
 1. Oracle Enterprise Linux

http://edelivery.oracle.com/EPD/LinuxLicense/get_form?ARU_LANG=US

 2. Oracle Technology Developer License Terms

http://www.oracle.com/technology/software/popup-license/standard-license.html

 Please enter the above URLs into your browser and review them.
To accept the agreements, enter 'y', otherwise enter 'n'.
Do you accept? [y/n]: y
Thank you.

You may now use this machine.
Welcome to Oracle Database on EC2!
This is the first time this EC2 instance has been started.

Please set the oracle operating system password.
	.
	.
Please specify the passwords for the following database administrative accounts:

SYS (Database Administrative Account) Password:
	.
	.

Now for the link to Apex on the new OracleXE instance:
  http://ec2-??-??-???-??.compute-1.amazonaws.com:8080/apex


Fig 3 – Oracle Apex running from an EC2 OracleXE instance

Looks like we have it.

Summary:
Oracle is the Big Daddy of spatial GIS. It is also the “Mother of all DBA complexity.” Running a spatial app with oracle in the background is not trivial, but it is getting easier. The EC2 OracleXE AMI makes starting an Oracle server instance a matter of minutes. Although lacking some of the capability of its free and open source competition, OracleXE can be useful for the garden variety web enabled spatial app. For the developer with lots of experience in Oracle, OracleXE provides a low cost entry onto the performance/price escalator.

Next on the agenda is adding SDO_GEOMETRY data along with some kind of real spatial rendering, which means in my case getting a tomcat server running with Geoserver on the same OracleXE instance. Alternatively it might be worth a try at installing the OracleXE .rpm on an AMI with a GIS stack already available. And, it will be useful to recompile ogr with oracle db support.

Of course the real mix and match challenge will be OracleXE on an EC2 (real soon now) Windows instance with Java, Tomcat, Geoserver serving a Google Map control coupled to Google Earth, OpenLayers, VirtualEarth. But really EC2 Windows will probably come preconfigured with the new MS SQL Server 2008 and all the promised geospatial goodies including Linq potential.

After just a short trip into the Ellison Empire, I must admit I still like the no frills PostgreSQL/PostGIS better.