Skip to content

Updating GDAL PostGIS Raster driver

August 25, 2012

NOTE: Please, update your feeds. This blog has been moved to http://www.libregis.org

Those past days, I’ve been working on the GDAL PostGIS Raster driver. I commited a version with relevant changes some days ago. It works with irregular blocking (tiles of different size), different pixel sizes in tiles and is pretty much faster than the previous version. Currently, I’m debugging to solve some errors, but I plan to have a stable version for next week. Of course, any feedback is welcome.

So, you can download the last GDAL nightly snapshot or checkout the code from svn repository. Also, I keep a github repository just for the driver’s code. Feel free to clone it, comment, collaborate or just say “Hi”

Apart from that, as this blog is now part of the OSGeo Planet, I’m going to update it way more frequently. For the time being, I can commit with one post per month. Probably more.

I’m moving to www.libregis.org

November 26, 2011

I’ve migrated all the blog content to a new domain:

http://www.libregis.org

So, you can update your markers if needed. I will keep this blog alive for some time.

Web-based PostGIS geometry/raster viewers

August 27, 2011

If you want to test PostGIS Raster (included in the next PostGIS 2.0) without installing any software in your computer, take a look at these web-based PostGIS geometry/raster viewers:

Thanks to Regina Obe for this, and all the PostGIS team for the excellent work. I’m working in the GDAL PostGIS Raster driver. So, I hope to have some good news soon.

BTW, if you’re attending FOSS4G 2011 in Denver, don’t miss the talk PostGIS 2.0, the new stuff

Free and ready to use anywhere GIS Virtual Machine!

April 7, 2011

Ricardo Pinho has announced on postgis-users list the availabilty of a VMWare-powered GIS Virtual Machine, based on Ubuntu 10.10 and with lots of great Open Source GIS stuff. There are 3 versions of this machine:

  • GISVM base: A VM with Ubuntu 10.10 Desktop and ready-to-execute scripts, that allow you to install the GIS software you need. The lighter version.
  • GISVM desktop: This VM includes PostgreSQL, PostGIS, GeoServer, Mapserver, FWTools, QGIS, GRASS, gvSIG, uDIG, Kosmo and OpenJump installed. Based on Ubuntu 10.10 Desktop too.
  • GISVM server: The sever version of the VM includes Linux, Apache, MySQL, Php,Tomcat,PostgreSQL, PostGIS, GeoServer, Mapserver, deegree and Geonetwork. Based on Ubuntu 10.10 server.

You can download the machines from here.

Collaborate with an Open Source project and win 5000$

March 31, 2011

I participated in Google Summer of Code 2009, and that was the way I started working on PostGIS and GDAL. Now, I’ve been selected as mentor in Google Summer of Code 2011. In this page you can see my project and other related ones, with Pierre Racine as proposed mentor.

Do you want to collaborate in an Open Source project? Contact me by e-mail or leave a comment here. This summer could be a great summer of code!

How to install and configure PostGIS Raster on Windows

March 10, 2011

A lot of people ask me about how to use PostGIS Raster and the GDAL PostGIS Raster driver in Windows systems. And most of them are, comprehensibly, using the last stable releases of both libraries (PostGIS and GDAL). The problem is PostGIS Raster and GDAL driver are being developed in these moments. There’s no stable release, so far.

PostGIS Raster was included as official part of PostGIS in October 2010. The old version, formerly named WKT Raster has been abandoned. So, if you want to use PostGIS Raster, with new functionalities and many bugs fixed, you’ll have to use the latest PostGIS development snapshot. The use of the old WKT Raster extension is strongly unrecommended, and there’s no support for it.

About GDAL, there was an important change in the driver on October 20st, 2010. A new version was committed, and this is the only version that works with PostGIS Raster. This new driver version was included in the release 1.8.0, currently the stable release. So, the use of an older release to connect with PostGIS Raster is also unrecommended.

With these facts in mind, I understand the Windows users who want to use PostGIS Raster and the GDAL driver are a bit unprotected. I’m not a Windows user, but I consider it my fault. For this reason, I’ve decided to create this howto. I’m going to install and configure PostGIS Raster and GDAL in Windows. And fortunately, you don’t need to compile anything. Let’s go for it:

Enviroment: Windows 7 32 bits

Step 1: Installing Python and Numpy

PostGIS Raster loader uses GDAL Python bindings and numpy. So, we’ll first install Python and numpy. Several options here:

  • Python 2.5, unofficial numpy 1.5.1 (download numpy-1.5.1.win32-py2.5.‌exe from here)
  • Python 2.6, unofficial numpy 1.5.1 (download numpy-1.5.1.win32-py2.6.‌exe from here)
  • Python 2.7, unofficial numpy 1.5.1 (download numpy-1.5.1.win32-py2.7.‌exe from here)

I’m using:

  • Python 2.6.6
  • Numpy 1.5.1 (numpy-1.5.1.win32-py2.6.‌exe)

Step 2: Installing GDAL

I think the easiest way to have a complete GDAL installation (library + binaries + Python bindings) working on Windows is using the GDAL binaries mantained by Tamas Szekeres, here. Last GDAL versions from trunk are compiled daily with MSVC 2003, MSVC 2005 and MSVC 2008. Pick one. For example, the version compiled with MSVC 2003, here.

The  faster way now would be to download the generic installer for the GDAL core components, and choose the complete installation

GDAL 1.8.0 core installer for Windows

GDAL 1.8.0 core installer for Windows

After the installation, you will need to add C:\Program Files\GDAL\ to the PATH enviroment var. And create this new var:

GDAL_DATA=C:\Program Files\GDAL\gdal-data

UPDATE 2011-10-07: About the GDAL Python bindings

After installing GDAL core components, you must install the Python bindings separately. You can use this installer (for Python 2.7 and GDAL 1.8.0), or grab one of the executables existent at http://www.lfd.uci.edu/~gohlke/pythonlibs/

Depending on your Python installation, you can choose GDAL-1.8.1.win32-py2.6.‌exe, GDAL-1.8.1.win32-py2.7.‌exe, GDAL-1.8.1.win32-py3.1.‌exe or GDAL-1.8.1.win32-py3.2.‌exe

UPDATE 2011-03-25: Alicia Duarte, from IDESF, told me she was to additionally set the next enviroment vars to make it work:

GDAL_DRIVER_PATH=C:\Program Files\GDAL\gdalplugins
PROJ_LIB=C:\Program Files\GDAL\projlib

WARNING: This directory (GDAL Python bindings location) may not exist in your GDAL installation. In my case, the bindings are in C:\Python27\Lib\site-packages\osgeo, and I didn’t need to define this variable.

PYTHONPATH=C:\Program Files\GDAL\python

UPDATE 2011-07-19:  I think I introduced some confusion in the comments. I apologize for that. Several people are having problems with GDAL Python bindings. I’ll  try to make it clearer:

  • On one hand, there’s a sure thing: you need GDAL library + GDAL Python bindings installed in your system. There’s an easy way of doing it, by executing the generic installer: gdal-18-1310-core.msi. This installer includes the Python bindings, and they’re installed at C:\Program Files\GDAL\python directory by default. This should be enough.
  • If you previously have a Python interpreter and a GDAL version installed on your system (check it from a console, executing gdalinfo –formats), you may want to simply install the proper GDAL Python bindings. Let’s assume your GDAL version is 1.8 (lower versions won’t work with PostGIS Raster), and your Python version is 2.7. Then, you simply need to install GDAL-1.8.0.win32-py2.7.msi. This installer will put GDAL Python bindings at C:\Python27\Lib\site-packages by default. This is a different location than above, but at the end, you should have a working version of GDAL Python bindings on your system.
  • Finally,  the PostGIS Windows experimental binaries with PostGIS Raster support (step 4) are packed with their own version of GDAL library, used only by the raster2pgsql loader.

GDAL installed. Now, let’s go with step 3

Step 3: Installing PostgreSQL

I recommend Postgresql 8.4.x or higher. You can download the 1-click-installer from here, and install it in the common Windows’ way.

During the installation, you’ll be asked to install some nice additional stuff using the Application Stack Builder, like PostGIS 1.5.x or PostGIS 1.4.x. Don’t install anything. The PostGIS versions that can be installed doesn’t include PostGIS Raster. And you can’t install it separately, because, remember, now is part of PostGIS. We’re going to install last PostGIS version by hand. Don’t be afraid, it’ll be easy.

Step 4: Installing PostGIS

We’re going to use the PostGIS Windows experimental binaries. We need the binaries compiled for PostgreSQL 8.4, these ones

Once uncompressed, you can see this

postgis20 windows

Contents of binary package PostGIS 2.0 for Windows

Following the instructions of README.txt file, let’s edit makepostgisdb.bat to adapt the vars to our enviroment. We may want to change these vars:

set PGPORT=5432
set PGHOST=localhost
set PGUSER=postgres
set PGPASSWORD=password_of_postgres_user
set THEDB=template_postgis20
set PGBIN=C:\Program Files\PostgreSQL\8.4\bin\
set PGLIB=C:\Program Files\PostgreSQL\8.4\lib\

After that, we make a backup of our PostgreSQL bin and lib directories (PGBIN and PGLIB). Just in case.

We now execute makepostgisdb.bat. Take into account you may want to execute the script as administrator, in case your Windows user doesn’t have needed privileges (it was my case).

UPDATE 2011-06-08: For all Windows XP Professional SP2 users:

when you run makepostgisdb.bat from the binaries of POSTGIS SVN2.0, please rename the libiconv-2.dll in bin folder of Postgresql/8.4/bin folder to any other name. Now run the makepostgisdb.bat file. You will not get any error of “operation aborted …. transaction block” (Thanks to C Patnaik)

Once executed, we have PostGIS 2.0 + PostGIS Raster installed and configured. A new database called template_postgis20 is created, enabled with PostGIS and PostGIS Raster. Easy! Let’s test it with the next query:

select postgis_full_version(), postgis_raster_lib_build_date(), postgis_raster_lib_version()

And the result:

PostGIS version installed

PostGIS version installed

Of course, you can create your own database using template_postgis20 as template. From the command line:

createdb -U postgres -T template_postgis20 your_database

Or from pgAdmin III:

Creating new database from template in pgAdmin III

Creating new database from template in pgAdmin III

If you want to use the SHP2PGSQL GUI, you must copy the postguisgui folder (inside bin folder in the downloaded PostGIS experimental binaries file) into pgAdmin III folder (mine is C:\Program Files\pgAdmin III\1.12) and add the next lines to plugins.ini file, inside pgAdmin III folder too:

;
;PostGIS shp2pgsql-gui (Windows):
;
Title=PostGIS Shapefile and DBF loader
Command="$$PGBINDIR\postgisgui\shp2pgsql-gui.exe" -h "$$HOSTNAME" -p $$PORT -U "$$USERNAME" -d "$$DATABASE" -W "$$PASSWORD"
Description=Open a PostGIS ESRI Shapefile or Plain dbf loader console to the current database.
KeyFile=$$PGBINDIR\postgisgui\shp2pgsql-gui.exe
Platform=windows
ServerType=postgresql
Database=Yes
SetPassword=Yes

After that, you can use the GUI inside pgAdmin III:

SHP2PGSQL GUI installed under pgAdmin III

SHP2PGSQL GUI installed under pgAdmin III

Now you can load raster data into your database in 2 steps. In the first step, you will generate a SQL file. Open a command line and execute this:

python C:\Program Files\Postgresql\8.4\bin\raster2pgsql.py -r image.tif -t tablename -o image.sql

As result of this step, a SQL file named image.sql will be created in the directory where you executed the Python line above.

In the second step, you’ll need to load the SQL file in your_database, by using your favourite PostgreSQL client. For example, pgAdmin. If your use the psql command line client, the line to execute is:

psql -h localhost -U postgres -d your_database -f image.sql

If you want to translate your raster data from the database to image files at disk again, execute gdal_translate.

gdal_translate PG:"dbname=your_database user=postgres table=your_table mode=2" your_file.tif

Basically, this is all what you need to work with PostGIS Raster in Windows. Anyway, take into account the next additional notes:

Additional notes:

In step 2, I installed the whole GDAL library, including binaries and Python bindings. In step 4, I installed PostGIS 2.0 with all its pre-requisites. One of them is GDAL. The optimal way would be to use the existent GDAL library, but we’re working with pre-compiled binaries and DLLs. So, we have 2 copies of GDAL library:

  • C:\Program Files\GDAL\gdal18.dll: Installed with generic installer for GDAL, in step 2. Used by gdal_translate, when reading data from database, and in any operation involving GDAL, except the raster loading with raster2pgsql.py script
  • C:\Program Files\PostgreSQL\8.4\bin\libgdal.dll: Installed with PostGIS Windows experimental binaries, in step 4. Used by raster2pgsql.py script, to load raster data into PostgreSQL.

Both library files were compiled from GDAL 1.8.0 from trunk. But they will be slightly different, for sure. This is a suboptimal solution, of course, but it shouldn’t cause problems, because libgdal.dll is only used when loading raster data into the database. The rest of operations involving GDAL will be executed linking with gdal18.dll

If you want to translate the raster data to any image format different from TIFF, you can use the -of option in gdal_translate. For example:

gdal_translate -of "PNG" PG:"dbname=your_database user=your_user password=your_password table=your_table mode=2" your_file.png

To get available raster formats:

gdalinfo --formats

UPDATE 2011-06-11: I changed the gdal_translate line. I’m using the ‘postgres’ user and the ‘trust’ method to authenticate this user (so, no password is needed), but it’s only a simplification. In general, you should use a different user and a password-enabled authentication method, like md5. Thanks to C.Patnaik again, because his comment remainded me this issue

Conclussions:

It’s possible to work with the last version of PostGIS Raster and GDAL PostGIS Raster driver in Windows, but assuming that:

  • You’re using experimental build. Not recommended for production enviroments.
  • The previous versions of PostGIS Raster (WKT Raster) and GDAL PostGIS Raster driver (before GDAL 1.8.0) are discontinued. Don’t use them.
  • It’s a suboptimal solution, because you install 2 copies of GDAL library. But I think this is the best available solution right now.

Do you have problems, something to add, modify or correct? Let me know it in the comments

 

PostGIS Raster meets Desktop

December 10, 2010

As I said in my previous post, there are people working to bring PostGIS Raster to desktop. Specifically:

Many thanks! These initiatives help to continue working

Follow

Get every new post delivered to your Inbox.