PostgreSQL template tables and rake db:test:prepare

January 13, 2011

I recently overcame the final hurdle to getting rspec hooked up within this legacy project that I’m working on (legacy in the sense that it has no tests, and is in production, and works).

There were three levels of fail with respect to running rake db:test:prepare

Fail #1: Configuration

The database configuration file had the test environment looking at the same database as development. The first time I ran rake db:test:prepare it dropped my development database. Usually this isn’t such a huge disaster, but in this case we’re using a slightly sanitized version of the production database. It takes 45 minutes to scp it down from the backup server, and 4+ hours to load.

The solution, of course, was to change this:

test:
  adapter: postgresql
  database: legacyproject_development

to this:

test:
  adapter: postgresql
  database: legacyproject_test

Fail #2: Bug in Schema Dumper

This project happens to be using a homegrown plugin for PostGIS (because at the time we needed one there were no other available options). Since we’ve never needed to actually load the database from the schema, nobody had noticed that the schema definition of geometry tables were lacking the parameter for the SRID.

wrong number of arguments (4 for 5)

Fixing the custom schema dumper for the geometry tables fixed this issue.

Fail #3: Missing PostGIS Functions and Tables

The production and development databases had been set up manually to include the PostGIS goodies. rake db:test:prepare was now correctly creating the test database based on the schema, but was lacking everything PostGIS. Three options immediately came to mind:

  • write a task that manually loaded all the postGIS stuff and have it run prior to the rake db:test:prepare task (uhm, no thanks).
  • add the PostGIS stuff to the default postgresql template table, template1 (I’d rather not).
  • create a special PostGIS template table (yes, please).

So I did:

psql -d postgres -U postgres
CREATE DATABASE template_postgis WITH TEMPLATE=template1 ENCODING='UTF8';
\c template_postgis;
CREATE LANGUAGE plpgsql;
\i /opt/local/share/postgresql90/contrib/postgis-1.5/postgis.sql
\i /opt/local/share/postgresql90/contrib/postgis-1.5/spatial_ref_sys.sql
UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template_postgis';
GRANT ALL ON geometry_columns TO PUBLIC;
GRANT ALL ON spatial_ref_sys TO PUBLIC;  

And then all I needed to do define the database config so that it used this template instead of template1 when created the test database.

test:
  adapter: postgresql
  template: template_postgis
  database: legacyproject_test

Ah, right. Not so simple. It would seem that noone has needed this config options, so rails doesn’t recognize the template option.

So we submitted a patch to rails. Smallest patch in the history of rails probably, at less than 30 characters

#win

A (silly) PostgreSQL gotcha

January 07, 2011

As I was setting up my latest project in Rails 3 with PostgreSQL, Cucumber, RSpec and various other goodies, I got this error when running rake db:migrate:

PGError: ERROR: permission denied for relation schema_migrations
SELECT version FROM schema_migrations

I took a quick look in my config/database.yml

And saw the following ridiculousness:

development:
  adapter: postgresql
  encoding: unicode
  database: myproject_development
  host: localhost
  pool: 5
  username:
  password: productionpassword

Updating username to myproject and password to be blank did the trick.