Database Migrations Using DBXExpress Metadata

by John 12/19/2007 5:56:00 AM

I recently spent some time looking at Ruby On Rails. One of the many things about Rails that caught my interest was the way that schema changes to databases are performed using Ruby. I think it was this post by Steve Shaughnessy about the new dbExpress 4.0 MetaData where it occurred to me perhaps I could use the DBExpress MetaData to do my something similar in Delphi.

I had several goals in mind:-
  • Produce a repeatable series of steps to upgrade database schemas.
  • Stay away from having to remember SQL syntax.
  • Automate the upgrade process
  • Target multiple db servers

Before presenting my solution, I would like to thank Leonel Togniolli, CodeGear R&D for helping me with how to go about calling the DBExpress MetaData API.

Project Layout

I have bundled everything together in a projectgroup which contains five projects.
Moshine.Migration.Framework.dll - Contains most of the core code
Moshine.Migration.Framework.Tests.dll - Contains some unit tests
Moshine.Migration.Test.dll - is an example of a database migration
Moshine.MigrationRunner - Is a VCL.Net application to execute migrations
Moshine.MSBuildMigration.dll - Contains a collection of MSBuild tasks that can be used from the IDE or standalone from the RAD Studio Command Line


A database schema generation or migration is composed of multiple steps. A step is the equivalent to a Delphi class.  A step can perform such actions as creating tables, adding columns, adding indexes, adding foreign keys, and executing sql. Sql could be used to add stored procedures and insert domain data into tables.

So if you take a look at Moshine.MigrationTest.dll, this contains seven units which each contain a Delphi Class. So after running the migration the database would be at version 7. For example this is the declaration for the first step

    procedure Up();override;
    procedure Down(); override;

Migrations can go backwards and forwards so I have Up and Down methods. The idea is that to create a step you create a new class derived from TMigration and decorate with an attribute which describes which step this is. Migrations must start at 1 and be sequential.

If you take a close look at the Up method in TBookMarksFirstMigration, its creating a table called Bookmarks using the DBXExpress metadata API. After table creation the column ID is set as the primary key.

The corresponding Down method for this class would drop the table.


All migration steps must be derived from TMigration. This base class is in Moshine.Migration.Framework in Moshine.Migration.Framework.MigrationBase.pas. I have provided a number of protected methods that Migration developers can call

procedure AddColumn(tableName:string;columnName:string;dataType:integer);
procedure RemoveColumn(tableName,columnName:string); procedure AddIndex(tableName, columnName: string; indexName: String);
procedure RemoveIndex(tableName, columnName: string; indexName: String);
procedure AddPrimaryKey(columnName,TableName:string); procedure AddForeignKey(columnName,tableName,primaryTableName,primaryColumn:string);
procedure RemoveForeignKey(columnName,tableName,primaryTableName,primaryColumn:string);

There are also 2 properties

property Provider:TAdoMetaDataProvider read FProvider write FProvider;
property Connection:DbConnection read FConnection write FConnection;

Connection is a database connection but Provider is the most important. Its a TAdoMetaDataProvider object and is essentially the DBXExpress metadata. The protected methods I supply are basically calling methods in the metadata API. Most of the time I would expect that you would call the helper methods but you can use use the Provider or Connection to execute raw sql.

MSBuild Setup

I have been using MSBuild target files to define targets that call the various tasks. The Moshine.MigrationTest.dll project contains one such targets file called TesTargets.Target

The MSBuild tasks are driven by two properties, a connection and assembly that contains your migration.

Moshine.MigrationTest is a sample migration. The project contains an app.config

This contains your connection specifications. For example mine contains

<?xml version="1.0" encoding="utf-8" ?>
        <add name="SomeConnectionString" connectionString="database=C:\develop\moshine\Migration\Moshine.MigrationRunner\App_Data\migration.jds;user=SYSDBA;password=masterkey;host=localhost;protocol=TCP;create=true" providerName="Borland.Data.BlackfishSQL.RemoteClient"/>

a single Blackfish connection called "SomeConnectionString"

The other item that must be added to a project are the MSBuild tasks. I have added TestTarget.targets to Moshine.MigrationTests.dll

<Project xmlns="">

    <UsingTask TaskName="Moshine.Migration.MSBuild.TMigrateDownTask"
            AssemblyFile="C:\develop\moshine\Migration\Moshine.MSBuildMigration\bin\Moshine.MSBuildMigration.dll" />
    <UsingTask TaskName="Moshine.Migration.MSBuild.TMigrateUpTask"
            AssemblyFile="C:\develop\moshine\Migration\Moshine.MSBuildMigration\bin\Moshine.MSBuildMigration.dll" />
    <UsingTask TaskName="Moshine.Migration.MSBuild.TMigrationTask"
            AssemblyFile="C:\develop\moshine\Migration\Moshine.MSBuildMigration\bin\Moshine.MSBuildMigration.dll" />
    <UsingTask TaskName="Moshine.Migration.MSBuild.TMigrationSchemaVersionTask"
            AssemblyFile="C:\develop\moshine\Migration\Moshine.MSBuildMigration\bin\Moshine.MSBuildMigration.dll" />


    <Target Name="Migration">
        <TMigrationTask  MigrationAssembly="$(MigrationAssembly)" ConnectionName="$(ConnectionName)" />

    <Target Name="Up">
        <TMigrateUpTask  MigrationAssembly="$(MigrationAssembly)" ConnectionName="$(ConnectionName)" />

    <Target Name="Down">
        <TMigrateDownTask  MigrationAssembly="$(MigrationAssembly)" ConnectionName="$(ConnectionName)" />

    <Target Name="SchemaVersion">
        <TMigrationSchemaVersionTask  MigrationAssembly="$(MigrationAssembly)" ConnectionName="$(ConnectionName)" />


The contents is above. I have setup two properties and four targets. The two properties define the Connection to use and the assembly that contains the migrations that will be run by the MSBuild targets.

You should now be able to open a RAD Studio Command Prompt. It you change to the Moshine.MSBuildMigration directory you add run the migrations. For example

msbuild TestTarget.targets -t:up
msbuild TestTarget.targets -t:down
msbuild TestTarget.targets -t:SchemaVersion
msbuild TestTarget.targets -t:Migration

You can also run the migrations from the IDE project menu as well. In the project manager select TestTarget.targets in the Moshine.MigrationTest project, right click and select the targets menu item. Listed as submenu items of the targets menu will be the same four targets as above, as can be seen below.

The SchemaVersion targets displays the database schema version.
Migration executes all steps defined in the assembly.
Up and down step up and down the migration.

So say I have my database at database schema version 4 and I execute the migration target. Steps 5, 6 and 7 will be performed.

Running a Migration

The Migration Framework looks for a database table called Migration_Schema to determine the database version and then looks at the steps provided in the metadata assembly to work out the steps to run.

If you issue the Migration target and the number of steps doesn't match the schema version the appropriate steps will be run.

You can also use the Up and down targets step up and down the migrations. MigrationSchemaVersion outputs the database schema version.


I have also provide a VCL.Net application which duplicates the MSBuild functionality. This is configured with app.config. An example of which can be seen below.


To be able to build the projectgroup you will need the December 2007 Rad Studio Update installed.

The Moshine.MSBuildMigration.dll assembly relies on Moshine.Migration.Framework.dll, so make sure that both assemblies are located wherever you set the using task.


<UsingTask TaskName="Moshine.Migration.MSBuild.TMigrateDownTask"
            AssemblyFile="C:\develop\moshine\Migration\Moshine.MSBuildMigration\bin\Moshine.MSBuildMigration.dll" />

In theory you should be able to perform migrations on any database that CodeGear provides MetaData for. I have done some testing with AdoDbx against MSSQL, Interbase and MySql. Most of the time I have been using the Blackfish native providers.

If this interests you, I have placed the code in codecentral at this location. Absolutely any feedback is welcome.

Currently rated 5.0 by 1 people

  • Currently 5/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5


migration | dbx4 | Database | CodeGear | Delphi | Blackfish | VCL.Net | msbuild

Powered by BlogEngine.NET
Theme by Mads Kristensen

About the author

Name of author John Moshakis
I'm a software developer living in Toronto..

E-mail me Send mail


<<  August 2022  >>

View posts in large calendar


    Recent posts

    Recent comments



    Don't show


      The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

      © Copyright 2022

      Sign in