Tuesday, April 26, 2011

Evolutionary Database Design

Howdy! Techy post, for a change. I recently read this great article from Martin Fowler which explains quite clearly and simply the benefits and patterns that come with embracing change in software development from start. The article speaks specifically about how to prepare both practically and mentally to change your database a lot, and often. It is funny how most of us reject the idea of changing design or database decisions in the middle of projects. Assuming they will change and preparing for it can be a quite stress releasing experience (especially knowing how requirements or their interpretations almost ALWAYS change, and very fast).

Here are some key ideas which I personally saw enlightening:

  • The design of the system has to evolve through the various iterations of the software. 
  • Look at design as an on-going process that is interleaved with construction, testing, and even delivery.
  • Agile processes run complete life cycles in each iteration, completing the iteration with working, tested, integrated code for a small subset of the requirements of the final product.
  • Most people consider that database design is something that absolutely needs up-front planning.
  • Changing a schema after deployment results in painful data migration problems.
  • The developer knows what new functionality is needed, and the DBA has a global view of the data in the application.
  • Everybody gets their own database instance: people learn by trying things out. It's important for each developer to have their own sandbox where they can experiment, and not have their changes affect anyone else.
  • As a rule of thumb each developer should integrate once a day.
  • The pain of integration increases exponentially with the size of the integration.
  • A database consists of schema and test data. By having sample data, we are forced to ensure that any schema changes also handle sample data. You should try to introduce real data from the very first iteration of your project.
  • Every database refactoring is automated by writing it in the form of SQL (these changes are never applied manually).  We keep hold of these script files to produce a complete change log of all the alterations done to the database.
  • Once we do a release,  we apply the full change log of database refactorings since the previous release. 
  • As well as automating the forward changes, you can consider automating reverse changes for each refactoring. If you do this you'll be able to back out changes to a database in the same automated way. 
  • Have a clear database access layer to show where the database is being used and how.
  • Creating a new database lineage is much the same as branching the source code on the application.
  • Whenever you run into repetitive tasks in software development you are ideally placed to automate them.

You can refer to the original article for more info, if you are a software developer I really recommend you to read it! You can tell me in the comments what you think about it.