How to update the data if the model definition has changed

[jakobkmar]

What is the recommended way of updating the data currently stored in the database if the definition of the model has changed (e.g. new fields (may be nullable) or just a new index over existing fields)?

I am asking this because if you for example just add a new index, the old database entries are not covered by this index and therefore not visible for some operations.

[Kasheen]

In his talk (here) @SalomonBrys mentioned that migrations will likely have to be manual.

I'm currently prototyping some code for a project using kodein-db and I've been wondering about this question also.

My only thought to it is perhaps, instead of updating your model class with new fields, to create a new class with a trailing version: for instance ModelDocument becomes ModelDocumentV2 which has extra fields. And then write a migration class which would db.find<ModelDocument>().all() and loop through them converting them to ModelDocumentV2 and writing those new documents back into the database, then delete all of the original type from the database.

The database would obviously have an empty "table" for the old type, but I can't see that as much of an issue. Iterating a large collection could be costly in terms of performance though.

[jakobkmar]

Yes I have the thought about that approach too.

In my case though, I have just added a new index (the values of the data class itself have not been changed at all) and it took me 1 hour to realize that old entries were not covered by that index.

Maybe an updateIndexes() function would be a good feature to add to KodeinDB, because KodeinDB encourages you to query by index, but of course you only add the index to the model if you need it in the code - so it is very likely that developers add indexes to an existing model later on.

[SalomonBrys]

Migration is the next big subject we need to be working on.
There are several cases we need to support:

• When the model changes (fields are added and/or removed)
• When the metadata changes:
  • Indexes are added and/or removed
  • IDs type or value is changed
• When classes are renamed and/or moved to different packages
• When classes are "split" into 2 sub-classes (Person was a class, becomes an interface, and 2 new Child and Parent are introduced), and inversely when multiple classes are merged into one.

I am currently working on a major shift to the API to allow for type-safe operations & queries, which changes the way models are defined.
Once this API is introduced (and the previous one properly deprecated), and the migration guide to this new API is good, then we'll have the basis for a proper migration API.