Data migration: the line everyone forgets to quote

The proposal covers everything: modules, integrations, training, warranty. Migration appears as a single line - "data will be transferred".

Halfway through the project it emerges that the legacy database holds five years of records, product names appear in three different spellings, units of measure are mixed, and half the fields are empty.

Whose work is that now? On which side, and who pays for it?

What migrates

The practical rule: what the system needs to operate migrates, everything else stays in an archive.

The last two cause the most debate. "We need the full history" is a natural request, but in practice it is rarely used.

The practical answer: the legacy system or files are kept as an archive and opened when needed. Only balances and open transactions go into the new system. That cuts the migration cost several times over.

Cleaning: the longest part

This is the organisational rather than technical part of migration, and it usually stays with the client.

What gets cleaned:

• Duplicate records - one product under two or three names
• Naming conventions - "Cement M400" and "cement m400" are the same thing
• Units of measure - received in tonnes, issued in kilograms
• Empty fields - no code, no price, no category
• Inactive records - products not sold in five years
• Bad contacts - old numbers and dead email addresses

A system cannot do this work, because only you know which two records mean the same product.

Scale: a catalogue of 500 lines usually takes two or three days. Five thousand lines takes two weeks.

Why a trial migration matters

This stage is often skipped, and it prevents more problems than any other.

A trial migration moves one month of data, or one department's data, and checks the result. It runs at the start of the project.

What it establishes:

Data quality. How many records are problematic and how long cleaning will take.

Technical compatibility. How data leaves the legacy system - is there an API, does the export work, is the format readable.

Volume. Record counts are often larger than expected.

Relationships. How product codes, counterparties and documents link to each other.

After that, the migration timeline and cost become a calculation rather than an assumption.

How the result is verified

Once the transfer is done the result has to be confirmed. Four checks are enough:

The last finds the most errors. A negative balance or a zero-priced product usually exposes a problem that existed in the legacy system.

The migration sequence

The date in step four matters. A migration mid-month splits monthly reporting across two systems. The start of a month or quarter is far easier.

Who pays, and how much

This is settled at proposal stage, not mid-project.

The second row matters most. Cleaning stays with the client, because only they know which data is correct. Fail to say so up front and the project stalls exactly there.

Cost: migration is typically ten to twenty per cent of project scope. Clean data lands at ten; messy data at twenty or more.

Questions to ask a vendor

• Does migration appear as its own line, and how many hours is it estimated at
• Is a trial migration included and when does it run
• Which side is responsible for cleaning the data
• What migrates and what stays archived - is there a list
• How is data extracted from the legacy system - API, export or database
• How is the result verified and who signs it off

The third question matters most, and it usually goes unanswered.

What happens to the legacy system

After migration the old system is not switched off immediately.

The practical approach: it stays available for at least three months in read-only mode. Someone will need an old document, and in that time everyone becomes confident the new system works.

After three months the data is kept as an archive copy - a file or a database backup. Licence payments stop.

One thing to record: where the archive copy is kept and who can open it. Two years later, when that data is needed, the person who knows how to retrieve it may no longer be there.

In summary

Migration is not a technical task. It is a decision about data quality - and it belongs as its own line in the proposal.

Practical steps:

1. Insist that migration appears as a separate line in the proposal
2. Put a trial migration into the first month of the project
3. Draw up the list in advance: what migrates and what stays archived
4. Agree in writing which side does the cleaning
5. Set the transfer date at the start of a month or quarter
6. Keep the legacy system read-only for three months