Over the last 30 years we have carried out thousands of data migrations accumulating a wealth of experience which we now apply to all data migration projects.
Below is a list of frequently asked questions to help alleviate any fears you may have when considering migrating to a new system.
Question: There are many library systems in the marketplace, where do I start?
The starting point in any analysis of a new system, begins with a determination of needs.
Question: What is data migration? How is it performed?
Most libraries and archives today have data in a database of some form or another.
When the Soutron system is selected to manage the library or archive, we undertake to migrate the data into Soutron. It is part of our System Implementation and is carried out within a formal project.
We ask the client organisation to do this extraction for a number of reasons. We do not have a licence to use their existing software and do not wish to contravene your licence terms. Doing it this way ensures that the data that is is handed over to us as a complete set of data, avoiding any issues of possible missing data.
The client knows their own data and as such can check and confirm that the data being provided is complete and accurate. The data can be in a variety of formats. Much depends on how data can be extracted from the existing software and database. We prefer XML but often receive data in MARC21 format or CSV. It is possible that there are multiple data sets if part of the project is bring these together to establish one database
The most important part of any new system implementation is migrating the data and ensuring that the data structures and the way in which the data is held and presented in the database is ideally suited for the library and its users.
Question: Can I use all of the data from my current archive / library management system?
Normally yes! This is certainly our goal providing that the data can be exported from the original legacy system and is structured data. The Soutron system accommodates different types of data and is flexible to accommodate all types of data in the one system – both library and archive data.
Question: Can you import and host all of my record attachments, images, PDFs?
Yes, but much of the loading of files such as PDFs, Office Documents and images depends on the quality of the data files and the way in which files are named and formatted. Image files differ considerably from document files and can have a variety of additional metadata and may involve resizing and clean up before they are deemed good enough to load into the database.
Question: How long does a typical migration take?
The timescale for the migration varies depending on a number of factors. Much of the time is dependent on the number of data loads needed and the extent of the data checking that is needed.
If the original data is particularly poor in quality or structure or both, it may take considerable time to reach a point where it is deemed suitable for live operation. The time taken to reach this stage can also depend on library staff availability, since the burden falls on them to check data and feedback to the project team.
No one else is in a position to determine how data should be structured and if it is ready. Some projects take only one data load, if the data is straightforward. Others can take 4 or 5 months especially where data is not provided at the outset but appears late in the project or where the data provided is very poorly structured and needs multiple loads.
Question: Can I run both systems (the old one and the new one) until I am happy to make the switchover?
Yes! We advise that the original system continues to be used and is operational until the data migration is signed off as being correct. This means that there is no disruption to services and only when the data is ready and everyone is happy is a final copy of the data taken from the original system and converted. (Remember that the conversion routine is automated using XML schemas as transforms so that it is prepared according to the needs explored in the test loads. This makes it possible to perform repeated loads and checks on the data without manual effort to make changes in the data.
Once the new system is deemed acceptable and everyone is happy using the new system, the old one can be decommissioned. We recommend that the old system is kept in place for a period of a few weeks just to check back on any item in the early weeks of use of the new system. Some clients, knowing the data loads are good and having conducted repeated tests during the conversion turn off the old system at the point of Go Live. Such is the confidence that the migration and conversion process provides.
Question: Will I lose any data in the migration process?
Data is never “lost” however if the data provided in electronic format is unstructured or in a corrupt format it is unlikely to be correctly interpreted or understood programmatically and as such it may be impossible to load into a field structure in the database.
In over 30 years of business, only on a very few occasions have we faced a situation where data could not be converted. We take great care to minimise data loss.
Often we are able to recover, clean and tidy up data, increase accuracy and consistency and present it far better than when it came out of the original host system.
Question: What happens on go-live day?
The URL for the Live system is activated and the database is loaded with the latest copy of data from the test system. The user file has been populated at this point and tested for Single Sign On. From this point on the library staff can point their web browser to the new URL and perform the tasks that they have been trained to do using the new software. The Search Portal(s) will be active and end users will now link to the URL for the Search Portal.
Preparation and attention to detail makes the Go Live event a natural and straightforward activity, one that everyone is prepared for in terms of training in how to use the application and in having the data loaded and available via the Search Portal.
The data from the test system that has been used throughout the implementation process will be loaded into the final database that has been configured and made ready in advance. Remember this will have been tested beforehand with the data, the only difference is that the data loaded is the very latest set of data to have been extracted from the original system which has been frozen and is only used for Search purposes. No new data will have been added since the last extraction of data.
Question: How do I explain the process to my managers?
The implementation of a new system is the same as any IT project that is undertaken by your organisation.
The project follows a proven methodology with a dedicated project manager in charge who communicates all tasks and activities via a project plan and Gannt chart to all project members and meets regularly with the team assigned to implement the system. A project coordinator from the client organisation is assigned to the project to provide decision making and assist the project manager. This is preferably someone with detailed knowledge of the needs of the organisation in respect of the library or archive.
A set of milestones are agreed to measure progress and a timeline established for the project to be implemented. There will be a need to consult with the internal IT team about matters relating to security of data and systems, set up of users for Single Sign On, and possibly any integration with the web site or other systems.
The Finance and Procurement teams need to sign off against any budgets and be involved in the purchase and contractual aspects of the project.
We hope the above helps to answer your questions regarding the whole process of data migration when moving data between systems.
At first it may all appear to be a mission impossible, a task not for the faint of heart. However, with our expertise and guidance, we can assure you that the process will be carried out professionally, securely and to the very highest standards in the industry today.