Migrating data to SharePoint Lists (MOSS 2007 / WSS 3.0)

When migrating data from non-SharePoint sources / legacy applications, it is important to design, organize and manage your list items and properties carefully so that there is minimal rework on your lists and properties without having to redesign the whole content framework. We will discuss below some of the common list management issues in SharePoint both during migration as well as management of content post migration.

1. Import list data and maintain or change folder structure when migrating data from various non-SharePoint sources or legacy applications to MOSS 2007 / WSS 3.0

As stated in the previous blog article related to document migration to SharePoint, most companies will want to retain the same structures for folders and contents to maintain operational consistency as well as business continuity. It will be easier to maintain the same folder structure in the newly setup SharePoint list without changing the user experience in handling folders and related information / content. It makes the navigation intuitive, eases migration process, minimizes user training and improves operational efficiency. In a few business situations, it will be a lot better to place the contents in a new structure to improve efficiency or due to changes in the business. In both cases, it is important to identify a solution that can organize content depending on the business or functional or end-user requirements.

2. Import information from other systems that are in different formats to specific SharePoint 2007 list types such as tasks, contacts, issue tracking, wiki pages etc.

In some cases, it will be easy to map existing content to the default list types provided by SharePoint 2007, such as project tasks, announcements, contacts, pictures, discussions etc. In the case of proprietary formats or contents residing in legacy applications or relatively unknown CMS, content migration could pose a problem when trying to map the existing assets to the correct SharePoint list type as we need to. In such cases, it is important to store the content in a neutral format with the ability to cleanse the information before migrating them to SharePoint and map the information to respective list types meaningfully.

3. Selectively update metadata fields with values that have changed over a period of time or update missing fields that got missed out during the normal upload process.

When the SharePoint system is in full use, it is likely that the users may miss out some critical piece of information / data when updating / uploading information in SharePoint or the business needs may have changed over time, thereby forcing the content owner to fill-in the missing information or fields in SharePoint.

4. Import metadata values and file attachments to SharePoint 2007 lists

Existing contents in non-SharePoint sources could be residing in various formats and file locations. When consolidating all the existing information assets, it will be vital to pull them from various data sources and associate all documents from different file server / network share locations. It is also important to import metadata and file attachments from several sources and append them to existing default / custom lists in SharePoint.

5. Selective migration of information based on user-defined conditions that can facilitate repetitive import of information from a single source file or multiple source files.

In a few instances, it may vital to append metadata records as different versions in the SharePoint list to track version history (along with file attachments) or may be required to replace the old metadata records with new ones and retain the latest values, depending on certain pre-defined conditions. The metadata records may be available in a single source metadata file or multiple metadata files when they exported from the non-SharePoint source.

6. Mass document migration application to work off your desktop instead of running right on the SharePoint servers directly.

If you are using third-party tools for migration, a solution that can run either on the desktop and or on the server will be ideal. Take the case of incremental migrations. The SharePoint server could already be in production mode while several different SharePoint lists could still be under migration. Server performance will be compromised if the migrating application is going to run on the server. It is better if the application runs on a desktop performing actions like data cleansing, processing etc. and just do a final bulk upload into SharePoint. In real life, it will be time-consuming task to implement business processes to handle each situation differently.

7. Migrate to your new SharePoint repository while retaining the existing taxonomy or migrate to a new taxonomy.

The metadata need to be presented to SharePoint during migration in a certain manner that makes the search and retrieval more powerful and elegant. You may migrate / propagate metadata fields from the external metadata file / database. Mapping and tagging list metadata and adding additional metadata / properties and documents in SharePoint, especially for a well structured taxonomy can be a laborious process.

8. Retain and carry-forward the Created Date and Last Modified file attributes to maintain business continuity for users and minimize user training when collaborating in the new SharePoint environment.

There are plenty of business reasons to retain the same Date field values once the metadata has been migrated to SharePoint. Unfortunately, SharePoint falls short in this aspect and you need third-party tools or some in-house programming to carry forward the original date fields for the list records.

9. Automated process to reduce the time necessary and labor involved to move large file repositories to SharePoint.

Almost all mid-sized to large-sized migrations require automation, especially when incremental/batch migrations from different sources take place. Automated batch jobs help you to take complete control of the migration process by handling errors and triggering events that can be managed efficiently. Otherwise, you have to spend hours on ad hoc problems and tracking down repetitive errors thrown by non-automated migrations.

10. Ability to track the migration at a very granular level It will be ideal to track the status for each metadata that is migrated to a SharePoint list.

This will help the user take corrective action at the source data level (fixes or workarounds) and re-import the metadata to SharePoint. Though this could be a tedious task in complex migration / complex dataset scenarios, it is import to maintain the sanity of data that is migrated to SharePoint to ensure 100% data integrity.

If you are looking for a SharePoint List management solution that addresses many of the data migration challanges stated above, take a look at our SPList Manager for SharePoint 2007 (SPListM) solution at: