Vyapin Blogs

Our products install and maintain their respective application files in Program Files or Program Files (x86) folder, unless the default location is changed by the user. Post-installation, when launching the product the first time, the user has the option to specify a drive location for storing the application worker files such as settings, error log, tasks and task history files which we refer to as <Application Data Folder>.

The <Application Data Folder> can be found from the Help -> About screen. The default path of <Application Data Folder> is as follows:

==> Windows XP, Windows 2003 - C:\Documents and Settings\All Users\Documents
==> Windows 7, Windows 2008, Windows Vista - C:\Users\Public\Documents

You can change the application data folder at any time by browsing to the desired folder location in Help -> About menu.

At a later date, users may want to upgrade to the newer version of the product or uninstall and re-install the product in another computer. For this very purpose, our products provide options to uninstall just the application or uninstall the application and the application related worker files that are stored in the computer.

Let us take a detailed look by taking DocKIT for SharePoint 2010 product as an example:

Upgrading products:

The process of upgrading to the new version of the product by retaining the settings is a 6-step process, as outlined below:

Uninstalling products:

Cleanup import tasks and their history instances

This option cleans up the import tasks created in DocKIT and their history instances generated during import tasks run. The import of content from file system to SharePoint will be configured as import tasks. These import tasks during their run will create the import task activity logs, error logs and the respective task settings in the <Application Data Folder>. This option when selected will clean-up the entire folders and files created by the import tasks.

Cleanup scheduled tasks that are associated with the import tasks

This option cleans up the Windows Scheduled Tasks corresponding to each import task, when the task is set to run at scheduled intervals. This is the sub-option of the previous clean-up option. Selecting the first option will delete the scheduled tasks automatically.

Cleanup stored user profiles

DocKIT maintains the user credential used for connecting to SharePoint in the Credential Manager, when user instructs DocKIT to store the user name and password for repeated use. Selecting this option will clean up the stored user profiles maintained by DocKIT application.

Uninstall DocKIT application

This option uninstalls DocKIT application by removing the application files from the application installation folder. This option is similar to using the Control Panel - Programs and Features OR Add / Remove Programs applet option.

Upon clicking Next in the Uninstall Wizard will prompt for confirmation to start the uninstall process.

Clicking Finish button in the Uninstall Wizard will perform cleanup based on the selected cleanup options.

Most of the content migration to SharePoint involves migration of archived and active contents from different systems such as file servers, network shared folders and other document management systems (e.g., Humming Bird, DocuShare, DocsOpen). This is an impending task apart from migrating frequently changing folders and files in the current storage system on a day-to-day basis.

It is a well-known fact that the contents stored in these systems might have their folder and file names coined as per legacy naming conventions or standards prevailed at that time. There also exists a possibility of the folder and file names having special characters that are valid in legacy systems but invalid (disallowed) in SharePoint. During migration of contents from legacy systems to SharePoint, users might want to get rid of the old names and replace them with new names, rename invalid to valid characters in folder and file names.

Apart from this, users may have a possible need for renaming folders and files that are already available in SharePoint.

In such cases, organizations and users who migrate bulk contents into SharePoint need user-friendly means with varied options to rename those folders and files in bulk during migration / post migration.

In a nutshell, we highlight the following general and specific cases of renaming requirements:

	Rename folders & files while migrating from legacy systems to SharePoint
	Rename folders & files with invalid characters to valid characters in SharePoint
	Rename folders & files that already exist in SharePoint

Folder level requirements:

	Create a new folder hierarchy (restructure existing contents) and import files into them during migration
	Create new document sets by way of renaming folders

Document / File level requirements:

	Rename files of different names and related contents and import them as document versions (maintain version history)
	Rename files having names for each version (appended with version number) and import them as document versions

To cater to the above mentioned renaming requirements, DocKIT for SharePoint 2010 product provides easy renaming of folders and files in the following two ways:

1. Rename folders & files using the values specified in an external metadata file or batch descriptor file.
2. Rename folders & files using folder & file naming rules.

Rename folders & files using the values specified in an external metadata file / batch descriptor file

I will explain how to use the external metadata file or a batch descriptor file to rename folders and files in the most common cases using a few examples below:

1. Rename files simply by providing alternate names

2. Rename folders simply by providing alternate names

3. Rename files that have same extension

Let’s take an example where files generated by an external system with the same extension (e.g., “.01”). In this case, the end user could be interested in providing suitable alternate file names for meaningful representation. Moreover, since MS Office files (like - .docx, .doc, .xlsx, .ppt, .doc, .xls,) are handled by SharePoint differently, renaming those files would be useful if they are stored with correct file extension.

It would most helpful if the end user is able to provide suitable alternate name by appending the extension corresponding to the native application that generated the file / document.

You can achieve this by specifying (source file path, new file name) in an external metadata file or the batch descriptor file.

By using an external metadata file

4. Create a new folder hierarchy and import files into them during migration

One other specific usage scenario that you may encounter is to migrate all files in a folder to a new sub-folder in the destination location. This requirement can be easily achieved through “Batch Descriptor file” task option in DocKIT.

You can achieve this by stating the desired folder hierarchy under New Folder column while specifying batch file entries. Also, you have to set ‘No’ for ‘Do you want to create the top-level folder(s) included for import?’ in ‘Folder Options’ step in DocKIT Task Wizard.

5. Create a new document set by way of renaming folders on the fly and import files into them during migration
   

In order to leverage the use of newly introduced Document Set feature in SharePoint 2010, you would be interested in creating Document Set by using the existing folders during migration into SharePoint. DocKIT comes in handy to address this unique requirement.

To accomplish this, you have to specify the new folder name which has to be created as document set and the corresponding folder content type (created using ‘Document Set’ as its parent content type) to be used for creating the new folder as document set.

After creating the specified folder as document set, DocKIT will create its sub-folders as document sets and migrate the files under the respective sub-folders.

Adding custom document set content type to SharePoint library:

6. Rename files that already exist in SharePoint by providing alternate names

DocKIT provides the option to rename the files that are already available in SharePoint. It works in a similar fashion like new files that are imported into SharePoint, but for the Path column. The
Path column should be replaced by Destination Path column and should contain the file URL of the list item available in SharePoint.

You can use only batch file syntax to rename files that are already available in SharePoint. In the batch file, apart from the New Name column, only Destination Path and not the Path will be specified.

7. Rename folders that already exist in SharePoint by providing alternate names

Similar to renaming an existing file in SharePoint, DocKIT facilitates existing folder rename option. As indicated already, you can use only the batch file syntax to rename folders and files that are already available in SharePoint. In the batch file, you have to specify the New Folder column, apart from the Destination Path column.

Rename folders & files using folder & file naming rules

We will discuss about another interesting and useful option in DocKIT to rename folders / files based on patterns, remove illegal characters in folders/file names while migrating to SharePoint in the next blog post.

Read: DocKIT: Rename folders/files based on patterns using renaming rules

In continuation with the previous blog post that deals with the folder and file rename options, here we discuss how to deal with renaming of folders/files that contain illegal characters.

As said earlier, folders and files residing in legacy systems will have having special characters that are invalid (disallowed) in SharePoint. DocKIT facilitates this renaming by providing folder and file naming rules while migrating into SharePoint.

Certain users might also need migration of different versions of a file with their version named with certain patterns. DocKIT also supports this file version history creation seamlessly.

Rename folders & files using naming rules

This option is intended /recommended for renaming folders & files that have characters not accepted in SharePoint as part of their names.

SharePoint disallows certain invalid characters in the folder and file names. In order to overcome this behavior, you need to replace certain characters that appear in Windows folder & file names and are invalid folder / file characters in SharePoint libraries, with valid characters (all characters except ~ # % & { } * | \ : < > / ?) in SharePoint.

For this, you can use the “Folder & File Renaming Rules” feature in DocKIT. This feature in DocKIT uses ‘Regular Expressions’ technique that is widely used in software applications where character pattern matching is crucial.

Consider the following cases:

Example 1: Order of entries when constructing renaming rules

The Folder and File Renaming rule in DocKIT requires the rule entries to be correctly placed for successful and meaningful replacement of invalid characters. The order of the entries (sequence) is important depending on the name of the files and folders. For example, if you want dot and double underscore OR triple dots to be treated as a single entity, then a rule entry with this value has to be given first. After this, you should provide dot and single underscore OR double dots value as entry.

Example:

Equivalent Naming Rules:

Figure 1: Naming rule as specified in DocKIT application

Example 2: Usage of reserved characters

The ‘Regular Expressions’ technique used by DocKIT for pattern matching categorizes some of the operators as reserved and has its own meaning for the operators in the pattern syntax. In order to treat the reserved characters as normal characters like ‘A’, ‘1′, you have to use ‘\’ (backslash) as an escape sequence character in DocKIT Renaming rule as given below:

Create file version history by using certain naming patterns

We will discuss a situation where certain files in a folder have their names appended with version numbers. If you would like these version numbers to stack up nicely as version history of the file / document, this is what you can do.

Example 1: Uniform file naming pattern for file version

Example 2: Random file naming pattern for file version:

Note: Entries under the Find section should be in the same order as listed.

Figure 2: File Settings in DocKIT Task Wizard

To create version history, select ‘Create new version’ option in ‘File Settings’ step in DocKIT Task Wizard.

For more information, please refer the following help sections in the online help document. You can launch the help document from Help menu (Help -> Contents menu) in the application.

• DocKIT Features -> Folder & File Renaming Rules
• DocKIT Special Cases -> Rename Documents in File System

Why do I get this error in DocKIT? - “The column values could not be assigned for this file, since there was no corresponding entry in the external metadata file”

DocKIT assigns metadata when migrating documents to SharePoint. DocKIT can also assign metadata to documents that already reside in SharePoint. You can assign the same metadata for all documents / files in a folder or different metadata for each of the documents / files in the folder. 

In order to accomplish this, you have to specify absolute path of file or folder with the appropriate metadata for the SharePoint columns. The absolute file / folder path is dependent on the source from where files/folders are added for migration or on the destination SharePoint location. 

If the path in the external metadata file and the path of the added source contents do not match, then this error message: [”The column values could not be assigned for this file since there was no corresponding entry in the external metadata file.”] will be reported by DocKIT application.

Please note the following basic requirements about the contents in the external metadata file (provided as input to DocKIT software): 

	DocKIT expects the first column in the metadata file to be named as ‘Path’.
	[PATH] in the metadata file should refer to the path of the file or folder location, whose location is specified in any of the following formats:
		Local path: C:\My Documents\Versions Demo\SampleTest1.doc (or)
		UNC path: \\SPServer\MyDocs\Sample1.doc (or)
		Mapped drives: M:\MyDocs\Sample1.doc
	The path from which folders / files were added for import using DocKIT user interface must exactly match the path specified under ‘Path’ field in the metadata file. In other words, if you added folders / files from the D: drive (e.g., D:\My Documents) in the DocKIT explorer window, you must ensure that the path field in the external metadata file uses the D: drive to point to this specific file or folder.
	The columns listed in the metadata file should be available in SharePoint prior to metadata assignment.

I will now explain how to assign document metadata correctly in a few common scenarios using some examples.

Example 1: Import files and folders from local file system to SharePoint and assign metadata

Example 2: Import files and folders from network share (UNC) to SharePoint and assign metadata.

Example 3: Import files and folders from mapped network drive to SharePoint and assign metadata.

Example 4: Assign metadata for files already available in SharePoint

SharePoint Server 2010’s yet another new capability Document Sets facilitate managing documents as an entity based on their metadata (document property). The entity can be categorized based on the need or task which shares a common set of attributes and can be versioned as a whole (instead of individual documents) in SharePoint. The Content Type forms the underlying technique for Document Set and hence multiple document sets can be maintained in SharePoint.

As a natural course for a SharePoint migration product, our DocKIT for SharePoint 2010 product supports creation of Document Sets and classifying documents within them. With the existence of document sets in SharePoint, it optionally creates versions.

Here is a ready reckoner for migrating folders as document sets and importing the folder contents into them using DocKIT:

Let us consider the following folder structure maintained for plumbing system in the file system for migration into SharePoint.

• Fixtures
  • bathtub.dwg
  • coldwater.dwg
  • hotwater.dwg
  • piperack.dwg
  • shower.dwg
  • washsink.dwg
    

• Plan
  • drillguide.dwg
  • floorplan.dwg
  • pipingplan.dwg

1. In DocKIT, create an import task using Import folders, files and metadata to SharePoint Libraries (Explorer Mode) or Import folders, files and metadata to SharePoint Libraries (Batch File Mode) option.

2. In Import folders, files and metadata to SharePoint Libraries (Explorer Mode) option: Select the desired SharePoint site to be used as destination.

In Select items to import dialog, add the desired folders that you want to create as document sets in the destination SharePoint libraries.

In Import folders, files and metadata to SharePoint Libraries (Batch File Mode) option:

In Batch File Options step, select the desired batch descriptor file to be used as reference for import folders and files.

Note:Ensure that you specify the desired folders that you want to create as document sets in the destination SharePoint libraries, destination path.

3. Click Next to go to Metadata Options step. In this step, you can provide an external metadata file in (CSV/TXT/XLS) format to assign custom metadata for columns included in document set.

4. Click Next twice to go to System Dates step. This step will assist you to retain the original date field (Created Date, Modified Date) values as in source file system to destination SharePoint.

5.  Proceed with the subsequent steps to reach Content Type Settings step

6. In this step, under Folder Content Type Options, specify a content type that is derived from ‘Document Set’ content type. You can either enter a content type name or select a column in the external metadata file (that contains the content type name) to be used.

7. Proceed with the subsequent steps to reach Folder Options step.  

8. In this step, select a suitable folder option so as to create the added (top level / its sub-folders) folders as document sets in destination SharePoint library.  

9. Proceed with the subsequent steps to save and run the task.

10. On completion of the import process, the folders will be created as document sets with files imported underneath them. 

For further information on DocKIT for SharePoint 2010, please visit our product home page at:

http://www.vyapin.com/products/sharepoint/sharepoint-2010/dockit/sharepoint-migration.htm 

You can download a 30-day trial version from the above product web page.

Our much anticipated major release of SPListX version 4.0 (SPList Export for SharePoint 2010) is finally here with the following new features included,