Vyapin Blogs

We have made a significant release of our powerful SharePoint list management solution – SPList Manager for SharePoint 2007 (SPListM) version 1.2.

The following are some of the improvisations made to the product solution in this new release:

1. Support to Forms based authentication: SPListM now supports Forms Authentication validating the users against the corresponding SharePoint environment. SPListM would support websites configured with Forms based authentication mechanism.
2. Supports Fully Qualified Domain Name format: SPListM now supports a Fully Qualified Domain Name, i.e. ‘username @domain’ format in its entire validation schema.
3. Profile Manager Tool: Users’ logon information can be stored as User profiles and the same can be retrieved without necessarily entering the credentials every time. Profile manager supports both Windows based and Forms based authentication credentials. Using this, the users can just select their corresponding credentials without manually entering them every time when connecting to SharePoint.
4. Folder options: SPListM now allows users to create folders (if unavailable) on the fly and helps in restructuring folder hierarchy during content migration. The folders to be created can be defined in the external batch descriptor file.
5. Web Services Component Indicator: It is now easy to determine the existence of Web Services Component. The Web Services Component Indicator would now determine the presence of Web Services Component in the corresponding destination SharePoint server. The user can access the same from the Tools Menu of the application (ToolsàTest SPListM Web Service). This would avoid Web Service Component mismatch, if any.
6. SPListM application can now run on Windows 7 / Vista / XP computers. SPListM Web Services component can run on Windows Server 2008 and 2003.

For a free 30-day trial version or to upgrade to the latest version of SPListM software, visit the product home page at http://www.vyapin.com/products/sharepoint/moss-2007/splistm/sharepoint-2007-list-management-tool.htm .

Businesses process and share enormous information today, thanks to SharePoint collaborative environment in which most of them operate. It is no simple task to make information available in the collaborative environment by importing them to SharePoint Lists from various external information sources.

SPListM, our list management solution makes it extremely easy for the user to import and update list items in a SharePoint environment. The information or the list items that are to be imported are usually held in a Microsoft Excel Spreadsheet or CSV format. In fact, our previous blog post was about the limitations the users would confront when using an Excel spreadsheet to import data to a SharePoint list
(http://www.vyapin.com/blogs/index.php/2010/03/02/importing-microsoft-excel-spreadsheet-data-to-sharepoint-2/).

This blog post would discuss how SPListM would help the users address some of these list management issues in SharePoint.

1. SPListM supports standard data types / columns: SPListM supports all standard data types such as, single or multiple lines of text, number, currency, date and time, calculation, lookup, hyperlink etc. SPListM can help the users with data pertaining to almost all business data types such as People, Lookup, Business Data etc.

SPListM through its compatibility with the Business Data Catalog feature has the ability to retrieve information associated with a reference column from the back-end data source. It acts as a connector and fetches related information stored across tables. The same is the case with Lookup columns. SPListM helps users retrieve all the values in the reference column corresponding to the Look Up column. The following screenshots would demonstrate SPListM’s support to ‘Look Up’ data type:

The Property Name ‘Department’ being defined as a Look Up column allows the user to choose from the options based on which the corresponding List items are updated as follows:

The above screenshot shows the column values, the Look Up->Department refers to. SPListM fetches the corresponding data and makes it easy for the user to update the List Item based on the options listed.

2. Import data to all SharePoint list types: SharePoint has in it numerous list types like Project Tasks, Calendars, Surveys, Issue Tracking, Links, Discussion Boards etc. which exist for more specific business reasons to organize information. SPListM helps users update list items to any of these list types, including custom lists created by the user. Updating the list items manually would be a daunting task for the user. SPListM can make use of a simple batch descriptor file to accomplish this task. In fact using an external metadata file allows the user to update more than one data fields for a particular list item simultaneously. The following screenshots would better explicate this:

The above screenshot shows an external batch descriptor with the corresponding Lists and the updates to be made to the corresponding SharePoint list. The field ‘Base URL’ denotes the Lists to be updated. As you can see, SharePoint List Types such as Tasks and Calendars can be updated through SPListM.

Screenshot of SPListM-Lists in a SharePoint site:

The above screenshot displays the SharePoint lists corresponding to the given site URL. The Lists pane displays SharePoint list types such as Tasks, Calendars and Discussions etc. The corresponding Lists can be selected and the necessary updates can be done using SPListM.

3. Automated List items update: Take a situation where in a huge chunk of business information undergoes constant updates and the same information has to be available to everyone through SharePoint. In this situation, the list has to be imported every time it undergoes a change or the changes should be replicated in the SharePoint list manually. SPListM can make amendments to the existing data in list items in an automated import task, which can be scheduled to run at periodic intervals. In SPListM, you can make use of the basic mode wizard or an external metadata file to update the SharePoint list.

Consider the situation where the user has to update the project status daily at the end of the day or update file / document attachments from another source in a SharePoint list. If it is has to be done manually, he or she has to look at each list item and update the corresponding entries. SPListM on the other hand would allow the user to define the changes to list items in a separate batch file. If the task is a daily routine, all he or she has to do is to define the changes in a batch file and schedule it for automatic list updates. The task would get executed automatically and the changes can be made without much difficulty and human intervention.

4. Ability to maintain versions: SPListM allows the users to maintain versions to effectively track changes to the list items. It also allows the user to go without storing minor changes as a separate version, depending on the user preference. In other words, maintaining version history of list items is left to the discretion of the user and SPListM effectively does what is expected out of it.

Screenshot of SPListM Versioning Settings:

The above screenshot shows SPListM’s ability to store changes in list item as a new version. It is left to the discretion of the user to enable the changes to be stored as a new version or to make the changes in the existing version.

5. Conditional Import of data: SPListM allows the users to conditionally import data, a feature that would help users to precisely import documents out of huge silos of information. The users can define complex logical conditions at multiple levels to ensure that the list items or records satisfying the conditions alone get updated. The updates can be done either through the basic mode wizard or through an external metadata file depending on its complexity.

Consider the following table containing Employee details. SPListM allows the user to selectively import these details to SharePoint. The user can impose import conditions satisfying which the data is imported. The conditions as said earlier can be imposed at multiple levels and precise information can be imported.

Suppose if the user wants to selectively import data into SharePoint, the user can impose conditions using SPListM. In the given example, if the user wants to have only the details of employees who are either from Sales or Marketing department, SPListM allows the user impose a logical “OR” condition to support the given cause. Take a look at the following screenshot:

Screenshot of SPListM-Conditional update of list items:

The following screenshot shows the SharePoint site with details of employees from Marketing or Sales departments:

In all of the above scenarios, SPListM helps users address most of the common issues confronted in their daily SharePoint usage. SPListM could very well prove to be indispensable tool for your SharePoint environment particularly for the list management challenges that one could face.

For a free trial version of SPListM, visit the product home page at

http://www.vyapin.com/products/sharepoint/moss-2007/splistm/sharepoint-2007-list-management-tool.htm

One of the most common scenarios in SharePoint migration is the migration of File shares and folders from various sources – file servers, desktops and other storage devices. While the need for migrating files and folders is quite common, the task of getting this accomplished becomes very complex if there are a large number of files and folders to migrate. You will need an automated solution that will take care of some of these complexities. A large migration typically involves migrating and tagging tens of thousands of files (documents) from legacy file shares and folders into a new SharePoint repository while retaining the existing taxonomy or moving to a completely new taxonomy.
Most file system users will not be familiar with the concept of document metadata. They are more familiar with the term file properties. The concept of document metadata originates from Document Management Systems (DMS), where documents are stored and made searchable on a wider set of keywords and phrases. Document libraries in SharePoint are akin to Document Management Systems in the way they store documents and properties for search and retrieval. However, these properties need to be presented to SharePoint during migration in a certain manner that makes the search and retrieval more powerful and elegant. You may migrate / propagate metadata fields from the document properties stored within the file or using an external metadata file / database. Mapping and tagging existing file properties and adding additional properties to documents in SharePoint, especially for a well structured taxonomy can be a laborious process. Thus, migrating files and folders along with metadata needs an appropriate solution that will break down the migration into several different tasks (automated batch processes), with several different rules to process the files and folders.
Our SharePoint migration tool for File shares and folders, DocKIT, allows users to move folders and files along with metadata (using an external metadata file and file properties) from file system to SharePoint libraries based on the folders/files added by the user using the DocKIT user interface. Some of the supported metadata elements that typically define the source content are:

• Custom properties defined in the external metadata file
• Summary file system properties - Title, Subject, Author, Category, Keywords and Comments
• File properties such as Manager, Company, DateLastPrinted, DateLastSaved, RevisionNumber, Version, WordCount etc. in the case of MS-Office documents
• Original Created Date & Last Modified Date of source documents
• Author (Created By) and Editor (Modified By) of source documents
• Content Approval Status & Approval Comments

Please click the following link to know more about Vyapin’s DocKIT: http://www.vyapin.com/products/sharepoint/moss-2007/dockit/sharepoint-2007-file-migration.htm

Businesses using Microsoft SharePoint technologies host loads of information that are constantly created, modified, accessed and processed by its users. SharePoint servers store this information in what are called ‘Lists’ that typically resembles an Excel spreadsheet.
As you may be aware already, you can import Microsoft Excel Spreadsheet to SharePoint [In your SharePoint site, select Site Actions -> Create -> Custom Lists -> “Import Spreadsheet” hyperlink under the Custom Lists header]

Microsoft Excel Spreadsheets may do what organizations generally require, but have limitations when the data volume is large or when there are several participants, as may be the case in many organizations.

The following are some of the areas where the users would sometimes feel the pain when importing Microsoft Excel Spreadsheet data to SharePoint 2007:

1) Limited support of data types or data classification
2) Import data to all SharePoint list types, including project tasks, issue tracking, custom lists etc.
3) Missing link between SharePoint and source data
4) Tracking changes to the data
5) Handling complex queries and conditions to filter data

SPList Manager for SharePoint 2007 (SPListM), our list management solution helps users overcome these pain points and in this blog post we will explain on how it does it.

1) Limited Support of data types:

Importing an Excel spreadsheet in SharePoint supports only custom lists and is not compatible with built-in data types such as People, Lookup and Business Data etc. Consider the situation where the user has to retrieve a huge chunk of business data relating to a unique id say Customer ID. All the data associated with Customer ID spread across various spreadsheets has to be referred and interpreted manually. SPListM on the other hand can help users retrieve information across multiple lists by referring to the Customer ID. It retrieves all information associated with the Customer ID and presents it for the user. SPListM supports all SharePoint standard data types. It would be appropriate to term SPListM as a generic SharePoint ‘list processing tool’ as it supports all widely used data types. This feature has managed to attract several SPListM customers and is rightly its value proposition.

2) Import data to all SharePoint list types:

As said earlier, importing list types such as project tasks, issue tracking, custom lists etc. to SharePoint would be a tough task, especially the ones with file attachments. SPListM offers a simple solution to import metadata (or list items) and associated file attachments to SharePoint lists such as Tasks, Calendar, Contacts, Survey, Discussion Boards, Announcements, Links, Issue Tracking and Custom Lists. For example, SPListM can easily let the user regularly update the project tasks or a custom list through an external batch file and could prove quite efficient by saving several man hours.

3) Missing link between SharePoint and the source data:

There is only a one-way communication between a Microsoft Spreadsheet and SharePoint services. Changes made to SharePoint list items is restored back to a spreadsheet (content source) while the vice versa is not possible. If the changes made to the spreadsheet source are to be updated in SharePoint list, it has to be exported to SharePoint as a new list. The link between the content source and the destination (SharePoint list) is terminated and this could prove costly when data undergoes constant changes. Consider a situation where loads of information is constantly dumped into an Excel spreadsheet and needs to be updated in the SharePoint site. It would be increasingly difficult for the user to export it every time or update the voluminous information manually in a datasheet view. SPListM could prove extremely handy in this situation as the updates could be defined in the import task wizard or through an external metadata file. SPListM does not necessarily establish a two-way communication with the SharePoint. But it still makes a one-way communication complete and more effective.

4) Tracking changes to data (versioning):

It is increasingly difficult for the user to keep track of the changes made to the content sources as the changes made would only be exported to SharePoint as a new list. As a result of which, ‘versioning’ is made impossible. In situations where a list has to be approved by other SharePoint users, it would be impossible to keep track of changes made by each user. Updating a list item with SPListM can help you maintain versions and track changes made by users in a collaborative environment.

5) Handling complex queries and conditions:

It would prove a daunting task for the user while attempting to conditionally import data using complex queries from a spreadsheet to a SharePoint site in an automated manner and more so if it involves a lot of data. In a situation where data is often exported to a spreadsheet and the user has to migrate the same to a SharePoint site, Excel’s filter options could be very useful. But what if the user wants to impose conditions at various levels and do a major cleansing of data? SPListM has the ability to impose conditions at various levels and could do this cleansing as easy as a cake walk. It allows the user to apply multiple logical conditions and can update the list items with ease. The updating could be done either through the basic import task wizard option or through an external batch descriptor file.

SPListM has been engineered in such a way as to fill the potential pitfalls a user faces while importing Microsoft Excel spreadsheet data to a SharePoint list. SPListM supplements Microsoft Excel’s capabilities in enhancing business productivity.

To try and evaluate SPListM, you can download a 15-day trial version at http://www.vyapin.com/products/sharepoint/moss-2007/splistm/sharepoint-2007-list-management-tool.htm.

The much anticipated release of SharePoint 2010 is on the cards. It’s almost official that the final cut would be released by June 2010. The beta version is already out and is certainly creating a buzz among the business community. The latest version that is coming up has flexible deployment options and has kindled our excitement as well. Several customers have inquired about our products’ compatibility with the latest version of SharePoint. Our answer to them is yes, we have all our products ready for the next big release of SharePoint.

Our reporting tool for SharePoint, ARKSP is ready for its major upgrade for the SharePoint 2010 release and should be up and running by next week. DocKIT, our flagship document migration solution is to follow suit with its upgraded version for SharePoint 2010. We have been getting a lot of queries from keen customers about DocKIT’s compatibility with SharePoint 2010 and we are not going to make them wait any further than the first week of Feb 2010. The SharePoint to file system migration solution, SPListX is gearing up for the release with additional ‘Search Criteria’ features, which, the users are going to find it extremely useful in pulling out precise ‘spot-on’ information. Our SharePoint list management solution, SPListM comes out with a beta version by March 2010. XPlica, our solution to your migration needs (from SharePoint to SharePoint libraries) would be the last to be released. We have scheduled it for April 2010. We are on full speed to keep our solutions abreast with the biggest release of SharePoint and to continue helping our customers leverage their SharePoint resources. The beta version of our products has already started to roll out and is being evaluated by some of our renowned customers. We have already started to listen to some excellent feedback from our customers and we are looking forward to keep the momentum going on. The next couple of months are going to be extremely significant for us as we deliver the latest versions to our customers and prospects.
Looking forward to an absorbing month ahead…

When migrating data from non-SharePoint sources / legacy applications, it is important to design, organize and manage your list items and properties carefully so that there is minimal rework on your lists and properties without having to redesign the whole content framework. We will discuss below some of the common list management issues in SharePoint both during migration as well as management of content post migration.

1. Import list data and maintain or change folder structure when migrating data from various non-SharePoint sources or legacy applications to MOSS 2007 / WSS 3.0

As stated in the previous blog article related to document migration to SharePoint, most companies will want to retain the same structures for folders and contents to maintain operational consistency as well as business continuity. It will be easier to maintain the same folder structure in the newly setup SharePoint list without changing the user experience in handling folders and related information / content. It makes the navigation intuitive, eases migration process, minimizes user training and improves operational efficiency. In a few business situations, it will be a lot better to place the contents in a new structure to improve efficiency or due to changes in the business. In both cases, it is important to identify a solution that can organize content depending on the business or functional or end-user requirements.

2. Import information from other systems that are in different formats to specific SharePoint 2007 list types such as tasks, contacts, issue tracking, wiki pages etc.

In some cases, it will be easy to map existing content to the default list types provided by SharePoint 2007, such as project tasks, announcements, contacts, pictures, discussions etc. In the case of proprietary formats or contents residing in legacy applications or relatively unknown CMS, content migration could pose a problem when trying to map the existing assets to the correct SharePoint list type as we need to. In such cases, it is important to store the content in a neutral format with the ability to cleanse the information before migrating them to SharePoint and map the information to respective list types meaningfully.

3. Selectively update metadata fields with values that have changed over a period of time or update missing fields that got missed out during the normal upload process.

When the SharePoint system is in full use, it is likely that the users may miss out some critical piece of information / data when updating / uploading information in SharePoint or the business needs may have changed over time, thereby forcing the content owner to fill-in the missing information or fields in SharePoint.

4. Import metadata values and file attachments to SharePoint 2007 lists

Existing contents in non-SharePoint sources could be residing in various formats and file locations. When consolidating all the existing information assets, it will be vital to pull them from various data sources and associate all documents from different file server / network share locations. It is also important to import metadata and file attachments from several sources and append them to existing default / custom lists in SharePoint.

5. Selective migration of information based on user-defined conditions that can facilitate repetitive import of information from a single source file or multiple source files.

In a few instances, it may vital to append metadata records as different versions in the SharePoint list to track version history (along with file attachments) or may be required to replace the old metadata records with new ones and retain the latest values, depending on certain pre-defined conditions. The metadata records may be available in a single source metadata file or multiple metadata files when they exported from the non-SharePoint source.

6. Mass document migration application to work off your desktop instead of running right on the SharePoint servers directly.

If you are using third-party tools for migration, a solution that can run either on the desktop and or on the server will be ideal. Take the case of incremental migrations. The SharePoint server could already be in production mode while several different SharePoint lists could still be under migration. Server performance will be compromised if the migrating application is going to run on the server. It is better if the application runs on a desktop performing actions like data cleansing, processing etc. and just do a final bulk upload into SharePoint. In real life, it will be time-consuming task to implement business processes to handle each situation differently.

7. Migrate to your new SharePoint repository while retaining the existing taxonomy or migrate to a new taxonomy.

The metadata need to be presented to SharePoint during migration in a certain manner that makes the search and retrieval more powerful and elegant. You may migrate / propagate metadata fields from the external metadata file / database. Mapping and tagging list metadata and adding additional metadata / properties and documents in SharePoint, especially for a well structured taxonomy can be a laborious process.

8. Retain and carry-forward the Created Date and Last Modified file attributes to maintain business continuity for users and minimize user training when collaborating in the new SharePoint environment.

There are plenty of business reasons to retain the same Date field values once the metadata has been migrated to SharePoint. Unfortunately, SharePoint falls short in this aspect and you need third-party tools or some in-house programming to carry forward the original date fields for the list records.

9. Automated process to reduce the time necessary and labor involved to move large file repositories to SharePoint.

Almost all mid-sized to large-sized migrations require automation, especially when incremental/batch migrations from different sources take place. Automated batch jobs help you to take complete control of the migration process by handling errors and triggering events that can be managed efficiently. Otherwise, you have to spend hours on ad hoc problems and tracking down repetitive errors thrown by non-automated migrations.

10. Ability to track the migration at a very granular level It will be ideal to track the status for each metadata that is migrated to a SharePoint list.

This will help the user take corrective action at the source data level (fixes or workarounds) and re-import the metadata to SharePoint. Though this could be a tedious task in complex migration / complex dataset scenarios, it is import to maintain the sanity of data that is migrated to SharePoint to ensure 100% data integrity.

If you are looking for a SharePoint List management solution that addresses many of the data migration challanges stated above, take a look at our SPList Manager for SharePoint 2007 (SPListM) solution at:

http://www.vyapin.com/products/sharepoint_list_management_migration_splistm.htm

This is a significant release of SPListX, not only for us but also for our customers who have been waiting for some time to get these new features. I am sure it was worth the wait for them, because we made sure we addressed all of the important features that will make this an important component in our solution suite for SharePoint List management, List Migration, List Backup & Restore and List Synchronization requirements. 

You can now backup / export all the list items along with their metadata to a file server or local computer using SPListX and reimport them using DocKIT or SPListM depending on the SharePoint list type. You can also synchronize your SharePoint list contents with your file system using SPListX. 

In fact, the entire combo of DocKIT, SPListM and SPListX is a powerful tool chest that all SharePoint managers and administrators would love to have. All the 3 tools are a must for any organization that has a serialized process for moving documents and other list items in and out of SharePoint using networked file shares. 

And ofcourse, the best part about all the 3 products is their ability to retain the salient list item / document properties - Created Date/Time, Modified Date/Time, Created By and Modified By when working in and out of SharePoint (both MOSS and WSS). 

What’s new in SPListX 3.0? 

» SPListX latest version supports all SharePoint list types such as Tasks, Calendar, Contacts, Survey, Discussion Boards, Announcements, Links, Issue Tracking and Custom Lists in Microsoft Office SharePoint Server 2007 (MOSS 2007) and Windows SharePoint Services 3.0 (WSS 3.0).

» Export SharePoint lists based on user-defined export conditions  

» Enhancements to the GUI 

Features

» Export folders & documents and metadata to the file system

» Export SharePoint lists to the file system using a batch descriptor file

» Automate Tasks using the Tasks Manager

» Export SharePoint Columns, including system columns such as Created, Modified, Approval Status, etc. SPListX exports metadata into CSV, XLS, MDB and XML file formats.

» Export document / list item versions

» Export folder / document system dates to metadata file

» Propagate folder / document system date fields - Created Date/Time and Modified Date/Time to NTFS 

Almost all Sharepoint implementations have a need to reorganize lists at regular intervals based on changes in business as well as user needs. SPListM addresses these needs by letting administrators do this on the fly or automate such tasks. Here are some of the salient features/benefits of SPListM:

• Upload or create new list items, including attachments to a SharePoint list 
• Update metadata / properties / column values for list items in the SharePoint list
• Delete existing list items in a SharePoint list
• Create new list items or Update metadata for list items in a SharePoint list using in a CSV / Excel / TXT file
• Update SharePoint’s built-in fields, namely - Content Approval Status & related Comments, Content Type, Created Date, Modified Date, Created By and Modified By fields for each folder and list item in a SharePoint list
  (Note: SPListM for SharePoint 2007 Web Services component must be installed in the SharePoint Server to activate this feature)
• Apply filter conditions based on SharePoint columns prior to an update or delete operation
• Manage SPListM tasks through command-line or Windows Task Scheduler interface