Vyapin Blogs

We have made a significant and much improvised release of SPListX for SharePoint 2007 version 3.1 recently. We have included a powerful feature to export documents and list items from SharePoint lists based on ‘Search Criteria’ provided by the user. This new feature uses the built-in SharePoint Search facility in the background to deliver the results.
With the new SPListX for SharePoint 2007 version 3.1, users have the following new provisions:
1. Search Criteria: Export SharePoint list items with file attachments, version histories and metadata based on SharePoint Search results. Previously the user had the choice to export list items through the browse option or export them using an external batch file. Now, with the new ‘Search Criteria’ feature choice, the user can search for matching list items to export using Keywords or Advanced Search options at a site collection level or farm level. This feature was designed to leverage SharePoint’s Search feature, which is its value proposition.
2. Search Validation Settings: The user can validate the search results for a given search criteria (Keywords or Advanced Search). The ‘test search’ feature helps the user to verify the search results, thereby allowing the user to fine tune the search criteria to retrieve better results.
3. Support for Forms authentication: Prior to the current version, SPListX allowed only the users registered with the corresponding Active Directory domains to be validated against the SharePoint environment. But with the current release, SPListX supports users registered outside the Active Directory environment too. SPListX would initiate Windows authentication or Forms authentication automatically, depending on the destination SharePoint site.
4. Fully qualified domain name format: SPList X now supports a fully qualified domain name, i.e. username@domain format in all its user validation.

The ‘Search criteria’ function and the other functionalities were eagerly awaited by many our existing customers. SPListX with these new features incorporated is certainly bound to add value to your existing SharePoint infrastructure.

For a 30-day free trial version of SPListX, visit the product home page at: http://www.vyapin.com/products/sharepoint/moss-2007/splistx/sharepoint-2007-list-export.htm or contact our sales team at sales@vyapin.com for pricing related information.

One of the most common scenarios in SharePoint migration is the migration of File shares and folders from various sources – file servers, desktops and other storage devices. While the need for migrating files and folders is quite common, the task of getting this accomplished becomes very complex if there are a large number of files and folders to migrate. You will need an automated solution that will take care of some of these complexities. A large migration typically involves migrating and tagging tens of thousands of files (documents) from legacy file shares and folders into a new SharePoint repository while retaining the existing taxonomy or moving to a completely new taxonomy.
Most file system users will not be familiar with the concept of document metadata. They are more familiar with the term file properties. The concept of document metadata originates from Document Management Systems (DMS), where documents are stored and made searchable on a wider set of keywords and phrases. Document libraries in SharePoint are akin to Document Management Systems in the way they store documents and properties for search and retrieval. However, these properties need to be presented to SharePoint during migration in a certain manner that makes the search and retrieval more powerful and elegant. You may migrate / propagate metadata fields from the document properties stored within the file or using an external metadata file / database. Mapping and tagging existing file properties and adding additional properties to documents in SharePoint, especially for a well structured taxonomy can be a laborious process. Thus, migrating files and folders along with metadata needs an appropriate solution that will break down the migration into several different tasks (automated batch processes), with several different rules to process the files and folders.
Our SharePoint migration tool for File shares and folders, DocKIT, allows users to move folders and files along with metadata (using an external metadata file and file properties) from file system to SharePoint libraries based on the folders/files added by the user using the DocKIT user interface. Some of the supported metadata elements that typically define the source content are:

• Custom properties defined in the external metadata file
• Summary file system properties - Title, Subject, Author, Category, Keywords and Comments
• File properties such as Manager, Company, DateLastPrinted, DateLastSaved, RevisionNumber, Version, WordCount etc. in the case of MS-Office documents
• Original Created Date & Last Modified Date of source documents
• Author (Created By) and Editor (Modified By) of source documents
• Content Approval Status & Approval Comments

Please click the following link to know more about Vyapin’s DocKIT: http://www.vyapin.com/products/sharepoint/moss-2007/dockit/sharepoint-2007-file-migration.htm

NTFS Permissions reports on Files, Folders and Shares using Admin Report Kit for Windows Enterprise.

There are several powerful features available in Admin Report Kit for Windows Enterprise to generate reports on NTFS permissions on files and folders residing in servers and workstations across multiple domains in the network. All reports may be scheduled and generated for multiple computers, users, and groups for multiple domains as a batch job.

The Permissions Reports section under the built-in reports feature (out-of-the-box reports) includes specific reports that report exclusively on reporting the access permissions assigned to users and groups on files, folders and shares. Our NTFS permissions reporting tool has several flavors of reports designed specifically for the administrator’s convenience. The following questions can be easily answered using these multi-dimensional reports:
1. Given a selected set of Users and Groups, which files and folders do they have access to across computers in a domain?
2. Given a selected set of files, folders and shares across computers, which users and groups have access to these?
3. Which users have inherited access permissions by virtue of their group membership (even though they may not have been granted explicit permissions)?
4. What permissions have been assigned to users both explicit and inherited through nested groups? One single report showing both.
5. What are the net effective permissions for users and groups on a set of folders?
6. How are nested groups affecting NTFS permissions on files and folders?

Here is a walkthrough of how to generate NTFS Permissions Reports using Admin Report Kit for Windows Enterprise (ARKWE):

Click on the Permissions Reports menu item under the Built-in Reports button in the toolbar.

The following NTFS Permissions Reports are available:

List of permissions for specific users and groups on folders
Reports the folder permissions assigned to specific users and/or groups on a selected set of folders.

List of permissions for folders
Reports the permissions associated with a selected set of folders.

List of permissions for specific users and groups on files
Reports the files permissions assigned to specific users and/or groups under a selected set of folders.

List of permissions for files
Reports the permissions associated with files under a selected set of folders.

List of all permissions for folders (Inherit & Explicit)
Reports the permissions for users assigned in the folders directly and inherited by means of nested groups.

List of effective permissions for users and groups on folders
Reports the effective permissions for users and groups for a set of folders.

List of effective permissions for users and groups on files
Reports the effective permissions for users and groups for files available in a set of folders.

Apart from the above out-of-the-box NTFS Permissions Reports, several standard customizable reports on various share and folder resources are available. These may be customized and scheduled as batch jobs for multiple computers and domains.

Please click on the following to download and evaluate the above features in Admin Report Kit for Windows Enterprise.
http://www.vyapin.com/products/windows-audit/windows-reports.htm

Businesses using Microsoft SharePoint technologies host loads of information that are constantly created, modified, accessed and processed by its users. SharePoint servers store this information in what are called ‘Lists’ that typically resembles an Excel spreadsheet.
As you may be aware already, you can import Microsoft Excel Spreadsheet to SharePoint [In your SharePoint site, select Site Actions -> Create -> Custom Lists -> “Import Spreadsheet” hyperlink under the Custom Lists header]

Microsoft Excel Spreadsheets may do what organizations generally require, but have limitations when the data volume is large or when there are several participants, as may be the case in many organizations.

The following are some of the areas where the users would sometimes feel the pain when importing Microsoft Excel Spreadsheet data to SharePoint 2007:

1) Limited support of data types or data classification
2) Import data to all SharePoint list types, including project tasks, issue tracking, custom lists etc.
3) Missing link between SharePoint and source data
4) Tracking changes to the data
5) Handling complex queries and conditions to filter data

SPList Manager for SharePoint 2007 (SPListM), our list management solution helps users overcome these pain points and in this blog post we will explain on how it does it.

1) Limited Support of data types:

Importing an Excel spreadsheet in SharePoint supports only custom lists and is not compatible with built-in data types such as People, Lookup and Business Data etc. Consider the situation where the user has to retrieve a huge chunk of business data relating to a unique id say Customer ID. All the data associated with Customer ID spread across various spreadsheets has to be referred and interpreted manually. SPListM on the other hand can help users retrieve information across multiple lists by referring to the Customer ID. It retrieves all information associated with the Customer ID and presents it for the user. SPListM supports all SharePoint standard data types. It would be appropriate to term SPListM as a generic SharePoint ‘list processing tool’ as it supports all widely used data types. This feature has managed to attract several SPListM customers and is rightly its value proposition.

2) Import data to all SharePoint list types:

As said earlier, importing list types such as project tasks, issue tracking, custom lists etc. to SharePoint would be a tough task, especially the ones with file attachments. SPListM offers a simple solution to import metadata (or list items) and associated file attachments to SharePoint lists such as Tasks, Calendar, Contacts, Survey, Discussion Boards, Announcements, Links, Issue Tracking and Custom Lists. For example, SPListM can easily let the user regularly update the project tasks or a custom list through an external batch file and could prove quite efficient by saving several man hours.

3) Missing link between SharePoint and the source data:

There is only a one-way communication between a Microsoft Spreadsheet and SharePoint services. Changes made to SharePoint list items is restored back to a spreadsheet (content source) while the vice versa is not possible. If the changes made to the spreadsheet source are to be updated in SharePoint list, it has to be exported to SharePoint as a new list. The link between the content source and the destination (SharePoint list) is terminated and this could prove costly when data undergoes constant changes. Consider a situation where loads of information is constantly dumped into an Excel spreadsheet and needs to be updated in the SharePoint site. It would be increasingly difficult for the user to export it every time or update the voluminous information manually in a datasheet view. SPListM could prove extremely handy in this situation as the updates could be defined in the import task wizard or through an external metadata file. SPListM does not necessarily establish a two-way communication with the SharePoint. But it still makes a one-way communication complete and more effective.

4) Tracking changes to data (versioning):

It is increasingly difficult for the user to keep track of the changes made to the content sources as the changes made would only be exported to SharePoint as a new list. As a result of which, ‘versioning’ is made impossible. In situations where a list has to be approved by other SharePoint users, it would be impossible to keep track of changes made by each user. Updating a list item with SPListM can help you maintain versions and track changes made by users in a collaborative environment.

5) Handling complex queries and conditions:

It would prove a daunting task for the user while attempting to conditionally import data using complex queries from a spreadsheet to a SharePoint site in an automated manner and more so if it involves a lot of data. In a situation where data is often exported to a spreadsheet and the user has to migrate the same to a SharePoint site, Excel’s filter options could be very useful. But what if the user wants to impose conditions at various levels and do a major cleansing of data? SPListM has the ability to impose conditions at various levels and could do this cleansing as easy as a cake walk. It allows the user to apply multiple logical conditions and can update the list items with ease. The updating could be done either through the basic import task wizard option or through an external batch descriptor file.

SPListM has been engineered in such a way as to fill the potential pitfalls a user faces while importing Microsoft Excel spreadsheet data to a SharePoint list. SPListM supplements Microsoft Excel’s capabilities in enhancing business productivity.

To try and evaluate SPListM, you can download a 15-day trial version at http://www.vyapin.com/products/sharepoint/moss-2007/splistm/sharepoint-2007-list-management-tool.htm.

The much anticipated release of SharePoint 2010 is on the cards. It’s almost official that the final cut would be released by June 2010. The beta version is already out and is certainly creating a buzz among the business community. The latest version that is coming up has flexible deployment options and has kindled our excitement as well. Several customers have inquired about our products’ compatibility with the latest version of SharePoint. Our answer to them is yes, we have all our products ready for the next big release of SharePoint.

Our reporting tool for SharePoint, ARKSP is ready for its major upgrade for the SharePoint 2010 release and should be up and running by next week. DocKIT, our flagship document migration solution is to follow suit with its upgraded version for SharePoint 2010. We have been getting a lot of queries from keen customers about DocKIT’s compatibility with SharePoint 2010 and we are not going to make them wait any further than the first week of Feb 2010. The SharePoint to file system migration solution, SPListX is gearing up for the release with additional ‘Search Criteria’ features, which, the users are going to find it extremely useful in pulling out precise ‘spot-on’ information. Our SharePoint list management solution, SPListM comes out with a beta version by March 2010. XPlica, our solution to your migration needs (from SharePoint to SharePoint libraries) would be the last to be released. We have scheduled it for April 2010. We are on full speed to keep our solutions abreast with the biggest release of SharePoint and to continue helping our customers leverage their SharePoint resources. The beta version of our products has already started to roll out and is being evaluated by some of our renowned customers. We have already started to listen to some excellent feedback from our customers and we are looking forward to keep the momentum going on. The next couple of months are going to be extremely significant for us as we deliver the latest versions to our customers and prospects.
Looking forward to an absorbing month ahead…

Consider the following scenario:

There are two domains in a forest with different namespaces namely SPACENET (SPACENET.local) and OtherDomain (OtherDomain.local). Let us assume that SPACENET is the domain that needs to access resources in OtherDomain. In order to allow domain users from SPACENET to access resources in the domain OtherDomain, we need to add SPACENET’s users as members in the ‘domain local group’ of
OtherDomain.

If any user from the SPACENET domain is a member of ‘domain local group’ of OtherDomain (within the same forest), then the ‘Member Of’ tab for that user will not show that he is a member of ‘domain local group’ of OtherDomain. So, if you would like to know the ‘member of’ details of a particular User in a domain, a comprehensive listing should show all groups the User is a member of, including those groups in other domains that the user is a member of. 

Please see following figures to understand this better.

Active Directory Users and Computers for

‘OtherDomain.local’

Active Directory Users and Computers for ‘SPACENET.local’

If an administrator wants the ‘Member of’ details for users for the entire forest, he needs to view each group’s ‘Members tab’ in the AD console to see whether the specified user is a member of this group. The administrator needs to repeat this step for all domains and all groups in those domains.
 
So, How does ARKAD help show Users ‘Member of’ details for all domains in a forest in a single report view?
 
With the help of Admin Report Kit for Active Directory (ARKAD) you can view the users ‘Member Of’ details for an entire forest. The following image depicts the report generated by ARKAD for the above scenario.

A user may be assigned to multiple groups in an Active Directory organization. A group member may have membership in other groups in the same domain (or) in a different domain within the same forest (or) in a different domain in a different forest.

An in-depth user/group membership report must include all the groups that a user is member of across the entire AD organization (and not just the groups within one domain).

In a multiple forest environment, When we add a member from one domain to a group in another domain (from a trusted domain outside of that forest) , Active Directory automatically creates a special object called a foreign security principal (FSP) in the CN=ForeignSecurityPrincipals container in the domain NC.

Active Directory creates a foreign security principal object in a forest when objects from its trusted external forest are assigned group membership and security for trusting the forest’s objects. The users and groups of the external forest are represented by foreign security principals in the trusting forest and is necessary for them to access domain resources that exist in that forest. When a trust is established between domains across forests, these foreign security principals can become members of ‘domain local groups’ in the source domain.

In order to generate a report on all user memberships, you need a tool that runs through all user memberships across domains and if there are multiple forests with FSPs, then the membership across forests will have to be generated. For example, a complete membership listing of a User A, who is present in multiple domains across multiple forests, will show all groups that User A is a member of (including Domain Local Groups).

Vyapin’s Admin Report Kit for Active Directory (ARKAD) generates such complex user/group membership reports.

How to view all security principals in all domains within a single forest in ARKAD? (A security principal can be a user, group, service, or Computer). The Forest Reports feature in ARKAD allows the user to generate reports across domains in a forest. (Select ‘Forest Reports…’ under New Report button in the tool bar. The Forest Reports window with the list of reports will be displayed; Select a report from the list of reports. Click Next to proceed to the next steps).

When migrating data from non-SharePoint sources / legacy applications, it is important to design, organize and manage your list items and properties carefully so that there is minimal rework on your lists and properties without having to redesign the whole content framework. We will discuss below some of the common list management issues in SharePoint both during migration as well as management of content post migration.

1. Import list data and maintain or change folder structure when migrating data from various non-SharePoint sources or legacy applications to MOSS 2007 / WSS 3.0

As stated in the previous blog article related to document migration to SharePoint, most companies will want to retain the same structures for folders and contents to maintain operational consistency as well as business continuity. It will be easier to maintain the same folder structure in the newly setup SharePoint list without changing the user experience in handling folders and related information / content. It makes the navigation intuitive, eases migration process, minimizes user training and improves operational efficiency. In a few business situations, it will be a lot better to place the contents in a new structure to improve efficiency or due to changes in the business. In both cases, it is important to identify a solution that can organize content depending on the business or functional or end-user requirements.

2. Import information from other systems that are in different formats to specific SharePoint 2007 list types such as tasks, contacts, issue tracking, wiki pages etc.

In some cases, it will be easy to map existing content to the default list types provided by SharePoint 2007, such as project tasks, announcements, contacts, pictures, discussions etc. In the case of proprietary formats or contents residing in legacy applications or relatively unknown CMS, content migration could pose a problem when trying to map the existing assets to the correct SharePoint list type as we need to. In such cases, it is important to store the content in a neutral format with the ability to cleanse the information before migrating them to SharePoint and map the information to respective list types meaningfully.

3. Selectively update metadata fields with values that have changed over a period of time or update missing fields that got missed out during the normal upload process.

When the SharePoint system is in full use, it is likely that the users may miss out some critical piece of information / data when updating / uploading information in SharePoint or the business needs may have changed over time, thereby forcing the content owner to fill-in the missing information or fields in SharePoint.

4. Import metadata values and file attachments to SharePoint 2007 lists

Existing contents in non-SharePoint sources could be residing in various formats and file locations. When consolidating all the existing information assets, it will be vital to pull them from various data sources and associate all documents from different file server / network share locations. It is also important to import metadata and file attachments from several sources and append them to existing default / custom lists in SharePoint.

5. Selective migration of information based on user-defined conditions that can facilitate repetitive import of information from a single source file or multiple source files.

In a few instances, it may vital to append metadata records as different versions in the SharePoint list to track version history (along with file attachments) or may be required to replace the old metadata records with new ones and retain the latest values, depending on certain pre-defined conditions. The metadata records may be available in a single source metadata file or multiple metadata files when they exported from the non-SharePoint source.

6. Mass document migration application to work off your desktop instead of running right on the SharePoint servers directly.

If you are using third-party tools for migration, a solution that can run either on the desktop and or on the server will be ideal. Take the case of incremental migrations. The SharePoint server could already be in production mode while several different SharePoint lists could still be under migration. Server performance will be compromised if the migrating application is going to run on the server. It is better if the application runs on a desktop performing actions like data cleansing, processing etc. and just do a final bulk upload into SharePoint. In real life, it will be time-consuming task to implement business processes to handle each situation differently.

7. Migrate to your new SharePoint repository while retaining the existing taxonomy or migrate to a new taxonomy.

The metadata need to be presented to SharePoint during migration in a certain manner that makes the search and retrieval more powerful and elegant. You may migrate / propagate metadata fields from the external metadata file / database. Mapping and tagging list metadata and adding additional metadata / properties and documents in SharePoint, especially for a well structured taxonomy can be a laborious process.

8. Retain and carry-forward the Created Date and Last Modified file attributes to maintain business continuity for users and minimize user training when collaborating in the new SharePoint environment.

There are plenty of business reasons to retain the same Date field values once the metadata has been migrated to SharePoint. Unfortunately, SharePoint falls short in this aspect and you need third-party tools or some in-house programming to carry forward the original date fields for the list records.

9. Automated process to reduce the time necessary and labor involved to move large file repositories to SharePoint.

Almost all mid-sized to large-sized migrations require automation, especially when incremental/batch migrations from different sources take place. Automated batch jobs help you to take complete control of the migration process by handling errors and triggering events that can be managed efficiently. Otherwise, you have to spend hours on ad hoc problems and tracking down repetitive errors thrown by non-automated migrations.

10. Ability to track the migration at a very granular level It will be ideal to track the status for each metadata that is migrated to a SharePoint list.

This will help the user take corrective action at the source data level (fixes or workarounds) and re-import the metadata to SharePoint. Though this could be a tedious task in complex migration / complex dataset scenarios, it is import to maintain the sanity of data that is migrated to SharePoint to ensure 100% data integrity.

If you are looking for a SharePoint List management solution that addresses many of the data migration challanges stated above, take a look at our SPList Manager for SharePoint 2007 (SPListM) solution at:

http://www.vyapin.com/products/sharepoint_list_management_migration_splistm.htm

One of the headaches in finding out the NTFS permissions on files and folders is determining the effective permissions. Files and folders may have permissions explicitly set on them for users and groups and also have implicit/inherited permissions (e.g. users having access or deny to a folder by virtue of  their membership in a group or a nested group). Determining the effective permissions is a complex coding task, especially when you take into account the local and other built-in groups.

We have now included this feature in our Admin Report Kit for Windows Enterprise (ARKWE) and this is scheduled for release in a week or so. This has been a long awaited feature for many of our customers and prospects and I am glad that we have addressed it in the upcoming release.

We have released a new and improved version of DocKIT for SharePoint 2007 about a week back. We included some nice features based on the customer feedback we received over the last few months. (BTW, we are very attentive when it comes to customer feedback and feature suggestions, which has truly brought us to where we are today).

If you are a customer waiting for just these features, then DocKIT for SharePoint 2007 v3.8 version is right for you.

1)    Support for Business Data Catalog (BDC) column data type in SharePoint libraries. Business Data Catalog (BDC) column values must be provided with the appropriate filter name to use in order to query the LOB records. You must provide the BDC column value in the format: <filter name>:<value>. For example: “Company Name:NorthWind”, where Company Name is the filter name and NorthWind is the value you are searching for and would like to assign to the column.

2)    You can upload documents to Link to Document SharePoint default content type. This content type allows you to store a link to a document in a document library instead of the document itself. This content type is useful when referencing documents that are stored in http(s) locations.

3)    With the new version you can update the same metadata to all the existing files and folders based on wildcard characters.For example:http://sharepoint/library1/folder1/*      -   Update all folders and sub-folders only, but skip fileshttp://sharepoint/library1/folder1/*.*  -  Update all files and sub-folders

4)    When migrating folders and files from the file system to a SharePoint library, you can update the folder metadata to the file, in case there is no specific metadata given for the file that is imported to SharePoint. In other words, you can apply the same folder metadata to all files / documents within a folder, without the need to specify unique metadata entry for each file.

5)    You can now use complex HTML syntax to update SharePoint columns in HTML format. This feature is especially useful for Wiki pages.

6)    One of the common practices while using DocKIT is to interchangeably use UNC Path and Mapped Drive Letters when adding files to import and providing metadata entries. The earlier versions required that the user must be consistent in UNC Path or Mapped Drive letter usage when creating the import task (cannot specify both interchangeably). This new version automatically resolves UNC Path and Mapped Drives so that users can use them interchangeably without the need to fix this error and re-import documents.For any new feature suggestions or feedback, send us an e-mail to support at vyapin dot com.