I’ll start by saying it straight up: SharePoint 2010 Document IDs are not guaranteed to be unique.  If you route a document from one site collection to another, where the target site collection uses the same document ID prefix, then you could end up with duplicate document IDs.  And worse is that if you search for documents using a duplicate document ID, SharePoint will only return the first match.

Document ID Good Intentions

The new Document ID feature in SharePoint 2010 is one of many great additions to the SharePoint platform.  The generation and assignment of document IDs to content, and having control over the document ID format address a common request I’ve fielded from clients requiring automated assignment of unique document IDs.  Sure, there’s always been a unique ID assigned to each document but it was a GUID.  Asking users to use GUIDS is one way to loose the love.

How to Cook Up Duplicate Document IDs

1. Create two new site collections and enable the “Document ID” and “Content Organizer” features in each.  I’ll refer to one site collection as the source and the 2nd one as the target.
2. Configure the “Document ID” settings for each site collection using the same prefix.  For example:
3. Configure the content organizer in the source site collection to route documents to the target site collection.  This will require a “Send To” connection to be created in Central Administration that points to the target site collection.  This will also require a target site collection content organizer rule that drops incoming documents into the document library.
4. In the target site collection, upload a single document into the document library; a document ID will be assigned to that document.
5. In the source site collection, upload several documents to the “Drop Off Library”.  Those documents should be routed to the target site collection if the content organizer rule has been correctly configured.When you look at the documents in the target document library, you should see the duplicate document IDs.  For example:
   

Duplicate Document ID Search Misses

If you end up with duplicate document IDs (and you don’t have to, as I’ll explain later), and you search for a document with a duplicated ID, then only the first match will be returned.  That applies to both document ID search mechanisms:

• The document ID “redirect” URL “_layouts/DocIdRedir.aspx?ID=<doc id>”.
• The “Find by Document ID” web part.

If searching for content by document IDs is a key search use case for an organization, then ending up with duplicate IDs may make it difficult to locate content having duplicated document IDs.

It’s especially problematic if external systems need to store a link to the document using the document ID “redirect” URL.  Should the link reference a document with a duplicate doc ID, then the system/process could pull the wrong document (i.e. pulling the first match when the desired document is another document with the same ID).

All Doom and Gloom in Document ID Land?

No.  The SharePoint 2010 Document ID feature is still a great addition to the product.  Hopefully upcoming service packs will correct this issue and ensure that incoming duplicate doc IDs are updated to make then truly unique.

To prevent duplicate document IDs, ensure that every site collection that uses the document ID feature has a unique prefix (configured in the Document ID settings).  This should guarantee that all your documents where ever they may be routed in your farm, are assigned unique document IDs.

Writing a custom document ID provider is the other approach to ensuring the generation of unique document IDs.  The custom provider code could be implemented to derive a farm-wide unique identifier to ensure uniqueness across all farm site collections without the need to specify unique ID prefixes for each site collection.

Related posts:

1. SharePoint Document Management – Folder Discontent
2. SharePoint IRM Issue Opening Documents

For Michael Burry and other fund managers, there was plenty of information out there showing the terminal decline in lending standards.  Unlike others, Burry wasn’t satisfied with just the information.  He wanted the knowledge that would enable him to bet against the subprime mortgage bubble.  To build and manage the knowledge from the available information, and therefore to optimize his fund management business,  he spent months scanning and reading mortgage bond prospectuses and pairing that knowledge with the use of credit default swap (CDS) financial instruments.  CDS instruments (essentially insurance policies against the defaulting of corporate bonds) enabled Burry to put his knowledge into action and systematically build up a large and successful bet against subprime mortgage bonds.

Burry’s story, like many others, illustrates that having information is not enough.  Information (in all forms: electronic, printed, in people’s heads) needs to be processed and managed in a way that elicits and disseminates knowledge.  Knowledge management also includes processes and activities that elicit both tacit and explicit knowledge from people in the organization.  For example, your company may have people in the field (e.g. repair and service personnel working at customer sites) that have built up valuable field knowledge over the years.  Is that knowledge being captured, disseminated and reflected in your current business processes and analytics?  If not, you’re missing out on a key avenue for business optimization that could help improve customer service while lowering the costs of delivering that service.

Knowledge Management systems and platforms provide the tools that help capture, classify and disseminate knowledge but they are not the key to capturing knowledge.  The key is people.  That may be obvious, but it’s not uncommon to see IT systems implemented with the expectation that knowledge will follow.  Mining the tacit and explicit knowledge within your organization needs more than an intuitive Knowledge Management system.  It needs an approach focused on encouraging and rewarding knowledge contribution.  The service professional in the field making service calls needs both a way to easily contribute their knowledge (the “system”), and a reason (the incentive) to contribute their knowledge.  Do the service personnel carry hand-held devices that can easily be used to capture knowledge?  If not, can hand-held “knowledge capture” devices be effectively introduced into their working practices?  Do you instead (or also) schedule regular meetings with service personnel with the explicit agenda of capturing their experiences, information, and feedback in order to dig out and capture tacit knowledge?  What do the service personnel receive in return for their additional efforts to record their knowledge?  Of course, you don’t have to reward your service personnel for their additional knowledge capture efforts, but you’ll likely find that they contribute less without some kind of incentive.  And, if knowledge management is one of the keys to business optimization, it thereby follows that any hindrance to knowledge capture will slow business optimization efforts.

My experiences with many organizations are that knowledge management is ad-hoc and minimal at best.  How many times have you heard such questions after someone has left an organization: “How did John …”, “Why did John …”, “How did John figure out …”, etc?  Clearly, John’s knowledge was either not captured and/or people aren’t trained to know where to look and/or systems were not in place to effectively capture John’s knowledge.  Net result is less than optimal: knowledge is lost, work is redone, inefficiencies continue, mistakes are made, etc.  The fix is Knowledge Management, one of the keys to business optimization.

Related posts:

1. Microsoft Business Intelligence Conference 2008

Leadership ≠ Charisma + Intelligence

Was sorting through my old magazines the other day and came across an Aug 2007 edition of Scientific American Mind which contains an interesting article on leadership called “The New Psychology of Leadership“.  It dispels the conventional wisdom that effective leaders must always have charisma, intelligence and other “domination” personality traits.  And it confirms what I’ve observed for years working in IT for many different companies: slick, charismatic, command and control leadership does not work for technology teams.

Clearly, for many leadership roles intelligence is required.  Try putting yourself into a talented group of software developers without intelligence and see how far you get as a leader.  What the article suggests is that emphasizing maximum leader intelligence is short sighted in the face of research showing the importance of a leader’s ability to establish a shared identity.  Traits such as being down-to-earth and trustworthy can provide much more leadership value than being an Einstein, depending on the leadership context.

Leadership = Shared Identity

Analysis has revealed that the key to effective leadership is establishing and building a shared identity with one’s group, aligning the identity of the group towards planned strategies and policies.  Your team/group/country must be see you as one of them, positioned amongst them, not above them.  I include “country”, because it was noted that George Bush’s skill as a leader was very much related to his ability to be seen as a regular guy, not an intellectual.

Other Article Highlights

• Effective leadership is not about enforcing compliance.  It’s about defining and shaping the identity of one’s group.
• From the 60s to the late 70s, there’s been a flip-flop between the view that effective leadership is borne of charisma and the view that it’s borne of the ability to identify with one’s “followers”.
• Psychology studies conducted in the 70s illustrated the power of social identities and shared goals.  Although not directly focused on leadership, those studies helped clarify the need for leaders to act and be seen as group members.
• Historical perspective on Europe shows the correlation between social/national identity and the type of leader that arose.  No/little social identity = autocratic ruler focused on power over true leadership.  National identity = monarchs ruling as patriots.
• A social behavior experiment called the BBC Prison Study showed that effective leadership arose within the prisoners, but not the guards, because of the much stronger social identity shared by the prisoners.  The prisoners more desired to resist the guards, than the guards desired to guard the prisoners.
• No surprise that favoritism and large salary gaps between leaders and their groups can easily turn into destructive forces, undermining a leader’s credibility.

No related posts.

Problem
A document library configured with information rights management protection will not display the contents of Office documents after they’re originally created/uploaded.  When you try and open, for example, a Word document from the IRM protected SharePoint library all you see is this:

Solution
SharePoint IRM and Windows RMS require that the user’s email address be present for both the user account (AD account or local account) AND for the user’s SharePoint profile.

The reason why the email attribute is so important goes something like this: RMS uses the email address to uniquely identify each user and the first time a user tries to protect content, RMS will provision a client certificate from the RMS server.  So, I guess this infers that without the email attribute, RMS can’t get a certificate for the user and, therefore, doesn’t know if the user can legitimately see the document and, therefore, does not show the document just in case?  Make sense?

Anyway, I had seen/heard somewhere that the email attribute must be present for the user’s Active Directory account.  So, I ensured that my test user had an AD email attribute.  But things still didn’t work (i.e. blank documents as above).  Then, on a call with Microsoft Product Support, we were told that when using SharePoint Information Rights Management (which uses RMS), the user’s SharePoint profile must also have an email address for the “Work  e-mail” attribute as shown here:

After ensuring this e-mail attribute was set then, voila, things started working with IRM protected SharePoint document libraries.  If your environment is using Active Directory and the user’s AD “E-mail” attributes contain email addresses, then using SharePoint profile import will automatically import the email addresses for the SharePoint user profiles.

Related posts:

1. SharePoint Document Management – Folder Discontent
2. SharePoint 2010 Document ID Non-Uniqueness

… one company, based in Paris, used [macro and micro level offshoring diversification] to its advantage… The company was looking to offshore 2,000 specialized, high-end IT jobs and initially planned on sourcing the entire project in India. It opted for an alternative scenario, however, after running the numbers as part of its due diligence. With a view to minimizing its exposure to geographic, currency, and labor issues, the company tiered the work across several locations, placing roughly two-thirds of the project in India and splitting the remaining third across three other regions. It kept 100 jobs in Lille, France, and nearshored 300 more in low-cost Romania, because of the proximity of these locations to European markets. A further 300 were placed in Egypt, where government programs have substantially broadened the talent base. The company then housed the remaining 1,300 roles in Bangalore. By diversifying in this way, the company significantly lowered its overall portfolio risk while incurring only marginally higher costs than it would have under the all-India approach.

Wow!  Such a diversified offshoring model begs lots of questions about the details of the Paris company’s project, products, services, IT roles being outsourced, how it “ran the numbers” as part of its due diligence, etc.  Anyone happen to know the details?  What degree of collaboration (communication, coordination, etc) is required between all those offshore/outsourced centres and the client company in Paris?  Have they fully taken into account all the hidden costs of offshoring?  I would be very surprised if they did not factor in the hidden costs as part of their due diligence exercise.  It’s hard to believe that such a diversified offshoring model could yield the right balance of cost savings and quality.

That excerpt is from McKinsey Quarterly’s recent publication called “Rethinking the model for offshoring services”.  (If you’re on Facebook and become a Fan of the McKinsey Quarterly page your should be able to find and access the full publication … otherwise you will need a premium membership to read the full article from the McKinsey site).

The McKinsey publication makes the case that offshore providers should diversify at both the macro level (geographic … outside of the standard offshoring locations such as India) and micro level (expansion of range of work performed in any one offshore centre) in order to mitigate risk, obtain better cost certainty, and to foster better coordination, flexibility, and responsiveness.  They liken it to the portfolio diversification approach taken by financial managers to manage investment risk.

Sure, diversification as a means to manage risk makes a lot of sense for all kinds of things, but the valuation of a multi-geographic, multi-language, multi-time zone offshoring model obviously needs to account for much more than the valuation of a portfolio of financial instruments.  I sense that the same valuation mistakes will be made as when outsourcing first became popular.  That is not factoring everything into the valuation and thereby overvaluing the diversified offshoring model (especially for offshored software development services).  The publication does make mention of some factors that can diminish the value of offshoring but does not go into a deeper analysis:

Qualitative factors—such as time zone, the suitability of the local skill base, the region’s proximity to key customers, and the existence of government initiatives—also play an important role. Although Eastern European countries are more expensive, for example, they bring strong specialist talent, the requisite language skills, and excellent infrastructures; these factors may often compensate for the higher cost.

Related posts:

1. An Outsourcing “Mess”
2. Rural Sourcing – An Alternative to Outsourcing

ZoomIt is a fantastic little zoom tool from Windows Sysinternals that I regularly use to ensure my audience can clearly see the areas of my demo screens that I’m discussing.  When adeptly used, ZoomIt can add a lot of professional polish to your software demos and help retain your audience’s attention.  I’m constantly amazed at how many people do not use such a tool to present their demos and lose their audience’s focus by presenting busy, hard to see, screens.  Smart people, good presenters, good material, lots of effort put into building the demo, then they apologize for the screen being hard to see.  Come on!  Zoom tools have been around for a while and yet I rarely see them used; I would say less than 10% of the many software demo presentations I’ve attended.

If you present software demos, use ZoomIt or risk having your audience fall asleep, leave, talk, do email, play with their iPhone which they’ll be doing regardless … anything except try to read your demo screen.  It’s easy to become adept at using ZoomIt and it will add professional polish to your demos.

No related posts.

• India: Development centres comprising dozens of developers.
• Canada: Technical management and some development.
• USA: Data center hosting test and production servers.

My exposure to the company was relatively brief but from what I saw and heard the model was a mess.  It was a mess because:

• There were software quality issues and inadequate quality metrics.
• There were network communication latency and availability issues between India and North America.  Acquiring high volumes of software artefacts from India was a time consuming process and there were regular network outages.
• Language and time zone differences created communication issues between India and Canada.
• The huge time zone difference sometimes required Canadian personnel to regularly meet during evening hours when Indian personnel were available.

Pretty standard issues that have plagued outsourcing since it became in vogue.

I am sure some of their quality issues can be addressed with the introduction of better practices and tools.   But I bet that a number of quality issues are directly related to the software development being done in a different country, in a radically different time and “language” zone.

I am not suggesting that all software development outsourcing models are subject to quality issues.  And I am not suggesting that a well thought out strategic approach to developing an outsourcing partnership is going to yield a mess.  However, I’ve been “around” outsourcing enough now to know that if the company in question added up all the hidden costs of their outsourced model (related to poor quality, management/translation overhead, communication, etc), they would likely find the costs savings to be either minimal or non existent compared to a “home sourced” model.

A “home sourced” model includes what I see as the very viable rural sourcing approach as provided by companies such as Rural Sourcing.  Monty Hamilton has recently taken over as CEO, and has big plans to increase and improve the company’s presence and profile.  Rural Sourcing’s labor cost is reported to be about $60/hour, around double what it would cost in India but at least half the cost of using a big consultancy.  So, although the “on paper” labor rate is higher than the Indian labor rate, taking the hidden costs of oursourcing into account should make a rural sourced rate of $60/hour very attractive.  Costs are certainly very important but think of the huge advantages of having your development team based in North America and within a reasonable time zone.  It’s especially advantageous when developing business oriented solutions that require extensive collaboration and communication around the team of peers (and I’m including team leads and management as well in the term “peers”!).  Of course, that comparison will be changing as the Indian market changes to address the current economic conditions and what I’m seeing as an ever increasing propensity by businesses to either not outsource, or to “backshore” their tech shops.

Back to the company in question that’s got itself in an outsourcing “mess”.  I would love to be involved in building the business case to move that company from an Indian based development centre to a rural sourced presence somewhere in North America.  I think it would be a slam dunk win for backshoring.

Related posts:

1. Rural Sourcing – An Alternative to Outsourcing
2. McKinsey Quarterly Offshoring Rethink
3. Zero Feature Release – The First Iteration

You’re having trouble connecting to the internet from your Windows Server (or Vista as I’ve read elsewhere).  You ping an internet site and see the IP address but receive “destination host unreachable”. You go to the command prompt and type “ipconfig /all” to investigate your IP settings.  You see that the default gateway is set to “0.0.0.0” as the first default gateway IP address.  Doesn’t look right, of course.  You check the IP settings and don’t see “0.0.0.0” specified as a default gateway.  What to do?  This is what I did to resolve the problem for my Windows Server 2008 machine:

1. Open a command prompt.
2. Type “route delete 0.0.0.0”.
3. Configure the IP settings to add back your correct default gateway IP address.

That did the trick.

No related posts.

1. Giving end users more powerful desktop-based OLAP capabilities.
2. Giving IT departments the ability to better control organizational data assets.

Desktop-based OLAP capabilities will be achieved as an extension to Microsoft Excel.  Better control of organizational data assets (and spreadmarts) will be achieved via tight integration of Gemini spreadsheets with SharePoint Server.  Better control means better visibility into previously isolated data sets, improved ability to established authoritative data sets and analytics, and improved sharing of information.  Much has been written about Gemini so I’ll leave it to you to Google (or Bing) Gemini.

What I really like about the Gemini project is the approach, or philosophy if you want to call it that.  Rather than “fight” the spreadsheet users, join them.  Someone involved with the Gemini project must have recognized that continuing to slap users on the wrists for contributing to spreadmarts is the wrong approach and counter-productive.  Instead, why not provide an easy mechanism to capture their data/analytics in a controlled server environment without confronting them with policies and lectures that force Excel users “underground” and exacerbate the rifts between IT and the business.  At the same time, embrace the popularity, agility, and ubiquity of Microsoft Excel by providing powerful new OLAP features the further increase a user’s ability to quickly capture, model and analyze business data.

The Gemini philosophy of emphasizing user business agility (desktop OLAP) and centralized data capture (SharePoint integration) over rigid data control mechanisms is a refreshing approach.  An approach that will hopefully see wider adoption in IT.  I wouldn’t be surprised to see Microsoft address the proliferation of Access databases in the same manner.

Not to be treated lightly is the area of records management and the need to ensure that organizational records/data (whether they exist in Excel spreadsheets or Access databases) are governed in an effective and timely manner.  Gemini helps facilitate a solution to that problem for OLAP data by providing tight integration with SharePoint where information management policies can be applied to published Gemini spreadsheets.  But it doesn’t solve the entire problem.  If Gemini spreadsheets contain records that need to be governed by disposition policies, and the users do not publish those spreadsheets to SharePoint … well … you obviously continue to face the risks of having uncontrolled data scattered about the company on users’ desktops.

Update (Sep 11, 2009):

Found this suite of YouTube videos explaining Gemini: http://www.youtube.com/user/geminute. I haven’t watched any of them yet.  Drop me a comment and let know what you think.

Related posts:

1. Microsoft Business Intelligence Conference 2008

• Documents are scattered deep and wide in a vast hierarchy of folders across many shared folder locations.
  
  
• Duplicate content is everywhere.
  
  
• Organizational records become buried amongst the shared folders and are overlooked by information management policies that govern things such as the record’s retention policy.
  
  
• Security holes are common because access rights are not propagated to sub-folders leaving sensitive content exposed.
  
  
• In many cases, a shared folder hierarchy represents an department’s or an organization’s content taxonomy.  A document’s presence in a shared folder hierarchy is implicitly “tagged” with metadata being the names of the folders in the hierarchy.  Problem is, when the document is copied or moved, the metadata (i.e. the names of the folders) does not travel with the document.
  
  
• Finding information can be a time consuming and costly endeavour.  Yes, costly; see “The high cost of not finding information”.

It’s obviously a no brainer to move those organizations from shared folders to a document management system.  And many companies are either using (or thinking about using) SharePoint as a document management solution.

You might think that using SharePoint 2007 as a document management system will cure all the shared folder ills that I highlighted above.  Wrong.  SharePoint 2007 has a core performance limitation that will require you to use folders (or indexed views) when the number of documents in a single view starts to exceed 2000.  The same applies to SharePoint lists.

Tests show that the performance of libraries starts tanking as the number of documents/items in a library view approaches and exceeds 2000.  The recommended way to avoid the performance degradation trap is to create folders within libraries to break down the documents into sub-2000 document folders.  The TechNet article “Plan for software boundaries (Office SharePoint Server)” talks in detail about this performance limitation and how to deal with it using folders.  It very clearly states that folders are “critical for scaling“.  So, unfortunately, if you expect to store more than 2000 documents in a library you’re faced with either:

1. Using folders in a document library and having to deal with many of the same problems as dealing with server-based shared folders.
   
   
2. Using indexed views (views that used indexed columns).  Helps improve performance for libraries with 2000+ documents but performance is not as good as using folders.
   
   
3. Using multiple document libraries that store no more than 2000 documents.

Options 2 (indexed views) and 3 (multiple document libraries) may alleviate the need to use folders.  However they potentially force you to deviate from your information architecture (IA) by using multiple libraries and views that do not conform to IA requirements for the organization and presentation of content.  Those options can also introduce additional administrative overhead managing the many libraries, views, and indexed columns.

Ultimately, if an organization needs to store many thousands of documents in SharePoint 2007 they will likely have to break down the libraries into hierarchies of sub-folders.  And now you’re back to square one dealing with folder hell.  Will the next version of SharePoint address the performance limitation and eliminate the need for folders?

Related posts:

1. SharePoint 2010 Document ID Non-Uniqueness
2. SharePoint IRM Issue Opening Documents