In a nutshell the Microsoft Gemini project is about two things:
- Giving end users more powerful desktop-based OLAP capabilities.
- Giving IT departments the ability to better control organizational data assets.
Desktop-based OLAP capabilities will be achieved as an extension to Microsoft Excel. Better control of organizational data assets (and spreadmarts) will be achieved via tight integration of Gemini spreadsheets with SharePoint Server. Better control means better visibility into previously isolated data sets, improved ability to established authoritative data sets and analytics, and improved sharing of information. Much has been written about Gemini so I’ll leave it to you to Google (or Bing) Gemini.
What I really like about the Gemini project is the approach, or philosophy if you want to call it that. Rather than “fight” the spreadsheet users, join them. Someone involved with the Gemini project must have recognized that continuing to slap users on the wrists for contributing to spreadmarts is the wrong approach and counter-productive. Instead, why not provide an easy mechanism to capture their data/analytics in a controlled server environment without confronting them with policies and lectures that force Excel users “underground” and exacerbate the rifts between IT and the business. At the same time, embrace the popularity, agility, and ubiquity of Microsoft Excel by providing powerful new OLAP features the further increase a user’s ability to quickly capture, model and analyze business data.
The Gemini philosophy of emphasizing user business agility (desktop OLAP) and centralized data capture (SharePoint integration) over rigid data control mechanisms is a refreshing approach. An approach that will hopefully see wider adoption in IT. I wouldn’t be surprised to see Microsoft address the proliferation of Access databases in the same manner.
Not to be treated lightly is the area of records management and the need to ensure that organizational records/data (whether they exist in Excel spreadsheets or Access databases) are governed in an effective and timely manner. Gemini helps facilitate a solution to that problem for OLAP data by providing tight integration with SharePoint where information management policies can be applied to published Gemini spreadsheets. But it doesn’t solve the entire problem. If Gemini spreadsheets contain records that need to be governed by disposition policies, and the users do not publish those spreadsheets to SharePoint … well … you obviously continue to face the risks of having uncontrolled data scattered about the company on users’ desktops.
Update (Sep 11, 2009):
Found this suite of YouTube videos explaining Gemini: http://www.youtube.com/user/geminute. I haven’t watched any of them yet. Drop me a comment and let know what you think.
A large number of organizations still live in shared folder hell, plagued by the following issues:
- Documents are scattered deep and wide in a vast hierarchy of folders across many shared folder locations.
- Duplicate content is everywhere.
- Organizational records become buried amongst the shared folders and are overlooked by information management policies that govern things such as the record’s retention policy.
- Security holes are common because access rights are not propagated to sub-folders leaving sensitive content exposed.
- In many cases, a shared folder hierarchy represents an department’s or an organization’s content taxonomy. A document’s presence in a shared folder hierarchy is implicitly “tagged” with metadata being the names of the folders in the hierarchy. Problem is, when the document is copied or moved, the metadata (i.e. the names of the folders) does not travel with the document.
- Finding information can be a time consuming and costly endeavour. Yes, costly; see “The high cost of not finding information”.
It’s obviously a no brainer to move those organizations from shared folders to a document management system. And many companies are either using (or thinking about using) SharePoint as a document management solution.
You might think that using SharePoint 2007 as a document management system will cure all the shared folder ills that I highlighted above. Wrong. SharePoint 2007 has a core performance limitation that will require you to use folders (or indexed views) when the number of documents in a single view starts to exceed 2000. The same applies to SharePoint lists.
Tests show that the performance of libraries starts tanking as the number of documents/items in a library view approaches and exceeds 2000. The recommended way to avoid the performance degradation trap is to create folders within libraries to break down the documents into sub-2000 document folders. The TechNet article “Plan for software boundaries (Office SharePoint Server)” talks in detail about this performance limitation and how to deal with it using folders. It very clearly states that folders are “critical for scaling“. So, unfortunately, if you expect to store more than 2000 documents in a library you’re faced with either:
- Using folders in a document library and having to deal with many of the same problems as dealing with server-based shared folders.
- Using indexed views (views that used indexed columns). Helps improve performance for libraries with 2000+ documents but performance is not as good as using folders.
- Using multiple document libraries that store no more than 2000 documents.
Options 2 (indexed views) and 3 (multiple document libraries) may alleviate the need to use folders. However they potentially force you to deviate from your information architecture (IA) by using multiple libraries and views that do not conform to IA requirements for the organization and presentation of content. Those options can also introduce additional administrative overhead managing the many libraries, views, and indexed columns.
Ultimately, if an organization needs to store many thousands of documents in SharePoint 2007 they will likely have to break down the libraries into hierarchies of sub-folders. And now you’re back to square one dealing with folder hell. Will the next version of SharePoint address the performance limitation and eliminate the need for folders?
