Archive

The Gemini Approach

In a nutshell the Microsoft Gemini project is about two things:

  1. Giving end users more powerful desktop-based OLAP capabilities.
  2. Giving IT departments the ability to better control organizational data assets.

Desktop-based OLAP capabilities will be achieved as an extension to Microsoft Excel.  Better control of organizational data assets (and spreadmarts) will be achieved via tight integration of Gemini spreadsheets with SharePoint Server.  Better control means better visibility into previously isolated data sets, improved ability to established authoritative data sets and analytics, and improved sharing of information.  Much has been written about Gemini so I’ll leave it to you to Google (or Bing) Gemini.

What I really like about the Gemini project is the approach, or philosophy if you want to call it that.  Rather than “fight” the spreadsheet users, join them.  Someone involved with the Gemini project must have recognized that continuing to slap users on the wrists for contributing to spreadmarts is the wrong approach and counter-productive.  Instead, why not provide an easy mechanism to capture their data/analytics in a controlled server environment without confronting them with policies and lectures that force Excel users “underground” and exacerbate the rifts between IT and the business.  At the same time, embrace the popularity, agility, and ubiquity of Microsoft Excel by providing powerful new OLAP features the further increase a user’s ability to quickly capture, model and analyze business data.

The Gemini philosophy of emphasizing user business agility (desktop OLAP) and centralized data capture (SharePoint integration) over rigid data control mechanisms is a refreshing approach.  An approach that will hopefully see wider adoption in IT.  I wouldn’t be surprised to see Microsoft address the proliferation of Access databases in the same manner.

Not to be treated lightly is the area of records management and the need to ensure that organizational records/data (whether they exist in Excel spreadsheets or Access databases) are governed in an effective and timely manner.  Gemini helps facilitate a solution to that problem for OLAP data by providing tight integration with SharePoint where information management policies can be applied to published Gemini spreadsheets.  But it doesn’t solve the entire problem.  If Gemini spreadsheets contain records that need to be governed by disposition policies, and the users do not publish those spreadsheets to SharePoint … well … you obviously continue to face the risks of having uncontrolled data scattered about the company on users’ desktops.

Update (Sep 11, 2009):

Found this suite of YouTube videos explaining Gemini: http://www.youtube.com/user/geminute. I haven’t watched any of them yet.  Drop me a comment and let know what you think.

Technorati Logo , , , , ,

SharePoint Document Management - Folder Discontent

A large number of organizations still live in shared folder hell, plagued by the following issues:

  • Documents are scattered deep and wide in a vast hierarchy of folders across many shared folder locations.

  • Duplicate content is everywhere.

  • Organizational records become buried amongst the shared folders and are overlooked by information management policies that govern things such as the record’s retention policy.

  • Security holes are common because access rights are not propagated to sub-folders leaving sensitive content exposed.

  • In many cases, a shared folder hierarchy represents an department’s or an organization’s content taxonomy.  A document’s presence in a shared folder hierarchy is implicitly “tagged” with metadata being the names of the folders in the hierarchy.  Problem is, when the document is copied or moved, the metadata (i.e. the names of the folders) does not travel with the document.

  • Finding information can be a time consuming and costly endeavour.  Yes, costly; see “The high cost of not finding information”.

It’s obviously a no brainer to move those organizations from shared folders to a document management system.  And many companies are either using (or thinking about using) SharePoint as a document management solution.

You might think that using SharePoint 2007 as a document management system will cure all the shared folder ills that I highlighted above.  Wrong.  SharePoint 2007 has a core performance limitation that will require you to use folders (or indexed views) when the number of documents in a single view starts to exceed 2000.  The same applies to SharePoint lists.

Tests show that the performance of libraries starts tanking as the number of documents/items in a library view approaches and exceeds 2000.  The recommended way to avoid the performance degradation trap is to create folders within libraries to break down the documents into sub-2000 document folders.  The TechNet article “Plan for software boundaries (Office SharePoint Server)” talks in detail about this performance limitation and how to deal with it using folders.  It very clearly states that folders are “critical for scaling“.  So, unfortunately, if you expect to store more than 2000 documents in a library you’re faced with either:

  1. Using folders in a document library and having to deal with many of the same problems as dealing with server-based shared folders.

  2. Using indexed views (views that used indexed columns).  Helps improve performance for libraries with 2000+ documents but performance is not as good as using folders.

  3. Using multiple document libraries that store no more than 2000 documents.

Options 2 (indexed views) and 3 (multiple document libraries) may alleviate the need to use folders.  However they potentially force you to deviate from your information architecture (IA) by using multiple libraries and views that do not conform to IA requirements for the organization and presentation of content.  Those options can also introduce additional administrative overhead managing the many libraries, views, and indexed columns.

Ultimately, if an organization needs to store many thousands of documents in SharePoint 2007 they will likely have to break down the libraries into hierarchies of sub-folders.  And now you’re back to square one dealing with folder hell.  Will the next version of SharePoint address the performance limitation and eliminate the need for folders?

Technorati Logo , , ,

Visual Communication Model

Since coming across Jame’s post about Idiagram I’ve been thinking about how Marshall’s approach to complex problem solving and visual modeling might help contribute to solving corporate communication problems.  One of my biggest frustrations in the workplace is poor communication.  It destroys morale, reduces quality, kills teamwork, etc.  No mystery to those of you who also understand the critical role that good communication plays in maintaining high performance teams.  It would be interesting to apply Marshall’s approach, along with his visual modeling style, towards the corporate  communication problem space, for both external (public) and internal (workplace) communication.  How do you think the communication problem space could be visually modeled given the approach you’ve seen from Idiagram?  If I’m feeling creative one day I may make the brave attempt to draft up a visual model for internal employee communication that encompasses things like:

  • Speaking and writing clearly.
  • Providing complete information.
  • Credibility: your actions match your words.
  • Clear and timely communication of issues.
  • Responsiveness.
  • Open communication at all levels.

The idea would be to illustrate how these communication characteristics tie into high performance teams, professional development, and profitability.

Cloud Thoughts

Since the days of the first SaaS (Software as a Service) entrants the idea of provisioning computing resources from external hosting providers has clearly become mainstream.  Search for “Cloudsourcing”, “Cloud Computing” or whatever you call it and you’ll see a wealth of information and a wide selection of mature and incumbent offerings.

From a technical and architectural perspective there is much to hold my interest.  Architecturally, the cloud paradigms bring latency and other considerations to the forefront that generally did not figure strongly for in-house architectures.  An architecture that entertains cloud storage, for example, clearly cannot accommodate regular high volume data requests and expect to be responsive.  That does not mean that cloud-based storage should be discounted.  Perhaps a hybrid architecture provides the best business value where large volumes of archived data are hosted in the cloud, with day-to-day operational data retained in-house.  Such a hybrid data services model is put forward by Eugenio Pace and Gianpaolo Carraro in their article “Head in the Cloud, Feet on the Ground” in the October 2008 edition of the Microsoft Architecture Journal.  Their article is definitely worth a read and I’ll be referring to it later on in this post.

The surge in Cloud Computing, like many IT paradigm surges, will incite a gamut of opinions and assessments.  Some will hail Cloud Computing as a revolutionary, game changing force.  Some will see it as yet another distributed computing model.  Like others, I see it as evolutionary, a natural progression of distributed computing capabilities that reflects the evolution of hardware, networks, standards, computing economics, etc.  Cloud Computing is significant and will have a profound impact for the following reasons: provisioning, sustainment, scaling, and cost.  Not having to buy hardware and software, install it, scale it, patch it, cool it, upgrade it, fix it, hire Systems Engineers, etc, is a big deal for many businesses.  Start-ups need not get side tracked mucking around provisioning and sustaining their own hardware and software and can, instead, apply those cycles to what will be their core competencies.  Established businesses may be able to reduce computing costs and improve service levels by leveraging the Cloud.  And therein lies my primary Cloud Computing interest at the moment: making sense of the Cloud from a business value perspective (i.e. understanding the value proposition).  The subject of value, especially for enterprise-class customers, is addressed in the Sand Hill Group article “Capitalizing on the Cloud“.

When to go to the Cloud (the business decision) and how to go to the Cloud (the planning and architectural decisions) are obvious questions that require careful analysis.  No doubt some will rush to the Cloud based on a cursory analysis of costs in a similar way that many rushed to outsource without accounting for the hidden costs.  I recall the outsourcing rush that infected and pained the investment bank I was working for at the time.  Senior execs mandated to IT managers that 25% of IT projects be outsourced.  On paper it all looked too good to be true and it was.  Outsourced developers cost a fraction of the rate of a London-based developer.  Simply engage offshore developers to build the application, reap the savings and collect your big banking bonus.  Anyone familiar with IT outsourcing knows how the story ended for many businesses bitten by the outsourcing bug: the hidden costs of additional management overhead, communication issues, quality issues, etc, quickly negated the labour rate savings and, in many cases, increased costs for many projects.

Making sense of Cloud Computing and its value to your business will require an understanding and analysis of more than just costs and architecture.  On paper, compute time rates, data storage rates, and software usage rates may seem attractive.  Quick calculations may illustrate savings.  But what about regulations that govern data storage (e.g. healthcare regulations, the US Patriot Act)?  Do regulations governing your business require that data be stored within your national boundaries?  If so, are there any Cloud data storage providers that will provide geographically constrained storage to ensure your data is physically stored within your national borders?  Amazon have introduced Availability Zones for their Cloud Compute (EC2) offering that let you specify distinct locations for your compute server instances.  Their Cloud storage offering (S3) will likely follow suite allowing you to confine data storage to specific geographic regions.

In addition to regulatory considerations, there are many other factors that need to be understood before migrating applications to the Cloud.  For mission critical applications that are architectural candidates for the Cloud, an assessment of Cloud providers will require an understanding of how they support high availability and disaster recovery.  High availability considerations not only touch on the Cloud but also on an organization’s Internet connectivity.  High availability requirements for Cloud applications will require redundant Internet connectivity on the organization’s part in addition to the high availability infrastructure provided by the Cloud provider.  When it comes to disaster recovery perhaps the Cloud will provide better value.  Many organizations either don’t have a proper DR plan or have not tested the plan sufficiently, exposing the company to an elevated risk of being unable to survive a disaster scenario.  A solid Cloud provider will have a solid and fully tested DR plan since DR will be (or should be) one of their core competencies.

For a good architectural perspective on Cloud opportunities, I recommend the article “Head in the Cloud, Feet on the Ground” by Eugenio Pace and Gianpaolo Carraro.  The bulk of their article focuses on what they call localized optimization through selective specialization (LOtSS).  LOtSS is concerned with lowering software architecture “transloading” costs; the costs associated with managing the architecture across the enterprise-cloud boundary.  They point out that security, application management, and integration are typical aspects that can incur unacceptable transloading costs.  Organizations that are planning on adopting Cloud Computing will need to account for those transloading costs to effectively gauge the value of Cloud architectures.

The successful adoption of Cloud Computing to align with and support business strategies will obviously need to be backed by a value-focused plan which should include the assessment of architectural transloading costs.  The plan could take the form of an adoption model such as “The Cloud Computing Adoption Model” which is loosely based on the Capability Maturity Model (CMM) with five maturity levels: Virtualization, Cloud Experimentation, Cloud Foundations, Cloud Advancement, and Cloud Actualization.  Planning, adoption models, guidance, road maps, and assessments will be areas of continued and future focus as I work towards more formalization of Cloud Computing approaches and best practices for our customers.

Technorati Logo ,

Microsoft Business Intelligence Conference 2008

microsoft_bi_conf_2008 

Business intelligence (BI) refers to technologies, applications and practices for the collection, integration, analysis, and presentation of business information and sometimes to the information itself.
Wikipedia
BI systems provide historical, current, and predictive views of business operations, most often using data that has been gathered into a data warehouse or a data mart and occasionally working from operational data. Software elements support the use of this information by assisting in the extraction, analysis, and reporting of information.
Wikipedia

I’m wayyyyyyy overdue in getting my blurb published about my attendance at the Microsoft BI Conference.  Thousands have already beat me to it but, nevertheless, below is my summary.

Technology takeaways from the 2008 Microsoft Business Intelligence Conference:

  • Project Gemini.  Microsoft’s “self service” analytical capabilities that add significant data processing and analytical power to the Excel client experience.  Gemini reflects Microsoft’s ongoing efforts to surface integrated BI capabilities to users in a natural manner where users aren’t always aware they’re working with, or doing, BI.  Also part of Gemini is the ability to publish the resulting cube to a “Gemini enabled” SharePoint library for sharing, social-style interaction, and performance and usage analysis.  This is a significant development along the road of mining those important ad-hoc Excel data analysis sheets that breed and spread like bacteria across an organization.  And, with Gemini, they are mined in a very non-intrusive manner.
  • SQL Server Kilimanjaro.  The next release of SQL Server slated for delivery during the first half of 2010.  Kilimanjaro will include Gemini and self-service reporting.  Self-service reporting is all about giving workgroups/teams the ability to self-service their reporting needs and to provision security for their reports.  Sharing report components, “mashing up” reports, and getting notified about changes to shared components, are part of the self-service reporting initiative.
  • Project Madison.  Advanced data warehousing capabilities via the integration of DATAllegro’s technology into SQL Server.  (DATAllegro is a recent Microsoft acquisition.)  Project Madison will allow SQL Server to scale to 100s of terabytes of data with, it is claimed, exceptional data loading and query performance.
  • The Microsoft Master Data Management (MDM) stack.  Microsoft envision an integrated MDM stack to include an MDM application platform and domain-specific MDM solutions provided by partners.  The plan is to ship the MDM platform as part of Microsoft Office SharePoint Server.  A working product is expected for next year’s Business Intelligence Conference.

A couple of non-technology takeaways:

  • Requirements for successful business intelligence and performance management solutions:
    1. A culture of performance.
    2. A culture driven by business value.
    3. A culture of continuous improvement.
    4. Incremental delivery.

    This really isn’t a takeaway at all because it’s just common sense.  Still, many organizations fail to successfully deliver their BI and performance management initiatives (and many other types of IT projects) due to cultural issues.  In many cases blame is heaped on the technology or the technologists.  Inevitably, properly conducted  project retrospectives will highlight that certain leaders, stakeholders and consultants are accountable for not prioritizing their focus on the business climate and business drivers.  Without a wide spread cultural shift in support of change (backed, of course, by sound business cases), most projects will fail.

  • The importance of deploying your performance management solution to your partner ecosystem to support those that contribute to your bottom line.  This speaks to the concept of pervasive performance management which is “enterprise-wide, enterprise deep, and outside-in, and includes unstructured data which is becoming an important component of the BI puzzle“.
Technorati Logo , , , , ,



Bad Behavior has blocked 81 access attempts in the last 7 days.