The Gemini Approach

In a nutshell the Microsoft Gemini project is about two things:

  1. Giving end users more powerful desktop-based OLAP capabilities.
  2. Giving IT departments the ability to better control organizational data assets.

Desktop-based OLAP capabilities will be achieved as an extension to Microsoft Excel.  Better control of organizational data assets (and spreadmarts) will be achieved via tight integration of Gemini spreadsheets with SharePoint Server.  Better control means better visibility into previously isolated data sets, improved ability to established authoritative data sets and analytics, and improved sharing of information.  Much has been written about Gemini so I’ll leave it to you to Google (or Bing) Gemini.

What I really like about the Gemini project is the approach, or philosophy if you want to call it that.  Rather than “fight” the spreadsheet users, join them.  Someone involved with the Gemini project must have recognized that continuing to slap users on the wrists for contributing to spreadmarts is the wrong approach and counter-productive.  Instead, why not provide an easy mechanism to capture their data/analytics in a controlled server environment without confronting them with policies and lectures that force Excel users “underground” and exacerbate the rifts between IT and the business.  At the same time, embrace the popularity, agility, and ubiquity of Microsoft Excel by providing powerful new OLAP features the further increase a user’s ability to quickly capture, model and analyze business data.

The Gemini philosophy of emphasizing user business agility (desktop OLAP) and centralized data capture (SharePoint integration) over rigid data control mechanisms is a refreshing approach.  An approach that will hopefully see wider adoption in IT.  I wouldn’t be surprised to see Microsoft address the proliferation of Access databases in the same manner.

Not to be treated lightly is the area of records management and the need to ensure that organizational records/data (whether they exist in Excel spreadsheets or Access databases) are governed in an effective and timely manner.  Gemini helps facilitate a solution to that problem for OLAP data by providing tight integration with SharePoint where information management policies can be applied to published Gemini spreadsheets.  But it doesn’t solve the entire problem.  If Gemini spreadsheets contain records that need to be governed by disposition policies, and the users do not publish those spreadsheets to SharePoint … well … you obviously continue to face the risks of having uncontrolled data scattered about the company on users’ desktops.

Technorati Logo , , , , ,

SharePoint Document Management - Folder Discontent

A large number of organizations still live in shared folder hell, plagued by the following issues:

  • Documents are scattered deep and wide in a vast hierarchy of folders across many shared folder locations.

  • Duplicate content is everywhere.

  • Organizational records become buried amongst the shared folders and are overlooked by information management policies that govern things such as the record’s retention policy.

  • Security holes are common because access rights are not propagated to sub-folders leaving sensitive content exposed.

  • In many cases, a shared folder hierarchy represents an department’s or an organization’s content taxonomy.  A document’s presence in a shared folder hierarchy is implicitly “tagged” with metadata being the names of the folders in the hierarchy.  Problem is, when the document is copied or moved, the metadata (i.e. the names of the folders) does not travel with the document.

  • Finding information can be a time consuming and costly endeavour.  Yes, costly; see “The high cost of not finding information”.

It’s obviously a no brainer to move those organizations from shared folders to a document management system.  And many companies are either using (or thinking about using) SharePoint as a document management solution.

You might think that using SharePoint 2007 as a document management system will cure all the shared folder ills that I highlighted above.  Wrong.  SharePoint 2007 has a core performance limitation that will require you to use folders (or indexed views) when the number of documents in a single view starts to exceed 2000.  The same applies to SharePoint lists.

Tests show that the performance of libraries starts tanking as the number of documents/items in a library view approaches and exceeds 2000.  The recommended way to avoid the performance degradation trap is to create folders within libraries to break down the documents into sub-2000 document folders.  The TechNet article “Plan for software boundaries (Office SharePoint Server)” talks in detail about this performance limitation and how to deal with it using folders.  It very clearly states that folders are “critical for scaling“.  So, unfortunately, if you expect to store more than 2000 documents in a library you’re faced with either:

  1. Using folders in a document library and having to deal with many of the same problems as dealing with server-based shared folders.

  2. Using indexed views (views that used indexed columns).  Helps improve performance for libraries with 2000+ documents but performance is not as good as using folders.

  3. Using multiple document libraries that store no more than 2000 documents.

Options 2 (indexed views) and 3 (multiple document libraries) may alleviate the need to use folders.  However they potentially force you to deviate from your information architecture (IA) by using multiple libraries and views that do not conform to IA requirements for the organization and presentation of content.  Those options can also introduce additional administrative overhead managing the many libraries, views, and indexed columns.

Ultimately, if an organization needs to store many thousands of documents in SharePoint 2007 they will likely have to break down the libraries into hierarchies of sub-folders.  And now you’re back to square one dealing with folder hell.  Will the next version of SharePoint address the performance limitation and eliminate the need for folders?

Technorati Logo , , ,

Visual Communication Model

Since coming across Jame’s post about Idiagram I’ve been thinking about how Marshall’s approach to complex problem solving and visual modeling might help contribute to solving corporate communication problems.  One of my biggest frustrations in the workplace is poor communication.  It destroys morale, reduces quality, kills teamwork, etc.  No mystery to those of you who also understand the critical role that good communication plays in maintaining high performance teams.  It would be interesting to apply Marshall’s approach, along with his visual modeling style, towards the corporate  communication problem space, for both external (public) and internal (workplace) communication.  How do you think the communication problem space could be visually modeled given the approach you’ve seen from Idiagram?  If I’m feeling creative one day I may make the brave attempt to draft up a visual model for internal employee communication that encompasses things like:

  • Speaking and writing clearly.
  • Providing complete information.
  • Credibility: your actions match your words.
  • Clear and timely communication of issues.
  • Responsiveness.
  • Open communication at all levels.

The idea would be to illustrate how these communication characteristics tie into high performance teams, professional development, and profitability.

Cloud Thoughts

Since the days of the first SaaS (Software as a Service) entrants the idea of provisioning computing resources from external hosting providers has clearly become mainstream.  Search for “Cloudsourcing”, “Cloud Computing” or whatever you call it and you’ll see a wealth of information and a wide selection of mature and incumbent offerings.

From a technical and architectural perspective there is much to hold my interest.  Architecturally, the cloud paradigms bring latency and other considerations to the forefront that generally did not figure strongly for in-house architectures.  An architecture that entertains cloud storage, for example, clearly cannot accommodate regular high volume data requests and expect to be responsive.  That does not mean that cloud-based storage should be discounted.  Perhaps a hybrid architecture provides the best business value where large volumes of archived data are hosted in the cloud, with day-to-day operational data retained in-house.  Such a hybrid data services model is put forward by Eugenio Pace and Gianpaolo Carraro in their article “Head in the Cloud, Feet on the Ground” in the October 2008 edition of the Microsoft Architecture Journal.  Their article is definitely worth a read and I’ll be referring to it later on in this post.

The surge in Cloud Computing, like many IT paradigm surges, will incite a gamut of opinions and assessments.  Some will hail Cloud Computing as a revolutionary, game changing force.  Some will see it as yet another distributed computing model.  Like others, I see it as evolutionary, a natural progression of distributed computing capabilities that reflects the evolution of hardware, networks, standards, computing economics, etc.  Cloud Computing is significant and will have a profound impact for the following reasons: provisioning, sustainment, scaling, and cost.  Not having to buy hardware and software, install it, scale it, patch it, cool it, upgrade it, fix it, hire Systems Engineers, etc, is a big deal for many businesses.  Start-ups need not get side tracked mucking around provisioning and sustaining their own hardware and software and can, instead, apply those cycles to what will be their core competencies.  Established businesses may be able to reduce computing costs and improve service levels by leveraging the Cloud.  And therein lies my primary Cloud Computing interest at the moment: making sense of the Cloud from a business value perspective (i.e. understanding the value proposition).  The subject of value, especially for enterprise-class customers, is addressed in the Sand Hill Group article “Capitalizing on the Cloud“.

When to go to the Cloud (the business decision) and how to go to the Cloud (the planning and architectural decisions) are obvious questions that require careful analysis.  No doubt some will rush to the Cloud based on a cursory analysis of costs in a similar way that many rushed to outsource without accounting for the hidden costs.  I recall the outsourcing rush that infected and pained the investment bank I was working for at the time.  Senior execs mandated to IT managers that 25% of IT projects be outsourced.  On paper it all looked too good to be true and it was.  Outsourced developers cost a fraction of the rate of a London-based developer.  Simply engage offshore developers to build the application, reap the savings and collect your big banking bonus.  Anyone familiar with IT outsourcing knows how the story ended for many businesses bitten by the outsourcing bug: the hidden costs of additional management overhead, communication issues, quality issues, etc, quickly negated the labour rate savings and, in many cases, increased costs for many projects.

Making sense of Cloud Computing and its value to your business will require an understanding and analysis of more than just costs and architecture.  On paper, compute time rates, data storage rates, and software usage rates may seem attractive.  Quick calculations may illustrate savings.  But what about regulations that govern data storage (e.g. healthcare regulations, the US Patriot Act)?  Do regulations governing your business require that data be stored within your national boundaries?  If so, are there any Cloud data storage providers that will provide geographically constrained storage to ensure your data is physically stored within your national borders?  Amazon have introduced Availability Zones for their Cloud Compute (EC2) offering that let you specify distinct locations for your compute server instances.  Their Cloud storage offering (S3) will likely follow suite allowing you to confine data storage to specific geographic regions.

In addition to regulatory considerations, there are many other factors that need to be understood before migrating applications to the Cloud.  For mission critical applications that are architectural candidates for the Cloud, an assessment of Cloud providers will require an understanding of how they support high availability and disaster recovery.  High availability considerations not only touch on the Cloud but also on an organization’s Internet connectivity.  High availability requirements for Cloud applications will require redundant Internet connectivity on the organization’s part in addition to the high availability infrastructure provided by the Cloud provider.  When it comes to disaster recovery perhaps the Cloud will provide better value.  Many organizations either don’t have a proper DR plan or have not tested the plan sufficiently, exposing the company to an elevated risk of being unable to survive a disaster scenario.  A solid Cloud provider will have a solid and fully tested DR plan since DR will be (or should be) one of their core competencies.

For a good architectural perspective on Cloud opportunities, I recommend the article “Head in the Cloud, Feet on the Ground” by Eugenio Pace and Gianpaolo Carraro.  The bulk of their article focuses on what they call localized optimization through selective specialization (LOtSS).  LOtSS is concerned with lowering software architecture “transloading” costs; the costs associated with managing the architecture across the enterprise-cloud boundary.  They point out that security, application management, and integration are typical aspects that can incur unacceptable transloading costs.  Organizations that are planning on adopting Cloud Computing will need to account for those transloading costs to effectively gauge the value of Cloud architectures.

The successful adoption of Cloud Computing to align with and support business strategies will obviously need to be backed by a value-focused plan which should include the assessment of architectural transloading costs.  The plan could take the form of an adoption model such as “The Cloud Computing Adoption Model” which is loosely based on the Capability Maturity Model (CMM) with five maturity levels: Virtualization, Cloud Experimentation, Cloud Foundations, Cloud Advancement, and Cloud Actualization.  Planning, adoption models, guidance, road maps, and assessments will be areas of continued and future focus as I work towards more formalization of Cloud Computing approaches and best practices for our customers.

Technorati Logo ,

Microsoft Business Intelligence Conference 2008

microsoft_bi_conf_2008 

Business intelligence (BI) refers to technologies, applications and practices for the collection, integration, analysis, and presentation of business information and sometimes to the information itself.
Wikipedia
BI systems provide historical, current, and predictive views of business operations, most often using data that has been gathered into a data warehouse or a data mart and occasionally working from operational data. Software elements support the use of this information by assisting in the extraction, analysis, and reporting of information.
Wikipedia

I’m wayyyyyyy overdue in getting my blurb published about my attendance at the Microsoft BI Conference.  Thousands have already beat me to it but, nevertheless, below is my summary.

Technology takeaways from the 2008 Microsoft Business Intelligence Conference:

  • Project Gemini.  Microsoft’s “self service” analytical capabilities that add significant data processing and analytical power to the Excel client experience.  Gemini reflects Microsoft’s ongoing efforts to surface integrated BI capabilities to users in a natural manner where users aren’t always aware they’re working with, or doing, BI.  Also part of Gemini is the ability to publish the resulting cube to a “Gemini enabled” SharePoint library for sharing, social-style interaction, and performance and usage analysis.  This is a significant development along the road of mining those important ad-hoc Excel data analysis sheets that breed and spread like bacteria across an organization.  And, with Gemini, they are mined in a very non-intrusive manner.
  • SQL Server Kilimanjaro.  The next release of SQL Server slated for delivery during the first half of 2010.  Kilimanjaro will include Gemini and self-service reporting.  Self-service reporting is all about giving workgroups/teams the ability to self-service their reporting needs and to provision security for their reports.  Sharing report components, “mashing up” reports, and getting notified about changes to shared components, are part of the self-service reporting initiative.
  • Project Madison.  Advanced data warehousing capabilities via the integration of DATAllegro’s technology into SQL Server.  (DATAllegro is a recent Microsoft acquisition.)  Project Madison will allow SQL Server to scale to 100s of terabytes of data with, it is claimed, exceptional data loading and query performance.
  • The Microsoft Master Data Management (MDM) stack.  Microsoft envision an integrated MDM stack to include an MDM application platform and domain-specific MDM solutions provided by partners.  The plan is to ship the MDM platform as part of Microsoft Office SharePoint Server.  A working product is expected for next year’s Business Intelligence Conference.

A couple of non-technology takeaways:

  • Requirements for successful business intelligence and performance management solutions:
    1. A culture of performance.
    2. A culture driven by business value.
    3. A culture of continuous improvement.
    4. Incremental delivery.

    This really isn’t a takeaway at all because it’s just common sense.  Still, many organizations fail to successfully deliver their BI and performance management initiatives (and many other types of IT projects) due to cultural issues.  In many cases blame is heaped on the technology or the technologists.  Inevitably, properly conducted  project retrospectives will highlight that certain leaders, stakeholders and consultants are accountable for not prioritizing their focus on the business climate and business drivers.  Without a wide spread cultural shift in support of change (backed, of course, by sound business cases), most projects will fail.

  • The importance of deploying your performance management solution to your partner ecosystem to support those that contribute to your bottom line.  This speaks to the concept of pervasive performance management which is “enterprise-wide, enterprise deep, and outside-in, and includes unstructured data which is becoming an important component of the BI puzzle“.
Technorati Logo , , , , ,

Adobe Photoshop Scripting

I’ve used Adobe Photoshop for a while now and never really progressed beyond the basics.  That is, until recently, when faced with the challenge of putting together a collage image for an Enterprise Search presentation that I’m working on.  The idea for the image was not mine.  I saw the document icon collage idea when researching online material.  Of course, I sought to acquire an acceptable copy of the image but, to no avail.  So, I designed my own, shown here:

document_icon_collage_med

Clearly, with very basic Photoshop skills in hand, the prospect of inserting and aligning different sized document icons, in random order, was daunting to say the least.  And, even worse would be the prospect of having to redo the layout if either the results were crap, or different icons or icon arrangements were needed.  One could easily end up going blind and getting carpel tunnel syndrome.

Automation was needed and this is where Adobe’s ExtendScript Toolkit saved the day.  The toolkit provides a scripting editor where you can write ExtendScript to manipulate and build a Photoshop image.  ExtendScript is Adobe’s extended implementation of JavaScript, so anyone familiar with JavaScript will have no problems writing ExtendScript.

So, putting my geek hat on, and with the ExtendScript Toolkit to hand, my high-level approach was as follows:

  1. Create two different sized sets of icons.  Each set stored in their own folder.
  2. Create the framework image which contains the effects layers (e.g. shading, shadowing, etc) and separate layers for the icons: small icons collage, large icons collage, and the spotlighted icon.
  3. Write an ExtendScript function to open the icon image files and randomly copy each icon image (from the “pool” of opened icon images) into the icon layer in aligned rows, repeating until the entire image is covered.  The function would be run for each icon collage layer.
  4. Manually add in the spotlighted icon as an “illuminated search hit”.

Now, I have script that I can reuse to build alternative icon collages or build other type of collages.  The script has literally saved me hours and hours of work.

Maybe there’s way of easily achieving the same thing without resorting to scripting?  If so, please let me know.

Get in touch if you’re interested in more details about the script implementation.

Technorati Logo , , , ,

Growth at the Expense of Brand Value

On the subject of coffee parlours, today I read a great diagnosis of the growth-induced erosion of Starbucks’ brand value and, consequentially, the erosion of their growth.  Clearly Starbucks have lessons to learn about balancing growth with brand, but they certainly aren’t new lessons.  What happened?  Read John Quelch’s article “How Starbucks’ Growth Destroyed Brand Value” to find out.  Allen Roberts posted a comment that nicely sums up the value of the brand:

This maintenance of the integrity of the brand as THE vital component of the value proposition to the consumer is basic marketing 101. Starbucks when they started knew this, and practised this as if their life depended on it, as it did!  However, it is the easiest thing to compromise when the driver becomes numbers, and short term financials, as tends to happen after an IPO, as brand integrity and the relationship consumers have with a brand is not easily quantifiable, so is less visible, easily ignored by (often) new management under the pump for results who have not “lived” the brand.

Technorati Logo , ,

The Last Mile

I like the coffee from Take 5 Cafe. Beats Starbucks hands down.  I mean, when all that the Starbucks “barista” has to do is press a button to brew a shot … well, I’m sure Starbucks was aiming for infallible consistency while maximizing throughput but they lost the plot when it comes to the ceremony and “romance” of the coffee shop.  Starbucks CEO Howard Schultz admitted as much in a leaked memo:

…when we went to automatic espresso machines, we solved a major problem in terms of speed of service and efficiency.  At the same time, we overlooked the fact that we would remove much of the romance and theatre that was in play with the use of the La Marzocca machines.  This specific decision became even more damaging when the height of the machines, which are now in thousands of stores, blocked the visual sight line the customer previously had to watch the drink being made, and for the intimate experience with the barista.

The romance and theatre of a serious coffee parlour are core ingredients of the business, not last mile ingredients and Take 5 have got it right compared to Starbucks.  A beautifully crafted espresso machine, a delight to behold, is used by the barista (occasionally a delight to behold) to brew espresso shots.  The sounds of the production are classic espresso machines sounds.  Coffee aromas are untrapped in their portafilters.  European styling. Granite countertops to receive the coffee product.  We have theatre. Delivered, finally, is your coffee order and we’re now at the last mile stage.  To prevent the heat of the coffee from burning your hand and pissing you off so much you decide to sue because, ummmmm, you were never properly taught thermodynamics in high school, a sleeve is used to give you a comfortable, cool, area to hold the cup. Yep, the sleeve is one of those “last mile” items.  Take 5 have screwed up at the last mile by using an ugly plastic sleeve that feels like you’ve grabbed hold of a squid.

plastic_sleeve1

So, it looks and feels like crap.  Does Take 5 really want to have their carefully designed, eye pleasing, coffee cup adorned with such a sleeve?  What are they saving per sleeve over the cardboard variety?  0.5 cents?  1 cent?  Yes, it must be about money because I can’t believe the Take 5 management would vote for the plastic sleeve over the cardboard sleeve if the costs were the same.  Or, maybe I’m wrong, and the plastic crap keeps the heat at bay better than the cardboard sleeves?  If so, then give me the heat.

Headline news: “The Sleeve That Sunk Take 5 Cafe”.  Not going to happen and that’s not what my herniated sleeve yarn is implying.  But, no doubt, businesses loose custom beyond a sustainable level (i.e. they fail) because they’ve overlooked or ignored last mile aspects.

I’m a details guy.  It’s the nature of my job in Information Technology. So “last mile” stuff is always on my radar and I’m continually miffed at how many people and companies don’t understand the importance of hitting the ball out of the park for those last mile items.  Tom Peters wrote a book called “The Pursuit of WOW!” in which he explains how important it is for businesses to have pleasant, clean bathrooms.  If memory serves, I believe he solidly drove home the point that you’re an idiot if you neglect your washrooms as a business owner in a competitive marketplace.  You establish your business, plan your brand, build your brand, take care of all the core pieces, cut the ribbon, unveil the business …. and the “last mile” toilets, well, go down the toilet.  Sure, a majority of new business starts don’t build hole-in-the-ground toilets; unless you’re starting a business in France or China.  But there’s still a noticeable number of establishments that screw up either right out of the gate, or through neglect over time.

Speaking of washrooms, one of the premier examples of a business hitting it out of the park for their washrooms is Cactus Club Cafe.  Talk about the pursuit of toilet WOW.  The washrooms at the Cactus Club on the corner of Broadway and Ash are fantastic. Immaculate, spacious, beautifully designed and decorated, a leather couch, etc … basically a posh studio apartment with modern decor.  People talk about the Cactus Club washrooms as part of the Cactus Club experience; they add value to the Cactus Club brand. Flush with style.

What aspects of your business operate in the last mile zone AND could serve as differentiators in a competitive marketplace?  Find out, focus on them, and improve.

Technorati Logo , , ,

MSIL Foray

Recently I took a dive into the world of Microsoft Intermediate Language (MSIL).  Why? I didn’t yet have the source code for an assembly I was referencing and I needed to remove a date validation constraint that was affecting the solution.  This is how I resolved the situation:

  • Disassembled the assembly into Microsoft Intermediate Language (IL) using the Microsoft IL Disassembler (ildasm.exe) located in “C:\Program Files\Microsoft Visual Studio .NET 2003\SDK\v1.1\Bin”.
  • Edited the MSIL file and updated the date validation code to pass validation.
  • Assembled the IL back into an assembly using the Microsoft IL Assembler (ilasm.exe) located in “C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322″.

Pretty straight forward except for having to learn some MSIL.  Oh, and there was also the effort required to identify the area of code in question.  To cut that story short, I’ll simply say that I pinned down the boolean variable that was used to determine if today’s date was within or outside of the hard coded date range … and that variable is simply “named” variable “2″. I searched the MSIL file for all areas where variable “2″ was updated and then came across the date validation logic that I had to amend.

Following is the MSIL code snippet containing the date validation check which I’ve amended to pass validation by “widening” the year values.  This snippet is asking the question: Is the current date between Oct 3, 2009 and Aug 7, 2000? If so, then the “date invalid” flag is set to true, otherwise leave the “date invalid” flag as false.

  • IL_0017: Get the current date and push it on to the stack.
  • IL_001c: Push the value 0×7d9 (2009) on to the stack.
  • IL_0021: Push the value 10 on to the stack.
  • IL_0023: Push the value 3 on to the stack.
  • IL_0024: Instantiate a DateTime instance for the date Oct 3, 2009. This will pop the year, month, and day values from the stack leaving the current date (from IL_017) at the top of the stack.
  • IL_0029: Compare the current date to Oct 3, 2009 and place the result on the stack.
  • IL_002e: Load the value 0 on to the stack.
  • IL_002f: Compare the value 0 (from top of stack) to the DateTime::Compare result (second from top on the stack) and branch to IL_004a if greater than or equal. In other words, if “Now” is greater than or equal to Oct 3, 2009, branch to IL_004a where the date validation flag is set to true, meaning “failed validation”.
  • IL_0031 to IL_0048: Compare “Now” to the date Aug 7, 2000 and branch to IL_004c if “Now” is greater than or equal to Aug 7, 2000. In other words, branch past the code that sets the date validation flag “invalid” (value of 1).
  • IL_004a to IL_004b: Push the value 1 (true, in this case) on to the stack and store that value (stloc) in location 2 (”variable 2″). In order words set the “date is invalid” variable to true.
Technorati Logo ,

Roberto Announces Connected Health Framework

Roberto Ruggeri has just posted an article announcing the publication of the Microsoft Connected Health Framework.  As usual, Roberto has done a good job of introducing and explaining things and, therefore, you should be able to easily discern how the Connected Health Framework (CHF) is being positioned.  I really like Roberto’s focus on how CHF is not intended as a vertically targeted, technology-centric framework but is, instead, geared towards the interaction of services and business components in a industry-neutral manner. As Roberto says:

What makes a solution specific to healthcare is the type of messages that are exchanged, the security and policies for authentication, authorization and information access and of course the nature of services and business components

Technorati Logo , ,



Bad Behavior has blocked 694 access attempts in the last 7 days.