Tag Archive for 'microsoft'

The Gemini Approach

In a nutshell the Microsoft Gemini project is about two things:

  1. Giving end users more powerful desktop-based OLAP capabilities.
  2. Giving IT departments the ability to better control organizational data assets.

Desktop-based OLAP capabilities will be achieved as an extension to Microsoft Excel.  Better control of organizational data assets (and spreadmarts) will be achieved via tight integration of Gemini spreadsheets with SharePoint Server.  Better control means better visibility into previously isolated data sets, improved ability to established authoritative data sets and analytics, and improved sharing of information.  Much has been written about Gemini so I’ll leave it to you to Google (or Bing) Gemini.

What I really like about the Gemini project is the approach, or philosophy if you want to call it that.  Rather than “fight” the spreadsheet users, join them.  Someone involved with the Gemini project must have recognized that continuing to slap users on the wrists for contributing to spreadmarts is the wrong approach and counter-productive.  Instead, why not provide an easy mechanism to capture their data/analytics in a controlled server environment without confronting them with policies and lectures that force Excel users “underground” and exacerbate the rifts between IT and the business.  At the same time, embrace the popularity, agility, and ubiquity of Microsoft Excel by providing powerful new OLAP features the further increase a user’s ability to quickly capture, model and analyze business data.

The Gemini philosophy of emphasizing user business agility (desktop OLAP) and centralized data capture (SharePoint integration) over rigid data control mechanisms is a refreshing approach.  An approach that will hopefully see wider adoption in IT.  I wouldn’t be surprised to see Microsoft address the proliferation of Access databases in the same manner.

Not to be treated lightly is the area of records management and the need to ensure that organizational records/data (whether they exist in Excel spreadsheets or Access databases) are governed in an effective and timely manner.  Gemini helps facilitate a solution to that problem for OLAP data by providing tight integration with SharePoint where information management policies can be applied to published Gemini spreadsheets.  But it doesn’t solve the entire problem.  If Gemini spreadsheets contain records that need to be governed by disposition policies, and the users do not publish those spreadsheets to SharePoint … well … you obviously continue to face the risks of having uncontrolled data scattered about the company on users’ desktops.

Update (Sep 11, 2009):

Found this suite of YouTube videos explaining Gemini: http://www.youtube.com/user/geminute. I haven’t watched any of them yet.  Drop me a comment and let know what you think.

Technorati Logo , , , , ,

Microsoft Business Intelligence Conference 2008

microsoft_bi_conf_2008 

Business intelligence (BI) refers to technologies, applications and practices for the collection, integration, analysis, and presentation of business information and sometimes to the information itself.
Wikipedia
BI systems provide historical, current, and predictive views of business operations, most often using data that has been gathered into a data warehouse or a data mart and occasionally working from operational data. Software elements support the use of this information by assisting in the extraction, analysis, and reporting of information.
Wikipedia

I’m wayyyyyyy overdue in getting my blurb published about my attendance at the Microsoft BI Conference.  Thousands have already beat me to it but, nevertheless, below is my summary.

Technology takeaways from the 2008 Microsoft Business Intelligence Conference:

  • Project Gemini.  Microsoft’s “self service” analytical capabilities that add significant data processing and analytical power to the Excel client experience.  Gemini reflects Microsoft’s ongoing efforts to surface integrated BI capabilities to users in a natural manner where users aren’t always aware they’re working with, or doing, BI.  Also part of Gemini is the ability to publish the resulting cube to a “Gemini enabled” SharePoint library for sharing, social-style interaction, and performance and usage analysis.  This is a significant development along the road of mining those important ad-hoc Excel data analysis sheets that breed and spread like bacteria across an organization.  And, with Gemini, they are mined in a very non-intrusive manner.
  • SQL Server Kilimanjaro.  The next release of SQL Server slated for delivery during the first half of 2010.  Kilimanjaro will include Gemini and self-service reporting.  Self-service reporting is all about giving workgroups/teams the ability to self-service their reporting needs and to provision security for their reports.  Sharing report components, “mashing up” reports, and getting notified about changes to shared components, are part of the self-service reporting initiative.
  • Project Madison.  Advanced data warehousing capabilities via the integration of DATAllegro’s technology into SQL Server.  (DATAllegro is a recent Microsoft acquisition.)  Project Madison will allow SQL Server to scale to 100s of terabytes of data with, it is claimed, exceptional data loading and query performance.
  • The Microsoft Master Data Management (MDM) stack.  Microsoft envision an integrated MDM stack to include an MDM application platform and domain-specific MDM solutions provided by partners.  The plan is to ship the MDM platform as part of Microsoft Office SharePoint Server.  A working product is expected for next year’s Business Intelligence Conference.

A couple of non-technology takeaways:

  • Requirements for successful business intelligence and performance management solutions:
    1. A culture of performance.
    2. A culture driven by business value.
    3. A culture of continuous improvement.
    4. Incremental delivery.

    This really isn’t a takeaway at all because it’s just common sense.  Still, many organizations fail to successfully deliver their BI and performance management initiatives (and many other types of IT projects) due to cultural issues.  In many cases blame is heaped on the technology or the technologists.  Inevitably, properly conducted  project retrospectives will highlight that certain leaders, stakeholders and consultants are accountable for not prioritizing their focus on the business climate and business drivers.  Without a wide spread cultural shift in support of change (backed, of course, by sound business cases), most projects will fail.

  • The importance of deploying your performance management solution to your partner ecosystem to support those that contribute to your bottom line.  This speaks to the concept of pervasive performance management which is “enterprise-wide, enterprise deep, and outside-in, and includes unstructured data which is becoming an important component of the BI puzzle“.
Technorati Logo , , , , ,

Microsoft SOA & Bus. Process Conference 2006: Day 4

Denny’s Restaurant Session
Bellevue, Washington, 3:00am
At approximately 3:00am, me and two of my colleagues decided to do a bit of food research at a Denny’s restaurant in Bellevue.  You could call this our first morning session.

There is More to Life in Healthcare Integration than Technical Compliance
Elizabeth Redding, Partner2Learn
Elizabeth has been helping healthcare organizations with technical compliance over the past 10 years and has a lengthy background developing healthcare plans. Based on the title of the session, I suspected that we would be getting slides that state the obvious. Sure enough, one of the slides stated that “Focus should be on the business…”. I really didn’t think it was necessary for the presentation to include such basics when the focus should be more on providing deeper insight into service oriented architecture and business process management from a healthcare perspective.

She spent some time talking through the benefits of the BizTalk integration model using an HL7 patient administration message workflow as an example and also eluded to the benefits of using the BizTalk BAM (Business Activity Monitor) and BRE (Business Rules Engine) in the mix.

On the subject of the administrative overhead in healthcare she said that when HIPPA was first introduced, the [American] healthcare industry was spending 40% of their budget on administrative overhead. From a Canadian perspective, she said that the administrative overhead is currently massive and one of the reasons has to do with the reporting of data which is apparently a burdensome process. She wasn’t specific about the nature of the reporting and how the data is reported so if anyone knows what she’s talking about please post a comment to explain.

Developing and Maintaining Business Rule Solutions
Richard Seroter, Technology Specialist, Microsoft

I thought this presentation would focus solely on BizTalk BRE but Richard did a great job delivering useful content in other areas which made the session worthwhile.

Why have business rules? Richard provided the 3 standard reasons: 1) Accessibility, 2) Flexibility, and 3) Manageability. He then talked briefly about the iterative BPM lifecycle comprised of: Model and Design, Develop and Deploy, Manage and Interact, Analyze and Optimize. What Richard was saying, was that establishing and implementing business rules requires the same level of professional analysis and design as you would conduct for other business solution areas. I guess that’s pretty obvious but I imagine that are lots of organizations/people out there who design and implement business rules in an ad-hoc manner in the wrong places. Actually, Richard did point out that many people implement business rules in inappropriate places like stored procedures, user interfaces, and web service implementations. Yes, I agree, many, many people have done this and it can severely impair a solution’s adaptability making it a very costly endeavor to upgrade applications to accommodate changing business rules.

In addition to BizTalk, Workflow Foundation (WF) also provides a rules engine. The following table is a comparison of the two:

BizTalk BRE Workflow Foundation Rules
Rete-based algorithm Forward executing algorithm
Xml/database/.NET facts Workflow variables
Vocabularies No vocabularies
“If … Then” semantic “If … Then … Else” semantic
Explicit chaining Implicit chaining
Runtime infrastructure (e.g. tracking, security) Custom coded
Rules built using the BizTalk BRE Composer Built in Visual Studio .NET

Other takeaways include:

  • In BizTalk, when rules are executed via an orchestration, the rule execution can be tracked, allowing you to view the tracking information to discover which rules were used/fired. However, if you call the BRE directly via a .NET class, you cannot do tracking by default. To enable BRE tracking via .NET, you need to implement a certain interface provided by one of the BizTalk BRE assemblies (can’t remember which one; will leave it to you to do the search) and pass the implementation of that interface to the BRE call.
  • Richard feels there is a business opportunity around the development of customized rule solutions (e.g. rule templates, function libraries) for verticals (e.g. healthcare).
  • RuleBurst is a product that is used to administer complex business rules. Richard did a quick demo of RuleBurst and it did look impressive (much more comprehensive than the BRE composer). It uses a Microsoft Office rule design surface (e.g. rules can be edited directly in MS Word), has a robust management console, and, apparently some great QA (Quality Assurance) testing capabilities.

Web Services Factory
Don Smith, Product Manager, Patterns & Practices, Microsoft
Web Services Factory can be summed up as follows (from Microsoft’s Web Service Software Factory article):

The Web Service Software Factory (also known as the Service Factory) is an integrated collection of tools, patterns, source code and prescriptive guidance. It is designed to help you quickly and consistently construct Web services that adhere to well known architecture and design patterns.

Don was a very enthusiastic presenter and did a decent job of explaining the concept and implementation of the web services software factory. At a high level, Don explained that a software factory is about “communicating” between the architect and the developer and between the architect and the business; in other words, to help increase “communication” across the board. That sounds rather vague but he did go on to explain a bit more: “It’s about allowing the developer to have a conversation with Visual Studio“. Ok, still vague. How about: “an implied or suggested process that you go through to build a solution“. Or, better yet, he also described a software factory as something that helps you build a specific kind of application via the incorporation of a variety of guidance content type including: architecture and design guidance, patterns, and help, reference implementations, application blocks, and guidance packages. To be fair, Don did a pretty good job of trying to explain the concept of a software factory. However, from an implementation perspective, the term “software factory” seems loosely defined (like SOA), starting out as something based on DSLs (domain specific languages), then something that uses GAT and GAX, then “morphing” into something that is being wrongly interpreted as simply a code generator. Microsoft’s answer to the question “What is a Software Factory?” can be found here.

Underpinning the web services factory are two core components: GAT (Guidance Automation Toolkit) and GAX (Guidance Automation Extensions). These are the components that allow “factory builders” to extend Visual Studio 2005 to author and run guidance packages which contain “templates, wizards and recipes, which help developers build solutions in a way consistent with the architecture guidance“.

The services factory contains guidance for designing messages and service interfaces, applying exception shielding and handling, designing business entities in the domain model, translating messages to and from business entities, and designing, building, and invoking the data access layer. The first release of the Services Factory was an ASMX (web services) release in July 2007; the WCF release is scheduled for December 2007.

This session was a great demonstration of the capabilities of the GAT and GAX components and how they can provide substantial benefits in driving solution development.

Technorati Logo , , , , , , , , , ,

Microsoft SOA & Bus. Process Conference 2006: Day 3

Data and Transaction Management
Steve Swartz and Clemens Vasters
Connected Systems Division, Microsoft
The core foci of this presentation were the characteristics and architectural patterns related to data storage and data access. The core takeaways were the importance of early data architecture envisioning and that the correct data architecture is the ultimate performance optimization. For experienced architects and developers in the audience, the session likely served as a refresher on the topic of data architecture.

Steve talked about data characteristics from two perspectives: scope of the data and class/type of data. Data scope included: My Database, My Shared Data, Our Database, My Huge Database, My Distributed Database. Classes/types included: Reference Data, Fresh Data, Stale Data.

As Steve puts it: “Increased collaboration changes the architecture perspective”. Thinking about data scope, the growth in data scope, data concurrency and correctness, should be one of the central tasks of early data architecture. That may be obvious, but Steve suggested that too few people consider the larger “enterprise wide” data picture when architecting solutions. I agree.

Next were the architectural patterns which included: Direct Access, Remote Access, Intermediated Access, Error Handling via ACID, Error Handling via Accounting, Error Handling via Compensation, Distribution via Caching, Distribution via Federation, Distribution via Read-Only Replication, Distribution via Read/Write Replication, and Distribution via Reporting.

The intermediated access pattern is a rich and important pattern that places processing in front of the database to control access. It is the mechanism used to achieve federated queries and therefore is a part of the federation distribution pattern. When talking about the “Error Handling via Compensation” pattern, Steve mentioned something about Microsoft working on automatic compensation capabilities? Anyone know about this? BizTalk allows you to implement the compensation pattern for long running transactions (which he mentioned) but I’m sure Steve wasn’t referring to BizTalk. “Error Handling via Accounting” is the error pattern that persists errors and then reports them for “manual” resolution. This is in contrast to the ACID error handling pattern which uses transaction management to provide automatic error handling. ACID performs much better (obviously) but does not scale as well as the “accounting” approach.

Driving Business Process Automation through Vertical Accelerators
Brennan O’Reilly & Mark Smith, EMC Microsoft Practice
This was a weak presentation with plenty of platitudes, basics and generalities on how to design integrated business systems with some focus on manufacturing, financial services, and healthcare verticals. I was hoping to get some good insight into the design of accelerators for the BizTalk platform, but left disappointed.

The healthcare part of the presentation referred to the American healthcare industry and dealt almost exclusively with billing and insurance solutions. Actually, they didn’t explicitly refer to the American healthcare industry but it was obvious from their focus on billing and HIPPA.

Effective Techniques for Handling Large Messages in SOA
Thomas Abraham, Senior Consultant/Architect
Microsoft Solutions Practice
Biggest takeaway from this session was his idea about carving up a large message in a custom pipeline component. The idea is to slice up the message, writing the “large” piece to temporary storage and then using a proxy ID in a cut-down version of the message that flows into the message box. The proxy ID “points” to the “large” message piece in temporary storage and is used to retrieve that piece for pipeline assembly on the way out of BizTalk. Not rocket science but a very useful tip for reducing the load on the message box and improving performance.

When Thomas was discussing ASP .NET 2.0 coding practices, he stated that it’s important to think about memory management. Since the introduction of garbage-collected languages, I have been saying the same thing. Ever since I started working with Java, and now C#, I have regularly come across developers who ignore the memory usage characteristics of their software and then wonder why they are having performance issues.

BizTalk Server 2006 R2 Adapter Framework
Chandramouli Venkatesh, Group Manager
Connected Systems Division, Microsoft
Chandramouli did a good job of presenting on the upcoming BizTalk adapter framework. Microsoft’s motivation behind the creation of an adapter framework is to unify their approach to adapter development enabling “…easy development of metadata-driven, host-agnostic, custom adapters to LOB systems”. This is great to see and should hopefully make the prospect of developing a custom BizTalk adapter a less intimidating experience; I haven’t developed a custom BizTalk adapter myself but I keep hearing that it’s not a trivial task.

In keeping with the harmonies that Microsoft is creating around WCF, the adapter framework will extend WCF and adapters will be surfaced as WCF bindings which will make the adapter’s consumption exactly the same as WCF service consumption. This means that custom adapters will be usable across different consuming hosts (e.g. BizTalk, a custom WCF host) which is a big win.

From a development perspective the adapter framework will ship with a rich set of development tools which will automate and simplify much of the coding required for adapter development. Microsoft’s current thinking is that the adapter framework will be made available as a separate download.

Technorati Logo , , , ,

Microsoft SOA & Bus. Process Conference 2006: Day 2

With a good dose of business-focused keynotes and sessions under my belt, I was ready for a bit of a technical dive. So, my first session of the morning was WCF: Extensibility in the WCF Service Oriented Platform, presented by Craig McMurty, Technical Evangelist at Microsoft. It started with an introduction to Windows Communication Framework (WCF) which probably could have been shortened. A couple of highlights from this session:

  • The presentation’s central theme was the WCF ability to “inject” custom functionality (i.e. extensions) at various stages along the path that a WCF message travels on both the client side and the server side. Such custom functionality can deal with specific requirements around serialization, threading, etc.
  • Craig said that WCF was a “software factory” for communication. There’s that expression again, “software factory”. Since its inception, “software factory” has been liberally bandied about when talking about a number of software development spaces. If I recall correctly, the term “software factory” had a precise meaning that doesn’t encompass each and every domain-specific API that comes along. Ok, I’ve just returned from an evening out with my colleagues where we discussed my objections to the liberal use of the expression “software factory” which now seems to include the realm of WCF. My colleague, Adam Bowron put forth an excellent argument in defense of the association of “software factory” with the code generated by WCF. He explained that the generation of the WCF code infrastructure to support a specified contract and protocol requirement (i.e. a specific service communication domain) can legitimately be labeled as a software factory example. When the term “software factory” was first “born” I believe this was the basic description:“A Software Factory is a software product line that configures extensible development tools like Visual Studio Team System with packaged content like DSLs, patterns, frameworks and guidance, based on recipes for building specific kinds of applications.”http://www.theserverside.net/news/thread.tss?thread_id=29651

    Does this description apply to WCF? Perhaps. WCF has patterns and guidance but are they packaged and primed to kick-start the development of WCF solutions?

The next morning session in line was a presentation called Avoiding 3 Common Pitfalls in Service Contract Design. Tim Ewald (Principal Architect at Foliage Software Systems) did a reasonably good job at explaining his position against building consensus around the definition of enterprise-wide canonical representations of business entities. One of his central tenants is that flexibility in data models is one of the keys to SOA success. The 3 command pitfalls he talked about were:

  1. Too much required data. The trouble is that you may want to adopt a corporate data model for a certain business entity (e.g. a customer) but the system you are dealing with simply cannot supply some of the schema’s required information; an issue of cardinality. This forces you to take one of two approaches: 1) Gather data just to fulfill the schema requirements, or 2) Use garbage data. The solution is to make the schema elements optional (i.e. minOccurs=”0″). With a schema you should be saying “if you can supply the data in this shape (i.e. the correct sequence of elements) I’m ok but if you change the shape (i.e. out of order elements/types) then we have a problem”. Bottom line: enforce the occurrence constraints down at the system level.
  2. No Solution for Versioning. Before designing data contracts is it crucial to first establish a clear schema versioning policy that governs how the schema can change. Tim suggested that changing namespaces makes systems incompatible and offers “lower value” and goes on to say that namespace changes can be minimized with a few simple rules which include; 1) Create an instance document based on the schema version you have, 2) Always consume an instance based on the schema version you have, 3) If you want to do validation, make sure you do careful validation that doesn’t fail if, for example, new elements are detected at the end of types (i.e. the core schema version has changed to include new elements). If you want to develop flexible, loosely coupled systems then you need to be prepared to ignore some schema validation errors (e.g. unknown elements that appear at the end of a sequence). Flexibility is enhanced by a versioning policy that allow clients to extend their data contract to include new types or new elements at the end of a sequence.
  3. No System-Level Extensions. What if you want to add new elements to your version of the schema? According to Tim you should be able to extend your version of the schema and, therefore, the core schema authors should support this by “leaving a slot in the schema for clients to extend their version”. Tim went on to say that by enabling and promoting extension capabilities, the core schema team could “learn” from the extensions applied to “local/departmental” versions and then incorporate those extensions, as needed, into the core schema over time (i.e. a “harmonious” and flexible evolution of the core canonical representation driven by “feedback from the trenches”).

Tim summed up by asking us to rethink the “big bang” canonical design approach and, instead, start with a base model designed to facilitate the evolution of incremental value. This approach, according to Tim, would allow the user to adopt core schema versions at their own pace without sacrificing business goals.

After a quick bite of lunch, my colleague (Adam Bowron) and I hooked up with Jim Boyer (a great Technical Specialist from Microsoft). We walked across the street to the BizTalk product building for a meeting to discuss BizTalk 2006 end-to-end message ordering with one of the Microsoft BizTalk product/program managers. I won’t go into the details at this point expect to say that the discussions centered around enabling BizTalk to maintain FIFO message order regardless of the impact of multiple orchestrations and other BizTalk activities which can alter the original order.

Our meeting at the BizTalk building cut into the first afternoon sessions and I arrived half-way through the presentation on Building an ESB on the Microsoft Platform. I really didn’t take away much from the remainder of the session except that it looks like there is some interesting work going on around the BizTalk platform to provide a compelling ESB-esque story. My colleague mentioned that there is good work being done around enterprise-level exception management (leveraging the BizTalk 2006 failed message routing feature).

My final session of the day was entitled Get Your Processes Talking! Speech, Workflow & BizTalk United. Presented by Jon Fancey (Principal Consultant, NetStore) and Albert Kooiman (Senior Business Development Manager in the Unified Communications Group), the aim was to show how Communication Server 2007 and BizTalk can work together to deliver speech-oriented workflow solutions. This session was really all about showcasing Communication Server 2007 with a little dash of BizTalk. The speech capabilities in Communication Server appeared impressive and the design experience in Visual Studio looked great (i.e. creating a new speech workflow solution). The intention is to optimize human workflow processes via automation and thus improve service levels. My thinking at this point was that organizations need to be extremely careful about fronting their service function with a machine. Not all customer service scenarios are suitable for speech-based human workflow processes. I would go so far as to say that few customer service scenarios are suitable for the machine interface. When customers make a service call to an organization they typically do not want to talk to a machine.

Technorati Logo , , ,



Bad Behavior has blocked 89 access attempts in the last 7 days.