Thursday, August 21, 2008

Limitations of documents - and why consultants like them

Documents are ineffective in supporting the operationalisation of processes, and the management of knowledge, about complex enterprises.

Accounting processes and knowledge
Imagine the accounting process - if it was implemented using documents (Word, Visio, Excel) i.e. if I am a simple organisation I could create invoices, make payments, create balance sheets, calculate tax, do reports, keep track of assets, answer questions from my customers on what they owe, etc. Now if I were the accountant - I could do this work using documents if the number of elements (invoices, payments etc.) were few, the rate of change of information about by assets was slow, and there were not many people asking questions or changing things.

Now if I were the accountant - I could do this work using documents if the number of elements (invoices, payments etc.) where few, the rate of change of information about assets was slow, and there were not many people asking questions or changing things.

I would effectively either do it using documents (Word) where the semantics are not explicit (e.g. where rate, hours and charge are just in written in the word document) or in a model (e.g. which calculates charge from hours and a table of rates). Clearly the model would allow me scale more quickly (make me more productive). This might encourage the business to see the value of the models and get me to extend their scope so I could record other information and answer new questions.

However as the number of elements I have to dealt with increases, the rate of change increased, and the number of people I have to communicate with increased. I will quickly become a bottle neck i.e. typing everything in to my models (or worse still documents).

Clearly I would be come a bottleneck. It would take a very large teams of clerks to get through the work - and still we would not be that quick at answering even simple ad-hoc questions. Now to do the job properly I would of course need sound methods and structures (frameworks) - but they would not make me efficient or agile.

Document and content management systems could exacerbate the problem - if it allowed lots of people to send me word documents about things that I had to deal with i.e. turn into invoices, payments, etc. It might give the illusion that the all the knowledge is being managed - but I would just get swamped.

The answer for a complex business must be a system (accounting) which allows information to be captured as a by-product of day to day work (entered by people at source), analysed and presented in the way that suits each audience (i.e. the system provides reports directly to people who need them). This has the advantage of meaning I don't require a team or clerks and the accountant can be removed from administrative tedium to focus on more strategic decisions.

Of course this would not suit the person who sells me the teams of clerks (consultants) to do this work - because now I would not need most of them.

Technology processes and knowledge
Critical knowledge about an enterprise is knowledge of: the environmental constraints (market, regulatory etc.); and how the environment affects the business strategy (goals, strategies, plans, products etc.), and how the strategy affects business operations (processes, people, information, etc.); and how technologies supports the business operations; and what changes are planned to all of the above. Some of the knowledge is required for operations and some for decisions on possible changes.

It used to be that the number of elements (systems, reports, applications etc.) were few, the rate of change of information about the assets was slow, and there were not many people asking questions or changing things (because usage was limited and constrained).

The number of CPUs per person in a business seems a reasonable proxy for complexity. Thirty years ago there was perhaps 1 CPU for 1000 people (e.g. in a thousand person organisation), twenty years perhaps there were 10 (for the same organisation), fiften years ago perhaps 100, ten years ago perhaps 1000 (as everyone had PCs) and now perhaps 3,000 (various servers, PCs, phones, NW devices etc.). So the complexity has increased by perhaps a factor of 3000 (i.e. 300,000%).

Now if I were the CIO and IT team - I could do this work using documents if there were few elements, a slow rate of change, and I were not asked about changes that often. Certainly thirty years ago, twenty years ago and perhaps fifteen years about - but over the last 10 years things have started to get too complex.

IT has become a bottleneck i.e. typing everything in to my models (or worse still documents), or often just keeping knowledge in our heads.

I need large teams of clerks to get through the work, I am not answering most simple ad-hoc questions. I get told that methods and structures (frameworks) are the answer - but no one can make them work.

Document and content management systems give the business the illusion that the all the knowledge is being managed - but actually I can't join the dots.

Of course the consultants in the areas (BA, BP, EA, Systems Analysts, Data Analysts etc.) are happy as I have endless need for them. Funnily enough none of them mention any systems based answers (unless they sell me a bureaux offering locking me into them).

Enterprise processes and knowledge need to be supported by enterprise systems
One can clearly see how enterprise recognise the need for enterprise systems in other areas as they have become complex and critical. It now needs to be recognised that processes for strategy, architecture, governance and transitioning of IT (and arguably the entire business) needs to be treated as an enterprise problem and provided an enterprise solution.



See related item: http://ict-tech-and-industry.blogspot.com/2008/08/stages-of-maturity-in-communications.html

Sunday, August 17, 2008

IT Business Alignment

In a recent A&G magazine item "IT-Business Alignment" identifies IT-Business alignment as a top a concern amongst CIOs.

It is suggested the top enablers to alignment include: Senior executive support for IT;
IT understanding the enterprise's business environment; Business units’ understanding of the enterprise IT environment; a close partnership between IT and business; IT plans linked to business plans; good communications between IT and business; Business units's support (including staff and investment) for enterprise wide IT initiatives; Clear predictable corporate goals and directions; Business units sharing (not competing for) IT resources; IT and the business governance; Business units’ prioritization of IT needs; Clear ownership of IT-business alignment

It is suggested that: “Business people must recognize the most important thing they can do is act as a sponsor for an IT initiative...To do that, they need the IT management team to come in with specific ideas. Business executives are usually more than willing to support
them, but the initiative must come from IT.” I think that the reality is that business people can see that IT has not got its own house in order and simply don't trust IT. To say that “Business people must recognize ...” seems to me a little rich. I think "the IT unit must recognize that the business doesn't relly trust them and they need to come in with specific ideas on how to win trust, and that this IS an initiative must come from IT.”

So until CIO's take the initiative to put in place solutions that will ensure that they can communciate i.e.
- what they understand the corporate goals and directions to be (and how these link to business plans)
- what IT seeks to do and why clearly to senior executive (including IT plans linked explicitly to business plans)
- the enterprise IT environment to the business
- their understanding of the business (via their views of key relationships e.g. environment, goals, projects, operations and technology)
- how they have addressed IT governance.
- their model for prioritizing IT needs based on the business plans.
How can they expect to win the confidence of the business?

If they seriously think (being IT professionals) that they way to enable this communication is to use sets of disconnected Office suite documents (plans, project definitions, systems diagrams, process models etc.) - they must be dreaming.

See: http://ict-tech-and-industry.blogspot.com/2008/08/stages-of-maturity-in-communications.html
http://ea-in-anz.blogspot.com/2007/10/improving-its-reputation-and-alignment.html
http://ea-in-anz.blogspot.com/2007/09/why-dont-most-it-processes-work-well.html

Projects are artificial constructs and often managed in ways unsuited to enterprise change initiatives

(project management for enterprise systems needs to be suited to the task)
Much of the approach management comes from project disciplines developed and honed in the construction industry e.g. focused on building a building. Unfortunately enterprises, and many of their elements, are quite different in nature to buildings (e.g. enterprises are more like a city). This is especially so of enterprise systems. Until the differences are recognised, understood and analysed - and the project management approaches revised to better reflect the differences - project management in an enterprise will under perform on the expectations (i.e. they will not typically result in on-time, on-budget and on-scope delivery). This extends to a set of projects or programmes and non-time bounded initiatives

How buildings differ from enterprise systems

Usage is clear and doesn't change - of a building and most components of a building. Before building commences (and usually almost at conception i.e. before design commences) the expected use is clear to all parties and doesn't change substantially. That is to say that we all know what a house, garage, etc. is for (how we expect to use it); what a bathroom, kitchen, bedroom, etc. is for; what a bath, window, bed, etc. is for; what a door handle, sink tap, bed side lam, etc. is for. In all cases usage is clear and doesn't change and it is known before construction starts, and in the vast majority of cases before design starts. To suggest that all parties clearly know what an enterprise system is for (and exactly how it is to be used), what a major module is for (precisely and how it will be used), what a user interface is for, or what a UI widget will do precisely - at the start of most projects is simply untrue.

Construction of a building is largely assembly - Constructing systems (systems: development, integration, tuning, customisation, configuration etc.) are not assembly tasks they are largely design tasks. The small design decisions made by the various building trades do not approximate in scope (e.g. the extent to which the design could be got wrong) to the level of design undertaken on the fly for systems.

Building's design is defined and explicit - As a result of a clear understanding of usage, material, methods and a way of communicating about buildings (that has evolved and matured), a building's scope is very well understood before construction commences. New systems enterprise's design is very seldom defined with the same degree of certainty so in practice much of the project management is associated with design of the enterprise systems

Buildings are completed - Buildings can be built and once built are essentially complete (and often remain largely unchanged for extended periods of time). Most enterprise systems are never really complete, they continue to evolve on an ongoing basis - so while the there may be a period during which major changes are made, the nature of work does not stop at the end of a project (it may diminish, or be postponed briefly).

Buildings are more discrete externally - A building can be seen as successful by itself - and while they connect to other built-elements (things) the nature of the boundaries are, by enlarge, well understood by all parties. Another way of seeing this is to recognise that my house can be complete and useful irrespective of the state of most of the other buildings in my street - but most enterprise systems rely on a very complex web of interconnections with other systems to operate (networks, identity, office suites, printers, third party systems etc.).

Buildings are more discrete internally - Similarly the internal elements of a building can be seen quite discretely by all parties. Another way of seeing this is to recognise that my bedroom can be completed and useable - even if my kitchen is not. In enterprise systems it is far more difficult to confirm that discrete elements are working.

Technologies and materials understood - The characteristics of a materials used are well understood. Frequently enterprise systems are built using technologies that have not be used extensively before by the people doing the work (at best similar of analogous technologies, or earlier versions, may have been used). Often regulatory codes are in place reflecting what does and does not work

Methods are mature - The exact methods of design and construction are not understood or agreed to on an industry wide basis. With buildings the set of documents used to describe the building is fairly standardised irrespective of which organisation does the design and within each area the detailed documents are also well defined (structural, HVAC, electrical, plumbing, etc.). So codes of practice exist and usually WBS for sub-aspects. Compliance with best practice is by regulation and not determined by a planners powers of persuasion. This is not true for enterprise systems - practioners argue over both high level methods, and how detailed methods should be applied.

Reasonable tests are well understood - As usage, materials and methods are well understood the functional and non-functional behaviour is well understood e.g. I know I should be able to walk through a door, and that a brick that is thrown can be expected to break a window - but not destroy a column.

The reason the words "by all parties" occur so often in the above is because specialists associated with enterprise systems may assert that their level of knowledge approximates that of their building counterparts. But the issue is that everything needs to work together - so what we really need is for all parties (user, designer, construction trades, operators) to all know how everything is intended to work together. For buildings there are exceptions to all of these rules - by in large they are true - and where they are not true specialised buildings (opera houses, stadia etc. ) these projects run into difficulty as well.

So projects are, in many of the uses they are put by an enterprise, artifical constructs used to provide certainty regarding cost and time - implicitly recognising the scope is a variable. Unfortunately most traditional project management assumes scope is fixed - and often project management becomes a function that communciates how far time and cost are to exceeded.

Essentially, enterprises want to make changes to how their business operates (which can be described in a business architecture) and the technologies, assets and services they use (which can be described in a technology architecture) based on strategy (constraints, goals, strategies, initiatives, plans etc.).

What is needed is a way to describe the changes required (holistically) then to understand what aspects of change are to be made in each project silo (and where common elements are affected) and to allow analysis to be undertaken across all projects as new information comes to light e.g. usage becomes clearer, design matures (as more is learned about the materials and how they are best assembled); as boundaries with and between elements clarify; as tests of quality and success are refined. The exact requirements; exact design; and exact project definitions and boundaries and inter-dependencies emergence from mists in parallel.

The approach I advocate achieves this.

Enterprise transformations and knowledge management and the prisoners' dilemma

Why is that while objective analysis indicates that only a very small change of behaviour is required by many people in an enterprise, especially one undergoing transitions, to significantly improve the outcomes for all - but no one is prepared to make the changes.

Stages of maturity in communications - strategy, architect and governannce functions

(to communicate a simple organisational problem with a few simple pictures)

Overview
When large enterprises are going through transformation (new products, mergers etc.) documents are ineffective in communicating the complex issues (and document and content management systems can exacerbate problems). As a result people involved in transitions spend time much time in meeting, in emailing and on the phone.

Strategy, architecture and governance are roles are established, but they can't manage the knowledge explicitly, don't scale and the reliance on individuals represents a risk to the business. Modelling manages the semantics, and provide powerful analysis and visualisation capabilities, and provides a source of record (reduce the risk) - but they don't scale and most models don't suit most audiences most of the time - so as communications mechanism they fail.

A knowledge repository is required which allows information to be captured as a by-product of day to day work, analysed and presented in the way that suits each audience and allows internal people to focus on actual strategy, architecture and governance while minimising the need for external consulting services.

Documents are ineffective in communicating complex issues in large enterprises Documents which encapsulate business concepts e.g. strategies, plans, initiatives, programmes , designs etc. are ineffective in communicating to most of the people most of the time. Many people create documents that communicate things to them, and their peers i.e. from their perspective and oriented at their interests. They make documents available to others in the expectation that they will effectively communicate to different people, with different interests and different perspectives. Most people if they looked at the evidence would realise this doesn't work (i.e. it demonstrably fails).



The set of all documents contains sets of related concepts (constrains, goals etc.). The set of concepts may relate to any aspect of the business (product, organisation, location, behaviour, risk, projects, technologies etc.). In the documents, usually most of the relationships between concepts are implied and unweighted (semantics are inexplicit) - because the author or original audience would be expected to know these relationship implicitly.


In the real world people need answers to questions quickly (without lots of reading)
When any issue arises or any decision needs to be made what people want is to understand an issue, or set of issues that is currently of interest to them i.e. How the concepts are related.

If one read, understood and internalised all the related documents (assuming they can be located - and were all consistent) one may perhaps get the understanding required. Unfortunately few have the time and capacity to do this.

Document and content management systems can exacerbate the problem
Document and content management systems, or content may help maintain the set of documents - but they don't address the issue knowledge held within the documents it is not explicitly related (often it is not even very explicitly defined) i.e. the semantic is not explicitly recorded (within documents, or across sets of documents), consequentially they can't address the issue that the set of documents are inconsistent. So in a sense these systems exacerbate the problem by facilitating the publication of even more documents, and maintaining the illusion that if the documents can be located people will understand all that they contain.

Meeting mayhem follows
As the business people need to know the answers, and don't have the capacity to read all the documents - what they do is communicate amongst themselves to get a handle on the issue of interest (workshops, meetings, emails, phone calls etc.). They work out what they need to do (though it takes time) and move on - only to repeat the exercise with next issue (or on the same issue sometime latter).


Strategy, architecture and governance functions are implemented
In most organisations eventually people realise that there is an issue and they put in place people (strategy, architect, governance) who aim to consolidate this knowledge. These people may even create their own documents - that aim to consolidate information around different perspectives (there are of course an uncountable number of different perspectives that occur over time in an complex enterprise). They become someone you can ask questions of. Unfortunately their capacity is usually fairly quickly exceeded - so they become people who know a lot (but seldom enough on any topic), but produce little of value.



Strategy, architecture and governance models eventuate
Some organisations realise that implementing this knowledge management function in a person or team (and their documents) doesn't address the issue of the semantics not be explicitly understood so they move to putting models in place. Models can explicitly manage the semantics and aggregate knowledge from all the sources. The models can, in theory, now answer all the questions. However there are two problems.




Models don't suit most people most of the time
The 1st problem is that while the models may be able to answer all the questions most people don't want to deal with these complex models (i.e. so for many they don't act as an effective communciation mechanism).

Models are time consuming to maintain
The 2nd problem is that one creates a bottle neck i.e. in the effort involved in maintaining the models. Usually people start by aggregating a small subset of the knowledge in the enterprise (i.e. the inputs and outputs that relate to a small set of people and issues). People quickly realise that the more knowledge that is in the model the more powerful the models can be a relating information and answering questions. Unfortunately this increases the load of model maintenance and modelling function become a bottleneck. There are bottlenecks on both the input side (maintaining models) and the output side (explaining how to use the models to answer questions).



A knowledge repository is required
Eventually people realise what is required is a knowledge repository - which contains the information held in the models (all the models) . This allows the strategy, architect and governance functions to be effective i.e. they are no longer bottlenecks and can add much more value. The repository can be used by all parties to record information, answer questions and produce the documents that each requires.



Why consulting firms don't get this
Well they do get. It just doesn't suit them. Smart consulting firms know that if models are not used at all then the knowledge and expertise they offer will always be required (an almost endless demand for their services). If models are used - and they provide the modellers - they will likewise ensure lots of modelling work. Similarly some of internal the people tasked with peforming these functions may fear that if they used such a solution - their work would evaporate. Ironically what actually happens is that they no longer have to act as typing pool (transcribing information) and actually do what most of them are most interested in doing i.e. doing actual strategy, architecture and governance thinking.

Wednesday, August 13, 2008

Industrialisation of IT

(prompted by an organisation with a method for really focusing on business value)

A confluence of factors now allows some organisations to adopt dramatically new ways of getting more value from the systems more quickly - with less investment and less risk. Essentially this involves a non-incremental change (revolutionary) as aspects of IT move from being a craft to an industrial process.

The factors include: the existance in many organisations of extant suites that are capable of supporting most of the existing business processes; the adoption of standards for controlling how systems processes are orchestrated; adoption of common mechanism for systems to communicate; and maturity of the approaches to modelling processes and reporting on the execution of those processes. The acronyms associated with these factors include: ERP, BPEL, SOA, BPM and BAM.

These new approaches allow models to be created that represent how the organisation operates and that at the same time link directly to services (existing, well engineering, industrially tested and documented) in the enterprise suites. Optimising the business performance then becomes a matter of analysing the process models and

It will substantially reduce the level of expenditure that has traditionally been associated with the wanton customisation through low level coding of large application suites (e.g. ERP systems). This customisation is usually poorly conceived from a business perspective i.e. it is expensive, time consuming, risky and worst of all moves the understanding of how the organisation actually operates into code (the organisation doesn't really understand or control). What the customisation does ensure is that large development teams are required - usually provided by one of the "independent" consultants who have lead the organisation down the garden path of customisation in the 1st place. Frequently it will also require additional hardware and systems software so all the vendors of products and services win i.e. everyone wins except the end user.

The class of organisations are those whose major systems are these ERP systems (and other such systems as they mature) and scope is determined by what these suites are designed to do.

As with any revolution the people who benefit from status quo (the craft model) will resist these changes. The crafts people in this case are traditional developers who wish to hand craft systems in preference to assembling solutions from parts.

During the industrialisation of manufacturing to stop the resistance they made machine breaking (industrial sabotage) a capital crime, executed some saboteurs and deported transported many as prisoners to Australia. Almost two hundred years latter we can only hope the resistance will be less vociferous.

If organisation move to this paradigm they can largely remove IT as a blocker to change and the IT organisation can be downsized and redesigned to focus on innovation.