IT Directions

In the 3 years since publication of my book “Human Interactions - the Heart and Soul of Business Process Management”, reactions from BPM specialists have been mixed.
One frequent response is: “at last someone is saying what I’ve always thought”. In other words, the mixture of consultancy observations and insights from varied disciplines that is Human Interaction Management (HIM) seems like simple common sense.
Another frequent response is “that can’t be right”. In other words, please don’t let this be right - because if it is, we’ve got to start again, almost from scratch. Both consultants and software vendors are going to have to make dramatic changes to the solutions they are offering.
Current BPM solutions (both method and theory) relegate human collaboration to the fringes of organizational life, treating as an backwater of unmanageable “ad-hoc” work where “here be dragons”. This is not good enough, is it. Organizations need knowledge workers to be:

1. More efficient - work faster and leave tidy information behind. Basex estimate that 28% of knowledge worker time is lost to poor control of interactions. So we need better teams, better communication channels, better generation of knowledge from information, better time management, and better planning tools (these are the 5 HIM principles, by the way). How are current BPM and case management solutions going to give us any of these? To the worker, “human-centric” software offerings are still just task lists, with the added complication that they are now expected to create their own tasks.
2. More effective - adapt their work to changing objectives. The key word is “changing”. It is not enough to allow people to plan their own activities in ad-hoc fashion - this is just a gateway to chaos, in which people go off at tangents the whole time, leaving behind a sea of unstructured audit trails. Human work needs to be part of a management framework geared around driving top-down change (i.e., from current strategy) while empowering bottom-up decision-making (i.e., allowing people to negotiate next steps in a structured way as they go through a process).

In this blog series, I am relating these needs specifically to BPMN, by showing how BPMN notation cannot express constructs that are critical to the efficient and effective management of human work - while HIM notation can do so easily. Here is another example.
In the HIM diagrams featured in previous posts in this series, you will notice that each Role (yellow rectangle) has the word INSTANCE underneath its name. This means that the Role is active in the process - it represents a real, working participant. Some Roles, however, are used differently. Here is another version of the HIM solution to Respond to RFP from a few posts back, again generated using the free HumanEdj:

Notice that this time the Technical Consultant Role is shown as a TYPE rather than an instance. Also, the Account Manager has a new Activity, to assign a Technical Consultant. What is going on?
The new diagram reflects the fact that, in some cases, the Salesperson will not need technical consultancy - they will be familiar enough with the solution to be offered to the client that they can prepare the proposal unaided. In the new version, the Salesperson and Account Manager can discuss this via the Initiate Submit Proposal Interaction - and depending on the outcome of that discussion, the Account Manager may or may not create an “instance” of the Technical Consultant Role “type”.
How would this be depicted in BPMN? You would have to create a swimlane for the Technical Consultancy work, and make all its activities optional in the process. In other words, work has been assigned but may never happen. Further, there is no explicit depiction of how the work will be assigned - under what circumstances, when, who by, to whom, the sort of skills and experience required, and so on. Critical aspects of human resource planning are omitted from the process - of necessity, since there is nowhere to put them.
TAKE AWAY
To remedy process description deficiencies such as those described in this blog series, software vendors typically advocate the use of business rules in conjunction with process execution software based on BPMN and/or XPDL.
While one of the 6 key HIMS object types is a Rule, which can be applied in many ways, Rules alone are not the answer, since in large quantities they are unmaintainable. For routine work, we are moving toward an agent-based approach in which rules are key - when you run a process a million times, the effort of maintaining the rules is worth it. But for human-driven work, in which every process instance is different, there is no value-add in burdening it with huge numbers of rules. Not only does it constrain the work artificially (just ask a doctor whether they would let a system diagnose for them), but there is zero motivation for the people involved to help maintain the rules, since they are only putting themselves out of a job - so people will work around the rules and let them degrade. I’ve seen this in many organizations over the years.
The key issue is that human-driven work is fundamentally different to routine work, for a variety of reasons ranging from socio-economic issues such as globalization (in particular, the availability of very cheap workers) to psychological issues such as team building (as Handy predicted 20 years ago, corporations are becoming virtual). Organizations ignore this at their peril! As Jon Pyke, Chair of the Workflow Management Coalition, wrote in an article on this site in 2006:

Process-based technology that understands the needs of people and supports the inherent “spontaneity” of the human mind is the next logical step, and we might be tempted to name this potential paradigm shift “Knowledge Intensive Business Processes.”

KIBPM falls into two main categories, which will probably merge over time, and the vendor that recognizes that potential will steal a march on the others. At the simplest level we have case management, and secondly, we have Human Interaction Management. I doubt there are many BPM products on the market today which will be able to meet this seismic shift in requirements - certainly those that rely on BPEL and SOA won’t; what’s more, any that have been in the market for longer than five years will need radical surgery to meet the coming challenge.

Why Workflow Sucks

In other words, radical new ideas and accompanying new technology are required, that go way beyond case management. So why have the major software vendors stuck with the kind of workflow and state-based case management solutions that have been available for years? Well, all suite vendors have a huge code base to maintain, and it’s easier to tinker with what they’ve got than to develop new code from scratch. But this isn’t serving the market, or ultimately, themselves.
To thrive in the next 50 years, we need new insights, new practices and (ultimately) new forms of software.

In the last entry to this blog, I challenged readers to depict a very simple process in BPMN - development of a branding package (colours, fonts, logo, artwork, etc), in which:

1. The account manager, designer and artist throw ideas back and forward for review and debate (concepts, images, documents, etc), usually but not always by email;
2. The same person acts as designer and artist at the start, but has the option of handing over the artwork creation to others if their workload becomes too heavy;
3. It is possible to add more artists as needed during the life of the process, all of whom participate in the same communications.

So far no-one has responded to the challenge. This is not surprising, since it is not possible to show any of this in BPMN.
With regard to “throwing ideas back and forward”, message flow in BPMN is limited to a single, one-way send from one pool to another. The send can be repeated if the appropriate looping constructs are used, but it is very hard to depict message flow between more than 2 parties, and any attempt to reproduce the flexible manner in which people exchange messages is doomed to failure. Mainstream BPMS software currently deals with this limitation by claiming that message exchange between colleagues is not really part of a work process - rather, they claim, it is an ad-hoc activity on which no structure can (or needs to) be placed. In other words, what is perhaps the most fundamental tool of knowledge work is relegated to floating around under the organizational radar, in an unmanageable backwater.
With regard to “handing over work”, BPMN has no concept of who does the work. It simply shows what needs to be done, by someone or something. Hence it is impossible to include in a process diagram any indication of the people, or sorts of people, involved. Again, fundamental aspects of knowledge work (acceptance and delegation of responsibility, capabilities, personal characteristics, and so on) are quite literally out of the picture.
Finally, with regard to “adding more” of a certain role, there is no means to achieve this in a BPMN process. It is hard even to imagine how the notation could be extended to support the notion, since the flowchart principles on which BPMN are based do not support such a concept. Yet human resource planning is fundamental to process management, as it is to all management. How is process software supposed to support human resource planning using a notation in which it is not possible to depict, let alone adjust, the resource levels assigned to a work package?
By contrast, here is a simple diagram using Human Interaction Management (HIM) notation that shows all the above constraints:

The horizontal, dashed lines are Interactions - they show message exchange between the 3 parties to the process. Messages can flow in any direction, repeatedly as necessary.
The yellow rectangles are Roles in the process, and the ovals at the top are Users of those Roles. At the start, Dee Zeiner is playing both the Designer and the Artist Roles. However, the diagram includes an extra User, Richard Tist, to whom the Artist Role can be reassigned during the process if Dee takes on other work and has to delegate.
Finally, should it become necessary to add another Artist as extra resource, this can be done in a Human Interaction Management System (HIMS) simply by right-clicking on the Artist Role and asking for another one. The HIMS will prompt for the details of the person to be assigned the work and then duplicate the Role, automatically assigning the new Role to the new person and including the new Role in the same Interactions as the current Role.
TAKE AWAY
There are many other aspects of HIM notation that make it more suitable for collaborative, adaptive human work than BPMN. I will discuss these aspects in future posts in this series. While waiting for the next post in this series, you might like to consider the following puzzle.
The Artist in the process above must create a draft set of colours before starting to create fonts. However, once they have created a draft set of colours, [the semantics of a HIM diagram mean that] they can repeat and interleave the colour and font creation activities as they wish. Similarly, they must create a draft set of fonts before starting to create a logo. However, once they have created a draft set of fonts they can repeat and interleave the colour, font and logo creation activities as they wish. Simple common sense! Yet can you depict this incredibly simple and commonplace pattern of work in BPMN?
You will find that it is necessary to jump through some very artificial hoops in order to draw such a pattern in BPMN - just try to show how the Artist can go directly from colour creation to logo creation only once they have created fonts. Unlike people, a flowchart has no memory. A HIMS does, however. If you are interested to try out the next generation of workplace software, the free HumanEdj (via which the above diagram was generated) is available online for download.

This week I spoke at BPM Europe 2008 in London. Talking to people there, and listening to the other presentations, it seems to me that BPM has reached, at last, an inevitable watershed. Home truths that people have been trying to ignore for years are finally and painfully apparent to everyone.
As you will read on this Web site and others, BPM software is now mainstream - the market has consolidated and all major suite/platform vendors offer some kind of BPM solution. Further, the cultural practices that practitioners consider actually to be BPM (software deployment being advisable only after adopting a process-oriented approach to management) have become reasonably consistent.
Unfortunately, however, the management practices are not keeping up with needs - and as a consequence, the software market is shrinking.
To see this, consider various analyst reports of the last few years. Although all analyst firms and hence the reports are funded mainly by software vendors, the reports themselves tell a grim story. For instance, Gartner in 2004 reported that you could expect ROI of “at least 15%” from BPM. But you can get 10% from a high street savings account! Why go to the expense, trouble, and risk of a BPM project to get 15%?
Around the same time, Butler Group reported that you were unlikely to get any ROI from BPM, since most corporate processes were already fairly efficient. This makes sense, since the management practices most usually associated with BPM date from the 1950s - Lean, Six Sigma, et al are essentially refinements of TQM and Hoshin Kanri as developed by Juran, Deming and others during the reconstruction of post-war Japan.
That was 2004. Now, in 2008, a highly publicised report by BEA, “State of the BPM Market in 2008″, tacitly confirms Butler’s viewpoint.
For a start, the report makes no quantitative estimate of ROI from a BPM project at all. In a document whose only purpose can be to promote adoption of BPM software, this is a telling omission.
Further, the BEA report also includes estimates of the size of the BPM software market from various sources. On average, the size is believed to be around $1bn, predicted to rise to several times that in a few years. This is exactly the same prediction that has been given year on year for a long time now - every year the total size is still around $1bn, predicted hopefully to rise “soon”. When you take into account the fact that every year more and more of this sum is residual revenue - support and maintenance payments on existing installations - the market as a whole is clearly decreasing in size.
The reason that BPM software appears to be doing OK is that the market has consolidated dramatically, from hundreds of vendors to a handful of major players. With so many less vendors, for now at least there is enough to go around.
On a less quantitative note, any BPM practitioner will tell you that enterprise BPM - the Third Wave predicted by Smith and Fingar - does not exist. Many of the speakers at BPM Europe 2008 made this point, including Howard Smith himself. BPM is currently deployed, whether as management practice alone or also using software to automate work, as a point solution - to fix a specific low-level process such as invoicing, claims processing, or membership approval.
What has all this to do with BPMN?
TAKE AWAY
BPMN is a symptom of a deep problem with BPM, which is that its management practices completely ignore leading management thinking since 1960. The great figures of the last half century - Drucker, Handy, Senge, and others - all pointed out that organizations are systems, in which feedback loops cross boundaries. For the organization to operate effectively, people must collaborate in order to make real time changes to running processes. This requires both visibility of what is happening at many levels and empowerment to implement such changes.
Yet mainstream BPM practice, as it currently stands, is this: describing an organization as a hierarchical tree of processes, starting with separate value streams/chains, and descending with inexorable rigour to routine, repetitive flowcharts more suited to enactment by CPUs than to enactment by humans. Such an approach is well suited to the factories with which Deming and Juran were concerned in 1950, but not at all to a modern, globalized organization in which a huge part of the work is collaborative human activity.
In this blog series, I am focusing on a specific aspect of mainstream BPM practice - the use of flowcharts to represent processes, via the de facto standard notation BPMN. To end this post, consider how you might use BPMN to depict the following process - development of a branding package (colours, fonts, logo, artwork, etc), in which:

1. The account manager, designer and artist throw ideas back and forward for review and debate (concepts, images, documents, etc), usually but not always by email;
2. The same person acts as designer and artist at the start, but has the option of handing over the artwork creation to others if their workload becomes too heavy;
3. It is possible to add more artists as needed during the life of the process, all of whom participate in the same communications.

Next time, I will show how these requirements, which are typical of human collaborative activity, cannot be represented in BPMN in any way that is useful. I will also show how very simple it is to represent the process in HIM notation. In doing so, I will (as promised) explain more about the notation demonstrated during my last post.
In the meantime, you may like to find out more about HIM, and the associated software, a HIMS (of which the reference implementation is free). It is the only way you will ever realize the Third Wave.

A commenter on the last post to this blog wrote:
“I agree that BPMN is not targeted to the behaviour of people to people and people to system interaction. Using it for such a purpose is akin to using a chisel to open a paint tin.”
This sums up very neatly a typical attitude to what I call human-driven processes. Like paint swirling around in a tin, collaborative human work is seen as lacking any structure. Human interactions appear to be so complex, so ad-hoc, that it is impossible to apply precise descriptive techniques to what is going on.
The result of this viewpoint is that collaborative processes are brushed under the carpet by business analysts, process designers, solution architects, and other people in a position to help organizations work both more efficiently and more effectively. The latest crop of “human-centric” BPM tools, for example, treat knowledge work as an inconvenient adjunct to routine processes - a sort of exceptional case, in which the polished simplicity of a routine process is unfortunately disrupted by the human desire to think, communicate and achieve. The manner in which editing documents, sending emails, and other mainstays of business life is treated by such BPM products relegates them to the fringes of management awareness - a short-sighted strategy, to say the least, since it is this kind of work that delivers most value. Treating such activities as a temporary step outside the “real” process certainly gives little opportunity for the 59%-80% of the US workforce who are knowledge workers to contribute to the sucess of their organization.
Regular readers of this blog will know that the principles and patterns of Human Interaction Management exist to remedy the situation. Using the simple notation of HIM, in conjunction with a small number of standard patterns, it is possible to understand and improve any type of collaborative work. However, these principles and patterns are relatively new (they were documented for the first time in my 2005 book), so in this blog series I am attempting to illustrate them by example.
The particular example used is a simplified process for Response to Request For Proposal. For comparison, in my last post I drew the process in BPMN and discussed the weaknesses of the representation. In this post I will illustrate how the process can be represented using HIM notation.

You will see immediately that the diagram looks quite different from BPMN. The similarities that exist are in fact superficial - don’t be misled by them into thinking that the notational constructs are equivalent. Here are some key aspects of the diagram:
Roles
Instead of lanes/pools as in a BPMN diagram, with documents/data floating about in thin air, each player in a HIM process has a Role - the vertical rectangles with yellow background. A Role is more than a collection of actions - it is a mini-workspace, that provides you with a lot more than “things to do in a specific order”. For a start, each Role has its own goals and responsibilities - which by agreeing to play a Role, you commit to meeting. To help you meet the goals and responsibilities, each Role has its own private documents/data that you can update and share with colleagues when appropriate. A Role also contains business rules that help you decide when to do what, and that help ensure that your work stays in line with higher-level policies and regulations.
Interactions
Instead of messages from one party to another as in a BPMN diagram, a HIM process has Interactions between Roles - the purple, mainly horizontal lines. A HIM Interaction is a purposeful communication channel - a means by which messages can be exchanged, repeatedly and in any direction, between any number of parties. Think of Interactions as “email plus process context”, although any form of messaging can be used to implement an Interaction.
Collaborative Transactions
Instead of sub-processes as in a BPMN diagram, a HIM process has Collaborative Transactions - stages or phases of a process, as highlighted in the marked-up version of the diagram below. Collaborative Transactions start with an Interaction between all parties (to establish the purpose of the ensuing work), contain various actions for each party, and conclude with another Interaction (to agree that the work is completed and decide on next steps). Collaborative Transactions can be nested, as shown here, without introducing a hierarchical structure foreign to human interactions - in reality, people can talk to one another at any time, whatever part of a process they are supposed to be carrying out.

There is a lot more to say about this diagram, and I will return to it in future posts.
TAKE AWAY
Collaborative human work is a quite different thing from the routine, mostly automated work handled by mainstream BPM techniques and tools. Trying to capture human work using BPMN is not so much like using a chisel to open a paint tin as it is like using a chisel to play the violin - the end result will be frustration with your chisel, serious damage to your violin, and a horrible noise.
The diagram above was drawn using HumanEdj, the reference implementation of a Human Interaction Management System - a modelling and execution environment for human work. The diagram not only shows the work, but can be used by all parties to carry it out. If this blog post strikes a chord with you, why not try out HumanEdj - design some collaborative processes for yourself, and see how easy it is to bring order to the apparent chaos. It may not be possible to eliminate real-world complexity, but it is certainly possible to bring it under control. You just need new techniques and new tools.
NOTE: The diagram shown here was drawn using the latest version of HumanEdj, which will be on public release later this week. If you are using an earlier version, your diagrams will look different.

As a first step towards exploring the depiction of human-driven processes in BPMN, I have chosen a very common process - responding to a Request for Proposal (RFP). Here is my attempt to show a simplified version of this work in BPMN:

From a manager’s perspective, the questions underneath the image are important. The process as it stands could be carried out exactly as shown, but completely fail to deliver what the business needs. Whatever your organization’s “process maturity level”, and however sophisticated your BPM software, implementation of the process without answering these questions is more likely to harm the business than to improve it.
The underlying problem is that the diagram itself offers no hooks on which to hang the answers. Let’s take them in turn.
What are the goals and responsibilities of each player?
A BPMN lane or pool is simply a grouping of activities - it is not an organizational role, with associated goals and responsibilities
How can the Salesperson know what the others are looking for?
Starting from a diagram such as this, the proposal author is effectively working blind - they have no idea on what basis their work will be reviewed, or even who by
To what policies and regulations must the players adhere?
The diagram shows no indication of organizational context - to what conditions the proposal must conform
What skills, experience and personality type should each player possess?
Encoding processes in standard format can be dangerous - without providing any information about the players, it gives the false impression that the work is somehow independent of the people carrying out the activities
What if the Salesperson needs help from another Salesperson to write the proposal?
If the proposal turns out to be too much work for one person in the time available, they may need to share it with someone else - but the diagram offers no means to achieve this
What if the Salesperson needs to discuss matters with the others?
The most efficient way to prepare a document draft is to allow communication with reviewers prior to submission - but BPMN does not allow the depiction of interactive, multi-party communication channels, only one-off messages sent from 1 pool to another as part of a workflow
What if other work apart from document writing is necessary to prepare the proposal?
Writing the proposal document is actually the tip of the iceberg compared to the research, evaluation and analysis that underpins the document - such activities tend to be hard to predict in advance, yet BPMN makes no allowance for on-the-fly adjustment to the process
What supporting information is needed?
It is critical to supply each player with the reference material they need - yet BPMN allows artefacts to be associated with a process only as activity inputs/outputs
How is material containing that supporting information made available to the participants?
It is not enough just to show reference material - the players need to know what form it is in, where to find it, and how to access these locations (account details for a technical journal subscription, for example)
DOES THIS PICTURE CORRESPOND TO THE REAL WORLD?
Anyone who has ever prepared a proposal in response to an RFP will see immediately that this diagram is totally unrealistic - the real world is neither so simple nor so rigid as the workflow depicted
TAKE AWAY
BPMN is a fine process notation - when used for its natural purpose, which is to capture routine work in which the only human activities are data entry and low-level decision making. Work suited to depiction using BPMN is either automated, semi-automated, or so repetitive that one day it will be automated. Effectively, BPMN is a high-level software design notation.
When it comes to human knowledge work, a BPMN diagram simply does not contain the right information. Even if business people can be persuaded to try and understand a notation stuffed full of engineering symbols, a software design notation is the wrong tool to capture collaborative, adaptive, innovative human activity.
In the next postings to this blog, I will continue to explore the application of BPMN to knowledge work, and look at an alternative - the simple notation of Human Interaction Management, which is designed both to capture human work and to support it with software (a Human Interaction Management System).

The last few posts to this blog have featured quotes illustrating the need for 21st century organizations to adopt new management practices. In a nutshell, senior management need to recognize the importance of collaborative, adaptive human work as the true driver of competitive success.
This idea has been promoted by luminaries such as Peter Drucker, Peter Senge and Charles Handy for many years now. However, for the most part organizations are still struggling to improve their knowledge work processes. In fact, it is generous even to call human interaction in most corporate environments “processes” Why is this? When knowledge work makes up somewhere between 59% and 80% of the US workforce, you would think that by now it would be managed properly. However, most organizations are only just getting to grips with their routine work - applying BPM techniques to processing orders, arranging deliveries, payment settlement, and returns handling. Taking a process view of product research, project management, market development, sales support, problem solving, organizational improvement, business growth and other non-routine work is the next step.
Nevertheless, some organizations are preparing now to take this step. This is wise, since the early bird catches the worm. “Asia, Automation and Abundance” have only just started to shake up the global market - the eventual survivors will be those who are more efficient and effective not only in their routine work, but also in their knowledge work.
However, many early adopters of a process viewpoint on knowledge work have had a hard time. This is because, despite the claims of analysts and software vendors for “human-centric BPM”, current mainstream techniques and tools do not cut the mustard. I have emails from (literally) hundreds of organizations who have tried to improve knowledge work processes using solutions from a mainstream vendor, only to find that they are using the wrong tool for the job.
TAKE AWAY
This post is the introduction to a new series, in which I will show why this is, using as an example the de facto standard BPM notation - BPMN. I will take simple scenarios drawn from everyday knowledge work and show how BPMN cannot express these situations. I will also show how easy they are to express in the Human Interaction Management notation (and thus to implement using a Human Interaction Management System, or HIMS).
Please join me in future posts for Keith’s adventures in BPMN.

“In a wired world, the possibilities for assembling, then disbanding, geographically scattered project teams are almost endless. A growing number of companies are relying on electronically-connected virtual project teams to get things done. In time, the primary work unit in the enterprise is likely to be ‘the project’, not the function performed somewhere in the Org Chart.
Virtual, matrixed teams, composed of diverse competencies, knowledge and capabilities, with traditional hierarchical structures replaced by multi-company teams, will be assembled as needed, sometimes ‘on the fly’. Such dynamically formed teams will tackle specific projects to pursue specific opportunities and to counter specific threats in the brave new world of total global competition.”
“Are You Ready for Extreme Competition?”, Peter Fingar

“Companies aren’t going to deal with [the commodity trap] by squeezing another 1% of cost savings out of their invoice settlement processes, or reducing their order fulfillment time by a day - typical results for today’s BPM implementations. Agility in the high-change global economy is all about human-driven business processes: unique case management, ad-hoc collaborations, dynamic and just-in-time sourcing from a network of far-flung suppliers, complex sales proposals, open innovation collaborations, new product development and the like.”
“Shift Happens”, Peter Fingar