Friday, August 24, 2012

New Version of Type Visualiser

Version 1.0.18 has been released of my Type Visualiser Tool.  This release contains some cool new features  but the biggest change was by far shifting to a more strict MVVM approach.

When writing the first release the agile-ist in me wanted to get a prototype up and running as quickly as possible.  This resulted in a great deal of code behind to dynamically draw the visualisation.  The driver behind the change was the increasing number of bugs, inconsistency of behaviour and ultimately difficultly of adding new features.

By changing to a strict MVVM approach and heavily leveraging data-binding most of the code behind shifted into controllers and the model.  Once complete the justification was evident by the ease of adding new some new features.

Here's a summary of the new features:

  • All diagrams will visualise all parent types back to but not including System.Object.
    This includes interfaces and inherited interfaces.
  • Secondary associations can be shown by toggling the show/hide option from the Hide menu.  This will show associations between all other types on the diagram. By default only relationships back to the main subject of the diagram are shown.  It can get a little busy which is the reason why secondary associations are hidden by default.
  • Navigating to another type will now display the selected type in a new tab rather than changing the current diagram.  This allows much more information to be visualised more easily.

  • The canvas that the diagram sits on can be expanded to better accommodate dragging elements around. By mouse dragging a type and pushing the edge of the diagram canvas the canvas will expand.

Saturday, August 4, 2012

The role of an Architect in Agile Part 1

I had an interesting conversation with a highly experienced architect recently:
"Is what you're doing working? Really? Are you sure?"
If you answered yes, then don't change anything.
More than likely however, there are recurring quality problems, defect count is high, bug-fixes are rejected, and developers spend a great deal of time refactoring. You need an architect and better architecture processes.

I pondered about the title of this post, should it be the role of an architect in Agile or just modern architect? Probably the latter, but the question of architecture in agile is something that bugs more than a few people. I certainly don't have all the answers nor proclaim to be an expert, but these are my experiences and observations.

So what's the problem?

Scrum and agile narrations focus on writing code, building code, not the "ideation" and design that is necessary.  Who isn't following Scrum or some self proclaimed "flavour of agile" these days right? Almost never does agile documents mention architecture or design.  They expect the team to design and build as part of a sprint, or maybe spend the first sprint just on design but no help on how this should work. In my experience this is fraught with difficulty. Design by committee doesn't work well, its usually better to have the most experienced person responsible for design and to properly think through a design and proof of concepts takes time. They should, of course, engage other team members for input and review, but one person needs to be responsible. Things go better if the senior(s) have a vision of what the end-goal architecture should look like, building this vision with clarity is the issue.

Stephen Cohen (Chief Architect, Microsoft): "Scrum went through 3-4 years in its original form before admitting that the architect had any role at all to play."

Juval Lowy (Master Architect, IDesign): "The agile priests would like you to believe that following agile will magically produce architecture that is adaptable, resistant to poor coding, and is scalable."
(Apologies for the paraphrasing).

Some practioners claim there is no need for design and it should be simply part of the implementing the story. This works in only the most trivial software, but in my experience most of us veterans would not often descibe what we do as simple. More often than not the code produced is badly structured in retrospect and not properly limiting volatility when business change occurs. Agile addresses this by saying "continuously refactor". How much time could be saved by mapping out an over arching architecture up front?

Wait, aren't you saying that you'd rather do waterfall?

No.  Architecture != Waterfall.

A wise friend once said:
If you put a bunch of extreme programmers in the middle of a city, let’s say Marrakech, and ask them to visit five tourist hot spots without using a “map”, they will wander around for days exploring every little passage way by brute force. If you give another bunch of developers a “map” and put them in the same city, they will use the map to go directly to each of the five tourist hot spots in a matter of minutes or hours.

Architecture is the map. You'd be crazy not to have a map (or get started creating one).

Simply sitting down with a one liner on a piece of card does not mean any developer from graduate to senior will be able to first estimate, then write tests, then magically produce well designed code that properly encapsulates volatility. This is a naive and utopian ideal that never happens.  If the architecture is already mapped out and clearly articulated then it absolutely is.


Once you have a map, in enough detail to be understood by your team, Scrum works okay. It should be done before the team is assembled to begin building.  

You should strive to create a shadow board architecture that allows making design decision for new business features easy, consistent and fast.

Ok, so architecture is important, but do we need an Architect?

What skills does agile require of its team members?
  • Cross-skilled (Dev, QA, SOA, UI, UX, DBA, Design, and anything else required);
  • Able to work on and explain any part of the system;
  • TDD & Unit testing;
  • QA practices;
  • Proven architecture and design skills;
  • Industry context knowledge and patterns;
  • Ability to write SDK documentation;
  • Ability to consistently review code;
  • Mentoring skills;
  • Able to create processes and procedures to ensure predictable outomes (ie adapt);
  • Good communication skills;
  • Able to effectively peer program;
Sounds very much like a very senior experienced veteran developer, if not an architect.  Every company I've been involved with would like all their developers to be at this level, but all never are. You'd be lucky if 20% of developers are at this level, usually not enough to have one per team. The people with these skills are in high demand, but the company will want to make do with what they have. Which means these people will likely be shared around out of necessity. Agile more than frowns on team members changing teams all the time, and there is no room for negotiation on that, believe me!  Even with no management direction senior developers will attempt to fill the design void. So, these highly skilled and experienced people are your architects, if not in name, then they are performing the role de-facto.

No one would argue that business needs are changing faster than ever. This necessitates good design that encapsulates volatility as best it can. The role of an Architect is most definitely STILL REQUIRED and because of faster pace of change, more relevant than ever.  

How should architecture fit into an agile process?

It is common to begin (and sometimes complete) the UX work before the Scrum team starts work. Architecture is no different, it should be running in parallel to UX.


Architecture should not be a heavy process, it should be mapped out once at the beginning for the whole new system, and sometimes before each major feature addition to an existing product.  According to  IDesign's "The Method" architecture can be completed in 3 to 5 weeks.  It is not something that happens for each user-story.


Some highly respected architects I have spoken to simply state a proper architecture process does not fit with Agile, period. I won't join this debate, however, agile isn't going to step aside any time soon. To date, I haven't ever won the debate on Agile versus "The Method". It will take time. For now, I believe we should work with agile. 


Scott Ambler posted his take on fitting architecture into agile processes. The idea is to envision the architecture during the requirements and analysis phase. (Before the development sprints begin).  The key is just-enough, then during development keep reiterating over it during the implementation phase.  Just-enough for your target team to comprehend.  Amend and refine when necessary.

Architecture is not something that is needed for every story.  It is the end-goal vision of your system.  This may happen once per release or even less frequently. You must have a clear idea of where you want your system to be 3 or 4 releases from now.  Its ok to change the vision.  The vision doesn't have to consider business features in detail, its the more like an overarching strategy for implementing them, but you should consider likely business extensions and how your architecture will support the changes.

Are you building a pluggable SOA system? If so, are there standard patterns all communications should use?  Is it worthwhile applying aspects to all components? What would happen if services organically grew where developers thought it was quickest and easiest to slap them in? What about realistic and likely business changes? Be cynical, its healthy. These are the questions important to consider when designing an architecture.

It takes effort to make something appear easy, and its easy to make something complex. - Unknown.


The role of an Architect in Agile Part 2

What should an architect be responsible for?

An architect or team of architects should be responsible for: 
  • Assisting in gathering and analysing requirements.
  • Sell and negotiate technical constraints and aspects to stakeholders.
  • Articulate the end goal architectural vision designed to encapsulate change.
  • Communicate with diagrams the intended architecture and implementation plan.
  • Prove (or disprove) technologies and techniques (prototyping).
  • Own the definition of integration points and service interfaces.
  • Establish processes to ensure predictable outcomes and quality in the SDLC.
  • Oversee delivery of software in accordance with the intended architecture (reviews).
  • Acceptable quality levels (coding standards, code metrics and performance parameters).
The architect is primarily responsible for creating a big picture of how a system should be assembled, integrated and where extension points are for probable business additions. This should include mostly diagrams, lists and pseudo-code, because developers don't read. The key is to keep it simple, just enough for the targeted team.

Controversially IDesign's "The Method" argues that the architect should also project manage. The explanation was simple, who else knows best how things integrate and their dependencies?

During implementation the job isn't done. Architecture is also governance of ensuring all these things have happened. An architect should oversee a feature from inception to implementation to final load and concurrency testing.  A Scrum team should only be working on one feature at a time (this is a founding principle of agile - focus and don't context switch).  This allows an architect to potentially oversee two features across two teams at the same time. Its also worth considering the architect joining the team as a development resource, if resources permit.

Any respectable Scrum coach will tell you that Scrum does not mandate no design. It simply states do what is required. If that means design can be done inside the same sprint as implementation, do it, although that is highly unlikely and I wouldn't advise it.  The best plan is to complete the necessary design work prior to initiating a stream of work with the Scrum team. The trick is don't do too much. Too much will vary depending on strength of experience in the team, and how effective they are at communicating. The more inexperienced the team the more design work needed.

Having architecture artifacts up front before starting a piece of work, and for grooming sessions, means it is highly visible and scrutised. This inevitably improves the design, and reduces unknowns.


Who's the boss?

Answer: The client. The Product Owner (PO) represents and speaks for the client. That is not to say the architect shouldn't have direct contact with the client, the PO is there to help you with this. Embracing change is the key to a successful software product. Often more valuable information comes to light during the project. Expect change. However, an inexperienced PO will change their mind more often than necessary. This can be addressed with a little more architecture work, this will keep the PO's reasoning honest. Any whiff of analysis paralysis means you either dump the feature/use-case or dumb it down and start the Scrum process with basic a implementation only. Unfortunately a PO that changes their mind too often results in poor qualilty software that costs more than it should to maintain or extend.

A good PO will work with their architect during the sprint review to ensure the quality metrics are adaquate and all agreed standards have been met.  Although the architect is technical, they are definitely in the Product Owner camp. The PO should lean on their architect for technical advice and verification of the product delivered.  The PO should fail the sprint if the architect can prove that the implementation doesn't comply with the design, or standards have not been met.

IDesign's "The Method" advocates the architect head up the project team and be the one to call the shots and sign-off the final product. This is a very tough sell to the establishment today, but it could be the future.

Some tips on how an architect can go about their job?

(Thanks to IDesign's Michael Montgomery for the inspiration for this list).
  • Formalise and document your requirements gathering approach.
  • Formalise document templates to store the gathered information. 
  • Don't gather information using sticky notes! Save it digitally using defined consistent templates.
  • No gold-plating. Everything is “just enough” based on known and likely use-cases, team composition, and hand-off-point.
  • Iterate on everything, including requirements, getting “just enough” before the Design Phase begins. Gather - Refine - Review/Assess, Gather - Refine - Review/Assess, until your ready to start implementation.
  • Continually reinforce to the BA's and PO's that they bring the ‘when, what, why, who’, but never the how.
  • Developers don’t read. Format use cases as ‘pseudo-code’ (i.e. numbered/bulleted ‘lists’) and diagrams.
  • Never use the term ‘spec’. Call it the ‘requirements lists’ or 'acceptance criteria'.
  • With the advent of UX, separate functional (UX/UI) from back-end service operational (SOA) requirements lists. Treat back end services as a different product to a client UI. They will have their own lifetimes driven by the same use cases.
  • UX is different from UI is different from SOA is different from Framework. Each may need its own requirements treatment, depending on the size and complexity of the system.
  • Developers don’t do UX (although they may think they can - don't believe them).
  • The UX guys must involve stakeholders early and often. This will need to be quite mature before implementation can begin.
  • For user driven applications (which is most), it’s well-crafted UX workflows that produce clearly defined use cases that produce succinct SOA. If you can get the UX guys ‘out in front of the ball’, you put yourself in a sweet spot. This particularly holds true in composite UX.

Conclusion

Given a choice, I would prefer to use IDesign's "The Method". My problem with it has been convincing employers and stakeholders, even though it's a new approach, it's a proven one. Others with superhuman sales and negotiation skills may have more success than I. Working in with Scrum, its roles, and other established roles has been a given for me; and I'm betting you too. However, I believe this is completely realistic in a hybrid model. The role of an architect is absolutely essential, but the role and its skills are far more than a senior developer's.  An architect's number one skill, is their communication skills.  In my opinion as an industry we need to formalise roles, and the architect role is paramount and akin to the building industry's architect in responsibility but more similar to a head-chef in activities.

References and more information

Scott Ambler on User Stories
Scott Ambler on Architecture in Agile
IDesign.Net
http://www.idesign.net/articles/agile_and_the_architect.htm
Able Architecture Part 1: 
Able Architecture Part 2: