I had an interesting conversation with a highly experienced architect recently:
"Is what you're doing working? Really? Are you sure?"
If you answered yes, then don't change anything.
More than likely however, there are recurring quality problems, defect count is high, bug-fixes are rejected, and developers spend a great deal of time refactoring. You need an architect and better architecture processes.
I pondered about the title of this post, should it be the role of an architect in Agile or just modern architect? Probably the latter, but the question of architecture in agile is something that bugs more than a few people. I certainly don't have all the answers nor proclaim to be an expert, but these are my experiences and observations.
So what's the problem?
Scrum and agile narrations focus on writing code, building code, not the "ideation" and design that is necessary. Who isn't following Scrum or some self proclaimed "flavour of agile" these days right? Almost never does agile documents mention architecture or design. They expect the team to design and build as part of a sprint, or maybe spend the first sprint just on design but no help on how this should work. In my experience this is fraught with difficulty. Design by committee doesn't work well, its usually better to have the most experienced person responsible for design and to properly think through a design and proof of concepts takes time. They should, of course, engage other team members for input and review, but one person needs to be responsible. Things go better if the senior(s) have a vision of what the end-goal architecture should look like, building this vision with clarity is the issue.
Stephen Cohen (Chief Architect, Microsoft): "Scrum went through 3-4 years in its original form before admitting that the architect had any role at all to play."
Juval Lowy (Master Architect, IDesign): "The agile priests would like you to believe that following agile will magically produce architecture that is adaptable, resistant to poor coding, and is scalable."
(Apologies for the paraphrasing).
Some practioners claim there is no need for design and it should be simply part of the implementing the story. This works in only the most trivial software, but in my experience most of us veterans would not often descibe what we do as simple.
More often than not the code produced is badly structured in retrospect and not properly limiting volatility when business change occurs. Agile addresses this by saying "continuously refactor". How much time could be saved by mapping out an over arching architecture up front?
Wait, aren't you saying that you'd rather do waterfall?
No. Architecture != Waterfall.
A wise friend once said:
If you put a bunch of extreme programmers in the middle of a city, let’s say Marrakech, and ask them to visit five tourist hot spots without using a “map”, they will wander around for days exploring every little passage way by brute force. If you give another bunch of developers a “map” and put them in the same city, they will use the map to go directly to each of the five tourist hot spots in a matter of minutes or hours.
Architecture is the map. You'd be crazy not to have a map (or get started creating one).
Simply sitting down with a one liner on a piece of card does not mean any developer from graduate to senior will be able to first estimate, then write tests, then magically produce well designed code that properly encapsulates volatility. This is a naive and utopian ideal that never happens. If the architecture is already mapped out and clearly articulated then it absolutely is.
Once you have a map, in enough detail to be understood by your team, Scrum works okay. It should be done before the team is assembled to begin building.
You should strive to create a shadow board architecture that allows making design decision for new business features easy, consistent and fast.
Ok, so architecture is important, but do we need an Architect?
What skills does agile require of its team members?
- Cross-skilled (Dev, QA, SOA, UI, UX, DBA, Design, and anything else required);
- Able to work on and explain any part of the system;
- TDD & Unit testing;
- QA practices;
- Proven architecture and design skills;
- Industry context knowledge and patterns;
- Ability to write SDK documentation;
- Ability to consistently review code;
- Mentoring skills;
- Able to create processes and procedures to ensure predictable outomes (ie adapt);
- Good communication skills;
- Able to effectively peer program;
Sounds very much like a very senior experienced veteran developer, if not an architect. Every company I've been involved with would like all their developers to be at this level, but all never are. You'd be lucky if 20% of developers are at this level, usually not enough to have one per team. The people with these skills are in high demand, but the company will want to make do with what they have. Which means these people will likely be shared around out of necessity. Agile more than frowns on team members changing teams all the time, and there is no room for negotiation on that, believe me! Even with no management direction senior developers will attempt to fill the design void. So, these highly skilled and experienced people are your architects, if not in name, then they are performing the role de-facto.
No one would argue that business needs are changing faster than ever. This necessitates good design that encapsulates volatility as best it can.
The role of an Architect is most definitely STILL REQUIRED and because of faster pace of change, more relevant than ever.
How should architecture fit into an agile process?
It is common to begin (and sometimes complete) the
UX work before the Scrum team starts work. Architecture is no different, it should be running in parallel to UX.
Architecture should not be a heavy process, it should be mapped out once at the beginning for the whole new system, and sometimes before each major feature addition to an existing product. According to IDesign's "The Method" architecture can be completed in 3 to 5 weeks. It is not something that happens for each user-story.
Some highly respected architects I have spoken to simply state a proper architecture process does not fit with Agile, period. I won't join this debate, however, agile isn't going to step aside any time soon. To date, I haven't ever won the debate on Agile versus "The Method". It will take time. For now, I believe we should work with agile.
Scott Ambler posted his take on fitting architecture into agile processes. The idea is to envision the architecture during the requirements and analysis phase. (Before the development sprints begin). The key is just-enough, then during development keep reiterating over it during the implementation phase. Just-enough for your target team to comprehend. Amend and refine when necessary.
Architecture is not something that is needed for every story. It is the end-goal vision of your system. This may happen once per release or even less frequently. You must have a clear idea of where you want your system to be 3 or 4 releases from now. Its ok to change the vision. The vision doesn't have to consider business features in detail, its the more like an overarching strategy for implementing them, but you should consider likely business extensions and how your architecture will support the changes.
Are you building a pluggable SOA system? If so, are there standard patterns all communications should use? Is it worthwhile applying aspects to all components? What would happen if services organically grew where developers thought it was quickest and easiest to slap them in? What about realistic and likely business changes? Be cynical, its healthy. These are the questions important to consider when designing an architecture.
It takes effort to make something appear easy, and its easy to make something complex. - Unknown.
