Here's a great "step back and think" article I came across on StackOverflow on class design.
Its always so tempting to just dive in and write some code, which usually turns into a heavy refactor cycle.
http://stackoverflow.com/a/4203836/553404
Showing posts with label Architecture. Show all posts
Showing posts with label Architecture. Show all posts
Monday, November 25, 2013
Tuesday, September 17, 2013
Documenting Technical Design - Ideas Smorgasbord
I get asked a lot what Technical Design is, sometimes what it isn't, and what to include in a technical document. As always the answer is it depends. It depends on your audience, their skill set and level, and the environment.
Here's a few ideas that I have found work. This isn't a comprehensive list that should always be included, again think about who your audience is and would they be interested (relevance) in this section. Consider the analogy of building a car: The person who will use it doesn't care about the tensile strength of the second and fourth piston. They care about how to drive it, maybe maintaining it, and what to do when something abnormal happens. But remember it depends on the audience, the mechanic will want different information, but he still won't necessarily care about the technology used to build the piston.
Technical Design should consider the following topics/sections: (In no particular order).
Tech Design is not:
A See Also Section.
List links to other relevant or related information.
Overview Section.
No more than two paragraphs. This is for business or non-techies who just want to know something about how the system works and how it fits into
the bigger picture with other systems. What is it used for? What problem does it solve?
System Context.
A single diagram showing this system as a single box and the other systems it integrates into.
Main Sequences and Flows.
This is the main or significant work flows thru the system. Use Abstract high level sequence diagrams. Each line should not represent a class but
a major component / library / namespace within the system. Do not get into class and object detail, it will get out of date too quickly.
Hosting and Deployment.
List all different options on how to host in a production environment. Also state the recommended or preferred hosting model. State why there are
multiple models. Use deployment diagrams not text. Try to show one box per deployed application, not one box per library. Draw lines only where
external calls are made between applications. Add text to each line describing http/tcp, authentication, authorisation, json/soap etc.
Could include details of configuration options and environment settings required, if necessary. This is not a installation guide.
How to integrate.
Link to a document (best to keep it separate as it will not be relevant to all readers and could be long) describing how a developer takes the
system as a framework and uses it. How do they install it, use it, and what do they need to do, how long is this getting up and running process
going to take. Obviously only applies if the system is a framework for developers rather than for consumers.
Multi-Tenancy
Is the system multi-tenanted. If so how have you acheived this, trade-offs made, limitations etc. How many tenants can it handle, how few?
Concurrency
Multi-threading? STA? UI Threading? ASP.Net async? Task Factory - how?
Code Organisation.
How is the code organised and why. Possibly code metrics too so another party can get a sense for the size and complexity of the application.
Service Contracts, Endpoints and Data Contracts
Describe all inbound services and their contracts. Describe all outbound service calls, their purpose, when they happen. Also define or link to a
data dictionary that describes all SOAP / Json payloads and fields.
Diagnostic Logging, Monitoring and Auditing
Database tables involved. What logging framework? Links to configure it. Default config (debug/release builds). What is auditing vs logging.
Why is auditing required. Performance counters and definition. Other standard recommended performance counters. How do the Ops team support
the system? Monitor it? Can they tell when it is failing before it fails? Windows Event Log. Health / Support service endpoints that might be available.
(Ping, heartbeat, health check services).
Resilience
Estimated usage stats (year 1,2,3). Estimate db size, requests per second, tenants, 24 hour (or other time period) load patterns.
Where are the fail points from a business perspective? Is 1000ms per request ok? 3000? 10,000ms? 2 hours to process the last item in a queue behind a
large batch?
Exceptions: Different scenarios - (db is down, external 3rd party service is down, one server in the farm is down, entry point service endpoint is down,
validation issues with inbound service data, inflight data cannot be updated back to db after reading it, system crashes during processing - how to
recover? etc) General exception strategy within the app. Where are they logged or not.
Security
What security is applied to all endpoints. User security. Pen testing considerations. OWASP. STRIDE. Known security concerns to address and how they
are addressed.
Database.
Trusted subsystem?
Required infrastructure security.
Deployment time required security config not covered by installer.
Configuration Options for Runtime System.
Are there runtime settings applied immediately without restart? App.config only? Other config? How are server farms updated?
All config options listed and described. Or links to other doc.
Database Schema
Data Dictionary or link to
Describe data access strategy - stored procs, ORM, both, etc
Backwards Compatibility
Are the existing devices out there with a version of the application. Ie are you upgrading backend services and there are existing UI's out there?
Forwards Compatibility: How will future versions be compatible with this?
Are your service contracts / Json transmissions versioned adequately?
Are you recommending to clients that the URL contain a version number?
Other important / architecturally significant items
Queueing / ServiceBus / Asynchronous design
Installation issues - not an install guide
Availability (do you need a draining shutdown, how much down time can you get / need)
Usability
Accessibility
Here's a few ideas that I have found work. This isn't a comprehensive list that should always be included, again think about who your audience is and would they be interested (relevance) in this section. Consider the analogy of building a car: The person who will use it doesn't care about the tensile strength of the second and fourth piston. They care about how to drive it, maybe maintaining it, and what to do when something abnormal happens. But remember it depends on the audience, the mechanic will want different information, but he still won't necessarily care about the technology used to build the piston.
Technical Design should consider the following topics/sections: (In no particular order).
Tech Design is not:
- A complete War And Peace of how every line works. (Tech Design should be as short as possible using as many diagrams as possible to avoid text).
- Sequence Diagrams based exactly on code or Class diagrams based exactly on code. (This will go stale too quickly and creates a maintenance nightmare)
- An install guide.
- Xml Code Comments/SandCastle.
- Any Documentation Generated from code.
- An API Guide.
- A Test Plan.
- How-to-use / user guide (this should be a separate document).
A See Also Section.
List links to other relevant or related information.
Overview Section.
No more than two paragraphs. This is for business or non-techies who just want to know something about how the system works and how it fits into
the bigger picture with other systems. What is it used for? What problem does it solve?
System Context.
A single diagram showing this system as a single box and the other systems it integrates into.
Main Sequences and Flows.
This is the main or significant work flows thru the system. Use Abstract high level sequence diagrams. Each line should not represent a class but
a major component / library / namespace within the system. Do not get into class and object detail, it will get out of date too quickly.
Hosting and Deployment.
List all different options on how to host in a production environment. Also state the recommended or preferred hosting model. State why there are
multiple models. Use deployment diagrams not text. Try to show one box per deployed application, not one box per library. Draw lines only where
external calls are made between applications. Add text to each line describing http/tcp, authentication, authorisation, json/soap etc.
Could include details of configuration options and environment settings required, if necessary. This is not a installation guide.
How to integrate.
Link to a document (best to keep it separate as it will not be relevant to all readers and could be long) describing how a developer takes the
system as a framework and uses it. How do they install it, use it, and what do they need to do, how long is this getting up and running process
going to take. Obviously only applies if the system is a framework for developers rather than for consumers.
Multi-Tenancy
Is the system multi-tenanted. If so how have you acheived this, trade-offs made, limitations etc. How many tenants can it handle, how few?
Concurrency
Multi-threading? STA? UI Threading? ASP.Net async? Task Factory - how?
Code Organisation.
How is the code organised and why. Possibly code metrics too so another party can get a sense for the size and complexity of the application.
Service Contracts, Endpoints and Data Contracts
Describe all inbound services and their contracts. Describe all outbound service calls, their purpose, when they happen. Also define or link to a
data dictionary that describes all SOAP / Json payloads and fields.
Diagnostic Logging, Monitoring and Auditing
Database tables involved. What logging framework? Links to configure it. Default config (debug/release builds). What is auditing vs logging.
Why is auditing required. Performance counters and definition. Other standard recommended performance counters. How do the Ops team support
the system? Monitor it? Can they tell when it is failing before it fails? Windows Event Log. Health / Support service endpoints that might be available.
(Ping, heartbeat, health check services).
Resilience
Estimated usage stats (year 1,2,3). Estimate db size, requests per second, tenants, 24 hour (or other time period) load patterns.
Where are the fail points from a business perspective? Is 1000ms per request ok? 3000? 10,000ms? 2 hours to process the last item in a queue behind a
large batch?
Exceptions: Different scenarios - (db is down, external 3rd party service is down, one server in the farm is down, entry point service endpoint is down,
validation issues with inbound service data, inflight data cannot be updated back to db after reading it, system crashes during processing - how to
recover? etc) General exception strategy within the app. Where are they logged or not.
Security
What security is applied to all endpoints. User security. Pen testing considerations. OWASP. STRIDE. Known security concerns to address and how they
are addressed.
Database.
Trusted subsystem?
Required infrastructure security.
Deployment time required security config not covered by installer.
Configuration Options for Runtime System.
Are there runtime settings applied immediately without restart? App.config only? Other config? How are server farms updated?
All config options listed and described. Or links to other doc.
Database Schema
Data Dictionary or link to
Describe data access strategy - stored procs, ORM, both, etc
Backwards Compatibility
Are the existing devices out there with a version of the application. Ie are you upgrading backend services and there are existing UI's out there?
Forwards Compatibility: How will future versions be compatible with this?
Are your service contracts / Json transmissions versioned adequately?
Are you recommending to clients that the URL contain a version number?
Other important / architecturally significant items
Queueing / ServiceBus / Asynchronous design
Installation issues - not an install guide
Availability (do you need a draining shutdown, how much down time can you get / need)
Usability
Accessibility
Friday, September 6, 2013
REST/Web API and WCF
Recently I had an opportunity to ask some questions to Daniel Roth from the Microsoft Asp.Net Web API team directly. I found the answers quite interesting, so I thought I'd share. It almost seems like WCF-Http is falling out of favour: "...all future innovations around REST and HTTP services will be focused on WebApi..."
Daniel: While you can use WCF to develop RESTful services, WCF at its core was intended to be a SOAP stack and this bleeds through many of its abstractions. ASP.NET Web API was intended to be a pure HTTP stack from the its inception. It provides a first-class HTTP programming model (HttpRequestMessage, HttpResponseMessage) that is symmetrically supported on both client and server. It provides first class support for things like content-negotiation, JSON, XML, form URL encoded data, OData, custom formats, HTTP headers and status codes, etc. If your goal is to create front end HTTP services that can reach the broadest set of clients, then ASP.NET Web API is probably your best choice. ASP.NET Web API is also a great platform choice for following the REST architectural style. WCF is still a great choice for backend services where you have complete flexibility over the transport protocol and encoding.
Me: What is the future road map for Wcf and WebApi? Will WCF change into a non-http service platform and WebApi take over in the Http space?
Daniel: The WCF Web HTTP programming model will continue to be supported, but all future innovations around REST and HTTP services will be focused on ASP.NET Web API.
Me: Is it possible to use Transactions and/or Message Security over WebApi? If not what do you recommend?
Daniel: There is no standard way of flowing transaction scopes over HTTP outside of SOAP that I know of. We are seeing more folks move to a RESTful compensation model instead of trying to manage a global transaction. In general flowing transactions across the Web is probably not a good idea. Message Security is typically referred to in the context of SOAP and ASP.NET Web API is not a SOAP stack. You can of course manually encrypt the request or response payloads if you’d like or just use HTTPS.
Me: My current understanding of the WebApi from a security stand point is that it is best suited for simple services that don't have a need for strong security.
Daniel: Web APIs generally do need strong security as they often have broad exposure to a variety of clients including malicious ones. Standard HTTP security mechanisms apply (HTTPS, HTTP auth schemes, OAuth, etc) WebApi is best suited for RESTful Json and Pox in terms of performance and simplicity. JSON is very common for Web APIs, but ASP.NET Web API allows you to easily plug in support for any format that you can think of. There are a lot of MIME types out there!
Me: WCF has adequate support for Json, and would be a safe choice, if the service is intended to be exposed using other protocols as well as Json.
Daniel: Right, there is an argument to be made that WCF makes it really easy to add an HTTP endpoint along with your SOAP endpoints. However, we find that most customers quickly run into limitations with WCF when working with HTTP directly and start to feel like they are swimming upstream. ASP.NET Web API and WCF can cleanly sit side-by-side for these scenarios.
Me: WCF is still the best choice for Tcp, MsMq, and Named Pipes.
Daniel: Yup
Daniel: All standard HTTP security mechanisms apply: HTTPS+client cert, Basic, Forms, Windows, OAuth2, claims based auth, etc.
Me: WCF has better runtime flexibility for tweaking hosting and endpoint configuration.
Daniel: In general ASP.NET Web API has the same level of flexibility as WCF and it is much easier to configure. The main difference is that ASP.NET Web API is HTTP specific and WCF is a transport agnostic SOAP stack.
Daniel: For vanilla web APIs this is true. Most public web APIs rely on some form of public docs and SDKs to deal with this issue (like https://dev.twitter.com/). However, we are building out support for OData which does have a metadata and client code generation story.
Me: WCF4 has support for routing (both transparent and not).
Daniel: WCF has virtually no support for routing (no parameters, no constraints). The route template is a constant base address and then you use UriTemplates to dispatch requests. ASP.NET Web API has full support for ASP.NET Routing.
Daniel: Hope this helps! You can also send future questions to webapitalk.
After this some colleagues of mine on a private architecture forum discussed this at length.
Me: I am a little confused how Web API fits in with WCF. Basically, I believe WCF to be the full-featured choice for flexibility, extensibility and security. Web API seems to be a quick “time-to-market” approach that is simple but offers little in the form of extensibility and security.
Daniel: ASP.NET Web API is a first-class HTTP stack for building HTTP services that target a broad range of clients including browsers and mobile devices. ASP.NET Web API is very extensible and powerful although it is specifically targeted at HTTP scenarios only. ASP.NET Web API makes it very easy to treat HTTP as an application layer protocol, unlike WCF which in general treats HTTP as a transport. ASP.NET Web API leverages standard HTTP mechanism for security. We do plan to publish guidance about when to use WCF vs ASP.NET Web API later this month on MSDN.
Me: There's a lot of ambiguity out there on how to use WebApi, most blogs claim its a bolt-on to your existing MVC site to enable mobile App consumption. It doesn’t sound like good architecture to use a Web site (which is a client), as a service API? What factors have driven Ms to infer that business logic should be tied up in the client tier and not encapsulated in a service?
Daniel: ASP.NET Web API is not tied to a web site client. Web APIs can and generally are implemented as a loosely coupled services tier. Public examples include the Facebook and Twitter APIs. You can even host ASP.NET Web API in your own process if you want to like you can with WCF.
Me: My current thinking is that WCF is my best bet for my current design, but I’m wondering what is there to gain by switch to Web API service end point?
Daniel: While you can use WCF to develop RESTful services, WCF at its core was intended to be a SOAP stack and this bleeds through many of its abstractions. ASP.NET Web API was intended to be a pure HTTP stack from the its inception. It provides a first-class HTTP programming model (HttpRequestMessage, HttpResponseMessage) that is symmetrically supported on both client and server. It provides first class support for things like content-negotiation, JSON, XML, form URL encoded data, OData, custom formats, HTTP headers and status codes, etc. If your goal is to create front end HTTP services that can reach the broadest set of clients, then ASP.NET Web API is probably your best choice. ASP.NET Web API is also a great platform choice for following the REST architectural style. WCF is still a great choice for backend services where you have complete flexibility over the transport protocol and encoding.
Me: What is the future road map for Wcf and WebApi? Will WCF change into a non-http service platform and WebApi take over in the Http space?
Daniel: The WCF Web HTTP programming model will continue to be supported, but all future innovations around REST and HTTP services will be focused on ASP.NET Web API.
Me: Is it possible to use Transactions and/or Message Security over WebApi? If not what do you recommend?
Daniel: There is no standard way of flowing transaction scopes over HTTP outside of SOAP that I know of. We are seeing more folks move to a RESTful compensation model instead of trying to manage a global transaction. In general flowing transactions across the Web is probably not a good idea. Message Security is typically referred to in the context of SOAP and ASP.NET Web API is not a SOAP stack. You can of course manually encrypt the request or response payloads if you’d like or just use HTTPS.
Me: My current understanding of the WebApi from a security stand point is that it is best suited for simple services that don't have a need for strong security.
Daniel: Web APIs generally do need strong security as they often have broad exposure to a variety of clients including malicious ones. Standard HTTP security mechanisms apply (HTTPS, HTTP auth schemes, OAuth, etc) WebApi is best suited for RESTful Json and Pox in terms of performance and simplicity. JSON is very common for Web APIs, but ASP.NET Web API allows you to easily plug in support for any format that you can think of. There are a lot of MIME types out there!
Me: WCF has adequate support for Json, and would be a safe choice, if the service is intended to be exposed using other protocols as well as Json.
Daniel: Right, there is an argument to be made that WCF makes it really easy to add an HTTP endpoint along with your SOAP endpoints. However, we find that most customers quickly run into limitations with WCF when working with HTTP directly and start to feel like they are swimming upstream. ASP.NET Web API and WCF can cleanly sit side-by-side for these scenarios.
Me: WCF is still the best choice for Tcp, MsMq, and Named Pipes.
Daniel: Yup
Me: What forms of authentication does WebApi support?
Daniel: All standard HTTP security mechanisms apply: HTTPS+client cert, Basic, Forms, Windows, OAuth2, claims based auth, etc.
Me: WCF has better runtime flexibility for tweaking hosting and endpoint configuration.
Daniel: In general ASP.NET Web API has the same level of flexibility as WCF and it is much easier to configure. The main difference is that ASP.NET Web API is HTTP specific and WCF is a transport agnostic SOAP stack.
Me: WebApi doesn't have any built in mechanism for defining a message level contract and the exchange of this metadata (WSDL).
Daniel: For vanilla web APIs this is true. Most public web APIs rely on some form of public docs and SDKs to deal with this issue (like https://dev.twitter.com/). However, we are building out support for OData which does have a metadata and client code generation story.
Me: WCF4 has support for routing (both transparent and not).
Daniel: WCF has virtually no support for routing (no parameters, no constraints). The route template is a constant base address and then you use UriTemplates to dispatch requests. ASP.NET Web API has full support for ASP.NET Routing.
Daniel: Hope this helps! You can also send future questions to webapitalk.
After this some colleagues of mine on a private architecture forum discussed this at length.
Juval Lowy: This is a great exchange and it covers a lot, but none of it is new – this is a pretty good statement of where things are, and have been for a long time. Even before WebAPI it was a dead-end to use REST/WCF where you could use WCF, which is why I never covered it in my books (the books were about how to use WCF, not how best to do without it). The bottom line is and has always been: if you can use the full spectrum of WCF with reliability, transactions, security, discovery, encoding and the 20 other features - great. If you can't – choose carefully from a variety of options, WebApi being just one of them.
Anonymous: On your question about the lack of a WSDL... Check out the ApiExplorer.
Anonymous: Is there a connectivity technology more in need of WS-* than Mobile?
Michael "Monty" Montgomery: Ever notice how everyone’s JSON packets are looking more and more like SOAP envelopes?
I’m just sayin’…
Juval: Very much agreed on all counts. Inspired me to get on a soapbox (pun intended) and make a few bigger picture points on what the proper motives should be for using REST to connect applications.
There is some context that should be considered when making general (and true) statements such as “the web is REST.” The web is indeed REST, and if one watches a WCF client talk to a WCF service (using some wire sniffer) over one of the SOAP http bindings, you’ll see REST. The RESTful interactions are simply lower down in the stack. We all know this quite well. So taking this statement: “the web is REST and the web is HTTP“ allow me to refactor it in a more architecturally minded manner:
HTTP is REST.
The question is, do we just adopt a SOAP stack to go over HTTP (for the web in our current conversation) and have the option to use any number of its features which are already there and ready to go? Or do we skip SOAP and just form the RESTful interactions ourselves? So we skip supporting SOAP, it’s too difficult…Then what? Are we not just re-implementing the features of the WS-* standards in a non-standard way (as Monty alluded to)?
That’s not an ideal situation to be in… The only reason to use REST, in my opinion, is only after we’ve admitted the situation is indeed not ideal.
We can’t wait for the vendors to adopt a common set of standards tomorrow, so for today, we’ll do it ourselves over REST. Eventually, all the disparate groups who are trying to implement SOAP’s features in their own manner (without admitting to themselves that’s what they’re actually doing) over REST (because it’s so much “simpler”) will have to admit that all their effort must be shot in the head (like so many VB6 diehards have had to do). Our responsibility is to make smart decisions in how invested we really should get into implementing needed fanciness over REST, while we wait for opportunities to adopt something better (*COUGH* WS-* *cough*).
Me: Hey, many thanks to all. Especially Juval and Monty, I couldn't agree more. With my involvement in Mobile over the last two years coming from a purely .NET to .NET background, mobile is definitely screaming for better standards, like some of what is already in WS-*. I really don't see why it wouldn't be possible to write a fully WS-* Json binding. Imho - Mobile-philes are too caught up in what is fashionable rather than looking at robust solutions. Example: a great deal of mobile apps and their services have shocking levels of security and secret-data management. Favouring instead investigating cool new open source frameworks for UX and others. Famous last words: "It uses SSL so it's secure".
Ok, so my choices are keep it simple in the absence of any real standards support in mobile and go with HTTP/REST/JSON with hand crafted support for message security. Reliable sessions and indempotency would be nice, but I'll have to hand craft these on my own. Or stick with WCF - Http+Soap and the future support of the full WS-* and SOAP (and possible JSON maybe) in Mobile.
Anonymos: REST is something very different [to Soap]. "True" REST (as defined by the guy who coined the term, Roy Fielding) requires you to satisfy a number of architectural constraints (similar to SOA tenets) that include things like:
Being resource and representation focused, not remote procedure call focused
Using the URI for more than just how to get to the service, but to also include information about what resource you are accessing and how you want to manipulate that resource
Leveraging HTTP as more than just a transport protocol, including using Verbs, response codes, headers, content negotiation. This becomes important when you notice little things like GET requests can be cached by web infrastructure, POST cannot, and SOAP is all POST.
Being Hypermedia-driven (which involves embedding URIs in representations that guide the caller through the allowable interactions with the service)
In my experience, there is a very wide range of compliance with these constraints in service people claim to be REST services. Most I would say are doing the first three bullets to some degree. Almost none are doing the last. But if you give any credit to the guy who defined REST, he says it ain't REST without the last bullet:
http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
But bottom line, SOAP service calls are not REST in any way shape or form because they do not do any of the above. Now I have in the past been one of the people saying "meh, close enough...", but my opinion on that has changed. We can't expect architects to communicate precisely about SOA concepts and then say it is OK to be loose and sloppy about the use of the term REST.
Anonymous: How come when you transfer money from your bank to another bank there's no distributed transaction? Why don't Expedia wrap a transaction around booking your airline tickets, hotel stay and transfers? Why don't the airlines even use a transaction around just the airline ticketing part? Where's the transaction around payment, inventory etc.. when you make a purchase on Amazon? Is the reason because they don't have WS-*?
Anonymous: I'm not saying never use transactions, just that you should keep the scope small with resources that are close geographically .. distributing them all over is never going to work at scale.
Here's an interesting article : http://blogs.msdn.com/b/clemensv/archive/2011/10/06/achieving-transactional-behavior-with-messaging.aspx
Anonymous: Is there a connectivity technology more in need of WS-* than Mobile?
Michael "Monty" Montgomery: Ever notice how everyone’s JSON packets are looking more and more like SOAP envelopes?
I’m just sayin’…
Juval: Very much agreed on all counts. Inspired me to get on a soapbox (pun intended) and make a few bigger picture points on what the proper motives should be for using REST to connect applications.
There is some context that should be considered when making general (and true) statements such as “the web is REST.” The web is indeed REST, and if one watches a WCF client talk to a WCF service (using some wire sniffer) over one of the SOAP http bindings, you’ll see REST. The RESTful interactions are simply lower down in the stack. We all know this quite well. So taking this statement: “the web is REST and the web is HTTP“ allow me to refactor it in a more architecturally minded manner:
HTTP is REST.
The question is, do we just adopt a SOAP stack to go over HTTP (for the web in our current conversation) and have the option to use any number of its features which are already there and ready to go? Or do we skip SOAP and just form the RESTful interactions ourselves? So we skip supporting SOAP, it’s too difficult…Then what? Are we not just re-implementing the features of the WS-* standards in a non-standard way (as Monty alluded to)?
That’s not an ideal situation to be in… The only reason to use REST, in my opinion, is only after we’ve admitted the situation is indeed not ideal.
We can’t wait for the vendors to adopt a common set of standards tomorrow, so for today, we’ll do it ourselves over REST. Eventually, all the disparate groups who are trying to implement SOAP’s features in their own manner (without admitting to themselves that’s what they’re actually doing) over REST (because it’s so much “simpler”) will have to admit that all their effort must be shot in the head (like so many VB6 diehards have had to do). Our responsibility is to make smart decisions in how invested we really should get into implementing needed fanciness over REST, while we wait for opportunities to adopt something better (*COUGH* WS-* *cough*).
Me: Hey, many thanks to all. Especially Juval and Monty, I couldn't agree more. With my involvement in Mobile over the last two years coming from a purely .NET to .NET background, mobile is definitely screaming for better standards, like some of what is already in WS-*. I really don't see why it wouldn't be possible to write a fully WS-* Json binding. Imho - Mobile-philes are too caught up in what is fashionable rather than looking at robust solutions. Example: a great deal of mobile apps and their services have shocking levels of security and secret-data management. Favouring instead investigating cool new open source frameworks for UX and others. Famous last words: "It uses SSL so it's secure".
Ok, so my choices are keep it simple in the absence of any real standards support in mobile and go with HTTP/REST/JSON with hand crafted support for message security. Reliable sessions and indempotency would be nice, but I'll have to hand craft these on my own. Or stick with WCF - Http+Soap and the future support of the full WS-* and SOAP (and possible JSON maybe) in Mobile.
Anonymos: REST is something very different [to Soap]. "True" REST (as defined by the guy who coined the term, Roy Fielding) requires you to satisfy a number of architectural constraints (similar to SOA tenets) that include things like:
Being resource and representation focused, not remote procedure call focused
Using the URI for more than just how to get to the service, but to also include information about what resource you are accessing and how you want to manipulate that resource
Leveraging HTTP as more than just a transport protocol, including using Verbs, response codes, headers, content negotiation. This becomes important when you notice little things like GET requests can be cached by web infrastructure, POST cannot, and SOAP is all POST.
Being Hypermedia-driven (which involves embedding URIs in representations that guide the caller through the allowable interactions with the service)
In my experience, there is a very wide range of compliance with these constraints in service people claim to be REST services. Most I would say are doing the first three bullets to some degree. Almost none are doing the last. But if you give any credit to the guy who defined REST, he says it ain't REST without the last bullet:
http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
But bottom line, SOAP service calls are not REST in any way shape or form because they do not do any of the above. Now I have in the past been one of the people saying "meh, close enough...", but my opinion on that has changed. We can't expect architects to communicate precisely about SOA concepts and then say it is OK to be loose and sloppy about the use of the term REST.
Anonymous: How come when you transfer money from your bank to another bank there's no distributed transaction? Why don't Expedia wrap a transaction around booking your airline tickets, hotel stay and transfers? Why don't the airlines even use a transaction around just the airline ticketing part? Where's the transaction around payment, inventory etc.. when you make a purchase on Amazon? Is the reason because they don't have WS-*?
Anonymous: I'm not saying never use transactions, just that you should keep the scope small with resources that are close geographically .. distributing them all over is never going to work at scale.
Here's an interesting article : http://blogs.msdn.com/b/clemensv/archive/2011/10/06/achieving-transactional-behavior-with-messaging.aspx
The articles were enlightening though! :) Especially that second one by Roy himself. http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
"There are probably other rules that I am forgetting, but the above are the rules [he has 6 bullets] related to the hypertext constraint that are most often violated within so-called REST APIs. Please try to adhere to them or choose some other buzzword for your API."
Anonymous: Designing service API's in a purely RESTful manner does seem a little functional decomp to me.
Get, Post, Delete, Put == CRUD right?
CRUD services don't sound like they are encapsulating behaviour very well and will result in business logic in many different clients.
Its extremely oriented towards resource access, create me one of these... now update it so that this property has this value... ok now delete it. This doesn't appear to be encapsulating the business use case inside the service. When another client wants to use the same functionality it will need to duplicate this orchestration code.
Unless I'm missing something using full blown REST doesn't seem prudent to me?
Anonymous: Its easiest to think about RESTful design in terms of CRUD operations because of the natural mapping of those verbs onto CRUD. And really a major portion of most business services involves CRUD. But it is not limited to that and you don't have to do functional decomposition with REST services any more than you need to with SOAP RPC services. It is more a different way of thinking about things. Instead of thinking about invoking a manager method, you think about it in terms of manipulating the resource that that manager represents. Calling a service operation that kicks off a workflow is ultimately manipulating a resource or a set of resources on the back end. If a set, then you can define a resource that represents that aggregate so that the service still represents a single resource. In aggregate it is either creating new resources, modifying existing resources, removing those resources, or simply retrieving information about resources. There can be any degree of CRUD, business logic, workflows, activities and so on under the covers of a REST service call.
You do have to get into the intentions of HTTP for each verb (the idempotency of each) to figure out how to map them onto service operations. And in reality, most complex business operations map to a POST anyway because they are not idempotent (they cause side effects and changes to the back end and cannot be repeated with the same outcome). But as I mentioned one of the benefits is the fact that for large scale services with lots of consumers and a significant number of retrieval call chains, those can be done with GET and can be far more efficient and scalable due to caching at the web server, router, or CDN level.
So nothing about REST forces you into functional decomp, but I'll admit that it is even easier to slip into that with REST. And really the big benefit to me of REST over SOAP RPC (WCF specifically) for some scenarios is the simplicity for things that don't need to be complex and the ease with which these services can be consumed from dumber clients (Win 8 WinRT, iOS, Android specifically).
Like anything else, REST is not a silver bullet. You have to pick the right technology based on your requirements. And big enterprise services being consumed by .NET clients or other services it is still usually better to use WCF. It is only for those internet facing services where others are going to write the consuming applications and you want to make it as easy as possible for them to be able to consume your functionality or for targeting device consumption that REST becomes attractive to me, and even then I have yet to embrace the hypermedia constraint fully. The only time I really do that is if I am using OData because it handles a lot of that automatically for you. But then you are dealing with pure CRUD, so I use that with caution as well.
Anonymous: I read Roy Fielding's thesis some time ago and my understanding is that REST is an architecture style specifically for the Web (WWW). Not an architectural style for distributed applications such as SOA. The main goal (requirements) of the Web as summarised by Berners-Lee "...was to be a shared information space through which people and machines could communicate" (see section 4.1 p.66 for more details). As Roy states on page 82 "The REST interface is designed to be efficient for large grain hypermedia data transfer, optimizing for the common case of the Web, but resulting in an interface that is not optimal for other forms of architectural interaction."
Therefore REST is best used for information sharing and not for service interactions (albeit its CRUD nature where REST targets especially the Read part). SOAP is best suited for service interactions. Unfortunately as the WS-* standards are not as ubiquitously implemented and easy as HTTP, developers resorted to HTTP for Service interactions and then mistakenly call their APIs as RESTful. It reminds me of a quote I read somewhere that went something like “Just because you code in C++ it doesn’t mean you are writing an object-oriented application”.
My conclusion is that if we are building a SOA and use HTTP and various Web APIs in place of WCF and SOAP (because of technology limitations etc) we mustn't try to adhere to REST but to SOA. However if that part of the application is for information sharing over the internet then we should adhere to REST.
Balance dogma with pragmatism
The true verification of good service encapsulation is 3 or more clients exposing the same functionality to different devices.
As it turns out the question concerning whether to use WCF or the new Web API depends on the context on how it will be used. If you are looking for a quick development time that doesn't require strict security (although security can be implemented) and ease of defining the data output (JSON, XML, etc.) then the Web API is the route that makes the most sense. A hybrid solution could be created where the API exists in the same middle layer as the WCF.
"There are probably other rules that I am forgetting, but the above are the rules [he has 6 bullets] related to the hypertext constraint that are most often violated within so-called REST APIs. Please try to adhere to them or choose some other buzzword for your API."
Anonymous: Designing service API's in a purely RESTful manner does seem a little functional decomp to me.
Get, Post, Delete, Put == CRUD right?
CRUD services don't sound like they are encapsulating behaviour very well and will result in business logic in many different clients.
Its extremely oriented towards resource access, create me one of these... now update it so that this property has this value... ok now delete it. This doesn't appear to be encapsulating the business use case inside the service. When another client wants to use the same functionality it will need to duplicate this orchestration code.
Unless I'm missing something using full blown REST doesn't seem prudent to me?
Anonymous: Its easiest to think about RESTful design in terms of CRUD operations because of the natural mapping of those verbs onto CRUD. And really a major portion of most business services involves CRUD. But it is not limited to that and you don't have to do functional decomposition with REST services any more than you need to with SOAP RPC services. It is more a different way of thinking about things. Instead of thinking about invoking a manager method, you think about it in terms of manipulating the resource that that manager represents. Calling a service operation that kicks off a workflow is ultimately manipulating a resource or a set of resources on the back end. If a set, then you can define a resource that represents that aggregate so that the service still represents a single resource. In aggregate it is either creating new resources, modifying existing resources, removing those resources, or simply retrieving information about resources. There can be any degree of CRUD, business logic, workflows, activities and so on under the covers of a REST service call.
You do have to get into the intentions of HTTP for each verb (the idempotency of each) to figure out how to map them onto service operations. And in reality, most complex business operations map to a POST anyway because they are not idempotent (they cause side effects and changes to the back end and cannot be repeated with the same outcome). But as I mentioned one of the benefits is the fact that for large scale services with lots of consumers and a significant number of retrieval call chains, those can be done with GET and can be far more efficient and scalable due to caching at the web server, router, or CDN level.
So nothing about REST forces you into functional decomp, but I'll admit that it is even easier to slip into that with REST. And really the big benefit to me of REST over SOAP RPC (WCF specifically) for some scenarios is the simplicity for things that don't need to be complex and the ease with which these services can be consumed from dumber clients (Win 8 WinRT, iOS, Android specifically).
Like anything else, REST is not a silver bullet. You have to pick the right technology based on your requirements. And big enterprise services being consumed by .NET clients or other services it is still usually better to use WCF. It is only for those internet facing services where others are going to write the consuming applications and you want to make it as easy as possible for them to be able to consume your functionality or for targeting device consumption that REST becomes attractive to me, and even then I have yet to embrace the hypermedia constraint fully. The only time I really do that is if I am using OData because it handles a lot of that automatically for you. But then you are dealing with pure CRUD, so I use that with caution as well.
Anonymous: I read Roy Fielding's thesis some time ago and my understanding is that REST is an architecture style specifically for the Web (WWW). Not an architectural style for distributed applications such as SOA. The main goal (requirements) of the Web as summarised by Berners-Lee "...was to be a shared information space through which people and machines could communicate" (see section 4.1 p.66 for more details). As Roy states on page 82 "The REST interface is designed to be efficient for large grain hypermedia data transfer, optimizing for the common case of the Web, but resulting in an interface that is not optimal for other forms of architectural interaction."
Therefore REST is best used for information sharing and not for service interactions (albeit its CRUD nature where REST targets especially the Read part). SOAP is best suited for service interactions. Unfortunately as the WS-* standards are not as ubiquitously implemented and easy as HTTP, developers resorted to HTTP for Service interactions and then mistakenly call their APIs as RESTful. It reminds me of a quote I read somewhere that went something like “Just because you code in C++ it doesn’t mean you are writing an object-oriented application”.
My conclusion is that if we are building a SOA and use HTTP and various Web APIs in place of WCF and SOAP (because of technology limitations etc) we mustn't try to adhere to REST but to SOA. However if that part of the application is for information sharing over the internet then we should adhere to REST.
Balance dogma with pragmatism
The true verification of good service encapsulation is 3 or more clients exposing the same functionality to different devices.
As it turns out the question concerning whether to use WCF or the new Web API depends on the context on how it will be used. If you are looking for a quick development time that doesn't require strict security (although security can be implemented) and ease of defining the data output (JSON, XML, etc.) then the Web API is the route that makes the most sense. A hybrid solution could be created where the API exists in the same middle layer as the WCF.
Saturday, August 4, 2012
The role of an Architect in Agile Part 1
I had an interesting conversation with a highly experienced architect recently:
"Is what you're doing working? Really? Are you sure?"
If you answered yes, then don't change anything.
More than likely however, there are recurring quality problems, defect count is high, bug-fixes are rejected, and developers spend a great deal of time refactoring. You need an architect and better architecture processes.
I pondered about the title of this post, should it be the role of an architect in Agile or just modern architect? Probably the latter, but the question of architecture in agile is something that bugs more than a few people. I certainly don't have all the answers nor proclaim to be an expert, but these are my experiences and observations.
Stephen Cohen (Chief Architect, Microsoft): "Scrum went through 3-4 years in its original form before admitting that the architect had any role at all to play."
Juval Lowy (Master Architect, IDesign): "The agile priests would like you to believe that following agile will magically produce architecture that is adaptable, resistant to poor coding, and is scalable."
(Apologies for the paraphrasing).
Some practioners claim there is no need for design and it should be simply part of the implementing the story. This works in only the most trivial software, but in my experience most of us veterans would not often descibe what we do as simple. More often than not the code produced is badly structured in retrospect and not properly limiting volatility when business change occurs. Agile addresses this by saying "continuously refactor". How much time could be saved by mapping out an over arching architecture up front?
A wise friend once said:
If you put a bunch of extreme programmers in the middle of a city, let’s say Marrakech, and ask them to visit five tourist hot spots without using a “map”, they will wander around for days exploring every little passage way by brute force. If you give another bunch of developers a “map” and put them in the same city, they will use the map to go directly to each of the five tourist hot spots in a matter of minutes or hours.
Architecture is the map. You'd be crazy not to have a map (or get started creating one).
Simply sitting down with a one liner on a piece of card does not mean any developer from graduate to senior will be able to first estimate, then write tests, then magically produce well designed code that properly encapsulates volatility. This is a naive and utopian ideal that never happens. If the architecture is already mapped out and clearly articulated then it absolutely is.
Once you have a map, in enough detail to be understood by your team, Scrum works okay. It should be done before the team is assembled to begin building.
You should strive to create a shadow board architecture that allows making design decision for new business features easy, consistent and fast.
No one would argue that business needs are changing faster than ever. This necessitates good design that encapsulates volatility as best it can. The role of an Architect is most definitely STILL REQUIRED and because of faster pace of change, more relevant than ever.
Architecture should not be a heavy process, it should be mapped out once at the beginning for the whole new system, and sometimes before each major feature addition to an existing product. According to IDesign's "The Method" architecture can be completed in 3 to 5 weeks. It is not something that happens for each user-story.
Some highly respected architects I have spoken to simply state a proper architecture process does not fit with Agile, period. I won't join this debate, however, agile isn't going to step aside any time soon. To date, I haven't ever won the debate on Agile versus "The Method". It will take time. For now, I believe we should work with agile.
Scott Ambler posted his take on fitting architecture into agile processes. The idea is to envision the architecture during the requirements and analysis phase. (Before the development sprints begin). The key is just-enough, then during development keep reiterating over it during the implementation phase. Just-enough for your target team to comprehend. Amend and refine when necessary.
Architecture is not something that is needed for every story. It is the end-goal vision of your system. This may happen once per release or even less frequently. You must have a clear idea of where you want your system to be 3 or 4 releases from now. Its ok to change the vision. The vision doesn't have to consider business features in detail, its the more like an overarching strategy for implementing them, but you should consider likely business extensions and how your architecture will support the changes.
Are you building a pluggable SOA system? If so, are there standard patterns all communications should use? Is it worthwhile applying aspects to all components? What would happen if services organically grew where developers thought it was quickest and easiest to slap them in? What about realistic and likely business changes? Be cynical, its healthy. These are the questions important to consider when designing an architecture.
It takes effort to make something appear easy, and its easy to make something complex. - Unknown.
"Is what you're doing working? Really? Are you sure?"
If you answered yes, then don't change anything.
More than likely however, there are recurring quality problems, defect count is high, bug-fixes are rejected, and developers spend a great deal of time refactoring. You need an architect and better architecture processes.
I pondered about the title of this post, should it be the role of an architect in Agile or just modern architect? Probably the latter, but the question of architecture in agile is something that bugs more than a few people. I certainly don't have all the answers nor proclaim to be an expert, but these are my experiences and observations.
So what's the problem?
Scrum and agile narrations focus on writing code, building code, not the "ideation" and design that is necessary. Who isn't following Scrum or some self proclaimed "flavour of agile" these days right? Almost never does agile documents mention architecture or design. They expect the team to design and build as part of a sprint, or maybe spend the first sprint just on design but no help on how this should work. In my experience this is fraught with difficulty. Design by committee doesn't work well, its usually better to have the most experienced person responsible for design and to properly think through a design and proof of concepts takes time. They should, of course, engage other team members for input and review, but one person needs to be responsible. Things go better if the senior(s) have a vision of what the end-goal architecture should look like, building this vision with clarity is the issue.Stephen Cohen (Chief Architect, Microsoft): "Scrum went through 3-4 years in its original form before admitting that the architect had any role at all to play."
Juval Lowy (Master Architect, IDesign): "The agile priests would like you to believe that following agile will magically produce architecture that is adaptable, resistant to poor coding, and is scalable."
(Apologies for the paraphrasing).
Some practioners claim there is no need for design and it should be simply part of the implementing the story. This works in only the most trivial software, but in my experience most of us veterans would not often descibe what we do as simple. More often than not the code produced is badly structured in retrospect and not properly limiting volatility when business change occurs. Agile addresses this by saying "continuously refactor". How much time could be saved by mapping out an over arching architecture up front?
Wait, aren't you saying that you'd rather do waterfall?
No. Architecture != Waterfall.A wise friend once said:
If you put a bunch of extreme programmers in the middle of a city, let’s say Marrakech, and ask them to visit five tourist hot spots without using a “map”, they will wander around for days exploring every little passage way by brute force. If you give another bunch of developers a “map” and put them in the same city, they will use the map to go directly to each of the five tourist hot spots in a matter of minutes or hours.
Architecture is the map. You'd be crazy not to have a map (or get started creating one).
Simply sitting down with a one liner on a piece of card does not mean any developer from graduate to senior will be able to first estimate, then write tests, then magically produce well designed code that properly encapsulates volatility. This is a naive and utopian ideal that never happens. If the architecture is already mapped out and clearly articulated then it absolutely is.
Once you have a map, in enough detail to be understood by your team, Scrum works okay. It should be done before the team is assembled to begin building.
You should strive to create a shadow board architecture that allows making design decision for new business features easy, consistent and fast.
Ok, so architecture is important, but do we need an Architect?
What skills does agile require of its team members?- Cross-skilled (Dev, QA, SOA, UI, UX, DBA, Design, and anything else required);
- Able to work on and explain any part of the system;
- TDD & Unit testing;
- QA practices;
- Proven architecture and design skills;
- Industry context knowledge and patterns;
- Ability to write SDK documentation;
- Ability to consistently review code;
- Mentoring skills;
- Able to create processes and procedures to ensure predictable outomes (ie adapt);
- Good communication skills;
- Able to effectively peer program;
No one would argue that business needs are changing faster than ever. This necessitates good design that encapsulates volatility as best it can. The role of an Architect is most definitely STILL REQUIRED and because of faster pace of change, more relevant than ever.
How should architecture fit into an agile process?
It is common to begin (and sometimes complete) the UX work before the Scrum team starts work. Architecture is no different, it should be running in parallel to UX.Architecture should not be a heavy process, it should be mapped out once at the beginning for the whole new system, and sometimes before each major feature addition to an existing product. According to IDesign's "The Method" architecture can be completed in 3 to 5 weeks. It is not something that happens for each user-story.
Some highly respected architects I have spoken to simply state a proper architecture process does not fit with Agile, period. I won't join this debate, however, agile isn't going to step aside any time soon. To date, I haven't ever won the debate on Agile versus "The Method". It will take time. For now, I believe we should work with agile.
Scott Ambler posted his take on fitting architecture into agile processes. The idea is to envision the architecture during the requirements and analysis phase. (Before the development sprints begin). The key is just-enough, then during development keep reiterating over it during the implementation phase. Just-enough for your target team to comprehend. Amend and refine when necessary.
Architecture is not something that is needed for every story. It is the end-goal vision of your system. This may happen once per release or even less frequently. You must have a clear idea of where you want your system to be 3 or 4 releases from now. Its ok to change the vision. The vision doesn't have to consider business features in detail, its the more like an overarching strategy for implementing them, but you should consider likely business extensions and how your architecture will support the changes.
Are you building a pluggable SOA system? If so, are there standard patterns all communications should use? Is it worthwhile applying aspects to all components? What would happen if services organically grew where developers thought it was quickest and easiest to slap them in? What about realistic and likely business changes? Be cynical, its healthy. These are the questions important to consider when designing an architecture.
It takes effort to make something appear easy, and its easy to make something complex. - Unknown.
The role of an Architect in Agile Part 2
What should an architect be responsible for?
An architect or team of architects should be responsible for:
- Assisting in gathering and analysing requirements.
- Sell and negotiate technical constraints and aspects to stakeholders.
- Articulate the end goal architectural vision designed to encapsulate change.
- Communicate with diagrams the intended architecture and implementation plan.
- Prove (or disprove) technologies and techniques (prototyping).
- Own the definition of integration points and service interfaces.
- Establish processes to ensure predictable outcomes and quality in the SDLC.
- Oversee delivery of software in accordance with the intended architecture (reviews).
- Acceptable quality levels (coding standards, code metrics and performance parameters).
Controversially IDesign's "The Method" argues that the architect should also project manage. The explanation was simple, who else knows best how things integrate and their dependencies?
During implementation the job isn't done. Architecture is also governance of ensuring all these things have happened. An architect should oversee a feature from inception to implementation to final load and concurrency testing. A Scrum team should only be working on one feature at a time (this is a founding principle of agile - focus and don't context switch). This allows an architect to potentially oversee two features across two teams at the same time. Its also worth considering the architect joining the team as a development resource, if resources permit.
Any respectable Scrum coach will tell you that Scrum does not mandate no design. It simply states do what is required. If that means design can be done inside the same sprint as implementation, do it, although that is highly unlikely and I wouldn't advise it. The best plan is to complete the necessary design work prior to initiating a stream of work with the Scrum team. The trick is don't do too much. Too much will vary depending on strength of experience in the team, and how effective they are at communicating. The more inexperienced the team the more design work needed.
Having architecture artifacts up front before starting a piece of work, and for grooming sessions, means it is highly visible and scrutised. This inevitably improves the design, and reduces unknowns.
Who's the boss?
Answer: The client. The Product Owner (PO) represents and speaks for the client. That is not to say the architect shouldn't have direct contact with the client, the PO is there to help you with this. Embracing change is the key to a successful software product. Often more valuable information comes to light during the project. Expect change. However, an inexperienced PO will change their mind more often than necessary. This can be addressed with a little more architecture work, this will keep the PO's reasoning honest. Any whiff of analysis paralysis means you either dump the feature/use-case or dumb it down and start the Scrum process with basic a implementation only. Unfortunately a PO that changes their mind too often results in poor qualilty software that costs more than it should to maintain or extend.A good PO will work with their architect during the sprint review to ensure the quality metrics are adaquate and all agreed standards have been met. Although the architect is technical, they are definitely in the Product Owner camp. The PO should lean on their architect for technical advice and verification of the product delivered. The PO should fail the sprint if the architect can prove that the implementation doesn't comply with the design, or standards have not been met.
IDesign's "The Method" advocates the architect head up the project team and be the one to call the shots and sign-off the final product. This is a very tough sell to the establishment today, but it could be the future.
Some tips on how an architect can go about their job?
(Thanks to IDesign's Michael Montgomery for the inspiration for this list).
- Formalise and document your requirements gathering approach.
- Formalise document templates to store the gathered information.
- Don't gather information using sticky notes! Save it digitally using defined consistent templates.
- No gold-plating. Everything is “just enough” based on known and likely use-cases, team composition, and hand-off-point.
- Iterate on everything, including requirements, getting “just enough” before the Design Phase begins. Gather - Refine - Review/Assess, Gather - Refine - Review/Assess, until your ready to start implementation.
- Continually reinforce to the BA's and PO's that they bring the ‘when, what, why, who’, but never the how.
- Developers don’t read. Format use cases as ‘pseudo-code’ (i.e. numbered/bulleted ‘lists’) and diagrams.
- Never use the term ‘spec’. Call it the ‘requirements lists’ or 'acceptance criteria'.
- With the advent of UX, separate functional (UX/UI) from back-end service operational (SOA) requirements lists. Treat back end services as a different product to a client UI. They will have their own lifetimes driven by the same use cases.
- UX is different from UI is different from SOA is different from Framework. Each may need its own requirements treatment, depending on the size and complexity of the system.
- Developers don’t do UX (although they may think they can - don't believe them).
- The UX guys must involve stakeholders early and often. This will need to be quite mature before implementation can begin.
- For user driven applications (which is most), it’s well-crafted UX workflows that produce clearly defined use cases that produce succinct SOA. If you can get the UX guys ‘out in front of the ball’, you put yourself in a sweet spot. This particularly holds true in composite UX.
Conclusion
Given a choice, I would prefer to use IDesign's "The Method". My problem with it has been convincing employers and stakeholders, even though it's a new approach, it's a proven one. Others with superhuman sales and negotiation skills may have more success than I. Working in with Scrum, its roles, and other established roles has been a given for me; and I'm betting you too. However, I believe this is completely realistic in a hybrid model. The role of an architect is absolutely essential, but the role and its skills are far more than a senior developer's. An architect's number one skill, is their communication skills. In my opinion as an industry we need to formalise roles, and the architect role is paramount and akin to the building industry's architect in responsibility but more similar to a head-chef in activities.References and more information
Scott Ambler on User StoriesScott Ambler on Architecture in Agile
IDesign.Net
http://www.idesign.net/articles/agile_and_the_architect.htm
Able Architecture Part 1:
Able Architecture Part 2:
Tuesday, May 29, 2012
Calls between layers should be WCF service calls
Everyone knows about creating layers in software architecture. During the IDesign Architecture Clinic I attended in March Juval talked about separation of layers and stopping leaking of concerns between layers.
The best way to achieve this is by using WCF between layers. When this is applied correctly WCF will provide:
Consistency between layers (transactions);
Scalability (general code performance, farming, or partitioning layers onto different app-servers etc);
Fault isolation (resilience from unhandled exceptions and isolate each thread from another's exceptions);
Security (message encryption, transport encryption, authentication, impersonation etc);
Clean Separation of presentation from business logic (maintainability and flexibility);
Availability (striving for 24/7 99.99% up time);
Responsiveness;
Throughput;
and Synchronisation.
Juval Lowy: "Tell me which one of these you don't like and I'll tell you in which way your system is going to die." (reference)
Juval is also known for saying "...every class as a service...". This might be a little controversial and extreme, but imagine for a moment that the cost of serialisation and deserialisation was eliminated for internal App-Domain calls. Why would you not want all the above aspects WCF brings to the table not only between layers but also between classes? An insight into the future perhaps?
Sunday, April 1, 2012
Juval Löwy's (IDesign.net) Architecture Clinic
Recently I had the privilege of attending the IDesign Architecture Clinic in Sydney. It was a week long intensive mix of the architecture theory, the responsibilities of an architect, and practical coaching. IDesign runs two similar courses in the software architecture stream: The Architect's Master-Class and the Architecture Clinic. The idea behind the Clinic is to keep the attendee numbers relatively small and have the opportunity to learn from a true master architect. In Sydney my group was lucky to have Juval himself run the course.
Day one was full on. Juval gave us a light speed overview of some of the critical material from the Architect's Master-class. The rest of the week was then practical hands-on experience using IDesign's "The Method". Days two, three and four followed the pattern of choose one of the attendee's systems and use The Method to draw up a proposed new architecture. Then Juval and the group will critique. Day five was all about project design, planning, and tracking.
The amount of knowledge and experience imparted during this course was simply incredible. No other speaker I have ever listened to has packed this amount of information into five days. There isn't more than a few handfuls of other people in the software industry that have the same level of industry insight and where it is heading in the next 10-20 years. During conversations there were more than a few statements that gave us a glimpse of where the industry is heading. As a bonus Juval also kindly donated two of his evenings and run us through some other topics in depth, from the Architect's Master-class.
Anyone who has listened to Juval speak will know he is both highly intellectual and controversial. He often makes statements that send murmurs through an audience. This course was no exception. Some examples (not direct quotes):
Day one was full on. Juval gave us a light speed overview of some of the critical material from the Architect's Master-class. The rest of the week was then practical hands-on experience using IDesign's "The Method". Days two, three and four followed the pattern of choose one of the attendee's systems and use The Method to draw up a proposed new architecture. Then Juval and the group will critique. Day five was all about project design, planning, and tracking.
The amount of knowledge and experience imparted during this course was simply incredible. No other speaker I have ever listened to has packed this amount of information into five days. There isn't more than a few handfuls of other people in the software industry that have the same level of industry insight and where it is heading in the next 10-20 years. During conversations there were more than a few statements that gave us a glimpse of where the industry is heading. As a bonus Juval also kindly donated two of his evenings and run us through some other topics in depth, from the Architect's Master-class.
Anyone who has listened to Juval speak will know he is both highly intellectual and controversial. He often makes statements that send murmurs through an audience. This course was no exception. Some examples (not direct quotes):
- Every class as a WCF service.
- The Method doesn't fit well with Agile.
- The Architect is the responsible party not the project manager.
- Most project managers actively kill their projects.
- Self managing teams don't work.
For most questions the general answer was "there is no time, you need to attend the Architect's Master-class". Other vague answers sounded as though he was hinting at some future technology now under development but is bound by NDA's. There were some answers, but with anything new and revolutionary every question reveals 3 more.
My personal opinion on this course was that it was worth while (it is definitely not a cheap course), but I would have liked to have completed the Architect's Master-class first. I also believe that there was too much after-the-fact review based criticism and not enough coaching. For days two, three, and four we were left to decompose the system and draw up our own plan for the whole day, whereas I would have preferred to have been walked through it from start to completion on day two before the practical on days three and four. Again just my personal opinion, and like I said still very worth while.
So, did I rush back to the office and evangelise all my co-workers and stakeholders?
No.
Why?
Because I cannot sell a new idea to my stakeholders without having prepared answers to the questions I know they will ask. Questions like: "We are a Scrum-Software-House, this doesn't fit with Scrum. You are not permitted to undertake projects in this way". Or "We hired you as an Architect not a project manager, you are operating outside your responsibilities". Or "Some of what you are prescribing infringes on senior developers responsibilities of technical design, architects should not be prescribing technical design". And my favourite: "We don't have enough architects to operate this way and no-one wants to be an architect". The answers to these questions are not the kind you can Google and Juval simply recommends go to the Architect's Master-class. So I'm filing this learning under "Interesting future insights that can't be used right now".
Can I justify going to the next Architect's Master-class?
Probably not for a while. Due to corporate politics its unlikely I can sell another course in the next 12 months just for me given the overall cost of fees, flights and accommodation (planned courses are only in the States). Also given that this one has not set my architecture team on fire with new levels of efficiency.
There's no easy solutions, just broad wholesale change.
Saturday, July 23, 2011
A good article on domain driven design (DDD)
I read a good article on DDD the other day, and its worth keeping a reference to it.
http://msdn.microsoft.com/es-es/architecture/en
http://msdn.microsoft.com/es-es/architecture/en
Tuesday, June 7, 2011
DTOs vs Unified Object Models
Great blog post on Info Q discussing differing opinions on the Data Transfer Object pattern and unified object models (used for ORM, business logic, as wire transfer objects, and as Models in Model-View-Controller pattern).
http://www.infoq.com/news/2011/06/DTOs-vs-Objects
In regard to unified models, I have always felt this was forcing to many concerns on a class. In my opinion at least it breaks the single responsibility principle.
http://www.infoq.com/news/2011/06/DTOs-vs-Objects
In regard to unified models, I have always felt this was forcing to many concerns on a class. In my opinion at least it breaks the single responsibility principle.
Saturday, April 16, 2011
MEF versus IoC
The Microsoft Extensibility Framework (MEF) has been around for a couple of years now and is used to allow extension to Windows applications without changing the original code (Recomposition). It can be used with either Silverlight, WPF, and WinForms. MEF is not a replacement to PRISM. The PRISM toolkit to help you implement the MVVM pattern and is similar to MVVM Light or any of the other MVVM Toolkits. PRISM is also only intended for WPF and Silverlight. So what is MEF? The MEF Website describes it as
"Application requirements change frequently and software is constantly evolving. As a result, such applications often become monolithic making it difficult to add new functionality. The Managed Extensibility Framework (MEF) is a new library in .NET Framework 4 and Silverlight 4 that addresses this problem by simplifying the design of extensible applications and components."
I've been a long standing fan of IoC, and after reading many articles about MEF and writing a few test applications, there seems to be a cross over with IoC. IoC is far more generic and is a more broad pattern to use across your entire system, whereas MEF is specifically targeting extending and recomposition of the UI. So where's the cross over? MEF is a kind of object factory (aka container), it makes replacement of UI Views and Controllers (aka View-Models) easy. By decorating a public property with an attribute the MEF framework with find all subclasses of (or a chosen one) and instantiate it and inject it into the property. Very much like an IoC container. But MEF does have some cool features for searching application folders for runtime add-ins etc. But so does StructureMap.
"Application requirements change frequently and software is constantly evolving. As a result, such applications often become monolithic making it difficult to add new functionality. The Managed Extensibility Framework (MEF) is a new library in .NET Framework 4 and Silverlight 4 that addresses this problem by simplifying the design of extensible applications and components."
I've been a long standing fan of IoC, and after reading many articles about MEF and writing a few test applications, there seems to be a cross over with IoC. IoC is far more generic and is a more broad pattern to use across your entire system, whereas MEF is specifically targeting extending and recomposition of the UI. So where's the cross over? MEF is a kind of object factory (aka container), it makes replacement of UI Views and Controllers (aka View-Models) easy. By decorating a public property with an attribute the MEF framework with find all subclasses of (or a chosen one) and instantiate it and inject it into the property. Very much like an IoC container. But MEF does have some cool features for searching application folders for runtime add-ins etc. But so does StructureMap.
Here's an excellent comparison from MSDN:
My take on it, is MEF will suit people new to IoC better, and might be more approachable. The documentation for MEF is pretty good, better than StructureMap's documentation for sure. Using a mature feature rich IoC container like StructureMap however you get most of the functionality of MEF, but it requires a little more work to use. But it gives you the advantage of a unified approach and good consistency.
Tuesday, October 26, 2010
Software Design Principles
Principles Of Software Design
8-September-2010
|
Monday, July 5, 2010
Service Oriented Architecture Resources
This page is a organic list of links and information on SOA and SaaS.
Some service oriented architecture links:
"Design Patterns for .Net"'s Pattern for flexible WCF Services
Subscribe to:
Posts (Atom)