Home > Archive > Extreme Programming > August 2006 > Using Agile for Development of a Platform
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Using Agile for Development of a Platform
|
|
|
| Hi,
I'm fairly new to agile methodology. Most of the examples I've seen for
successful projects come from the IT domain - 3-tier projects, some
GUI, some business logic, database layer. In this domain I can easily
see how this works. However, since this is not what I do, then I'm not
sure these success story apply to my problem. What I do can be compared
with developing an application server, or even better, to having an
application server written in C and port it to Java. More accurately, I
have a platform written in J2SE, which is analogous to application
server, and now I want to port it to use J2EE. Basically what I would
do (without agile) is to analyze which are the problems that I was
solving with J2SE tools and frameworks I had to add myself since they
don't exit in pure Java, and find the right J2EE mechanisms to solve
the same problems, or satisfy the same needs. This means:
1. Analysis
2. Architecture
3. Design
4. Implementation
So actually I have a few questions:
1. Do you think this kind of software project is suitable for agile
methodology (any of the existing ones)?
2. Any success stories can be shared?
3. If it is suitable, then how is it done, in a non-waterfallish way? I
know that agile does not mean no design or no architecture, but I'm not
sure as to how it should be really done.
4. Who should be playing the customer under this circumstances?
architect? platform product manager?
Can't wait to your comments!
Thanks.
| |
| Laurent Bossavit 2006-08-28, 6:58 pm |
| Zohar,
> I know that agile does not mean no design or no architecture, but I'm not
> sure as to how it should be really done.
Start with one question:
* What constitutes "success" in your domain ?
This branches into a number of sub-questions:
- Who gets to decide whether your project is a success or not, and
(among other decisions) to keep funding it or not ? That gives you a
first idea of who plays the Customer role.
- How stable is that definition of success, over time ? The more it
changes, the shorter your iterations and increments need to be.
- How stable are the *details* of the definition, over time ? The more
likely that some details will change, the more you need constant and
frequent interaction with your customer(s).
- Which details are closest to the core of what success means, and which
are mere conveniences ("bells & whistles") ? - that will help you
prioritize features.
- What might *prevent* success ? Having a short list of your main
project-specific risks will help determine if your choice of process was
adequate.
To expand a bit on that last, note that people risks usually dominate,
followed by business risks, with technical risks a distant third. If
your number one risk is that you plan to use relatively unskilled but
cheap labor, be aware e.g. that Extreme Programming relies extensively
on developers' "craftsmanship" and deep knowledge of technical trade-
offs. If you see "bugs" and other kinds of software defects as a risk
(if your clients care about things like stability, for instance), take
particular notice of what "agile" processes have to say about testing,
test automation or the role of QA.
The above four questions are the main ones that come to mind, but you'll
want to brainstorm a longer list, always starting with the question at
the top. Anything in "agile" or other processes that you can't clearly
relate to that question is baggage, to be dispensed with (at least
initially).
Laurent
| |
|
| Laurent,
You're obviously right, but I think you ignored the essence of my
question, being, do agile methods suite this kind of project at all.
Where "this kind of project" means the development of a framework,
rather than an application. It is an interesting question even as it
is, if agile suits development of platform/framework from scratch, but
my case is even more complicated, as the platform is already
implemented in J2SE, and the goal of this project is only to port it to
J2EE, and here it makes me wonder if and how it can be done in agile.
There are so many questions, that it's even hard to begin:
1. How do you do the planning for such project?
2. Who writes the user stories? is it developers/architect/product
manager?
3. Assume I collected all my user stories, now what? do I try to create
an architecture, or let the architecture emerge from recurring
implementations of the user stories? (hope not!)
4. What about design? I know it's not excluded, but how is it done?
Would appreciate any more help!
Thanks,
Zohar
| |
|
| zets wrote:
> You're obviously right, but I think you ignored the essence of my
> question, being, do agile methods suite this kind of project at all.
What problem do you imagine?
One myth of Agile development is it targets one-shot applications, not
products or frameworks. The founders of the Agile Alliance were specifically
focussing on the problems of scaling, productizing, and creating frameworks.
> Where "this kind of project" means the development of a framework,
> rather than an application.
Don't write code without a target user population. Don't write a framework
without a target of at least three product programs to share the framework.
If you put these rules together, then a framework should emerge as the best
design for code shared by these three products.
> It is an interesting question even as it
> is, if agile suits development of platform/framework from scratch,
By "from scratch", do you mean development without end-user requirements? No
methodology works in that situation; Agile methodologies simply admit (and
reveal) the problem.
> but
> my case is even more complicated, as the platform is already
> implemented in J2SE,
And you have end-users, and they have requirements that they expect in the
program? Then you are set to go!
> and the goal of this project is only to port it to
> J2EE, and here it makes me wonder if and how it can be done in agile.
Does the original code have unit tests?
What's the smallest amount of J2EE code you could write that would replace a
small amount of J2SE code? Can you write that code, and put it online?
How would it improve the users' experience? If it won't, then why are you
rewriting something that works?
> There are so many questions, that it's even hard to begin:
> 1. How do you do the planning for such project?
By taking the end-user's most important feature request, and satisfying it
with J2EE. Deliver, and repeat until the J2EE framework can take over the
J2SE code.
> 2. Who writes the user stories? is it developers/architect/product
> manager?
Who knows these end-users best? Who has their interests at heart? Who will
profit if they profit?
> 3. Assume I collected all my user stories, now what?
Never try to collect all user stories. That leads to "Big Requirements Up
Front", which is a very bad way to start a project.
> do I try to create
> an architecture, or let the architecture emerge from recurring
> implementations of the user stories? (hope not!)
Why should you hope the architecture should not emerge?
Many programmers have no experience with TDD and refactoring, so the only
"emergent" design they ever experience is code-and-fix, followed by
emergency debugging, followed by generally sick and feeble rework of
designs. TDD turns that situation from a problem into the best way to
design. Without debugging, designs are more plastic.
For example, suppose you have an idea for your framework's model. Regardless
whether you write down or target that framework, if you implement high-value
features using TDD, you will refactor to remove duplication. This will grow
a real architecture. Now it might have some elements in common with your
planned design - and it might have fewer details. The gain here is the
ability to reliably _remove_ design elements, knowing the resulting
framework still functions. Each removal improves the design. So TDD can be
seen as a way to validate and improve the quality of a design plan.
> 4. What about design? I know it's not excluded, but how is it done?
Okay, my guess was correct. You are asking, "I want to use Agile, but
emergent design won't work for frameworks, so how do I avoid emergent
design?"
The answer is that Agile is _for_ frameworks. The root principle of design
is removing duplicated structures. This principle applies to designs both on
paper and in code. It's why the book /Design Patterns/ says to "abstract the
thing that varies". That is the reverse way to say, "merge the thing that
duplicates".
If you have many different user populations, you need different products.
All those products will have code in common, so you merge that into common,
shared modules. You will get frameworks.
--
Phlip
[url]http://c2.com/cgi/wiki?Z Land[/url] <-- NOT a blog!!!
| |
|
| Thanks you!
We develop a platform, maybe a framework was a bad term. Again, take
the analogy of development of a compiler, or a application server, or
JVM. The target audience is application developer, that's the end
users.
I don't think the myth you're describing is really THE myth. It's
trivial that agile will also work for evolving system, when
requirements keep coming, but nevertheless, this is really not the
case. Think of a compiler, you know all the requirement, or enough
requirements to start with on day one of the project, because it's
either standard, or because you have compilers in the market already.
You promote writing a platform only when you have enough users for
that. I completely agree, but trust me, end users for the platform
exist, they're too many...
So I follow your rules but I still don't get my questions answered...
1. I have target user population
2. I have way more than 3 product programs using the platform
There's one thing I'm missing here, and I think there's one thing you
are...
The thing you're missing is that from user's perspective, the platform
doesn't change. The change is implementation detail. This is the
equivalent of Sun changing the implementation of their JVM. The API for
clients doesn't get affected, but the internal implementation does. Now
assume Sun has already a JVM, but from many reasons they want to change
technology. You may argue that if it's working why change, but the
reasons for change of technology may result not only from technical
considerations, but also from market reasons (partnerships, etc.).
Now it's the part I'm missing. On this analogy, how do I make the move
from a JVM written in one technology to a JVM written in another
technology, using agile methods?
The way I would do it with waterfallish techniques is to analyse what I
have in the original JVM, identify the techniques to do the same with
the new technology, prepare the architecture of the JVM using the new
technology, work on the design, write the tests and code, and get it
done.
One problem is also that the original code DOES NOT have unit tests, at
least most of it.
I completely resent the thought that TDD is a panacea. TDD can result
in a higher quality code faster. I don't see how it can replace a good
design or architecture or analysis. Try to think of any known
architecure, SOA, J2EE, COM, .NET. From your text, it could emerge from
TDD+refactoring to user stories of the platforms users. Well, I'm not
sure about that...Maybe the term framework is mapped into two different
things in our minds. It seems to me that you talk about a framework as
a bunch of design patterns connected, and I'm talking about a platform
like JVM, compiler, or any other analogy which is targeted for
application developers.
Hope it helps to clarify my questions and doubts,
Thanks,
Zohar
| |
|
| zets wrote:
> We develop a platform, maybe a framework was a bad term. Again, take
> the analogy of development of a compiler, or a application server, or
> JVM. The target audience is application developer, that's the end
> users.
>
> I don't think the myth you're describing is really THE myth. It's
> trivial that agile will also work for evolving system, when
> requirements keep coming, but nevertheless, this is really not the
> case.
XP also works when the requirements are graven in stone.
> Think of a compiler, you know all the requirement, or enough
> requirements to start with on day one of the project, because it's
> either standard, or because you have compilers in the market already.
> You promote writing a platform only when you have enough users for
> that. I completely agree, but trust me, end users for the platform
> exist, they're too many...
>
> So I follow your rules but I still don't get my questions answered...
> 1. I have target user population
> 2. I have way more than 3 product programs using the platform
>
> There's one thing I'm missing here, and I think there's one thing you
> are...
> The thing you're missing is that from user's perspective, the platform
> doesn't change. The change is implementation detail. This is the
> equivalent of Sun changing the implementation of their JVM. The API for
> clients doesn't get affected, but the internal implementation does. Now
> assume Sun has already a JVM, but from many reasons they want to change
> technology. You may argue that if it's working why change, but the
> reasons for change of technology may result not only from technical
> considerations, but also from market reasons (partnerships, etc.).
> Now it's the part I'm missing. On this analogy, how do I make the move
> from a JVM written in one technology to a JVM written in another
> technology, using agile methods?
Under XP, you express the user's needs as storytests. So in the case of an
API, you write storytests that nail down the API. Then you can refactor and
emerge all you like, behind the API, and the storytests will preserve your
user-programmers' experience.
> The way I would do it with waterfallish techniques is to analyse what I
> have in the original JVM, identify the techniques to do the same with
> the new technology, prepare the architecture of the JVM using the new
> technology, work on the design, write the tests and code, and get it
> done.
> One problem is also that the original code DOES NOT have unit tests, at
> least most of it.
Write the tests for the old version, then port each test to the new version.
The tests will preserve the old API. I don't see what you think the problem
will be.
BTW you still haven't said _why_ you are rewriting? Did Sun break the
migration from JSEE to J2EE?
> I completely resent the thought that TDD is a panacea.
Prevention is better than a cure. A "panacea" is the next worst thing - just
a cure.
> TDD can result
> in a higher quality code faster. I don't see how it can replace a good
> design or architecture or analysis.
Use TDD in association with
- frequent iterations
- sorting features in order of business value
- storytests for each user feature
All that stuff _is_ analysis. Or were you going to consider your "analysis"
finished even if it didn't lead to an automated test?
> Try to think of any known
> architecure, SOA, J2EE, COM, .NET. From your text, it could emerge from
> TDD+refactoring to user stories of the platforms users. Well, I'm not
> sure about that...
One completely amazing story there is the journey of CORBA. The Object
Management Group owns its definition. They are the most expensive consortium
of the smartest software engineers on the planet, right?
Their technology is currently in major decline. Modern CORBA is
over-specified, difficult to implement, and a nightmare to use. The culprit
is OMG's process. This thread discusses why:
http://groups.yahoo.com/group/extre.../message/121012
What's amazing about the relationship between Michi Henning's article (
http://tinyurl.com/nrlbf ) and XP is this: With our training, we only need
to read Michi's premise before guessing exactly what will happen.
The OMG does not require reference implementations before it will ratify a
CORBA specification. Reference implementations are a Best Practice. It's how
ISO collects new features for C++.
Reference implementations would be the equivalent of one team (the
user-programmers) writing failing acceptance tests, and patches, for other
teams (the vendors) to pass. That system would prove the resulting features
will be technically viable, and actually useful.
Instead, OMG members debate via paperwork. And we can easily predict the
cruft, politics, low velocity, uncompetitive features, and loss of market
share that the OMG's BRUF and BDUF will cause.
Then we read Michi's conclusion, and it's true: CORBA's modern versions are
besotted with cruft, politics, low velocity, uncompetitive features, and
loss of market share.
> Maybe the term framework is mapped into two different
> things in our minds. It seems to me that you talk about a framework as
> a bunch of design patterns connected, and I'm talking about a platform
> like JVM, compiler, or any other analogy which is targeted for
> application developers.
I have talked all along about a platform - about the Core at the center of
SEI's Software Product Lines.
I have used XP to generate such a framework; a platform. TDD, emergent
design, everything. The platform was a success, despite opposition to the
point of executive sabotage, trying to prevent it from working.
--
Phlip
[url]http://c2.com/cgi/wiki?ZLand[/url] <-- NOT a blog!!!
|
|
|
|
|