| sirgilligan 2004-03-26, 11:51 pm |
| Here is the latest version of my document. I have tried to learn from
many sources including this group. Thanks.
Geoff
______________________________
Maverick Agile Methodology (Work in progress)
By Geoffrey Slinker
Abstract
Maverick Agile Methodology is an approach to achieve the goals of 1)
developing software where the requirements are not known or are
changing, 2) developing software of high quality, 3) the team members
are not necessarily co-located, 4) the system is integrated from many
tiers and layers, and 5) the team is larger than 40 individuals.
Terms
2D requirements. Agile Development. Iterative Development.
Requirements
The system is designed with the customer or inventor present. The goal
is to accurately capture the requirements of the system up to the
point the customer can clearly communicate the requirements. Also, the
requirements must be easily browsed to be able to refresh people's
mind. Another goal of the requirements gathering is to present the
requirements in a way that is appropriate for the complexity of the
system. Also, the requirements must be easily modified.
Modification of requirement specifications can be like tightly coupled
code, and a modification in one place breaks something in another
place. It is amazing how the entire development process is similar to
coding. It is the management of complexity and the inability of the
human mind to conceptualize very complex systems. Since one goal is
that the requirement specifications are easily modified then there are
several practices and methods that can be used to achieve this goal.
One aspect of a quality requirements specification is that it
minimizes the possibilities of misinterpretation. Requirement
specifications should be kept as simple as possible and broken down to
the most concise description possible.
If electronic versions of the requirement specification are used it
can greatly ease their modification and dissiminitation. Remember that
Maverick Development says that any means is right if it meets the
goal. Hand drawn diagrams can be finished in just minutes on a white
board. It is amazing that the same diagram drawn with a modeling tool
can take all day because we fuss and fight with the tool to get
everything just perfect. So, if hand draw diagrams meet the goal, use
them and move forward. Scan them and save bitmaps of them. Maybe you
should invest in a tablet and pen for inputting these diagrams.
Remember, technology can hinder productivity as well is improve it.
Since agile development assumes that the requirements are incomplete
and that the customer is the only one able to refine the requirements
and make changes when the customer perceives a market shift then the
customer must be easily accessible. Since Maverick Agile Development
is for large teams, it is not possible that one customer can address
every developer's need at any given time. If multiple client
representatives are not available then local experts are needed to
make inter-iteration decisions. The local expert may not have a
perfect idea of how the system should be, but with three w
iterations the deviation from the ideal should be minimal. Since there
are not enough customer representatives Maverick Development realizes
that incorrect choices and false starts are going to happen and there
is no panicking when it does. If the local experts continuously
misdirect the project then the customer will correct the problem with
more personal visibility, retraining of the local experts, or with new
local experts.
Agile approaches seem to assume a "thin thread" or "steal thread"
approach to development. This is the aspect of agile development where
there is always a working "slice" or "thread" of the system available
to the customer. This is natural if the product has a UI. What if the
product is a plug-in for an application or a record manager or
indexing engine? This is where you have to be a computer scientist
instead of a programmer. How do you approach a product so that there
is something that you can demonstrate early and then build up or
around that functionality? A functional decomposition could meet the
goal. A bottom-up or top-down approach with the appropriate wrappers
and stubs could do the job.
Maverick Development has some similarities to Crystal methodologies.
There will be a Maverick Methodology emerge for each type of
development need. In time there will be a specific Maverick
methodology that addresses software that doesn't have a UI, software
that has to meet rigorous reviews for a critical system, and many
other types of software development tasks.
Design and the 2D Model
Maverick Agile Development assumes the use of an object oriented
language.
If the system will be implemented by several modules, different tiers,
and such, then the interaction of each piece most be considered.
Simple diagrams to sketch the interface and interactions are all that
are required. A simplifying of UML or some technique is sufficient.
Remember electronic versions of diagrams can be a trap and many hours
can be wasted perfecting the look of an interaction diagram.
All objects are built on what is referred to as the two-dimensional
model. This is a new concept but not an unfamiliar one. Objects exist
to satisfy requirements in two dimensions. The first dimension is that
of the real world entity the object models. The second is the behavior
of the object in a computer system.
There will be methods that meet the requirements for both dimensions.
For example you have an object that models a Car. There may be methods
like get the color or set the color. These are methods that have to do
with the real world object. But the Car object exists in the computer
world as well. And in the computer world we do things like compare
objects. You normally don't think about saying, is this car equal to
that car. But in the computer system such comparisons exist.
Therefore, Maverick Agile Development uses requirements for the real
world and well as the computer world and these are called 2D
requirements.
In Maverick Quality Assurance (MQA) three requirements are identified
that are of this second dimension.
1) Object factories. Object factories are needed by MQA to allow for
the develoment of test harnesses, stubs, and mock objects that are
needed to develop complete unit tests.
2) Object Validators are validation facilities that examine an object
for comformance with required values. MQA component tests needs object
validators to check objects returned from a component call.
3) Object Equivalence Verifier. MQA tests will need to compare two
objects to see if the are the equivalent. A test that needs this
functionality would be a persistence test. Equivalence can be a
difficult problem to solve (e.g. The Record Linkage Problem) and there
may be a valid need to relax the definition of equivalence.
Factories, Validators, and Equivalence Verifiers may not be part of
the model for the real world object. However, since the object exists
in the dimension of a computer system these are necessary requirements
that must be met.
Testing
Maverick Agile Development addresses the issues of unit, component,
integration, system, performance and acceptance tests as a full part
of the system. Maverick Quaility Assurance goes into detail of the
methods for development to improve the quality of the product.
Maverick processes and models are based on the premise that it is
better to have those with domain knowledge perform a task instead of
an external team if the requirements of the system are in continuous
flux or if the transfer of the domain knowledge is expensive. Because
Maverick Agile Development is for large teams working on a product
with changing requirements development are responsibile for unit,
component, and integration tests.
Unit tests are tests on the smallest units in the system. The unit is
completely tested. Some object unit tests only test the properties
(getters and setters). Maverick Agile Development require testing of
all methods, and messages. This requires the use of test harnesses,
stubs, and mock objects. Maverick Quality Assurance goes into detail
on the subject matter.
Component tests are the same unit tests with the stubs and mock
objects removed. The component tests are simply a different "build" of
the unit tests. The component tests are linked to real system
components instead of mock system components. Because of this,
component tests are more of a build issue than a seperate issue from
unit tests.
In Maverick Agile Development a system is integrated when no mock
objects are used in the build and the system passes all of the
component tests.
Maverick Agile Development uses pair programming for component
integration issues. When one method calls into another part of the
system, the developers of each component of the system pair with the
each other to make sure the component is called correctly. They review
the call, the return values, and the exceptions and refactor
accordingly.
Continuous Build
One of the areas that is difficult for other agile methods is the
continuous build process if the team becomes large or the unit tests
take too much time to run.
The goal of the continuous build is to present a version of the
product that is always functioning correct for the features
implemented. Another goal of the build process is to eliminate broken
builds at the main repository.
This allows developers to never be broken and always working. This
allows the customer to use the system at anytime and see the current
state of the product during the iteration.
This presents a particularly difficult task when the team is large. If
a developer updates the code on his machine he may get part of the
files with changes that another has committed. Transactions are needed
to prevent getting only part of the files of a large commit.
If the tests take too long to run developers will save up their
changes to avoid the wait and wasted time. When they wait to commit
changes the conflicts with other changes cause the developer to get
caught in a refactoring frenzy trying to keep up with changes. Since
files are changing so often it requires the idea of relaxing the
constraint of developing from the "head" of the revision control
system.
For Maverick Agile Development code is always updated from a good
build which is labeled as a specific revision. The thought comes to
mind that people will be working on "stale" code, but on a large team
with many changes the code is stale so fast it doesn't matter when you
get it. One reason to get from a revision is if the "head" build is
broken and you have made changes and you want to update your local
code to do a build and run the tests locally before you commit
changes. If you update to a broken build you will have to wait until
that build is fixed, or rollback to a good revision, or worse yet you
don't notice the build was broken when you updated and you think that
your changes have broken the build.
Other processes for updating and committing code can be used. A patch
process can be used where the differences of the local files are sent
to a build machine and that machine builds the system and if it
compiles and passes the tests it is then committed to the "real" build
machine (the machine that has the revision control repository). This
process can take a long time to perform. On large systems the setting
up of test data can take several minutes and the running of all of the
tests could be a significant amount of time as well. If you are trying
to commit a change that another team member is waiting on it could be
a few hours before the changes propagate through the process.
The continuous build must be as fast as possible. Distributing the
build process can help with a patch process. Each feature team or some
topological division of the developers has a patch build machine. But
that alone is not enough. To improve turn around time the builds
themselves have to be changed. The ideal is to have a build for a
large project (which includes running tests) be no longer than 15
minutes. Doing a clean build in the continuous build process takes to
long. For large teams a clean build is done every fourth build or
every hour. Interim builds are not clean builds. In the interim only
changes and their dependencies are built.
A goal of the build process is to eliminate broken builds. Before
submitting changes to the build process the developer should get an
updated version of the system and build it locally and run the tests.
Usually this local build would not be a clean build. If the build
process is fast enough this step should be eliminated. In Maverick
Development you do whatever moves you down the road, and if there no
need for a step you eliminate it. This is built on the idea of being a
thinking individual that understands the ramifications of decisions
and that the individual is concerned and will not do anything to
interfere with another team member's ability to work. So, if the build
process is a patch process and it is fast enough the local build is
redundant.
Iterations
The goal of iterative development is to always provide an improving
product. Improvements are not limited to added functionality alone.
Improvements in quality or performance are valid as well as anything
else that is a real improvement.
are no more than 3 ws. It is about planning, refocusing,
refactoring, and staying on target. Any more than three ws is too
long to readjust. If there is a concern that the iteration planning
takes to much time or is too expensive to do every three ws, then
the way the planning is done needs to be improved.
There is some overhead in pulling everyone together. If the Maverick
Development Model is followed and meetings have goals reflected by
agendas then the meeting will be as optimal as a meeting can be.
Maverick development is based on an idea that software is done when it
is done and there are at least a certain minimal number tasks that
will be performed before the software is done. Since these essential
tasks will be done no matter how long the iteration it does not slow
the development momentum to have short iterations.
Offshore/Offsite
If offshore/offsite teams are used in the development, the iterations
are two ws or less. This increases the communication to a level
that is needed for redirection. The problem is that offshore teams
suffer from poor communication and get off target and no one knows
until it is too late. The method is to use an iterative approach where
there is always a working subset of the product until it is developed
entirely. With shortened iterations you will be able to use the
iteration to quickly see if the offshore team is working as planned
and if not to make a course correction. Like other agile methods, this
depends on the other processes of the methodology in place. The
continuous build is essential for this to work. The offshore team must
be working from a shared source base and a patch process will address
the build needs for a distributed work force.
Development Platform
One goal for large development teams is to keep developers up and
working and to get new employees setup quickly.
If the project is using Java then the development machines should be
setup the same using the same OS, same versions of the SDK and such,
same versions of Ant, Bash, etc. Since Java can run on Mac OS X,
Linux, Solaris, HP-UNIX, BSD, ... it becomes an unnecessary burden to
manage special shell scripts and suttle differences of each platform
and configuration.
There is an argument that if the product is going to be delivered on
multiple platforms and configurations it is good to have such
platforms and configurations spread about development. This is where
Maverick Development comes in and says "Why?" One of the most
prevalent reasons for having the various machines is to detect any
problems with the product running on the different configurations.
That is a valid goal, but the solution of having developers with
different platforms and configurations is not satisfactory for large
teams.
With large teams the different development platforms quickly find
problems with the build process across the various platforms. The
problems with the build become the predominant issue with the
different platforms instead of finding issues with the product
deployed on various machines.
The various platforms that will host the product will be maintained by
those in charge of deployment tasks. They will keep the various
platforms configured and working. Problems that have to do with code
will be brought to the attention of the developers. Problems that have
to do with configuration and deployment will be handled by the
deployment team. The build will be run continuously on all the
platforms and any broken tests will be addressed by development. If a
particular platform is becoming problematic then maybe it is good that
it was discovered early and you can eliminate that platform from the
list of supported systems.
Having an exact setup of the desired deployment environment is not
usually possible early on in the development of the product. Bring the
various systems on line as necessary are dictated by driving forces
such as budget, schedule, hosting facilities, and important decision
factors.
The configuration of each development build will be the same. The
location of items on each machine will be the same. The environment
variables and paths will be the same.
If the configuration is standard then new employees can receive an
"image" that can be easily installed. If a developer's machine
experiences hardware trouble the machine can be restored very quickly
to a usable state. With the same setup on each machine it is easier to
do pair development. The difficulties for large teams are enough and
we do not need to add unnecessary difficulties with various
configurations.
If the development is in a language like C++ and there are multiple
target platforms development will need to have access to these
platforms continuously. Languages that have compiler directives and
the ability to conditionally specify code for compilation (#ifdef
comes to mind) require that the tests be ran on each target platform
in order to catch compile errors. The continuous build process becomes
more complicated as well. An important aspect of this methodology is
that there is no porting team. The developers are responsible for
their code to run on every supported platform. This will give many
benefits. One is that they will not write the code in an optimized
fashion for their preferred platform. Another is that you do not have
two different individuals working on the same code that might
interpret the requirements in subtly different ways. When organizing
the teams and if continuous pair programming is used, then wisely pair
people with experience on different platforms.
THIS METHODOLOGY IS STILL UNDER DEVELOPMENT.
Geoffrey_slinker@yahoo.com
|