Home > Archive > Extreme Programming > August 2005 > Smells versus beauty
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Smells versus beauty
|
|
| Juancarlo Añez 2005-08-06, 9:01 am |
| I'm trying to help some juniors (one to three years of
programming-for-a-living) learn how to write better code. We've done
some great work in the projects we've done together, but they get into
trouble quickly because of bad code when I'm not around.
The notion of "code smell" has certainly been helpful in this
endeavour. It is easy to show how copy/paste repetition is risky and
always ends up meaning much more work than refactoring into
generalizations, for example. [Where's the rederence to "coding debt",
BTW?]
Yet, this morning I was thinking that the "smell" notion by itself
doesn't really cut it because refactoring out "smells" is too much
about ths symptoms of bad health and not enough about good health, just
like taking pain killers may do away with the recurrent headache but
will not improve your health. "Smells" are too much about the current
situation and too little about where you want to be.
There needs to be a way to let juniors learn about the aesthetics of
good code so they can s it. To learn that experienced programmers
rely a lot on "intuition", and that such intuition is as much objective
expertice as it is about inspiration. That the sinergies that happen in
pair-programming can only be explained on epistemological (and even
metaphysical) grounds. That sing beauty through creativity touches
on the divine.
Christopher Alexander deals with these themes in beautiful ways in his
books, but the notion of code beauty (of "the way") has yet to make it
into the software writing literature. The books "The Pragmatic
Programmer" and "Refactoring into Patterns", and the writings of
"Richard Gabriel" are steps in the right direction, but we're still
missing serious treatment about the aesthetics of software creation.
It is my impresion that, without the aesthetics, the motivation, the
goal, are simply lacking, and that thus things like smells,
refactoring, and patterns end up as temporary patches: the moment you
turn away the code quality starts to degrade because the team is
lacking in the notions of beauty, of harmony, equilibrium, simplicity,
....
Pair programming addresses that but, without consideration for the
aesthetic, it is yet just another patch.
Sorry for the long note, but I was short on time.
Juanco
| |
| Daniel Parker 2005-08-07, 5:01 pm |
|
"Juancarlo Añez" <apalala@gmail.com> wrote in message
news:1123335407.299864.181120@z14g2000cwz.googlegroups.com...
> I'm trying to help some juniors (one to three years of
> programming-for-a-living) learn how to write better code. We've done
> some great work in the projects we've done together, but they get into
> trouble quickly because of bad code when I'm not around.
>
My view: you're creating too much of a dependency on yourself. Juniors
have to be allowed to make their own mistakes, that's how they get better.
Stop monitoring them so much, stop correcting every little thing that they
do, give them some room. Juniors' are supposed to make mistakes, that's
part of the learning process. Focus on output, not input; have acceptance
tests in place that will catch errors, but don't be too quick to help them
fix the errors, let them take it as far as they can before you step in. At
a certain point, you should stop even answering questions they should be
able to figure out, force them to come up with solutions themselves, even if
they're not as good as your solutions. After they've done the best they can
do, maybe provide some feedback, but leave it up to them whether to
incorporate the feedback. The only thing you should insist on is the
correctness of the final output.
> It is my impresion that, without the aesthetics, the motivation, the
> goal, are simply lacking, and that thus things like smells,
> refactoring, and patterns end up as temporary patches: the moment you
> turn away the code quality starts to degrade because the team is
> lacking in the notions of beauty, of harmony, equilibrium, simplicity,
>
I think you should focus on correctness of output rather than the beauty of
input. The two may be correlated, but likely not in a causal way, both
being correlated with the third factor, the programmer.
>
Regards,
Daniel Parker
| |
| Juancarlo Añez 2005-08-07, 5:01 pm |
| Daniel,
| My view: you're creating too much of a dependency on yourself.
Sorry, but that's speculation. My job is officially, formally, and
practically not that of a mentor. Like for most people, my job is to
get profitable things done. There are many projects going on in which
I'm not involved in (of which I don't have the slightest idea).
But when I get called into a project that's in trouble, I invariably
observe that the problems are mostly due to bad design and bad code.
Part of the work is pointing out the problems and mentoring the team on
how to solve them, but my ultimate task is getting the project moving,
even if it means replacing part of the team.
The juniors I talk about are not my pupils. My observation about having
done good work with them was meant to emphasize that they _can_ do good
work, and my note had the intention of pondering on why they would do
good work under certain circumnstances and bad work the rest of the
time.
The idea I proposed was that juniors lacked an aesthetic sense about
how good code and good designs should be (feel?) like. I strongly
believe that's the issue.
I agree that experience is paramount, but a three-year graduate has
exactly three years of experience, and will get exactly one year of
experience every year you're watching, if the environment is right.
I have a problem. I don't have the solution, which might be a book that
inspires, better mentoring, process, or team building, or whatever. But
I am certain that it all has to do with subjective appreciation of what
good programming is.
Can you help with that?
| I think you should focus on correctness of output rather than the
| beauty of input. The two may be correlated, but likely not in a
| causal way, both being correlated with the third factor, the
| programmer.
We do all that! But projects get into trouble anyway, and the most
frequent cause is bad code. I insist: the cause of that is lack of an
aesthetic perception of what good code should look like.
--
Juanco
| |
| Phlip 2005-08-07, 10:00 pm |
| Juancarlo Añez wrote:
> The idea I proposed was that juniors lacked an aesthetic sense about
> how good code and good designs should be (feel?) like. I strongly
> believe that's the issue.
Are they pair programming?
--
Phlip
[url]http://www.greencheese.org/ZLand[/url] <-- NOT a blog!!!
| |
| Tim Haughton 2005-08-08, 4:00 am |
| "Juancarlo Añez" <apalala@gmail.com> wrote in message
news:1123335407.299864.181120@z14g2000cwz.googlegroups.com...
>
> Yet, this morning I was thinking that the "smell" notion by itself
> doesn't really cut it because refactoring out "smells" is too much
> about ths symptoms of bad health and not enough about good health, just
> like taking pain killers may do away with the recurrent headache but
> will not improve your health. "Smells" are too much about the current
> situation and too little about where you want to be.
>
If your olfactory palate is up to scratch, removing smells should get you to
where you want to go in the most expedient fashion.
> There needs to be a way to let juniors learn about the aesthetics of
> good code so they can s it. To learn that experienced programmers
> rely a lot on "intuition", and that such intuition is as much objective
> expertice as it is about inspiration. That the sinergies that happen in
> pair-programming can only be explained on epistemological (and even
> metaphysical) grounds. That sing beauty through creativity touches
> on the divine.
>
"Dire warning, team."* A mistake I made numerous times when I was a junior,
was to get overly interested in the aesthetics of the code. Object Oriented
Programming is addictive if not taken in moderation. I spent (read wasted) a
lot of time looking for the 'best' solution to problems, when an 'adequate'
solution would have sufficed. Only later did I come to redefine the 'best'
solution as the quickest smell free solution to satisfy my acceptance tests.
> Christopher Alexander deals with these themes in beautiful ways in his
> books, but the notion of code beauty (of "the way") has yet to make it
> into the software writing literature. The books "The Pragmatic
> Programmer" and "Refactoring into Patterns", and the writings of
> "Richard Gabriel" are steps in the right direction, but we're still
> missing serious treatment about the aesthetics of software creation.
It's odd. The word 'beauty' is the same word I used to use back then. My
code had to be a shining beacon of object oriented mastery.
>
> It is my impresion that, without the aesthetics, the motivation, the
> goal, are simply lacking, and that thus things like smells,
> refactoring, and patterns end up as temporary patches: the moment you
> turn away the code quality starts to degrade because the team is
> lacking in the notions of beauty, of harmony, equilibrium, simplicity,
> ...
The absolute best way to ensure the best code design, his to follow the red
light, green light, refactor mantra. I've honestly never had better results
with anything else. Refactoring is an essential skill for any developer, and
the act of refactoring shouldn't be thought of as a bad thing. It seems to
me like you might be responding only to duplication, or smells, in the code
domain. Don't ignore duplication or smells in the time domain either. If you
find yourself making the same kind of refactoring, the same change, the same
activity twice, always ask yourself, "what can I do to remove this
duplication."
>
> Pair programming addresses that but, without consideration for the
> aesthetic, it is yet just another patch.
>
I'm curious, what do you see as the purpose of pair programming?
--
Regards,
Tim Haughton
Agitek
http://agitek.co.uk
http://blogitek.com/timhaughton
* 80's English children's TV reference. Prizes to any who can tell me where
it's from.
| |
| Esteban A. Maringolo 2005-08-08, 9:06 am |
| Juancarlo Añez escribió:
> | I think you should focus on correctness of output rather than the
> | beauty of input. The two may be correlated, but likely not in a
> | causal way, both being correlated with the third factor, the
> | programmer.
> We do all that! But projects get into trouble anyway, and the most
> frequent cause is bad code. I insist: the cause of that is lack of an
> aesthetic perception of what good code should look like.
Code style is something the team gets over time, newcomers are
teached about what to do, and what not to do, however, between teams
and companies, some patterns emerges naturally.
If you do Smalltalk, you could watch this:
<http://www.amazon.com/exec/obidos/t...316125?v=glance>
Perhaps there is something similar for languages also (C++, Java, etc).
Best regards,
--
Esteban
| |
| Thomas Gagne 2005-08-08, 9:06 am |
| Refactoring, or optimizing, after writing a bunch of new code is itself
a step that should be expected and embraced. It is often difficult to
anticipate what might become of some new code until it exists.
Our brains spend all day storing information suboptimally then refactor
it during sleep. It's a process that works well for the most
sophisticated computational machinery known to man, why would it not be
valuable for coding?
Look forward to creating software that smells, and enjoy the process of
cleaning it up. Get better at doing it and with increasing frequency.
| |
| Laurent Bossavit 2005-08-08, 9:06 am |
| Juancarlo,
> generalizations, for example. [Where's the rederence to "coding debt",
> BTW?]
http://www.martinfowler.com/bliki/TechnicalDebt.html
http://c2.com/cgi/wiki?TechnicalDebt
> There needs to be a way to let juniors learn about the aesthetics of
> good code so they can s it.
Give them lots of beautiful (and not so beautiful) code to read. One of
the ways we learn aesthetics is by imitation; you have to read a lot
before you can write well. Have them discuss that code in groups,
ideally in groups that include more senior people.
Have them change code (maintenance) as well as (or rather than) write
code (development). That teaches the practical consequences of writing
code in certain ways. When they've suffered through hours of fixing the
same bug in 100 different places, they'll never copy and paste lightly
again.
Give them space to practice - write code purely for training, not
production. (See recent threads about the Coding Dojo.) You need to
write a lot of bad code before you can write good code, the bad had
better be code that you can afford to throw away.
Laurent
| |
| Ron Ruble 2005-08-08, 9:06 am |
|
A few comments:
Juancarlo Añez wrote:
> Daniel,
>
> | My view: you're creating too much of a dependency on yourself.
>
> Sorry, but that's speculation. My job is officially, formally, and
> practically not that of a mentor. Like for most people, my job is to
> get profitable things done. There are many projects going on in which
> I'm not involved in (of which I don't have the slightest idea).
Understood. But nothing in your first post indicated
this. Your first post indicated a concern with more
long term development of programmers, and less on
immediate results.
That said, I agree that you may be creating too much
of a dependency on you.
Philip mentioned pair programming. Do the junior
programmers do -any- kind of peer code review?
In similar situations I have seen, each programmer
does his own thing without any form of review. Once
that habit is broken, they become much more
productive.
> But when I get called into a project that's in trouble, I invariably
> observe that the problems are mostly due to bad design and bad code.
Ehh, maybe. The immediately evident problems are
undoubtedly related to bad design and code. The
root problems are those that caused a bad design
to be used and bad code to fester.
> Part of the work is pointing out the problems and mentoring the team on
> how to solve them, but my ultimate task is getting the project moving,
> even if it means replacing part of the team.
Steve McConnell's "Software Project Survival Guide",
and "Rapid Development" both have some good recommendations
about turning around a failing or stalled project.
Yes, this sometimes means replacing people. But,
usually, replacing people is secondary to fixing
other problems.
> The juniors I talk about are not my pupils. My observation about having
> done good work with them was meant to emphasize that they _can_ do good
> work, and my note had the intention of pondering on why they would do
> good work under certain circumnstances and bad work the rest of the
> time.
Often this is due to a lack of practice "shifting
gears". They need someone to enforce stepping back,
looking at [1] what they are doing, [2] why they
are doing it, [3] how well (or badly) it is working.
It's a natural human behavior to push harder when
doors don't open; to continue down the same road
when you're lost. Changing gears, looking to see
if you need to pull on the door or press an
electronic release button; turning the car around
and backtracking are -unnatural-, learned behavior.
Most junior programmers haven't acquired these habits.
They need help and reminders. Code reviews and design
reviews are part of this. So is a manager who looks
at progress, and makes the programmer sit down, away
from the computer, and talk to him about the problems
he's facing. Often the programmer doesn't need help;
he just needs to calmly think these things through.
Our typical high-pressure, short-schedule development
environments don't help this.
> The idea I proposed was that juniors lacked an aesthetic sense about
> how good code and good designs should be (feel?) like. I strongly
> believe that's the issue.
I strongly believe that's a symptom, not the disease.
> I agree that experience is paramount, but a three-year graduate has
> exactly three years of experience, and will get exactly one year of
> experience every year you're watching, if the environment is right.
What is experience worth? If they only have poor
examples, they learn the wrong things. And a skilled
programmer is not always the best teacher. A lot of
times, those to whom a skill comes naturally make
lousy teachers, because they can't break down what
they do, and they can't understand the mistakes
others make.
> I have a problem. I don't have the solution, which might be a book that
> inspires, better mentoring, process, or team building, or whatever. But
> I am certain that it all has to do with subjective appreciation of what
> good programming is.
>
> Can you help with that?
I don't believe I can. I really feel that this is
-not- a root cause, but a readily observable result
of the underlying problems.
> | I think you should focus on correctness of output rather than the
> | beauty of input. The two may be correlated, but likely not in a
> | causal way, both being correlated with the third factor, the
> | programmer.
>
> We do all that! But projects get into trouble anyway, and the most
> frequent cause is bad code. I insist: the cause of that is lack of an
> aesthetic perception of what good code should look like.
Can't help you then. I'm not being dismissive; please
accept that I am trying to be helpful. Perhaps the
books I mentioned earlier will be of use.
But I feel that "aesthetic perception of what good code
should look like" is a -result- of good programming
experience, not a stepping stone to good programming.
Good luck.
| |
| calmly_relaxing@hotmail.com 2005-08-08, 5:02 pm |
| If it passes the unit tests (assuming they exists), can you not just
push on and critique later (if you have time)?
Are the aestetics realy that important?
| |
|
| calmly_relaxing wrote:
> If it passes the unit tests (assuming they exists), can you not just
> push on and critique later (if you have time)?
>
> Are the aestetics realy that important?
The problem with unit tests is they lower your quality floor. Without tests,
quality can only be so low before you suffer and start cleaning up. With
tests, quality can be absurdly low, and you can still add features rapidly.
So, clean up relentlessly, as you go, so new features become even easier.
--
Phlip
[url]http://www.greencheese.org/ZLand[/url] <-- NOT a blog!!!
| |
| Rich MacDonald 2005-08-09, 4:02 am |
| "Tim Haughton" <timhaughton@gmail.com> wrote in
news:SsDJe.960039$ub.631953@fe07.news.easynews.com:
> "Dire warning, team."* A mistake I made numerous times when I was a
> junior, was to get overly interested in the aesthetics of the code.
> Object Oriented Programming is addictive if not taken in moderation. I
> spent (read wasted) a lot of time looking for the 'best' solution to
> problems, when an 'adequate' solution would have sufficed. Only later
> did I come to redefine the 'best' solution as the quickest smell free
> solution to satisfy my acceptance tests.
>
[snip]
>
> It's odd. The word 'beauty' is the same word I used to use back then.
> My code had to be a shining beacon of object oriented mastery.
>
[snip]
>
> The absolute best way to ensure the best code design, his to follow
> the red light, green light, refactor mantra. I've honestly never had
> better results with anything else.
Wonderful post, especially the "addictive if not taken in moderation".
Although I do think a recovering OOer has a great vantage point.
Somehow we need to combine "good enough pragmatism" with a deep need to
*care* about the code. If you don't care deeply, you may reach adequacy
but you'll never reach beauty and brilliance.
As to the original post, is it possible to "nibble away" at a poor design
via refactoring to overcome the local optima and reach a global optimum?
I'd guess that until you know what great design is, then no. But nibbling
can create a powerful "toolkit" of awareness and experience and that is
valuable. When the team is stuck in local oscillations, its possible, as
an expert, to "inject" new refactorings and major overhauls and thus have
a major educational impact via a series of "ahas". After all, everyone
improves in stages and the great designers didn't get there all at once.
| |
| Tim Haughton 2005-08-09, 4:02 am |
| "Rich MacDonald" <rich@@clevercaboose.com> wrote in message
news:Xns96AD606028richclevercaboosecom@2
4.94.170.102...
> Somehow we need to combine "good enough pragmatism" with a deep need to
> *care* about the code. If you don't care deeply, you may reach adequacy
> but you'll never reach beauty and brilliance.
An adequate solution should be at most an arms length away from an
aesthetically beautiful solution. Judging that arms length is something that
probably comes with experience more than anything.
> As to the original post, is it possible to "nibble away" at a poor design
> via refactoring to overcome the local optima and reach a global optimum?
Absolutely. The essence of refactoring with unit tests is 'little steps'.
> I'd guess that until you know what great design is, then no.
Refactoring is a skill, no doubt about it. Simply removing duplication
should guide you to a better design, I think experience will dictate how
quickly your design will improve. When I refactor, I do so with the benefit
of a good understanding of OOP, and I apply that knowledge a little bit at a
time.
> But nibbling
> can create a powerful "toolkit" of awareness and experience and that is
> valuable. When the team is stuck in local oscillations, its possible, as
> an expert, to "inject" new refactorings and major overhauls and thus have
> a major educational impact via a series of "ahas". After all, everyone
> improves in stages and the great designers didn't get there all at once.
Refactoring is a business exercise. The bigger the nibble, the bigger the
risk but the larger the potential pay back. Experience tends to improve your
bite size. No...that's not right. Experience makes you *think* you've
improved your bite size. I've seen plenty of very talented developers bite
off more than they could chew in a refactoring.
All these oral metaphors are reminding me that I have a dentist's
appointment this afternoon. Apparently I didn't brush my teeth properly
after refactoring.
--
Regards,
Tim Haughton
Agitek
http://agitek.co.uk
http://blogitek.com/timhaughton
| |
| Donald Roby 2005-08-09, 10:00 pm |
| Rich MacDonald wrote:
>
<snip>
>
> As to the original post, is it possible to "nibble away" at a poor design
> via refactoring to overcome the local optima and reach a global optimum?
Since refactoring is normally done in small steps, it tends to be local,
and only likely to achieve local optima naturally.
I'd suspect that in fact, a global optimum probably doesn't always even
exist.
Design quality measurements are not too precise anyway and the space may
not be sufficiently ordered to even sensibly contemplate a global optimum.
| |
| Phlip 2005-08-09, 10:00 pm |
| Donald Roby wrote:
> Rich MacDonald wrote:
>
> Since refactoring is normally done in small steps, it tends to be local,
> and only likely to achieve local optima naturally.
That's one reason to implement features in order of business priority. The
most important code gets written first, and subsequent code supports it.
--
Phlip
[url]http://www.greencheese.org/ZLand[/url] <-- NOT a blog!!!
| |
| hansewetz@hotmail.com 2005-08-10, 9:03 am |
| Donald Roby wrote:
> Rich MacDonald wrote:
> <snip>
>
> Since refactoring is normally done in small steps, it tends to be local,
> and only likely to achieve local optima naturally.
Re-factoring is not the cause of the problem (of ending up in local
minima) - it is a symptom. The cause is not having a good enough
understanding of the problem domain.
Obviously I'm exaggerating a little. However, re-factoring is
something that is seen as something 'normal' by many developers. In
my mind, re-factoring should be avoided as much as possible. The way to
avoid it is to understand as much as possible before doing.
Regards,
Hans Ewetz
| |
| Tim Haughton 2005-08-10, 9:03 am |
| <hansewetz@hotmail.com> wrote in message
news:1123665833.910883.16980@g14g2000cwa.googlegroups.com...
> Re-factoring is not the cause of the problem (of ending up in local
> minima) - it is a symptom. The cause is not having a good enough
> understanding of the problem domain.
>
> Obviously I'm exaggerating a little. However, re-factoring is
> something that is seen as something 'normal' by many developers. In
> my mind, re-factoring should be avoided as much as possible. The way to
> avoid it is to understand as much as possible before doing.
Behold the waterfall everyone. Isn't it pretty?
Wait...I'm being unkind. Perhaps you *are* talented enough to create a
design upfront that doesn't have to change. Perhaps you *can* guide a
project along its life cycle without having to change code. Given the
learning of the last 10+ years, you would seem to be in the tiniest 'elite'
minority.
--
Regards,
Tim Haughton
Agitek
http://agitek.co.uk
http://blogitek.com/timhaughton
| |
| Donald Roby 2005-08-10, 9:03 am |
| hansewetz@hotmail.com wrote:
> Donald Roby wrote:
>
>
>
> Re-factoring is not the cause of the problem (of ending up in local
> minima) - it is a symptom. The cause is not having a good enough
> understanding of the problem domain.
>
In the portion of my note that none of the responders seem to be
bothering with, I think I suggested a much more likely reason for
failure to find a global optimum. It doesn't generally exist.
> Obviously I'm exaggerating a little. However, re-factoring is
> something that is seen as something 'normal' by many developers. In
> my mind, re-factoring should be avoided as much as possible. The way to
> avoid it is to understand as much as possible before doing.
>
Are you able to preconceive an optimal design by complete understanding
of the problem domain? Very impressive! How do you define optimal
design anyway?
And for how many years of maintenance while the problem domain is
changing does that design remain optimal?
> Regards,
> Hans Ewetz
>
| |
| Donald Roby 2005-08-10, 9:03 am |
| Phlip wrote:
> Donald Roby wrote:
>
>
>
>
>
>
> That's one reason to implement features in order of business priority. The
> most important code gets written first, and subsequent code supports it.
>
Yes. It gets you to local design quality optimization that supports the
most important stuff.
| |
| hansewetz@hotmail.com 2005-08-10, 9:03 am |
| Tim Haughton wrote:
> <hansewetz@hotmail.com> wrote in message
> news:1123665833.910883.16980@g14g2000cwa.googlegroups.com...
>
> Behold the waterfall everyone. Isn't it pretty?
>
> Wait...I'm being unkind. Perhaps you *are* talented enough to create a
> design upfront that doesn't have to change. Perhaps you *can* guide a
> project along its life cycle without having to change code. Given the
> learning of the last 10+ years, you would seem to be in the tiniest 'elit=
e'
> minority.
I think my point was pretty clear: re-factoring should not be viewed as
something good. Instead it should be viewed as a necessary evil =EF=81=8A
I don=E2=80=99t like the idea of pushing re-factoring to the degree where
even the smallest designs are =E2=80=98evolved=E2=80=99 through re-factorin=
g when
it is simple to sit down and think for an hour or two and get it right
the first time. Clearly, the larger the design is the less likely this
is to happen. Also, in any domain there are properties that are stable
and do not change. It is important to identify such properties, and it
can be done without using re-factoring. Everything is not variable =E2=80=
=93
if it were, there would be no talk about domains.
But, I=E2=80=99m probably using the wrong language =E2=80=93 if I drop down=
to your
inflamed way of expressing your viewpoints I would say:
=E2=80=9CGod forbid that we should understand anything before =E2=80=98evol=
ving=E2=80=99
solutions to problems through the magic of re-factoring. After all,
software developers ARE experts in all areas of life. Old men with long
gray beards having spent a life time developing sophisticated methods
for solving problems in all kinds of domains have simply been wasting
their time =E2=80=93 they should have solved them through re-factoring :-)=
=E2=80=9D
Regards,
Hans Ewetz
| |
| hansewetz@hotmail.com 2005-08-10, 9:03 am |
| Donald Roby wrote:
> Are you able to preconceive an optimal design by complete understanding
> of the problem domain?
Never said I was. ... ???
Sounds as if you are trying to misunderstand my comment on purpose...?
> And for how many years of maintenance while the problem domain is
> changing does that design remain optimal?
Everything in a domains does not change randomly. There are stable
properties that define the domain.
Regards,
Hasn Ewetz
| |
| Tim Haughton 2005-08-10, 9:03 am |
| <hansewetz@hotmail.com> wrote in message
news:1123670336.162569.215370@g43g2000cwa.googlegroups.com...
> I think my point was pretty clear: re-factoring should not be viewed as
> something good. Instead it should be viewed as a necessary evil ?
Refactoring is "improving the design of existing code." Not sure how it
could be construed as evil.
You seem to be equating understanding with designing. Would you concede that
it is possible to understand the problem domain without designing?
Does your experience tell you that doing big design upfront (BDUF) is more
economically viable than using emergent design? If so, great! But like I
said before, you are in a tiny minority. Empirical evidence tends to suggest
the opposite. My experience tends to suggest the opposite. I have used BDUF
*and* emergent design.
Of course, any process requires a heaped table spoon of pragmatism. If I'm
implementing a discrete Fourier transform, of course I'm going to reach for
my Numerical Recipes book, I'm not going to let a DFT 'emerge'. Although, on
occasion, unlearning something and letting a solution emerge can surprise
you.
--
Regards,
Tim Haughton
Agitek
http://agitek.co.uk
http://blogitek.com/timhaughton
| |
| Ron Ruble 2005-08-10, 9:03 am |
| hansewetz@hotmail.com wrote:
<snip>
> Re-factoring is not the cause of the problem (of ending up in local
> minima) - it is a symptom. The cause is not having a good enough
> understanding of the problem domain.
I feel you may be overstating something here.
Folks, my interpretation follows; correct me if I
am wildly off base.
"Refactoring" is a tool, not a sacred icon. Anymore
than "requirements analysis" is a sacred icon.
Refactoring -because you botched this part of the
program- is not good. But the other respondents are
not assuming that refactoring is done because the
code was crap.
Most -normal- refactoring is done to extend code
or to make it more versatile. This is based on the
YAGNI principle. We made the bet that we weren't
going to need more than the original code. This
allowed us to reduce time up front. Mostly, this
seems to pay off in timed gained by not coding
the things we don't need.
But "you ain't gonna need it" isn't the same as
"you ain't -never- gonna need it". XP never said
the latter was true.
If you're doing it right, refactoring isn't
-lost- time; it's invested time. It's just
invested on the back end rather than the front.
> Obviously I'm exaggerating a little. However, re-factoring is
> something that is seen as something 'normal' by many developers.
Refactoring should be seen as normal by practitioners
of XP. Many coders elevate code as sacred, never to
be changed once it works. But when the needs change,
the code should change, otherwise we get end up
with a 'big ball of mud'.
The emphasis on YAGNI and refactoring tries to
develop the habit of doing things better by
not perpetuating defensive coding habits
that don't work.
> In my mind, re-factoring should be avoided as much as possible. The way to
> avoid it is to understand as much as possible before doing.
I think this is the biggest conflict. For more on
my point of view, read Tom Demarco's book "Slack :
Getting Past Burnout, Busywork, and the Myth of
Total Efficiency".
In a perfect system, refactoring would not be
necessary. You'd nail the design, requirements
would never change, surprises would never happen.
I've never seen a perfect system; I don't think
most people here have. Managers who try to remove
the slack in systems end up breaking the system.
Thanks for your input.
| |
| hansewetz@hotmail.com 2005-08-10, 9:03 am |
| Tim Haughton wrote:
> Refactoring is "improving the design of existing code." Not sure how it
> could be construed as evil.
I believe that if you can avoid it you should avoid it. It is not my
first choice when selecting a tool. If I have to use it I will use it.
I never said that it should not be used.
> You seem to be equating understanding with designing. Would you concede that
> it is possible to understand the problem domain without designing?
I'm not equating the two. You are assuming I am.
Of course it is possible to understand a problem domain without knowing
how to design software for it. However, it is a lot easier to design
the software of you also understand the domain.
> Does your experience tell you that doing big design upfront (BDUF) is more
> economically viable than using emergent design? If so, great!
I'm not sure how you are able to draw this conclusion about what I
think.
> But like I
> said before, you are in a tiny minority. Empirical evidence tends to suggest
> the opposite. My experience tends to suggest the opposite. I have used BDUF
> *and* emergent design.
So have I, and depending on the type of problem one of them usually
works better than the other.
I don't like to rule out any technique that can be used to produce a
good design. I'm not sure about you, but appear as if you do just that.
Again, you are drawing a conclusion that I'm in a tiny minority based
on what you imagine I think.
> Of course, any process requires a heaped table spoon of pragmatism. If I'm
> implementing a discrete Fourier transform, of course I'm going to reach for
> my Numerical Recipes book, I'm not going to let a DFT 'emerge'. Although, on
> occasion, unlearning something and letting a solution emerge can surprise
> you.
That's good and I agree with you. However, there are many developers
that would not do so. Instead they would use some re-factoring
technique that lets the design 'emerge'. This is exactly what I'm
against.
Regards,
Hans Ewetz
| |
| scottf3095@aol.com 2005-08-10, 9:03 am |
|
"Tim Haughton" <timhaughton@gmail.com> wrote in message
news:<G1lKe.90971$vm2.32061
> Does your experience tell you that doing big design upfront (BDUF) is more
> economically viable than using emergent design? If so, great! But like I
> said before, you are in a tiny minority. Empirical evidence tends to suggest
> the opposite. My experience tends to suggest the opposite. I have used BDUF
> *and* emergent design.
>
Must everything be BDUF or emergent design? Isn't it better not going
to either extreme?
-scott frye
| |
| hansewetz@hotmail.com 2005-08-10, 9:03 am |
| Ron Ruble wrote:
> hansewetz@hotmail.com wrote:
> <snip>
>
>
> I feel you may be overstating something here.
> Folks, my interpretation follows; correct me if I
> am wildly off base.
>
> "Refactoring" is a tool, not a sacred icon. Anymore
> than "requirements analysis" is a sacred icon.
Agree.
The problem is that re-factoring is often treated as a sacred icon.
> Refactoring -because you botched this part of the
> program- is not good.
Agree.
>
> Most -normal- refactoring is done to extend code
> or to make it more versatile. This is based on the
> YAGNI principle. We made the bet that we weren't
> going to need more than the original code. This
> allowed us to reduce time up front. Mostly, this
> seems to pay off in timed gained by not coding
> the things we don't need.
Agree. However, re-factoring is sometimes done because the domain was
not understood well. Often properties of the domain that 'define' the
domain are not treated as stable but as any other property of the
problem. This, I consider to be a problem.
Re-factoring for the purpose of changing a design because of
requirements that are inherently not stable is necessary. OK, I did
exaggerate a little in my initial postings ... sorry :-)
> But "you ain't gonna need it" isn't the same as
> "you ain't -never- gonna need it". XP never said
> the latter was true.
Agree.
>
> Refactoring should be seen as normal by practitioners
> of XP. Many coders elevate code as sacred, never to
> be changed once it works. But when the needs change,
> the code should change, otherwise we get end up
> with a 'big ball of mud'.
Agree that it should be seen as 'normal' under certain circumstances.
It should not be seen as normal if re-factoring must be done because of
violation of properties that are inherently stable in the domain or
business.
>
> I think this is the biggest conflict. For more on
> my point of view, read Tom Demarco's book "Slack :
> Getting Past Burnout, Busywork, and the Myth of
> Total Efficiency".
My statement '... as much as possible.' I still stand for. If it is
possible to avoid re-factoring without an unreasonable expenditure of
time and effort it should be avoided. This may seem as a copout because
of the '... if ...'. However, most developers with experience of large
projects knows that there it is important to distinguish between
properties that are inherently stable and other properties. If this is
done, many reasons for re-factoring can be avoided.
Just to make sure I'm not being misunderstood (again). I agree with
some comments that my initial statements were too sweeping and general.
However, I also believe that there are different types of issues that
must be solved when creating a design. To manage some issues such as
inherently unstable requirements, re-factoring can help. On the other
hand, management of inherently stable issues such as invariants in the
domain or the business, should not be done through re-factoring -
unless of course the damage is already done.
Regards,
Hans Ewetz
| |
| Tim Haughton 2005-08-10, 9:03 am |
| > Must everything be BDUF or emergent design?
Of course not.
>Isn't it better not going to either extreme?
What makes you think that an extreme has to be bad?
--
Regards,
Tim Haughton
Agitek
http://agitek.co.uk
http://blogitek.com/timhaughton
| |
| hansewetz@hotmail.com 2005-08-10, 9:03 am |
| scottf3...@aol.com wrote:
> "Tim Haughton" <timhaughton@gmail.com> wrote in message
> news:<G1lKe.90971$vm2.32061
>
> Must everything be BDUF or emergent design? Isn't it better not going
> to either extreme?
Now, that's a novel idea!
I wish you good luck in pushing such an 'extreme' idea among XP
developers.
You don't have to push it on BDUF supporters because I don't believe
there are many of them - I'm not one of them even though a previous
sender assumed I was.
I believe you nailed the right question - I was a little too careless
when I framed my initial comment.
Regards,
Hans Ewetz
| |
| hansewetz@hotmail.com 2005-08-10, 5:03 pm |
| Tim Haughton wrote:
> <hansewetz@hotmail.com> wrote in message
> news:1123679808.906424.135960@z14g2000cwz.googlegroups.com...
> more
>
> OK, we must have been talking at cross purposes. When you said that one
> should avoid the use of refactoring, I had guessed that this was a decision
> based on economics. If it's *not* economics, what *does* steer you away from
> refactoring and emergent design? I have to admit that economics is my God.
Economics in terms of 'time to market' or economics in terms of 'pay
big penalties if we fail in production'?
>
> What kind of problems do you feel are best solved by the use of BDUF?
> Projects with unchanging requirements?
I don't think it's wise to lump an entire project into one single
technique - re-factoring or BDUF. Also, I have yet to see a project
with all requirements being stable.
However, there are, as I have mentioned in previous postings, stable
properties/requirements in any project that benefits from BDUF -
typically, properties that define the domain. Other
properties/requirements that are not stable clearly do not benefit from
a BDUF.
> Even if a system has unchanging requirements, that could potentially be a
> significant proportion of the project time spent designing without
> delivering anything. How do you mitigate this?
How do you mitigate for 'discovering' stable properties through a long
series of re-factoring exercises?
I can't see that there is a general answer to yours or mine question.
You have to judge based on your knowledge of the domain and past
experience. I have found that what you call BDUF works well for
properties that are judged as stable. Others may have different
experiences.
>
> I use emergent design by default because I'm not smart enough to create a
> design upfront that will still be valid when the code is complete. My love
> of economics also leads me to emergent design because I know that I produce
> software more efficiently using continuous design rather than upfront
> design.
I'm not smart enough or maybe not skilled enough to use code as a tool
for understanding the underlying nature of a domain. I need paper and
pen for such a task. Neither am I clever enough to ensure that
important properties of a domain are encoded correctly unless I have a
good understanding of the domain before coding.
Regards,
Hans Ewetz
| |
| Tim Haughton 2005-08-10, 5:03 pm |
| <hansewetz@hotmail.com> wrote in message
news:1123682682.352167.55310@g49g2000cwa.googlegroups.com...
> Economics in terms of 'time to market' or economics in terms of 'pay
> big penalties if we fail in production'?
>
I think economics is whatever the customer wants it to be. Hitting a
deadline, zero fault count, early RoI, managing risk etc..
[snip]
a[color=darkred]
>
> How do you mitigate for 'discovering' stable properties through a long
> series of re-factoring exercises?
When I talk about mitigation, I'm talking about making a bad thing less
serious. Spending a large amount of time designing, i.e. delaying RoI, is a
bad thing, wouldn't you agree?
Discovering a stable domain property isn't a bad thing, so I don't need to
mitigate it.
> I can't see that there is a general answer to yours or mine question.
> You have to judge based on your knowledge of the domain and past
> experience. I have found that what you call BDUF works well for
> properties that are judged as stable. Others may have different
> experiences.
[snip]
> I'm not smart enough or maybe not skilled enough to use code as a tool
> for understanding the underlying nature of a domain. I need paper and
> pen for such a task. Neither am I clever enough to ensure that
> important properties of a domain are encoded correctly unless I have a
> good understanding of the domain before coding.
So we're each coding to our own abilities. Happy days :¬) Pragmatic process
engineering.
--
Regards,
Tim Haughton
Agitek
http://agitek.co.uk
http://blogitek.com/timhaughton
| |
|
| scottf3095 wrote:
> Must everything be BDUF or emergent design? Isn't it better not going
> to either extreme?
That question assumes there's something wrong with emergent design.
The big obvious problem with BDUF is it produces over-engineered solutions,
with too many moving parts. These solutions notoriously resist unexpected
changes.
hansewetz wrote:
> Re-factoring is not the cause of the problem (of ending up in local
> minima) - it is a symptom. The cause is not having a good enough
> understanding of the problem domain.
Rework is a symptom of poor process. Refactoring is a small, safe, easy step
you do between adding lines of code. Many small refactors reduce the odds of
future refactors, and of future rework.
To avoid rework, do not speculate about design out to the limit of your
ability, implement your design, and then change the code beyond that limit.
Everyone can only plan so far, and speculating about a design without
proving it incrementally will increase the odds of rework.
> I don't like the idea of pushing re-factoring to the degree where
> even the smallest designs are 'evolved' through re-factoring
> when it is simple to sit down and think for an hour or two and
> get it right the first time.
The best big designs are generated incrementally, by repeatedly adding
features and repeatedly squeezing the code back down. This anneals and
laminates the design, like forging a sword. Think repeated steps of heating,
folding, and hammering. Swell the steel crystals up, stretch them, squeeze
them flat, and repeat thousands of times. Code developed like this strongly
resists bugs.
The BDUF direction on this spectrum typically generates a design, and then
debugs the behavior until it's correct. Emergent design generates behavior,
and then refactors under test until the design improves. No debugging.
You can't discuss emergent design in isolation from its effect on
bug-resistance.
> Clearly, the larger the design is the less likely this
> is to happen. Also, in any domain there are properties that are
> stable and do not change. It is important to identify such properties,
> and it can be done without using re-factoring. Everything is not
> variable - if it were, there would be no talk about domains.
That's one reason to implement features in order of business priority. The
most important code gets written first, and subsequent code supports it.
That gets you to local design quality optimization that supports the most
important stuff.
--
Phlip
[url]http://www.greencheese.org/ZLand[/url] <-- NOT a blog!!!
| |
| scott frye 2005-08-10, 5:03 pm |
|
"Tim Haughton" <timhaughton@gmail.com> wrote ...
>
> Of course not.
>
>
> What makes you think that an extreme has to be bad?
>
> --
> Regards,
>
> Tim Haughton
>
My point wasn't that an extreme is bad. Earlier in the conversation
when hansewetz made a negative comment about re-factoring (necessary
evil), you immediately assumed he must be for BDUF. I was trying to
point out the flaw in this logic. There are more than two techniques
here and many gradients of applying those techniques. BDUF is one
extreme and relying on emergent design exclusively is another.
Is an extreme bad? If generally applied my answer would have to be
yes. It has been already discussed that an emergent design might not
be the way to go if a non-locally optimized solution is desired. On
the flip side of the coin, we all know the dangers of trying to design
everything before writing a single line of code.
I think it is important that a software designer keep one eye on the
future and one eye on the present and design software accordingly.
There may be two equally good ways to do something in the present but
one lends itself to known future needs better that the other. (I
frequently have this situation myself). The Architect/Developer must
weight the cost of using the future looking technique against the
probability of the future becoming reality and factor in the cost of
being wrong. And there are many other variables that might effect the
decision.
This is more practical putting on blinders and making sweeping rules
such as NEVER program for something you don't need today or ALWAYS
design everything before coding. These should be expressed as
guidelines with suggestions to when they are most applicable.
-scott frye
| |
| hansewetz@hotmail.com 2005-08-10, 5:03 pm |
| Tim Haughton wrote:
> I think economics is whatever the customer wants it to be. Hitting a
> deadline, zero fault count, early RoI, managing risk etc..
Agree. However, if you pay non-negligible penalties when the software
fails in production, it may be important to know that the software has
certain properties. One way to increasing the chances of the software
having those proerties is to understand the domain before designing.
This topic has been discussed quite a lot in another threads related to
TDD so I won't get into it here.
I don't believe the notion of 'Economics' is a sharp enough concept to
use as a general tool for determining if 'emergent' design is more
economical than 'up-front design.
As a side point, even though you state (in my mind the pretty 
statement) that 'I have to admit that economics is my God', it is not
the only issue that affect IT development.
> When I talk about mitigation, I'm talking about making a bad thing less
> serious. Spending a large amount of time designing, i.e. delaying RoI, is=
a
> bad thing, wouldn't you agree?
You are not necessarily delaying delivery to production when you spent
a large amount of time designing. You may not deliver as often as you
would, had you used some other way of designing.
I can equally well say that you are delaying delivery because you have
used re-factoring as a method for 'discovering' certain properties in a
very complex domain. You may have been able to discover those
properties by thinking for a while before designing.
It seems as if you have made the assumption (correct me if I'm wrong)
that basing a design on the understanding of a domain is less efficient
time-wise than discovering it through re-factoring.
Just to be sure that there are no misunderstandings; I am not stating
that upfront design is 'better', I am simply stating that there is no
rule that can be applied in general to determine which way to develop
is 'better'.
> Discovering a stable domain property isn't a bad thing, so I don't need to
> mitigate it.
You do need to mitigate for lost time if you spend lots of time
'discovering' such properties by inefficient methods based on using
re-factoring as a discovery tool. I have seen it happen many times.
> So we're each coding to our own abilities. Happy days :=AC) Pragmatic pro=
cess
> engineering.
No, we are all 'developing' to our own abilities. Coding is simply one
out of many activities done during development.
Regards,
Hans Ewetz
| |
| hansewetz@hotmail.com 2005-08-10, 5:03 pm |
| Phlip wrote:
>
> hansewetz wrote:
>
>
> Rework is a symptom of poor process. Refactoring is a small, safe, easy step
> you do between adding lines of code. Many small refactors reduce the odds of
> future refactors, and of future rework.
Even though I did over-generalized in my initial comments (I FULLY
ADMIT THIS), I believe I have a point here. I do believe re-factoring
is a symptom of not understanding 'certain types' of problems. If, we
know that certain things will not change and if we can understand how
to map our understanding into a design (obviously without spending an
unreasonable amount of time and effort), I believe re-factoring is a
symptom of screwing up in the development process.
In general, coding and re-factoring is not a general purpose tool for
discovering things.
> To avoid rework, do not speculate about design out to the limit of your
> ability, implement your design, and then change the code beyond that limit.
> Everyone can only plan so far, and speculating about a design without
> proving it incrementally will increase the odds of rework.
Almost agree. The hard question is where this limit is. Also, you use
the word 'speculate' as if there was never the case when we can have a
high degree of certanty that the design is correct. I believe that in
some cases you can have a high degree of certainty that the design is
'right'.
> The best big designs are generated incrementally, by repeatedly adding
> features and repeatedly squeezing the code back down. This anneals and
> laminates the design, like forging a sword. Think repeated steps of heating,
> folding, and hammering. Swell the steel crystals up, stretch them, squeeze
> them flat, and repeat thousands of times. Code developed like this strongly
> resists bugs.
Sowftare development hardly ever comapares well with 'real world
tangeble stuff' :-)
> The BDUF direction on this spectrum typically generates a design, and then
> debugs the behavior until it's correct. Emergent design generates behavior,
> and then refactors under test until the design improves. No debugging.
The term 'emergent' implies that something appears that could not have
been predicted. My view is that in many cases it is possible to
'predict' before hand what the design should be - before coding it.
However, it is only possible with enough understanding of the domain
and of requirements.
Also, 'non-emergent' based design is not the same as BDUF. Calling
non-emergent-based' design BDUF would be like calling the design you
promote NDWE (No Design Whatsoever).
> You can't discuss emergent design in isolation from its effect on
> bug-resistance.
Sure you can. The logical correctness of a design is something
different than code bugs.
One problem with proponents of 'emergent' design techniques is that
they have a problem with analyzing the separation of concerns at the
level of the technique they are using. This is strange since they are
in general very good at separating unrelated concerns at the code level
:-)
> That's one reason to implement features in order of business priority. The
> most important code gets written first, and subsequent code supports it.
> That gets you to local design quality optimization that supports the most
> important stuff.
Hmmmm ... lets see. Using a smart pointer has low business priority -
actually it has 'no' business priority. If I sit down and think for an
hour I will realize that I will need smart pointers in the project. But
I won't do this since the most important business feature does not
require them. So, I implement the feature without using smart pointers.
Now, the next feature on the band wagon does require smart pointers so
I go ahead and code smart pointers and the new feature. But, now I
cannot compile because the use of smart pointers in feature 2 require
me to use smart pointers in feature 1. Aha ... I re-factor and the
world is again a stable place.
This is the extreme behavior of the methodology you propose. I hope
that this is not really how it works.
Regards,
Hans Ewetz
| |
| Ron Ruble 2005-08-10, 5:03 pm |
| hansewetz@hotmail.com wrote:
> Phlip wrote:
>
>
<snip>
[color=darkred]
> Even though I did over-generalized in my initial comments (I FULLY
> ADMIT THIS), I believe I have a point here. I do believe re-factoring
> is a symptom of not understanding 'certain types' of problems.
There are at least 2 reasons we may not understand the
problem sufficiently well to avoid refactoring:
-analysis which can more fully and accurately specify
the problem has not been done, due to error on the part
of the analysts. This seems to be what you are assuming
is the case.
-analysis which can more fully and accurately specify
the problem has not been done, for good and sufficient
reasons. This is what many XPers see.
Reasons analysis may not be that complete include:
-the problem is a moving target. This happens a -lot-
in banking and insurance, where we must modify
requirements based on changing legislation.
-the problem is completely new. Some code needs to do
t new things, which no one has done before. In
these cases, producing running code is often part
of the analysis process.
-the cost of analysis is known empirically, in this
case, to be greater than or approximately equal to
the cost of refactoring the emergent design.
> In general, coding and re-factoring is not a general purpose tool for
> discovering things.
Not general-purpose, certainly. But of frequent value.
>
> Almost agree. The hard question is where this limit is. Also, you use
> the word 'speculate' as if there was never the case when we can have a
> high degree of certanty that the design is correct. I believe that in
> some cases you can have a high degree of certainty that the design is
> 'right'.
Without doubt. The question is always determining which
cases these are.
>
> Softare development hardly ever compares well with 'real world
> tangible stuff' :-)
Doesn't it? I wonder. More and more, as I deal with people
outside the software arena, it seems like many of the
differences we see are illusory.
>
> The term 'emergent' implies that something appears that could not have
> been predicted.
"Could not have been predicted" is overbroad. Most people
here are not arguing that the design -could not- have been
specified up front in most cases. Most people are arguing
about whether it is more cost-effective; about whether the
results of such analysis are better than those that emerge
using XP.
>
> Hmmmm ... lets see. Using a smart pointer has low business priority -
> actually it has 'no' business priority. If I sit down and think for an
> hour I will realize that I will need smart pointers in the project. But
> I won't do this since the most important business feature does not
> require them. So, I implement the feature without using smart pointers.
> Now, the next feature on the band wagon does require smart pointers so
> I go ahead and code smart pointers and the new feature. But, now I
> cannot compile because the use of smart pointers in feature 2 require
> me to use smart pointers in feature 1. Aha ... I re-factor and the
> world is again a stable place.
>
> This is the extreme behavior of the methodology you propose.
> I hope that this is not really how it works.
Nope; it's extreme behavior of an -unbalanced subset-
of XP methodology. Smart pointers may be specified by
the programmer as necessary at a given time to produce
"the simplest thing that could possibly work".
It's always easy to show how a given XP practice,
-in isolation from- other XP practices, can return
incorrect results when taking to excess.
The Google archives are full of examples, from threads
in this newsgroup.
Thanks for your comments.
| |
| hansewetz@hotmail.com 2005-08-10, 5:03 pm |
| Ron Ruble wrote:
> hansewetz@hotmail.com wrote:
A few comments before I have to bail out.
>
> Not general-purpose, certainly. But of frequent value.
The problem is that it is often treated as a general tool for
discovery.
I have seen re-factoring being used as a discovery tool for
understanding how accounting works when reading a basic book on the
subject would have done the job.
>
> "Could not have been predicted" is overbroad. Most people
> here are not arguing that the design -could not- have been
> specified up front in most cases. Most people are arguing
> about whether it is more cost-effective; about whether the
> results of such analysis are better than those that emerge
> using XP.
That's why I said 'implies'. I do agree with you though - the question
is which way is the more cost-effective one (whatever 'cost effective'
means). Looking at other engineering endeavors, it seems as if
understanding is the preferred method. On the other hand, someone
mentioned in another thread that IT development is not engineering,
it's a craft.
I believe the key to efficient development is non-dogmatism and
understanding when and in what context some technique or methodology
works well. I hear too often that the XYZ technique is THE PREFERRED
method in IT development.
> It's always easy to show how a given XP practice,
> -in isolation from- other XP practices, can return
> incorrect results when taking to excess.
.... in the same way that it is easy to show how 'design up front' will
in the extreme be an inefficient way of developing software.
Regards,
Hans Ewetz
| |
| scottf3095@aol.com 2005-08-10, 5:03 pm |
|
"Phlip" wrote
> That question assumes there's something wrong with emergent design.
>
Or more accurately that emergent design has its advantages and
divantages. To my knowledge, no "perfect" design technique has been
invented yet. If it had, there would be no reason for this newsgroup
and all software development would have a 100% success rate.
-scott frye
| |
|
| hansewetz wrote:
> ...I do believe re-factoring
> is a symptom of not understanding 'certain types' of problems.
Absolutely. Leveraging the effect is better than repressing it.
> If, we
> know that certain things will not change and if we can understand how
> to map our understanding into a design (obviously without spending an
> unreasonable amount of time and effort), I believe re-factoring is a
> symptom of screwing up in the development process.
>
> In general, coding and re-factoring is not a general purpose tool for
> discovering things.
Combined with relentless testing, and sorting features in order of business
priority, it _is_ a general purpose discovery tool.
The attempt to avoid refactoring will produce code that's difficult to
change for unexpected reasons. Don't invest in your code the fear you won't
able to change it, or your fear will come true.
> Almost agree. The hard question is where this limit is. Also, you use
> the word 'speculate' as if there was never the case when we can have a
> high degree of certanty that the design is correct. I believe that in
> some cases you can have a high degree of certainty that the design is
> 'right'.
In which case, you would still use UML or a whiteboard to iterate a design.
You would start the model, imagine it expressing one use case, grow the
model, and so on.
That system refactors without the benefit of feedback from code, and the
benefit of forcing the code to change for each refactor.
> Hmmmm ... lets see. Using a smart pointer has low business priority -
> actually it has 'no' business priority.
All teams follow a base level of "community agreements" regarding coding
style. Don't test-first a stinkin' smart pointer.
> If I sit down and think for an
> hour I will realize that I will need smart pointers in the project.
Hopefully you won't need to model them in UML, right?
> But
> I won't do this since the most important business feature does not
> require them.
Yes it does; it requires clean robust code. If your language doesn't supply
smart handles, you must raise the floor a little. The same goes for default
copy operations, 'finally' blocks, private data members, logging files, etc.
> So, I implement the feature without using smart pointers.
> Now, the next feature on the band wagon does require smart pointers so
> I go ahead and code smart pointers and the new feature. But, now I
> cannot compile because the use of smart pointers in feature 2 require
> me to use smart pointers in feature 1. Aha ... I re-factor and the
> world is again a stable place.
>
> This is the extreme behavior of the methodology you propose. I hope
> that this is not really how it works.
We were supposed to discuss macro-design. The interaction of hundreds of
classes. At that level, the Once and Only Once principle decrees you should
limit the number of kinds of smart pointers they all use. So that level of
design is directly influenced by both business considerations and code
quality considerations.
--
Phlip
[url]http://www.greencheese.org/ZLand[/url] <-- NOT a blog!!!
| |
|
| hansewetz wrote:
> I have seen re-factoring being used as a discovery tool for
> understanding how accounting works when reading a basic book on the
> subject would have done the job.
You describe a rework session. Refactoring does not happen in isolation from
the other practices, such as analysis to produce acceptance tests.
>
> ... in the same way that it is easy to show how 'design up front' will
> in the extreme be an inefficient way of developing software.
The studies cited in the book /Agile & Iterative Development: A Manager's
Guide/ by Craig Larman show an amazing statistic. Among the shops attempting
pure Waterfall (don't deny it happens), the ones that tried the hardest to
work in perfectly strict phases had the lowest success rate. The ones that
slacked off and re-worked, a little, had the second-worst rate.
--
Phlip
[url]http://www.greencheese.org/ZLand[/url] <-- NOT a blog!!!
| |
|
| scottf3095 wrote:
>
> Or more accurately that emergent design has its advantages and
> divantages. To my knowledge, no "perfect" design technique has been
> invented yet. If it had, there would be no reason for this newsgroup
> and all software development would have a 100% success rate.
Research the defect rate for XP shops. No bugbase. One defect per quarter.
Etc.
It's not about a perfect design, guys. It's about rapid turnaround with zero
bugs!
--
Phlip
[url]http://www.greencheese.org/ZLand[/url] <-- NOT a blog!!!
| |
| hansewetz@hotmail.com 2005-08-10, 5:03 pm |
| Phlip wrote:
> hansewetz wrote:
>
> Combined with relentless testing, and sorting features in order of business
> priority, it _is_ a general purpose discovery tool.
Hmmmm ... I don't see many philosophers, researchers in empirical
sciences, mathematicians and others using coding augmented with
re-factoring as their main tool for conducting research. After all,
research is about discovering things :-)
Regards,
Hans Ewetz
| |
| hansewetz@hotmail.com 2005-08-10, 5:03 pm |
| Phlip wrote:
> The studies cited in the book /Agile & Iterative Development: A Manager's
> Guide/ by Craig Larman show an amazing statistic. Among the shops attempting
> pure Waterfall (don't deny it happens), the ones that tried the hardest to
> work in perfectly strict phases had the lowest success rate. The ones that
> slacked off and re-worked, a little, had the second-worst rate.
I have never proposed that a waterfall process will be effective ... so
I'm not sure why your comment is here.
But, let's also take a look at reality - anyone believing that a book
called 'Agile & Iterative Development' will be neutral would be a fool
- and based on your previous postings I believe you are very far from
being one :-)
Regards,
Hans Ewetz
| |
|
| hansewetz wrote:
> Phlip wrote:
>
>
> Hmmmm ... I don't see many philosophers, researchers in empirical
> sciences, mathematicians and others using coding augmented with
> re-factoring as their main tool for conducting research. After all,
> research is about discovering things :-)
E = mc^2 was the result of generating a big formula and then refactoring it,
right?
At its heart, refactoring is just algebraic substitutions to normalize
expressions...
--
Phlip
[url]http://www.greencheese.org/ZLand[/url] <-- NOT a blog!!!
| |
|
| hansewetz wrote:
> Phlip wrote:
>
>
> I have never proposed that a waterfall process will be effective ... so
> I'm not sure why your comment is here.
The bigger your up-front design effort, the higher your risks of failure.
At the other end of the scale, the smaller your increments, the higher the
risks of success. An increment cannot be _small_ unless it _forbids_
speculative designing beyond the current iteration.
> But, let's also take a look at reality - anyone believing that a book
> called 'Agile & Iterative Development' will be neutral would be a fool
> - and based on your previous postings I believe you are very far from
> being one :-)
"the studies cited in the book..." Argue with them.
--
Phlip
[url]http://www.greencheese.org/ZLand[/url] <-- NOT a blog!!!
| |
| hansewetz@hotmail.com 2005-08-10, 5:03 pm |
| Phlip wrote:
> hansewetz wrote:
>
> E = mc^2 was the result of generating a big formula and then refactoring it,
> right?
First, I said 'coding augmented by refactoring' - you are stating
something different.
Second, there is a big difference between empirical sciences and
designing IT systems. In empirical sciences we are trying to discover
what the stuff out there is, how it behaves etc. We are not building
the system - something built it for us. In IT development we are the
ones building the system. Since we are building the system we have
certain advantages. In theory we can design everything upfront (notice
I say in theory). We can also, in theory, predict how our system will
behave without conducting experiments, i.e. execution tests. In
empirical sciences it is not even in theory possible to determine how
the world works without conducting experiments.
What I am saying is that we should use the fact that we are the ones
building the system to our advantage as much as possible within
practical limits. My impression is that you are applying the same type
of methods as is done by scientists in the empirical fields. If this is
true I disagree with your development method.
Regards,
Hasn Ewetz
| |
| hansewetz@hotmail.com 2005-08-10, 5:03 pm |
| Phlip wrote:
> hansewetz wrote:
>
> The bigger your up-front design effort, the higher your risks of failure.
Not always true. If you understand relationships among some domain
objects and base a design on that you will not incur a higher risk than
if you don't have the same understanding and start 'evolving' your
design from a very low level.
You are confusing 'bugs', i.e. mistakes in the code, with logical flaws
in your design.
> At the other end of the scale, the smaller your increments, the higher the
> risks of success. An increment cannot be _small_ unless it _forbids_
> speculative designing beyond the current iteration.
Again, not always true. See previous comment.
> "the studies cited in the book..." Argue with them.
I know better than to do that! I'm just highly skeptical to
non-sceintific literature that proposes ABC and then shows statistics
that ABC is much better' than the complement of XYZ. Especially in the
IT world.
Regards,
Hans Ewetz
| |
| Laurent Bossavit 2005-08-10, 10:01 pm |
| Hans,
> I have seen re-factoring being used as a discovery tool for
> understanding how accounting works when reading a basic book
> on the subject would have done the job.
And that was bad because ?..
(Contrast: "I have seen people read books to discover how accounting
works, when refactoring some fundamental test cases would have done the
job." Unless you meant to point out that different people have different
ways of learning, something is missing from your argument.)
Laurent
| |
| Laurent Bossavit 2005-08-10, 10:01 pm |
| Hans,
> You are confusing 'bugs', i.e. mistakes in the code, with logical flaws
> in your design.
I don't see where Phlip wrote something that suggests he's confusing the
two; could you please point it out ?
> Not always true. If you understand relationships among some domain
> objects and base a design on that you will not incur a higher risk than
> if you don't have the same understanding and start 'evolving' your
> design from a very low level.
I think the "risk" Phlip is concerned about is that you're only *fooling
yourself* that you understand the relationships, and you base a design
on that flawed understanding.
Say you take on a big part of the domain, attempt to understand that,
take a wrong turning somewhere in your understanding, and base a design
on *that*; what's going to reveal your error ? The bigger the domain
stack, the more hay for your needle to get lost in.
But say you focus on a small part of the domain, consequently base a
small part of the design on that understanding, write tests that stress
that design, then implement it - if you're fooling yourself, I'd expect
it will become apparent somewhere along that process.
Refactoring arises naturally as a consequence of iterating on that
process: as you move on to the next small domain chunk, some design
resulting from the first chunk will need rearranging. (Whenever you buy
new furniture, you move old furniture around to balance the whole - you
don't call that "rework", and position every table in its ultimate
position the day you buy it.)
Laurent
| |
| Rich MacDonald 2005-08-11, 4:00 am |
| hansewetz@hotmail.com wrote in news:1123665833.910883.16980
@g14g2000cwa.googlegroups.com:
> Re-factoring is not the cause of the problem (of ending up in local
> minima) - it is a symptom. The cause is not having a good enough
> understanding of the problem domain.
>
> Obviously I'm exaggerating a little. However, re-factoring is
> something that is seen as something 'normal' by many developers. In
> my mind, re-factoring should be avoided as much as possible. The way to
> avoid it is to understand as much as possible before doing.
I must disagree in the most vehement, yet polite, of terms. Yes this
would be true in a perfect world. But we haven't solved world hunger
either :-)
Just today I refactored some code I'd refactored last w. Last w I
reversed the duplication of some cut-and-paste code. I had no
understanding of the domain, I simply eliminated the duplication by
creating a common superclass and shoving everything up that I could.
Today I realized the domain drivers for that duplication, so I was able
to create a new class hierarchy and make sense of that duplication. In
one sense that work I did last w was wasted since I undid it all
today. In another sense, last w's work helped removed the cobwebs to
allow today's work. So yes, you are correct, understanding as much as
possible before doing would have eliminated last w's work. But in
another sense I'm probably further along today because I did, rather than
sit back and ponder. And if I'm not further along I'm a close second.
Of course you are correct in that a good understanding of the problem
domain is required in order to avoid the local minima. No dispute here.
My dispute is to suggest that refactoring should be the exception not the
rule. In my daily work, refactoring is the key driver and its key to my
productivity. Even when I understand the domain beforehand I've still yet
to encounter the time when I get it exactly right the first time. So
rather than wait, I do then improve. And I encourage other programmers to
try it as well. I see far more problems caused by a lack of refactoring
than I do caused by a lack of domain understanding.
| |
| Rich MacDonald 2005-08-11, 4:00 am |
| hansewetz@hotmail.com wrote in news:1123682682.352167.55310
@g49g2000cwa.googlegroups.com:
> I'm not smart enough or maybe not skilled enough to use code as a tool
> for understanding the underlying nature of a domain. I need paper and
> pen for such a task. Neither am I clever enough to ensure that
> important properties of a domain are encoded correctly unless I have a
> good understanding of the domain before coding.
Forgive my vulgarity but I call bullshit. You write well, you think well, I
bet you can.
I used to be unable to code OO until I'd drawn with paper and pencil.
Unfortunately, I wasn't very good with paper and pencil either and my final
code looked nothing like my original schematic. There was always some
discovery during coding that sent me in a different direction. After
repeating the experiment over and over again with the same results, I
eventually learned how to use pencil and paper (whiteboard, actually) and
keyboard simultaneously. Now I code when I know and I whiteboard when I
need to sketch the next step(s), then I code with an eye on the whiteboard.
Generally its the code evolution that sends me back to the whiteboard, not
vice-versa.
Its a skill that can be learned.
Now if the domain is big enough, then I do work to keep the whiteboard up-
to-date (actually I switch to UML Visio). But all I do is print the
diagrams and put them on the wall for quick reference. Once again, the code
drives the diagrams, not vice-versa.
| |
| Laurent Bossavit 2005-08-11, 4:00 am |
| Rich,
> Forgive my vulgarity but I call bullshit. You write well, you think well, I
> bet you can.
It's never a good idea to wager about something that is entirely under
the other player's control.
Also, you're appealing to the wrong kind of motive. That is one reason
why managerly games concerning task estimates are bound to be counter-
productive. A manager to negotiate a task estimates is saying: "You're
telling me this will take you two days. Hah! I bet you're smart enough
to do it in one."
The manager is appealing to emotion, not rationality; and creating a
double bind situation. If you bring in the task in one day, you've won
the cleverness contest, but you've lost the negotiation bet. If you take
two days or longer, you've won the bet but you've lost the race. Either
way you look at it you lose.
The better bet is that whatever Hans tells you about his cognitive
capabilities is true, until and unless disproved by his voluntary
experiment. And expect the reciprocal courtesy.
Laurent
| |
| hansewetz@hotmail.com 2005-08-11, 4:00 am |
| Laurent Bossavit wrote:
> Hans,
>
>
> And that was bad because ?..
Hmmm ... rediscovering the wheel comes to mind ...
Regards,
Hans Ewetz
| |
| hansewetz@hotmail.com 2005-08-11, 4:00 am |
| Laurent Bossavit wrote:
> Hans,
>
>
> I don't see where Phlip wrote something that suggests he's confusing the
> two; could you please point it out ?
No I can't - I might have made an incorrect assumption.
It does happen quite often in these newsgroups - but few admit it.
I'll go back and check it later ... don't have the time right now.
>
> I think the "risk" Phlip is concerned about is that you're only *fooling
> yourself* that you understand the relationships, and you base a design
> on that flawed understanding.
That is always a possibility.
I also believe that there is a risk of fooling oneself if one believes
that using coding augmented with re-factoring is a reliable tool for
discovering fundamental properties in a domain or a business. I also
believe a similar statement holds for design.
> Say you take on a big part of the domain, attempt to understand that,
> take a wrong turning somewhere in your understanding, and base a design
> on *that*; what's going to reveal your error ? The bigger the domain
> stack, the more hay for your needle to get lost in.
That's a risk - in the same way that relying on re-factoring for
getting to a design that is flexible is also a risk. Analogous to
optimization in math, you could end up in a local minimum which is very
difficult to find your way out of. However, understanding before doing
has yielded pretty good results in engineering..
> But say you focus on a small part of the domain, consequently base a
> small part of the design on that understanding, write tests that stress
> that design, then implement it - if you're fooling yourself, I'd expect
> it will become apparent somewhere along that process.
See previous comment.
> Refactoring arises naturally as a consequence of iterating on that
> process: as you move on to the next small domain chunk, some design
> resulting from the first chunk will need rearranging.
The question here is really if you are picking chunks that are suitable
for re-factoring or, if you picking small chunks and re-factoring is a
natural way of processing them. This is not really clear.
Another issue is if picking small chunks and re-factoring them in an
iterative fashion is 'always' better than understanding some, maybe
larger chunk, and avoid re-factoring. Here I don't think there is a
single answer that aligns with all types of problems. I get the
impression that you (and Phlip) believe that there is one single answer
and that the answer is the re-factoring alternative.
> (Whenever you buy
> new furniture, you move old furniture around to balance the whole - you
> don't call that "rework", and position every table in its ultimate
> position the day you buy it.)
The coupling between the parts in software is of a different nature
than the one in your furniture analogy.
Regards,
Hans Ewetz
| |
| hansewetz@hotmail.com 2005-08-11, 4:00 am |
| Rich MacDonald wrote:
> hansewetz@hotmail.com wrote in news:1123665833.910883.16980
> @g14g2000cwa.googlegroups.com:
>
>
> I must disagree in the most vehement, yet polite, of terms. Yes this
> would be true in a perfect world. But we haven't solved world hunger
> either :-)
Check the previous comments in this thread. I have ADNMITTED that I did
not express myself very clearly and that I did exaggerate - typed too
fast and thought to little. I'll issue a suitable punishment for myself
:-)
Also, noticed that I did say 'avoided as much as possible'. I believe
that understanding before doing is always better than doing without
understanding. However, this is not always feasible.
It is interesting to notice though that some postings from others view
re-factoring as a primary way of understanding fundamental properties
in a domain or a business. This I do not agree with.
Regards,
Hans Ewetz
| |
| Tim Haughton 2005-08-11, 4:00 am |
| <hansewetz@hotmail.com> wrote in message
news:1123688530.704133.105160@f14g2000cwb.googlegroups.com...
> Agree. However, if you pay non-negligible penalties when the software
> fails in production, it may be important to know that the software has
> certain properties. One way to increasing the chances of the software
> having those proerties is to understand the domain before designing.
I think we can all agree that having a software product fail in production
is a bad thing. That's why I choose the methodology which has been shown to
deliver the lowest bug rate. I don't dispute that understanding the domain
is a good thing, but if we're developing iteratively, and if the customer is
deciding in which order functionality should appear, do I need to understand
all of the domain at the start of the project, or just enough to satisfy the
acceptance tests for that iteration?
> I don't believe the notion of 'Economics' is a sharp enough concept to
> use as a general tool for determining if 'emergent' design is more
> economical than 'up-front design.
If economics isn't the best tool for determining whether something is
economical, I'm stuck.
> As a side point, even though you state (in my mind the pretty
> statement) that 'I have to admit that economics is my God', it is not
> the only issue that affect IT development.
Of course, my statement was tongue in ch, but it emphasises a point:
developing software is a business endeavour, and we should develop software
in a manner that best serves the business.
> You are not necessarily delaying delivery to production when you spent
> a large amount of time designing. You may not deliver as often as you
> would, had you used some other way of designing.
These two sentences seem to be contradictory, I'll let you elaborate.
> I can equally well say that you are delaying delivery because you have
> used re-factoring as a method for 'discovering' certain properties in a
> very complex domain. You may have been able to discover those
> properties by thinking for a while before designing.
I don't think you *can* say that. Agile methods give earlier RoI than
non-Agile methods, if you're disputing that fact, we can discuss it. Like I
said earlier, you don't need to understand the whole problem domain before
undertaking an iteration, you need to understand just enough for that
iteration. The output of that iteration is defined in advance using
acceptance tests.
> It seems as if you have made the assumption (correct me if I'm wrong)
> that basing a design on the understanding of a domain is less efficient
> time-wise than discovering it through re-factoring.
I think I'm going to correct you. I assume (read: have learned though
experience) that it is more efficient to base the *behaviour* of the code on
the understanding of the domain. I assume (read: have learned though
experience) that it is more efficient to crystallise the understanding of
the domain in the acceptance tests rather than the design. I assume (read:
have learned though experience) that it is more efficient to let a design
grow organically through ruthless refactoring to support *current*
functionality rather than designing upfront.
> Just to be sure that there are no misunderstandings; I am not stating
> that upfront design is 'better', I am simply stating that there is no
> rule that can be applied in general to determine which way to develop
> is 'better'.
I'm prepared to accept that. However, given what I know of Agile
development, it would always be my starting point, and I would need to find
a good reason to depart from it.
> You do need to mitigate for lost time if you spend lots of time
> 'discovering' such properties by inefficient methods based on using
> re-factoring as a discovery tool. I have seen it happen many times.
Customer collaboration is a discovery tool. Co-creation of acceptance tests
is a discovery tool. Refactoring tells you more about the *functionality*
and design of your code than it does the problem domain.
Your job as a developer is usually to produce software with a certain
functionality, not a certain logical design.
--
Regards,
Tim Haughton
Agitek
http://agitek.co.uk
http://blogitek.com/timhaughton
| |
| hansewetz@hotmail.com 2005-08-11, 4:00 am |
| Rich MacDonald wrote:
> hansewetz@hotmail.com wrote in news:1123682682.352167.55310
> @g49g2000cwa.googlegroups.com:
>
>
> Forgive my vulgarity but I call bullshit. You write well, you think well, I
> bet you can.
Thanks!
> I used to be unable to code OO until I'd drawn with paper and pencil.
> Unfortunately, I wasn't very good with paper and pencil either and my final
> code looked nothing like my original schematic. There was always some
> discovery during coding that sent me in a different direction. After
> repeating the experiment over and over again with the same results, I
> eventually learned how to use pencil and paper (whiteboard, actually) and
> keyboard simultaneously. Now I code when I know and I whiteboard when I
> need to sketch the next step(s), then I code with an eye on the whiteboard.
> Generally its the code evolution that sends me back to the whiteboard, not
> vice-versa.
I have no problem 'coding OO' without drawing a design first. It's just
a very inefficient way of developing software once the software is
non-trivial - in my experience. I have found that understanding the
fundamentals of domain/business before touching the keyboard saves me a
lot of time.
Both you and I can most likely come up with compelling support for our
opinions. However, there will always be some assumptions or objectives
that differ. I believe that the key in resolving these types of
discussions is to make the objectives and assumptions clear before
hand.
> Its a skill that can be learned.
That's not the issue. I can also learn how to slice bread with a dollar
bill. The issue is if your development technique is more or less
effective than some other technique.
> Now if the domain is big enough, then I do work to keep the whiteboard up-
> to-date (actually I switch to UML Visio). But all I do is print the
> diagrams and put them on the wall for quick reference. Once again, the code
> drives the diagrams, not vice-versa.
If it works for you that's good. I let my models drive my code - it
works for me.
Regards,
Hans Ewetz
| |
| hansewetz@hotmail.com 2005-08-11, 9:01 am |
| Tim Haughton wrote:
> <hansewetz@hotmail.com> wrote in message
>
> If economics isn't the best tool for determining whether something is
> economical, I'm stuck.
Sorry, I did not express it correctly. I should have said:
I believe Economics can be used as a measure after the fact. I can't
see that you can use economics as a general tool that can be used
beforehand to determine which methodology or technique that will yield
the best results measured in economic terms. It is not a sharp eneough
tool.
>
> Of course, my statement was tongue in ch, but it emphasises a point:
> developing software is a business endeavour, and we should develop software
> in a manner that best serves the business.
I wasn't sure ... some people are serious about it. Anyway, that issue
belongs in a different type of newsgroup. Maybe the (one o | | |