Home > Archive > Software Engineering > September 2006 > In trouble, sincerely ask for kind advice.
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
In trouble, sincerely ask for kind advice.
|
|
| shuisheng 2006-09-04, 6:59 pm |
| Dear All,
We meet some problem in our project development and sincerely ask for
your kind advice.
We took eight months to develop a project system with about 50,000
lines of code. The code is integrated into our system without careful
testing. So after all the integration, we found there are many bugs.
And I believe if we test more, more bugs will appear. Some bugs lead to
the computer to crash. And they are very hard to find and remove. For
example, we took a whole w on a particular bug and still no
significant progress. And also I am not satisfied with the design and
think the project will be hard to maintain.
We have no experience on a big project. What should we do about the
project? Redo it or debug it or others? What is the most efficient way
on our situation?
I appreicate your kind help!!!
Shuisheng.
| |
| James Bond 007 2006-09-04, 6:59 pm |
|
"shuisheng" <shuisheng75@yahoo.com> wrote in message news:1157387787.556296.258620@m73g2000cwd.googlegroups.com...
> Dear All,
>
> We meet some problem in our project development and sincerely ask for
> your kind advice.
>
> We took eight months to develop a project system with about 50,000
> lines of code. The code is integrated into our system without careful
> testing. So after all the integration, we found there are many bugs.
> And I believe if we test more, more bugs will appear. Some bugs lead to
> the computer to crash. And they are very hard to find and remove. For
> example, we took a whole w on a particular bug and still no
> significant progress. And also I am not satisfied with the design and
> think the project will be hard to maintain.
>
> We have no experience on a big project. What should we do about the
> project? Redo it or debug it or others? What is the most efficient way
> on our situation?
>
> I appreicate your kind help!!!
>
> Shuisheng.
It depends on the importance of the project and any customer commitments or schedules that you are trying to meet. If you have the time, it might be best to start over again and do it the right way. If time is a constraint, bring in some outside engineers who know how to tackle a problem and debug software.
A system with 50,000 lines of code is not really a big project. But regardless of the size of a project, it needs to be done using proper software development methodology. Do you follow a structured software engineering process? Documented requirements, design, test cases, test plan. It sounds like you may not even have these things. It's almost impossible to produce a quality software project if there is no quality process being followed. In addition, the fact that you spent a whole w trying to correct one bug and have failed to do so tells me that your staff is poorly skilled and unorganized. They obviously don't have a clue on how to debug something or perform a structured problem analysis and decomposition. Before you redo anything, get your people the training they need or replace them with engineers who have the training and a proven track record of success, and put the right quality processes in place. And make sure that the people doing the testing are not the same ones who did the design or wrote the code.
| |
| H. S. Lahman 2006-09-04, 6:59 pm |
| Responding to Shuisheng...
> Dear All,
>
> We meet some problem in our project development and sincerely ask for
> your kind advice.
>
> We took eight months to develop a project system with about 50,000
> lines of code. The code is integrated into our system without careful
> testing. So after all the integration, we found there are many bugs.
> And I believe if we test more, more bugs will appear. Some bugs lead to
> the computer to crash. And they are very hard to find and remove. For
> example, we took a whole w on a particular bug and still no
> significant progress. And also I am not satisfied with the design and
> think the project will be hard to maintain.
>
> We have no experience on a big project. What should we do about the
> project? Redo it or debug it or others? What is the most efficient way
> on our situation?
Alas, 50 KLOC is a pretty small project by today's standards. Projects
aren't large until they have about 100 times as much code.
The big question is: are there so many bugs because the development
process was poor or simply because there wasn't any testing before
integration? I assume that the rest of the system doesn't have as many
bugs at integration time. If so,...
case 1: this code was developed by the same group or a group using the
same process as the rest of the system. In that case, the difference in
bugs probably is due purely to insufficient testing. IOW, if this code
had been tested to the same level as the rest of the system before
integration, it would probably have had about the same number of bugs
during integration. In this situation you should have the authors go
back an test it to the same level as the rest of the system and then try
to integrate it again.
case 2: this code was developed by a different group using a different
development process than the rest of the system. In this situation it
is possible they just did a very bad job and no amount of testing is
going to fix that. In this situation one should probably throw it out
and start over using the process used on the rest of the system.
Unfortunately there is strong empirical evidence that says that the
number of escapes to the field is directly proportional to the number of
defects found in the last round of testing. It is also well-established
that integration (system) testing typically has relatively poor fault
coverage. Therefore, even if case (1) applies, you should not attempt
to use just integration testing to eliminate the bugs. Testing should
go back to proper unit and subsystem testing before attempting to
evaluate integration tests.
*************
There is nothing wrong with me that could
not be cured by a capful of Drano.
H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
"Model-Based Translation: The Next Step in Agile Development". Email
info@pathfindermda.com for your copy.
Pathfinder is hiring:
http://www.pathfindermda.com/about_us/careers_pos3.php.
(888)OOA-PATH
| |
|
| shuisheng wrote:
> We meet some problem in our project development and sincerely ask for
> your kind advice.
Your team has used development practices that generally don't work. However,
many of the books you read will seem to tell you to develop like this. (And
quite a few will actually say it.) You need to learn better development
practices.
> We took eight months to develop a project system with about 50,000
> lines of code.
Next time, determine the smallest set of high-value features you can
possibly deliver, and deliver these, long before 8 months.
> The code is integrated into our system without careful
> testing.
Next time, write unit tests at the same time as you write the tested code.
All developers should do this, because it makes development very fast. Done
right, your tests can resist bugs before they happen.
> So after all the integration, we found there are many bugs.
Read /Working Effectively with Legacy Code/ by Mike Feathers. Then add a
unit test module to your source code. If you use Java, get JUnit, if C# get
NUnit or CsUnit, etc.
> And I believe if we test more, more bugs will appear. Some bugs lead to
> the computer to crash. And they are very hard to find and remove.
This tells me your design is poor. Read the book /Refactoring/ to learn to
improve the design of code you already wrote.
Next, you will find that as you fix bugs, you will cause more bugs. You
_must_ add unit tests, to reduce the chances this happens.
--
Phlip
[url]http://c2.com/cgi/wiki?ZLand[/url] <-- NOT a blog!!!
| |
|
| H. S. Lahman wrote:
> Therefore, even if case (1) applies, you should not attempt
> to use just integration testing to eliminate the bugs. Testing should
> go back to proper unit and subsystem testing before attempting to
> evaluate integration tests.
H.S. Lahman just also told you to add JUnit or NUnit, and then write unit
test cases for everything. (I'm not sure why that wasn't clearer here...)
James Bond 007 told you to document your system before writing it. That
might help, but it won't fix or prevent bugs. And he did not mention unit
tests. They are the most important part of development, and someday every
programming book will discuss them. The quality of your team, and their
power to debug, is less important than their stamina to write lots of simple
unit test cases!
--
Phlip
[url]http://c2.com/cgi/wiki?ZLand[/url] <-- NOT a blog!!!
| |
|
| H. S. Lahman wrote:
> ...Therefore, even if case (1) applies, you should not attempt
> to use just integration testing to eliminate the bugs. Testing should
> go back to proper unit and subsystem testing before attempting to
> evaluate integration tests.
Another issue. I don't think the OP has "integration tests", because I think
they used "integrate" to mean "put online with the rest of the users' code".
--
Phlip
[url]http://c2.com/cgi/wiki?ZLand[/url] <-- NOT a blog!!!
| |
| JXStern 2006-09-04, 6:59 pm |
| On 4 Sep 2006 09:36:27 -0700, "shuisheng" <shuisheng75@yahoo.com>
wrote:
>Dear All,
Hello.
>We meet some problem in our project development and sincerely ask for
>your kind advice.
OK.
Late thought it seems.
>We took eight months to develop a project system with about 50,000
>lines of code.
Out of curiosity, how many developers were involved?
Let me guess: about 3 fulltime, and a couple of others helping?
Students on a summer project maybe?
> The code is integrated into our system without careful
>testing.
Aha. And what did you expect would happen then?
> So after all the integration, we found there are many bugs.
No surprise.
>And I believe if we test more, more bugs will appear.
Generally true, even for the best of code.
> Some bugs lead to
>the computer to crash.
Very bad.
What kind of application is this, some kind of real-time control, not
just a little business application?
> And they are very hard to find and remove.
Not too surprising.
> For
>example, we took a whole w on a particular bug and still no
>significant progress.
If this is a real-time system, this is not surprising, but perhaps you
need better tools, certainly you need better management.
>And also I am not satisfied with the design and
>think the project will be hard to maintain.
That would be unfortunate, but of course how can we judge from here?
>We have no experience on a big project.
Well, you do now.
(is 50k lines "big"? well, let's just say things get even tougher as
the size goes to 500k or 5m LOC)
> What should we do about the
>project? Redo it or debug it or others? What is the most efficient way
>on our situation?
Whatever you do will probably be a major effort.
Whether your developers are given a chance to learn better processes,
then go back and apply them, or else you go out and find an
experienced manager and developers, who can look at the system and
clean it up - but who first have to learn all about your application -
either way is going to be slow and expensive.
If you or your company don't have the resources, good news - you can
give up now!
>
>I appreicate your kind help!!!
>
>Shuisheng.
Jiaoshi
| |
| xpyttl 2006-09-04, 9:59 pm |
| Shuisheng
You have gotten a lot of advice here, most of it good, some of it useful.
It should be clear that you are not going to wish yourself out of this
problem.
The first thing you need to do is get new management. If you cannot do
that, then Jiaoshi's advice of giving up -- that is, bail out and go
somewhere else, is the best advice you have gotten. If you can get a
software manager who has been able to get real software out the door
successfully, the first thing he will demand is to get someone from outside
to come in and asses the damage.
Probably starting over isn't the answer, but it looks as if you don't have
the skills on your team to be able to tell whether it is or not. A lot of
folks around here are zealots around various flavors of so-called agile
methods, especially, XP, and their knee-jerk reaction to everything is to
test. Now, you cannot get away without testing, and testing is a formal
discipline at least as challenging as development. These guys don't mean
have a few foks randomly bang and the keyboard and see if it works. Real
testing takes a lot of thought and planning by people who know what they are
doing.
However much the zealots wish it to be otherwise, you cannot test quality
into a product. Not a car, not software, not any product. There are
techniques for eliminating bugs that are many times more effective than
testing, but they require skilled practitioners that are trained. In other
words, once again, you need some help from people who know what they are
doing.
Oh, and surprise, surprise, people like that don't come cheap. If you
actually want results, be prepared to swallow hard when you see the bill.
Anyone can call themselves a consultant, but consultants that can actually
get results can command astonishing fees.
...
| |
| Paul E. Black 2006-09-05, 7:00 pm |
| On Monday 04 September 2006 12:36, shuisheng wrote:
> We took eight months to develop a project system with about 50,000
> lines of code. The code is integrated into our system without careful
> testing. So after all the integration, we found there are many bugs.
>
> ...
>
> ... What should we do about the
> project? Redo it or debug it or others?
What kind of bugs are you finding? If they are requirements bugs
("Oh, I didn't realize overdue customers should be charged interest at
the start of every month"), keep debugging: you are learning what the
system should be. This is really a form of requirements capture.
If the bugs are code errors, it sounds like troublesome modules (or
even the whole thing) should be rewriten. Or at least fall back to
unit and subsystem testing, as others advised.
-paul-
--
Paul E. Black (p.black@acm.org)
| |
|
| Paul E. Black wrote:
> If the bugs are code errors, it sounds like troublesome modules (or
> even the whole thing) should be rewriten. Or at least fall back to
> unit and subsystem testing, as others advised.
You XP Zealot! Don't you know that automated tests are never a panacea or
silver bullet?!
;-)
--
Phlip
[url]http://c2.com/cgi/wiki?ZLand[/url] <-- NOT a blog!!!
| |
| Gabriel Claramunt 2006-09-07, 7:59 am |
|
"Phlip" <phlipcpp@yahoo.com> wrote in message
news:_zZKg.20792$kO3.6744@newssvr12.news.prodigy.com...
....
> The quality of your team, and their power to debug, is less important than
> their stamina to write lots of simple unit test cases!
Hey! That is against most agile methodologies!!
Agile or non agile, IMHO the quality of the team is a LOT more important and
decicive for the project than writing lots of simple unit test cases!
I will say that is the most significant factor to succeed in a SW
development project.
But YMMV...
| |
|
| Gabriel Claramunt wrote:
[color=darkred]
> Hey! That is against most agile methodologies!!
It's against all of them. But it's just a rhetorical flourish at the end of
yet another paragraph declaring unit tests to be the most important aspect
of coding.
> Agile or non agile, IMHO the quality of the team is a LOT more important
> and decicive for the project than writing lots of simple unit test cases!
Indeed; the sentence works better if we transpose the words 'stamina' and
'quality'.
> I will say that is the most significant factor to succeed in a SW
> development project.
> But YMMV...
I personally would pick a team of juniors using TDD over a high-quality team
not using it...
--
Phlip
[url]http://c2.com/cgi/wiki?ZLand[/url] <-- NOT a blog!!!
| |
| Gabriel Claramunt 2006-09-08, 4:00 am |
| I liked the discussion of Cockburn:
http://alistair.cockburn.us/index.p...elopment
Anyway, "high quality team" is a good excuse for any process: a high quality
team will suceed in spite of the process.
(Althogh a good process will enable a good team perform at their best)
Saludos
Gabriel
"Phlip" <phlipcpp@yahoo.com> wrote in message
news:fKULg.9473$q63.7043@newssvr13.news.prodigy.com...
> Gabriel Claramunt wrote:
>
>
>
> It's against all of them. But it's just a rhetorical flourish at the end
> of yet another paragraph declaring unit tests to be the most important
> aspect of coding.
>
>
> Indeed; the sentence works better if we transpose the words 'stamina' and
> 'quality'.
>
>
> I personally would pick a team of juniors using TDD over a high-quality
> team not using it...
>
> --
> Phlip
> [url]http://c2.com/cgi/wiki?ZLand[/url] <-- NOT a blog!!!
>
| |
| Ron Ruble 2006-09-09, 10:00 pm |
| shuisheng wrote:
> Dear All,
>
> We meet some problem in our project development and sincerely ask for
> your kind advice.
>
> We took eight months to develop a project system with about 50,000
> lines of code. The code is integrated into our system without careful
> testing. So after all the integration, we found there are many bugs.
> And I believe if we test more, more bugs will appear. Some bugs lead to
> the computer to crash. And they are very hard to find and remove. For
> example, we took a whole w on a particular bug and still no
> significant progress. And also I am not satisfied with the design and
> think the project will be hard to maintain.
>
> We have no experience on a big project. What should we do about the
> project? Redo it or debug it or others? What is the most efficient way
> on our situation?
>
> I appreicate your kind help!!!
>
> Shuisheng.
>
In addition to the replies you have already received, if
you start the process of trying to fix the existing codebase:
- Start tracking where the bugs are. It's a truism that most of
the bugs are in a minority of the code modules. When you
identify these modules, you can retrofit tests into
these as a higher priority.
- Use a revision control system and a -plan- for revision
control. Allow anyone to modify the branch, but
only allow promotion of code that passes automated
tests and least brief interactive testing, according
to a written test plan. This last part is necessary
because most of the code won't be using automated
tests for some time.
- Start integrating automated tests, and ensure that -no- new
code modules or changed or added functions in existing
modules are promoted until there are unit tests.
This process should be automated. There are diff
utilities that can be called from shell scripts
as a quick way to do start this.
-Triage will be critical; don't just start changing things
because the style isn't ideal. Start with fixing
the defective code. Fixing the defective code is
-not- just coding away the error' it's also adding
the unit test code to ensure the changed code is
functioning as expected.
- Code has to: (1) work properly, (2) meet the user's needs,
and (3) be delivered (more or less) in a timely fashion.
It sounds like your folks may have fallen into the trap
of "code like hell" in order to meet #3. But you will
never meet #1 and #2 until you control and guide the
development process instead of it controlling you.
Steve McConnell's books have some good tips. Good luck.
Comments, anyone?
| |
| JXStern 2006-09-10, 9:59 pm |
| On Sat, 09 Sep 2006 17:22:15 GMT, Ron Ruble <raffles2@att.net> wrote:
>In addition to the replies you have already received, if
>you start the process of trying to fix the existing codebase:
>
>- Start tracking where the bugs are. It's a truism that most of
> the bugs are in a minority of the code modules. When you
> identify these modules, you can retrofit tests into
> these as a higher priority.
>- Use a revision control system and a -plan- for revision
> control. Allow anyone to modify the branch, but
> only allow promotion of code that passes automated
> tests and least brief interactive testing, according
> to a written test plan. This last part is necessary
> because most of the code won't be using automated
> tests for some time.
>- Start integrating automated tests, and ensure that -no- new
> code modules or changed or added functions in existing
> modules are promoted until there are unit tests.
> This process should be automated. There are diff
> utilities that can be called from shell scripts
> as a quick way to do start this.
>-Triage will be critical; don't just start changing things
> because the style isn't ideal. Start with fixing
> the defective code. Fixing the defective code is
> -not- just coding away the error' it's also adding
> the unit test code to ensure the changed code is
> functioning as expected.
>- Code has to: (1) work properly, (2) meet the user's needs,
> and (3) be delivered (more or less) in a timely fashion.
> It sounds like your folks may have fallen into the trap
> of "code like hell" in order to meet #3. But you will
> never meet #1 and #2 until you control and guide the
> development process instead of it controlling you.
>
>Steve McConnell's books have some good tips. Good luck.
>
>Comments, anyone?
All good, but especially for newbies who don't have an iterative
process down pat, an old cliche applies:
When you think you're 90% done with a project,
all that's left
is the other 90%.
J.
|
|
|
|
|