Software Engineering - "Modern" Software Framework?

Author

"Modern" Software Framework?

philip_b_taylor@yahoo.co.uk

2007-08-17, 8:06 am

I am looking at the architecture and possible framework solutions for
a system which has the following requirements:
1. Distributed, networked control/data handling/analysis system
2. Multi-platform (Linux, Windows, + Real-time platforms TBD)
3. Multi-language (probably Java, C++, Python)
4. Widely distributed development (worldwide), including academic as
well
as industrial participants.
5. Development time 5-10 years, operational lifetime 20-30 years (yes
really).

The CORBA solution currently proposed is being criticized (amongst
other reasons) as
being "old technology".

Any suggestions as to what the "new technology" solution might be?

Philip Taylor

Dmitry A. Kazakov

2007-08-17, 8:06 am

On Fri, 17 Aug 2007 02:34:23 -0700, philip_b_taylor@yahoo.co.uk wrote:

> I am looking at the architecture and possible framework solutions for
> a system which has the following requirements:
> 1. Distributed, networked control/data handling/analysis system
> 2. Multi-platform (Linux, Windows, + Real-time platforms TBD)
> 3. Multi-language (probably Java, C++, Python)

Real-time Python, huh.

> 4. Widely distributed development (worldwide), including academic as
> well as industrial participants.
> 5. Development time 5-10 years, operational lifetime 20-30 years (yes
> really).
>
> The CORBA solution currently proposed is being criticized (amongst
> other reasons) as being "old technology".

CORBA is a middleware. Google for:

distributed middleware, data distribution layer, SCADA infrastructure

> Any suggestions as to what the "new technology" solution might be?

That depends on what you want in *technical* terms. The list above looks
more like a TV advertisement. How many nodes, which topology of
connections, how many process variables, which frequencies, which QoS,
types system?...

If you mean a data distribution / process automation middleware, then there
exist alternatives to CORBA. That does not mean they were better than
CORBA, because much depends on the requirements, see above.

Requirements are greatly different for different application areas. For
example, one middleware initiative for automotive is AUTOSAR
(http://www.autosar.org), it would not well fit avionics. Robotic platforms
is yet another thing, etc. Then there are so-called "business" middlewares
which have little to do with the former, beyond the common name.

As for "new technology," there is no technology for distributed real time,
sorry. It requires a lot of work and care.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

H. S. Lahman

2007-08-17, 7:10 pm

Responding to Philip_b_taylor...

> I am looking at the architecture and possible framework solutions for
> a system which has the following requirements:
> 1. Distributed, networked control/data handling/analysis system
> 2. Multi-platform (Linux, Windows, + Real-time platforms TBD)
> 3. Multi-language (probably Java, C++, Python)
> 4. Widely distributed development (worldwide), including academic as
> well
> as industrial participants.

IT is beginning to look more like R-T/E every day, isn't it? B-)

> 5. Development time 5-10 years, operational lifetime 20-30 years (yes
> really).
>
> The CORBA solution currently proposed is being criticized (amongst
> other reasons) as
> being "old technology".

Architecture and frameworks are at a higher level of abstraction than
CORBA. CORBA is a specific implementation of a distributed architecture
infrastructure.

For a project this size one needs to think about bigger issues first
(e.g., who is talking to whom, why they are talking, how intimately they
should talk, will the communications be message-based or remote object
access, which conversations will be two-way, etc., etc.). One figures
out the overall system structure and the strategies to deal with
communications constraints and THEN worries about what technologies to
use to implement those strategies.

*************
There is nothing wrong with me that could
not be cured by a capful of Drano.

H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
"Model-Based Translation: The Next Step in Agile Development". Email
info@pathfindermda.com for your copy.
Pathfinder is hiring:
http://www.pathfindermda.com/about_us/careers_pos3.php.
(888)OOA-PATH

xpyttl

2007-08-17, 7:10 pm

<philip_b_taylor@yahoo.co.uk> wrote in message
news:1187343263.648716.93040@22g2000hsm.googlegroups.com...

> 2. Multi-platform (Linux, Windows, + Real-time platforms TBD)
> 3. Multi-language (probably Java, C++, Python)

These two requirements seem to be mutually exclusive. As Dmitry implied,
Python certainly has nothing to do with real time, but you coud make the
same argument for Java, and maybe even C++.

You might consider having some engineers review the marketing hype before
you get too far down the road.

...

spence_m@ociweb.com

2007-08-17, 7:10 pm

On Aug 17, 4:34 am, philip_b_tay...@yahoo.co.uk wrote:
> I am looking at the architecture and possible framework solutions for
> a system which has the following requirements:
> 1. Distributed, networked control/data handling/analysis system
> 2. Multi-platform (Linux, Windows, + Real-time platforms TBD)
> 3. Multi-language (probably Java, C++, Python)
> 4. Widely distributed development (worldwide), including academic as
> well
> as industrial participants.
> 5. Development time 5-10 years, operational lifetime 20-30 years (yes
> really).
>
> The CORBA solution currently proposed is being criticized (amongst
> other reasons) as
> being "old technology".
>
> Any suggestions as to what the "new technology" solution might be?
>
> Philip Taylor

Hi Philip,

you may need a family of products.

try TAO for real time C++ CORBA. OpenDDS for real time pub/sub, ACE
for a real time, high performance (socket level) C++ abstraction
layer, JacORB for a Java CORBA implementation, and OpalORB for a perl
ORB. Boost is also a good source of libraries for C++ use.

All are open source, often have the same large community of users,
have been around for a while and have commercial support, good
documenation, training available, good interoperability, share the
same platforms etc. Many Universities do research using them.

Those products are used in defence systems, trading systems, telecom
systems, shop floor control etc.

TAO also has a component model called CIAO which is leading edge. In
general CORBA may be old but the implementations are contemporary,
mutli language, and are being used in demanding situations where other
technology frameworks fall short.

Only open source can gve you the long term use you need. Most of the
installations we support are 10 year minimum systems.

regards Malcolm Spence
Director of Business Development
OCI St. Louis MO USA
www.ociweb.com www.theaceorb.com

TEL: 1-314-579-0066 ext 206
FAX: 1-314-579-0065

philip_b_taylor@yahoo.co.uk

2007-08-17, 7:10 pm

On 17 Aug, 16:33, "xpyttl" <xpyttl_NOS...@earthling.net> wrote:
> <philip_b_tay...@yahoo.co.uk> wrote in message
>
> news:1187343263.648716.93040@22g2000hsm.googlegroups.com...
>
>
> These two requirements seem to be mutually exclusive. As Dmitry implied,
> Python certainly has nothing to do with real time, but you coud make the
> same argument for Java, and maybe even C++.
>
> You might consider having some engineers review the marketing hype before
> you get too far down the road.
>
> ..

Sorry perhaps I was not clear enough. There is, of course, no
intention of
using Python on the real-time platforms - the requirement is that
Python be
inter-operable within the framework used, not that it will be used on
every host in
the system. Python (and to some extent Java) will be mainly used in
the supervisory and scripting tiers of the system. Some (soft) real-
time hosts
will use C++, perhaps Java. There will be specialist hard real-time
processors which may have to be excluded from the framework.

As regards size, I don't know, except 'large'. There should be no size
limitations
imposed by the framework. My guess is ~100 hosts and 100K+ I/O points?
There's
a couple of years detailed design yet to go.

The reason I mention CORBA is that (like many large organisations) we
are not
free to start from a clean sheet and design a system from scratch. An
existing
CORBA-based solution is being proposed for sensible financial and
political
reasons. I was asking about new technology options that may be being
ignored
because of these constraints.

There has certainly been no marketing hype anywhere (so far), as the
proposed
solution is not sold commercially,

Philip

Phlip

2007-08-17, 10:08 pm

philip_b_taylor wrote:

> The CORBA solution currently proposed is being criticized (amongst
> other reasons) as
> being "old technology".

CORBA's main problem is it grew under a broken "request for proposals"
model. The stakeholding committees chartered their proposal system to
discourage and inhibit reference implementations for new feature requests.
This simple but horrific mistake allowed client companies to compete with
each other by submitting elaborate requests, allowing the CORBA
specification to clutter up with endless cruft. It would be hard to think of
a system worse than MS's ActiveX, but CORBA did it.

I predict that CORBA as we know it will not last another few years, and that
something else will take its spot. Maybe a simplified version of the
ACE-TAO-CIAO framework(s).

> Any suggestions as to what the "new technology" solution might be?

Let's try Ruby on Rails...

> 1. Distributed, networked control/data handling/analysis system

You distribute Ruby with dRB. A Ruby object can operate over a wire just as
easily as locally. To synchronizing objects from other languages, you use
SWIG or something to bond them to Ruby objects. End to end, this system is
very light, flexible, and feature-per-feature competitive with DCOM or
CORBA.

> 2. Multi-platform (Linux, Windows, + Real-time platforms TBD)

Uh, check, including self-hosting systems that put a lite server inside its
own web browser. That is the GUI of the future, folks.

> 3. Multi-language (probably Java, C++, Python)

For the love of the diety of your chosing... why? So that each programmer on
a project can write in their own language?

Modern applications are multi-lingual in their layers. Rails, for example,
goes from SQL in your database (any database) thru Ruby in your middleware
to HTML and JS in your browser. What other languages are there??

> 4. Widely distributed development (worldwide), including academic as
> well
> as industrial participants.

Check.

> 5. Development time 5-10 years, operational lifetime 20-30 years (yes
> really).

You mean uptime of 20 to 30 years? I don't know. But open source projects,
with wide community support, tend to only get better over time...

You forgot item 0. - wall-to-wall support for unit tests. You weren't going
to start this heroic 30 year project without unit tests, were you??

--
Phlip
http://www.oreilly.com/catalog/9780596510657/
"Test Driven Ajax (on Rails)"
assert_xpath, assert_javascript, & assert_ajax

Phlip

2007-08-17, 10:08 pm

philip_b_taylor wrote:

> There's a couple of years detailed design yet to go.

Please google for "waterfall". Such a long _detailed_ design phase is
setting you up for _assured_ disaster.

> The reason I mention CORBA

Quite like CORBA's request for proposal system...

--
Phlip
http://www.oreilly.com/catalog/9780596510657/
"Test Driven Ajax (on Rails)"
assert_xpath, assert_javascript, & assert_ajax

kjin101@gmail.com

2007-08-18, 4:27 am

On Aug 17, 9:04 am, spenc...@ociweb.com wrote:
>
> TAO also has a component model called CIAO which is leading edge.
>

CORBA component model (CCM) was largely influenced by the techniques
developed during later 90's in the apache Avalon project and early EJB
1.0 work. These tehcniques have been abandoned in the mainstream
industry for years and the EJB 1.x has became a classic negative
example of architect design. If CCM is still hyped as a "leading edge"
technique, it is most likely out of an ostrich attitude or an "if you
can't convince them (users), confuse them!" tactic.

>
> Only open source can gve you the long term use you need. Most of the
> installations we support are 10 year minimum systems.
>

CCM introduced a compliance barrier that not only tightly locks in
compliant component implementations, but also locks out almost all
existing CORBA business logic implementations (servants) and even
certain client applications and important common services (such as
Event/Notification). To be reusable in CCM (regardless whether the
given CCM container is in open source), these CORBA application
implementations and common services have to be refactored or even
completely redefined and reimplemented. All of these additional
engineering cost largely defeat the very purpose of having this
component framework even if it was in open source.

Regards,
Ke

kjin101@gmail.com

2007-08-18, 4:27 am

lakshmi js

2007-08-18, 4:27 am

Hi Lahman,

Well said. And we need to think of these aspects of software systems
before thinking of low level technology details. Assuming that
architecture and framework you will use is not tightly coupled with
development technology.

But still, it is little tricky not to consider technology choices
available, esp technology that are proven and have longer shelf life
than others, for building such systems. This becomes more relevant
when we are talking of softwares with some specific requirement, like
distributed, heterogeneity, Realtime, longevity etc.

So probably you would be looking to design your own architecture, and
this should be linked with development technology of your choice. Even
if we don't decide about technology of choice at the start, it does
provides relief to know that you could be relieved of addressing
complexities of hetereogeneity, Realtime, distribution, deployment
etc, if you consider CORBA technology. Within context of MDA it does
provide a very powerful tool for software architecture and
development.

Considering CORBA as old and so boring/outdated could be a fashion
statement than with real meaning.

CORBA has relevance as it is a defacto standard for distributed,
heterogeneous software system development, distribution, deployment
etc. CORBA has longevity, reliability, and possibly only viable
solution for developing software for the nature described as in
original post.

As mentioned by Malcolm, we have some opensource CORBA solutions for
type of system you want to build. Also you have some proven commercial
solutions like ORBexpress that are widely used in RT and embedded
systems, and supported in JAVA, C++ and Ada.

What other viable/ feasible solutions have you heard?

regards,
Lakshmi JS
www.invariantlogic.com

On Aug 17, 7:49 pm, "H. S. Lahman" <h.lah...@verizon.net> wrote:
> Responding to Philip_b_taylor...
>
>
> IT is beginning to look more like R-T/E every day, isn't it? B-)
>
>
>
> Architecture and frameworks are at a higher level of abstraction than
> CORBA. CORBA is a specific implementation of a distributed architecture
> infrastructure.
>
> For a project this size one needs to think about bigger issues first
> (e.g., who is talking to whom, why they are talking, how intimately they
> should talk, will the communications be message-based or remote object
> access, which conversations will be two-way, etc., etc.). One figures
> out the overall system structure and the strategies to deal with
> communications constraints and THEN worries about what technologies to
> use to implement those strategies.
>
> *************
> There is nothing wrong with me that could
> not be cured by a capful of Drano.
>
> H. S. Lahman
> h...@pathfindermda.com
> Pathfinder Solutionshttp://www.pathfindermda.com
> blog:http://pathfinderpeople.blogs.com/hslahman
> "Model-Based Translation: The Next Step in Agile Development". Email
> i...@pathfindermda.com for your copy.
> Pathfinder is hiring:http://www.pathfindermda.com/about_us/careers_pos3.php.
> (888)OOA-PATH

Dmitry A. Kazakov

2007-08-18, 4:27 am

On Fri, 17 Aug 2007 15:30:32 -0700, philip_b_taylor@yahoo.co.uk wrote:

> Sorry perhaps I was not clear enough. There is, of course, no
> intention of using Python on the real-time platforms - the requirement is that
> Python be inter-operable within the framework used, not that it will be used on
> every host in the system. Python (and to some extent Java) will be mainly used in
> the supervisory and scripting tiers of the system. Some (soft) real-
> time hosts will use C++, perhaps Java. There will be specialist hard real-time
> processors which may have to be excluded from the framework.

But in which sense interoperable? Consider Python software being a
publisher. The things it publishes should have certain QoS (say, jitter
<5ms), so? The point is, it is very difficult to insulate an RT part from
the rest of the system. Even HMI has RT requirements. For instance, you hit
the "emergency stop" button and in 20ms it has to reach all actuators.

IMO, a quite common misunderstanding is that an RT transport layer would
automatically make the system RT. It does not. The software does.

> As regards size, I don't know, except 'large'. There should be no size
> limitations imposed by the framework.

This is in a sort of contradiction with RT requirements. Can the system be
modified on-the-fly (process variables/connections added/removed, QoS
parameters changed)? Can the system be able to estimate QoS before run?
Should the system control and optimize QoS?

> My guess is ~100 hosts and 100K+ I/O points?

Yes, this is close to the upper end of the systems we dealt with, ~30
nodes, ~300K variables.

> There's a couple of years detailed design yet to go.

I am very intrigued. If possible of course, could you tell more about the
project?

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

philip_b_taylor@yahoo.co.uk

2007-08-18, 8:05 am

Thanks for the suggestions so far - the only really new one is Ruby on
Rails, which I
I had not considered for this type of system. I know very little about
it, but the Ruby on Rails website
states "Rails is all about infrastructure so it's a great fit for
practically any type of web application Be it
software for collaboration, community, e-commerce, content management,
statistics, management." So
it never struck me as as relevant. I may be wrong.

Someone was surprised that we want to use Java, C++ and Python. Simple
reason: re-use of
legacy code and available skills. We may be introducing new languages
but don't want to add
many more, ideally reduce. Good point about C++ being older technology
than CORBA,
another reason for phasing it out...the performance hit of Java (the
main reason C++ was used) is
surely becoming a thing of the past.

As regards RT constraints on a Python-based system like 5/20 ms
jitter. We would never require
that kind of performance throughout the whole networked system.
Millisecond range stability
will be confined within single nodes or perhaps closely-coupled
subsystems. Safety-critical
systems will be separate, direct hardware/PLC interlocks for close-
down etc. As software engineers
we leave that stuff to the hardware guys :)

As regards the 2 years design - I mean that the hardware is being
designed during the next 2 years
rather than detailed software design (which yes would be unwise and
anyway premature).

Philip

Dmitry A. Kazakov

2007-08-18, 8:05 am

On Fri, 17 Aug 2007 19:39:04 -0700, Phlip wrote:

> philip_b_taylor wrote:
>
>
> Please google for "waterfall". Such a long _detailed_ design phase is
> setting you up for _assured_ disaster.

No, you can't sell TDD here.

Let me try to explain why. There are two general archetypical approaches to
design: top-down and bottom-up. What you refer as "waterfall" falls under
top-down. Your TDD is bottom-up. In OO terms moving top-down is equivalent
to specialization. You start with the most general system functionalities
and them put [implementation] constraints on them. TDD is equivalent to
generalization, you start with a minimal working set and then add new /
extend old functionalities and features.

This is why you cannot do TDD for a system having many non-functional
requirements (like QoS). This is fundamentally incompatible with the
evolutionary approach of TDD. For all, non-functional requirements are
non-testable in the sense that a successful test does not tell you anything
useful. It might work with 255 variables and collapse with 256. The
induction: it worked for n => it will work for n+1 is obviously wrong. The
reverse works, you can add constraints keeping the system working. This is
why up-front design is the only way to go for such system.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Phlip

2007-08-18, 7:06 pm

Dmitry A. Kazakov wrote:

> No, you can't sell TDD here.
>
> Let me try to explain why.

I stopped reading here.

--
Phlip
http://www.oreilly.com/catalog/9780596510657/
"Test Driven Ajax (on Rails)"
assert_xpath, assert_javascript, & assert_ajax

Phlip

2007-08-18, 7:06 pm

philip_b_taylor wrote:

> another reason for phasing it out...the performance hit of Java (the
> main reason C++ was used) is
> surely becoming a thing of the past.

I'm getting a lot of "management by magazine" here. C++ is more
programmer-friendly than Java, and of course faster.

All the hype about Java being "programmer friendly" is just that - hype.
Java's speed is irrelevant if you just install more memory to avoid
thrashing. The point of Java was supposed to be "C++ without stray pointers
and index overruns. Instead, we got a language that makes you jump thru
hoops and write excessive cruft just to get anything done.

Java - the elegant simplicity of C++, the blazing speed of Smalltalk.

You need to settle on two languages. The one closest to the metal should be
compiled, deterministic, statically type, and as programmer-friendly as
possible. The one closest to the users (the soft layer) should be
interpretive, emergent, dynamically typed, and much more
programmer-friendly.

Adding Java (interpretive, deterministic, statically typed, and
programmer-hostile) will just slow you down.

> As regards the 2 years design - I mean that the hardware is being
> designed during the next 2 years
> rather than detailed software design (which yes would be unwise and
> anyway premature).

Ooookay. Why wait 2 years to start _building_ the software? Why not let
software development assist the hardware development?

There's nothing "premature" about starting as early as possible, to give you
the longest runway.

--
Phlip
http://www.oreilly.com/catalog/9780596510657/
"Test Driven Ajax (on Rails)"
assert_xpath, assert_javascript, & assert_ajax

H. S. Lahman

2007-08-18, 7:06 pm

Responding to Js...

> But still, it is little tricky not to consider technology choices
> available, esp technology that are proven and have longer shelf life
> than others, for building such systems. This becomes more relevant
> when we are talking of softwares with some specific requirement, like
> distributed, heterogeneity, Realtime, longevity etc.

I think it depends on where the observer is standing. B-)

In a translation shop one can provide an OOA model that fully resolves
all functional requirements and is fully testable for those
requirements. Such a model will not even <directly> identify distributed
boundaries, much less the technologies needed to communicate across
them. That's because translation OOA models need to be independent of
the computing environment to avoid the review team burning the designer
at the stake. One can then turn that model over to an MDA transformation
engine that will do full code generation for a particular computing
environment and will optimize for the specific technologies available there.

For example, probably the single most important architectural activity a
Systems Engineer does for a large project is define logical partitioning
of the system. That can be done at a very high level of abstraction
during OOA in a UML Component Diagram that is completely devoid of
technical detail. But that logical partitioning (subject matters, levels
of abstraction, flows of requirements) will drive distributed boundaries
and interface/communication strategies.

Even if one is doing manual elaboration development, in the OOD one
addresses nonfunctional requirements at the strategic level. So the
architectural decisions are around things like caching, memory
management, and distributed messaging strategies. One really doesn't get
to tactical design decisions like DCOM vs. CORBA until one implements
the OOD strategies during OOP.

[An oversimplification because the Systems Engineer/Architect is likely
to decree what technologies will be used much earlier in the development
cycle based on other factors than just functional and nonfunctional
requirements. Some would even argue that specifying a technology is a
nonfunctional requirement driven by the development environment (e.g.,
existing developer skills, tool costs, existing hardware and software,
etc.).]

*************
There is nothing wrong with me that could
not be cured by a capful of Drano.

H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
"Model-Based Translation: The Next Step in Agile Development". Email
info@pathfindermda.com for your copy.
Pathfinder is hiring:
http://www.pathfindermda.com/about_us/careers_pos3.php.
(888)OOA-PATH

xpyttl

2007-08-18, 7:06 pm

"Phlip" <phlipcpp@yahoo.com> wrote in message
news:srOdnU9-UM2cY1vbnZ2dnUVZ_umlnZ2d@adelphia.com...
> Dmitry A. Kazakov wrote:
>
>
> I stopped reading here.

as he covers his ears and sticks out his tongue -- nya, nya, nya

...

Johnny Willemsen

2007-08-19, 4:27 am

Hi,

> Thanks for the suggestions so far - the only really new one is Ruby on
> Rails, which I
> I had not considered for this type of system. I know very little about
> it, but the Ruby on Rails website
> states "Rails is all about infrastructure so it's a great fit for
> practically any type of web application Be it
> software for collaboration, community, e-commerce, content management,
> statistics, management." So
> it never struck me as as relevant. I may be wrong.

We as Remedy IT have a CORBA product available for Ruby, see
www.theaceorb.nl With that solution we can also use and implement CORBA in
Ruby.

Johnny

Markus Elfring

2007-08-21, 8:05 am

> There's a couple of years detailed design yet to go.

Would you like to look at recent design methodologies to avoid any pitfalls?
http://en.wikipedia.org/wiki/Agile_software_development

Regards,
Markus

Gabriel Claramunt

2007-08-21, 8:05 am

Yes, following modern process and methodologies will reduce the risks of the
project and increases the chances of success.
They're many process to follow, and one must adopt/adapt the one that best
fits to the team and the project (and the organization). Yes, seems obvious,
but people seems to forget that ;-)

--
Gabriel Claramunt
http://gabrielsw.blogspot.com

"Markus Elfring" <Markus.Elfring@web.de> wrote in message
news:5j04n5F3qq6umU1@mid.individual.net...
>
> Would you like to look at recent design methodologies to avoid any
> pitfalls?
> http://en.wikipedia.org/wiki/Agile_software_development
>
> Regards,
> Markus

Phlip

2007-08-21, 10:05 pm

Markus Elfring wrote:

>
> Would you like to look at recent design methodologies to avoid any
> pitfalls?
> http://en.wikipedia.org/wiki/Agile_software_development

You total process zealot! Don't you know that one size doesn't fit all, and
that many great _heroic_ projects have been delivered, on time and under
budget, with pure Waterfall??!

--
Phlip

Markus Elfring

2007-08-22, 8:06 am

> You total process zealot! Don't you know that one size doesn't fit all, and
> that many great _heroic_ projects have been delivered, on time and under
> budget, with pure Waterfall??!

There are different impressions about the success statistics. How big is the
probability that requirements will change during the years while it is tried to
finish each project phase sequentially? How stable are any dependencies?

Agile software development contains a continuum from predictive to adaptive
processes. How much would you like to balance agility and disciplined work?
Would you like to consider any of the following approaches to embrace change in
your project management?
- Crystal Clear
- Extreme Programming
- Scrum

Regards,
Markus

Phlip

2007-08-22, 7:06 pm

Forgive me for getting serious again, but...

Markus Elfring wrote:

> Agile software development contains a continuum from predictive to
> adaptive processes.

That's as may be, but proudly declaring "our project is so important that
its requirements phase will last longer than a year!" is a major warning
bell. Study after study has shown that the longer a requirements and design
phase goes, the higher the odds the resulting product has major problems.
These studies don't track aphorisms, like "the programmers thought the
project was fun" (though that's important). The studies track the billions
of dollars these projects waste.

The reason is simple: Those phases accumulate many mistakes, without any
feedback. Thou shalt increment and iterate.

> How much would you like to balance agility and
> disciplined work?

You mean as disciplined as writing a test case for every line of code?

--
Phlip
http://www.oreilly.com/catalog/9780596510657/
^ assert_xpath
http://tinyurl.com/23tlu5 <-- assert_raise_message

Alexei Polkhanov

2007-08-22, 10:06 pm

On Aug 17, 2:34 am, philip_b_tay...@yahoo.co.uk wrote:
> I am looking at the architecture and possible framework solutions for
> a system which has the following requirements:
> 1. Distributed, networked control/data handling/analysis system
> 2. Multi-platform (Linux, Windows, + Real-time platforms TBD)
> 3. Multi-language (probably Java, C++, Python)
> 4. Widely distributed development (worldwide), including academic as
> well
> as industrial participants.
> 5. Development time 5-10 years, operational lifetime 20-30 years (yes
> really).
>
> The CORBA solution currently proposed is being criticized (amongst
> other reasons) as
> being "old technology".

Hehehehehe!!! You have not listed even a SINGLE requirement in that
list. All these items are parts for proposed solution for problem that
have not been even articulated. When you see something like "System
should send customer information to server over XML protocol" - first
thing that should ring the bell is that it is not a requirement
because requirements should never suggest solution or have parts of
design embedded in it. "System should be able to submit customer
information to central shared storage and provide feedback to user in
2 seconds". - this is requirement.

Ironically I would excuse someone with little knowledge about Software
Development, but for someone who knows

words like "CORBA and C++"
you should at least be aware of what User Requirements are and thing
that go in there and things that NOT. It even more amazing that most
people who participated in this thread did not even noticed that :-)

Alexei.

Phlip

2007-08-22, 10:06 pm

Alexei Polkhanov wrote:

> Hehehehehe!!! You have not listed even a SINGLE requirement in that
> list. All these items are parts for proposed solution for problem that
> have not been even articulated. When you see something like "System
> should send customer information to server over XML protocol" - first
> thing that should ring the bell is that it is not a requirement
> because requirements should never suggest solution or have parts of
> design embedded in it. "System should be able to submit customer
> information to central shared storage and provide feedback to user in
> 2 seconds". - this is requirement.
>
> Ironically I would excuse someone with little knowledge about Software
> Development, but for someone who knows

words like "CORBA and C++"
> you should at least be aware of what User Requirements are and thing
> that go in there and things that NOT. It even more amazing that most
> people who participated in this thread did not even noticed that :-)

Why, that's almost like saying that long-term requirements should not be
planned, up-front, as technical details!

--
Phlip
http://www.oreilly.com/catalog/9780596510657/
"Test Driven Ajax (on Rails)"
assert_xpath, assert_javascript, & assert_ajax

Dmitry A. Kazakov

2007-08-23, 4:19 am

On Wed, 22 Aug 2007 18:26:14 -0700, Alexei Polkhanov wrote:

> When you see something like "System
> should send customer information to server over XML protocol" - first
> thing that should ring the bell is that it is not a requirement
> because requirements should never suggest solution or have parts of
> design embedded in it.

Yep, the XML hype becomes damaging. Alas, even serious customers start to
place that nonsense into their requirements. There is nothing to do about
it, just hold your breath and wait until the dirty wave passes...

> "System should be able to submit customer
> information to central shared storage and provide feedback to user in
> 2 seconds". - this is requirement.

Well, actually the middleware is also a solution. QoS requirements for the
middleware should be a superset of the application system's requirements.
The difference is that XML fulfils no "physical" system requirements, but
fictional ones.

> Ironically I would excuse someone with little knowledge about Software
> Development, but for someone who knows

words like "CORBA and C++"
> you should at least be aware of what User Requirements are and thing
> that go in there and things that NOT. It even more amazing that most
> people who participated in this thread did not even noticed that :-)

Huh, that's rather normal nowadays. There is a whole generation of
programmers grown with the idea of "spinal programming."

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

apm35@student.open.ac.uk

2007-08-23, 8:05 am

On 18 Aug, 03:37, "Phlip" <phlip...@yahoo.com> wrote:
> CORBA's main problem is it grew under a broken "request for proposals"
> model. The stakeholding committees chartered their proposal system to
> discourage and inhibit reference implementations for new feature requests.
> This simple but horrific mistake allowed client companies to compete with
> each other by submitting elaborate requests, allowing the CORBA
> specification to clutter up with endless cruft.

> I predict that CORBA as we know it will not last another few years, and that
> something else will take its spot. Maybe a simplified version of the
> ACE-TAO-CIAO framework(s).

I agree that CORBA is the worst example of committee-itus that the
world has ever seen. The C++ language binding is horrific and IMO will
never ever be fixed. But the binding for java is ok and I think CORBA
works quite well with java. Also despite the poor language bindings in
other languages the fact that there are other language binding makes
CORBA a good way to interface to legacy systems (with java on the
frontend of course). IMO. So I think its got a good for several years
yet.

There is a project called ICE from ZeroC, developed by ex-CORBA and ex-
OMG people which tries to remedy the mistakes made with CORBA. You
might want to take a look at it. This seems like a much cleaner system
to me and shows alot of promise. Time will tell.

Regards,

Andrew Marlow

kjin101@gmail.com

2007-08-23, 10:09 pm

ap...@student.open.ac.uk wrote:
> On 18 Aug, 03:37, "Phlip" <phlip...@yahoo.com> wrote:
>
>
> I agree that CORBA is the worst example of committee-itus that the
> world has ever seen. The C++ language binding is horrific and IMO will
> never ever be fixed. But the binding for java is ok and I think CORBA
> works quite well with java. Also despite the poor language bindings in
> other languages the fact that there are other language binding makes
> CORBA a good way to interface to legacy systems (with java on the
> frontend of course). IMO. So I think its got a good for several years
> yet.
>

The C++ binding is out-of-date but not that horrific and is an
insignificant reason in CORBA's failure (or fall). Even if CORBA had a
perfect C++ binding, it would still as miserable as it today. The
major technique problem of CORBA is not its some how out-of-date
language mappings, not the data aligment in CDR encoding, not the
unnatural OBV, the chatty OTS, the braindead AMI, FT, etc. etc., but
the lack of a simple, light while still superb component assembly,
deployment and configure model/framework that canceals complexities in
building CORBA server, and/or publish/subscribe (either event/
notification or data distributation), secured, and/or transactional
(as well as persistent, fault tolerant, etc.) applications.

Regards,
Ke

Markus Elfring

2007-08-23, 10:09 pm

> You mean as disciplined as writing a test case for every line of code?

There might be a kind of competition between agility and engineering discipline.
Are you used to the explanations and advices from the information source
"http://www.infoq.com/agile/"?
Example presentations:
- "The Role of Testing and QA in Agile Software Development"
by Scott W. Ambler

- "Agile Styles: Feature Driven Development and the Crystal Methodologies"
by David Anderson and Alistair Cockburn

Regards,
Markus

Phlip

2007-08-23, 10:09 pm

Markus Elfring wrote:

>
> There might be a kind of competition between agility and engineering
> discipline.

The general topic here is "reclaming the language". Agility is _very_
disciplined. For example, under XP, you _only_ write production code in
pairs, and you _always_ write a test for every line of code.

Claiming that Brand X is "disciplined", therefor XP is "sloppy", is playing
the language game.

--
Phlip
http://www.oreilly.com/catalog/9780596510657/

Alexei Polkhanov

2007-08-23, 10:09 pm

On Aug 22, 7:38 pm, "Phlip" <phlip...@yahoo.com> wrote:
>
>
> Why, that's almost like saying that long-term requirements should not be
> planned, up-front, as technical details!
>
Nope, that is not "technical details" that is "suggesting design in
requirements" - which is a NO-NO. Phrase that sounds like "Multi-
platform (Linux, Windows, + Real-time platforms TBD)" should never
appear in requirements document because it is not a requirement. It is
not only most common and most dangerous mistake made during
requirement specification it is the most costly mistake as well.
Requirement should only say what system should do for and with what
constraints. So for example you should not say "System should be multi-
platform" - that only the software engineer who will design the system
can decide if it should be multi-platform or single platform, should
it be real time embedded system written in C++ or 100 line program
written in interpreted BASIC. As soon as it serves the purpose and
satisfies all end-user requirements. Requirements should not say if
software need to be a part of it. If you have requirements for the
oven which only need to control temperature +- 1C and have timer - are
you sure it needs software? Would it be cheaper and more practical
without it?

So again - design, parts of it or any suggestions to possible design
decisions should be avoided in requirements specifications at all
costs.

Some my favorite references on requirements:
[1] Software Requirements: Objects, Functions and States, Second
Edition [FACSIMILE] (Paperback) by Alan M. Davis (Author)
[2] Software Requirements, Second Edition (Paperback) by Karl E.
Wiegers (Author)

> --
> Phlip
> http://www.oreilly.com/catalog/9780596510657/
> "Test Driven Ajax (on Rails)"
> assert_xpath, assert_javascript, & assert_ajax

Ajax on rails ... as for me I like filet mignon on charred onions and
zucchini with balsamic vinegar sauce. Sorry just kidding ;-)

Alexei.

Phlip

2007-08-23, 10:09 pm

Alexei Polkhanov wrote:

> Nope, that is not "technical details" that is "suggesting design in
> requirements" - which is a NO-NO.

Violent agreement achieved... (-;

--
Phlip
http://www.oreilly.com/catalog/9780596510657/
^ assert_xpath
http://tinyurl.com/23tlu5 <-- assert_raise_message

David Lightstone

2007-08-23, 10:09 pm

"Phlip" <phlipcpp@yahoo.com> wrote in message
news:HYqdnZy5KY0EEFbbnZ2dnUVZ_ramnZ2d@ad
elphia.com...
> Markus Elfring wrote:
>
>
> You total process zealot! Don't you know that one size doesn't fit all,
> and that many great _heroic_ projects have been delivered, on time and
> under budget, with pure Waterfall??!
>
> --

Huh, Are you not the official "Use Agile first" troll?

> Phlip
>
>

Michi Henning

2007-08-24, 8:05 am

kjin101@gmail.com wrote:
>
> The C++ binding is out-of-date but not that horrific and is an
> insignificant reason in CORBA's failure (or fall). Even if CORBA had a
> perfect C++ binding, it would still as miserable as it today.

I tend to agree. It's death by a thousand wounds. All the little bits
that are wrong in minor ways (and sometimes major ways) add up to a
considerable amount of pain. I believe that, had this pain not been
experienced by users, the current SOA/WS/SOAP/XML craze would never
have happened. Talk about jumping from the frying pan into the fire...

Cheers,

Michi.

S Perryman

2007-08-24, 8:05 am

"Michi Henning" <michi@zeroc.com> wrote in message
news:YAszi.19436$tp3.99946@nasal.pacific.net.au...

> kjin101@gmail.com wrote:

[color=darkred]
> I tend to agree. It's death by a thousand wounds. All the little bits
> that are wrong in minor ways (and sometimes major ways) add up to a
> considerable amount of pain.

Yes, it is quite interesting how all the parties in the OMG managed to mess
up CORBA, when the ANSA platform (a pre-cursor to CORBA, and a
large input into the OMG/ODP stds) as far back as *1990* already had :

- C/Objective-C/C++ IDL code generators
- Full Trader capability (including federation)
- Inter-working with Lisp/CLOS envs

Regards,
Steven Perryman

Phlip

2007-08-24, 10:05 pm

Michi Henning wrote:

> I tend to agree. It's death by a thousand wounds. All the little bits
> that are wrong in minor ways (and sometimes major ways) add up to a
> considerable amount of pain. I believe that, had this pain not been
> experienced by users, the current SOA/WS/SOAP/XML craze would never
> have happened. Talk about jumping from the frying pan into the fire...

Why is the cost of any of them justified? What am I missing here?

This is the Ruby dRB call to make an object available on a server:

DRb.start_service("druby://server:7777",
SongNameServer.new("/tmp/songname"))

The innards of SongNameServer is just a standard Ruby object.

Here's the call to connect from a client to that object:

DRb.start_service
ro = DRbObject.new(nil, 'druby://server:2001')

And 'ro.songname' will call one of the object's methods.

What does CORBA purport to have that this lite rig doesn't have? And would
do we need to write the billions of lines of CORBA (or SOAP or whatever)
required just to say "hello world" across a wire?

--
Phlip
http://www.oreilly.com/catalog/9780596510657/

kjin101@gmail.com

2007-08-24, 10:05 pm

Phlip wrote:
> Michi Henning wrote:
>
>
> Why is the cost of any of them justified? What am I missing here?
>
> This is the Ruby dRB call to make an object available on a server:
>
> DRb.start_service("druby://server:7777",
> SongNameServer.new("/tmp/songname"))
>
> The innards of SongNameServer is just a standard Ruby object.
>
> Here's the call to connect from a client to that object:
>
> DRb.start_service
> ro = DRbObject.new(nil, 'druby://server:2001')
>
> And 'ro.songname' will call one of the object's methods.
>
> What does CORBA purport to have that this lite rig doesn't have? And would
> do we need to write the billions of lines of CORBA (or SOAP or whatever)
> required just to say "hello world" across a wire?
>

1. For a simple hello world application, a CORBA server doesn't
require user to write billions of lines of code. if you talk about
CORBA ORB's code, then you can ask the similar question of: "Why we
need billions of lines of code of linux/unix or windows just to
display hello world on a screen?". The billions lines (I guess more
than 99% of your billions LoC ORB code are blank and/or code comments)
of ORB code are for things like worker thread management, optimized
concurrent request dispatching, various local IPC mechanisms (e.g.
unix-domain socket, shared memory or memory mapping file, solaris
door, etc. etc.) or co-host optization, memory/object pooling,
connection management, request intercepting callback, request context
handling, objects to implementations resolvations (mapping to
different server policies and scenarios), instead of a primitive,
coarse-grained server that use one-shoe-fit-all default scenario to
handle a "hello world" request from remote clients.

2. your example discuss about the deployment model of a server
application. That is exactly what I mentioned yesterday, namely CORBA
fails to define a straightforward and effective server deployment
model. if it did and did it at the right time, then history could be
different ....

Regards,
Ke

> --
> Phlip
> http://www.oreilly.com/catalog/9780596510657/

Gabriel Claramunt

2007-08-25, 7:07 pm

Regarding process discipline, I agree with the conclutions of A. Cockburn in
his paper
"Characterizing people as non-linear, first-order components in software
development"
(
http://alistair.cockburn.us/index.p...elopment
)
"...The second is that people tend to inconsistency. The prediction is that
methodologies requiring disciplined consistency are fragile in practice...."
Those process need a special form of support (coaches, hard rules, etc) to
maintain discipline.
Probably agility and discipline are orthogonal concepts, what seems to me
somewhat oppossed concepts are discipline and adaptability.
Personally, I'll preffer a process with the least amount of discipline
possible to deliver results...

--
Gabriel Claramunt
http://gabrielsw.blogspot.com

"Phlip" <phlip2005@gmail.com> wrote in message
news:wclzi.57518$GO6.7156@newsfe21.lga...
> Markus Elfring wrote:
>
>
> The general topic here is "reclaming the language". Agility is _very_
> disciplined. For example, under XP, you _only_ write production code in
> pairs, and you _always_ write a test for every line of code.
>
> Claiming that Brand X is "disciplined", therefor XP is "sloppy", is
> playing
> the language game.
>
> --
> Phlip
> http://www.oreilly.com/catalog/9780596510657/

Rob Ratcliff

2007-08-25, 7:07 pm

I guess it is good to see that there is no lack of hyperbolic opinions
and extreme dogma in our industry! :-) Dumping rather than incrementally
improving working solutions seems to be common place and perhaps even
encouraged to spur on new business opportunities. It'd be nice to see
continual progress such as you might see in other hard engineering
industries (I came from the aerospace industry) rather than all of this
sideways and even backwards movement and churn.

I've been developing with CORBA in various languages for 7 years now and
I fail to see it as "horrific" failure. It performs great, isn't that
hard to use (especially for the typical RPC or event-driven
application), provides lots of mature verticals and with all the
interoperable open source implementations offers a lot of choice. Of
course there are things that I would like to see improved like: modern
language bindings based on current best practices, making bi-directional
comms the norm, support for simple versioning, friendly reflective-type
parsers to better handle additions to data structures without
recompiling, friendlier error messages, IDL with annotations and such,
but those are all things that could be incrementally improved rather
than dumping the solution completely.

I was happy to see that some of the current OMG members are pushing for
improvements to the Java and C++ language bindings. I think there is
still lots of future potential for CORBA, especially if people remain
flexible and plan for success!

JMHO,

Rob

kjin101@gmail.com wrote:
> ap...@student.open.ac.uk wrote:
[color=darkred]

[color=darkred]
>
> The C++ binding is out-of-date but not that horrific and is an
> insignificant reason in CORBA's failure (or fall). Even if CORBA had a
> perfect C++ binding, it would still as miserable as it today. The
> major technique problem of CORBA is not its some how out-of-date
> language mappings, not the data aligment in CDR encoding, not the
> unnatural OBV, the chatty OTS, the braindead AMI, FT, etc. etc., but
> the lack of a simple, light while still superb component assembly,
> deployment and configure model/framework that canceals complexities in
> building CORBA server, and/or publish/subscribe (either event/
> notification or data distributation), secured, and/or transactional
> (as well as persistent, fault tolerant, etc.) applications.
\

Phlip

2007-08-25, 7:07 pm

Rob Ratcliff wrote:

> I was happy to see that some of the current OMG members are pushing for
> improvements to the Java and C++ language bindings. I think there is
> still lots of future potential for CORBA, especially if people remain
> flexible and plan for success!

I sure wish I could flame three whole newsgroups so subtly and
devastatingly!

(-;

--
Phlip
http://www.oreilly.com/catalog/9780596510657/
"Test Driven Ajax (on Rails)"
assert_xpath, assert_javascript, & assert_ajax

JPWoodruff@gmail.com

2007-09-02, 7:12 pm

On Aug 17, 4:30 pm, philip_b_tay...@yahoo.co.uk wrote:

<... all context elided ...>

I would direct your attention to the International Conference on
Accelerator and Large Experimental Physics Control Systems. ICALEPCS
is a biennial series of conferences where large distributed
real time systems are discussed.

Some ongoing projects give progress reports per two years.

CORBA is well represented in these systems -- at least the "new"
ones (the ones that were new 5 years or so ago).

I worked on one of them - the National Ignition Facility laser - at
the end of my work, then retired in 2002. You can read reports of the
software framework used there in both ICALPECS and SigAda
publications.

>
> There has certainly been no marketing hype anywhere

I for one thought your reasoning made sense so far.

John

- sorry for long latency. I wasn't paying attention.....

philip_b_taylor@yahoo.co.uk

2007-09-02, 7:12 pm

On 23 Aug, 02:26, Alexei Polkhanov <apolkha...@relic.com> wrote:
> Hehehehehe!!! You have not listed even a SINGLE requirement in that
> list. All these items are parts for proposed solution for problem that
> have not been even articulated. When you see something like "System
> should send customer information to server over XML protocol" - first
> thing that should ring the bell is that it is not a requirement
> because requirements should never suggest solution or have parts of
> design embedded in it. "System should be able to submit customer
> information to central shared storage and provide feedback to user in
> 2 seconds". - this is requirement.
>
> Ironically I would excuse someone with little knowledge about Software
> Development, but for someone who knows

words like "CORBA and C++"
> you should at least be aware of what User Requirements are and thing
> that go in there and things that NOT. It even more amazing that most
> people who participated in this thread did not even noticed that :-)
>

Funnily enough, in this instance these *are* requirements. It just
happens that they include some required solutions. Unfortunately, in
real large systems of this type, non-functional requirements involving
politics, cost, legacy and prejudice are much more influential than
things like a clean design or even
a system that actually satisfies the functional requirements. At its
worst, a feasible solution is first
decided on, then you find someone who's willing to implement this
solution. Eventually you
see some User Requirements - so modify them to fit the solution...

Philip

Phlip

2007-09-02, 10:05 pm

philip_b_taylor wrote:

> Funnily enough, in this instance these *are* requirements. It just
> happens that they include some required solutions.

I heard a really

term for these situations recently; it applies here
very well.

The details you listed are "prosthetic goals". Your real goals - your _real_
requirements - are a system with such-and-so users, such-and-so throughput,
and such-and-so costs.

> Unfortunately, in
> real large systems of this type, non-functional requirements involving
> politics, cost, legacy and prejudice are much more influential than
> things like a clean design or even
> a system that actually satisfies the functional requirements.

So, whoever added all these finished decisions to your "requirements"
documents, do they have the authority to decree you all _must_ use them? If
so, do they also have the responsibility to see the project finished?

--
Phlip
http://www.oreilly.com/catalog/9780596510657/
"Test Driven Ajax (on Rails)"
assert_xpath, assert_javascript, & assert_ajax

Dmitry A. Kazakov

2007-09-03, 4:19 am

On Sun, 2 Sep 2007 17:19:45 -0700, Phlip wrote:

> So, whoever added all these finished decisions to your "requirements"
> documents, do they have the authority to decree you all _must_ use them? If
> so, do they also have the responsibility to see the project finished?

Huh, people making such decisions never have any responsibility other than
to continue to do so... The backbone of modern SW management is decoupling
between producers of shit and the consumers of. (:-))

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Phlip

2007-09-03, 7:06 pm

Dmitry A. Kazakov wrote:

> Huh, people making such decisions never have any responsibility other than
> to continue to do so... The backbone of modern SW management is decoupling
> between producers of shit and the consumers of. (:-))

He's doing a heckuva job, and I stand behind him 110%.

--
Phlip

Sponsored Links