Home > Archive > Scheme > June 2004 > why does nobody like us was Re: programming in the large, with scheme
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
why does nobody like us was Re: programming in the large, with scheme
|
|
| Andrew 2004-06-25, 7:48 pm |
|
I am a beginner at both programming in general and Scheme in
particular (a condition which appears, alas, to be permanent), so take
the following with the appropriate grain of salt. However, I have been
observing from the sidelines for a while, and I've thought about these
sorts of things from time to time, so hopefully I'll have something
useful to say on these issues. I know this topic is a perennial
one... so my apologies for inflicting it on everyone again, but I
think there are matters here that need to be addressed, or at least
kept in mind.
The thread I'm replying to seems drifting away from module systems to
Scheme's popularity (or lack thereof). That there are seemingly few
examples of large projects done in Scheme presumbably has a close
relationship to why there are few projects attempted in Scheme at
all. I'm talking FOSS projects, or at least source-code available
projects here: it's difficult for the community to learn from the
experiences of a closed-shop project, and presumably there is some
relation between the public use of Scheme and the private.
From time to time I've looked in the usual places for projects done in
Scheme that I can learn from. Ideally I've been looking for a
small-to-medium sized thing that I'd use in my every day life that I
could contribute to, but actually it's difficult to turn up any
'end-user' programs at all. I've looked in the usual places:
sourceforge, freshmeat, the web, and I read this newsgroup at
irregular intervals. It's rare to find something that isn't:
*) a scheme implementation
*) libraries of scheme code for various things (binary trees, hash
tables, ncurses etc)
*) scheme used as a scripting/extension language in another program
(eg The Gimp).
(stuff which could be vaguely grouped together as 'web-programming',
like BRL and its ilk are a notable exception).
In other words, there is a lot of stuff around that could be used to
_make_ programs with, but an apparent paucity of programs that
actually _use_ them. If there are in fact significant end-user
programs around using scheme, I haven't been able to find them, which
is a problem in itself.
Bear points us to a few connected issues, but I suspect for him it
boils down to this:
> It always comes back to this point: If you want to do
> anything large, you have to write your own code for every
> last damn scrap of it, because nobody else is going to write
> code that works together and agrees on what versions of
> library functions (or in the case of set! even primitive
> syntax(!)) it wants. And if that's the way it's going to be,
> I can't live there and I will have to use something else.
An expression of frustration if ever there was one. He points to two
possible reasons for this. The first is a technical one: the lack of a
common module system, which appears to result in a lot of
wheel-rewriting. The second, is more sociological:
> Wrong. It's actually more common for them to make
> something available with NO FOUNDATIONS AT ALL. For
> example, somebody was agitating for being able to
> open "any url" as a file, and completely ignoring the
> fact that there is no foundation at all for network
> communications in scheme. You can't write a driver
> for your network card; you can't twiddle the hardware
> port to send bits out over ethernet; you can't read
> raw bytes (as opposed to characters) from a hardware
> buffer; you can't register an interrupt response routine.
(by the by, if anyone knows how to get gnus to quote from more than
one message, email me privately and tell me. thanks :] )
This quote was in reply to Alex Shinn's Reason #7 below. I think
they're actually addressing the same issue here, just phrased
differently which makes it seem as though there's a contradiction.
Alex Shinn <foof@synthcode.com> writes:
> At 20 Jun 2004 21:36:06 +0100, Patrick May wrote:
>
> 7) Because Scheme users are perfectionists and spend too much time
> trying to get the foundations right before building on them.
I think 6, 7 and Bear are pointing to a similar feature about the
people who end up using Scheme. They (you, we) tend to be people who
lean towards the abstract, the logical, and the aesthetic: the further
they are removed from the dirty, grubby details of coping with sending
bits down to that dodgy no-name peripheral card the happier they are;
they'd rather contemplate the perfect S-exp based syntax for
transversing red-black trees than write a device driver. That's a
sweeping generalisation, of course, but it seems to me to be broadly
true: there's far more abstract icy-slopes-of-reason stuff discussed
in c.l.s than practical ditch-digging bit-twiddling stuff, and there
don't appear to be that many device drivers written in Scheme. Alex is
talking about the theoretical foundations, which schemers seem to do
well at; Bear is talking about the practical foundations, which
schemers don't do so well at.
For the sake of simiplification, let us presume that there are two
types of people, the icy-slopes-of-reason denizens and the
ditch-diggers. Apologising in advance for such a lame pop-culture
reference, I am going to term the first 'elves', and the second
'dwarves', following Coates (
http://www.plasticbag.org/archives/2004/03/from_pirate_dwarves_to_ninja_elves.shtml
).
Obviously there are not really merely two kinds of people, and at
the very least we're dealing with a spectrum: more accurately each
individual probably has a mixture of both 'dwarven' and 'elven'
traits: so when I say an individual or a group is 'elven' let us
presume that means the 'elven' traits predominate.
Scheme has a lot of elven afficionardos, and not so many dwarven ones,
at least, so it seems to me. It also seems to me that in order to get
more projects underway with scheme, we need more dwarven nature going
here. It is dwarven to want an application to do X, and to undertake
the work necessary to build said application. It is elven to want to
do this in a way which is as aesthetically pleasing, logically sound,
etc. I don't insist on this stuff, but it does seem to me generally
true that people wanting to code application X go elsewhere for their
solutions.
Co-running with this piece of amature sociopsychology, there is also
the issue of popularity. Scheme has an image problem: it's not viewed
as a language to 'do stuff' in, and I've spoken to many people who
have a negative view towards it from school. Of course, it ends up
being one big complex feedback situation: (1) there aren't many people
coding applications in scheme, this is partly because (2) people don't
see other people coding applications in scheme (because of (1), of
course) and partly because (3) there seems to be barriers to doing so:
it's hard to use the libraries there are, no common module systems,
etc, what libraries there are available (for a given scheme
implementation) are not as comprehensive as those available for other
languages; of course, this is somewhat because of (4) there isn't
throngs of people clamouring in the corridors for them, or writing
them, which just comes back to (1) again. Threaded through all of this
is the fact that this stuff might be not only putting off a 'critical
mass' of just anyone, but the very kinds of people who are needed to
address these very problems.
I think this might be playing a part also:
> 8) Bad luck/timing. The *vast* majority of software for a long time was
> written in C/C++ because of momentum. Java required immense
> marketing to get into the scene. Perl bootstrapped itself into
> serious projects through first becoming ubiquitous on Unix systems
> and then by being useful for web development at a time when there was
> desperate need for lots of rapid web programming. Python actually
> played off Perl's popularity by providing similar libraries while not
> looking like line noise. There's no longer much room left in the web
> programming niche, and no commercial support to get into the serious
> application market.
>
> --
> Alex
But for now, I want to look at the things that put people off
scheme. The scheme community needs to be aware of these issues, and
not take a dismissive stance towards them. I'm going to assume that
it's people who already know at least something about programming that
are going to be exploring the possibility of using scheme: these
problems may not exist if you have a captive audience of
non-programmers to teach (such as your own children), but until we
manage to set up more scheme kubbitzen we will have to deal with
converting the heathen.
1) The syntax.
The first thing anyone sees of scheme (or lisp) and
it's scary: it looks nothing like anything you've seen before, all
these brackets everywhere (what do they all do? how do you tell how
they're matched? is that even important) and nary an equals sign or
anything that looks like a line terminator. The first time I had a
look at my .emacs file, I nearly voided my bladder: I was expecting
something more like a .ini file and I found it absolutely daunting. I
closed the file and didn't look at it again for a few years.
The problem is that almost everything else out there (that's likely to
be familiar to our potential convert, at least) has a roughly C-like
syntax. Our potential convert (let's call him J.) thinks he know how
to make some sense of computer code, and suddenly there's something
quite alien sitting in front of him.
I know what the responses to this are: it can't be changed, in any
case we wouldn't want to as it's a strength, and in any case it's not
a lasting difficulty as you quickly get used to it, and with proper
indenting and perhaps a decent editor it's no worse than anything else
for reading. I agree. But the initial reaction is a negative one, and
likely to result in a reaction such as "I'm not capable of
understanding this" or (perhaps more likely, given that g s tend to
have a distinct arrogant streak, especially in intellecutal matters)
"LISP is dumb".
2) The paradigm.
Many programmers, especially enthusiasts who are inclinded to s out
new languages and code applications in their spare time, learnt to
program in their 'formative years' at high school or earlier. Even if
they happen to go to a university where they are taught scheme early
on, chances are that they are already immersed in an imperative style
of programming. Futhermore, the imperative style is the dominant
paradigm in the 'real world' where you get paid to program.
Functional and imperative are worlds apart. Imperative views a
computer program essentially as a list of step by step instructions
for an idiot: put this value here, give me this amount of memory
there, etc. Imperative languages are just abstractions attempting to
make this instruction-giving easier than writing in assembly: even
with object-orientated programming you're just packaging up the
instructions in an organised fashion. Functional programming, by
contrast, emphasises abstraction for, as it were, exposing the logical
structure of the task to a mathemetician. A paradigmatic example of
this is scheme's use of recursion over iteration: scheme uses
recursion because the task has a recursive structure; other languages
use iteration because that's 'what the computer does'.
Another relevant difference is that other languages tend to shy away
from dealing with complexities such as continuations, closures, the
details of how the object system or exceptions work, etc --- these are
'extra for experts', but Scheme seems to almost revel in them. You can
even define your own syntax.
I keep encountering people who have learnt Scheme at university, but
range in their attitudes to it from thinking of it as an interesting
excercise at best, to resenting it as a useless thing their lectures
'made them' learn at worse. I don't think many of them really grokked
the language. They don't see the weird syntax, the use of
continuations and closures, or the emphasis on functional style as
useful (to write their instant messaging client or their spam filter
or whatever they want to do) , but rather a confusing hinderance of
academic interest at most. No doubt this is due to a lack of
understanding as much as anything else: you can't appreciate the
utility of something you don't understand. But they view themselves as
being able to program already, as already having adequate tools for
the job, so why do they need to bust their brains with these oddities?
Particularly as scheme seems to have other shortcomings:
3) Fragmentation
This topic has been discussed to death on c.l.s, it comes up every few
ws. Why don't all the scheme people get together and write One
Implementation to Rule Them All? I don't want to rehash this ground
again, but it has to be recognised that this is a common thought for
newcomers to have. They see all the different implementations as a
sign of a fractured community with no direction. The community can try
to explain to them to see them as different languages in their own
right etc. etc., but even if they're still vaugely interested after
the discussion has died down, there's the problem of which one to
choose?
4) lack of libraries.
There is also the perception that there are not the extensive
libraries that one can get with most other languages (Perl, Python,
C...), which is probably closer to the truth, as discussed above. If
you go for Python, you know you'll be able to find code for 'most
anything, including bindings for whatever GUI widget library you want
to use, whereas with Scheme you need to commit to an implementation,
and then you're stuck with their choice of a GUI library (I don't know
this for sure). Plus it's less than likely there will be a lot of
pre-written libraries for your purpose, so you're faced with writing a
lot of them yourself.
5) lack of any compelling examples of applications
With other languages in common use, there are many examples of FOSS
applications developed with the language that you can download and
check out, ranging from wordprocessors through IM clients to little
applets that sit on your desktop and tell the time (or something). C
has entire desktop environments written in it, Python has things like
Zope, and there are moderatley sized applications written in Perl
also. With Scheme, they seem thin on the ground. Yes, there's DrScheme,
but a newcomer could be forgiven for thinking that the only thing
Scheme is good for is for writing more Scheme implementations in.
If there aren't any compelling examples of other people using the
language for writing the sorts of apps that you're interested in
(broadly speaking) then it's not exactly striking a chord likely to
inspire you to use Scheme to write your application.
All of this is more than enough to fuel the impression
6) Scheme is a toy language for academics, students, and the occasional
purist.
In the face of the above, I don't think it's surprising that the
people who are still attracted to scheme as their language of choice
tend do so for aesthetic reasons (and I mean 'aesthetic' in a wide
sense), elven reasons, rather than practical "I can write my IM app
and I can write it now" kind of dwarven reasons. And I suspect, as I
indicated above, is that the community as a result has plenty of
people to contemplate language design issues and write the odd SRFI,
but a lack of people to get out there and code with dwarven resolve
the low-level network interface libraries that Bear wants.
The above issues with scheme obviously are interrelated and form a
kind of problematic nexus for the propagation of the use of the
language. If they could be addressed, and I hope they can be
addressed, we could see Scheme getting caught up in a virtuous cycle
of people deciding to use scheme for their app, writing libraries
needed for it, which will inspire yet more people to build on those
libraries and improve those libraries (if they have shortcomings) and
write more apps, and around about it will go.
Now, I could, as I've mentioned, be wrong about some of the above
issues. If I'm wrong, I'm sure I will be corrected very quickly. They
might not actually _be_ issues, but the point is they are _perceived_
to be issues, and I don't think that can be denied.
So what to do? Although there are definitely practical challenges to
be met (a common module system would be a substantial step forward),
there are perceptual issues to be combated also. Obviously we're not
going to change the syntax, or give up the use of continuations, but
perhaps we can do a better job of explaining why Scheme is different
from other languages and why those things are good things.
One suggestion I would make (if I may) is that thought could be given
to improving Scheme's face on the web. The first two links given by
google for scheme are a page at MIT, and schemers.org.
http://www.swiss.ai.mit.edu/projects/scheme/
http://www.schemers.org/
The MIT page makes it sound dreadfully academic:
"Scheme is a statically scoped and properly tail-recursive dialect of
the Lisp programming language invented by Guy Lewis Steele Jr. and
Gerald Jay Sussman." The rest goes on about all the nifty computer
sciencey stuff that Scheme has incorporated. I'm sorry, but this does
nothing to entice anyone interested in actually programming
applications into looking further.
Schemers.org is very studenty, which might be fine for that site, but
is hardly going to inspire hearty confidence in Scheme's suitability
for Real Work.
The other Scheme related sites on the first page are a couple of PLT
scheme pages, scheme.org which did not respond when I looked at it,
and The Internet Scheme Repository home page, which tells you that
it's not being maintained: not a good look!
Consider by contrast how a newcomer gets treated at python.org or
php.org.
Perhaps a page could be made with some introductory and explanatory
material, a breif overview of the major implementations, perhaps some
case studies or at least links to projects implemented in scheme or
other stuff to tout scheme's practical merits, and something like
Scheme Wly News
http://mjr.towers.org.uk/swn/
It would be a good place to pass on the sage advice newbies invariably
get poured upon them in this newsgroup about why there are many
schemes and why they don't use infix notation and how they should pick
an implementation and stick with it, etc. etc.
Perhaps scheme.org could be hijacked for such a purpose? Google's
cache suggests it was last updated in 2002.
Bigloo's homepage does a reasonable job of impressing a new reader
with it's practicality etc.
Anyway, just my 2 pennies worth...
--
--
|Andrew Tarr | arc@stuff.gen.nz | http://arc.stuff.gen.nz
|GPG Public Key:- http://arc.stuff.gen.nz/andrew.gpg
|_____
"There is no excellent beauty that hath not
some strangeness in the proportions"
--Francis Bacon
|~~~~~
|
| |
| Bruce Lewis 2004-06-25, 7:48 pm |
| Andrew <arc@stuff.gen.nz> writes:
> Functional and imperative are worlds apart. Imperative views a
> computer program essentially as a list of step by step instructions
> for an idiot: put this value here, give me this amount of memory
> there, etc. Imperative languages are just abstractions attempting to
> make this instruction-giving easier than writing in assembly: even
> with object-orientated programming you're just packaging up the
> instructions in an organised fashion. Functional programming, by
> contrast, emphasises abstraction for, as it were, exposing the logical
> structure of the task to a mathemetician.
I disagree with this comparison.
I gave myself a headache yesterday beating my head against a wall. This
is because of a bug I finally discovered after months of tinkering with
the source code of someone who left the company a few months ago. The
bug was a direct result of the hazardous nature of imperative
programming, specifically mutation. Certain variables were being
mutated in the wrong order, causing results that were sometimes correct,
usually close to correct, and sometimes (seemingly unpredictably) quite
wrong. I think a mathematician could have implemented this formula in
an imperative style and avoided the bug. Non-mathematicians would find
it a lot easier to get the program right using a functional style.
| |
| Shriram Krishnamurthi 2004-06-26, 4:01 am |
| "Neil W. Van Dyke" <neil@neilvandyke.org> writes:
> Andrew <arc@stuff.gen.nz> writes:
>
> Every few months for the last couple years, I try to contact the owner
> of the "scheme.org" domain, in hopes of getting a respectable showcase
> site there. [But all end in failure.]
I can testify to this. It would appear that not enough other people
are miffed by this, or at least not enough of those who are live in
Sweden. But if you're reading this thread and *do* live in Sweden,
please get in touch!
Shriram
| |
| Andrew 2004-06-28, 3:57 am |
| Bruce Lewis <brlspam@yahoo.com> writes:
> Andrew <arc@stuff.gen.nz> writes:
>
>
> I disagree with this comparison.
>
> I gave myself a headache yesterday beating my head against a wall. This
> is because of a bug I finally discovered after months of tinkering with
> the source code of someone who left the company a few months ago. The
> bug was a direct result of the hazardous nature of imperative
> programming, specifically mutation. Certain variables were being
> mutated in the wrong order, causing results that were sometimes correct,
> usually close to correct, and sometimes (seemingly unpredictably) quite
> wrong. I think a mathematician could have implemented this formula in
> an imperative style and avoided the bug. Non-mathematicians would find
> it a lot easier to get the program right using a functional style.
I think you misunderstand my metaphor. The 'idiot' and
'mathematician' are the metaphorical recipients of the code, not the
programmer.
Your example shows that in writing a program as though it were a
long list of simple instructions for an idiot, you can make mistakes:
in your account it seems that the programmers did not have the logical
form of the problem well articulated to themselves, hence the
mistakes. If they had written the program as though they were
articulating the logical structure of the problem to the satisfaction
of a mathematician (i.e. functionally) they wouldn't make these
errors. A proper mathematician would of course understand the
logical form of the program to start with, so could thus write correct
imperative code.
--
|Andrew Tarr | arc@stuff.gen.nz | http://arc.stuff.gen.nz
|GPG Public Key:- http://arc.stuff.gen.nz/andrew.gpg
|_____
"There is no excellent beauty that hath not
some strangeness in the proportions"
--Francis Bacon
|~~~~~
|
| |
| Bruce Lewis 2004-06-28, 4:12 pm |
| Andrew <arc@stuff.gen.nz> writes:
> in your account it seems that the programmers did not have the logical
> form of the problem well articulated to themselves, hence the
> mistakes. If they had written the program as though they were
> articulating the logical structure of the problem to the satisfaction
> of a mathematician (i.e. functionally) they wouldn't make these
> errors.
Actually, I think they *did* have the logical form clearly in mind. By
merely cutting one chunk of code and pasting it to just before the
variables it used were mutated, the bug was fixed. What the original
programmer missed was the additional discipline necessary to get the
program right in the presence of mutation. A functional style would
have allowed the programmer to get it right without having to think so
much about the order in which things got mutated.
|
|
|
|
|