[pdftex] pdftex compression -- proposed addition to manual

[M. Wroth]

At 01:42 PM 8/26/01 +1000, Greg Black wrote:
>Timothy Murphy wrote:
>
>| One of the advantages of literate program a la Knuth
>| is that it makes it much easier to keep documentation and coding in sync.
>| In fact it is quite difficult for them to diverge.
>
>Can you substantiate this claim?  It seems to me that it is a
>simple matter for the documentation to get out of whack with any
>programming scheme.

In fact it is possible for documentation to diverge from code even with 
literate programming.  I've been working with two different projects at 
work that are using literate programming.  Their code and documentation 
have diverged in places, at times. Having the code and documentation 
immediately adjacent to each other helps make this clear when it happens, 
and appears to make it less likely.

The main places we have seen divergence have been in the general 
introductory material -- which is separated from the code, as it is 
intended to describe the code as a whole.

>| I think this is an important issue.
>| For example, I would say that at least 50% of the Linux HOWTOs are useless
>| because of changes in coding since they were written.
>
>I don't doubt this, but I have seen no evidence that LP might
>solve it.

Quantitative evidence either way in this area is hard to come by, although 
the trend to CMM level 4 may change that.  Anecdotally, I think what we've 
seen on the projects I've been working on suggests that it helps -- but 
there is no control sample to compare directly against.

>| The great Ken Thompson said "A program should do one thing, and do it well".
>
>To me, that provides support for the anti-LP brigade --- let the
>documentation people write and maintain their part of the world;
>and let the programmers, inspired by the documents, write the
>code.

If we had professional documenters (aka technical writers) working on 
pdfTeX along with the coders, I might find this line of reasoning more 
convincing (although I'm skeptical even then -- the amount of 
"professional" code that gets thrown away because it cannot be understood 
is scary).  It is true that good programmers are not necessarily good 
technical writers (in whatever mode of documentation is in use).

It appears to me that having no separate technical writers working on 
documenting the pdfTeX code means that even if the model of having separate 
coders and documenters were in fact superior, it's a moot point.  The 
comparison has to be between coders documenting their work using literate 
programming, and coders documenting their work using some other technique 
(presumably embedded comments). (Unless of course I'm wrong, and there are 
people working with the code whose primary focus is documenting it :-)
[....]

>But I keep seeing toy programs that
>are far less readable/maintainable than they would have been
>without the LP baggage.

>  And I have seen no serious LP-based
>software that can compete with the non-LP code I write in my day
>job.

The two major projects I've been dealing with at work are not necessarily a 
fair comparison.  While they are not "toy" programs by any stretch of the 
imagination, they are simulations written by people who are primarily 
analysts rather than computer programmers.  They appear to me to be much 
more maintainable than equivalent projects written without literate 
programming (and I've seen more than two or three of those).

If pdfTeX is to outlive the ability of its current authors to maintain it, 
it must be maintainable without their direct participation.  I will argue 
strongly for making that true.  I believe that literate programming is the 
best way to do that.  But writing understandable, well documented code is 
the issue.
[...]
>It's not enough to say: "This method will make everything better
>because of x, y and z."  It's necessary to conduct experiments
>that demonstrate the truth of the claims.  I have not seen any
>such research.  What I have seen is evidence that points the
>other way.

My experience is the opposite.  But it appears neither of us has 
quantitative evidence to support our respective positions, unfortunately.

>I am convinced that other programming paradigms --- including
>such ideas as structured programming and practices that
>encourage clarity of expression and maintainability --- have
>much more to offer than LP.

I have no quarrel with clarity of programming. I just think the techniques 
I've seen for that are complementary to literate programming, not 
antagonistic to each other.

Mark B. Wroth
<mark at astrid.upland.ca.us>