[texhax] Re: BibTeX and its limitations

[Gerald Weber]

Dear Richard,

I've spent the last two years writing a bibtex style called abnt-num.bst and 
abnt-alf.bst (it's the same style, one uses numeric citation the other 
alphabetical). They are part of a project trying to implement latex/bibtex 
styles for the Brazilian ABNT norms (a derivative of the ISO norms).
The project is hosted at http://abntex.codigolivre.org.br (unfortunately it's 
mostly in portuguese).

The ABNT norm 6023/2002 for bibliographic references is probably one of the 
dumbest bibliographic norms ever made in human history and I probably ran 
across every possible bibtex limitation. But I also worked out several 
interesting mechanisms to make a bibtex style more flexible.

An important limitation are the name handling functions. If you have a simple 
"John Doe" style name, everything is OK, but problems arise with more 
complicated names from different cultural origins. 
The ABNT norms introduced and extra (and unnecessary) complication of the 
last-last name (or sur-sur name). For example, Brazil's current president is 
Luiz Inácio Lula da Silva. His surname is "Lula da Silva", so if you set 
author={Lula da Silva, Luiz Inácio}
everything should be just fine. But for ABNT it would be just "Silva", i.e., 
the last name of the last name. I had to wrestle a lot with bibtex's 
functions in order to get this reasonable working without having to rewrite 
every author field in the database.

In the absence of an author, editor or organization for your citation, the 
ABNT norm invented another strange approach, they will use the title with the 
first word capitalized. A entry with 
title={The strange limitations of {BibTeX}}
will become
THE STRANGE limitations of BibTeX
This is rather complicated to implement with the few string handling functions 
provided by bibtex, but things get worse when you have accented characters 
like \c{c}.

On the other hand we were able to insert a lot of customization options into a 
single bst file without having to rewrite the bst itself. This is done by 
writing a \citeoption command derived from the \cite command and implementing 
a bst entry called @abnt-options. We provide a bib file called 
abnt-options.bib with entries like
@ABNT-options{abnt-dont-use-etal=yes,
 abnt-etal-list=0,
 key="aaaa"}

the user will write in his latex file \citeoption{abnt-dont-use-etal=yes} and 
the bst file will process the field abnt-etal-list=0 and modify it's behavior 
accordingly. We noticed that this is a very effective way to introduce 
options to a bst style because many options can be grouped into a single 
bibtex entry. But there is a memory penalty to pay. Actually we had to ask 
the MikTeX project maintainer to increase bibtex's memory tables and we have 
to be always very careful with strings.
I think this mechanism could be used to write a very generic but highly 
customizable bibtex style.

BibTeX's programming language is hard to learn and even harder to debug. It 
has lots of limitations, the worst (in my opinion) is the lack of an 
include-file feature. 
It would be a lot simpler writing bibtex code by placing common functions into 
a file which could be included by several bst styles.
Probably bibtex's language should be replaced by some modern string handling 
language, but it would be a pity to loose all those bst files already 
written.

Of course one big improvement would be using xml bibliographic bases directly 
with bibtex. See http://bibtexml.sourceforge.net/ for interesting material 
about this.

hope you may find these comments useful

best wishes

Gerald Weber
gweber at codigolivre.org.br