TeX (= tau epsilon chi, and pronounced similar to "blecch", not to the state known for `Tex-Mex' chili) is a computer language designed for use in typesetting; in particular, for typesetting math and other technical (from greek "techne" = art/craft, the stem of `technology') material.
In the late 1970s, Donald Knuth was revising the second volume of his multivolume magnum opus The Art of Computer Programming, got the galleys, looked at them, and said (approximately) "blecch"! He had just received his first samples from the new typesetting system of the publisher's, and its quality was so far below that of the first edition of Volume 2 that he couldn't stand it. Around the same time, he saw a new book by Patrick Winston that had been produced digitally, and ultimately realized that typesetting meant arranging 0's and 1's (ink and no ink) in the proper pattern, and said (approximately), "As a computer scientist, I really identify with patterns of 0's and 1's; I ought to be able to do something about this", so he set out to learn what were the traditional rules for typesetting math, what constituted good typography, and (because the fonts of symbols that he needed really didn't exist) as much as he could about type design. He figured this would take about 6 months. (Ultimately, it took nearly 10 years, but along the way he had lots of help from some people who are well known to many readers here—Hermann Zapf, Chuck Bigelow, Kris Holmes, Matthew Carter and Richard Southall are acknowledged in the introduction to Volume E, Computer Modern Typefaces, of the Addison-Wesley Computers & Typesetting book series.)
A year or so after he started, Knuth was invited by the American Mathematical Society (AMS) to present one of the principal invited lectures at their annual meeting. This honor is awarded to significant academic researchers who (mostly) were trained as mathematicians, but who have done most of their work in not strictly mathematical areas (there are a number of physicists, astronomers, etc., in the annals of this lecture series as well as computer scientists); the lecturer can speak on any topic s/he wishes, and Knuth decided to speak on computer science in the service of mathematics. The topic he presented was his new work on TeX (for typesetting) and Metafont (for developing fonts for use with TeX). He presented not only the roots of the typographical concepts, but also the mathematical notions (e.g., the use of Bezier splines to shape glyphs) on which these two programs are based. The programs sounded like they were just about ready to use, and quite a few mathematicians, including the chair of the Math Society's board of trustees, decided to take a closer look. As it turned out, TeX was still a lot closer to a research project than to an industrial strength product, but there were certain attractive features:
Mathematicians are traditionally, shall we say, frugal; their budgets have not been large (before computer algebra systems, pencils, paper, chalk and blackboards were the most important research tools). TeX came along just before the beginnings of the personal computer; although it was developed on one of the last of the "academic" mainframes (the DECsystem ("Edusystem")-10 and -20), it was very quickly ported to some early HP workstations and, as they emerged, the new personal systems. From the start, it has been popular among mathematicians, physicists, astrophysicists, astronomers, any research scientists who were plagued by lack of the necessary symbols on typewriters and who wanted a more professional look to their preprints.
To produce his own books, Knuth had to tackle all the paraphernalia of academic publishing - footnotes, floating insertions (figures and tables), etc., etc. As a mathematician/computer scientist, he developed an input language that makes sense to other scientists, and for math expressions, is quite similar to how one mathematician would recite a string of notation to another on the telephone. The TeX language is an interpreter. It accepts mixed commands and data. The command language is very low level (skip so much space, change to font X, set this string of words in paragraph form, …), but is amenable to being enhanced by defining macro commands to build a very high level user interface (this is the title, this is the author, use them to set a title page according to AMS specifications). The handling of footnotes and similar structures are so well behaved that "style files" have been created for TeX to process critical editions and legal tomes. It is also (after some highly useful enhancements in about 1990) able to handle the composition of many different languages according to their own traditional rules, and is for this reason (as well as for the low cost), quite widely used in eastern Europe and with other scripts outside of western Europe/North America.
Some of the algorithms in TeX have not been bettered in any of the composition tools devised in the years since TeX appeared. The most obvious example is the paragraph breaking: text is considered a full paragraph at a time, not line-by-line; this is the basic starting algorithm used in the HZ-program by Peter Karow (and named for Hermann Zapf, who developed the special fonts this program needs to improve on the basics).
In summary, TeX is a special-purpose programming language that is the centerpiece of a typesetting system that produces publication quality mathematics (and surrounding text), available to and usable by individuals.
Metafont - the font creation program - is separate from TeX. It generates only bitmap fonts (although internally it creates outlines on which the bitmaps are based). There is still research to be done on combining overlapping outlines into a single outline that can be used like the outline of a Type 1 font; Knuth has "frozen" Metafont, so further research and development are being done by someone else, and the result will not be called "Metafont". In fact, it's also possible to use Type 1 fonts with TeX, and nearly all TeX installations routinely use free or commercial Type 1 fonts, especially if they're not producing heavily technical material; only Computer Modern (the font Knuth developed), Lucida Bright, and Times have anywhere near a comprehensive symbol complement, and none of the symbol sets are from "major" font suppliers (too much work, not enough money; the demise of Monotype's symbol catalog - metal only - is a particularly great loss).
TeX is the composition engine (strictly speaking, an interpreter, not a compiler). It is essentially a batch engine, although a limited amount of interactivity is possible when processing a file, to allow error recovery and diagnosis. Thus it *is* a page layout application.
It does not require applications like PageMaker, Quark Express, Fontographer or FontLab. A TeX system stands on its own, provided all the fonts one needs are available. TeX uses only the metrics, and produces a "device independent" output file - .dvi - that must then be translated to the particular output device being used (an imagesetter, laser printer, inkjet printer; in the "old days" even daisy-wheel printers were used). The DVI translator actually accesses the font shapes, either as bitmaps, Type 1 fonts, or pointers to fonts installed in a printer with the shapes not otherwise accessible. PostScript and PDF are two of the most popular "final" output forms for TeX.
One of the major areas where TeX holds its own is as a "back end" to SGML and XML systems, where no human intervention is expected between data input (structured, not WYSIWYG) and removing the output from the printer or viewing it on a screen. Granted, this isn't "creative" in the sense most often discussed, but it's still important to the readability and usefulness of such documents that care is taken with the design and typography, and the flexibility and programmability of TeX makes that possible.
As an aside, TeX can be the cause of religious wars. For those of us who need the capabilities of TeX - for production of books and journal articles in research mathematics - no other current composition tool, proprietary or otherwise, can handle the material and produce high-quality, publication-worthy output, and simultaneously be usable by the writer of the document. We'll be glad to provide test material to anyone who wants to prove us wrong. (It hasn't happened yet; the audience is much too small, and the problem too complex for a Microsoft or Quark or Adobe to be interested.) On the other hand, if you want a tool for producing a slick advertisement or a letter to Aunt Henrietta, unless you're already using it for something else, TeX is not the tool for you.
Donald Knuth, rewards the first finder of each typo or computer program bug with a check based on the source and the age of the bug. Since his books go into numerous editions, he does have a chance to correct errors. Typos and other errors in books typically yield $2.56 each once a book is in print (pre-publication "bounty-hunter" photocopy editions are priced at $.25 per), and program bugs rise by powers of 2 each year from $1.28 or so to a maximum of $327.68. Knuth's name is so valued that very few of his checks—even the largest ones—are actually cashed, but instead framed. (Barbara Beeton states that her small collection has been worth far more in bragging rights than any equivalent cash in hand. She's also somewhat biased, being Knuth's official entomologist for the TeX system, but informal surveys of past check recipients have shown that this holds overwhelmingly for nearly everyone but starving students.) This probably won't be true for just anyone, but the relatively small expense can yield a very worthwhile improvement in accuracy. (2008 update: Knuth no longer writes checks, but instead personal certificates of deposit to the Bank of San Seriffe.)
If you become known for this sort of interest in accuracy and reward for the first report, we suspect that it will attract readers to the material itself, as well as just to the finding of errors.
If you want to install a usable TeX for use on your computer, please consult the resources mentioned on the TUG home page.
If you want to inspect Knuth's own sources for educational or other such purposes, without any of the scaffolding and enhancements that have come to surround them in modern systems, you can get them from Stanford; the material is also mirrored on CTAN.