Lore

Lore - A Document Generation System

Gimmick -- Gilmore girls quotes
Goal - take something which is easy to write, transforms to something easy to read
For correct definitions of 'easy', of course

Rory (Concert Interruptus, season 1) -- Yeah, well I've always thought easy is completely overrated.

Source Format

Subset of XHTML 1.0
- Except for some new attributes
- Shouldn't bother browsers
Slanted towards logical markup

Alex (I Solemnly Swear, season 3) -- That would've been far too logical.

Output Formats

Screen and paper
- Screen - 'fancy HTML'
- Paper - LaTeX
  - Use LaTeX to produce PDF or PostScript

Madelaine (The Lorelais' First Day at Chilton, season 1) -- You don't know she's going out for the paper.

Minimal Lore Document

<html>
<head><title>Title</title></head>
<body><h1>Title</h1></body>
</html>

Luke (There's the Rub, season 2) -- You said minimal

Minimal Lore Document Explained

title element in head -- a must
h1 element in head -- a must
- Must have same content

Tom (There's the Rub, season 2) -- Hey, this is minimal

External Listings

Advantage -- no need to quote
Advantage -- test your examples
Example:
- <a class="python-listing" href="/usr/lib/python2.2/os.py">os.py</a>

Kirk (Red Light on the Wedding Night, season 2) -- I include it as an example of the excellence I aspire to.

Using Lore to Generate HTML

Write template
[optional] Write stylesheet
Run lore

Paris (Run Away, Little Boy, season 2) -- I went on the web and found this site

Generating LaTeX

lore -olatex file.html --> produces file.tex
Default is to create an 'article'
Creating PostScript
- latex file.tex
- latex file.tex
- dvips -o file.ps file.dvi
Creating PDF
latex file.tex
pdflatex file.tex

Rory (Christopher Returns, season 1) -- He had already printed like a million

Using Lint

lore -olint doc/howto/*.html
lore -n -olint doc/howto/*.html #no output except warnings

Max (The Deer-Hunters, season 1) -- I know a D seems pretty dismal

Questions?

Lorelai (Forgiveness and Stuff, season 1) -- A person needs details.

Bonus Slides

Miss James (The Lorelais' First Day at Chilton, season 1) -- If you do it in Latin you get extra credit.

Lore Alternatives - LaTeX

Very good at printed results
Model makes alternative parsers near-impossible
Renderers to HTML are buggy and fragile
People find it hard to use

Michel (Love, Daisies and Troubadors, season 1) -- It increases my ennui

Lore Alternatives - HTML

Too flexible
No support for needed idioms
- Special-purpose Python markup
- Tables of contents
- Inlining
- Footnotes
Renders badly to dead trees with current tools

Lorelai (Love, Daisies and Troubadors, season 1) -- It was broken [...] I'm not crazy

Lore Alternatives - Docbook

Using correctly requires too much work
- Write a DTD with special elements
- Write Jade stylesheets
Lore is probably smaller than docbook specialisation
People find it hard to use

Rory (Hammers and Veils, season 2) -- What do you want me to do it?

Lore Alternatives - Texinfo

Next slide, please

Man (Hammers and Veils, season 2) -- There's a ton of hurt that almost happened here.

Lore Alternatives - reST

Completely new language (no editor support)
Hard to add new tags
No linter

Emily (Hammers and Veils, season 2) -- And this is what we need to discuss right now?

Lore Alternatives - LyX

Dependent on GUI

Rory (Hammers and Veils, season 2) -- Well, it's just dressed up a little.

Some Standard Tags -- XHTML Primer

<p>paragraph</p>
<em>emphasis</em>
<strong>strong emphasis</strong>
Headers
- <h2>sectionheader</h2>
- <h3>subsection</h3>
Lists
- <ol><li>ordered list item</li></ol>
- <ul><li>unordered list item</li></ul>
<img src="http://example.com/img.png" />
- Note '/' at end!

Rory (Kiss and Tell, season 1) -- See, even a little information in your hands is dangerous.

More HTML

Indicating authorship -- <link rel="author" href="author@example.com" title="Author Name" />
Put in <head>
sub/sup -- subscripts, superscripts

Max (The Lorelais' First Day at Chilton, season 1) -- Tolstoy's favourite author, for instance, was...

More HTML -- cross references

Label
Reference in file
- <a href="#label-name">reference text</a>
Reference in other file
- <a href="file-name#label-name">reference text</a>
Refer to URL
- <a href="http://example.com">reference text</a>

Christopher (Christopher Returns, season 1) -- It's just a weird reference.

Special Markup

Things not in XHTML are done with div/span classes
<div class="note">note</a> -- notes
<div class="doit">doit</a> -- something not implemented
<span class="footnote">footnote</a> -- put in a footnote

Taylor (Take The Deviled Eggs, season 3) -- Out attention spans are gnat-like tonight

API References

<code class="API">urllib</code>
<code base="urllib" class="API">urlencode</code>
<code base="twisted" class="API">copyright.version</code>

Lorelai (The Road Trip To Harvard, season 2) -- We're just kinda hanging out between classes

API References Explained

Integrate with systems for docstring generation
Generate links to auto-generated docs

Luke (Love and War and Snow, season 1) -- How do you know? Do you have written documentation?

Inline Listings

Use <pre>
Possible classes: python, shell, python-interpreter
Example:

<pre class="python">
def foo():
    return forbnicate(4)
</pre>

Taylor (Take The Deviled Eggs, season 3) -- That's not even English.

Inline Listings -- short

Use <code>
Possible classes: python, shell, py-signature
...and more

Rory (Double Date, season 1) -- It's like this weird code thing with her.

Generating HTML -- writing templates

Templates are XHTML documents
Put in reference to stylesheet -- head is mostly kept as is
Title will be prepended to document's title
<div class="toc" /> will be replaced by table of contents
<div class="body" /> will be replaced by processed body

Paris (I Can't Get Started, season 2) -- How's this sound for a template?

Generating HTML -- using commandline

Full details: the lore manpage
Basic format: lore file.html --> outputs file.xhtml
- --config template=template.tpl to use different template
- --config baseurl=format-string for the url of the auto-generated docstring docs

Richard (The Third Lorelai, season 1) -- Your wish is my command.

Generating HTML -- using commandline -- examples

lore --config template=strange.tpl foo.html
lore --docsdir doc/howto/
lore -p --docsdir doc/howto/ # use plain progress

Jackson (A Deep-Fried Korean Thanksgiving, season 3) -- Deep-fried cake!

Generating HTML -- using commandline -- examples (cont'd)

lore --docsdir doc/howto/ --config baseurl=../api/%s.html
lore --ext='' foo.html # produce 'foo' as output

Jackson (A Deep-Fried Korean Thanksgiving, season 3) -- Deep-fried shoe!

Generating HTML -- notes about stylesheets

Many 'class's in the output
The stylesheet Twisted uses can be used as example
- Especially the .py-src-* classes: used for syntax highlighting

Miss Patty (Cinnamon's Wake, season 1) -- If you had a better hair style I might consider dating

Generating LaTeX -- examples

lore -olatex --config section file.html
lore -olatex --config book file.html
lore -olatex --config section --docsdir doc/howto/

Luke (Hammers and Veils, season 2) -- Just an example

Using Lint -- notes

If there is an element which lint gives a warning you disagree with: <element hlint="off">mistake</element>
But usually the linter is right
lint exits with non-zero status iff some document was not clean -- useful in shell scripts

Paris (The Deer-Hunters, season 1) -- That would be cause for concern.

Understanding Lint Warnings

Format: file:line:column:warning
Line/column always point to start/end of element
Some justifications:
- <pre> with >80 characters/line renders badly, both in HTML and in LaTeX

Jess (Teach Me Tonight, season 2) -- I appreciate the warning.

Using Lore For Slides

lore -ilore-slides -omgp file.html for magic point
lore -ilore-slides -oprosper file.html for prosper
lore -ilore-slides -ohtml file.html for HTML next/prev
Splits on 'h2'
Dogfooding

Emily (Road Trip to Harvard, season 2) -- Why in the world do you insist on taking slides?

Extending Lore

Accept more input tags
Change how documents are processed
Add more output formats

Rory (The Lorelais' First Day at Chilton, season 1) -- Well, add a couple of plaid skirts

Extending Lore -- example

We want to add a way to blink: <span class="blink">
Modify HTML output
Modify lint output
Make it 'small caps' in LaTeX
Distribute as package 'blinker'

Lorelai (Presenting Lorelai Gilmore, season 2) -- No, no, if you wanna do it, I'll help. It's just weird.

Extending Lore -- example (cont'd)

# blinker/html.py
from twisted.lore import tree
from twisted.web import microdom, domhelpers

def doBlink(document):
    for node in domhelpers.findElementsWithAttribute(document, 'class',
                                                    'blink'):
        newNode = microdom.Element('blink')
        newNode.children = node.children
        node.parentNode.replaceChild(newNode, node)

def doFile(fn, docsdir, ext, url, templ, linkrel=''):
    doc = tree.parseFileAndReport(fn)
    doBlink(doc)
    cn = templ.cloneNode(1)
    tree.munge(doc, cn, linkrel, docsdir, fn, ext, url)
    cn.writexml(open(os.path.splitext(fn)[0]+ext, 'wb'))

Christopher (Presenting Lorelai Gilmore, season 2) -- I can't believe you're letting her do it.

Extending Lore -- example (cont'd 2)

# blinker/latex.py
class BlinkerLatexSpitter(latex.LatexSpitter):

    def visitNode_span_blink(self, node):
        self.writer('{\sc ')
        self.visitNodeDefault(node)
        self.writer('}')

Lorelai (Presenting Lorelai Gilmore, season 2) -- I'm sorry, I meant what scenario on my planet

Extending Lore -- example (cont'd 3)

# blinker/factory.py
from blinker import html, latex
from twisted.lore import default

class ProcessingFunctionFactory(default.ProcessingFunctionFactory):

    doFile = [doFile]

    latexSpitters = {None: latex.BlinkLatexSpitter}

    def getLintChecker(self):
        checker = lint.getDefaultChecker()
        checker.allowedClasses = checker.allowedClasses.copy()
        oldSpan = checker.allowedClasses['span']
        checker.allowedClasses['span'] = (lambda x:oldSpan(x) or
                                                     x=='blink')
        return checker

factory = ProcessingFunctionFactory()

# blinker/plugins.tml
register("Blink-Lore",
         "blinker.factory",
         description="Lore format with blink",
         type="lore",
         tapname="blinklore")

...and that's it!

Rory (Presenting Lorelai Gilmore, season 2) -- Sorry, we haven't tamed my wild ways yet.

Man page support

No output
Man->Lore conversion
lore -iman -olint file.1 --> generates file.html

Lorelai (Concert Interruptus, season 1) -- would you like to write out some sort of instruction manual to go with the dishes?

Man page support

No output
Man->Lore conversion
lore -iman -olint file.1 --> generates file.html

Lorelai (Concert Interruptus, season 1) -- would you like to write out some sort of instruction manual to go with the dishes?

Lore

Lore - A Document Generation System

Source Format

Output Formats

Minimal Lore Document

Minimal Lore Document Explained

External Listings

Using Lore to Generate HTML

Generating LaTeX

Using Lint

Further Reading

Questions?

Bonus Slides

Lore Alternatives - LaTeX

Lore Alternatives - HTML

Lore Alternatives - Docbook

Lore Alternatives - Texinfo

Lore Alternatives - reST

Lore Alternatives - LyX

Some Standard Tags -- XHTML Primer

More HTML

More HTML -- cross references

Special Markup

API References

API References Explained

Inline Listings

Inline Listings -- short

Generating HTML -- writing templates

Generating HTML -- using commandline

Generating HTML -- using commandline -- examples

Generating HTML -- using commandline -- examples (cont'd)

Generating HTML -- notes about stylesheets

Generating LaTeX -- examples

Using Lint -- notes

Understanding Lint Warnings

Using Lore For Slides

Extending Lore

Extending Lore -- example

Extending Lore -- example (cont'd)

Extending Lore -- example (cont'd 2)

Extending Lore -- example (cont'd 3)

Man page support

Man page support