Troubleshooters.Com, T.C Linux Library and Litt's LyX Library Present

Steve Litt's Sure Fire,
No Fail Way to Create Lyx Layouts

Copyright (C) 2009 by Steve Litt, All rights reserved. Material provided as-is, use at your own risk.

Contents

Why I Wrote Document

The way you add custom environments and character styles to a LyX-supported document class is by creating a layout file. The way it works is that the layout file creates a new document class, based on the existing document class, and adds environments and character styles, not existing in the existing document class, to the new document class. Environments of the existing document class can also be modified for the new document class. LyX users find themselves in many situations. Here are two of the many:

  1. Many people don't create their own layout files. This web page is not of interest to them.
  2. Many people create their own layout files but never have trouble connecting them to an existing or new LyX document. This web page is not of interest to them.

If you fit into either of the preceding two  categories, you needn't read farther. If you use layouts and have had trouble incorporating them into documents in the past, read on, you'll probably find the solution.

The Symptoms

You make a layout file. You go into LyX and click Tools->Reconfigure, just like you should. You exit LyX and rerun it, just like you should. You click Document->Settings->Document_Class and look at the list of document classes, and the new document class you just implemented with the new layout isn't in the list. Or maybe it's in there twice. You repeat the steps, over and over and over again, and maybe finally it shows up, and maybe it doesn't. After months or years, it starts appearing like the ability to link a layout with a document class is an intermittent problem that you keep solving by trial and error or dumb luck. What the heck is going on?

Worse yet, a document class stops working or starts showing up twice, and you find your document reverting to the Article document class, so that you lose not only your hand built environments, but the Chapter environment also. You have to find a way to re-connect the layout file with the document, and then you have to spend many gruelling hours manually restoring all the environments that reverted to Standard. It's a horror.

The Underlying Problem

If you've experienced problems like those described in the The Symptoms section, don't worry, you're not alone. There are some definite causes and some definite solutions.

Different LyX versions looked for layout files in different places. Older LyX versions looked for them only in the layouts directory located below your personal LyX data directory, which is typically /home/myself/.lyx. So they typically looked for layout files in /home/myself/.lyx/layouts. Basically, any correctly coded layout file in that directory, having a .layout extension, appears on the list of document classes when you mouse to Document->Settings->Document_Class.

Of course, the user LyX data directory is not always /home/myself/.lyx. It could be something like /home/myself/.lyx-1.5.6.  The exact directory is established at installation time, and can be observed by running LyX, choosing Help->About_LyX, and observing the Credits tab of the resulting popup. It will probably look something like this:
LyX Version 1.5.3 (Mon, Dec 17, 2007)
Library directory: /usr/local/share/lyx-1.5.3
User directory: ~/.lyx-1.5.3

In the preceding, the user's LyX data directory is /home/myself/.lyx-1.5.3. Note that other LyX versions may require other procedures to find out this directory.

Anyway, besides the LyX user data directory, newer LyX versions also look in the current directory for .layout files. It makes sense, because a lot of people like to have one and only one layout file for each LyX document, and each LyX document is placed in its own directory, together with its supporting documents (images and the like).

The trouble is, some LyX versions do a better job than others of locating the local directory layout file. The plot thickens when a local layout file has a symlink in the user LyX data directory. The plot thickens even more when you're not sure which is the right LyX user data directory, and thickens even more when you're not sure you've correctly coded the layout file.

A huge source of problems is eliminated if no files in the current directory have .layout extensions. Then the only place that's searched is LyX user data directory. There's only one problem: If you're anything like me you want each layout file in the same directory as the document it's created for.

The solution is simple enough. Put the layout file in the current directory, but don't use the .layout file extension. Then symlink it to a .layout name in the LyX user data directory, and LyX will find the symlink, and refer to the real file.

The solution I've come up with, which works every time for me, is the following in order to make new docuument class myclass:
  1. In the current directory, create an ultra simple document class, based on existing document class book, in a file called myclass_layout.local.
  2. In your LyX user data directory, create a symlink, named myclass.layout, which refers back to the current directory's myclass_layout.local.
  3. In the current directory, create a half page test file called test.lyx, and verify that you can apply document class myclass to it.
  4. Continue adding to myclass_layout.local., checking frequently with test.lyx to make sure things are going right.
  5. When the layout file is where you want it, it should be very easy to apply it to your real LyX document (call it mybook.lyx, for instance.)
The preceding is a simplification, so the next two sections will be a tutorial of this method.

Tutorial: Getting Your Document to Recognize a Trivial Layout File

Make $lyxdata point to the user LyX data directory

This step is just so this tutorial can be taken by all people, regardless of the directory name of their LyX user data directory. As mentioned before, the LyX user data directory can be found by clicking Help->About_LyX from within LyX. $lyxdata will contain only the part of the directory below your home directory. So, if your LyX user data directory is the typical ~/.lyx, then you'll set $lyxdata like this:
lyxdata=.lyx;export lyxdata
If, on the other hand, your LyX user data directory is ~/.lyx1.5.6, you'd set it like this:
lyxdata=.lyx1.5.6;export lyxdata
Be sure to do all further steps and run all further programs from the terminal or terminal emulator in which you typed the command to set $lyxdata, because without a properly set $lyxdata, this tutorial won't work.

Before proceeding, test that $lyxdata is really set using the following command:
echo $lyxdata
The preceding command should print the part of your LyX data directory name that's below ~/, so it should be .lyx, .lyx1.5.6, .lyx-1.5.3, or maybe .mylyxdata, or wherever your user LyX data resides.

Make the test file

Make the layout file locally

Symlink the layout file

Run the LyX reconfigure script

Connect the layout file to the test file

Connect the layout to another document

Once a layout has been properly constructed and symlinked from the layouts directory below the LyX user data directory, that layout should be available to any LyX document edited by that version of LyX. Therefore, you must verify that it works not only with mybook.lyx, but with any document. Do this...

If It Doesn't Work...

This tutorial has been tech edited with LyX 1.5.6 on Mandriva 2008 32 bit Linux. If this tutorial didn't work, it's probably because of you made a typo, specified a wrong directory, didn't copy the contents of the layout file exactly right, or the like. Check the following:

What This Tutorial Did and Didn't Accomplish

This tutorial demonstrated exactly how to create a trivially simple layout file and unerringly hook it to any LyX document. What it did not accomplish was explaining how to repeatedly code and test a layout file. Read on...

Tutorial: Repeatedly Coding and Testing a Layout File

Layout coding and testing has several hassles.

Hassle Number One

Hassle number one is the fact that, for a layout file change to be recognized by LyX, the LyX program must be run anew. The preferred way to do this is to exit all sessions of LyX, and run the program anew. You could also simply run the program anew with the original session still running, in which case the old session would have the old layout, and the new session would have the new layout. This could become a troubleshooting nightmare if you forgot which was which. A third way is to run LyX from a script. More on that later.

Hassle Number Two

If anything with filenaming of the layout file changes, you must reconfigure LyX and then start it anew. This is also true if you change the document class name within the layout file. For instance, changing this:
#  \DeclareLaTeXClass[book]{mybook}
to this:
#  \DeclareLaTeXClass[book]{myboogie}
would require a LyX reconfigure.

I don't know for sure, but there may be other situations in which reconfiguration is required to recognize changes. If you inadvertently troubleshoot something requiring LyX reconfiguration, and don't reconfigure, it becomes a troubleshooting nightmare as you try ploy after ploy to get your changes recognized, when in reality they'll never be recognized until you reconfigure.

Hassle Number Three

At any time, a document class that becomes unavailable for any reason, can cause all environments not included in the article class to be dropped from the document -- converted to the Standard environment. If this happens, it can take hours or even days to manually re-mark all paragraphs with their intended environments.

This is a huge reason to use only small test documents, rather than your almost-finished 300 page book, while coding and testing layouts.

Hassle Number Four

Filenames, directories and symlinks must be just right to assign your layout-created document class to a document. If your symlink is supposed to go in /home/myself/.lyx/layouts, but instead goes in /home/myself/.lyx or /home/myself/.lyx-1.5.3/layouts, the document class won't be accessible from the list of document classes, or it may remain accessible and useable until the next reconfigure or next running of a new LyX session.

Hassle Number Five

One misplaced brace, End, or any other syntax error within a layout file, can cause the layout file to silently fail. The first you know about it is when you run LyX on a LyX file having the document class created by the layout file, and LyX tells you it's reverting to the default document class. If you EVER see that error message, DO NOT save the file under its original name or you'll lose all your environments (see hassle number 3). Instead, save it to a brand new name, and then troubleshoot the layout file, testing with a disposable test file.

Hassle Number Six

Leaving and returning to LyX, loading the document and testing it by compiling it to a PDF takes a heck of a lot of time on a large document, like a 300 page book.

Hassle Number Seven

There are occasions on which, while trying to assign a document class to a document, you'll see the desired document class appear twice in the document class list. If that happens, be very afraid. This is very likely to precede a loss of environments (see hassle number 3)

Hasslebusters

You don't need the aggravation described in the hassles subsection. Use these hasslebusters to minimize the aggravation.

Hasslebuster Number One: Use a Small LyX Test File

Use only a small LyX test file to test changes in a layout file. NEVER use a large LyX document, especially a valuable one. Your LyX file can lose its environments, which is a tragedy on a valuable large file, but almost insignificant on a small test lyx file. Also, the exit/lyx/compile cycle takes much less time with a small test LyX file.

Hasslebuster Number Two: Reconfigure and Run a New LyX Every Layout Change

A layout file change will not be recognized by a LyX session started before the layout change. Therefore you must run a new LyX. By far the safest way to do that is to exit all current LyX sessions, and start a new one.

Most layout activities don't require a LyX reconfigure, but a few do. If you fail to reconfigure when you should have, it creates a troubleshooting nightmare. Although it takes more time to reconfigure every time, doing so will make you confident of your troubleshooting diagnostic tests.

Hasslebuster Number Three: Troubleshoot Immediately If a Document Class Lists Twice

There are occasions on which, while trying to assign a document class to a document, you'll see the desired document class appear twice in the document class list. If that happens, do not save your good document under its usual name -- save it under a new name in case it has lost its environments. Then troubleshoot the multiple listing of the document class, and get rid of the problem before it gets rid of your good document's environments. Here are a few possible causes:
Whether the problem is caused by one of the preceding or something else, fix it before continuing to work on your good document.

Hasslebuster Number Four: Use a Script

The preceding hassles and hasslebusters all point to one thing -- when you're developing and debugging a layout file, use a script to compile and test. It will save you tons of time and aggravation. When the script's debugged, THEN you can attach it to your real document instead of scripting through the test document.

Earlier we discussed reconfiguring LyX from inside the LyX program. What you want to do is run the reconfiguration from  a script. LyX comes with a program called configure.py, located in your system's LyX library directory. You can find the LyX library directory from inside LyX with Help->About_LyX. On my system right now, that directory is /usr/local/share/lyx1.5.6, so the path to the configure.py script is /usr/local/share/lyx1.5.6/configure.py. Running this script is just like pressing Tools->Reconfigure from within LyX.

Assuming your LyX test document is ~/mybook/test.lyx, and your reconfigure script is /usr/local/share/lyx1.5.6/configure.py., here's a good script to use while coding and debugging a layout file:
#!/bin/bash

libdir=/usr/local/share/lyx1.5.6	# Change to match your system
userdir=~/.lyx1.5.6			# Change to match your system
pdfreader=kpdf                          # What pdf reader you use
                                        #  Change to match your system

lyxdoc=$1                               # file to compile, without .lyx

cd $userdir
$libdir/configure.py  # Reconfigure LyX
pwd
cd -


##### PREDELETE ANY INTERMEDIATE FILES #####
rm -f $lyxdoc.dvi
rm -f $lyxdoc.ps
rm -f $lyxdoc.pdf

lyx --export pdf $lyxdoc.lyx

if ls $lyxdoc.pdf; then
	$pdfreader $lyxdoc.pdf
	echo COMPILE SUCCEEDED!
else
	echo COMPILE FAILED, TROUBLESHOOT!
fi


Notice the script switches to the LyX user data directory, then runs LyX's configure.py, then switches back to the original directory. The configure.py script, when run in the LyX user data directory, does the same thing as Tools->Reconfigure from within LyX. Such reconfiguration is necessary whenever the name of the document class is changed, and may be necessary other times, so the script does it always.

The argument of the preceding script shoud be the filename of the LyX document to be compiled, but without the .lyx extension. If you want to make this script more robust, you can use the sed and basename commands to enable it to work either with or without the .lyx extension.

Another enhancement you could make is to have the script export the LyX doc to LaTeX, and then compile the LaTeX file. That would give you exact error messages. For simplicities sake, I didn't put in these enhancements.

I name this script jj so that I can quickly and easily invoke it like this:
./jj mybook
Because compiling is done so often when debugging a layout file, the shorter and more memorable the command, the faster you work.

Start with mybook_layout.local

The following is the mybook_layout.local discussed previously:
#% Do not delete the line below; configure depends on this
#  \DeclareLaTeXClass[book]{mybook}

Input stdclass.inc

Preamble

EndPreamble

Style MyTest
  CopyStyle  Chapter
End

Add Story environment

Now we'll add an environment, calling it story:
#% Do not delete the line below; configure depends on this
#  \DeclareLaTeXClass[book]{mybook}

Input stdclass.inc

Preamble

EndPreamble

Style MyTest
  CopyStyle  Chapter
End

Style Story
  CopyStyle Standard
End
The preceding does nothing but install a style named Story, that's a clone (CopyStyle) of the standard environment. Run the jj script against mybook , go into LyX, and verify that the Story environment is now available. Assign the Story environment to a paragraph and recompile to PDF. You'll notice the Story paragraph looks the same as standard text, both in the LyX environment and in the final PDF. This is because it's a clone of Standard. It's up to you to make it look different...

Change the Appearance Within LyX

In my books, I like stories to appear in italic, with wider margins (narrower text). The margins within the LyX environment can be altered with LeftMargin and RightMargin, while the font can be modified with the Font/EndFont directive, as follows:
#% Do not delete the line below; configure depends on this
#  \DeclareLaTeXClass[book]{mybook}

Input stdclass.inc

Preamble

EndPreamble

Style MyTest
  CopyStyle  Chapter
End

Style Story
  CopyStyle Standard
  LeftMargin        "MMMMMM"
  RightMargin        "MMMMMM"
  Font
    Shape               Italic
  EndFont
End

The changed margins and fonts within the LyX environment are marked in red in the preceding listing. The margins are indented the width of five upper case M characters in whatever font is being used. The font shape is changed to italic. Because of the CopyStyle command, everything else is identical to the Standard environment.

When you pull up mybook.lyx in LyX, the paragraphs marked with the Story environment are indented and italic. At this point, you can intelligently use the Story environent, because it's instantly recognizeable within the LyX environment, even though it still appears as the Standard environment when you compile it to PDF.

The beauty of changing only the LyX appearance is it's quick and easy to do, unlike the LaTeX coding necessary to change the actual output. That means you can get back to your writing quickly, and later, when you have time, get the output to look just right. The next section discusses changing the appearance of the environment in the output...

Change the Appearance in the Output

The whole purpose of an environment is to change the look of the paragraph in the output (PDF or Postscript). To do that, you must code a LaTeX environment that changes the output appearance, and then reference that LaTeX environment within the LyX style, as follows:
#% Do not delete the line below; configure depends on this
#  \DeclareLaTeXClass[book]{mybook}

Input stdclass.inc

Preamble

\newenvironment{storyL}{%
  ~\\
  \leftskip .5in
  \rightskip .5in
  \it
}{
  \par
  ~\\
}

EndPreamble

Style MyTest
  CopyStyle  Chapter
End

Style Story
  CopyStyle Standard
  LatexType             Environment
  LatexName             storyL
  LeftMargin        "MMMMMM"
  RightMargin        "MMMMMM"
  Font
    Shape               Italic
  EndFont
End
The storyL LaTeX environment starts and ends with newlines (~\\) to set it off from the environments before and after it. It alters the margins with the \leftskip and \rightskip commands, and sets the font shape to italics with the \it command. The \par command in the closing part of the environment is necessary so the indentation applies to the final paragraph marked with the Story environment.






Two lines are added to the LyX style: LatexType and LatexName. The latter links the LyX style with the LaTeX environment. The former identifies the LyX style as an environment rather than a command or other LyX style type.

In the preceding code, you'll notice that the LaTeX environment goes between the Preamble and EndPreamble statements toward the beginning of the layout file. You could also physically group the LyX style and LaTeX environment like this:

#% Do not delete the line below; configure depends on this
#  \DeclareLaTeXClass[book]{mybook}

Input stdclass.inc

Preamble

EndPreamble

Style MyTest
  CopyStyle  Chapter
End

Style Story
  CopyStyle Standard
  LatexType             Environment
  LatexName             storyL
  LeftMargin        "MMMMMM"
  RightMargin        "MMMMMM"
  Font
    Shape               Italic
  EndFont
  Preamble
    \newenvironment{storyL}{%
      ~\\
      \leftskip .5in
      \rightskip .5in
      \it
    }{
      \par
      ~\\
    }
  EndPreamble
End

In the preceding, you see it's really nice to have all code related to the Story environment in one place. I don't do it that way. I put all my LaTeX in one Preamble/EndPreamble section toward the beginning of the layout file, even though that means I must refer from the LyX style to the LaTeX environment which is sometimes hundreds of lines before it. You might wonder why.

First, my experience is that if I have enough Preamble/EndPreamble pairs, I'll make an error with them, and such errors are very hard to troubleshoot. Secondly, if you use CopyStyle, embedding Preamble/EndPreamble sections within the LyX Style can cause compile errors, but if instead the LaXeX environment is coded outside of the LyX style, everything works. The following is a link to the email thread where this is discussed, and you'll notice in the end we had to agree to disagree :-).

http://thread.gmane.org/gmane.editors.lyx.general/48876/focus=48879

Summary

This document could have gone much farther, showing various environments and character styles, but the purpose of the document is to demonstrate a surefire way to get a layout file to work. That has been done.

The surefire way is to code the layout file, in the local directory, as mybook_layout.local, then symlink it as mybook.layout in the LyX user data directory. Initially the file should be as trivial as possible, because just getting LyX to recognize a new layout file can be tricky.

Once the layout file is recognized, new environments and other things can be added. This document describes a handy script to reconfigure LyX and compile the document to PDF. This script makes coding and debugging a layout file much quicker.

When adding a style, the first step is to add a LyX style that uses CopyStyle to copy an existing style. Next you modify the style so that you get a different appearance within the LyX environment. Once you've done that, you can actually start using the new environment because it's distinctive within the authoring environment. Getting this far is trivially easy, so if you're busy writing content, you can make a LyX environment quickly and get on with your writing, postponing the coding of the PDF appearance for later.

Coding the PDF appearance of an environment involves creating a LaTeX environment within a Preamble/EndPreamble pair, and then referencing that LaTeX environment within the LyX style with a LatexName command.

Working with layout files can be tricky, time consuming and error prone, but the methods discussed in this document make it much easier.


Back to Troubleshooters.Com * Back to Linux Library * Litt's LyX Library