LisTeX
Table of Contents
1. Introduction
This packages comes from the ocasional frustration of writing
latex in org-mode
and having a sane default formating
of the equations. Also it comes from the lack of frustration
formating and writing s-expressions in Lisp.
Let's say, it originates in a "frustration transfer".
This package is written as a literate program cite:knuth. If you want to contribute please consider writing in the style of the document.
You can see a rendered version of this document here.
1.1. TODO Philosophy
This package should be as simple as possible while providing good ergonomics (at least for me).
So, the main idea is that the user must provide
just a list that will be converted to a latex string
through the listex
function.
This means, only a list is given and every component of the list can be different kind of types recognized by the 5 section.
This package has the following concepts:
- command
- This should be the equivalent to latex commands, i.e.,
a latex symbol and some braces expressions denoting arguments,
for instance
\frac{A}{B}
rather than\frac A B
. In our DSL, this is denoted by a-
prefix, i.e. this frac would be'(-frac A B)
. Be aware that listex has no sense of the latex information for thefrac
command, i.e.'(-frac A B C D)
is allowed and gets rendered as\frac{A}{B}{C}{D}
. - keyword
- This is the counterpart to command, i.e. just latex keywords.
This can be at the beginning of the list of in the middle,
and gets introduced by the prefix
/
. For instance,'(/frac /alpha 2)
⇒\frac \alpha 2
'(-frac /alpha 2)
⇒\frac{\alpha}{2}
- operator
- TODOCUMENT
- lisp-macro
- TODOCUMENT
- tex-macro
- TODOCUMENT
- alias
- TODOCUMENT
- atom
- TODOCUMENT
- list
- TODOCUMENT
1.2. What is to be done
- Some real world testing
- to find bugs
- to finalize the DSL, right now I think it's quite ergonomic, but it might be wrong.
- Think about which rules to implement for the latex indentation.
2. org-src blocks
In org mode we want to have easily accessible code blocks like
#+begin_src listex :env al* :label dr-dv (let ((d '(-mathrm d))) `(-frac (,d r) (,d v))) #+end_src #+RESULTS: :results: \begin{align*} \label{dr-dv} \frac{\mathrm{d} r}{\mathrm{d} v} \end{align*} :end:
We have beyond the usual flags, the following:
- :label
- This is the latex label that should be used included in the expression automatically. Also a local lisp variable will be created with the same name so that one can reuse the expression somewhere else.
- :lisp-label
- Sometime we want to reuse the expression
but we do not want a tex label, so
:lisp-label
sets the lisp variable to the expression as in:label
but then no tex label is necessary. - :env
- It is possible to wrap the expression in an environment
before the expression gets rendered to latex.
Possible values are for instance
$, $$, eq, eq*, al, al*
etc.
(defvar org-babel-default-header-args:listex '((:exports . "results") (:results . "value drawer") (:eval . "t"))) (defvar org-babel-header-args:listex '((:label . :any) (:lisp-label . :any) (:env . :any) (:eval . :boolean)) "listex-specific header arguments.") (defun org-babel-execute:listex (body params) (cl-labels ((maybe-symbol (key) (let ((maybe-string (cdr (assq key params)))) (when maybe-string (intern maybe-string))))) (let* ((should-eval (maybe-symbol :eval)) (raw-expr (car (read-from-string body))) (expr (if should-eval (eval raw-expr) raw-expr)) (env (maybe-symbol :env)) (label (maybe-symbol :label)) (lisp-label (or (maybe-symbol :lisp-label) label))) (message "penis %s %s" should-eval (type-of should-eval)) (when lisp-label (message "new lisp-label defined: %s" lisp-label) (eval `(setq-local ,lisp-label ',expr))) (listex (if env `(,env ,(if label `((-label ,label) ,expr) expr)) expr)))))
3. Derived mode
We define a derived mode based on the emacs-lisp
mode
to make it easy to edit the s-expressions with your preferred bindings.
(define-derived-mode listex-mode emacs-lisp-mode "LiSTeX" "Major mode for listex. \\{listex-mode-map}")
4. Misc
(defvar listex-keyword-prefix "/") (defvar listex-command-prefix "-") (defvar listex-operator-prefix "%") (defvar listex-operator-list '(+ * - =)) (defun listex:indentation (len) (eval `(concat ,@(cl-loop for i from 1 to len collect " "))))
5. Types
(cl-deftype listex:keyword () '(and symbol (satisfies (lambda (k) (string-prefix-p listex-keyword-prefix (format "%s" k)))))) (cl-deftype listex:car-is-atom () '(satisfies (lambda (expr) (atom (car expr))))) (cl-deftype listex:command () '(and list listex:car-is-atom (satisfies (lambda (expr) (let ((name (format "%s" (car expr)))) (string-prefix-p listex-command-prefix name)))))) (cl-deftype listex:operator () '(and list listex:car-is-atom (satisfies (lambda (expr) (let ((name (car expr))) (or (member name listex-operator-list) (string-prefix-p listex-operator-prefix (format "%s" name)))))))) (defmacro listex:lisp-macro-get-fun (name) `(alist-get ,name listex-lisp-macro-alist)) (cl-deftype listex:lisp-macro () '(and list listex:car-is-atom (satisfies (lambda (expr) (listex:lisp-macro-get-fun (car expr)))))) (defvar listex-alias-alist nil "Alist holding all the aliases.") (defmacro listex:get-alias (name) `(alist-get ,name listex-alias-alist)) (cl-deftype listex:alias () '(and symbol (satisfies (lambda (expr) (listex:get-alias expr)))))
6. Macros definition
6.1. Implementation
(defvar listex-lisp-macro-alist nil "Alist storing all listex macros that are defined.") (defmacro listex:lisp-macro-alist-pair (alist key args list-or-fun) `(list '(alist-get ',key ,alist) ,(cl-etypecase list-or-fun (function list-or-fun) (list `(cl-flet ((f ,args ,list-or-fun)) (cl-function f)))))) (defmacro listex:defmacro (key !args list-or-fun) `(let ((args (listex:lisp-macro-alist-pair listex-lisp-macro-alist ,key ,!args ,list-or-fun))) (eval `(setf ,@args)))) (defmacro listex:newcmd--format-function (args fmt) `(format ,fmt ,@(cl-loop for a in args ;; make sure that a is not a & identifier ;; for functions like &rest if (not (string-prefix-p "&" (symbol-name a))) collect `(listex:render-tex ,a)))) (defmacro listex:newcmd (key args fmt) `(listex:defmacro ,key ,args (listex:newcmd--format-function ,args, fmt)))
and for the aliases we define defalias
(defmacro listex:defalias (alias key) `(setf (alist-get ',alias listex-alias-alist) ',key))
6.2. TeX macro definitions
This package defines some macros by default for use in the src code and for ease of use for others.
;; important macros (listex:newcmd braced (&rest body) "{%s}") (listex:newcmd progn (&rest body) "%s") (listex:newcmd list (&rest body) "%s") ;; left right stuff (listex:newcmd lr (l r &rest body) "\\left%1$s %3$s \\right%2$s") (listex:defmacro lrp (&rest args) `(lr \( \) ,@args)) (listex:defmacro lrs (&rest args) `(lr \[ \] ,@args)) (listex:defmacro set (&rest args) `(lr /{ /} ,@args)) ;; quantum mechanics (listex:defmacro <| (&rest args) `(lr /langle | ,@args)) (listex:defmacro |> (&rest args) `(lr | /rangle ,@args)) ;; exponents (listex:newcmd ^ (base &rest sup) "%s^{%s}") (listex:newcmd _ (base &rest sub) "%s_{%s}") (listex:newcmd ^_ (base sup sub) "%s^{%s}_{%s}") (listex:newcmd _^ (base sub sup) "%s_{%s}^{%s}") ;; wrapping (listex:newcmd begend (b &rest bod) "%s%s%1$s") (listex:newcmd env (env-name &rest body) "\\begin{%1$s}\n%s\n\\end{%1$s}") (listex:defmacro mat (&rest args) `(env pmatrix ,@args)) ;; Math environments (listex:defmacro $ (&rest args) `(begend $ ,@args)) (listex:defmacro $$ (&rest args) `(begend $$ ,@args)) (listex:defmacro eq (&rest args) `(env equation ,@args)) (listex:defmacro eq* (&rest args) `(env equation* ,@args)) (listex:defmacro al (&rest args) `(env align ,@args)) (listex:defmacro al* (&rest args) `(env align* ,@args)) ;; force newlines in the output (listex:newcmd terpri () "\n") (listex:newcmd br () "\n") (listex:newcmd nl () "\n") ;; more convoluted example (listex:defmacro matrix (rows cols &rest elements) (progn (cl-assert (eq (length elements) (* cols rows))) `(env pmatrix ,@(cl-loop for el in elements with i = 0 with buff = nil do (push el buff) do (cl-incf i) if (eq (% i cols) 0) do (push '\\\\ buff) and collect (reverse buff) and do (setf buff nil) else do (push '& buff)))))
7. Render
7.1. Implementation
(defun listex:render-tex (expr) "Main function to convert a listex DSL s-expression into a latex-compatible string." (cl-etypecase expr (listex:lisp-macro (let* ((args (cdr expr)) (name (car expr)) (f (listex:lisp-macro-get-fun name)) (new-expr (apply f args))) (listex:render-tex new-expr))) (listex:alias (let* ((replacement (listex:get-alias expr))) (listex:render-tex replacement))) (listex:keyword (format "\\%s" (string-remove-prefix listex-keyword-prefix (symbol-name expr)))) (listex:command (let* ((args (mapcar (lambda (e) (cl-etypecase e (vector e) (otherwise (format "{%s}" (listex:render-tex e))))) (cdr expr))) (name (format "\\%s" (string-remove-prefix listex-command-prefix (symbol-name (car expr))))) (args-strings (mapcar #'listex:render-tex args))) (concat name (string-join args-strings)))) (listex:operator (let* ((name (car expr)) (namestr (symbol-name name)) (op (if (> (length namestr) 1) (string-remove-prefix listex-operator-prefix namestr) namestr))) (string-join (mapcar #'listex:render-tex (cdr expr)) (format " %s " op)))) (list (string-join (mapcar #'listex:render-tex expr) " ")) (atom (format "%s" expr)))) (defun listex (expr) (listex:render-tex expr))
8. Macrolet
In order to have a macrolet-like behaviour we need
to have an expander of our lisp-like macros, i.e.
of expressions of the type listex:lisp-macro
that are defined by the listex:defmacro
macro.
However, since a lexical scope environment will be destroyed
after the scope, we need to expand the forms inside the
equation by the macros that have been defined.
Thankfully this is not very difficult and is done
in the listex:expand-lisp-macro
function.
(cl-defun listex:expand-lisp-macro (expr &key recursive) "This function should expand all listex:lisp-macro s-expressions by the s-expression that they expand to, so that in some cases you can just get the whole. This works as it follows: - if an expression is a lisp-macro, then it will first expand its arguments and then return the expansion of the parent with the expansion of the arguments replaced. - If an expression is a command, tex-macro or an operator expression, then it will replace the same expression just with the elements replaced by their expansions. - Otherwise, it should replace just the bare expression." (cl-flet ((expander (e) (listex:expand-lisp-macro e :recursive recursive))) (cl-typecase expr (listex:lisp-macro (let* ((name (car expr)) (args (mapcar #'expander (cdr expr))) (f (listex:lisp-macro-get-fun name)) (new-expr (apply f args))) (if recursive (expander new-expr) new-expr))) (listex:alias (let ((new-expr (listex:get-alias expr))) (if recursive (expander new-expr) new-expr))) ;; expand the arguments ((or listex:command listex:operator) (let ((name (car expr)) (args (mapcar #'expander (cdr expr)))) `(,name ,@args))) (list (mapcar #'expander expr)) (otherwise expr))))
The lt-macrolet
will expand the macros defined in the
let body using listex:expand-lisp-macro
so that they
are portable outside of this environment and you do not need
to define globally the macros and pollute the listex
macro environment.
(defmacro listex:letconstruct (recursive pair-constructor alist bindings &rest body) (let ((letf-args (cl-loop for b in bindings collect (eval `(,pair-constructor ,alist ,@b))))) `(cl-letf (,@letf-args) (listex:expand-lisp-macro (progn ,@body) :recursive ,recursive)))) (defmacro listex:alias-alist-pair (alist key replacement) `(list '(alist-get ',key ,alist) ,(cl-etypecase replacement ((or atom cons) `',replacement)))) (defmacro lt-aliaslet (bindings &rest body) `(listex:letconstruct nil listex:alias-alist-pair listex-alias-alist ,bindings ,@body)) (defmacro lt-aliaslet* (bindings &rest body) `(listex:letconstruct t listex:alias-alist-pair listex-alias-alist ,bindings ,@body)) (defmacro lt-macrolet (bindings &rest body) `(listex:letconstruct nil listex:lisp-macro-alist-pair listex-lisp-macro-alist ,bindings ,@body)) (defmacro lt-macrolet* (bindings &rest body) `(listex:letconstruct t listex:lisp-macro-alist-pair listex-lisp-macro-alist ,bindings ,@body)) (defmacro lt-cmdlet (cmds &rest body) `(lt-macrolet ,(cl-loop for cmd in cmds collect (let ((key (car cmd)) (args (cadr cmd)) (fmt (caddr cmd))) `(,key ,args (listex:newcmd--format-function ,args ,fmt)))) ,@body)) ;; set indentation for lt-macrolet and other let constructs correctly (progn (put 'lt-aliaslet 'lisp-indent-function 'defun) (put 'lt-aliaslet* 'lisp-indent-function 'defun) (put 'lt-macrolet 'lisp-indent-function 'defun) (put 'lt-macrolet* 'lisp-indent-function 'defun) (put 'lt-cmdlet 'lisp-indent-function 'defun))
9. Tools
9.1. Preview listex
A tool for previewing the latex created might be very useful when working in src-blocks
(defun listex:render-last-sexpr (&optional not-eval?) (interactive "P") (let ((sexp (thing-at-point 'sexp))) (message (listex:render-tex (if not-eval? (read sexp) (eval (read sexp))))))) (defun listex:render-defun (&optional not-eval?) (interactive "P") (let ((sexp (thing-at-point 'defun))) (message (listex:render-tex (if not-eval? (read sexp) (eval (read sexp)))))))
you can use these by doing
(use-package listex :config (evil-define-key '(insert normal) listex-mode-map (kbd "C-c C-c") #'listex:render-defun (kbd "C-c C-e") #'listex:render-last-sexpr))
if you are using evil
mode for instance.
10. Examples
10.1. Some equations
10.1.1. Newton
A simple equation will be
'(= (-mathbf F) (m (-mathbf v)))
\[\mathbf{F} = m \mathbf{v}\]
but of course you have all the power of emacs-lisp
at your disposal,
so you can get a little more creative with how you organize
things:
(let ((d '(-mathrm d)) (F '(-mathbf F)) (v '(-mathbf v)) (p '(-mathbf p)) (X '%\\times)) (cl-labels ((D (e x) `(-frac (,d ,e) (,d ,x)))) `(= ,F ,(D p 't) (+ (,X ,(D 'm 't) ,v) (,X m ,(D v 't))) )))
Up to now, everything has been using regular macros from elisp
.
Listex presents some convenience macros implementing similar lexical scopes
for latex macros and symbol aliases, for instance
<newton-macroletted>=
(lt-macrolet ((Dt (f) `(-frac ((-mathrm d) ,f) ((-mathrm d) t))) (bf (n) `(-mathbf ,n))) (lt-aliaslet ((*p* '(bf p)) (*v* '(bf v)) (*F* '(bf F)) (⨉ '%\\times)) '(= *F* (Dt *p*) (+ (⨉ (Dt m) *v*) (⨉ m (Dt *v*))))))
\[\mathbf{F} = \frac{\mathrm{d} \mathbf{p}}{\mathrm{d} t} = \frac{\mathrm{d} m}{\mathrm{d} t} \times \mathbf{v} + m \times \frac{\mathrm{d} \mathbf{v}}{\mathrm{d} t}\]
This last expression expands to
'(= (-mathbf F) (-frac ((-mathrm d) (-mathbf p)) ((-mathrm d) t)) (+ (%\times (-frac ((-mathrm d) m) ((-mathrm d) t)) (-mathbf v)) (%\times m (-frac ((-mathrm d) (-mathbf v)) ((-mathrm d) t)))))
10.1.2. Coupled cluster
This example writes the cluster operator in coupled cluster theory as an example for managing align environments.
(cl-labels (;; mathematics and second quantization (Σ (idx) `(_ (/sum /limits) ,idx)) (t (up down) `(_ (^ t ,up) ,down)) (a (i) `(_ (-hat a) ,i)) (á (i) `(^ ,(a i) /dagger)) (áa (up down) `(,(mapcar #'á up) ,(mapcar #'a (reverse down)))) ;; define particles and holes (parts (n) (seq-take '(a b c d e f g A B C D E F G) n)) (holes (n) (seq-take '(i j k l m n o I J K L M N O) n)) ;; Coupled cluster term of order n (term (n) (let ((pow (progn (require 'calc-bin) (math-power-of-2 n))) (abc (parts n)) (ijk (holes n))) (list (when (> n 1) (list '-frac 1 pow)) (Σ `(,@abc ,@ijk)) (t abc ijk) (áa abc ijk))))) ;; this inserts a plus and & and \\ for the align environment (let ((+ '%\\\\&+)) `(%&= (-hat T) (,+ ,@(cl-loop for i from 1 to 10 collect (term i)) /cdots))))
and we can try the same this time using
lt-macrolet
(cl-labels ((ps (n) (seq-take '(a b c d e f g A B C D E F G) n)) (hs (n) (seq-take '(i j k l m n o I J K L M N O) n))) (lt-macrolet* ((&+ (&rest args) `(%\\\\&+ ,@args)) (t (up down) `(_ (^ t ,up) ,down)) (^a (i) `(_ (-hat a) ,i)) (^á (i) `(^ (^a ,i) /dagger)) ;; holes and particles (áa (up down) `(,(cl-loop for p in up collect `(^á ,p)) ,(cl-loop for p in down collect `(^a ,p)))) (Σ (&rest idx) `(_ (/sum /limits) ,@idx)) (τ (n) (let ((pow (math-power-of-2 n)) (abc (ps n)) (ijk (reverse (hs n)))) `(,(when (> n 1) `(-frac 1 ,pow)) (Σ ,abc ,ijk) (t ,abc ,ijk) (áa ,abc ,ijk))))) '(%&= (-hat T) (&+ (τ 1) (τ 2) (τ 3)))))
10.2. Latex document example
Even though listex
is really thought for typesetting of formulas,
in principle writing documents in the style of
eltex is roughly possible, but consider just using org
for this.
Consider also reading this blog post if you really want to write
whole latex documents using s-expressions.
'(progn (-documentclass [12pt] article) (-usepackage hyperref) (env document (-section Introduction) (nl) (progn This document is is an example for the (-texttt LisTeX) domain specific language \(DSL\). (nl) You can also do inline math ($ (+ (^ A 5)) + 5) or displaystyle math (nl) ($$ (+ (^ A 5) 5) .) )))
10.3. A matrix macro
This is a simple but useful macro defined in the macro section:
(lt-aliaslet ((φ '/phi) († '/dagger)) '(matrix 2 3 (-hat A) B (-dot E) (-tilde (-hat C)) D (_ E (matrix 2 2 1 (* (-hat φ) †) 3 4))))
\[\begin{pmatrix} \hat{A} & B & \dot{E} \\ \tilde{\hat{C}} & D & E_{\begin{pmatrix} 1 & \hat{\phi} * \dagger \\ 3 & 4 \\ \end{pmatrix}} \\ \end{pmatrix}\]
And now imagine we want to find the \( D \)
and replace it with a \( \color{red}\psi \), well we can do it quite easily
since we defined a lisp reference for the above matrix called
weird-matrix
and we can use the emacs lisp function subst
to find and replace in the sexp tree:
weird-matrix
(cl-subst '(-color red /psi) 'D weird-matrix)
\[\begin{pmatrix} \hat{A} & B & \dot{E} \\ \tilde{\hat{C}} & \color{red}{\psi} & E_{\begin{pmatrix} 1 & \hat{\phi} * \dagger \\ 3 & 4 \\ \end{pmatrix}} \\ \end{pmatrix}\]
That's not bad at all.
10.4. Cmdlet
There should be an easy way of defining lexically macros, but I should think about tihs.
This is what works now, however the problem is that I would not want
to have to call listex
before the command.
(lt-cmdlet ((χ (a b c) "%s^{%s^{%s}}")) '(-frac (χ 5 5 5) /varphi))
\[\frac{5^{5^{5}}}{\varphi}\]
10.5. Schwarzschild's paper
This is a partial implementation of Schwarzschild's paper as a testcase for a paper.
(lt-aliaslet ((δ '/delta) (μ '/mu) (ν '/nu) (~% "\n") (=nl= '\\\\) (gμν '(_ g μ ν))) (lt-macrolet ((lisp (&rest args) (eval `(progn ,@args))) (main (&rest args) `(env document ,@args)) (text (&rest args) (progn (require 's) (s-word-wrap 80 (listex args)))) (code (&rest args) `(-texttt ,@args)) (* (&rest title) `("\\section{" ,@title "}" "\n\n")) (** (&rest title) `("\\subsection{" ,@title "}" "\n\n")) (*** (&rest title) `("\\ssubsection{" ,@title "}" "\n\n")) (eqlab (label &rest args) `(eq (-label ,label) ,@args)) (d! (var) `(progn (-mathrm d) ,var))) '(main (* Über das Gravitationsfeld eines Massenpunktes nach der Einsteinschen Theorie.) (text Hr. Einstein hat in seiner Arbeit über die Perihelbewegung des Merkur "(s. Sitzungsberichte vom 18. November 1915)" folgendes Problem gestellt: =nl= Ein Punkt bewege sich gemäß der Forderung (eqlab eq:main-metric-definition (lr /{ \. (env matrix (= (δ /int (d! s)) 0) =nl= =nl= (= (d! s) (-sqrt (/sum gμν (d! (_ x μ)) (d! (_ x ν)))))))) wobeit ($ gμν) Funktionen der Variabeln ($ x) bedeuten und bei der Variation am Anfang und Ende des Integrationswegs die Variablen ($ x) festzuhalten sind. Der Punkt bewege sich "also," kurz "gesagt," auf einer geodätischen Linie in der durch das Linienelement ($ (d! s)) charakterisierten Mannigfaltigkeit. =nl= Die Ausführung der Variation ergibt die Bewegungsgleichungen des Punktes) =nl= To be continued...)))
10.6. XML renderer
Here is an example of how to extend what is implemented in order to render XML easily.
(defun listex:render-xml (expr) "Main function to convert a listex DSL s-expression into a latex-compatible string." (cl-etypecase expr (listex:lisp-macro (let* ((args (cdr expr)) (name (car expr)) (f (listex:lisp-macro-get-fun name)) (new-expr (apply f args))) (listex:render-xml new-expr))) (listex:alias (let* ((replacement (listex:get-alias expr))) (listex:render-xml replacement))) (listex:command (let* ((name (string-remove-prefix listex-command-prefix (symbol-name (car expr)))) (args-strings (mapcar #'listex:render-xml (cdr expr)))) (format "%s=\"%s\"" name (string-join args-strings " ")))) (list (string-join (mapcar #'listex:render-xml expr) " ")) (atom (format "%s" expr)))) (defmacro with-xml (&rest body) `(cl-flet ((commandp (lambda (e) (typep e 'listex:command))) (join-exprs (l) (string-join (mapcar #'listex:render-xml l) " "))) (lt-macrolet ((> (name &rest content) (let ((commands (remove-if-not #'commandp content)) (tag-content (remove-if #'commandp content))) (format "<%s %s>\n%s\n</%1$s>\n" name (join-exprs commands) (join-exprs tag-content))))) ,@body)))
and we can use it in a minimal example that still is neat.
(lt-macrolet ((π (&rest body) `(> p (-class md-3 flex) ,@body))) (with-xml '(> html (> head (> title This is the inside of this title)) (> body (> div (-class "content") (π Here is a paragraph of my blog. I am pretty happy about how my paragraph is coming along.) And this is some content that I want to put outside of my paragraph.)))))
<html > <head > <title > This is the inside of this title </title> </head> <body > <div class="content"> <p class="md-3 flex"> Here is a paragraph of my blog. I am pretty happy about how my paragraph is coming along. </p> And this is some content that I want to put outside of my paragraph. </div> </body> </html>