Defining operators

As we know, LaTeX has many commands to typeset operators, which are traditionally typeset using upright font—for example, sin, log etc. Obviously, the catalog of LaTeX’s operators is far from complete, especially because new ones are being constantly introduced. The question arises: what to do when we need a custom operator?

Probably one of the most often mistakes is to write something like

${\rm sgn} x$

or, even worse,

$\mbox{sgn} x$

What are the problems with those solutions? The first one yields the result shown in the image.sgn-bad Notice the lack of a thin space between the “sgn” and the “x“. The second one gives a similar result, but not scaling properly when put into a subscript or superscript.

So what is the “right” way? As usually, there is more than one; but the simplest and cleanest one is to use the amsmath package:

\usepackage{amsmath}

(this is not necessary if you use one of the amsart, amsbook or amsrep classes) and then say

\operatorname{sgn}

But there is one caveat. Some time ago I saw in my friend’s file the following code:

\newcommand{sgn}{{\operatorname{sgn}}}

The result is disastrous: the space vanishes again! The explanation lies in the fact that TeX’s math mode treats everything in curly braces as an ordinary symbol, and we need an operator and not an ordinary symbol. The right way would be to define

\newcommand{sgn}{\operatorname{sgn}}

—i.e., this would be the right way if there were no special command for that in the amsmath package. If you use some operator more often, it’s best to define it using the following construct:

\DeclareMathOperator{\sgn}{sgn}

where the first argument is the command and the second one the name to be typeset.

Again, there is a bit more to say. Assume that you want to have an operator like lim, which takes “limits” (in TeX’s terminology, this means that the sub- or superscript is to be placed below or above when in “displayed” mode). Then, you should use the starred version of \DeclareMathOperator:

\DeclareMathOperator*{\esssup}{ess\,sup}

(notice the thin space—\,—which gives the resultess sup shown). Of course, there is an analogous operatorname* variant.

(This article is a rough translation of a Polish version, published on the author’s web site in 2008.)