Skip to content

MathJax v4.0.0-rc.4

Pre-release
Pre-release
Compare
Choose a tag to compare
@dpvc dpvc released this 25 Jun 22:42
· 45 commits to develop since this release
4.0.0-rc.4
4d1e195

This PR is the culmination of months of work to improve the performance of the assistive and speech-generation tools in MathJax. These are discussed in the sections below. There are also a number of new features and bug fixes. With this release, MathJax is moving to scoped npm packages, in the @mathjax scope; see below for more details. This is the last planned pre-release of v4.0, and it is the current release candidate. If all goes well, the official v4.0 should be released in July.

Note that 4.0.0-rc.1 through 4.0.0-rc.3 had issues (packaging issues, and CDN issues), and have been removed.

Accessibility Performance Improvements

When the MathJax accessibility features were turned on by default in the 4.0.0-beta.6 release, this led to a greater performance degradation in practice than in our testing. The 4.0.0-beta.7 attempted to address that issue by putting off the speech generation until after the mathematics was displayed, but this still did not perform as well as we wanted. As a result, we have spent the last 6 months working on alternative approaches.

The method we selected is to move the speech generation into a web-worker thread, which allows the browser to remain responsive even while the speech generation is being performed. This involved a substantial refactoring of both the MathJax assistive support code, as well as the speech-generation engine that underlies that support, but our tests indicate that this is a significant improvement over the results in the beta.7 release. Because web-workers are only available in browsers, not in node applications, we had to develop a means of supporting this new approach within node. Our solution was to use the node worker-threads library, along with some shim code to give it a corresponding interface to the features we were using from web-workers in the browser. This allows the same code to be used in both browsers and node applications. Currently this requires the use of the LiteDOM in node, but we may provide support in other DOM implementations in the future.

Because node applications that include speech generation now use worker-threads behind the scenes, the node application won't exit until the MathJax worker thread is shut down, so applications may need to tell MathJax that they are finished. This is done by calling the new MathJax.done() method in applications that use MathJax components, or by calling the MathDocument's done() method for this that use direct calls to MathJax's modules.

Expression Explorer Improvements

The expression explorer in the beta.7 release does not follow the ARIA standards properly, and while it worked in some browsers and with some screen readers, it did not work well in others, and, in particular, it failed rather badly in VoiceOver on MacOS. The interaction between browsers and screen readers is complicated, and the results vary widely among the different browser/os/screen-reader combinations. We spent several months searching for a mechanism that followed the ARIA specification while still giving consistent results across that various browsers and screen readers. The main goals were to:

  • Follow the ARIA standards.
  • Work across the major browser/OS/screen-reader combinations.
  • Properly read the full expression when reading the whole page or stepping through the page sentence-by-sentence (or by other units).
  • Automatically enter "focus mode" when the expression is focused via tabbing or clicking (in those screen readers that have separate focus and browse modes).
  • Allow control over the description of the math (e.g., "math", "clickable math", or no description) used during reading and focusing.
  • A "press H for help" message should be spoken when the math is first focused (but can be turned off by a menu preference).

We tested with 13 combinations: Chrome, Firefox, and Safari on MacOS with VoiceOver; Chrome, Firefox, and Edge on Windows with NVDA and JAWS; and Chrome and Firefox on Linux (Ubuntu 24 and 22) with Orca, and believe we have achieved these goals in all cases. The new explorer should give a much smoother experience to screen-reader users in all these settings.

Using the explorer remains largely the same. The explorer was enabled by default initially in beta.6, and continues to be in this release. That means that clicking on an expression will enter the explorer, highlight the clicked term, and cause a screen reader to speak that term; navigation via arrow keys will continue from that point. In previous versions, the speech and/or braille subtitles were shown as well, but these have been turned off by default in this release, as they caused confusion for users not aware of the explorer fetures. They can be re-enabled by use of the MathJax contextual menu.

Several new functions have been added to the explorer:

  • Pressing h will open a dialog box that describes the explorer's features. When an expression is first focused, a message informing the user of this feature is announced after the expression is read, but that can be disabled using the contextual menu.
  • Clicking on an expression starts the explorer and selects the clicked character, as in the past. New in this version is that double-clicking will start the explorer on the whole expression (just as tabbing to it would).
  • In the previous version, if an expression was explored and tab is used to leave the explorer, when that expression is refocused, the selection would remain where it was when the expression was exited. In this version, tabbing to an expression always starts at the top level of the expression, not the last selected term. The reason for this is so that if the page is read, wither as a whole or by smaller chunks, the expression will be read in full rather than just the previously selected term. There is a contextual menu item that can be used to select the older behavior, however, if a user wishes to be able to restart the explorer where they left off.

Speech-Generation Improvements

Aside from moving the speech generation into a web worker or node worker thread, the speech-generation has been improved by adding more sophisticated semantic analysis for complex user defined structures, including improved disambiguation of function applications, ellipses, and unusual large operators. MathJax now also automatically includes speech generation in precise SSML markup that is used for MathJax's auto-voicing feature as well as exposed in the speech attributes. Additional locales supported are Afrikaans and Korean as well as 8-dot Euro Braille output generated from original LaTeX input formulas.

Because the speech generation has been moved to a web-worker, the speech is no longer produced by the main MathJax thread, and so the timing options that were added in v4.0.0-beta.7 have been removed. That is, the speechTiming section of the options block of the MathJax configuration is not longer necessary, and will be marked as invalid if it is used.

Input Changes

In previous versions of MathJax, the macros in text-mode material (in \text{} and similar macros) were not processed unless the textmacros extension was loaded and enabled. This version now includes the textmacros extension in all the combined components that contain the TeX input jax (i.e., all the ones starting with tex-). This means that macros inside text-mode will be processed. Because only a limited number of text-mode-macros are defined, this can lead to errors in cases where the literal macro name would be displayed in the past. For example, prior to this version \text{The \item macro is used in lists} would produce the literal string The \item macro is used in lists, but in this version, it will lead to an error message about \item being undefined. So this may be a breaking change in some pages that take advantage of the old behavior.

You can disable the textmacros extension in combined components that include it by merging

MathJax = {
  tex: {
    packages: {'[-]': ['textmacros']}
  }
};

into your MathJax configuration.

One of the last remaining extensions from version 2 that was not ported to v3 is now available in this release: the begingroup extension that allows you to create temporary macro definitions. See the v2 begingroup documentation for more details (the v4 documentation is being written, but is not yet available).

The v4 version of begingroup defines two new non-standard macros: \begingroupReset and \begingroupSandbox. The first one ends any open \begingroup macros, reming any of their temporary macros or environments. The second can be used to isolate the definitions in one section of the page from those in another, so that sites like StackExchange can use this between user posts to make sure that one user doesn't redefine things to mess up another user. The \begingroupSandbox macro can't be redefined, and its action is essentially to do \begingroupReset\begingroup. This removes any previous user definitions and makes a new group for the next user's definition. It also directs any global definitions to this new group so that a user can create global macros in their own sandbox, but they are removed at the next \begingroupSandbox call. Any macros or environments created before the first \begingroupSandbox call are shared definitions that are available in every sandbox. Once \begingroupSandbox is performed, however, there is no going back; no new shared definitions can be made.

The mathtools extension has been updated to reflect the changes to the actual LaTeX package that were made in 2022 and 2024. In particular, there are some breaking changes to \coloneq and three other colon macros, several new colon-like commands, and several new extensible arrow macros, as described below.

The \coloneq, \Coloneq, \eqcolon and \Eqcolon macros now use the 2022 and later definitions (they use = rather than -, so \coloneq produces := not :- as in the past). A new legacycolonsymbols option controls which set to use (just as in actual mathtools). This can be set in the mathools section of the tex block of your MathJax configuration, of via the \mathtoolsset macro. The new colon macros \approxcolon, \Approxcolon, \simcolon, \Simcolon, \colondash, \Colondash, \dashcolon, and \Dashcolon are now defined, and are available regardless of the setting of legacycolonsymbols.

The new extensible arrow macros are \xlongrightarrow, \xlongleftarrow, \xLongrightarrow, and \xLongleftarrow.

Support for \MakeAboxedCommand, which was missing in the past, has been added in this release. This includes a non-standard starred version that handles box commands whose contents are in math-mode rather than text-mode, like \bbox and \boxed versus \fbox and \fcolorbox.

The \vcentercolon macro was incorrectly named \centercolon in previous versions, and has been corrected here. This is a breaking change for pages that used the incorrect name, but you can always define \centercolon to be \vcentercolon if that is an issue.

The mutlinedgap configuration option has been renamed to multlined-gap to correspond better with other option names (that all use dashes), and there is a new multlined-width option has been added to give the default width for multlined environments.

For the mhchem extension, several of the arrows were not previously stretchy. This release adds a new mhchem-specific font that includes the characters needed to stretch all the arrows available in mhchem, improving its output in both CHTML and SVG output. Note, however, that these fonts match the mathjax-newcm font set, and are used no matter what font is selected, so the arrows may not match other arrows used in the font if you are using one other than mathjax-newcm.

The digits options in the tex configuration block has been changed to numberPattern, and a new initialDigit pattern is available to specify what initial characters signal the start of a number. These make it easier to customize what constitutes a number for languages that use alternative characters, like Persian or Arabic. The digits option continues to be available for now, but will be removed in a future release.

There is also a new initialLetter pattern that is used to trigger multi-letter identifiers. This make using accented characters or other non-Latin characters for multi-letter identifiers easier to configure.

Output Changes

This version of MathJax includes a new font, mathjax-newcm, based on the New Computer Modern fonts, and this is now the default font for MathJax v4. Its look is largely the same as the previous default, mathjax-modern, but mathjax-newcm is slightly darker, and has greater character coverage. We hope this addresses the concerns that some expressed about the lightness of the mathjax-modern font.

An important improvement has been made to the way the CommonHTML (CHTML) output handle stretchy delimiters. In the past, the CHTML output would use CSS transforms to stretch the extender to the needed size. This lead to alignment issues between the extenders and the end pieces in some browsers, and in some cases, thin extenders disappeared entirely. In this version of MathJax, the extenders are made from repeated copies of the extender piece, slightly overlapping, and clipped to the proper size. This makes for cleaner and more reliable assemblies for stretched characters, both vertically and horizontally, and should eliminate the rendering problems seen in the past.

The MathJax layout uses a font that has larger depth below the baseline than most standard fonts. When the MathJax character are selected using drag selection, this leads to the selection bounding box being larger than expected. With this release, the CHTML output now uses clip paths to restrict the bounding boxes, making for more accurate selection backgrounds, and preventing unwanted vertical scroll bars in some displayed equations.

MathJax Scoped Packages

With this release, MathJax is moving to scoped packages for the source and font npm packages. The mathjax-full package is now @mathjax/scr, and the font packages are @mathjax/mathjax-stix2-font, @mathjax/mathjax-fira-font, and so on. Future extensions and other packages will be in the @mathjax scope as well. The only exception is that the mathjax package will remain un-scoped. Since that use of MathJax in browsers is primarily through the mathjax package, that means the URL for loading MathJax will remain the same, with only the version number needing to be changed. That is, you can use

https://cdn.jsdelivr.net/npm/[email protected]/tex-mml-chtml.js

to load this release candidate version of MathJax. But to obtain the source version to include in a node project, you would use

pnpm install @mathjax/[email protected]
pnpm install @mathjax/[email protected]

or similar commands. You will need to change any references to mathjax-full to @mathjax/src in your code. If you will use a different font, use

pnpm install @mathjax/mathjax-stix2-font

to get the latest mathjax-stix2 font package, for example. You will need to change any references to mathjax-[fontname]-font to @mathjax/mathjax-[fontname]-font in your code.

MathJax API Additions

In previous version of MathJax, the speech generation was performed within the semantic-enrich component along with the semantic enrichment of the internal MathML representation of the mathematical expressions that it processes. In this release of MathJax, these two functions have been separated from each other, and the speech-generation functionality is performed in a new speech component. This is included in all the combined components, but can be loaded individually by including a11y/speech in the load array of the loader block of your MathJax configuration.

The section on Accessibility Performance Improvements already mentions the new MathJax.done() function that is used to shut down the web-worker or node worker-thread invalid in speech production. There is a corresponding new done() method for the MathDocument class that can be used in application that don't use the MathJax Component framework, but rather call on MathJax modules directly.

Some actions in MathJax require loading extra code from an extension, which is an asynchronous action, as the browser must wait for the file to be loaded before it can use it. In MathJax v3, such asynchronous actions were mostly associated with loading TeX extension packages, and that could be avoided by pre-loading the extensions that are needed, so that typesetting could be performed synchronously. In v4, with fonts that have much greater coverage than in v3, some font data may need to be loaded asynchronously as well, and that means that typesetting may be asynchronous even if all the needed TeX extensions are pre-loaded. As a result, the MathJax.typesetPromise() function is more likely to be needed, and MathJax.typeset() will only work if not font data needs to be loaded.

Because of this greater need to handling asynchronous file loading, several new functions have been added to the MathDocument class to provide promise-based versions of the corresponding synchronous calls. These include convertPromise(), renderPromise(), and rerenderPromise(), which wrap the convert(), render(), and rerender() methods in the needed mathjax.handleRetriesFor() call and return its promise. This makes it easier to perform these actions when font data or TeX extensions need to be loaded than having to use handlerRetriesFor() yourself.

In the previous beta versions of v4, the promise-based functions like MathJax.typesetPromise() and MathJax.tex2mmlPromise() use MathJax.startup.promise to serialize the typesetting actions, preventing them from occurring concurrently (in version 3, you were expected to do that yourself). In this version, these functions now use the new promise-based functions from the MathDocument class. Because MathJax.startup.promise is part of the MathJax Components framework, and MathDocuments can be used outside that framework, the new MathDocument promise-based calls use their own private promise instead, and the MathJax.startup.promise reverts back to it original purpose of representing when he initial typeset operation has completed.

You nay wish to use the new MathDocument promise to synchronize other code with MathJax's typesetting operations, however, without having to keep track of the promises returned by the three new promise-based functions. For this reason, MathJax provides a new whenReady() method of the MathDocument class. It takes a function as its argument, and performs that action when its internal promise is resolved; that is, when any previous promise-based typesetting actions complete. You can think of whenReady() as queuing your action to be performed whenever MathJax has finished anything that has been queued previously.

The function you pass to whenReady() can return a promise (if it starts any asynchronous actions of its own, for example), in which case that promise must be fulfilled before any further whenReady() actions will be performed. For example

const doc = mathjax.document('', {});
doc.whenReady(() => console.log('A'));
doc.whenReady(() => {
  return new Promise((ok, fail) => {
    setTimeout(() => {
      console.log('B');
      ok();
    }, 1000);
  });
});
doc.whenReady(() => console.log('C'));

would print A to the developer console, then a second later print B followed by C..

The MathJax code base has undergone a major cleanup effort for this release, using eslint and prettier to format the code consistently, and new life-cycle scripts to perform these actions have been added to the package.json file. Other modernizations, like moving from String.substr() to String.substring() were also performed.

Finally, MathJax's test suite has been expanded to include more than 3,000 tests. We have full coverage for the TeX input jax and the ts/util directories, but more tests need to be written for other sections of the code base. This is an ongoing project that will take time to complete.

Breaking Changes in this Release

The Input Improvements section already discusses some potentially breaking changes for some pages, including the fact that the the textmacros extension is now included in all combined components, the update to the mathtools extension to include the changes in the corresponding LaTeX package from 2022 and 2024, and the change from digits to numberPattern for configuring the pattern used to identify a number in TeX input.

The MathJax Scoped Packages section describes MathJax's change to using scoped npm package names, and in particular, moving from mathjax-full to @mathjax/src.

Changes for Speech Generation

The API Changes section above describes the separation of the semantic enrichment from the speech generation and the introduction of a new speech component. Because the speech is now generated within a web worker or node worker thread, the speech-generation code is no longer in the main MathJax components, but is in a separate file that is run in the worker. That means there is no more access to speech generation directly within MathJax (it is only available in the worker). In particular, the ts/a11y/sre.ts file now only include the semantic-enrichment methods of the speech-rule engine, so Sre.toSpeech() is no longer available. In a node application, you can load this function directly from the speech-rule engine's API, however. Here is an example command-line script that takes a TeX expression and returns its speech string using this approach:

//
//  Load the modules needed for MathJax
//
import {mathjax} from '@mathjax/src/js/mathjax.js';
import {TeX} from '@mathjax/src/js/input/tex.js';
import {liteAdaptor} from '@mathjax/src/js/adaptors/liteAdaptor.js';
import {RegisterHTMLHandler} from '@mathjax/src/js/handlers/html.js';
import {SerializedMmlVisitor} from '@mathjax/src/js/core/MmlTree/SerializedMmlVisitor.js';
import {STATE} from '@mathjax/src/js/core/MathItem.js';

//
// Import the speech-rule-engine
//
import '@mathjax/src/components/require.mjs';
import {setupEngine, engineReady, toSpeech} from 'speech-rule-engine/js/common/system.js';

//
// Import the needed TeX packages
//
import '@mathjax/src/js/input/tex/base/BaseConfiguration.js';
import '@mathjax/src/js/input/tex/ams/AmsConfiguration.js';
import '@mathjax/src/js/input/tex/newcommand/NewcommandConfiguration.js';
import '@mathjax/src/js/input/tex/noundefined/NoundefinedConfiguration.js';

//
// The em and ex sizes and container width to use during the conversion
//
const EM = 16;          // size of an em in pixels
const EX = 8;           // size of an ex in pixels
const WIDTH = 80 * EM;  // width of container for linebreaking

//
// Create DOM adaptor and register it for HTML documents
//
const adaptor = liteAdaptor({fontSize: EM});
RegisterHTMLHandler(adaptor);

//
// Create input jax and a (blank) document using it
//
const tex = new TeX({
  packages: ['base', 'ams', 'newcommand', 'noundefined'],
  formatError(jax, err) {console.error(err.message); process.exit(1)},
  //
  // Other TeX configuration goes here
  //
});
const html = mathjax.document('', {
  InputJax: tex,
  //
  // Other document options go here
  //
});

//
//  Create a MathML serializer
//
const visitor = new SerializedMmlVisitor();
const toMathML = (node => visitor.visitTree(node, html));

//
// Convert the math from the command line
//
const mml = html.convert(process.argv[2] || '', {
  display: true,
  em: EM,
  ex: EX,
  containerWidth: WIDTH,
  end: STATE.CONVERT         // stop after conversion to MathML
});

//
// Set up the speech engine to use English
//
const locale = process.argv[3] || 'en';
const modality = locale === 'nemeth' || locale === 'euro' ? 'braille' : 'speech';
await setupEngine({locale, modality}).then(() => engineReady());

//
// Produce the speech for the converted MathML
//
console.log(toSpeech(toMathML(mml)));

With the speech generation being performed in a worker, the process is now inherently asynchronous, as the communication between the main thread and the worker thread is mediated by promises. That means that speech generation can't be done synchronously, and you must use the promise-based functions for handling typeset and conversion operations that involve speech.

The speech-generation process now applies the speech attributes to the DOM nodes themselves, rather than to the internal MathML structure, so serialized versions of the internal MathML will not include the speech, as they did in the past.

Because of the changes in how speech is being generated, the speechTiming section of the options block of the MathJax configuration is no longer supported, and will cause an error if it is used in your configuration.

Name Changes

Some name changes have occurred within the code to help clarify the purpose of some objects or methods. In particular, the Loader.preLoad() method has been renamed Loader.preLoaded() in order to make it clear that this does not itself load the given components, but that your code has done that and you are telling MathJax that they have already been loaded.

Another change involves the objects used to handle CSS styles. MathJax has two object classes that deal with CSS definitions, one that specifies CSS styles via object literals (essentially JSON structures), and one that parses a CSS string into an object structure that acts like a DOM element's style attribute. Moreover both modules declared a StyleList type, and these were not compatible. That caused both confusion and complicated their use together in the same module, so the names have for these types and objects have been changed in this release in order to make it clearer which is which.

To accomplish this, the ts/util/CssStyles.js file was renamed to ts/util/StyleJson.js, and with that, StyleList is changed to StyleJson, StyleData to StyleJsonData, and CssStyles to StyleJsonSheet. This more accurately describes what this object does (it is the one that takes JSON data), while the ts/util/Style.js file implements (a subset of) the DOM object style attribute.

Finally, the latest.js file has been removed, as jsdelivr.net and other CDNs handle providing the latest version automatically, and with more granularity than this file did.

List of Major Updates

The major fixes and changes in this release fall into the following categories.

Input Updates

  • Report failure of \require rather than throw an error. (#1255)
  • Handle non-breaking spaces correctly in \text{}. (mathjax/MathJax#3353) (#1240)
  • Allow \eval with no argument in physics package. (mathjax/MathJax#3346) (#1232, #1234)
  • Add texClass REL when combining mo nodes so they round-trip through SRE properly. (#1228)
  • Fix incorrect character for \char`\x (was incorrectly set to \char'\x). (#1225)
  • Error for colors that include semicolons. (#1223)
  • Update mathools to include changes in the LaTeX package from 2022. (#1213)
  • Add \MakeAboxedCommand to mathtools extension. (#1213)
  • Implement begingroup package. (#1182)
  • Map \perp to U+27C2 rather than U+22A5. (issue mathjax/MathJax#3322) (#1181)
  • Allow better configuration for patterns for numbers and identifiers. (#1178)
  • Add textmacros to the input/tex components. (#1172)
  • Update newcommand to handle delimiters vs macros better. (#1165)
  • Make over- and under-lines not be accents, so they operate as in actual TeX. (#1162)
  • Make over/under braces and matrices be full size, as in actual TeX. (mathjax/MathJax#3300) (#1146)
  • Have \let look up macros first then delimiters, and removes \\ delimiter. (#1145)
  • Add a template substitution count to avoid infinite loop. (#1142) (#1144)
  • Fix incorrect handling of user-defined environments introduced in #856. (#1137) (#1139)
  • Fixes the Dutch longdiv issue. (#1136)

Output Updates

  • Fix script spacing when MathML spacing is used. (mathjax/MathJax#1224) (#1259)
  • Fix cssText creation to handle border-radius and similar values. (mathjax/MathJax#3365) (#1252)
  • Adjust assisitve-mml output to prevent it from shifting outside the expression bounding box. (#1244)
  • Fix position of mmultiline scripts in WebKit when <none> is used. (#1244)
  • Don't allow inline breaks at negative spaces. (#1233)
  • Update default spacing for mo elements that aren't in the operator dictionary. (mathjax/MathJax#3288) (#1224)
  • Change order of root and radicand in mroot output. (mathjax/MathJax#3335) (#1222)
  • Fix position adjustment for scaled SVG items with ids. (#1221)
  • Add stretchy characters needed by mhchem. (#1174)
  • Remove use of css transforms for handling stretchy assemblies in CHTML output. (#1173)
  • Handle clipping of characters in CHTML better and avoid vertical scroll bars for overflow="scroll". (#1171)
  • Change default font to mathjax-newcm. (#1140)

Interface Changes

  • Add menu item to control what tabbing will focus on. (#1258)
  • Add defaultOptionError function for easier configuration. (#1256)
  • Add MathJax.whenReady() function. (#1244)
  • Add core promise-based typeset and convert functions. (#1231)
  • The getMetricsFor() method now includes the display property. (#1229)
  • Add ability to filter SRE and LaTeX attributes from SVG image. (#1184)
  • Include a11y in default settings, and fix a number of problems with changing menu values. (mathjax/MathJax#3310) (#1169)

Miscellaneous

  • Move highlighter module from SRE to MathJax. (#1250)
  • Change to using @mathjax/src rather than mathjax-full. (#1243)
  • Implement new explorer paradigm. (#1241)
  • Print webpack failure messages in build tools. (#1239)
  • Fix lookbehind assertions that are not widely supported in browser. (#1238)
  • Map components/js to components/cjs for require() or components/mjs for import. (#1227)
  • Updates for running workers in node. (#1219)
  • Rename StyleList to StyleJson, etc. (#1180)
  • Rename Loader.preLoad() to Loader.preLoaded() to make it clear that this doesn't do any loading itself. (#1177)
  • Move to scoped packages for fonts, and use woff2 format. (#1168)
  • Remove latest.js as jsdelivr can do this automatically. (#1157)
  • Improve merging of math lists (mathjax/MathJax#3301). (#1143)
Assets 2
Loading