Wiktionary:Lua memory errors

Wiktionary has persistent Lua (Scribunto) memory errors in certain entries with many language sections: some one- and two-Latin-letter entries and some single Han character entries. At the moment Wiktionary editors have not found a long-term solution.

BackgroundEdit

Lua is the scripting language found on Module-namespace pages (see WT:LUA). It is used in many templates. It is used when there is {{#invoke:module name|function name|optional arguments ...}} (a module invocation) on a page, or in text that is transcluded on the page, from a template for instance.

It has a garbage collector, which frees up memory after it is no longer accessible from the Lua code. Memory is not freed immediately, so memory usage is somewhat unpredictable. Memory can rise or fall each time you preview a page, and removing a module invocation from a page can actually increase memory.

The memory limit is 50 megabytes. When Lua memory goes over this limit, an error with the explanation "not enough memory" is displayed in place of the module invocation ({{#invoke:module name|function name|optional arguments ...}}) and the page is placed in Category:Pages with module errors.

DifficultiesEdit

Certain pages have had memory errors for a year or more. These include some one- and two-Latin-letter entries and some single Han character entries. Examples include a.

Some pages with many translations have had memory errors.

TacticsEdit

Translation subpagesEdit

On pages with many translations, the solution to memory errors has been to move translations to a fake translations subpage created by adding /translations to the title and {{translation subpage}} on the top of the page; see Category:Translation subpages for the full list.

These translation subpages are not true subpages because the subpage feature is turned off in the main namespace. This allows there to be entries for terms that contain slashes. A true subpage has a link to the parent page under its title. This does not happen to titles with slashes in the main namespace. For instance, water/translations does not have a link to water under its title.

Derivation subpagesEdit

These works similarly to translation subpages. These store all derived terms from an individual lemma on a separate, fake subpage, and are mostly used with Han/CJKV characters. It is created by adding /derivations to the title and {{derivative subpage}} on top of the page; they will then be added to Category:Derivative subpages.

Increasing the memory limitEdit

A Wiktionary admin posted a Phabricator task that suggested increasing the memory limit. This would probably not be a long-term solution because memory usage would probably continue increasing and eventually exceed the new limit.

Replacing letters templatesEdit

Some of the most vulnerable targets for Lua memory errors are single-letter Latin alphabet pages. In fact, all five original Latin vowel letters (a, e, i, o, and u) have faced module errors multiple times. Hence, the Lua-dependent {{letters}} or some other derivative appears often on these pages. One workaround is to replace some of the templates dependent on {{letters}} with "simple" versions composed of plain wikitext, in order to reshuffle the garbage collection. Unfortunately, garbage reshuffling doesn't always happen and sometimes replacing letters list templates may worsen module errors.

Optimizing Lua memory usageEdit

It might be possible to redesign some modules (including core modules) to reduce memory usage. Larger changes might be required in order to have a noticeable effect. It remains to be seen whether this can be achieved in practice.

Workaround for usersEdit

Content of a section that cannot be seen because it is replaced by “Lua error: not enough memory” can often be viewed by first clicking “[ edit ]” next to the section header, and then the button Show preview below the edit window.

Long-term solutionsEdit

Per-language pagesEdit

This is an old proposal, which predates the Lua memory issue, but might solve it. A very detailed overview and links to old discussions can be found at Wiktionary:Per-language pages proposal.