Tutorial¶

This tutorial introduces the localecmd library and its features. We will create a simple shell for drawing turtle graphics and translate it into other languages. For drawing, we will use the python builtin turtle module.

If you get the error No module named '_tkinter', you have to install tkinter first. See tkinter documentation on how to do that.

To translate, you need to edit .po files and export them to .mo files. In the tutorial, we will use a normal text editor to edit the files and babel to convert them. Babel should be installed along with the localecmd library.

Part of the tutorial is based on the example of builtin cmd library and it can thereby be seen as a comparison between these two libraries.

Create project¶

We will need a project folder for the tutorial which contains certain subfolders. Create the project folder with you favourite file explorer. In bash shell, run mkdir localecmd-tutorial (or use whatever descriptive name).

Go into that folder and create a python file named functions.py. You may use any name, but the tutorial presumes functions.py. Also create a python file __main__.py and subfolders locale and helptexts.

Build TurtleShell¶

First introduce the functions. Edit functions.py to look as

import turtle

from localecmd import programfunction
from localecmd.cli import CLI, run_script
from localecmd.localisation import get_translations

DOMAIN = 'messages'


def _(msgid: str) -> str:
    "Gettext for message domain."
    return get_translations().dgettext(DOMAIN, msgid)


@programfunction()
def forward(*args: int):
    'Move the turtle forward by the specified distance:  FORWARD 10'
    turtle.forward(*args)


@programfunction()
def right(*args: int):
    'Turn turtle right by given number of degrees:  RIGHT 20'
    turtle.right(*args)


@programfunction()
def left(*args: int):
    'Turn turtle left by given number of degrees:  LEFT 90'
    turtle.left(*args)


@programfunction()
def goto(*args: int):
    'Move turtle to an absolute position with changing orientation.  GOTO 100 200'
    turtle.goto(*args)


@programfunction()
def home():
    'Return turtle to the home position:  HOME'
    turtle.home()


@programfunction()
def circle(*args: int):
    'Draw circle with given radius an options extent and steps:  CIRCLE 50'
    turtle.circle(*args)


@programfunction()
def position():
    'Print the current turtle position: POSITION'
    pos = (int(p) for p in turtle.position())
    print(_('Current position is {0} {1}\n').format(*pos))


@programfunction()
def heading():
    'Print the current turtle heading in degrees:  HEADING'
    print(_('Current heading is {0}\n').format(int(turtle.heading())))


@programfunction()
def reset():
    'Clear the screen and return turtle to center:  RESET'
    turtle.reset()


@programfunction()
def bye():
    'Stop recording, close the turtle window, and exit:  BYE'
    print(_('Thank you for using Turtle'))
    turtle.bye()
    raise SystemExit


@programfunction()
def playback(from_line: int, to_line: int = -1):
    "Rerun all commands between two line numbers"
    cli = CLI.get_cli()

    run_script('\n'.join(cli.cmdlog.get(from_line + 1, to_line)))

Then set up the shell. Edit __main__.py to be

import functions
from functions import DOMAIN, _

from localecmd import CLI, builtins


def main():
    modules = [functions, builtins]
    greeting = _("Welcome to the turtle shell. Type help to list commands.")
    cli = CLI(modules, greeting=greeting, gettext_domains=[DOMAIN], use_file_docstrings=True)
    cli.cmdloop()
    cli.close()


main()

Now, you can start the turtle shell by typing python . into the shell. Here is a sample session with the turtle shell showing the help functions, using blank lines to repeat commands, and the simple record and playback facility:

The language of the command line is fallback language
Welcome to the turtle shell. Type help to list commands.
¤ help forward
forward args... ⠀                                                               

Move the turtle forward by the specified distance:  FORWARD 10                  
¤ position
Current position is 0 0

¤ heading
Current heading is 0

¤ reset
¤ circle 20
¤ right 30
¤ circle 40
¤ right 30
¤ circle 60
¤ right 30
¤ circle 80
¤ right 30
¤ circle 100
¤ right 30
¤ circle 120
¤ right 30
¤ circle 120
¤ heading
Current heading is 180

¤ forward 200
¤ right 90
¤ forward 200
¤ right 90
¤ forward 400
¤ right 90
¤ forward 500
¤ right 90
¤ forward 400
¤ right 90
¤ forward 300
¤ right 90
¤ forward 200
¤ playback 1
Current position is 0 0

Current heading is 0

Current heading is 180

¤ bye
Thank you for using Turtle

Translate¶

Localecmd uses gettext to translate strings. This includes also function and parameter names and their types. The procedure of translating the program is the following:

Extract strings from the source files. These are messages and names. The extracted strings can be found in .pot files. These are templates for the translations.
Initialise and update translation files from template. Every translation has its own folder containing the translation files
Translate strings.
Compile translation files to machine-readable format.
Localecmd should now recognise the new translations.

Hint

Several of the commands have to be run multiple times with different arguments. In a real case, one should consider to use a script or makefile to cover the process.

Extract strings¶

Localecmd only provides Python API to extract strings from source files. Here, we will extract the strings with a script. Create a file extract_pot.py with the content

#!/usr/bin/env python3
import functions

from localecmd import create_pot

create_pot([functions], 'locale', project='localecli_tutorial')

The folder locale should now contain the translation template for every domain: cli_functions, cli_messages and cli_types.

If we now open locale/cli_functions.pot, it should look like

# Translations template for localecli_tutorial.
# Copyright (C) <YEAR> ORGANIZATION
# This file is distributed under the same license as the localecli_tutorial
# project.
# FIRST AUTHOR <EMAIL@ADDRESS>, <YEAR>.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: localecli_tutorial 0.1\n"
"Report-Msgid-Bugs-To: user@example.com\n"
"POT-Creation-Date: YEAR-MO-DA HO:MI+ZONE\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel <BABEL-VERSION>\n"

# Function
# Change language of program
msgctxt "builtins"
msgid "change_language"
msgstr ""

# Parameter
msgctxt "builtins.change_language"
msgid "language"
msgstr ""

The first ~20 lines are header and metadata. The actual translation strings start below that, starting with the builtin functions.

Important

Don’t change this file. It is the template for the translations and will anyway be overwritten everytime tutorial/extract_pot.py is called.

Further down, the turtle functions appear. Note that the comments above the function names show the first line of the docstring. Also, the context of every function is the module name: The bye function is in module functions, so the context is functions. The context of a parameter name is the function name: Parameter args of function circle in module functions has context functions.circle. This means that the parameter name must be translated for every function separately, even if it has the same name.

# Function
# Stop recording, close the turtle window, and exit:  BYE
msgctxt "functions"
msgid "bye"
msgstr ""

# Function
# Draw circle with given radius an options extent and steps:  CIRCLE 50
msgctxt "functions"
msgid "circle"
msgstr ""

# Parameter
msgctxt "functions.circle"
msgid "args"
msgstr ""

# Function
# Set the color:  COLOR BLUE
msgctxt "functions"
msgid "color"
msgstr ""

The files cli_messages.pot and cli_types.pot are built up similarly, but contain no more info than typical for gettext.

Additionally we need to extract the messages shown in the tutorial functions. To extract those, run pybabel extract . -o locale/messages.pot.

Now translation templates for all domains are extracted. This message extraction has to be done every time sources have changed.

Initialise translations¶

Before the translation into a language can be started, it must be initialised. Initialisation is only needed the first time a domain for a language is being translated. We will use babel for this and every domain needs to be initialised separately.

As language, we first use English. Run

pybabel init -i locale/cli_functions.pot -d locale -D cli_functions -l en_GB
pybabel init -i locale/cli_messages.pot -d locale -D cli_messages -l en_GB
pybabel init -i locale/cli_types.pot -d locale -D cli_types -l en_GB
pybabel init -i locale/messages.pot -d locale -D messages -l en_GB

This should generate a subfolder of locale named en_GB which contains another folder LC_MESSAGES. Inside that, there are four files with the same names as the domains, but with ending .po. The content of those is practically the same as the templates.

To show differences, we also will translate the program into an other language. Select a language that you are comfortable to translate into. If you use an other language than German (Germany), make sure to replace de_DE with the language code you are translating into further in the tutorial. Run:

pybabel init -i locale/cli_functions.pot -d locale -D cli_functions -l de_DE
pybabel init -i locale/cli_messages.pot -d locale -D cli_messages -l de_DE
pybabel init -i locale/cli_types.pot -d locale -D cli_types -l de_DE
pybabel init -i locale/messages.pot -d locale -D messages -l de_DE

Update translations¶

When the sources have changed and templates updates as in last section, the translation files must be updated. This is done with

pybabel update -i locale/cli_functions.pot -d locale -D cli_functions
pybabel update -i locale/cli_messages.pot -d locale -D cli_messages
pybabel update -i locale/cli_types.pot -d locale -D cli_types
pybabel update -i locale/messages.pot -d locale -D messages

Again, every domain must be updated separately, but at least the update is done for all lanugages at the same time.

Translate strings¶

We will translate the string with a normal text editor as this is enough for simple use. The source language is already English, so the English files may be left unchanged. Go through all .po files in directory locale/de_DE/LC_MESSAGES and translate the string in msgid to msgstr. All strings may have non-ASCII characters. Confer a gettext software localisation tutorial for more details on translating.

Compile strings¶

This has again to be done separately for every locale and file. The list of commands to run now is

pybabel compile -l en_GB -d locale -D cli_functions
pybabel compile -l en_GB -d locale -D cli_messages
pybabel compile -l en_GB -d locale -D cli_types
pybabel compile -l en_GB -d locale -D messages
pybabel compile -l de_DE -d locale -D cli_functions
pybabel compile -l de_DE -d locale -D cli_messages
pybabel compile -l de_DE -d locale -D cli_types
pybabel compile -l de_DE -d locale -D messages

Use translated program¶

As a default, the CLI is started in the fallback language, that is how it was coded.

The language can be changed in runtime with the command change_language as demonstrated below. Note that helptexts are not translated as that is part of the next step.

The language of the command line is fallback language
Welcome to the turtle shell. Type help to list commands.
¤ position
Current position is 0 0

¤ heading
Current heading is 0

¤ reset
¤ circle 20
¤ change_language de_DE
Die Sprache der Befehlszeile ist Deutsch (Deutschland)
¤ position
Die jetzige Position ist 0 0

¤ kreis 20
¤ abspielen -ab-Zeile 6
¤ tschüss
Vielen Dank für Ihre Reise mit Turtle

Helptexts¶

Localecmd primarily loads helptext from provided markdown files. These are expected to be in the folder locale/<language>/docs. The name has to be the name of the module.

We will now create those four files (two modules, two languages). We will use a script that uses the docstrings to generate the helptexts. First, we need a to extract the messages to translate and translate them. Create extract_docs.py with the following content:

#!/usr/bin/env python3
import os
import subprocess
import sys
import tempfile

sphinx_build = sys.executable + ' -m sphinx.cmd.build'
sphinx_intl = sys.executable + ' -m sphinx_intl'
# Languages to translate helptexts to
languages = ['en_GB', 'de_DE']

home_dir = os.path.split(__file__)[0]
localedir = os.path.join(home_dir, 'locale')
# Content of conf.py file
# myst extensions are used in the builtins docstrings and enable use of myst in parameter
# descriptions.
conf = """
extensions = [
    'localecmddoc',
    'myst_parser',
    'sphinx_markdown_builder',
]
localecmd_modules = {
    'functions': 'functions',
    }


myst_enable_extensions = [
    "fieldlist",
    "colon_fence",
]

source_suffix = {'.md': 'markdown'}
gettext_compact = True
"""
if __name__ == '__main__':
    with tempfile.TemporaryDirectory() as tmpdirname:
        # Folder to dump the  raw docstrings
        folder = os.path.join(tmpdirname, 'source')
        # Folder where converted and translated files
        folder2 = os.path.join(tmpdirname, 'build')

        # Write conf.py
        with open(os.path.join(folder, 'conf.py'), 'w') as file:
            file.write(conf)

        # Sphinx options
        options = "-b gettext"
        # Extract translatable strings # No user input
        subprocess.run([*sphinx_build.split(), folder, folder2, *options.split()])  # noqa: S603

        options = f"update -p {folder2} -d locale"
        # Languages to update: -l en_GB -l de_DE
        # lang_str = ' '.join([f'-l {lang}' for lang in languages])
        # Update .po files
        subprocess.run([*sphinx_intl.split(), *options.split()])  # noqa: S603

To run it, the packages sphinx, myst-parser, sphinx-intl and sphinx-markdown-builder must be installed.

pip install myst-parser sphinx sphinx-intl sphinx-markdown-builder

Then run the script:

python extract_docs.py

It should generate builtins.po and functions.po in locale/en_GB and locale/de_DE folders.

Translate the messages. For example as

# SOME DESCRIPTIVE TITLE.
# Copyright (C)
# This file is distributed under the same license as the Project name not
# set package.
# FIRST AUTHOR <EMAIL@ADDRESS>, 2025.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: Project name not set \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2025-02-25 06:13+0100\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language: de_DE\n"
"Language-Team: de_DE <LL@li.org>\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.17.0\n"

#: ../source/functions.md:1
msgid "Module functions"
msgstr "Turtle-Funktionen"

#: ../source/functions.md:6
msgid "Stop recording, close the turtle window, and exit:  BYE"
msgstr "Aufnahme beenden, Turtle-Fenter schließen und Programm schließen: TSCHÜSS"

The files for English language do not have to be translated since docstrings already are in English. Compilation will be done by sphinx-intl, so that is not nessesary now.

Then we will actually create the translated docs with a new script. Create generate_helptexts.py containing

#!/usr/bin/env python3
import os
import shutil
import subprocess
import tempfile

import functions
from extract_docs import conf, languages, localedir, sphinx_build

from localecmd import start_cli
from localecmd.cli import create_docs

if __name__ == '__main__':
    for lang in languages:
        with tempfile.TemporaryDirectory() as tmpdirname:
            # Folder to dump the files to convert
            folder = os.path.join(tmpdirname, 'source')
            # Folder where converted and translated files
            folder2 = os.path.join(tmpdirname, 'build')

            # Start cli
            cli = start_cli([functions], lang)
            # Dump myst docstrings
            create_docs(folder)
            cli.close()
            # Write conf.py
            with open(os.path.join(folder, 'conf.py'), 'w') as file:
                file.write(conf)
            # Sphinx options
            options = f"-b markdown -D language={lang} -D locale_dirs={localedir}"
            # Place to save helptext files
            endfolder = os.path.join('locale', lang, 'docs')
            # Generate helptext files in markdown with sphinx # No user input
            subprocess.run([*sphinx_build.split(), folder, folder2, *options.split()])  # noqa: S603

            # Generated helptextfiles
            files = ['functions.md', 'builtins.md']
            # Move helptextfiles to correct directory
            for filename in files:
                os.remove(os.path.join(endfolder, filename))
                shutil.move(os.path.join(folder2, filename), endfolder)
            # Empty build folder to have empty dir for next language
            shutil.rmtree(folder2)

and run it with

python generate_helptexts.py

The translated helptexts should now be in folder locale/<language>/docs. If we open locale/de_DE/docs/functions.md the first 20 lines should look like

# Turtle-Funktionen

### tschüss ⠀

Aufnahme beenden, Turtle-Fenter schließen und Programm schließen: TSCHÜSS

### kreis args... ⠀

Kreis mit gegebenem Radius zeichnen: KREIS 50

### vorwärts args... ⠀

Schildkröte vorwärts bewegen:  VORWÄRTS 10

### geh_nach args... ⠀

Schildkröte gedreht an bestimmte Stelle bewegen.  GEH_NACH 100

### richtung ⠀

Last we check that the translations are loaded correctly by our turtle-program. Run python . in the shell to start it again and check that translations are correct.

The language of the command line is fallback language
Welcome to the turtle shell. Type help to list commands.
¤ change_language en_GB
The language of the command line is English (United Kingdom)
¤ help circle
                                circle args... ⠀                                

Draw circle with given radius an options extent and steps:  CIRCLE 50           
¤ change_language de_DE
Die Sprache der Befehlszeile ist Deutsch (Deutschland)
¤ hilfe kreis
                                kreis args... ⠀                                 

Kreis mit gegebenem Radius zeichnen: KREIS 50                                   
¤ hilfe sprache_wechseln
                           sprache_wechseln sprache ⠀                           

Programmsprache ändern                                                          

 • Parameter:                                                                   
   sprache (Zeichenkette) – Ordnername under locale der die Wörterbücher        
   enthält.Standard ist ‚‘ für die Rückfallsprache Englisch                     
¤ sprache_wechseln 
The language of the command line is fallback language
¤ bye
Thank you for using Turtle