Placeholders

Transifex supports adding Translation checks to fit the translation characteristics.

Adding custom placeholders causes the matched expression to be highlighted, be assigned unique numbers in that string, and to be warned if translation differs from the source string.

Some notes:

• Some roles allow being renamed (e.g. `` :ref:`this is a rename text <actual-link>`_ ``), and the renamed text has great chance of being translatable. Hence, these roles that have whitespaces were not added to the list of place holders, and will be rendered as normal text.

• The roles :dfn: and :term: were not included because their content are normally translatable, so making them placeholders would pop up too many false-positive warnings.

• ``$`` is being wrongly replaced with the placeholder `{TX-PL-LABEL}#x60;` by Transifex because of the custom placeholder that starts and ends with ``. Removing this custom placeholder ceases the issue, but it disables custom placeholders for all literal expressions.

About order of placeholders expression matching:

• Transifex match text to custom placeholders expression from top-down, which means first expressions added as custom placeholder will take precedence over the expression below. This means the order must be chosen wisely to avoid unexpected mismatch of placeholders.

• Order suggestion: - two-part roles (e.g. :c:func:` ``) - one-part roles (e.g. ``:class:` ``) - literals starting with two dashes (e.g. \`\`--bind) - literals starting with one dash (e.g. ``-n``) - literals (e.g. ``None``) - Sphinx variables with links (e.g. |tzdata|_) - Sphinx variables (without links) (e.g. |version|) - URL to end from anonymous links (e.g. ``<URL>`__ `` from ``some text ``) - URL to end from links (e.g. ``<URL>`_ `` from ``some text ``)

Get a list of potential placeholders

Extract strings from docs

To make easier to read the source strings, extract them from the doc files and put it in a temporary file.

1. Before extracting, you need to get source repositories, install dependencies and then generate POT files:

git clone https://github.com/python-docs-translations/transifex-automations
cd python-docs-tx-translations
git clone --depth 1 --branch 3.12 https://github.com/python/cpython
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt -r cpython/Doc/requirements.txt
sphinx-build -E -b gettext -D gettext_compact=0 -d build/.doctrees . locales/pot

2. Now get the sources strings from POT files into a strings.txt

for pot in $(find locales/pot -name '*.pot'); do (msgcat --no-wrap $pot > tmp; mv tmp $pot;); done
grep -h ^msgid $(find locales/pot -name '*.pot') | sed 's|^msgid "||;s|"$||;/^$/d' > strings.txt
git checkout locale

Now, strings.txt have all source strings.

Find potential placeholders from extracted strings

Use the following commands to find potential custom.

Note that the commands sort by alphabetic order sort -u, but it could be sorted by most used placeholders via sort | uniq -c | sort -grk 1 instead.

1. To list two-part roles (e.g. ``:c:func:` ``):

grep -Po '(^|\s+):([\w\-\_\~!]*:){2}`' strings.txt | sed -E 's|^ +||;' | tr '[:upper:]' '[:lower:]' | sort -u

2. To list one-part roles (e.g. ``:class:` ``):

grep -Po '(^|\s+):[\w\-\_\~\!]*:`' strings.txt | sed -E 's|^ +||;' | tr '[:upper:]' '[:lower:]' | sort -u

3. To list Sphinx variables with links (e.g. |tzdata|_):

grep -Po '\|[\w\_\-]*\|_' strings.txt | sed -E 's|^ +||;' | sort -u

4. Now, to list Sphinx variables with links (e.g. |version|):

grep -Po '\|[\w\_\-]*\|' strings.txt | sed -E 's|^ +||;' | sort -u

Note

Put ``literals`` before Sphinx |variables| (with and without links) in the list of custom placeholders in Transifex, otherwise the expression of a non-variable from inside a literal expression might match before actually match the literal itself.

5. Check for URL in anonymous external links (won’t be able to match the whole syntax, but it is a progress):

grep -Po '<[\w:/\_\-\.]*>`__ ' strings.txt | sort -u

6. Check for URL in external links:

grep -Po '<[\w:/\_\-\.]*>`_ ' strings.txt | sort -u

Current Custom Placeholders

This is the list of roles and variables currently set in python-doc organization in Transifex.

Starts with	Ends with	Allows whitespace	Note
:c:data:`	`	no	Here start two-part roles.
:c:enum:`	`	no
:c:expr:`	`	no
:c:func:`	`	no
:c:macro:`	`	no
:c:member:`	`	no
:c:struct:`	`	no
:c:type:`	`	no
:c:var:`	`	no
:py:attr:`	`	no
:py:class:`	`	no
:py:const:`	`	no
:py:data:`	`	no
:py:exc:`	`	no
:py:func:`	`	no
:py:meth:`	`	no
:py:mod:`	`	no
:abbr:`	`	no	Here start the one-part roles.
:attr:`	`	no
:class:`	`	no
:code:`	`	no
:command:`	`	no
:const:`	`	no
:data:`	`	no
:doc:`	`	no
:download:`	`	no
:envvar:`	`	no
:exc:`	`	no
:expr:`	`	no
:file:`	`	no
:func:`	`	no
:gh:`	`	no
:guilabel:`	`	no
:index:`	`	no
:issue:`	`	no
:kbd:`	`	no
:keyword:`	`	no
:macro:`	`	no
:mailheader:`	`	no
:makevar:`	`	no
:manpage:`	`	no
:menuselection:`	`	no
:meth:`	`	no
:mimetype:`	`	no
:mod:`	`	no
:monitoring-event:`	`	no
:newsgroup:`	`	no
:opcode:`	`	no
:option:`–	`	yes
:option:`-	`	yes
:option:`	`	yes
:pdbcmd:`	`	no
:pep:`	`	no
:program:`	`	no
:ref:`	`	no
:rfc:`	`	no
:samp:`	`	yes	This role normally presents shell commands and as such normally includes whitespaces. It possible includes pseudo-variables that should be translated.
:source:`	`	no
:sub:`	`	no
:sup:`	`	no
:sup:`	`	no
:title-reference:`	`	no
:2to3fixer:`	`	no	Put out of alphabetic order, because it has one single occurrence.
``–	``	yes	This is literal role to match double-dash command-line options. Must come before single-dash one because Transifex placeholders are detected according to their order.
``-	``	yes	This is literal role to match single-dash command-line options.
``	``	yes	This matches all literals, and spaces should be allowed.
|	|_ | no		This matches Sphinx variables with links (e.g. ‘|tzdata|_’.
|	| | no		This matches Sphinx variables (e.g. ‘|version|’.
<	>`__ | no		This matches URL part of links (e.g. <https://pypi.org>`__)
<	>`_ | no		This matches URL part of links (e.g. <https://cython.org>`_)