Gettext multiple po files

It is typically used in scripts that run particular programs. Not all programs have translations for all languages. By default, an English message is shown in place of a nonexistent translation. If you understand other languages, you can set up a priority list of languages. On Windows, consoles such as the one started by the cmd. Note: This problem does not exist for Cygwin consoles; these consoles do input and output in the UTF-8 encoding.

Note: This has an effect only on strings looked up in message catalogs; other categories of text are usually not affected by this setting. Languages are not equally well supported in all packages using GNU gettext , and more translations are added over time. Usually, you use the translations that are shipped with the operating system or with particular packages that you install afterwards.

But you can also install newer localizations directly. For doing this, you will need an understanding where each localization file is stored on the file system. Each ll. The GNU gettext toolset helps programmers and translators at producing, updating and using translation files, mainly those PO files which are textual, editable files. This chapter explains the format of PO files. A PO file is made up of many entries, each entry holding the relation between an original untranslated string and its corresponding translation.

All entries in a given PO file usually pertain to a single project, and all translations are expressed in a single target language. One PO file entry has the following schematic structure:. The general structure of a PO file should be well understood by the translator. When using PO mode, very little has to be known about the format details, as PO mode takes care of them for her.

Entries begin with some optional white space. Usually, when generated through GNU gettext tools, there is exactly one blank line between entries.

Then comments follow, on lines all starting with the character. There are two kinds of comments: those which have some white space immediately following the - the translator comments -, which comments are created and maintained exclusively by the translator, and those which have some non-white character just after the - the automatic comments -, which comments are created and maintained automatically by GNU gettext tools.

Comment lines starting with. Comment lines starting with , contain flags; more about these below. Comment lines starting with contain the previous untranslated string for which the translator gave a translation.

After white space and comments, entries show two strings, namely first the untranslated string as it appears in the original program sources, and then, the translation of this string. The original string is introduced by the keyword msgid , and the translation, by msgstr. The msgid strings, as well as automatic comments, are produced and managed by other GNU gettext tools, and PO mode does not provide means for the translator to alter these.

The most she can do is merely deleting them, and only by deleting the whole entry. On the other hand, the msgstr string, as well as translator comments, are really meant for the translator, and PO mode gives her the full control she needs.

The comment lines beginning with , are special because they are not completely ignored by the programs as comments generally are. The comma separated list of flag s is used by the msgfmt program to give the user some better diagnostic messages.

Currently there are two forms of flags defined:. This flag can be generated by the msgmerge program or it can be inserted by the translator herself. It shows that the msgstr string might not be a correct translation anymore. Only the translator can judge if the translation requires further modification, or is acceptable as is. Once satisfied with the translation, she then removes this fuzzy attribute. The msgmerge program inserts this when it combined the msgid and msgstr entries after fuzzy search only.

See Fuzzy Entries. These flags should not be added by a human. Instead only the xgettext program adds them. The c-format flag indicates that the untranslated string and the translation are supposed to be C format strings. When the c-format flag is given for a string the msgfmt program does some more tests to check the validity of the translation.

See msgfmt Invocation , c-format Flag and c-format. Likewise for Java MessageFormat format strings, see java-format. Likewise for Java printf format strings, see java-format. Likewise for Object Pascal, see object-pascal-format. Likewise for the GCC sources, see gcc-internal-format. The context serves to disambiguate messages with the same untranslated-string. It is possible to have several entries with the same untranslated-string in a PO file, provided that they each have a different context.

Note that an empty context string and an absent msgctxt line do not mean the same thing. Here also, a msgctxt context can be specified before msgid , like above. This flag is followed by a range of non-negative numbers, using the syntax range: minimum-value..

It designates the possible values that the numeric parameter of the message can take. In some languages, translators may produce slightly better translations if they know that the value can only take on values between 0 and 10, for example. The previous-untranslated-string is optionally inserted by the msgmerge program, at the same time when it marks a message fuzzy.

It helps the translator to see which changes were done by the developers on the untranslated-string. It happens that some lines, usually whitespace or comments, follow the very last entry of a PO file.

Such lines are not part of any entry, and will be dropped when the PO file is processed by the tools, or may disturb some PO file editors. The remainder of this section may be safely skipped by those using a PO file editor, yet it may be interesting for everybody to have a better idea of the precise format of a PO file. On the other hand, those wishing to modify PO files by hand should carefully continue reading on.

An empty untranslated-string is reserved to contain the header entry with the meta information see Header Entry. This header entry should be the first entry of the file. The empty untranslated-string is reserved for this purpose and must not be used anywhere else.

Each of untranslated-string and translated-string respects the C syntax for a character string, including the surrounding quotes and embedded backslashed escape sequences.

When the time comes to write multi-line strings, one should not use escaped newlines. Instead, a closing quote should follow the last character on the line to be continued, and an opening quote should resume the string at the beginning of the following PO file line.

For example:. In this example, the msgid keyword is followed by three strings, which are meant to be concatenated. Concatenating the empty string does not change the resulting overall string, but it is a way for us to comply with the necessity of msgid to be followed by a string on the same line, while keeping the multi-line presentation left-justified, as we find this to be a cleaner disposition.

Outside strings, white lines and comments may be used freely. For the programmer, changes to the C source code fall into three categories. First, you have to make the localization functions known to all modules needing message translation. Second, you should properly trigger the operation of GNU gettext when the program initializes, usually from the main function.

Last, you should identify, adjust and mark all constant strings in your program needing translation. Presuming that your set of programs, or package, has been adjusted so all needed GNU gettext files are available, and your Makefile files are adjusted see Maintainers , each C module having translated C strings should contain the line:.

The initialization of locale data should be done with more or less the same code in every program, as demonstrated below:. For now consult the gettext or hello sources for more information.

This latter category is responsible for determining character classes with the isalnum etc. The standards say that additional formats but the one known in the "C" locale might be recognized. But some systems seem to reject numbers in the "C" locale format.

In some situation, it might also be a problem with the notation itself which makes it impossible to recognize whether the number is in the "C" locale or the local format. This can happen if thousands separator characters are used. Some locales define this character according to the national conventions to '.

Before strings can be marked for translations, they sometimes need to be adjusted. Usually preparing a string for translation is done right before marking it, during the marking phase which is described in the next sections. What you have to keep in mind while doing that is the following.

Translatable strings should be in good English style. If slang language with abbreviations and shortcuts is used, often translators will not understand the message and will produce very inappropriate translations. This is nearly untranslatable: Is the displayed item a parameter or the parameter? The ambiguity in this message makes it unintelligible: Is the program attempting to set something on fire?

Does it mean "The given object does not match the template"? Does it mean "The template does not fit for any of the objects"?

In both cases, adding more words to the message will help both the translator and the English speaking user. Translatable strings should be entire sentences. It is often not possible to translate single verbs or adjectives in a substitutable way. Change this to. This way the translator will not only understand the message, she will also be able to find the appropriate grammatical construction. A French translator for example translates "write protected" like "protected against writing".

There are usually more interdependencies between words than in English. The consequence is that asking a translator to translate two half-sentences and then combining these two half-sentences through dumb string concatenation will not work, for many languages, even though it would work for English. If a sentence is output using two subsequent printf statements, like this. It is necessary to merge the two printf statements so that the translator can handle the entire sentence at once and decide at which place to insert a line break in the translation if at all :.

Should these two statements merged into a single one? I would recommend to merge them if the two sentences are related to each other, because then it makes it easier for the translator to understand and translate both.

On the other hand, if one of the two messages is a stereotypic one, occurring in other places as well, you will do a favour to the translator by not merging the two.

Identical messages occurring in several places are combined by xgettext, so the translator has to handle them once only. The reason is that when the translatable string changes, the translator is faced with the task of updating the entire translated string.

It is a courtesy towards the translators to split such a message into several ones of five to ten lines each. While doing that, you can also attempt to split the documented options into groups, such as the input options, the output options, and the informative output options. This will help every user to find the option he is looking for. In order to present to the translator only entire sentences, and also because in some languages the translator might want to swap the order of object1 and object2 , it is necessary to change this to use a format string:.

A similar case is compile time concatenation of strings. It expands to a constant string, usually "d" or "ld" or "lld" or something like this, depending on the platform. Assume you have code like. You can therefore simply write. This means, you put the platform dependent code in one statement, and the internationalization code in a different statement. Note that a buffer length of is safe, because all available hardware integer types are limited to bits, and to print a bit integer one needs at most 54 characters, regardless whether in decimal, octal or hexadecimal.

All this applies to other programming languages as well. For example, in Java and C , string concatenation is very frequently used, because it is a compiler built-in operator. Like in C, in Java, you would change. Unusual markup or control characters should not be used in translatable strings.

Translators will likely not understand the particular meaning of the markup or control characters. It might be better to have the translator translate the left-hand and right-hand part separately.

This is flawed. Some translators may convert it to a simple newline, some to blank lines. With some PO file editors it may not be easy to even enter a vertical tab control character.

All strings requiring translation should be marked in the C sources. Marking is done in such a way that each translatable string appears to be the sole argument of some function or preprocessor macro. There are only a few such possible functions or macros meant for translation, and their names are said to be marking keywords. The marking is attached to strings themselves, rather than to what we do with them. This approach has more uses. A blatant example is an error message produced by formatting.

This marking operation has two goals. The first goal of marking is for triggering the retrieval of the translation, at run time. The keyword is possibly resolved into a routine able to dynamically return the proper translation, as far as possible or wanted, for the argument string. Most localizable strings are found in executable positions, that is, attached to variables or given as parameters to functions.

But this is not universal usage, and some translatable strings appear in structured initializations. See Special cases. The second goal of the marking operation is to help xgettext at properly extracting all translatable strings when it scans a set of program sources and produces PO file templates.

However, for packages using the gettext interface more heavily, it is usually more convenient to give the main keyword a shorter, less obtrusive name. Indeed, the keyword might appear on a lot of strings all over the package, and programmers usually do not want nor need their program sources to remind them forcefully, all the time, that they are internationalized.

Further, a long keyword has the disadvantage of using more horizontal space, forcing more indentation work on sources for those trying to keep them within 79 or 80 columns. Further, the coding rule, from GNU standards, wanting that there is a space between the keyword and the opening parenthesis is relaxed, in practice, for this particular usage.

So, the textual overhead per translatable string is reduced to only three characters: the underline and the two parentheses. However, even if GNU gettext uses this convention internally, it does not offer it officially. It is also possible to define marking functions that take it at another argument position. Note also that long strings can be split across lines, into multiple adjacent string tokens. Later on, the maintenance is relatively easy. In PO mode, one set of features is meant more for the programmer than for the translator, and allows him to interactively mark which strings, in a set of program sources, are translatable, and which are not.

Even if it is a fairly easy job for a programmer to find and mark such strings by other means, using any editor of his choice, PO mode makes this work more comfortable. Further, this gives translators who feel a little like programmers, or programmers who feel a little like translators, a tool letting them work at marking translatable strings in the program sources, while simultaneously producing a set of translation in some language, for the package being internationalized. The set of program sources, targeted by the PO mode commands describe here, should have an Emacs tags table constructed for your project, prior to using these PO file commands.

This is easy to do. In any shell window, change the directory to the root of your project, then execute a command resembling:. This command will explore all said files and create a TAGS file in your root directory, somewhat summarizing the contents using a special file format Emacs can understand. For packages following the GNU coding standards, there is a make goal tags or TAGS which constructs the tag files in all directories and for all files containing source code.

Once your TAGS file is ready, the following commands assist the programmer at marking translatable strings in his set of sources. But these commands are necessarily driven from within a PO file window, and it is likely that you do not even have such a PO file yet.

This is not a problem at all, as you may safely open a new, empty PO file, mainly for using these commands. This empty PO file will slowly fill in while you mark strings as translatable in your program sources. Search through program sources for a string which looks like a candidate for translation po-tags-search. Mark the last string found with a keyword taken from a set of possible keywords. This command with a prefix allows some management of these keywords po-select-mark-and-mark. The , po-tags-search command searches for the next occurrence of a string which looks like a possible candidate for translation, and displays the program source in another Emacs window, positioned in such a way that the string is near the top of this other window.

If the string is too big to fit whole in this window, it is positioned so only its end is shown. In any case, the cursor is left in the PO file window. If the shown string would be better presented differently in different native languages, you may mark it using M-, or M-. Otherwise, you might rather ignore it and skip to the next string by merely repeating the , command. A string is a good candidate for translation if it contains a sequence of three or more letters.

A string containing at most two letters in a row will be considered as a candidate if it has more letters than non-letters.

The command disregards strings containing no letters, or isolated letters only. It also disregards strings within comments, or strings already marked with some keyword PO mode knows see below. If you have never told Emacs about some TAGS file to use, the command will request that you specify one from the minibuffer, the first time you use the command.

Each time you use the , command, the search resumes from where it was left by the previous search, and goes through all program sources, obeying the TAGS file, until all sources have been processed. However, by giving a prefix argument to the command C-u , , you may request that the search be restarted all over again from the first program source; but in this case, strings that you recently marked as translatable will be automatically skipped.

Using this , command does not prevent using of other regular Emacs tags commands. For example, regular tags-search or tags-query-replace commands may be used without disrupting the independent , search sequence. However, as implemented, the initial , command or the , command is used with a prefix might also reinitialize the regular Emacs tags searching to the first tags file, this reinitialization might be considered spurious. The M-. Both commands will automatically create a new PO file untranslated entry for the string being marked, and make it the current entry making it easy for you to immediately proceed to its translation, if you feel like doing it right away.

It is possible that the modifications made to the program source by M-, or M-. You may use the O command from PO mode, or any other window changing command from Emacs, to break out into the program source window, and do any needed adjustments. You will have to use some regular Emacs command to return the cursor to the PO file window, if you want command , for the next string, say.

The first such speedup is that you are presented with a preferred keyword, which you may accept by merely typing RET at the prompt. The second speedup is that you may type any non-ambiguous prefix of the keyword you really mean, and the command will complete it automatically for you.

This also means that PO mode has to know all your possible keywords, and that it will not accept mistyped keywords. If you reply? When the command is prefixed by an argument C-u M-. In this case, the command asks for a keyword, written in full, which becomes a new allowed keyword for later M-.

Moreover, this new keyword automatically becomes the preferred keyword for later commands. By typing an already known keyword in response to C-u M-. All keywords known for M-. If many PO files are opened simultaneously, each one has its own independent set of known keywords. There is no provision in PO mode, currently, for deleting a known keyword, you have to quit the file maybe using q and reopen it afresh.

In C programs strings are often used within calls of functions from the printf family. Assume we have the code. A C programmer, even if he cannot speak German, will recognize that there is something wrong here. This will most probably lead to problems because now the length of the string is regarded as the address.

To prevent errors at runtime caused by translations, the msgfmt tool can check statically whether the arguments in the original and the translation string match in type and number. Because not all strings in a program will be format strings, it is not useful for msgfmt to test all the strings in the. This might cause problems because the string might contain what looks like a format specifier, but the string is not used in printf.

Therefore xgettext adds a special tag to those messages it thinks might be a format string. There is no absolute rule for this, only a heuristic. In the. The careful reader now might say that this again can cause problems. The heuristic might guess it wrong. This is true and therefore xgettext knows about a special kind of comment which lets the programmer take over the decision.

If in the same line as or the immediately preceding line to the gettext keyword the xgettext program finds a comment containing the words xgettext:c-format , it will mark the string in any case with the c-format flag. This kind of comment should be used when xgettext does not recognize the string as a format string but it really is one and it should be tested.

Please note that when the comment is in the same line as the gettext keyword, it must be before the string to be translated. This situation happens quite often. The printf function is often called with strings which do not contain a format specifier. Of course one would normally use fputs but it does happen. In this case xgettext does not recognize this as a format string but what happens if the translation introduces a valid format specifier?

The printf function will try to access one of the parameters but none exists because the original code does not pass any parameters. In this case the msgfmt might give too many warnings and would prevent translating the.

The method to prevent this wrong decision is similar to the one used above, only the comment to use must contain the string xgettext:no-c-format. If a string is marked with c-format and this is not correct the user can find out who is responsible for the decision.

See xgettext Invocation to see how the --debug option can be used for solving this problem. The attentive reader might now point out that it is not always possible to mark translatable string with gettext or something like this. Consider the following case:. While it is no problem to mark the string "a default message" it is not possible to mark the string initializers for messages. What is to be done?

We have to fulfill two tasks. First we have to mark the strings so that the xgettext program see xgettext Invocation can find them, and second we have to translate the string at runtime before printing them. The first task can be fulfilled by creating a new keyword, which names a no-op.

For the second we have to mark all access points to a string from the array. So one solution can look like this:. Please convince yourself that the string which is written by fputs is translated in any case. But this has a drawback. A use of gettext could have in rare cases unpredictable results. One advantage is that you need not make control flow analysis to make sure the output is really translated in any case.

But this analysis is generally not very difficult. If it should be in any situation you can use this second method in this situation. Code sometimes has bugs, but translations sometimes have bugs too. The users need to be able to report them. Reporting translation bugs to the programmer or maintainer of a package is not very useful, since the maintainer must never change a translation, except on behalf of the translator.

Hence the translation bugs must be reported to the translators. Here is a way to organize this so that the maintainer does not need to forward translation bug reports, nor even keep a list of the addresses of the translators or their translation teams. Every program has a place where is shows the bug report address. In this place, instruct the translator to add her own bug reporting address. For example, if that code has a statement.

Should names of persons, cities, locations etc. People who only know languages that can be written with Latin letters English, Spanish, French, German, etc. However, in general when translating from one script to another, names are translated too, usually phonetically or by transliteration. For example, Russian or Greek names are converted to the Latin alphabet when being translated to English, and English or French names are converted to the Katakana script when being translated to Japanese.

This is necessary because the speakers of the target language in general cannot read the script the name is originally written in. As a programmer, you should therefore make sure that names are marked for translation, with a special comment telling the translators that it is a proper name and how to pronounce it. In its simple form, it looks like this:.

In this more comfortable form, it looks like this:. As a translator, you should use some care when translating names, because it is frustrating if people see their names mutilated or distorted.

If your language uses the Latin script, all you need to do is to reproduce the name as perfectly as you can within the usual character set of your language.

In this particular case, this means to provide a translation containing the c-cedilla character. If the programmer used the simple case, you should still give, in parentheses, the original writing of the name — for the sake of the people that do read the Latin script. Here is an example, using Greek as the target script:. Because translation of names is such a sensitive domain, it is a good idea to test your translation before submitting it.

When you are preparing a library, not a program, for the use of gettext , only a few details are different. Here we assume that the library has a translation domain and a POT file of its own. If it uses the translation domain and POT file of the main program, then the previous sections apply without changes. The typical idiom used to achieve this is a static boolean variable that indicates whether the initialization function has been called.

Like this:. In other words, dgettext is used instead of gettext. Similarly, the dngettext function should be used in place of the ngettext function. After preparing the sources, the programmer creates a PO template file.

This section explains how to use xgettext for this purpose. You should then rename it to domainname. The answer is: for historical reasons. Up: Template [ Contents ][ Index ]. Read the names of the input files from file instead of getting them from the command line. Add directory to the list of directories. Source files are searched relative to this list of directories. The resulting. Write output to specified file instead of name. Specifies the language of the input files. Specifies the encoding of the input files.

This option is needed only if some untranslated message strings or their corresponding comments contain non-ASCII characters. Place comment blocks starting with tag and preceding keyword lines in the output file. Without a tag , the option means to put all comment blocks preceding keyword lines in the output file.

Note that comment blocks supposed to be extracted must be adjacent to keyword lines. For example, in the following C source code:. The second comment line will not be extracted, because there is one blank line between the comment line and the keyword.

The option has an effect on all input files. To enable or disable checks for a certain string, you can mark it with an xgettext: special comment in the source file. The xgettext: comment can be followed by flags separated with a comma. If a flag is prefixed by no- , the meaning is negated. Some tests apply the checks to each sentence within the msgid, rather than the whole string. The number is specified with the --sentence-end option.

Specify keywordspec as an additional keyword to be looked for. Without a keywordspec , the option means to not use default keywords.

If keywordspec is a C identifier id , xgettext looks for strings in the first argument of each call to the function or macro id. Note that when used through a normal shell command line, the double-quotes around the xcomment need to be escaped. The default keyword specifications, which are always looked for if not explicitly disabled, are language dependent.

They are:. Specifies additional flags for strings occurring as part of the arg th argument of the function word. The effect of this specification is that xgettext will mark as format strings all gettext invocations that occur as arg th argument of function.

Use the flags c-format and possible-c-format to show who was responsible for marking a message as a format string. The latter form is used if the xgettext program decided, the former form is used if the programmer prescribed it. By default only the c-format form is used. The translator should not have to care about these details. This implementation of xgettext is able to process a few awkward cases, like strings in preprocessor macros, ANSI concatenation of adjacent strings, and escaped end of lines for continued strings.

While this mark is there, the translation won't show up in run-time. As long as the fuzzy label is part of the PO entry you provide in the PO translation file, our parser ignores this translation and no changes are applied to the corresponding entry in Transifex. Set up your projects 2. Project Workflows 3. Set up TM 4. Provide Context 5. Translations 6.

Set up a team 7. Communication 8. Manage translation process 9. After translation Sample data. Syntax in PO files. Transifex needs the PO files that are uploaded to Transifex to be syntactically correct. You can check a PO file for syntax errors by running the command:. A valid PO comment should start with the character Comments are handled as developers notes in Transifex and there are 2 ways to define them in your PO file: add entries which have some white space immediately following the character add entries which start with.

It is perfectly fine if you translate a POT file directly and rename it according to the intended locale later. The header of a gettext file might contain a locale name and data for pluralization that Phrase will extract during import:.

Gettext uses the msgctxt notation to distinguish different contexts for the same msgid. In Phrase every key name must be unique thus we will add the msgctxt as the first part of the key name, separated by two pipe symbols :. The fuzzy keyword is used for translators verification. Thus fuzzy will automatically invoke the unverification process within your app.

Back to home. What is Phrase?