Yii and PHP i18n & L10n

Goals

• Learn what's i18n, L10n
• Learn about how it's done
• Avoid common mistakes
• Explain what PHP intl really is
• Data translation

What's i18n?

Internationalization == globalization

Ability of an app to be adapted to various languages and regions without engineering changes.

i.e. regardless of locale chosen. Developer's tasks.

What's L10n?

Localization

The process of adapting software for a specific region or language by adding locale-specific components and translating text.

Translator tasks.

Locale?

A a data set that includes info about language, country, formats, currencies etc. Usually varies per country. Sometimes there are sub-variations.

i18n problems (developers)

• Support various characters (utf8, right collation)
• LTR / RTL
• String translation
• Locale switching
• Language/locale aware formatting (dates, currencies, geographical names)
• Language/locale based substitutions
• Timezones
• ...

Approaches

• ICU (PHP intl, Yii 2.0, Aura, Java)
• gettext (Django, Yii 2.0, Yii 1.1)
• Custom stuff based on CLDR. Not as powerful (Yii 1.1, RoR, Symfony, Laravel)
• Totally custom or just string replacement. Meh

intl

• PHP extension.
• A wrapper around icu4c.
• Uses CLDR.
• Awful docs.
• Has issues.

CLDR

Unicode Common Locale Data Repository

ICU

Libraries for C and Java that use CLDR data for message formatting, locale selection etc.

What intl can do

• Collator - locale-aware string collation
• Locale - various locale utils
  • Locale::acceptFromHttp
  • Locale::canonicalize
• IntlCalendar - locale-aware calendar
• ResourceBundle - loads resources (later)
• Transliterator - yum!
• IntlBreakIterator - locating boundaries in text
• UConverter - encoding converter
• NumberFormatter - locale-aware number formatting
• MessageFormatter - powerful message formatter
• and more

Message formatting

Numbers

echo \MessageFormatter::formatMessage(
    'en_US', 'Value: {value, number}', ['value' => 123456,789]
);
// Value: 123,456.789

echo \MessageFormatter::formatMessage(
    'en_US', 'Price: {price, number, currency}', ['price' => 100]
);
// Price: $100.00

echo \MessageFormatter::formatMessage(
    'en_US', 'Value: {value, number, percent}', ['value' => 123]
);
// Value: 123%
                    

Date and time

echo \MessageFormatter::formatMessage(
    'en_US', 'Date: {d, date, short} | {d, date, medium} | {d, date, long}
     | {d, date, full}', ['d' => $d]
);
// Date: 4/18/15 | Apr 18, 2015 | April 18, 2015
   | Saturday, April 18, 2015

echo \MessageFormatter::formatMessage(
    'ru_UA', 'Date: {d, date, short} | {d, date, medium} | {d, date, long}
     | {d, date, full}', ['d' => $d]
);

// Дата: 18.04.15 | 18 апр. 2015 г. | 18 апреля 2015 г.
   | суббота, 18 апреля 2015 г.
                    

Spellout

echo \MessageFormatter::formatMessage(
    'en_US', '{n,number} is spelled as {n, spellout}', ['n' => 42]
);
// 42 is spelled as forty-two

echo \MessageFormatter::formatMessage(
    'en_US', 'I am {n, spellout,%spellout-ordinal} agent', ['n' => 47]
);
// I am forty-seventh agent

Plurals

$message = 'Здесь {n, plural, =0{котов нет} =1{есть один кот} one{# кот}
            few{# кота} many{# котов} other{# кота}}!';
echo \MessageFormatter::formatMessage('ru_UA', $message, ['n' => 1]);
// Здесь есть один кот!

$message = 'There {n, plural, =0{are no cats}
            =1{is one cat} other{are # cats}}!';
echo \MessageFormatter::formatMessage('en_US', $message, ['n' => 1]);
// There is one cat!

Ordinals

echo \Yii::t('app', 'You are the {n, ordinal} visitor here!', ['n' => 42]);
// You are the 42nd visitor here!

Selection

echo \Yii::t('app', '{name} is a {gender} and {gender, select, female{she}
    male{he} other{it}} loves Yii!', [
    'name' => 'Snoopy',
    'gender' => 'dog',
]);
// Snoopy is a dog and it loves Yii!

There are more but... enough

Intl is powerful!

intl problems

• Docs
• Named parameter issues
• Sometimes it's not installed
• Sometimes ICU is outdated

Named parameter issues

• Named parameters fail w/ PHP < 5.5.0 or INTL < 4.8.
• Named parameters fail w/ selectordinal in latest PHP versions.

Positional ones could be used:

$message = 'There {0, plural, =0{are no cats}
            =1{is one cat} other{are # cats}} except {1}!';
echo \MessageFormatter::formatMessage('en_US', $message, [1, 'Simon']);
                    

intl not installed

Outdated intl or ICU data

• http://nix-tips.ru/obnovlenie-icu-na-centosred-hatfedora.html
• http://habrahabr.ru/post/254789/

Hidden gems

intl relies on CLDR data which is kinda more than is exposed via extension API. Can we access raw data?

Yes!

intl manual lies to us

new \ResourceBundle($resourceFileName, $resourceDirName);

private function dumpResourceBundle($bundle, $depth = 0)
{
    if ($bundle === null) {
        return 'NULL';
    } elseif (is_scalar($bundle)) {
        return $bundle;
    }

    $out = '';
    foreach ($bundle as $k => $v) {
        $out .= str_repeat(' ', $depth) . $k . ' = ';
        if ($v instanceof \ResourceBundle || is_array($v)) {
            $out .= " [\n";
            $out .= $this->dumpResourceBundle($v, $depth + 1);
            $out .= str_repeat(' ', $depth) . "\n]\n";
        }
        else {
            $out .= $v . "\n";
        }
    }
    return $out;
}
                    

new \ResourceBundle('en_UK', null);

Instead of en_UK try more locales. If there's no data, try en.

• month names
• weekday names
• quarter names
• various formats for various calendars
• units
• measurement system names

new \ResourceBundle('en_US', ...);

• ICUDATA-curr = currencies
• ICUDATA-region = regions
• ICUDATA-zone = time zone names
• ICUDATA-lang = languages

Less useful:

• ICUDATA-rbnf - rule based number format (spellout)
• ICUDATA-translit - transliteration rules and tables (huge!)

new \ResourceBundle(..., null);

• metadata = language aliases, region codes, script aliases, territory aliases, variant aliases
• plurals = rules for plural and selectordinal
• zoneinfo64 = Olson DB?!

Research results

Give it a locale and it will show you plurals format, selectordinal format etc.

L18n problems (translators)

As little work as possible

• Translation itself
• Template adjustments (Colors, legal requirements)

Source strings

English or keys.in.english.

Message storage format

Doens't really matter.

PHP arrays are OK for non-tech people.

Don't try Google Translate ;)

Message scanner

A tool that scans code and updates translation files automatically adding new strings and removing old ones.

We have it in Yii ;)

Data translation

How to store posts in X languages?

• UTF-8
• Filter by language

Many columns

• - Complex query
• - Hard to add languages

Same table, multiple records

• ! Each language record is a separate independent one
• + Simple
• + Easy to add translations

Single record + translation in another table

• + Simple maintenance
• + Easy to add translations
• + Only one join

Always normalize language!

Use ISO-639-2: es, en, fr or RFC-5645: zh_CN, ru_UA.

Locale::canonicalize($language);

More reading

• intl docs
• ICU
• CLDR

Questions time!

• http://slides.rmcreative.ru/2015/fwdays-i18n-l10n/
• http://www.yiiframework.com/