Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wip: dynamically load language locale maps #4089

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,7 @@
"url-loader": "^4.1.1",
"web-encoding": "^1.1.5",
"webpack-manifest-plugin": "^2.2.0",
"windows-locale": "^1.1.3",
"workbox-webpack-plugin": "^6.5.4"
},
"dependenciesMeta": {
Expand Down
49 changes: 48 additions & 1 deletion scripts/i18n.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,39 @@ const { writeFileSync } = require('fs')
const package = require('../package.json')
const { locales, sourceLocale } = package.lingui


const lcid = require('windows-locale')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you don't need the whole list.
lingui is actually based over Android-supported locales which are: https://developers.google.com/interactive-media-ads/docs/sdks/android/client-side/localization

(just the underscore char is replaced via dash)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here I started an whole map:

const locales = {
  af: {
    label: "Afrikaans",
    countryCode: "ZA"
  },
  am: {
    label: "ኣማርኛ",
    countryCode: "ET"
  },
  ar: {
    label: "العربية الفصحى",
    countryCode: "AE"
  },
  ar_eg: {
    label: "عربى",
    countryCode: "EG"
  },
  ar_sa: {
    label: "العربية الفصحى",
    countryCode: "SA"
  },
  eu: {
    label: "Euskara",
    countryCode: "ES"
  },
  be: {
    label: "Беларуская",
    countryCode: "BY"
  },
  bn: {
    label: "Bengali",
    countryCode: null
  },
  bg: {
    label: "Bulgarian",
    countryCode: null
  },
  ca: {
    label: "Catalan",
    countryCode: null
  },
  zh: {
    label: "Chinese (Simplified)",
    countryCode: null
  },
  zh_cn: {
    label: "Chinese (Simplified, China)",
    countryCode: null
  },
  zh_hk: {
    label: "Chinese (Traditional, Hong Kong)",
    countryCode: null
  },
  zh_tw: {
    label: "Chinese (Traditional, Taiwan)",
    countryCode: null
  },
  hr: {
    label: "Croatian",
    countryCode: null
  },
  cs: {
    label: "Czech",
    countryCode: null
  },
  da: {
    label: "Danish",
    countryCode: null
  },
  nl: {
    label: "Dutch",
    countryCode: null
  },
  en: {
    label: "English",
    countryCode: null
  },
  en_au: {
    label: "English (Australia)",
    countryCode: null
  },
  en_ca: {
    label: "English (Canada)",
    countryCode: null
  },
  en_gb: {
    label: "English (United Kingdom)",
    countryCode: null
  },
  en_ie: {
    label: "English (Ireland)",
    countryCode: null
  },
  en_in: {
    label: "English (India)",
    countryCode: null
  },
  en_sg: {
    label: "English (Singapore)",
    countryCode: null
  },
  en_za: {
    label: "English (South Africa)",
    countryCode: null
  },
  et: {
    label: "Estonian",
    countryCode: null
  },
  fa: {
    label: "Farsi",
    countryCode: null
  },
  fil: {
    label: "Filipino",
    countryCode: null
  },
  fi: {
    label: "Finnish",
    countryCode: null
  },
  fr: {
    label: "French",
    countryCode: null
  },
  fr_ca: {
    label: "French (Canada)",
    countryCode: null
  },
  fr_ch: {
    label: "French (Switzerland)",
    countryCode: null
  },
  gl: {
    label: "Galician",
    countryCode: null
  },
  de: {
    label: "German",
    countryCode: null
  },
  de_at: {
    label: "German (Austria)",
    countryCode: null
  },
  el: {
    label: "Greek",
    countryCode: null
  },
  gu: {
    label: "Gujarati",
    countryCode: null
  },
  he: {
    label: "Hebrew",
    countryCode: null
  },
  hi: {
    label: "Hindi",
    countryCode: null
  },
  hu: {
    label: "Hungarian",
    countryCode: null
  },
  is: {
    label: "Icelandic",
    countryCode: null
  },
  id: {
    label: "Indonesian",
    countryCode: null
  },
  it: {
    label: "Italian",
    countryCode: null
  },
  ja: {
    label: "Japanese",
    countryCode: null
  },
  kn: {
    label: "Kannada",
    countryCode: null
  },
  ko: {
    label: "Korean",
    countryCode: null
  },
  lo: {
    label: "Lao",
    countryCode: null
  },
  lv: {
    label: "Latvian",
    countryCode: null
  },
  ln: {
    label: "Lingala",
    countryCode: null
  },
  lt: {
    label: "Lithuanian",
    countryCode: null
  },
  ms: {
    label: "Malay",
    countryCode: null
  },
  ml: {
    label: "Malayalam",
    countryCode: null
  },
  mr: {
    label: "Marathi",
    countryCode: null
  },
  no: {
    label: "Norwegian",
    countryCode: null
  },
  pl: {
    label: "Polish",
    countryCode: null
  },
  pt: {
    label: "Portuguese",
    countryCode: null
  },
  pt_br: {
    label: "Portuguese (Brazil)",
    countryCode: null
  },
  pt_pt: {
    label: "Portuguese (Portugal)",
    countryCode: null
  },
  ro: {
    label: "Romanian",
    countryCode: null
  },
  ru: {
    label: "Russian",
    countryCode: null
  },
  sr: {
    label: "Serbian",
    countryCode: null
  },
  sk: {
    label: "Slovak",
    countryCode: null
  },
  sl: {
    label: "Slovenian",
    countryCode: null
  },
  es: {
    label: "Spanish",
    countryCode: null
  },
  es_419: {
    label: "Spanish (Latin America)",
    countryCode: null
  },
  es_ar: {
    label: "Spanish (Argentina)",
    countryCode: null
  },
  es_cl: {
    label: "Spanish (Chile)",
    countryCode: null
  },
  es_co: {
    label: "Spanish (Colombia)",
    countryCode: null
  },
  es_cr: {
    label: "Spanish (Costa Rica)",
    countryCode: null
  },
  es_do: {
    label: "Spanish (Dominican Republic)",
    countryCode: null
  },
  es_ec: {
    label: "Spanish (Ecuador)",
    countryCode: null
  },
  es_sv: {
    label: "Spanish (El Salvador)",
    countryCode: null
  },
  es_gt: {
    label: "Spanish (Guatemala)",
    countryCode: null
  },
  es_hn: {
    label: "Spanish (Honduras)",
    countryCode: null
  },
  es_mx: {
    label: "Spanish (Mexico)",
    countryCode: null
  },
  es_ni: {
    label: "Spanish (Nicaragua)",
    countryCode: null
  },
  es_pa: {
    label: "Spanish (Panama)",
    countryCode: null
  },
  es_pe: {
    label: "Spanish (Peru)",
    countryCode: null
  },
  es_pr: {
    label: "Spanish (Puerto Rico)",
    countryCode: null
  },
  es_py: {
    label: "Spanish (Paraguay)",
    countryCode: null
  },
  es_us: {
    label: "Spanish (United States)",
    countryCode: null
  },
  es_uy: {
    label: "Spanish (Uruguay)",
    countryCode: null
  },
  es_ve: {
    label: "Spanish (Venezuela)",
    countryCode: null
  },
  sw: {
    label: "Swahili",
    countryCode: null
  },
  sv: {
    label: "Swedish",
    countryCode: null
  },
  gsw: {
    label: "Swiss German",
    countryCode: null
  },
  tl: {
    label: "Tagalog",
    countryCode: null
  },
  ta: {
    label: "Tamil",
    countryCode: null
  },
  te: {
    label: "Telugu",
    countryCode: null
  },
  th: {
    label: "Thai",
    countryCode: null
  },
  tr: {
    label: "Turkish",
    countryCode: null
  },
  uk: {
    label: "Ukrainian",
    countryCode: null
  },
  ur: {
    label: "Urdu",
    countryCode: null
  },
  vi: {
    label: "Vietnamese",
    countryCode: null
  },
  zu: {
    label: "Zulu",
    countryCode: null
  }
}

how to fill it

  1. look at English name, find translation at https://www.omniglot.com/language/names.htm , fill own name in 'label' prop
  2. click on the locale own name at this site, you will see detailed article above language and in which country (ies) it's used.
  3. pick up the proper country code from here https://countrycode.org

you could ask @patpedrosa to do it or work on it if you don't have more prioritised tasks
for now I suggest to fill this map with languages we currently support only then merge PR and open a task to fill the whole list


// setup locale map for language selector

// some locales are not directly mappable to a country code and then would fail to retreive a flag icon
const customLocaleMap = [
{ code: 'en', countryCode: 'US', name: 'English-United States' },
{ code: 'es-419', countryCode: 'AR', name: 'Español Latinoamericano' },
{ code: 'hi', countryCode: 'IN', name: 'हिन्दी' },
{ code: 'zh', countryCode: 'CN', name: '中文' },
{ code: 'uk', countryCode: 'UA', name: 'Українська' },
{ code: 'ko', countryCode: 'KR', name: '한국어' },
{ code: 'vi', countryCode: 'VN', name: 'Tiếng Việt' },
{ code: 'es-us', countryCode: 'UM', name: 'Español-United States' },
]

// filter custom locales out of the locales array
const filteredLocales = locales.filter(locale => !customLocaleMap.map(customLocale => customLocale.code).includes(locale));

// map the filtered locales to a country code (used for retreiving flag icons)
// and their language name (used in language selector)
const localeCodeMap = filteredLocales.map(locale => {

Check notice

Code scanning / CodeQL

Unused variable, import, function or class

Unused variable localeCodeMap.
const { language, location } = lcid[locale]
const countryCode = locale.split('-').at(-1).toUpperCase()

// todo: get native translation of language name
const name = language + (location ? '-' + location : '')

return customLocaleMap.push({ code: locale, countryCode: countryCode, name: name })
})

// Prepare build + json files
const localeCode = locale => `'${locale}'`
const localeKey = locale => locale.includes('-') ? localeCode(locale) : locale
const localeFilesPath = join(__dirname, '../src/language/locales', 'index.js')
Expand All @@ -24,11 +57,25 @@ export const sourceLocale = <%= localeCode(sourceLocale) %>
export const localeFiles = {<% _.forEach(locales, function(locale) { %>
<%= localeKey(locale) %>: () => import('./<%= locale %>/catalog.js'),<% }) %>
}

// for supported country list in language selector
export const countryCodes = [<% _.forEach(customLocaleMap, function(locale) { %>
'<%= locale.countryCode %>',<% }) %>
]

export const countryCodeToLocale = {<% _.forEach(customLocaleMap, function(locale) { %>
<%= localeCode(locale.countryCode) %>: '<%= locale.code %>',<% }) %>
}

export const languageLabels = {<% _.forEach(customLocaleMap, function(locale) { %>
<%= localeCode(locale.countryCode) %>: '<%= locale.name %>',<% }) %>
}
`)

writeFileSync(localeFilesPath, localeFilesTmpl({
sourceLocale,
locales,
localeKey,
localeCode
localeCode,
customLocaleMap
}))
46 changes: 5 additions & 41 deletions src/components/dashboard/Settings.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import { useDebounce } from 'use-debounce'
import Wrapper from '../common/layout/Wrapper'
import { Icon, Section, Text } from '../common'
import { LanguageContext } from '../../language/i18n'
import { countryCodes, countryCodeToLocale, languageLabels } from '../../language/locales'
import { CountryFlag } from '../profile/ProfileDataTable'

// hooks
Expand Down Expand Up @@ -66,43 +67,6 @@ const PrivacyOption = ({ title, value, field, setPrivacy }) => {
</RadioButton.Group>
)
}
const supportedCountryCodes = ['US', 'GB', 'ES', 'FR', 'IT', 'KR', 'BR', 'UA', 'TR', 'VN', 'CN', 'IN', 'ID', 'AR']
type CountryCode = $ElementType<typeof supportedCountryCodes, number>

const countryCodeToLocale: { [key: CountryCode]: string } = {
US: 'en',
GB: 'en-gb',
ES: 'es',
FR: 'fr',
IT: 'it',
KR: 'ko',
BR: 'pt-br',
UA: 'uk',
TR: 'tr',
VN: 'vi',
CN: 'zh',
IN: 'hi',
ID: 'id',
AR: 'es-419',
}

const languageCustomLabels: { [key: CountryCode]: string } = {
US: 'English-US',
GB: 'English-UK',
ES: 'Spanish',
FR: 'French',
IT: 'Italian',
KR: 'Korean',
DE: 'German',
BR: 'Portuguese-Brazilian',
UA: 'Ukrainian',
TR: 'Turkish',
VN: 'Vietnamese',
CN: 'Chinese-Simplified',
IN: 'Hindi',
ID: 'Indonesian',
AR: 'Latin-Spanish',
}

const getKeyByValue = (object, value) => {
return Object.keys(object).find(key => object[key] === value)
Expand All @@ -111,7 +75,7 @@ const getKeyByValue = (object, value) => {
const DropDownRowComponent = props => {
const { containerStyles, textStyles, children } = props
const { children: countryCode } = children.props
const countryLabel = languageCustomLabels[countryCode] ?? 'Device Default'
const countryLabel = languageLabels[countryCode] ?? 'Device Default'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

declare

const getLabel = code => languageLabels[code] ?? t`Device default`

helper outside component and use it


return (
<TouchableOpacity {...containerStyles} onPress={props.onPress}>
Expand Down Expand Up @@ -291,13 +255,13 @@ const Settings = ({ screenProps, styles, theme, navigation }) => {
<Section.Row style={styles.languageRow}>
<View style={styles.languageInputContainer}>
<ModalDropdown
defaultValue={languageCustomLabels[countryCode] ?? t`Select a language...`}
options={[isWeb ? '' : 'DD', ...supportedCountryCodes]} // empty string breaks on native
defaultValue={languageLabels[countryCode] ?? t`Select a language...`}
options={[isWeb ? '' : 'DD', ...countryCodes]} // empty string breaks on native
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why we still using this 'DD' ? empty value should mean 'system default'.

Copy link
Collaborator Author

@L03TJ3 L03TJ3 Sep 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We removed it but I had to turn it back here (on android)
the component is not able to handle a empty string when rendered on android (as comment points out)

alignOptionsToRight={true}
saveScrollPosition={false}
showsVerticalScrollIndicator={true}
renderButtonText={option => {
const language = languageCustomLabels[option] ?? 'Device Default'
const language = languageLabels[option] ?? 'Device Default'
return t`${language}`
}}
renderRowComponent={DropDownRowComponent}
Expand Down
8 changes: 8 additions & 0 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -4403,6 +4403,7 @@ __metadata:
webpack-bundle-analyzer: ^4.5.0
webpack-manifest-plugin: ^2.2.0
websocket: ^1.0.31
windows-locale: ^1.1.3
workbox-webpack-plugin: ^6.5.4
languageName: unknown
linkType: soft
Expand Down Expand Up @@ -41364,6 +41365,13 @@ __metadata:
languageName: node
linkType: hard

"windows-locale@npm:^1.1.3":
version: 1.1.3
resolution: "windows-locale@npm:1.1.3"
checksum: c3c4a3887212d94442bf3d3c205b54ade1fbbf352ba83f7d32de6cceea8f14d3605ce16b88d9f4735448c0d9e1c7dd2db2afcaa04bc0da545a265aca832f1d57
languageName: node
linkType: hard

"winston@npm:2.x":
version: 2.4.7
resolution: "winston@npm:2.4.7"
Expand Down