How to support this blog?

To support this blog, you can hire me as an OmegaT consultant/trainer, or you can send translation and project management jobs my way.

Search the site:

Showing posts with label Okapi. Show all posts
Showing posts with label Okapi. Show all posts

OmegaT and the Okapi Framework plugin for OmegaT

Introducing the Okapi Framework plugin for OmegaT


This is a summary for people who want a "shortcut".

[Updated Feb. 1st 2022 with link and version updates]

The Okapi Framework is "a cross-platform and free open-source set of components and applications that offer extensive support for localizing and translating documentation and software."


From very early on the people behind the Okapi Framework have helped OmegaT users with a number of facilities, among which the "Filters Plugin for OmegaT", which brings transparent support for file formats such as:

  • Doxygen-commented files
  • HTML files
  • InDesign IDML files
  • JSON files
  • Markdown projects.
  • Transifex projects.
  • TS files
  • Trados TagEditor TTX files
  • Wordfast pro TXML files
  • XLIFF 1.2 files
  • XLIFF 2.x files
  • XML files (with or without ITS support)
  • YAML files

Basic requirement

Make sure you have a recent version of OmegaT (use version 4 and above).

To use the plugin in OmegaT

  1. Enter your OmegaT configuration folder by using the Options > Access Configuration Folder menu in OmegaT
  2. If there is no "plugins" folder in that configuration folder, create one
    Create a "plugins" folder in the OmegaT configuration folder if it is not already there.
    Create the "plugins" folder if it is not already there.
  3. Download the plugin from the distribution site (1.9-1.41 at the time of this writing): https://okapiframework.org/wiki/index.php?title=Okapi_Filters_Plugin_for_OmegaT#Download_and_Installation 
  4. Unzip if necessary, check the readme.html file if you want
  5. Copy the okapiFiltersForOmegaT-1.9-1.41.jar file to the OmegaT plugins folder that you identified above
  6. Restart OmegaT
Once you have installed the plugin, open OmegaT's File Filters preferences (Options > File Filters...) You should see a number of filters that have their name appended with "(Okapi)".

The Okapi plugin provides a number of additional filters that are identified in the OmegaT interface by the string "(Okapi)" appended to their names
The additional filters provided by the Okapi plugin

Et voilà.

Now, 3 things

  1. [obvious, but it doesn't cost much to repeat]
    If anything doesn't work as planned, carefully read the information in the links above and see that you have followed all the instructions and checked all the requirements. This article is only a summary.
  2. [beginner level]
    OmegaT already has native filters for some of the Okapi filters. If you want OmegaT to use the Okapi filters, uncheck the native filters.
  3. [advanced level]
    The Okapi filters have a default behavior that is reasonable enough. If you want to modify that behavior, you'll need to create a custom filter parameters file (
    .fprm extension) from within Rainbow. You'll use OmegaT's filter specific options (select the Okapi filter in OmegaT's Filter Filters interface, click on "Options...") to select that file so that OmegaT uses it.

    It is possible to customize the Okapi filters behavior by adding parameters files created in Okapi's Rainbow
    Adding a parameter file to customize the Okapi XLIFF filter behavior

XLIFF Editors...

[Updated on March 21 2021 to include info on the DGT OmegaT plugin, and the new Swordfish version]

There are plenty of "XLIFF Editors" that pop up on the Mac App market. A number of them are free, most are not super expensive. But all confuse "XLIFF" with the poor subset that Xcode outputs. And that confuses the people who need XLIFF editors the most: professional translators.

XLIFF is an industry standard used in all the localization/translation world. Xcode developers who need the ability to edit their output have all the rights to create quick tools that will help them with that task. But please, don't call that XLIFF editors. What you do is Xcode l10n files editors, basically just XML simple parsers outputing the resulting data in a 2 column table with a native GUI. That's pretty much it. And that's very fine. But it's not XLIFF.

If any of those developers had actually worked with a professional translator to see what are the features required to work with XLIFF (and all the other related standards: TMX, SRX, TBX, ITS, etc.) they would never call their tool an XLIFF editor, just like TextEdit is able to edit XML but nobody would think of calling it an XML Editor...

Mac developers are very picky when it comes to what looks good. Good for them. But would they rather develop in Xcode or in TextEdit? Professional translators on the Mac who need to work with XLIFF currently have the following not so good looking but rock solid choices (all Java based, by the way):

XLIFF is a serious standard, and translators need rock solid standard support to work. If your editor does not have inline tag/segmentation/legacy translation support, call it anything but XLIFF Editor, please.

Also, this is not a rant. This is a reminder: there is a market for robust native pro-level translation tools on Mac. With Microsoft Office for Mac feature for feature equivalent to the Windows version, translators and localizers have little need to stay on Windows machines. Except that the biggest pro-apps are Windows only. And that's a shame.

Using XLIFF as your point of entry into the l10n world is a good and relatively easy way to access a large market (pro conversion engines to and from XLIFF already exist: see the Okapi Framework). But the point where you can compete with the incumbent and actually make money is way higher than what you think.

Rainbow, XLIFF and OmegaT

(I've updated the contents of the file to make things clearer for people who are not familiar with XLIFF - 08/05)

OmegaT's support for complex XLIFF is not really optimal. I'm going to spare you the details but basically, unless you tweak the file significantly, OmegaT is likely not going to be able to leverage all the available data.


Let's say that we have a XLIFF file with the following data:

First segment, not yet translated:
ba bi bu
In OmegaT, this part should be in a supported source file located in /source/

Reference translation from a first reference file:
ba bi, translated to バ ビ, matches the segment to translate at 66%
In OmegaT, this part should be in a TMX file located in /tm/

Reference translation from a second reference file:
ba bi bu be, translated to バ ビ ブ ベ, matches the segment to translate at 75%
This part too should be in a TMX file located in /tm/

Second segment, already translated in the same document:
ba bi, translated to バ ビ
Since this data comes from the same document and is a validated translation, it should be in the project_save.tmx file located in /omegat/

Such an XLIFF file would look like this:

First segment, not yet translated(<target> is empty):

<trans-unit approved="no" id="source_file">
<source xml:lang="en">ba bi bu</source>
<target state="new" xml:lang="ja"/>

First reference translation for that segment:

<alt-trans match-quality="66" origin="first_reference_translation">
<source xml:lang="en">ba bi</source>
<target xml:lang="ja">バ  ビ</target>
</alt-trans>

Second reference translation for that segment:

<alt-trans match-quality="75" origin="second_reference_translation">
<source xml:lang="en">ba bi bu be</source>
<target xml:lang="ja">バ ビ ブ ベ</target>
</alt-trans>
</trans-unit>

(↑ end of the first segment)

Second segment, already translated in that document:

<trans-unit approved="yes" id="source_file">
<source xml:lang="en">ba bi</source>
<target xml:lang="ja">バ  ビ</target>
</trans-unit>



If you want to know more about the XLIFF specification, look at this page: XLIFF Version 1.2


Now, let's see how Rainbow can help us leveraging all the data directly from within OmegaT.


Rainbow


Here comes Rainbow. I'm mentioned Rainbow, or the Okapi Framework a number of times already.

Rainbow is a Java software that acts as a swiss army knife for localization processes. It is a filter to/from complex formats, it is a batch search/replace tool etc.

Don't forget to see the Okapi Webinar that was given a few weeks ago by Yves Savourel. It is about 1h long and goes into great details to show you what Rainbow/Okapi is about. OmegaT is used as an example too and the following text can be considered as a detailed explanation of what you see in the Webinar.

Webinar: http://www.translate.com/Language_Tech_Center/Webinar_Portal.aspx?id=132

The Okapi group has decided to frequently release snapshots of the code so that users don't have to wait for milestones to get a taste of the most recent features.
The snapshots are available here: http://okapi.opentag.com/snapshots/. Get the rainbow_carbon-macosx_... file.

When you unzip the package, you will find a rainbow.sh script file. Open the Terminal where the file is located, make the file executable by typing the following command at the prompt:
chmod +x rainbow.sh

Then, call the script from Terminal to launch Rainbow.


When you launch Rainbow, you're displayed with an empty file list.

  1. drag & drop your XLIFF file on that list. It should be the list corresponding to the "Input List 1" tab.
  2. click on the "languages and encoding" tab and set the appropriate information. For example, an English manual translated to Japanese should be:
    • Source=EN (you can use a variant, like EN-US)
    • Encoding=UTF-8
    • Target=JA
    • Encoding=UTF-8
  3. Then go to the "Utilities > Translation Package Creation"
  4. Select OmegaT
  5. click on the Package Location tab
  6. Select a directory where Rainbow will create the OmegaT project
  7. Click OK.


You're back to the main window where the status bar indicates that the XLIFF file has been processed. What has Rainbow done to your XLIFF file ?

XLIFF



A XLIFF file is a mix of a number of things. First, it contains strings to translate in <trans-unit> sections:

<trans-unit approved="no" id="source_file">
<source xml:lang="en">ba bi bu</source>
<target state="new" xml:lang="ja"/>
</trans-unit>


Then, it may contain reference data. The first type is, obviously, what has already been translated, in similar <trans-unit> sections:

<trans-unit approved="yes" id="source_file">
<source xml:lang="en">ba bi</source>
<target xml:lang="ja">バ  ビ</target>
</trans-unit>


The other type is what is embedded to serve as reference, it comes as <alt-trans> sections within a <trans-unit>:

<trans-unit approved="no" id="source_file">
<source xml:lang="en">ba bi bu</source>
<target state="new" xml:lang="ja"/>
<alt-trans match-quality="66" origin="first_reference_translation">
<source xml:lang="en">ba bi</source>
<target xml:lang="ja">バ  ビ</target>
</alt-trans>
<alt-trans match-quality="75" origin="second_reference_translation">
<source xml:lang="en">ba bi bu be</source>
<target xml:lang="ja">バ ビ ブ ベ</target>
</alt-trans>
</trans-unit>




When Rainbow is used to create OmegaT projects from XLIFF files it looks for the 3 types of information and separates them in a way that OmegaT can manage.


  1. OmegaT cannot manage "empty" <target> elements in <trans-unit> sections. So, Rainbow recreates the source file so that it contains the <source> data in the <target> section. That way, OmegaT only has to overwrite that data:

    <trans-unit id="source_file">
    <source xml:lang="en">ba bi bu</source>
    <target xml:lang="ja">ba bi bu</target>
    </trans-unit>


  2. OmegaT cannot manage already translated data in the XLIFF file. So, Rainbow creates a typical "already translated" OmegaT data file that contains the already translated segments: the project_save.tmx located in the /omegat/ folder of the project:

    <tu tuid="source_file">
    <tuv xml:lang="en"><seg>ba bi</seg></tuv>
    <tuv xml:lang="ja"><seg>バ ビ</seg></tuv>
    </tu>


  3. OmegaT cannot managet reference data in the XLIFF file. So, Rainbow creates a typical "reference translation" OmegaT data file that contains all the embedded translation segments: the alternate.tmx located in the /tm/ folder of the project:

    <tu tuid="first_reference_translation">
    <tuv xml:lang="en"><seg>ba bi</seg></tuv>
    <tuv xml:lang="ja"><seg>バ ビ</seg></tuv>
    </tu>

    <tu tuid="second_reference_translation">
    <tuv xml:lang="en"><seg>ba bi bu be</seg></tuv>
    <tuv xml:lang="ja"><seg>バ ビ ブ ベ</seg></tuv>
    </tu>




That way, OmegaT can seamlessly access all the data available in that complex XLIFF file. Since Rainbow creates a whole OmegaT project, the only thing necessary to proceed is to open the project with OmegaT and translate.

Delivery



Nothing could be simpler. When you are done with your translation, create the translated files from within OmegaT.

Then, drag and drop the manifest.xml file located at the root of the OmegaT project created by Rainbow on Rainbow's main window where the Input List 1 tab is displayed. When manifest.xml is visible on the list, use the "Utilities > Translation Package Post-Processing..." menu.

Rainbow will use all the data it saved in the project to recreate a "proper" XLIFF file for delivery. That file contains all the data that was contained by the original XLIFF, its structure is identical. The only difference is that now it includes the data you have translated in OmegaT.

Okapi webinar

A message from Okapi's Yves Savourel:


Just a note to let you know that I'll be giving a little introduction on using the Okapi Tools. Note that it will be focused on the new Java-based tools.

ENLASO is hosting this free webinar on July 16th at noon Mountain Time.
It's free and you can register here:
https://enlasoevents.webex.com/enlasoevents/onstage/g.php?t=a&d=662713198


The latest version of the Java Okapi tools is here:
http://okapi.opentag.com/

Okapi tools on OSX...

In my "Okapi tools for Mono" post, last November, I discussed the possibility to use Okapi on OSX without installing Parallels/VMWare/WhateverVirtualSolutionYouPrefer and Windows and .NET... Which in the end makes Okapi a very expensive free tool...

Okapi has recently been released to work on Mono, the free (as a bird) version of .NET and I've promised myself to install everything and try the package as soon as I'd have a job requiring it.

This job has come. At last. It is a small set of InDesign files (that will eventually end up being Illustrator files) that are part of a much bigger translation package...

So, at the time of writing, here is the procedure you need to follow to get everything to work:

(Beware, Okapi for mono does not come with a graphical interface and everything must be done in the Terminal, at the command line. It is not terribly hard but not super trivial either for the translator who is not used to that. For people who need an introduction to working with the Terminal on OSX, check this link.)


  1. Download Mono (1.2.6 at the time of the post):

    http://www.go-mono.com/mono-downloads/download.html

  2. Install it by clicking on the package file and following the instructions.

  3. Download Okapi for Mono (R00020 at the time of the post):

    http://sourceforge.net/project/showfiles.php?group_id=42949

  4. Unzip/untar the package and read the ReadMe_ForMono.htm file found in the Tikal folder

  5. Do what the file says under "Using the Okapi tools"

    (for some reason, I am yet unable to properly set the path to Okapi...)



When this is done, you'll need to take a look at the Mono for OSX page to know how to use the beast.

Basically, it comes down to opening the Terminal and typing:

$ mono ~/bin/OkapiMono/Tikal/Tikal.exe

Here, I've left my path to Tikal because I could not set it properly in the install step. But a proper install should accept:

$ mono Tikal.exe

Running only Tikal.exe without any arguments will have the application display its basic help where you'll see what you need to type to do a few basic things, like:

$ mono Tikal.exe -lf
to list the available filters.

Just for your information, since you can find that on the Okapi help pages too, here are the filters that are directly available from the Mono command line:

  • okf_po (Okapi PO Filter)

  • okf_properties (Okapi Properties Filter)

  • okf_script (Okapi Script Filter)

  • okf_netres (Okapi NETRes Filter)

  • okf_rc (Okapi RC Filter)

  • okf_xml (Okapi XML Filter)

  • okf_wordfast (Okapi Wordfast Filter)

  • okf_tradostext (Okapi Trados Text Filter)

  • okf_illustrator (Okapi Illustrator Filter)

  • okf_table (Okapi Table Filter)

  • okf_inx (Okapi INX Filter)

  • okf_html (Okapi HTML Filter (ALPHA TEST ONLY)
  • )
  • okf_json (Okapi JSON Filter)



If you use the parameter -lu you'll get a list of available utilities:

  • oku_set01:extraction

  • oku_set01:merging

  • oku_set01:rewriting

  • oku_set01:encodingconversion

  • oku_set01:lbconversion

  • oku_set01:bomconversion

  • oku_set02:qualitycheck

  • oku_set02:update

  • oku_set02:rtfconversion

  • oku_set02:xsltransformation

  • oku_set02:twbanalysis

  • oku_set02:rtfsplitting

  • oku_set02:alignment

  • oku_set02:rtftotmx

  • oku_set03:alignment

  • oku_set04:dnllistedit

  • oku_set04:xliffsplitting

  • oku_set04:xliffconcatenation

  • oku_set04:tmxsplittingdup

  • oku_set04:xliffconversion

  • oku_set04:proofreading

  • oku_set04:searchandreplace

  • oku_set04:uriconversion

  • oku_set04:cdataconversion

  • oku_set04:xmlpruning

  • oku_set04:ttx2tmx

  • oku_set04:xmlpartitioning


All the filters and utilities etc require specific parameters to run. Check the Okapi for Mono manual that comes with the download to know how to use all that.

Don't forget that Okapi has a very nice user group managed by Yves Savourel, the developer. It is possible to access the archives without having to subscribe to it.

Et voilà ! That is pretty much everything you need to know to get started... I hope that helped !

XLIFF 1.2, TMX 2.0, etc.

For some reason, my Google Alert for XLIFF decided to inform me today of the existence of the XLIFF 1.2 specification on the OASIS site...

Since we're at it, the other relevant standard body, LISA, is also working very hard, to produce the next version of TMX: TMX 2.0. With Heartsome's Rodolfo Raya as the standard editor, we can be sure that HTS will be one of the first application suite to support the standard.

Meanwhile, the Internationalization Activity of the World Wide Web Consortium has not ceased to produce very interesting documents, like an Updated Working Draft about the Best Practices for XML Internationalization.

The W3C's i18n group is also working on the Internationalization Tag Set. Yves Savourel of Okapi Framework fame being the chair of that Working Group, you've already guessed that the framework already supports a part of this tag set...

Transmug !

Yves, apologies !

Jost Zetzsche of The Tool Kit, the newsletter to read, even if Mac news are scarce, just reminded me of the existence of Transmug, your group of Mac using translators...

I found the Preaching to a Choir post and liked the PDF (10 mb) presentation attached to it.

ps: Amazing the number of Yves related to Mac and translation. Yves Averous of Transmug, Yves Savourel of the Okapi Framework, Yves Champollion of WordFast...

Okapi tools for Mono !

The Okapi framework is a set of applications designed to ease the work of the translator:

"The Okapi Framework is a set of interface specifications, format definitions, components and applications that provides an environment to build interoperable tools for the different steps of the translation and localization process.

The goal of the Okapi Framework is to allow tools developers and localizers to build new localization processes or enhance existing ones to best meet their needs, while preserving a level of compatibility and interoperability. It also provides them with a way to share (and re-use) components across different solutions. The project uses and promotes open standards, where they exist. For the aspects where open standards are not defined yet, the framework offers its own. The ultimate goal is to adopt the industry standards when they are defined and useable.

In short, the Okapi Framework aims at being a crucible where we forge common components that can be used in any localization and translation application, providing faster development time and better interoperability, but still allowing for the diversity of solutions."


(quote from http://okapi.sourceforge.net/)

Problem is, Okapi is developed on the .NET platform, basically a Windows only platform.

A few years ago, people on the Linux side have decided that .NET was a valuable platform and decided to create an implementation of .NET for Linux, that could run .NET applications out of the box. Mono was born.

Mono was also made to run on OSX... The problem was that until recently Mono's support for .NET was not sufficient to run the Okapi tools and that the Okapi tools had not been written with the lowest common denominator in mind to run on Mono.

Yesterday, Yves Savourel, lead developer of the Okapi Framework Project, released a first Okapi for Mono package for testing on existing Mono environments (including OSX and Linux). The totality of the tools is not yet available but command line tools are said to work.

As far as OSX workflows are concerned, Okapi can produce XLIFF files (or OmegaT projects, or XLIFF files for OmegaT) from a number of localization/translation formats. It is now relatively trivial for OSX translators to deal with inDesign files, for example, as long as they are saved in the inDesign XML format (.inx). Okapi will convert the .inx files to XLIFF for translation in OmegaT and will convert the target files back to .inx for delivery....

Very good news for translators on OSX and warm thanks to the Okapi team !

ps: I'll post a detailed description of how to install Mono and Okapi on OSX in a few days for the readers who don't feel too adventurous. Meanwhile, here are the respective download pages:

Popular, if not outdated, posts...

.docx .NET .pptx .sdf .xlsx AASync accented letters Accessibility Accessibility Inspector Alan Kay alignment Apple AppleScript ApplescriptObjC AppleTrans applications Aquamacs Arabic archive Automator backup bash BBEdit Better Call Saul bug Butler C Calculator Calendar Chinese Cocoa Command line CSV CSVConverter database defaults Devon Dictionary DITA DocBook Dock Doxygen EDICT Emacs emacs lisp ergonomics Excel external disk file formats file system File2XLIFF4j Finder Fink Font français Free software FSF Fun Get A Mac git GNU GPL Guido Van Rossum Heartsome Homebrew HTML IceCat Illustrator InDesign input system ITS iWork Japanese Java Java Properties Viewer Java Web Start json keybindings keyboard Keynote killall launchd LISA lisp locale4j localisation MacPorts Mail markdown MARTIF to TBX Converter Maxprograms Mono MS Office NeoOffice Numbers OASIS Ocelot ODF Okapi OLPC OLT OmegaT OnMyCommand oo2po OOXML Open Solaris OpenDocument OpenOffice.org OpenWordFast org-mode OSX Pages PDF PDFPen PlainCalc PO Preview programming python QA Quick Look QuickSilver QuickTime Player Rainbow RAM reggy regular expressions review rsync RTFCleaner Safari Santa Claus scanner Script Debugger Script Editor scripting scripting additions sdf2txt security Services shell shortcuts Skim sleep Smultron Snow Leopard Spaces Spanish spellchecking Spotlight SRX standards StarOffice Stingray Study SubEthaEdit Swordfish System Events System Preferences Tags TBX TBXMaker Terminal text editing TextEdit TextMate TextWrangler The Tool Kit Time Capsule Time Machine tmutil TMX TMX Editor TMXValidator transifex Translate Toolkit translation Transmug troubleshooting TS TTX TXML UI Browser UI scripting Unix VBA vi Virtaal VirtualBox VLC W3C web WebKit WHATWG Windows Wine Word WordFast wordpress writing Xcode XLIFF xml XO xslt YAML ZFS Zip Zotero