IceCat on High Sierra

[Catalina update: the build instructions for IceCat 52.3 do not work anymore with Catalina. There is a weird error that says "C++ compiler cannot create executables." I am investigating this and I'll update the post when I have found a solution.]

The other day I received a general mail from the Free Software Foundation informing me that even if Safari on High Sierra had pretty good privacy and security features, IceCat, the GNU version of Firefox, was even better.

I was interested and checked the provided links to see what that browser was. I'm not a super fan of Mozilla stuff, but if it's GNU, then why not.

The problem is that the links did not point at any binary or downloadable package. The only thing I could find was the FTP server, with builds for Linux.

Update: It looks like there are builds of IceCat for the Mac, up to version 38.8.0 (released in May '16) on that page.

I wrote to the FSF back to tell them that if they wanted to promote better solutions for macOS users, it would be more effective if the solutions were actually available, either as binaries, or as instructions to build the software.

After checking a bit further, I found this page with instructions to build an 11 months old version on El Capitan: A way to build IceCat 45.5.1 on MacOS El Capitan.

I decided to try with the latest version of IceCat (52.3) on my High Sierra machine, and here are the amended instructions.

If you do not want to go through the build process, I have uploaded the resulting icecat-52.3.0.en-US.mac64.dmg package here, but without ANY warranty about the contents and security, and privacy and everything that you REALLY should care about when you install new software on your machine...

Instructions:

1. Prepare the development environment:
1. install Xcode and Homebrew
2. install yasm with Homebrew
3. install autoconf@2.13 with Homebrew
4. if you miss dependencies, the configure step below will tell you and you just need to get them from Homebrew
2. Download IceCat Source & configure the compilation
1. wget https://ftp.gnu.org/gnu/gnuzilla/52.3.0/icecat-52.3.0-gnu1.tar.bz2
2. tar xjvf icecat-52.3.0-gnu1.tar.bz2
3. cd icecat-52.3.0
4. mkdir objdir
5. mkdir -p ./browser/branding/unofficial/moz.build
6. cd objdir
7. ../configure --with-l10n-base=../l10n --enable-official-branding
3. That's it for the preparations. On a side note, the other configure options found in the original instructions were all refused. If configure does not protest, you're ready. If it does, you either miss dependencies (see step 1.4. above), or you have other issues on your machine. Check the net for error messages, fix the errors, and you should be fine. Now comes compiling time, and it's going to take quite some time, so get a book ready.
1. make
2. make package
4. When make package quits without complaining, you're ready. IceCat is in the dist folder. There is one that works out of the box, and one that comes as a .dmg (icecat-52.3.0.en-US.mac64.dmg on my machine). Either seems to work fine.

During the make process, you may run into an error that says something like:

...(stuff) ... CLOBBER dependency dropped ... (more stuff) ...

and the script proposes you: 

To ignore this message, touch "/Users/suzume/Documents/Code/tmp/icecat-52.3.0/configure"

Well just do as suggested:

touch "/Users/suzume/Documents/Code/tmp/icecat-52.3.0/configure"

and relaunch the build process with make. That worked for me.

Ok, make and make package have worked, you have an IceCat build and a .dmg package into /dist, and you want to start and try the thing. Just launch it and enjoy the web, à la GNU.

Now, you may run into a problem when you want to go to a site. The first time you use IceCat, every time you want to go to a site, a error page gets displayed with the message "Content Encoding Error" (etc.)

The fix is really easy: quit IceCat and relaunch it. That's it.

Et voilà! You can now enjoy a (slightly) more secure and privacy aware GNU browser on your Mac! Have fun!

Transcription software for free?

Fast forward to 2023...

Most of what I wrote below is correct. But unless you have lots of time on your hands (or work on very difficult recordings), I would strongly suggest that you use Whisper, the OpenAI solution to automatic transcription. It requires basic command line skills, but nothing that you can’t learn in a 10-minute video tutorial. Your machine specs will also make a difference: my 2.8 GHz Quad-Core Intel Core i7 2015 MBP with 16gb of RAM requires about 1.5~2x the length of the soundtrack to produce a very good transcription (of Japanese interviews) that I will later have to edit to produce something useable.

The github page is here: openai/whisper, and includes plenty of links that you might want to read.

---

About 4 years ago I got a job where I needed to transcribe about 40 hours of interviews. I wrote most of this article then and let it to rest until now. The solution I propose is a workable solution for transcribing audio/video and can also can be used as a practical introduction to Applescript. I just tested everything in High Sierra, with the current versions of all the mentioned applications.

This article also demonstrates how a few macOS technologies can be put together to create a very robust and integrated solution in a number of very easy steps. The idea is:

1. Find a process that you need to automate
2. Use Applescript to code the automation
3. Use Automator to create a system-wide service to access the automation
4. Use System Preferences to assign a shortcut to the service, either available system-wide, or only in a given application

Step 3. and 4. are just one way to access the automation. There are many other ways.

Update (the day after...): a comment on reddit says that the title is not accurate because I do not propose speech recognition. For people not familiar with transcription, plain speech recognition is not a solution because it requires two things: good sound quality and that the software be used to the voice. That is not the case with most transcription situations. But, it is possible to dictate the audio that you are listening to, in which case you'll need the same tools as described here, and you just have to add macOS dictation to the workflow if you want to stick to macOS bundled software.

Homemade transcription software...

Transcribing 40 hours of interviews is a lot of time in front of the machine and even though the only thing you need is a text editor and an audio file player the lack of default integration between the two can make you waste a lot of time on tedious manipulations.

The problem is going from the text editor to the audio player each time you need to pause the stream, step backward, or start the stream again. Then you need to get the Time Code, write it at the beginning of the line and start transcribing again.

There seems to be plenty of professional solutions for this where you can start/pause/rewind the audio player with foot pedals directly from the text editor.

However, if you’re used to your keyboard, doing that directly from the text editor of your choice using shortcuts and without ever leaving the editor would also be a totally satisfying solution.

It is possible to use the media buttons available on your keyboard (except for inserting the Time Code). You can work in the editor and pause/resume and do a few other things directly with the keys that Apple has provided. Although, There are a few problems with that.

The first is that you may have set your keyboard to ignore the Apple supplied keys and directly access the Function keys instead. If you’ve done that, you’ll have to hit the fn key and the corresponding Function key to access the media function you need.

Another problem is that the media keys are not conveniently located on the keyboard. Accessing them often will require that you move your hand from your basic typing position and this will slow you down.

The last and most important problem is that you can’t easily and precisely rewind the file to go back to a moment you did not properly hear…

The media buttons are really made to easily play media, and not to do transcription, so we need a different solution.

One solution would be to have direct access to the appropriate functions of the media player while it runs in the background through shortcut keys while the text editor remains in the foreground.

Here is what I came up with, using the following bundled software/solutions:

• TextEdit
• QuickTime Player
• Applescript / Script Editor
• Automator
• System Preferences

TextEdit will be used to type the text. It comes with everything you need to write plain text and you can also use it as a minimalist word processor. You can use anything else as a replacement. The only condition is that the software you use supports macOS Services. Unless you use a very exotic text editor or a virtual environment, that should not be an issue.

QuickTime Player supports a number of audio file formats and its functions can be accessed through Applescript. In the second part of this article I'll use VNC instead because it is more convenient.

Applescript is the Apple solution to link all the applications together. Script Editor is the Apple bundled editor where you can code in Applescript.

Automator is how we will create the “Services” to send commands to QuickTime Player while working in TextEdit.

System Preferences will allow us to assign a shortcut to access the services we just created.

Quicktime Player and Applescript

First of all, we need to play a document in QuickTime Player. Instead of pressing the Play button every time we want to start QuickTime Player, we're going to automate that with AppleScript.

1) Open Script Editor

2) Select File > Open Dictionary, then select QuickTime Player from the list of applications that is displayed

What you see now is the list of QuickTime Player functions that you can access from Applescript. You’ll notice a “QuickTime Player Suite” where player specific functions are documented. There, you can see that the “play” command requires a “document” to run.

3) Select File > New and type the following command:

tell application "QuickTime Player" to play front document

Now, open an audio file with QuickTime Player (do not start playing it yet), go back to your script and click the “Run” button in the Script Editor window (the grey arrow pointing right in the toolbar). You should now see the QuickTime Player start playing your file.

Pretty cool right?

If we just replaced clicking on "Play" in QuickTime Player by clicking on "Run" in Script Editor, we would not have advanced much. We now need to package all that in a way that removes the need to click on a button.

Automator

4) Open Automator and select File > New. Select “Service” and in the search field on the left, above the list of possible actions, enter “Applescript”. You should end up with the command “Run AppleScript”.

5) Drag that command to the right side of the Automator panel. You can now see a “(* Your script goes here *)” where you paste what you just typed in the AppleScript Editor (“tell application "QuickTime Player" to play front document”).

6) Now, there are a few things you need to tweak to make that run properly. Above the command block you just created there is a “Service receives selected [text]” drop menu. Click on [text] and go down the list to select [no input]. Indeed, your code does not need any input to run, so [no input] is the right choice. Leave the “in [any application]” as is since you want to be able to use that service from any application that supports services, not only from TextEdit.

We’re almost done. Once you save the service (give it a name like “QuickTime Play” or anything that makes sense to you), it should appear in the “Services” menu that’s available from the Application menu next to the Apple menu (top left of the screen).

So, the service is available, you can work in your text editor and select it from the menu and it will play the front QuickTime Player document, but just like you want to avoid having to click around, you want to access that service by using a keyboard shortcut so that you don’t have to leave the keyboard while typing.

System Preferences and Services shortcuts

7) At the bottom of the Services menu you’ll find a “Services Preferences” item. Select it and you’ll find yourself in the Keyboard Shortcut section of the System Preferences. Down at the bottom of the list, in the “General” section, you’ll find your newly created service with an “add shortcut” button on the right. Click that button and enter the shortcut you want, but be careful not to use something that’s already used in your text editor. The best way to check that is to open a file in your text editor and to try the shortcut you’re thinking about in various contexts (on selected text, between letters, etc.) If nothing happens, it means you can assign it to the service.

8) Enter TextEdit or the editor you're working with and hit the shortcut. Your audio file should be playing in the background, and you have not left the editor.

After "Play": "Pause" and "Rewind"

We have created a “Play” service. Now we need to have a “Pause” service and a “Rewind” one. The only difference is the AppleScript command that we’ll use.

If the “Play” command was:

tell application "QuickTime Player" to play front document

the “Pause” command will be:

tell application "QuickTime Player" to pause front document

and the “Rewind” command (let’s say 5 seconds backward) will be:

tell application "QuickTime Player" to step backward front document by 20

The “20” is 20 “steps” and after testing a bit it seems that one step is 1/4 of a second, so 5 seconds will be 20 steps.

We seem to be almost done, but there are 2 problems with the above commands.

1) The Rewind command also pauses the file. But if you want to step backward to re-listen a part you did not hear clearly, you want to have the command resume play right after you’ve rewound the file. The solution is, well, to ask QuickTime Player to resume playing after rewinding the file... The new “Rewind” command would look like this:

tell application "QuickTime Player"

step backward front document by 20

play front document

end tell

2) You’ll notice that there is a small time difference between the time you pause and the time you resume the play. Depending of when you stop and resume the playing, the time lag seems to vary between a few hundredth and a few tenths of second. It seems that’s the way QuickTime works and that’s a problem if you need to resume playing when somebody is talking: a 0.2 to 0.3 second gap is enough to miss a sound and misunderstand a word. Now that we know that “step backward” pauses the file, we can use it to pause the file at a satisfying time position, like 2 step before QuickTime Player actually paused (that should be enough). The new “Pause” command would now loo like this:

tell application "QuickTime Player" to step backward front document by 2

We already have a Service and a shortcut to access “Play”, so we just need to follow the same procedure to create the “Pause” and the “Rewind” services.

Now we can start working on our files…

Time codes...

But what about inserting time codes in your document ?

Here is the code from which you’ll be able to create your own service:

tell application "QuickTime Player" to set QTPTIME to (current time of front document)

set MIN to (QTPTIME div 60 as integer)

set SEC to (QTPTIME - (MIN * 60) as integer)

if MIN < 10 then set MIN to 0 & MIN

if SEC < 10 then set SEC to 0 & SEC

set TC to (("TC: " & MIN & ":" & SEC) as text)

set the clipboard to (TC as text)

“current time” is the current time of the document being played. The value is given in seconds. So to  convert that into a time code we need some basic arithmetics, which we find in line 2 and 3.

The results are expressed as “normal” numbers and so, for numbers smaller than 10 we’ll need to add a “0” to the number so that the time code has always 2 digits, like “01:01” for “1 minute 1 second”. That is in line 4 and 5.

Then we need to create the time code strings, on line 6. The concatenation command is “&” as we saw in line 4 and 5.

The resulting string that is put into the clipboard would be something like:

TC: 01:01

The last line stores the time code into the OS clipboard so that you can paste it wherever (and whenever) you want in your document. You can use the standard Command+V shortcut to do so.

Now, if you’ve done everything right, you should have something like this in your Services menu:



Et voilà !

An strong alternative to QuickTime Player: VLC

There is an alternative to QuickTime Player that has none of the problems we just described. The software is VLC. It is Free Software and is available directly from the development site. You can make donations to contribute to the development too.

VLC can play a lot of media formats. If you open VLC’s AppleScript dictionary (step 2. above) you’ll see that the “play” command works on the current playlist item and also pauses the stream when it is running. Trying VLC, you can see that when it pauses, it resumes from the same position, so that you don’t have to think of a workaround like we did for the above “Pause” service. Also, the “step backward” command does not stop the stream. If the stream is playing then “step backward” just steps backward and proceeds with playing the file. If the file is paused, then stepping back will keep it paused.

Thanks to VLC’s behaviour we can reduce our 3 above scripts to 2. One would be “play/pause”, the other would be “Step backward”.

The code looks like that:

Play/Pause:

tell application "VLC" to play

Step Backward:

tell application "VLC" to step backward of 2

(the default is 10 seconds, the AppleScript dictionary gives 4 possible values which are, after testing “of 1” for 3 seconds, “of 2” for 10 seconds, “of 3 for 1 minute” and “of 4” for 5 minutes.

VLC makes it simpler to code the solution, but if you want to work only with macOS bundled software, it is also possible to work around QuickTime Player issues, as we've seen above.

There are a lot of areas where the above code can be improved, but the solutions we have work well enough and can be the basis for a lot of other relatively simple developments.


Update (11/9):

I've changed the code a bit after testing on a real job (in VLC).

First, the time code:

tell application "VLC" to set VLCTIME to (current time as integer)

set MIN to (VLCTIME div 60 as integer)

set SEC to (VLCTIME - (MIN * 60) as integer)

if MIN < 10 then set MIN to 0 & MIN

if SEC < 10 then set SEC to 0 & SEC

set TC to (("

" & "TC: " & MIN & ":" & SEC & "

") as text)

set TCstring to (TC as text)

return TCstring

What I changed here is the TC string: I added a line break at the beginning and at the end of the TC string, so that I don't have to insert them myself. That way, I can insert the time code at the end of any line I've just finished and I get

• a line break → move to the next line
• a time code → insert the time code
• another line break → move to the next line

so that I can start to type right away.

I have changed the way the Automator service works too. Instead of feeding the clipboard, and pasting the time code myself, I ask the code to return the string (return TCstring) and I checked the [Output replaces selected text] box at the top of the Automator actions list.

That way the returned value is automatically pasted where I have the cursor.

Another modification, minor this one, in the Step Backward function: I changed the delay from 2 to 1, which is in fact way enough when you just need to clarify a sound. A delay of 2 requires that you wait too long before you can resume typing.

tell application "VLC" to step backward of 1

Now, you must really be careful about the shortcuts so that the don't interfere with normal navigation in the text.

I chose the following:

Control + ]  for "VLC Play+Pause"
Control + [  for "VLC Rewind"
Control + ↓  for "VLC Time Code"

I've just finished transcribing a short 6 minutes interview with this setting and everything worked like a charm.

What is "System Events"?

This kind of open-ended question rarely gets a satisfying answer right away, and this one was no exception with a: "check the dictionary".

If you've done a bit of amateurish applescripting, including directly copy-paste-run code that's been given to you directly or not on the various help fora, as I seem to have done for the best of the last 10 years, you are bound to have encountered things like:

tell application "System Events" to tell application process "TextEdit" to set frontmost to true

or things that involve keystrokes:

tell application "System Events" to tell application process "Terminal"

keystroke "n" using {command down}

keystroke "f" using {command down, control down}

end tell

And if you've tried some more involved and also desperate GUI scripting too, you've probably seen code that accesses menus and menu items and clicks on them, all that within a "System Events" block.

tell application "System Events"

tell process "Microsoft PowerPoint"

tell menu bar 1

tell menu bar item 8

tell menu 1

click menu item 5

end tell

end tell

end tell

end tell

end tell

This one comes from one of my test scripts, but in all honesty I can't remember what it's doing by just taking a look at it (always put comments in your code)...

Ok, so you know you can do black magic with that "application" but you don't know the extent of the magic. As I'm willing to know more about what I'm actually doing with Applescript, I decided to bite the bullet and ask the question on both the official AS user list that Apple seems to be ready to dump any time now, and the new, user based AS list hosted on groups.io:

"Is there a place that describes what System Events is, how it works, why it is necessary for some sorts of scripting, etc.?"

I was not disapointed when the first answer I got turned out to be roughly "check the dictionary, and don't crosspost because it's annoying".

As Matt Neuburg states in his wonderful and always relevant "Applescript: The definite Guide" (O'Reilly, 2003) ", p344:

"A dictionary is merely a list of words. Words don't make a language. The problem [...] is how to do things with words. [...] a vocabulary list is not documentation."

First, as Neuburg wrote, the dictionary is specifically not the place where you're supposed to find the answer to such questions. It's kind of like replying "RTFM" when there is only a technical specification and no manual around. It's a bit silly.

Second, with the current massive and ongoing corporate takeover of the internet, if you have energy to be annoyed about a message that's relevant to the 2 lists it is sent to, you probably just had a bad day, and you should take a deep breath before hitting the Send button, or even before thinking of replying at all (especially if the relevant part of the reply is, well, irrelevant). Here. I said it!

Back to the topic.

The straightforward part of Applescript is that you know it is here to help you automate your work with the applications that run on your Mac (or at least most of them). The (sometimes "very much") less straightforward part of Applescript is that each supported application has its own dialect and to fully make use of it, you need to know what it is capable of.

I've used Finder for 22 years now, I've used Calendar for about a decade, so I think I'm relatively familiar with what they're doing, and what I want them to do. And that's where their dictionary can help, because I have an idea where I want to go, and that's also the problem with "System Events" because I've never used it as an application and I have thus no idea what I can do with it and what to look for in the dictionary.

GUI Scripting ?

Matt Neuburg is a genius, and Bill Cheeseman is his prophet. Or maybe it's the otherway around. Bill Cheeseman, as too few people know, is the man behind Pfiddlesoft, the place where UI Browser is developped. And UI Browser is the application that you'll need if you want to do any serious UI scripting.

Apple also provides a tool to explore the user interface of applications, as of 10.13 it is called Accessibility Inspector and is located somewhere within Xcode, but comparing it to UI Browser is like comparing Apple's Script Editor to Late Night Software's Script Debugger: the bundled application only allows you to go so far.

Bill Cheeseman literally wrote the book about GUI scripting (literally: Bill Cheeseman and  Sal Soghoian, "AppleScript 1-2-3", Peachpit Press, 2009, chapter 28, which is in fact "Lesson 28": "GUI Scripting"), and in the reply thread was nice enough to refer me to his site (which is a little old fashionned, to say the truth) where all sorts of GUI scripting information can be found.

After a lot of looking around I finally found a link to a marvelous article by Matt Neuburg published in the March 10 2003 issue of TidBits. In 2003, OSX (oops, macOS) was still very young and Apple was just beginning to set rules regarding what kind of access to the operating system it allowed applications to have so as to ensure that the system provided the most stability and security to its users.

Because of some US legislation regarding government contracts back then, Apple had worked on an accessibility layer to all the applications that used the Mac UI toolset. The layer allows all those applications to be accessed through the menus and windows and editable areas. And that layer itself is accessed by one invisible but scriptable application: "System Events" (and specifically its "Processes Suite"), and it is this application that Applescript gives orders to when you do GUI scripting.

As the above code snippets suggest, most of the people who use System Events probably do so because of the accessibility API that allows for GUI scripting. But System Events is not only that.

Scripting additions were very fashionable at the time. Scripting additions directly add(ed) functionality to Applescript itself without requiring the launch of an application. As Shane Stanley (the man behind pretty much anything you can read about ApplescriptObj-C)  added in the reply thread to my question, Apple at the time was discouraging the creation and use of scripting additions (see also Chapter 21 "Scripting Additions" of Neuburg's book for details) and thus pushed some of its own into a different corner of the operating system: faceless background applications under the System Events' banner.

That's the reason why today, a quick glance at System Events' dictionary shows such a considerable diversity of objects: as of macOS 10.13, there are 22 "suites" (GUI scripting being only about the "Processes Suite") offering 31 commands, 104 classes, 16 enumerations, 1 record and 12 types, in areas that go from XML to security, Quicktime, movies, power settings, etc.

Shane is though quick to add that if the "Swiss army knife analogy" holds, it is not always for the best:

"But if you've ever used some of the tools on a Swiss Army Knife, you'll know they're often not nearly as good as a proper screw driver or whatever. It has quirks and bugs, and things like the file suite -- which should be a replacement for a lot of Finder scripting -- has had plenty of bugs over the years. And unfortunately some of the functions, like the XML and property list stuff, don't scale well -- do anything remotely complex and they're very slow (more due to AS itself rather than anything peculiar to System Events)."

So I guess here is the answer I was looking for. "System Events" is eventually just a faceless background application that is scriptable, covers a number of areas not covered by the current Apple provided Scripting Additions, and of which the Processes Suite has been the most documented and used thanks to GUI Scripting. This mixed bag of Suites is not really up to the task if you want to do heavy duty work though. For that, I guess you're better off using dedicated solutions (specialized scripting additions or handler libraries, allowed since 10.10), or, as Shane would suggest if asked, dive into Applescript Objective-C that offers direct access to any Cocoa framework, also available since 10.10...

Et voilà! With that knowledge in hand, I'm ready to do some more diverse System Events scripting, and if I get bitten by a bug, that will give me good reason to really dive into all the scripting additions I have collected over the years or even ApplescriptObjC...

Back to scripting for now, and to Neuburg's book!

XLIFF Editors...

[Updated on March 21 2021 to include info on the DGT OmegaT plugin, and the new Swordfish version]

There are plenty of "XLIFF Editors" that pop up on the Mac App market. A number of them are free, most are not super expensive. But all confuse "XLIFF" with the poor subset that Xcode outputs. And that confuses the people who need XLIFF editors the most: professional translators.

XLIFF is an industry standard used in all the localization/translation world. Xcode developers who need the ability to edit their output have all the rights to create quick tools that will help them with that task. But please, don't call that XLIFF editors. What you do is Xcode l10n files editors, basically just XML simple parsers outputing the resulting data in a 2 column table with a native GUI. That's pretty much it. And that's very fine. But it's not XLIFF.

If any of those developers had actually worked with a professional translator to see what are the features required to work with XLIFF (and all the other related standards: TMX, SRX, TBX, ITS, etc.) they would never call their tool an XLIFF editor, just like TextEdit is able to edit XML but nobody would think of calling it an XML Editor...

Mac developers are very picky when it comes to what looks good. Good for them. But would they rather develop in Xcode or in TextEdit? Professional translators on the Mac who need to work with XLIFF currently have the following not so good looking but rock solid choices (all Java based, by the way):

• FOSS, very active, used by professionals all over the world:
  OmegaT and the Okapi Framework filter plugin (GLP/LGPL)
• The DGT-OmegaT site also offers a high quality XLIFF plugin for OmegaT:
  StaX Filters (GPL3), documented here.
• FOSS, active, XLIFF and ITS dedicated editor:
  Ocelot, by the Okapi Framework
• Proprietary, but personal use of the source code is free, by Maxprograms:
  Swordfish, and the code is hosted on Github: /rmraya/Swordfish
  Maxprograms has moved from Java to Node.js, but that matters only if you use Swordfish from the source code.
• FOSS, not active anymore, used to be used by professionals all over the world:
  Heartsome's Translation Studio (GPL)

XLIFF is a serious standard, and translators need rock solid standard support to work. If your editor does not have inline tag/segmentation/legacy translation support, call it anything but XLIFF Editor, please.

Also, this is not a rant. This is a reminder: there is a market for robust native pro-level translation tools on Mac. With Microsoft Office for Mac feature for feature equivalent to the Windows version, translators and localizers have little need to stay on Windows machines. Except that the biggest pro-apps are Windows only. And that's a shame.

Using XLIFF as your point of entry into the l10n world is a good and relatively easy way to access a large market (pro conversion engines to and from XLIFF already exist: see the Okapi Framework). But the point where you can compete with the incumbent and actually make money is way higher than what you think.

Evolution...

This is a follow-up to the "Applescript everywhere" article that I published yesterday.

As I mentioned in the update at the end of the article, I had written on this blog, back in November 2007, "I have never found a practical way to use AppleScript in my workflows..."

That same year, 8 months earlier, I was talking about OmegaT in Tokyo. It was in one of the first editions of the "Open Source Conference", a local (Japanese) conference that moves around the country and takes place in Tokyo twice a year: "翻訳者を支えるオープンソースの翻訳メモリソフト". At the time, OmegaT did not have most of the features that make it now a professional CAT. I wanted to promote it in FOSS circles here because I thought they would be the best places to find potential developers. The strategy did not work, but that gave me a chance to meet amazing people from Hokkaido to Okinawa that I am still in touch with today.

2 years later, invited by one such person, I wrote a short presentation of OmegaT where I mentioned Applescript, it was published in issue 45 of the Journal of the "Asia-Pacific Association for Machine Translation". I put a copy here: "自由に翻訳！". At the time I was exploring ways to extend OmegaT with easy to write scripts because developing new features in Java was something I never felt I'd be able to do (I never was, and that's the main reason why I eventually left the project, last July).

The script I quote in the article is:

on run { }
  do shell script
  "sed \"s/<[^>]*>/ /g\" ~/Library/Preferences/OmegaT/script/source.txt | /usr/local/bin/pbcopy"
end run

Nothing very Applescripty there, I'm only calling a one line shell script that removes OmegaT tags from the source segment and pasting the result into the pasteboard. This one must have been using Automator, released 4 years earlier. It was a 6 layered "machine": Key shortcut → Automator service → Applescript handler → shell script → sed process → Cmd-P.

At the time, OmegaT had no "API" or any sort of scriptability to speak of. But it supported the OSX services because the Java UI framework it used (and still uses) allowed for that (unlike other Java applications that use other frameworks less "connected", if connected at all, to the OS). Marc Prior, then project manager, invested some money to have a "source, target and selection text export" feature. The feature still exists. There are 2 files that are automatically created in the OmegaT preference folder (source.txt and target.txt) and with a shortcut (Shift+Cmd+C) you can export the contents of the selection to selection.txt in the same location.

And OmegaT did not have any sort of tag management either. What I did as a workaround (obsolete now, thanks to new developments) was create scripts that would parse the source.txt when entering a new segment that I'd use as Automator services called by a shortcut. The script would basically call a "do shell script" as above to extract the tags from the source.txt file (opposite to the above) and output them so that I could paste them in target. The first version was relatively trivial (get all the tags), then I had a more complex one where the service would call a dialog where I'd enter the tag number and the service would paste that around the selection, inside the OmegaT target segment.

It was all relatively smooth and fun to develop, except that Automator was quite slow if I remember well. Now that I think about it, I'm pretty sure that it the starting point of my Applescript coding. Considering the fact that the article was published in March 2009 I must have started developing  that around the end of 2008.

From what I can see in my job archives, I was still using an ad-hoc folder hierarchy for my jobs in November 2008 while in January 2009 I was using a more structured version that I had scripted. The script was something relatively simple that took a start date, and end date, combined the 2 with the client name code and an "→" in between and created a folder with that as a name, and then a sub-hierarchy of folders where I'd put the client mail in a folder, the attachments to translate in another one, references if any, and 2 extra folders for the delivered files and the invoice. Some computer archeology shows that I was probably still saving the mails manually as RTF...

A few checks in my emails show that indeed, in 2006 I was writing to a friend "I have no idea what Applescript is" but in December 2008 I was sending a mail to the Applescript official list about word tokenization (https://lists.apple.com/archives/applescript-users/2008/Dec/msg00025.html) So, I guess all this shows conclusively that my first attempts at introducing Applescript and automation in general in my workflow date back from sometimes in late 2008.

I was writing yesterday about how I copy-pasted code from the web, but my main source of information, programmatic help and mental support came from the above mentioned and still existing Applescript User List hosted by Apple (https://lists.apple.com/mailman/listinfo/applescript-users). There is extremely little that I could have accomplished without list members' help. Which may confirm that Applescripting is not that trivial, or maybe that I'm just a little bit too thick, or maybe that I am not committed enough. In any case, (almost) 10 years later, I say as I did yesterday, that without the level of automation we have on the Mac, my professional life would not be what it is now.

Applescript everywhere

Artificial languages are no different from natural languages. Either you're fluent and you will have fewer problems doing what you want to do, or you're not, and you're going to spend a lot of time on trivial issues that don't seem to make much sense.

I was in the middle of writing a very optimistic article on how automating your tasks could greatly enhance your workflow and your life in general, and how automation on Mac generally meant "Applescript" when I stumbled on some of those issues when trying to work with Terminal.

I will finish the article because computers are automation systems and using them without trying to automate more than what's provided by bundled programs means letting programs govern the way we work. But that episode painfully reminded me how non-trivial it was for me to work in Applescript, even though I am not exactly a beginner anymore, which does not mean that I know much, just that I know where to look for information when I don't know, and that I can more or less make sense of what I find.

With Applescript, I never read the "manual". I started with online examples, I copy pasted a lot, modified a few scripts, checked the errors, modified again until I got where I wanted, and checked the official reference only rarely (only 333 pages for the PDF I have, but still). And I'm saying to myself, now that I can sort of see how things fall in place, that I should definitely spare some time to go through either the reference or one of the excellent books that are out there.

If Applescript is a language, you're most likely going to use one of its "dialects" too, since any application that supports Applescript actually has a list of words that are specific to it and if the words look the same between applications, they are, like we say in French about some English words "false friends". They look the same but don't always mean the same. Applescript is not a language that you will use to make heavy-duty computations (although you could). Applescript was made to work with "Apple Events", which are used to connect applications together. As you'll eventually read in that other article, one of my script creates a Finder group of folders based on a job request I received in Mail that will end up as a Calendar item.

Applications that don't support Applescript can also be talked to: think about the primitive sign language that we use when we don't know a language. But that can be pretty powerful because just as you can show a person how to click the button that will move a robot, you can also tell Applescript to click on any button or menu item of pretty much any application that runs on a Mac. In fact, some parts of Applescript supporting applications are only available through their user interface, for example, asking a Terminal window to enter full-screen mode requires an Applescript that says "click on such menu item of such menu bar" or even better "hit F while pressing Command and Control" to simulate the shortcut. That's the part that bit me last weekend when I was trying to create a tabbed full-screen window in Terminal opened on the folder that was selected in Finder.

And Applescript can also call the services that are hidden under your Mac's hood: the command line applications that you can usually only access through Terminal, and reciprocally, you can call Applescript from the Terminal. I do that when I select an OmegaT project folder in Finder and use Applescript to tell the command line to use that folder as a parameter to launch OmegaT.

All this is very powerful but can be pretty daunting for amateurs. In my next article, I'll try to show you how I got where I'm at today, which is, in all honesty, not very far ahead of any person who's reading about Applescript today for the first time. I'll publish the Applescript code I'm using, with extensive comments, so that you don't have to just blindingly take for granted what I wrote but get to understand why I wrote it that way. And hopefully that will help you being more productive while working on your Mac.

Update (a few minutes after posting):
I clicked on the "Applescript" tag that's in the column on the right side of the articles, which brought me back to November 2007 when I wrote "I have never found a practical way to use AppleScript in my workflows..." Well, things have changed and now I'm pretty sure I could not live without Applescript on my machine...

Adventures in Kernel Panic Land

Jimmy McGill chose "Saul Goodman" as his pseudo for a new venture to sell his now useless TV commercial slots, because "it's all good, man!"
That's exactly what your Mac tells you every day. Everything works fine, and everything will work out fine tomorrow, until it doesn't, but then, it's a Mac, so you're all good.

You never know that you need a backup until you do. That's the rule. I learned it about 10 years ago, the hard way. Massive hard disk failure, no backup at all, no extra software to check/fix/save the disk. Nothing, but a big deadline 2 days away. At the time I was running an Itanium Powerbook. I had bought it second hand a while before and it was a real workhorse. I also had a greenish clamshell iBook that my wife used to work with, before her company lent her a Windows machine.

When the machine's hard disk died on me, I called the project manager. He roughly told me that I still had 2 days and that the schedule would not move an inch. I don't remember what I did to fix the mess, but I did manage to scrap most of the data from the disk, then I must have moved all that to the iBook, resumed work, delivered, celebrated, and decided to create regular backups.

In the 21st century, external drives are cheap. They are cheaper than whatever hourly rate you apply to the time spent on fixing a non-backed-up dead system. They come in all sorts of flavour, shape, colour and capacity. And if your main machine is a laptop and it's a pain to physically connect it to your backup disk every time you put it back on your desk, you even have Wifi backups.

We tried half-backed solutions for a while. I bought terabytes of external drives, ugly as crap, all Firewiry and USB-Cy, but when you want a transparent solution that just works, all this just doesn't scale. Full disclosure: I have no financial interests in Apple. But damn, Time Capsule rocks. So yes, we got ourselves a Time Capsule, just 2 TB. We had 2 iMacs and 2 notebooks at the time, we thought we'd be short with the capacity but the 3TB Capsule seemed too expensive for just one extra terabyte. But even with the "low" capacity, it's been with us since. It does everything over Wifi, no need to connect the machines, just turn them on and Time Machine does its thing, and in no time you have your incremental backup ready for when you need it. You do need to turn the Time Machine on and select Time Capsule as your backup target, but it's a one time setting, and then you can forget about it, pretty much.

When things work fine, the only time you need a thorough backup is when you upgrade your system. I don't know how many more OS upgrades this "early 2011 Macbook Pro" will be allowed (our 24" iMac could not go further than Yosemite) but I was not going to miss that High Sierra upgrade after seeing the APFS talks on internet. The rest of the icing, I was not very interested in, although I love the way Safari does a lot to keep advertisers from wasting our CPU (and brain) cycles.

What had been bothering me for a while was that the machine was getting slower and slower everywhere and after having an issue with how slow BBEdit was, the Barebones support team confirmed that it was most probably a hard disk issue. With APFS coming I was not going to replace my disk with a rotating thing again. I was done with that. I was going to go the SSD way. All flashy, all speedy, all slick electronics.

My son had done it a few months back. We bought him an MBP when he entered university, one of those new but still coming with the antiquated DVD player. In his department, he was the only freshman to use a Mac. When he came back for Christmas last year, I paid for the replacement Samsung SSD and he spent a few hours doing the thing: backup your data and follow the instructions on Youtube. Easy enough. Then he reboot the machine and everything was faster. And it was before High Sierra and APFS.

I was going to go incremental on this. First install High Sierra, then get an SSD, and last reinstall everything. But before all, the backup. Time Machine being automatic, you really don't have to do much usually. But when you want to do it "now", well, there is a menu item for that. It's called "Back Up Now". To make sure you don't forget to save a few last-minute settings, just quit all the applications you run, leaving only Finder on (although if you know how to quit it, you can too), then let Time machine prepare the backup, do the backup and clean up the backup.

Then, you can start the upgrade process itself. Go to apple.com, click on "Upgrade" and everything goes super smoothly. At one point you are presented with a dialog that asks whether you really agree with all the terms, you just click "Agree" unless you're a hard-core free software advocate, but in this case you're not running a Mac and you're not reading this page, and then files get downloaded and installed and you can go watch your favourite show in the meanwhile, or read a book.

That's what I had planned anyway. Either finish season 3 of "Better call Saul" or read "Consider Phlebas", but it did not happen that way. What happened is that after a long while and not much happening, the Mac complained that it could not be upgraded. Just like that, two thirds into the install process, when the machine had overwritten all those important parts of the disk and you know that if you don't find a solution, you're done.

Rebooting won't do it. The old OS has been blown to pieces by the newer files partially written over it and the machine is programmatically speaking a giant spaghetti mess that the CPU has no idea how to deal with. When that happens, you have a Kernel Panic, and when the kernel panics it is of the utmost importance that you don't panic. Whatever you imagine has happened. All the pics of the newborn, all the thousands of whatever currency you spent on music and movies. Everything is gone. Except that, it's on your backup, and the likeliness that your computer and your backup die on you at the exact same time is so low that it's even lower than the likeliness of you being under the atomic fire of a North Korean missile. Although, granted, the day this happens, you'll have things more important to worry about than a broken backup.

So, you see the Kernel Panic thing that actually looks like a personal reboot of The Matrix on your screen and you feel all warm inside because you know it's a Mac. The idea here is that you need a handy way to get information from the Net. I don't have a phone, but I have that extra 24" iMac that's now mostly used as a Minecraft machine by the 2 little ones, but it also runs Safari (mostly used by the same little ones to check new "mods" for said Minecraft) and so I have a powerful entry point into the internets of information.

What you need to know here (and Safari is your friend if you don't) is that your Mac comes with secret programs that you can launch when you need them because they are located somewhere that doesn't get to be erased by a gigantic mess up of overwritten OS files. One such program is "Recovery Mode". You can launch it by hitting Command+R right after you restart your machine. If your system is down, like mine was, and you don't have a boot disk or USB key, then you want to enter Internet Recovery Mode, which requires an internet connection to download to your Mac all the things necessary to fix your issues (you may want to physically connect to your router/Time Capsule with anything that is faster than Wifi).

"Recovery Mode" be it local or internet based can do four things. The first one is get a Time Machine backup and restore the full machine from there. The second one is reinstall the OS from scratch. The third one is launch Safari to give you access to Apple Support for help. And the fourth one is give you access to Disk Utility so that you can diagnose/fix disk problems.

I figured that since I did not have a separate boot system my best bet was a full recovery from Time Machine. But I tried Disk Utility before that, checked the disk and since it told me everything was normal I just went on and decided to test an OS install. Depending on the key you call Recovery with, the version of the OS that's proposed to you is different. Either you have the OS your machine came installed with when you bought it, or you get the latest version that was installed, or you get the latest compatible version.

I tried High Sierra. And it did not work. I got a number of "cannot access the recovery server" or similar messages and gave up. I rebooted, again in Recovery Mode, tried again with the originally bundled version (Lion ?) and install started. And never stopped. It was late in the night when a dialog announced that the install would really not work.

This kind of moments are when you desperately need to cling to your faith in Macs (slightly backed by actually very few horror stories of messed up machines that refused to recover), and try one last thing before you go to bed. Reboot/Recovery Mode/Reinstall, it was staring to become a routine. But for some reason, this time it worked, and in the morning I had a (not so High) Sierra machine up and running, or rather ready to accept my backup data. A few hours and 250 GB later, I was running my machine, with a fresh Sierra install and all my data in place. I was basically at the exact same point I was the day before, but stronger in my faith that Macs just work and when they don't they still do.

The SSD arrived a few days later. That kind of manipulation involves a slightly higher level of wizardry. You need a few tools and an iFix video. Tools mean screws, and you really don't want to lose the screws. So here we go. Old hard disk removed, screws put in separate boxes to not mix them all, new SSD thing installed, screws back in place, power supply back on, reboot and Internet Recovery Mode launch.

Normally, an all brand new disk is not formatted to accept data copying. It is just a blank slate that the OS must put in shape for it to be of any use. But the OS still recognizes that there is a disk willing to be formatted attached to the machine. Normally.

In my case, the SSD thing was nowhere to be seen. The law of probability was plain about that: the thing itself is not at issue. Maybe an internal connection ? Would I have to open the Mac again and check everything ? I'm lazy and I prefer the Aspirin of Computer Fixing: reboot. A reboot/recovery mode later, I had the thing under my eyes. Go figure. Disk Utility was offering me to format the disk to APFS, which would also erase the data I had spent one full day reinstalling. Go for it, I thought, I'd just spend one more day going through the recovery hoops to get my machine running with the new OS.

Here again, things did not happen as planned. It took a small number of attempts to get High Sierra to be installed on the disk, and then I had to reinstall my data again from the Time Capsule backup.

I was lucky. We had a 3 day weekend here and I had decided to remove the curtains in both our room and the kids' and replace everything with wood or cloth window shades. Plus I had a lot of TV shows to watch, and all the books I brought back from France last summer just waiting to be read. Every once in a while I would check the Mac's screen, just to make sure, and this morning, eventually, I was able to contemplate the new setup, running smoothly, with a few new visible features, but mostly the new boot and launch speed (and damn, that SSD boot is faster than anything I've seen in a long while).

I know that putting a new third party disk into a box that's starting to show its age, adding the new version of the OS when it's just fresh out of master copy and using a new file system that was still in beta not so long ago is the exact opposite of hedging one's bets. And after reading a few cautionary articles after the fact I'm realizing that I may have been a little bold, but whatever happens, I have a backup that's regularly and automatically updated and I can just start from scratch any time I need... So I guess, it's all good, man!