If you've done a bit of amateurish applescripting, including directly copy-paste-run code that's been given to you directly or not on the various help fora, as I seem to have done for the best of the last 10 years, you are bound to have encountered things like:
tell application "System Events" to tell application process "TextEdit" to set frontmost to true
or things that involve keystrokes:
tell application "System Events" to tell application process "Terminal"
keystroke "n" using {command down}
keystroke "f" using {command down, control down}
end tell
tell application "System Events"
tell process "Microsoft PowerPoint"
tell menu bar 1
tell menu bar item 8
tell menu 1
click menu item 5
end tell
end tell
end tell
end tell
end tell
Ok, so you know you can do black magic with that "application" but you don't know the extent of the magic. As I'm willing to know more about what I'm actually doing with Applescript, I decided to bite the bullet and ask the question on both the official AS user list that Apple seems to be ready to dump any time now, and the new, user based AS list hosted on groups.io:
"Is there a place that describes what System Events is, how it works, why it is necessary for some sorts of scripting, etc.?"
I was not disapointed when the first answer I got turned out to be roughly "check the dictionary, and don't crosspost because it's annoying".
As Matt Neuburg states in his wonderful and always relevant "Applescript: The definite Guide" (O'Reilly, 2003) ", p344:
"A dictionary is merely a list of words. Words don't make a language. The problem [...] is how to do things with words. [...] a vocabulary list is not documentation."
First, as Neuburg wrote, the dictionary is specifically not the place where you're supposed to find the answer to such questions. It's kind of like replying "RTFM" when there is only a technical specification and no manual around. It's a bit silly.
Second, with the current massive and ongoing corporate takeover of the internet, if you have energy to be annoyed about a message that's relevant to the 2 lists it is sent to, you probably just had a bad day, and you should take a deep breath before hitting the Send button, or even before thinking of replying at all (especially if the relevant part of the reply is, well, irrelevant). Here. I said it!
Back to the topic.
The straightforward part of Applescript is that you know it is here to help you automate your work with the applications that run on your Mac (or at least most of them). The (sometimes "very much") less straightforward part of Applescript is that each supported application has its own dialect and to fully make use of it, you need to know what it is capable of.
I've used Finder for 22 years now, I've used Calendar for about a decade, so I think I'm relatively familiar with what they're doing, and what I want them to do. And that's where their dictionary can help, because I have an idea where I want to go, and that's also the problem with "System Events" because I've never used it as an application and I have thus no idea what I can do with it and what to look for in the dictionary.
GUI Scripting ?
Matt Neuburg is a genius, and Bill Cheeseman is his prophet. Or maybe it's the otherway around. Bill Cheeseman, as too few people know, is the man behind Pfiddlesoft, the place where UI Browser is developped. And UI Browser is the application that you'll need if you want to do any serious UI scripting.
Apple also provides a tool to explore the user interface of applications, as of 10.13 it is called Accessibility Inspector and is located somewhere within Xcode, but comparing it to UI Browser is like comparing Apple's Script Editor to Late Night Software's Script Debugger: the bundled application only allows you to go so far.
Bill Cheeseman literally wrote the book about GUI scripting (literally: Bill Cheeseman and Sal Soghoian, "AppleScript 1-2-3", Peachpit Press, 2009, chapter 28, which is in fact "Lesson 28": "GUI Scripting"), and in the reply thread was nice enough to refer me to his site (which is a little old fashionned, to say the truth) where all sorts of GUI scripting information can be found.
After a lot of looking around I finally found a link to a marvelous article by Matt Neuburg published in the March 10 2003 issue of TidBits. In 2003, OSX (oops, macOS) was still very young and Apple was just beginning to set rules regarding what kind of access to the operating system it allowed applications to have so as to ensure that the system provided the most stability and security to its users.
Because of some US legislation regarding government contracts back then, Apple had worked on an accessibility layer to all the applications that used the Mac UI toolset. The layer allows all those applications to be accessed through the menus and windows and editable areas. And that layer itself is accessed by one invisible but scriptable application: "System Events" (and specifically its "Processes Suite"), and it is this application that Applescript gives orders to when you do GUI scripting.
As the above code snippets suggest, most of the people who use System Events probably do so because of the accessibility API that allows for GUI scripting. But System Events is not only that.
Scripting additions were very fashionable at the time. Scripting additions directly add(ed) functionality to Applescript itself without requiring the launch of an application. As Shane Stanley (the man behind pretty much anything you can read about ApplescriptObj-C) added in the reply thread to my question, Apple at the time was discouraging the creation and use of scripting additions (see also Chapter 21 "Scripting Additions" of Neuburg's book for details) and thus pushed some of its own into a different corner of the operating system: faceless background applications under the System Events' banner.
That's the reason why today, a quick glance at System Events' dictionary shows such a considerable diversity of objects: as of macOS 10.13, there are 22 "suites" (GUI scripting being only about the "Processes Suite") offering 31 commands, 104 classes, 16 enumerations, 1 record and 12 types, in areas that go from XML to security, Quicktime, movies, power settings, etc.
Shane is though quick to add that if the "Swiss army knife analogy" holds, it is not always for the best:
"But if you've ever used some of the tools on a Swiss Army Knife, you'll know they're often not nearly as good as a proper screw driver or whatever. It has quirks and bugs, and things like the file suite -- which should be a replacement for a lot of Finder scripting -- has had plenty of bugs over the years. And unfortunately some of the functions, like the XML and property list stuff, don't scale well -- do anything remotely complex and they're very slow (more due to AS itself rather than anything peculiar to System Events)."
So I guess here is the answer I was looking for. "System Events" is eventually just a faceless background application that is scriptable, covers a number of areas not covered by the current Apple provided Scripting Additions, and of which the Processes Suite has been the most documented and used thanks to GUI Scripting. This mixed bag of Suites is not really up to the task if you want to do heavy duty work though. For that, I guess you're better off using dedicated solutions (specialized scripting additions or handler libraries, allowed since 10.10), or, as Shane would suggest if asked, dive into Applescript Objective-C that offers direct access to any Cocoa framework, also available since 10.10...
Et voilà! With that knowledge in hand, I'm ready to do some more diverse System Events scripting, and if I get bitten by a bug, that will give me good reason to really dive into all the scripting additions I have collected over the years or even ApplescriptObjC...
Back to scripting for now, and to Neuburg's book!