**PRELIMINARY** This is a document-in-progress. Though we have made every effort to ensure that there are no errors, it is always possible that some errors slipped past us. If any information in this document is unclear, or if you find an error, please contact support@newts.com. This document is not intended as a tutorial, but as a reference. If you have a question about a specific section or concept, please refer to the sample scripts, or email support@newts.com. As time permits, we will be releasing supplemental documents, tutorials, and samples which illustrate different parts of this document. ---------------------------- The anatomy of a GestureScript ---------------------------- v 0.6b, by Dan Rowley last updated 9/7/97 Copyright(C) 1997, Innovative Computer Solutions All rights reserved The electronic posting, quoting, reposting, duplication (electronic or otherwise) or other distribution of this document, in whole or in part, is expressly prohibited without the prior written consent of Innovative Computer Solutions. [ NOTE - this document is not intended as an introduction to NewtonScript programming. It assumes a working knowledge of the NewtonScript programming language, and at least a passing knowledge of the Newton run-time object model. Documentation on programming the Newton in general can be found on line at the Newton, Inc web site - www.newton-inc.com. ] Some suggested references: New developer information - http://www.newton-inc.com/dev/newdevs.html Newton documentation - http://www.newton-inc.com/dev/docs.html NewtonScript reference - ftp://ftp.newton-inc.com/pub/developers/techinfo/npg2.0.pdf.sit.hqx Although it is not required, you may also find that having a copy of the Newton Toolkit can speed the development/debugging process, particularly if you connect the "inspector" to watch for errors in the execution of your script(s). What is a GestureScript? ---------------------- A GestureScript is a snippet of NewtonScript code which executes in response to an event from GestureLaunch (ie; in response to a user Gesture). GestureScript is not a language, per se, it is absolutely "vanilla" NewtonScript. The distinction between writing a standalone NewtonScript function and a GestureScript is really one of context. Starting with version 3.0 of GestureLaunch, a GestureScript executes in a context which provides access to all sorts of information about the current event (ie; the gesture the user has just made), as well as access to useful library functions which simplify certain common tasks. More on this later. Previous versions of GestureLaunch required that all GestureScripts be "context-free," meaning that they could make no assumptions about the world in which they would execute - in a context-free function, "self" evaluates to nil, and the script inherits directly from the globals frame - there was no inheritance chain which would be scanned to evaluate variable and method references, and the script knew nothing about the event which triggered it's execution. Beginning with version 3.0, GestureScripts now execute within a useful context. In particular, this context provides a valid "self" object which contains a number of useful slots, including a proto reference to GestureLaunch itself. There is no parent reference, so you may use this to provide alternative inheritance for your script - this technique is covered in more detail later in this document. Writing a script on the Newton is not quite the same as writing a script in the Newton Toolkit. In particular, you must follow these ground rules: 1) Do not use the CONSTANT keyword. Ever. Although it is valid syntax, it will cause your script to fail in strange and unpredictable ways. Use a regular variable instead. 2) You cannot use the NATIVE keyword to create a native function. This keyword will be ignored, and may cause an error. 3) You cannot use any of the PROTOxxx constants, or any of the other constants defined in the NTK environment. You must use the actual value for the constant instead. You can find these values in the appropriate "defs" file for the platform. "Newton 2.1 defs" is the file of constants you would use on a MessagePad 2000. For PROTO constants, use the appropriate magic pointer (the numbers that start with an '@') as found in the defs file. A handy shortcut for evaluating simple integer constants (like kNotifyAlert) is to evaluate them in an NTK inspector window, instead of looking them up in the defs file. 4) When using GestureScript to create/edit your scripts, do not begin your script with func(), and do not wrap the entire function in a begin/end block. This is assumed, and provided by GestureScript. Feel free, of course, to define func() blocks and begin/end blocks inside your code, as necessary. The Context of a GestureScript ---------------------------- GestureScripts now execute from within the context of a frame which contains a number of useful slots, which are described below. These slots are all found by inheritance, so you can reference them directly by name. Note that if you create local variables with the same names, you'll need to use "self.(slotname)" to get to these slots. _proto - a reference to GestureLaunch itself. There are a number of useful library protos and functions provided by GestureLaunch. These are described in the "Library Functions" section, below. Note that you don't need to actually specify _proto. You can send messages to yourself, and they will filter back to GestureLaunch by inheritance. icon - a reference to the icon on which the user wrote the gesture. This is the actual view on the buttonbar on which the user wrote, not the buttonbar as a whole. Note that if your script is called "out of context" (ie; in response to something other than a user gesture), this slot could be nil. Note also that if the user is using KillBB, the buttonbar may have been opened temporarily, and may be closed by the time your script executes, and you should fail gracefully in this case. stroke - a reference to the WORD INFO FRAME for the recognized letter. The format of this frame is described in the "Word Info Frame" section, below (it is also described in the Newton Programmer's Reference). Note that this slot could contain nil if the system did not recognize a letter/gesture, or in the case of the double-tap and select gestures. config - a reference to the GestureLaunch configuration frame for the icon on which the user wrote the gesture. The format of this frame is not yet documented, and is subject to change. Contact ICS at support@newts.com if you would like the documentation for this frame. Please be sure to include information about why you need the information. script - the actual function object which is executing (your script) scriptclass - a symbol representing the reason your script was called. The possible values are: 'GestureScript - your script was called in response to a particular gesture. Check the word slot to find out which gesture. '|~Default Script| - your script was called because it was configured as the default, and no other script was defined for this gesture. Check the word slot to find out which gesture. '|~BeforeScript| - your script was called because it was configured as a BeforeScript. The BeforeScript is executed before the GestureScript, if any. '|~AfterScript| - your script was called because it was configured as an AfterScript. The AfterScript is executed after the GestureScript, if any, but before the Default Script. nil - your script was called "out of context." In response to no particular event. word - a symbol which represents the recognized gesture. This slot will be nil if there was no recognized gesture. The special symbol 'DblClick is used to represent the Double Click gesture. Note that this slot contains a SYMBOL, not a character or string. If you are executing as a BeforeScript (ie; the scriptclass slot contains the '|~BeforeScript| symbol, you can change the value of this slot to change they way GestureLaunch interprets the gesture. You could, for instance, change '0 (the symbol zero) to 'o (the symbol for the letter 'o'), or implement your own gesture recognition by examining the stroke slot and putting the appropriate gesture symbol in this slot. The "numbersonly" sample BeforeScript uses this technique to transpose commonly misrecognized letters into their similar numbers (ie; the letter 'o' becomes zero, and the letter 'z' becomes the number 2). Valid gestures are: "!", "#", "$", "%", "&", "*", "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "?", "@", "A", "B", "C", "D", "DblClick", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", (A zero with a slash through it), "O", "P", "Q", "R", "S", "T", "U", "V", "W", "X", "Y", "Z", "^", "~", parm - this slot contains the value of the parameter, as defined by the user. The format of this slot will vary based on the type of parameter the script takes: 'Text - parm will be a string, or nil if the parameter was left blank. An empty string could also represent a blank parameter. Use strfilled() to check for empty text parameters. 'List - parm will be an array of strings. 'Package - parm will be a PACKAGE FRAME. See the "Package Frame" section, below, for the format of this frame. This slot could contain nil if the user did not provide a parameter. The Different Kinds of GestureScripts ---------------------------------- When you create your GestureScript, you define the way(s) in which it can be called. The different types of scripts are defined below. Note that a single script can be one or more of these types, and should behave accordingly depending on how it is called. Check the value of self.scriptclass to find out how/why a script was called. BeforeScript - a BeforeScript is executed BEFORE all other scripts for a gesture. If you want to change the interpretation of a stroke, you should do it in a BeforeScript. While inside the BeforeScript, you can change the value of stroke, word, and parm. The return value of the BeforeScript is generally ignored, unless the return value is the symbol 'stop. If you return 'stop from a BeforeScript, then all gesture processing will stop, and no other scripts will be executed for that gesture. GestureScript - a GestureScript is executed after the BeforeScript, if any, and is executed in response to a particular gesture. The return value of a GestureScript is ignored. AfterScript - an AfterScript is executed AFTER the BeforeScript and GestureScript. The return value of an AfterScript is ignored. Default Script - a default script is executed after the afterscript (if any), and only if no GestureScript was executed. The return value of a Default Script is ignored. Library Functions ---------------- GestureLaunch provides a library of useful library functions which can be accessed by sending messages to yourself (they will be found by proto inheritance). If you look at "self" in the NTK inspector you may see other slots in the _proto which interest you. If the slots are not documented here, they are subject to change at any time without notice, and relying on them is **strongly** discouraged. *****CMDKEY :cmdkey(keyparm, modifiers) - simulates the pressing of key(s) on a hardware keyboard. keyparm - a character literal or string which represents the character(s) to type. You may optionally include the words "shift", "ctrl", "opt", or "cmd" in lieu of using the modifiers parameter. Leading and trailing spaces are ignored. modifiers - a combination of zero or more of the following values - kCommandModifier := (1 << 25); kShiftModifier := (1 << 26); kCapsLockModifier := (1 << 27); kOptionModifier := (1 << 28); kControlModifier := (1 << 29); kNoModifiers := 0; *****DOOPEN :doOpen() - open the GestureLaunch config window. *****DO_CONFIGURE_FOR :do_Configure_For(icon,tag) - open the GestureLaunch config window, set up to configure a particular icon icon - a reference to the icon you wish to configure tag - the icon tag for the icon. You can get this value by calling GlaunchGetTag *****FLASH_LETTER :flash_letter(letter,word) - briefly flash a letter and word on-screen. Used internally to provide the "display recognized gesture" functionality. letter - the letter to display in the large font word - the word to display underneath the letter, in a smaller font *****GETPARTENTRYFROMAPPFRAME :GetPartEntryFromAppFrame(appframe) - returns the extras drawer part entry which corresponds to a give package frame. Package frames are returned by package pickers, and are used as the value for package parameters. This function will return nil if the package frame could not be resolved. appframe - the frame you wish to resolve. *****GLAUNCHACTIONFUNC :GLaunchActionFunc(icon, word) - this is the function that gets control immediately after a gesture is recognized. It is responsible for setting up the context frame and calling the BeforeScript, GestureScript, AfterScript, and Default Script. You could call this function to simulate a gesture being recognized on a particular icon. Note that if you call this function, certain values like the stroke field will not be available to any GestureScripts which are called as a result. icon - the icon on which the gesture was written word - a string representing the gesture. The special value "select" represents the selection gesture, and "DblClick" represents a double-tap. *****GLAUNCHGETCONFIG :GlaunchGetConfig(tag) - get the configuration frame for the icon identified by the tag parameter. This tag comes from the GLaunchGetTag function. The format of the returned frame is not yet documented, but it can be passed to other functions that expect a config frame as a parameter. This function returns nil if there is no config frame for a given tag. tag - the tag for the icon for which you wish to retrieve the config frame. *****GLAUNCHGETSCRIPT :GLaunchGetScript(scriptid) - returns the script frame for the script with the given id. scriptid - the script id of the script you wish to retrieve. (All scriptids are symbols) *****GLAUNCHGETSCRIPTFUNC :GLaunchGetScriptFunc(scriptframe) - returns the function object from the given script frame. This will be compiled for you from the script, if necessary. scriptframe - the scriptframe, as returned by GLaunchGetScript. *****GLAUNCHGETTAG :GLaunchGetTag(icon) - returns the icon 'tag' for a given icon. This tag is used by other GestureLaunch functions to identify a given icon. icon - the icon for which you which to have the tag. *****GLAUNCHPASSSCRIPT :GLaunchPassScript(self,scriptid) - passes control to another script, as if it had been called in the first place. self - always pass self as this parameter. scriptid - the id of the script to which you want to pass control. *****GLAUNCHRUNSCRIPT :GLaunchRunScript(scriptid, icon, stroke, parm, word, symbol) - use this function to call another script programmatically. The script will be looked up by id, and passed the parameter you provide. It's up to you to ensure that the parameter is of the correct type. The icon, stroke, and word slots of the script context frame will be filled with values you provide. You may pass nil in these parameters, so long as the script does not depend on their values. In this way, you can simulate a script being called under various conditions. This function will return the return value of the script, if any, or the symbol 'fail if execution failed. scriptid - required. The id of the script you wish to execute icon - optional. A reference to an icon to pass to the script. stroke - optional. A stroke Bundle Frame to pass to the script. parm - optional. The parameter to pass to the script. You are responsible for ensuring that the parameter is the right type. word - optional. The symbol of the gesture to pass to the script. symbol - optional. The symbol of the gesture to pass to the script, this value is usually the same sd the word parameter, or is one of the special symbols '|~BeforeScript|, '|~AfterScript|, or '|~Default Script|. *****LAUNCHPARTENTRY :LaunchPartEntry(partEntry) - launches a package represented by a part entry, as returned by GetPartEntryFromAppFrame. partEntry-the part entry from the extras drawer, as returned by GetPartEntryFromAppFrame. *****EDITORS/PICKERS GestureLaunch provides several built-in editors/pickers to make it easier to deal with parameter values. All editors respond to the following event(s): doOpen(context,datapath,callback) context - the frame in which datapath and callback are evaluated. This is usually self. datapath - a symbol or path expression which describes the location in context where the initial data will be found. callback - a symbol or path expression which identifies the callback method that will be called when the user dismisses the picker/editor. This function will receive one parameter - the data. The parameter will be nil if the user dismissed the picker/editor by tapping the cancel button. --LIST_EDITOR Allows the user to edit a list of text items. context.(datapath) should point to a valid array. The parameter to the callback function will be the modified array, or nil. --PACKAGE_PICKER Allows the user to select a package. context.(datapath) should point to a nil slot, or to a slot containing a valid package frame to use as the initial selection. The parameter to the callback function will be the selected package frame, or nil. See the "Launch Package" script for an example of how to use the package picker. --TEXT_EDITOR Allows the user to edit text. context.(datapath) should point to the initial value, and the parameter to the callback function will be the modified text, or nil. The Word Info Frame ------------------ Many of the slots in the Word Info Frame (as documented in the Newton Programmer's Guide) do not apply to programming GestureScripts. The following slots have meaning in the context of a GestureScript. You should not rely on the values of any slots in the Word Info frame which are not documented below: words - an array of ProtoWordInterp frames which represent the alternative recognition choices for the stroke. See the description of ProtoWordInterp, below, for more information. stroke - a stroke bundle representing the actual user stroke. See the description of Stroke Bundles, below, for more information. :GetWords() - send this message to the Word Info frame to get the array of recognized words (this is a simple array of strings, instead of the ProtoWordInterp frames contained in the words array) ProtoWordInterp Frames ---------------------- This frame is used in the words array of a Word Info frame to describe the alternative recognition choices for a given stroke. You should not rely on the values of slots which are not documented below. It contains the following slots: word - the word (a string). Note that this will virtually always be a single letter string. score - an integer which represents the accuracy, or confidence level in the match. The lower this number, the higher the accuracy of the match. Stroke Bundles ------------- The stroke bundle is used in the stroke slo of a Word Info frame to describe the actual stroke as written by the user. You could use this information to provide your own gesture recognition, or to fine-tune recognition to your purposes. The following slots in the stroke bundle frame are of interest: bounds - the bounding rectangle of the stroke. strokes - an array with one element for each stroke in the bundle. You can pass these binary objects to various system functions to get information about the stroke. Some interesting functions for working with Stroke Bundles, and stroke objects: GetStroke(StrokeBundle,index) - returns the stroke binary object at index i in the strokes array GetStrokePointsArray(stroke, format) - returns a point array which contains the points in the stroke binary object, at the resolution specified in the format parameter. Point arrays have two elements for each point in a stroke, and are in Y,X order (the reverse of what you might think). The first element in the array is the first Y coordinate, the second is the first X coordinate, the third is the next Y coordinate, and so on. Use the following values for format: 0 - Screen resolution (100dpi), no duplicate points 1 - Screen resolution (100dpt), duplicates not filtered 2 - Tablet resolution (varies, but always higher than screen resolution), no duplicates 3 - Tablet resolution (varies, but always higher than screen resolution), duplicates not filtered. The Package Frame ----------------- The Package Frame is used to describe a package parameter, as chosen by the user, or to describe the return value from a Package Picker (see "Library Functions" for a description of the package picker). There are several library functions which simplify using Package Frames. See the "Library Functions" section for more information. The Package Frame contains the following slots: class:'package - this slot will always contain the symbol 'package. app - the appsymbol, if any, of the package. Certain kinds of packages do not have appsymbols. packagename - the name of the package. Built-in apps will not have a packagename. text - the text from the extrasdrawer.