A novel asynchronous access method with binary interfaces

The problem of binary access

In order to communicate intent through a binary interface, a user must be able to intentionally determine, whenever necessary, which of the two possible states the interface should present. Thus, for example, in the case of a button, the user must be able to intentionally perform the mechanical actions required to press and release the button. Other binary interfaces may, for example, exploit the user's ability to produce a gesture [5] or blink [6] at will.

More recently, researchers have explored the detection of voluntary changes in physiological activity, such as brain [7] or electrodermal activity [8], in order to obtain a few distinct and repeatable patterns that, similarly to binary interfaces, may be used to communicate intent. These novel approaches may provide a means of access for those users whose intent may not be understood otherwise. Some of these physiological interfaces, although still minimal, are capable of respresenting more than 1 bit of information at once, however, due to a variety of design, measurement and contextual challenges, their implementation is generally simpler and more effective when only a binary mode of use is required.

In spite of all these advantages, binary interfaces also present significant limitations that preclude their use in a wide variety of access and control applications. Evidently, the binary nature of these interfaces makes them an ideal solution for the control of devices with intentional spaces that present only a dyad of possibilities (e.g., close-open, up-down, etc.) However, when access to more than two distinct outcomes is required for the successful control of a device, the limitations of the binary interface become immediately apparent. Figure 1 depicts this dilemma where a user is required to control a complex device by means of a binary interface.

Protocol-based binary access

Consider the set S = {s₀, s₁, s₂,...,s_ς-1} containing all ς states available in a typical interface. Note that, with a binary interface, ς = 2. This interface, is used to access the set C = {c₀, c₁, c₂,..., c_κ-1} of size κ, containing all possible outcomes available for selection. The initial limitation of the case κ > ς is typically overcome by the implementation of a time-bound protocol that enables the generation of a new set S _T = {f₀(t), f₁(t), f₂(t),...,f_κ-1(t)} where each element f _i (t) ∈ S _T is a time-dependent function composed by a unique sequence of channel states s _i ∈ S with duration T . This time-based coding enables the direct mapping of each member f _i (t), of the newly created set of functions S _T , to a unique message c _i ∈ C. Figure 2 shows two sample periodic state sequences f _i (t) used to communicate messages through a binary interface (i.e., ς = 2). The top sequence represents the hexadecimal number 9A _HEX as defined by the RS232 serial communication protocol. The bottom sequence represents the letter 'X' as defined by the Morse code. Evidently, there are significant similarities between early electronic communication challenges and the use of binary interfaces by Disabled users. These similarities were quickly identified by interface designers who transferred the application of time-bound communication protocols to the implementation of access solutions for the Disabled. In fact, Morse-based communication and computer access methods are still being actively researched [9, 10].

There are, however, some significant disadvantages with the use of time-bound protocols in the control of a device by a human operator. These stem mainly from the fact that both the transmitting and the receiving end must comply with the protocol used in the communication process. This requires users to either memorize all pairs {f _i (t), c _i } mapping every device outcome c _i ∈ C to its corresponding sequence f _i (t) ∈ S _T , or learn the time-coding rule g(t) : f _i (t) → c _i that may be used to generate the i-th sequence f _i (t) ∈ S _T corresponding to the desired outcome c _i ∈ C. Evidently, depending on individual abilities, this requirement will affect different users to varying degrees. However, the number κ of device outcomes that can be made available to the user will be largely limited by the user's memory capacity as well as the complexity of the protocol. Therefore, this requirement will impose, in all cases, an upper boundary κ _max on κ (i.e., κ ≤ κ _max ).

The exclusion mask

With the exception of Equation (2), there is no additional information that could help us determine, precisely, which of the remaining elements c ≠ c_[n-1]of C should be selected as the outcome c_[n]. The top trace in Figure 3 shows an alternative graphical representation of this knowledge, which may be formally defined as

Here, the element c_[n-1]is assigned a maximum value of ${\mathcal{X}}_{[n]}$(c = c_[n-1]) = 1. This value represents an absolute certainty that c_[n-1]should be excluded from the selection of the device behavior c_[n]as stated in Equation (2). Conversely, all other elements share the minimum value ${\mathcal{X}}_{[n]}$(c ≠ c_[n-1]) = 0, which represents absolute uncertainty about their possibility of exclusion from the selection of c_[n]. Thus, ${\mathcal{X}}_{[n]}$(c), which may only take values in the range [0, 1], constitutes a numerical representation of the certainty of exclusion of a given outcome c ∈ C from the selection of c_[n]. In other words, ${\mathcal{X}}_{[n]}$(c) may be used to describe a range of assumptions (from weak ${\mathcal{X}}_{[n]}$(c) ≃ 0 to strong ${\mathcal{X}}_{[n]}$(c) ≃ 1) regarding the unsuitability of outcomes in the choice c_[n]. This function will be termed the spatial exclusion mask of c_[n].

The representation of the NAK principle in Equation (2) by means of the spatial exclusion mask ${\mathcal{X}}_{[n]}$(c) may initially seem unnecessary. However, as it will be demonstrated in the following sections, this mask introduces a framework for the numerical representation of contextual knowledge that may be used to optimize the choice c_[n]in response to a single binary event ϵ.

The exclusion estimate

Consider the function ϒ(c, t) depicted in the discrete time sequence presented in Figure 4. This function describes the viscoelastic deformation of the 1-dimensional domain composed of all elements c ∈ C. The figure shows parallel bands representing the state of the domain at regular time intervals. In order to elucidate the progression of time, the bands have been colored from dark to clear corresponding to the transition from older to more recent states of the domain. We will assume ϒ(c, t) has been left undisturbed for a long time t <t_[n]allowing it to maintain its natural at state (i.e., ϒ(c, t <t_[n]) = 0 for all possible outcomes c ∈ C). Then, at a given time t = t_[n], the domain is subject to a deformation ${\mathcal{X}}_{[n]}$(c) as depicted by the bottom trace in Figure 3. By definition, the viscoelastic deformation process would allow ϒ(c, t) to recover its natural state. However, as depicted by subsequent bands, this will only happen gradually over time.

The sequence in Figure 4 depicts the temporal memory effect inherent to the mechanical property of viscoelasticity. Note that this property fulfills the requirements stated in the previous section for the incorporation of temporal assumptions in the choice c_[n]. In particular, in the context of asynchronous access, the deformation ϒ(c, t) subject to consecutive disturbances ${\mathcal{X}}_{[n]}$(c), would allow recent assumptions ${\mathcal{X}}_{[n]}$(c) (i.e., those in the temporal neighborhood of c_[n-1]) to be assigned higher values of exclusion than former ones. In other words, the function ϒ(c, t) may be used to record the the full set of historical assumptions represented by all spatial exclusion masks $\{{\mathcal{X}}_{[n-1]}(c),{\mathcal{X}}_{[n-2]}(c),{\mathcal{X}}_{[n-3]}(c)\mathrm{...}\}$ preceding the n-th event ϵ. A simple recursive algorithm may be used to represent this process.

Let Δt be the period between the time t_[n]of the n-th event ϵ and the time t_[n-1]of the preceding event, that is

Δt = t_[n]- t_[n-1]

and ϒ_[n](c) the function ϒ(c, t) evaluated at time t_[n], that is

ϒ_[n](c) = ϒ(c, t = t_[n])

where τ is a time constant always greater than zero. The exponential decay described in Equation (8) derives from the behavior of real viscoelastic systems such as the discharge of an electric capacitor or the restoration of a mechanical shock absorber [17]. In all these cases, the constant τ is termed the viscoelastic constant and it is directly proportional to the duration of the viscoelastic restoration of ϒ_[n](c).

Note that ${\mathcal{X}}_{[n]}$(c) and ${\mathcal{H}}_{[n]}$(Δt) are weighting functions acting on the spatial and temporal domains, respectively, of the exclusion estimate ϒ_[n](c). While the spatial exclusion mask ${\mathcal{X}}_{[n]}$(c) ensures that outcomes similar to c_[n-1]are excluded from the choice c_[n], the function ${\mathcal{H}}_{[n]}$(Δt) ensures that recent exclusion estimates ϒ(c, t) are remembered while old ones are forgotten. Thus, ${\mathcal{H}}_{[n]}$(Δt) is our temporal exclusion mask. Note that the support of ${\mathcal{H}}_{[n]}$(Δt) is defined for values in the range [0, α _t ] with α _t > 0. In the case of the family of functions in Equation (8), α _t = ∞.

The definition of the exclusion estimate ϒ_[n](c) in Equation (7), which now integrates spatial and temporal assumptions, suggests that the best possible choice of c_[n]should be the element c ∈ C that minimizes ϒ_[n](c). Thus,

C_[n]= argmin ϒ_[n](c)

Figure 5 shows a discrete time sequence of the evolution of ϒ(c, t), according to Equation (9), where three different events ϵ occur at consecutive times. Note that it has taken only two events ϵ, with corresponding exclusion masks ${\mathcal{X}}_{[n-2]}$(c) and ${\mathcal{X}}_{[n-1]}$(c), for ϒ(c, t) to converge from a state of absolute uncertainty for t <t_[n-2], to a unique solution c_[n-1]at t = t_[n-1].

Equation (9) summarizes the decision process proposed for the asynchronous selection of a new device outcome c_[n]∈ C in response to a single binary event ϵ, consisting, in our example of single-switch access, of an intentional button press. Thanks to the assumptions incorporated in ${\mathcal{X}}_{[n]}$(c) and ${\mathcal{H}}_{[n]}$(Δt), the number of eligible device outcomes in the choice c_[n]is significantly reduced. In fact, with the appropriate parameters, Equation (9) will consistently converge to a unique solution soon after the interaction between the user and the device under control is initiated.

The process for asynchronous access presented here incorporates a number of desirable properties that make it easy to implement and adaptable to a wide variety of contexts. Among these properties are:

There are no restrictions on the time at which a particular event ϵ may occur. For users, this translates into the ability to respond immediately to a change in their intentions or an unexpected external disturbance on the device under control.

The recursive nature of the exclusion estimate ϒ_[n](c) eliminates the need for the implicit calculation of the effects of the set of historical assumptions $\{{\mathcal{X}}_{[n-1]}(c),{\mathcal{X}}_{[n-2]}(c),{\mathcal{X}}_{[n-3]}(c),\mathrm{...}\}$ on the selection of c_[n], thus reducing the processing power and memory storage capacity required for the implementation of the proposed method for asynchronous access.

There is no limit on the number κ of outcomes C that may be made available to the user through this method. In fact, the set C may be defined as a continuous interval of all possible real valued outcomes c ∈ [c _min , c _max ], where c _min and c _max are the lower and upper boundaries of C, respectively. Evidently, in this case, κ = ∞.

Summary

1. According to Equation (2), when the n-th event ϵ occurs, the device outcome c_[n]must be different from the outcome c_[n-1]immediately preceding it. In other words, there is absolute certainty that c_[n-1]should be excluded from the selection of c_[n]. Thus, the event ϵ, which represents a voluntary, user-prompted change in the interface, should be employed by users as an error indicator. This requires users to generate events ϵ every time the behavior of the device is inconsistent with their intentions.

2. Even though the exclusion principle in Equation (2) is the only knowledge implied, with absolute certainty, by the occurrence of event ϵ, it is also possible to assume, although with a lower degree of certainty, that behaviors similar to c_[n-1]should also be excluded from the selection of c_[n]. This assumption is defined by the spatial exclusion mask ${\mathcal{X}}_{[n]}$(c), a function with values in the range [0, 1] and support c_[n-1]± α _s , decreasing monotonically from ${\mathcal{X}}_{[n]}$(c = c_[n-1])= 1 (i.e., the strongest assumption of exclusion) to ${\mathcal{X}}_{[n]}$(c = c_[n-1]± α _s ) (i.e., the weakest assumption of exclusion) as candidate outcomes c ∈ C become decreasingly similar to c_[n-1].

3. It may also be assumed that device outcomes resulting from recent selections (i.e., immediately preceding the n-th event ϵ), should be excluded from the selection of c_[n], while outcomes that belong to the remote past of n should become eligible. The incorporation of this assumption is made possible through the introduction of the exclusion estimate ϒ_[n](c) and the temporal exclusion mask ${\mathcal{H}}_{[n]}$(Δt), where Δt is, according to Equation (5), the period between the time t_[n]of the n-th event and the time t_[n-1]of its predecessor. According to Equation (7), the exclusion estimate ϒ_[n](c), which is recursively defined in terms of the exclusion estimate ϒ_[n-1](c) of the preceding event, acts as a viscoelastic domain storing the set of historical deformations $\{{\mathcal{X}}_{[n]}(c),{\mathcal{X}}_{[n-1]}(c),{\mathcal{X}}_{[n-2]}(c),\mathrm{...}\}$ subject to a viscoelastic decay described by the temporal exclusion mask ${\mathcal{H}}_{[n]}$(Δt). Thus, ${\mathcal{H}}_{[n]}$(Δt) must decrease monotonically from ${\mathcal{X}}_{[n]}$(Δt = 0) = 1 to ${\mathcal{H}}_{[n]}$(Δt = ∞) = 0. Note that the functions ${\mathcal{X}}_{[n]}$(c) and ${\mathcal{H}}_{[n]}$(Δt) act as weighting masks on ϒ_[n](c) updating the certainty of exclusion, from the choice c_[n], for every candidate outcome c ∈ C, according to reasonable spatial and temporal assumptions, respectively.

4. Once the exclusion estimate ϒ_[n](c) is calculated, it will be possible to make an informed decision regarding the best possible choice of c_[n]∈ C according to Equation (9).

Initialization

Note that the decision process described in Equation (9) does not specify the characteristics of the exclusion estimate ϒ_[0](c) before the first (i.e., n = 1) event ϵ is generated by the user. In fact, there is no information regarding the value of the initial device outcome c _[0] either, and without this knowledge, the recursive process described in Equation (7) may not be initialized. Thus, even before the user initiates interaction with the device, a virtual selection c_[0] must be made. Similarly to the case where the concept of viscoelasticity was first introduced, we may assume that before the first user-prompted event (i.e., t <t_[1]) the exclusion estimate had been left undisturbed for a long time, thus allowing it to recover its natural flat state. That is, ϒ_[0](c) = 0 for all outcomes c_[0] = c ∈ C. Moreover, since all values c ∈ C fulfill the condition in Equation (9), we would then be obliged to draw c_[0] from a uniform distribution of C. Consequently, this random selection of c_[0] may be used to initialize the decision process Equation (9). Note that it is not necessary to communicate the virtual choice c_[0] to the device under control. Thus, the device may remain undisturbed until after the first user-prompted event ϵ occurs. In this case, the virtual choice c_[0] will only be used to obtain the first exclusion mask ${\mathcal{X}}_{[1]}$(c) at t_[1], enabling the calculation of the estimate ϒ_[1](c). The resulting outcome c_[1] will then be the first to affect the device's behavior. From the perspective of the user, it will appear that the outcome c_[1] has been drawn randomly from a uniform distribution. However, as explained here, this is only the case for the virtual choice c_[0], since, according to Equation (9) c_[1] will be drawn from a more restricted distribution where a subset of the elements c ∈ C (i.e., ~ c_[0] ± α _s ) have already been excluded.

An alternative (and, in fact, more useful) procedure consists of initializing ϒ_[0](c) with random white noise in the interval of real numbers [0, 1]. This minimizes the probability of having multiple candidates for the virtual choice c_[0], since it is expected that, after this initialization, ϒ_[0](c) will present a unique minimum value, which may then constitute the virtual choice c_[0]. The advantage of this method over the one initially proposed, resides in the fact that with the latter, ϒ_[n](c) will more likely converge to a unique solution from the beginning (i.e., n = 1) of the interaction. In fact, this also allows for the prediction of future selections of c_[n]∈ C with a significant degree of confidence.

Anchorage

If the viscoelastic constant τ is too long, or a significant number of events ϵ occur in a short amount of time, the exclusion estimate ϒ_[n](c) may accumulate constant offsets from previous, but still remembered deformations $\mathcal{X}$(c). Due to the discretization process inherent to any numerical implementation of the proposed method (e.g., on a computer), this offset accumulation may in fact cause saturation of the exclusion mask ϒ_[n](c). That is, ϒ_[n](c) ≃ 1 for all outcomes c ∈ C. If saturation occurs, the information storage capacity of the exclusion estimate will be completely eliminated, thus, preventing the selection of reasonable outcomes c_[n]derived from the spatial and temporal assumptions introduced before.

In order to prevent the occurrence of saturation, constant offsets must be eliminated at all times from the exclusion estimate ϒ_[n](c). This may be achieved by subtracting the value of ϒ_[n](c = c_[n]) from the function ϒ_[n](c). That is

ϒ_[n](c) ⇐ ϒ_[n](c) - ϒ_[n](c = c_[n])

where ϒ_[n](c = c_[n]) is the value of ϒ_[n](c) evaluated at the recently obtained outcome c_[n]. Evidently, if ϒ_[n](c = c_[n]) is already zero, Equation (10) will have no effect on ϒ_[n](c). This process of elimination of the offset of the exclusion estimate ϒ_[n](c) is termed anchorage. The process of anchorage has no effect on the decision c_[n], since this decision only depends on the relative exclusion value of a given outcome c ∈ C as compared to the rest of the elements of C.

Algorithm

The following list summarizes the sequential steps required for the implementation of the proposed asynchronous access method.

1. Originally, nothing is known about the intention of the user regarding the behavior of the device. Thus, the exclusion estimate ϒ_[0](c) may be initialized with white noise in the range [0, 1]. This results in the definition of the virtual choice c_[0] and the exclusion mask ${\mathcal{X}}_{[1]}$(c), which precede any user interaction and, therefore, any change in the behavior of the device.

2. When the n-th intentional binary event ϵ occurs, the period Δt is calculated according to Equation (5) and used to obtain the decay ${\mathcal{H}}_{[n]}$(Δt) through a suitable function such as Equation (8). Subsequently, the intention estimate ϒ_[n](c) is updated according to Equation (7).

3. The corresponding n-th device outcome c_[n]may now be obtained according to Equation (9). This outcome is immediately transmitted to the device which experiences a change in behavior.

4. The exclusion estimate ϒ_[n](c) is anchored according to Equation (10).

5. The exclusion mask ${\mathcal{X}}_{[n]}$(c) is updated to ${\mathcal{X}}_{[n+1]}$(c) through a suitable function such as Equation (4).

6. For subsequent events ϵ, the process is repeated from (ii) above.

In addition, the fundamental spatial and temporal assumptions require their corresponding exclusion masks ${\mathcal{X}}_{[n]}$(c) and ${\mathcal{H}}_{[n]}$(Δt) to have the following properties:

The spatial exclusion mask ${\mathcal{X}}_{[n]}$(c) must decrease monotonically from ${\mathcal{X}}_{[n]}$(c = c_[n-1]) = 1 to ${\mathcal{X}}_{[n]}$(c = c_[n-1]± α _s ) = 0. The support of this function will be defined in the range [c_[n-1]- α _s , c_[n-1]+ α _s ].

The temporal exclusion mask ${\mathcal{H}}_{[n]}$(Δt) must decrease monotonically from ${\mathcal{H}}_{[n]}$(Δt = 0) = 1 to ${\mathcal{H}}_{[n]}$(Δt = α _t ) = 0. The support of this function will be defined in the range [0, α _t ] where α _t > 0.

Note that although these assumptions are reasonable given the access problem proposed, there is no limit to the number and/or kind of assumptions that may be incorporated into ${\mathcal{X}}_{[n]}$(c) and ${\mathcal{H}}_{[n]}$(Δt). For example, one could deliberately exclude a particular outcome c _i ∈ C (i.e., ${\mathcal{H}}_{[n]}$(c _i ) = 1) or all events occurring before a certain memory threshold Δt₀ (i.e., ${\mathcal{H}}_{[n]}$(Δt < Δt₀) = 0) in response to some contextual knowledge.

Modelling the user

As mentioned before, it was assumed that the model user was able to monitor the behavior of the device under control through visual means. This process would involve a series of delays as a result of the time required by the user to process the visual information and, if necessary, generate the event ϵ. Thus, in order to obtain an accurate model of the visual reaction time t _r , which includes both the visual perception and motor reaction times, an initial experiment was performed with a real user. During this experiment, the real user was requested to respond to simple visual stimuli presented on a computer screen. The stimuli consisted of the appearance of a white circle on a black background after random delays of 1 to 3 seconds. The user was instructed to press a button (defined as the event ϵ) immediately after the stimulus (i.e., the white circle) appeared on the screen. The experiment was performed using the open source software package PXLab, which can be used to accurately measure the user's reaction time t _r defined as the period from the presentation of the stimulus, to the generation of the intentional event ϵ. A histogram of the reaction times, t _r , was obtained with a total of 100 trials. This histogram was used to represent the model user in the Monte Carlo simulations introduced above. Thus, for each event ϵ, a reaction time, t _r , was randomly drawn from the histogram. The expected value $\overline{t_r}$ of this user's reaction time was ~213 ms, which is consistent with previous research on the topic [20]. Thus, it may be assumed that the statistical model, represented by the histogram obtained, was an accurate estimate of user behavior incorporating the stochastic nature of the interaction between a real user and a device.

Cases for evaluation

According to the proposed method of asynchronous control, there are few restrictions to the definitions of the exclusion masks ${\mathcal{X}}_{[n]}$(c) and ${\mathcal{H}}_{[n]}$(Δt). As a result, there is an infinite number of functions that comply with the basic requirements of both of these functions. We will focus on the evaluation of a single family of functions for each of the exclusion masks defined. These functions have already been introduced and correspond to some of the simpler sets of assumptions that may be made in compliance with the necessary requirements for the linear spatial exclusion mask ${\mathcal{X}}_{[n]}$(c) in Equation (4) and the exponential temporal exclusion mask ${\mathcal{H}}_{[n]}$(Δt) in Equation (8).

Each case for evaluation was defined by a set, $\underset{˜}{\theta }$, of three parameters:

The width ω = α _s ·(c _max - c _min )^-1 of the linear spatial exclusion mask ${\mathcal{X}}_{[n]}$(c). This parameter specified the fraction of the full length of C defining the support boundaries of ${\mathcal{X}}_{[n]}$(c) as defined in Equation (4).

The viscoelastic constant τ of the temporal exclusion mask ${\mathcal{H}}_{[n]}$(Δt) as defined in Equation (8). This parameter defined the expected size, in seconds, of the memory window of the exclusion estimate ϒ_[n](c).

In total, 1976 sets $\underset{˜}{\theta }$ = {σ, ω, τ} were evaluated, and, as mentioned before, each set consisted of 6000 separate trials where the model user was requested to reach a specific target c _γ using a single binary interface and the proposed method for asynchronous access.

Performance measure

where $P_{\underset{˜}{\theta }}$(N ≤ X) is the CDF obtained for a particular set $\underset{˜}{\theta }$ = {σ, ω, τ}. Note that Γ is a relative measure of performance with reference to a process of random selection with substitution subject to the given tolerance σ . This latter process is captured by the second term in Equation (13) and defined in Equation (11). In the limit X → ∞, both terms of the subtraction tend to 1 thus, in practice, it is only necessary to consider a sufficiently large number for X. For example, in the case presented in Figure 6, the value X = 40 would be an appropriate limit for the summations.

According to Equation (13), positive values of Γ($\underset{˜}{\theta }$) indicate lower usage costs. Thus, in order to optimize the cost, Γ($\underset{˜}{\theta }$) must be maximized. Conversely, negative values of Γ($\underset{˜}{\theta }$) would indicate a disastrous performance (i.e., even worse than a random guess). Finally, a value Γ($\underset{˜}{\theta }$) = 0 would indicate similar performance between a random guess and the proposed algorithm with the particular parameter set $\underset{˜}{\theta }$. However, in such cases, the additional complexity of the proposed method would not justify its application. Thus, these sets should also be avoided.

A sample application

In order to demonstrate the relevance of the proposed asynchronous access method in the control of real appliances, we implemented a single-switch drawing application termed the one-button doodler [21], this on-line tool allows minimal interface users to create "free-hand" drawings using only the left button of a computer mouse or a single keyboard key. Figure 8 depicts a sample drawing made by a minimal interface user by means of this software application. A particular implementation of the proposed method, with spatial exclusion mask ${\mathcal{X}}_{[n]}$(c) defined by Equation (14), allowed this user to access a total of 180 different angles, allowing a pointer to follow the trajectory defined by the depicted trace. Note that, given the angular nature of the domain under control, the mask introduced here was more appropriate for this application than the mask defined in Equation (4) above.

We have previously reported that, in order to minimize the delay between the user's action and the device's response, the outcome c_[n]is transmitted to the device immediately after it is selected. However, in recent experiments involving real users, it has been evident that this immediacy is not as important as the ability to select the intended behaviour with high accuracy. Thus, in some cases, users prefer to have some time to reject the most recent selection proposed by the algorithm. Furthermore, since the algorithm becomes highly predictable soon after the interaction with the user has been initiated, it is possible to display a list of suggested behaviors that will follow the most recent selection. When available, this additional information can improve accuracy significantly. Full results of these and other studies involving real users will be reported in subsequent publications.