Object-Associated Telepointer for Real-time Collaborative Document Editing Systems

Transcription

1 Objct-Associatd Tlpointr for Ral-tim Collaborativ Documnt Editing Systms Author Xia, Qian, Sun, David, Sun, Chngzhng, Chn, David Publishd 2005 Confrnc Titl Procdings of Th First Intrnational Confrnc on Collaborativ Computing: Ntworking, Applications and Worksharing DOI Copyright Statmnt 2005 IEEE. Prsonal us of this matrial is prmittd. Howvr, prmission to rprint/ rpublish this matrial for advrtising or promotional purposs or for crating nw collctiv works for rsal or rdistribution to srvrs or lists, or to rus any copyrightd componnt of this work in othr works must b obtaind from th IEEE. Downloadd from Griffith Rsarch Onlin

2 Objct-Associatd Tlpointr for Ral-Tim Collaborativ Documnt Editing Systms Xia Griffith Univrsity Brisban, QLD 4111, Australia David Sun Univrsity of California at Brkly Brkly, CA, USA Chngzhng Sun Nanyang Tchnological Univrsity Singapor David Chn Griffith Univrsity Brisban, QLD 4111, Australia Abstract Tlpointrs ar a ral-tim groupwar intrfac fatur to indicat whr othr usrs ar pointing. Non of xisting tlpointr tchniqus, howvr, is capabl of tracking th rfrnc objct and prsrving th rlativ position in th fac of dynamic contnt and viw changs in ral-tim collaborativ documnt diting systms. In this papr, w rport a novl Objct-Associatd Tlpointr (OAT) tchniqu with th following faturs in th fac of dynamic contnt and viw changs: (1) rlocating tlpointrs to track th rfrnc objct, (2) prsrving th position rlativ to th rfrnc objct, and (3) providing fdback of th tlpointr rlocation to th local usr with a virtual local cursor. Th ky tchniqu for supporting OAT is to xtnd th oprational transformation tchniqu with a gnric Rfr opration to support adjusting th rfrnc objct position. Th OAT tchniqu has bn implmntd in th CoWord systm basd on th transparnt adaptation approach so that it can provid th OAT support to a rang of raltim collaborativ applications. 1. Introduction Tlpointrs ar avatars of local usrs mous cursors displayd on rmot participants scrns in ral-tim groupwar systms. As an important groupwar intrfac lmnt, tlpointrs ar abl to provid a varity of group awarnss information including prsnc, location and activity. In addition, tlpointrs can act as a communication channl by convying gstural mssags [9][10]. Ths faturs mak tlpointrs a powrful mans for providing usrs with a collaboration contxt, hlping usrs coordinat th group work and improving groupwar usability [8][11]. W ar particularly intrstd in th tlpointr tchniqus in ral-tim collaborativ documnt diting systms which allow multipl usrs to dit any objcts in th shard documnt at any tim. In ths systms, awarnss of othr participants prsnc, location, and activity ar important for collaboration and coordination, so tlpointrs can b a usful tool. For usrs to obtain maningful location and activity information from a tlpointr, it is important that th tlpointr points to prcisly th sam rfrnc objct as th local cursor dos. This rfrnc objct is usually th on xisting at th local cursor position on th usr intrfac. Howvr, in diffrnt situations, th typs of th rfrnc objct can b diffrnt. For xampl, th rfrnc objct can b an intrfac widgt (.g. a button) in th shard window or a contnt objct in a shard documnt (.g. a charactr in a txt widgt). In xisting tlpointr tchniqus, th rfrnc objct pointd to by a tlpointr is static in th sns that th objct idntifir dos not chang [6][14]. For xampl, whil rfrring to a window widgt, th tlpointr is associatd with th targt widgt idntifir, which nvr changs. Whil rfrring to a charactr within a txt viwing widgt, th tlpointr is associatd with a constant widgt idntifir plus a constant indx of th charactr in th txt buffr. Th invariabl objct idntifirs nsur th corrctnss of th static rfrnc schm in a rang of groupwar systms. Unfortunatly, th static rfrnc schm dos not work in ral-tim collaborativ documnt diting systms. This is bcaus in such systms, usrs can dit any objcts in th shard documnt at any tim. As a rsult, positional rfrncs of contnt objcts ar subjct to dynamic changs. Ths changs may caus problms to tlpointing undr two circumstancs. First, whn a tlpointr is about to b rlocatd to a nw rfrnc objct in rspons to th local cursor movmnt, th rfrnc objct may hav bn movd or changd by diting oprations concurrnt with th cursor movmnt, causing th tlpointr to point to an incorrct objct. Scond, aftr a tlpointr is rlocatd to a rfrnc objct, th rfrnc objct may b movd by subsqunt diting oprations or viw changs, which may also caus th tlpointr to point to an incorrct objct /05/$ IEEE

3 Th root of ths problms is that th static rfrnc schm assums a on-to-on and static corrspondnc btwn th objct idntifir and th objct itslf and simply associats tlpointrs with objct idntifirs. If th corrspondnc btwn th objct and its idntifir changs dynamically, tlpointrs may fail to point to th corrct objct. To addrss this objct-association problm in dynamic and concurrnt shard workspacs such as ral-tim collaborativ documnt diting systms, w propos an Objct-Associatd Tlpointr (OAT) tchniqu in this papr. Th basic ida is to adjust tlpointr rfrnc paramtrs by mans of Oprational Transformation (OT) [17] so that a tlpointr is always associatd with a rfrnc objct in th fac of dynamic contnt and viw changs. Th OAT tchniqu is dvisd in th Transparnt Adaptation (TA) [21] framwork, which is abl to convrt singl-usr commrcial applications into collaborativ vrsions without changing th sourc cod. This framwork provids th ncssary tchnical infrastructur for implmnting th OAT tchniqu and facilitats th application of th OAT tchniqu to a rang of xisting commrcial singl-usr applications. Th rst of this papr is organizd as follows. First, background knowldg about OT and TA is brifly introducd. Aftrwards, problms with xisting tlpointr tchniqus ar discussd as th motivation of this work. Nxt, th ffcts that th OAT tchniqu should achiv ar discussd. Thn tchnical issus rlatd to achiving OAT ffcts ar discussd and OTbasd solutions ar proposd. Furthrmor, issus rlatd to implmnting OAT in th CoWord systm ar discussd. Finally, contributions and futur work ar summarizd. 2. Background on OT and TA 2.1 Basics of th OT Tchniqu OT was originally dsignd to support multipl usrs to insrt and dlt charactrs in rplicatd txt documnts concurrntly [4][17]. Du to its uniqu capability in achiving systm consistncy without imposing any rstrictions on usrs, OT has bcom th choic of consistncy maintnanc and group undo tchniqu for many collaborativ diting systms [1][12] [19]. Th basic ida of OT can b illustratd by using a simpl txt-diting scnario as follows. Givn a txt documnt with a string "abc" rplicatd at two collaborating sits; and two concurrnt oprations: O1 = Insrt [0, "x"] (to insrt charactr "x" at position "0"), and O2 = Dlt [3, "c"] (to dlt th charactr "c" at position "3") gnratd by two usrs at collaborating sits 1 and 2, rspctivly. Suppos th two oprations ar xcutd in th ordr of O1 and O2 (at sit 1). Aftr xcuting O1, th documnt bcoms "xabc". To xcut O2 aftr O1, O2 must b transformd against O1 to bcom: O2 = Dlt [4, "c"], whos positional paramtr is incrmntd by on du to th insrtion of on charactr "x" by O1. Excuting O2 on "1abc" shall dlt th corrct charactr "c" and th documnt bcoms "xab". Howvr, if O2 is xcutd without transformation, thn it shall incorrctly dlt charactr "b", rathr than "c". In summary, th basic ida of OT is to transform an diting opration (.g. O1) dfind on a prvious documnt stat according to th ffcts of xcutd concurrnt oprations (.g. O2), so that th transformd opration (.g. O2 ) can achiv th corrct ffct in th currnt documnt stat. 2.2 Basics of th TA Approach TA is an innovativ approach to convrting xiting or nw singl-usr applications for multi-usr ral-tim collaboration, without changing th sourc cod of th original application [21]. Th TA approach is basd on a rplicatd systm architctur whr th shard singlusr application is rplicatd at all collaborating sits, th us of th singl-usr application s API (Application Programming Intrfac) to intrcpt and rplay th usr s intractions with th shard application, and th us of th OT tchniqu to manipulat th intrcptd usr oprations for supporting rsponsiv and unconstraind (i.. concurrnt and fr) multi-usr intractions with th shard application. Th cntral ida of th TA approach is to adapt th data addrss and opration modls of th shard application s API to that of th OT tchniqu. Mor prcisly, th TA approach can b dscribd by a rfrnc modl, as shown in Figur 1. This rfrnc modl consists of thr componnts: Singl-usr Application (SA), Collaboration Adaptor (CA), and Gnric Collaboration Engin (GCE). Th main functionalitis of ths componnts ar sktchd blow. Th SA componnt provids convntional singl-usr intrfac faturs and functionalitis. This componnt can b ithr an xisting commrcial off-th-shlf singl-usr application, or a nw singl-usr functionality componnt in a multi-usr collaborativ systm, but this componnt itslf has no knowldg about multi-usr collaboration. Th CA componnt provids application-spcific collaboration capabilitis and plays a cntral rol in adapting th SA for collaboration. This componnt has th knowldg of th SA API but not its intrnals. At th cntr of this componnt is th modul of Adaptd Opration (AO), which rprsnts th SA functionalitis xposd by th API. Th AO can b gnratd by th Local Opration Handlr (LOH) modul by intrcpting

4 local usr s intractions, or rcivd by th Rmot Opration Handlr (ROH) modul from rmot usrs. With th AO rsiding btwn th API and OT, th task of adaptation btwn th API and OT is dcomposd into two moduls: SA CA GCE L O H C M API API-AO Adaptation A O AO-PO Adaptation W A O T S M R O H G U AO Adaptd Opration API Application Programming Intrfac CA CM Consistncy Maintnanc GCE Gnric Collaboration Engin LOH Local Opration Handlr OT PO Primitiv Opration ROH Rmot Opration Handlr SA Collaboration Adaptor GU Group Undo Oprational Transformation Sing-Usr Application SM Sssion Managmnt WA Workspac Awarnss Figur 1: Th TA rfrnc modl. 1. Th API-AO Adaptation modul is rsponsibl for bridging th smantic gap btwn th API and th AO so that th AO can b corrctly rplayd on th SA. 2. Th AO-PO Adaptation modul is rsponsibl for mapping btwn th AO and OT-supportd Primitiv Opration (PO) so that th undrlying OT tchniqu can b usd to nsur th corrctnss of th AO paramtrs in th prsnc of concurrncy. Th GCE componnt provids applicationindpndnt collaboration capabilitis. This componnt has no knowldg of th singl-usr application functionality and thrfor can b usd in adapting diffrnt applications. This componnt ncapsulats a packag of collaboration supporting tchniqus, including Consistncy Maintnanc (CM), Group Undo (GU), Workspac Awarnss (WA), and Sssion Managmnt (SM), tc. OT is at th cor of this componnt for supporting consistncy maintnanc, usr-initiatd undo, and workspac awarnss (.g. th tlpointr discussd in this papr) in a collaborativ nvironmnt. For comprhnsiv discussions of th tchniqus ncapsulatd in th OT modul, th radr is rfrrd to [19][17][18][20]. 3. Rlatd Work Th tlpointr has a history almost as long as th pointing dvic (mous) [5]. Th tlpointr s rfrnc schm has bn volving ovr th yars. 3.1 Window Coordinat-Associatd Tlpointrs In arly application sharing and tlconfrnc systms lik Colab [16] and MMConf [3], a tlpointr is associatd with th coordinats in th shard window spac. In ths systms, a strict WYSIWIS viw mod is adoptd. As a rsult, all usrs hav xactly th sam viw of th shard window. This nsurs that ach objct is placd at th sam position in th shard window, and th sam coordinats point to th sam objct at all sits, so th window coordinats ar sufficint for a tlpointr to locat any objcts in th shard window. 3.2 Widgt-Associatd Tlpointrs In a rlaxd WYSIWIS viw mod, a shard window can hav diffrnt layout among participating sits. To accommodat th viw diffrnc, tchniqus associating tlpointrs with window widgts hav bn proposd, including Smart Tlpointr [14] and GroupKit [15]. With ths tchniqus, a tlpointr is associatd with idntifirs of a window widgt, and providd with th rlativ position insid th widgt spac. For xampl, in Smart Tlpointr, th tlpointr s rfrnc paramtrs includ a path in th widgt tr from th root to th laflvl widgt which contains th tlpointr and rlativ position information within th laf-lvl widgt spac. 3.3 Objct Position-Associatd Tlpointrs Som widgts may hav intrnal structurs or contnts (.g. a txt ditor or an HTML viwr). In a rlaxd WYSIWIS viw mod, th intrnal contnt may b formattd and displayd diffrntly du to diffrnt viw customization among collaborating sits. For such widgts, th widgt association is not nough. Smart Tlpointr associats a tlpointr with th contnt objct position by attaching th indx of th rfrnc objct (.g. th charactr indx in a txt buffr) in th tlpointr rfrnc paramtrs, so that th tlpointr can point to th sam contnt objct as th local cursor dos. This tchniqu is also adoptd in GroupWb [7] and Flxibl JAMM [1]. Although ths tlpointr tchniqus work wll in thir own nvironmnts and could achiv th ffcts thy wr dsignd for, thy ar not applicabl for tlpointing contnt objcts in ral-tim collaborativ documnt diting systms du to dynamic contnt and viw changs. To solv this problm, an Objct-Associatd Tlpointr (OAT) tchniqu is proposd in this papr. Unlik xisting tlpointr tchniqus, th OAT tchniqu rally associats a tlpointr with th rfrnc objct rathr than its position. It is abl to corrctly point to th rfrnc objct in th fac of

5 dynamic contnt and viw changs causd by both concurrnt and squntial diting oprations. In particular, it is abl to achiv th objct-associatd ffcts which will b discussd in th nxt sction. 4. Objct-Associatd Effcts for Tlpointing In this sction, w shall discuss th objct-associatd ffcts that th OAT tchniqu should achiv. 4.1 Positional Rfrnc Adjusting (PRA) Effct Th tlpointr idntifis th rfrnc objct by its positional rfrnc in th documnt. Th tlpointr should b abl to adjust its positional rfrnc in ordr to track th rfrnc objct in th fac of dynamic contnt changs causd by diting oprations. Exampls of th PRA ffct ar shown in Figur 2. At th initial stat (Figur 2-), th tlpointr is pointing to th charactr p at position 4. Aftr th xcution of an insrt or dlt opration, th positional rfrnc of th objct may b changd. To achiv th PRA ffct, th tlpointr positional rfrnc must b adjustd so that it still points to th charactr p, as shown in Figur 2- and (c). It should b pointd out that diting oprations could b gnratd concurrntly with or squntially aftr a tlpointr moving opration, th PRA ffct must b achivd undr all circumstancs. (c) T l point r T h point r T l poi nt r Figur 2: Th PRA ffct (th tlpointr tracks th rfrnc charactr p ). Th initial stat; Th stat aftr xcuting an insrt; and (c) Th stat aftr xcuting a dlt. 4.2 Rlativ Position-Prsrving (RPP) Effct Th tlpointr s position rlativ to th rfrnc objct should b prsrvd in th fac of dynamic changs to th documnt. An xampl of th RPP ffct is shown in Figur 3. At th initial stat (Figur 3-), th tlpointr is pointing at th cntr of th pictur. Aftr th xcution of a siz-updating opration, th pictur is rsizd to 50% of th original siz. To achiv th RPP ffct, th tlpointr position must b adjustd to accommodat th ffct of th updating opration on th objct so that it still points to th cntr of th pictur (Figur 3-). Rsizing Figur 3: Th RPP ffct. Th initial stat; Th stat aftr xcuting a rsiz opration: th tlpointr rmains insid th pictur. Whn th usr is prforming gsturs with th cursor, th cursor is mor oftn outsid, rathr than insid, th rfrnc objct. Th RPP ffct should also b achivd whn th tlpointr is outsid th rfrnc objct or in a blank ara. In this cas, w associat th tlpointr with th narst objct. An xampl is shown in Figur 4. In th initial stat (Figur 4-), th tlpointr is in th blank ara nar th pictur. Aftr th pictur is rsizd, th tlpointr is rlocatd accordingly so that it still points at th sam position rlativ to th rfrnc objct (Figur 4-). Rsizing Figur 4: Th RPP ffct whn th tlpointr is in a blank ara. Th initial stat; Th stat aftr xcuting a rsiz opration on th pictur: th tlpointr rmains outsid th pictur. 4.3 Th Virtual Local Cursor Tlpointrs ar usd to rprsnt th positions of thir corrsponding local cursors, and thrfor thy should b kpt consistnt with th local cursors. To achiv th PRA and RPP ffcts, tlpointrs may b rlocatd dynamically to track th rfrnc objcts. Aftr rlocation of th tlpointrs, th positions of ths tlpointrs at rmot sits may no longr b consistnt with thir corrsponding local cursor. On way to kp thm consistnt is to rlocat th local cursor as wll, but this can b disruptiv to th usr. To solv this problm, w introduc th notion of a virtual local cursor, which is th sam as a tlpointr but displayd at th local sit. Whn rlocation of any tlpointr occurs at a rmot sit, th virtual local cursor shall b rlocatd to track th rfrnc objct, but th local cursor is not movd. This virtual local cursor provids a fdback to th local usr about th locations of his/hr tlpointrs at rmot sits.

6 T l point r Insrting T h Th virtual local cursor T l poi nt r Figur 5: Th virtual local cursor for tracking th rfrnc objct. Th initial stat; Th stat aftr th rfrnc objct is pushd to th right, th virtual local cursor follows th rfrnc objct, but th ral local cursor is not affctd. Considr th xampl shown in Figur 2. Whn th string Th is insrtd, th tlpointr is rlocatd to track th charactr p (Figur 2-). What happns at usr s local sit at th sam tim is shown in Figur 5. In th initial stat (Figur 5-), th local cursor is pointing at th charactr p at position 4. Aftr th string is insrtd, th virtual local cursor tracks th rfrnc objct (Figur 5-), but th ral local cursor is not affctd. 4.4 Objct-Associatd Tlpointing Opration Dfinition Th objct-associatd tlpointing opration is dfind as follows: Cursor_mov [RfrncAddr, <RlativX, RlativY> ]: to mov th cursor to th position spcifid by th paramtrs. In an application supporting OAT, th Cursor_mov opration is gnratd from th local sit whn th local cursor movs and propagatd to rmot sits to rlocat tlpointrs accordingly. Th RfrncAddr paramtr is a linar indx to addrss th rfrnc objct in th documnt. Although this linar addrssing schm is modld aftr plain txt ditors, graphs and othr complx objcts can also b mappd to a linar addrssing spac and b addrssd with linar indics [21]. Th pair <RlativX, RlativY> rprsnts th cursor position rlativ to th rfrnc objct. Th rlativ position is masurd as ratios to th rfrnc objct siz, rathr than absolut pixl numbrs. Thortically, any point insid th rfrnc objct can b chosn as th rfrnc point as long as all collaborating sits mak th sam choic. In this papr, w tak th lft top cornr of th rfrnc objct as th rfrnc point. Whn applying th OAT tchniqu in a concrt application, th Cursor_mov opration may nd additional paramtrs, such as sit idntifirs, documnt idntifirs and window idntifirs. 5. Achiving th Objct-Associatd Effcts In this sction, w shall discuss tchnical problms and solutions rlatd to achiving th objct-associatd ffcts. 5.1 Achiving th PRA Effct Lik diting oprations (.g. Insrt, Dlt), th Cursor_mov opration also rfrs to th rfrnc objct with linar addrss. Thrfor, inconsistncy problms including Causality Violation and Intntion Violation [19] could also occur to th Cursor_mov oprations. Gnrally, th solution to th Causality Violation problm is to timstamp ach opration basd on vctor logic clocks [2][13], and to xcut an opration only whn all oprations causally bfor it hav bn xcutd. Th solution to th Intntion Violation problm is to transform ach opration with OT bfor xcuting (for dtails of th Causality Violation and Intntion Violation problms and solutions, th radr is rfrrd to [19]). Howvr, xisting OT tchniqus do not support transforming th Cursor_mov opration. Hr w propos an approach to supporting th transformation of th Cursor_mov opration by xtnding th OT tchniqu. Rathr than dirctly transforming th Cursor_mov opration, w transform it by mans of a primitiv opration Rfr. Th Rfr opration is dfind as follows: Rfr[pos]: to rfr to th objct at th position pos. Th pos paramtr indicats th addrss of th rfrnc objct in th documnt. Whn convrting a Cursor_mov opration into a Rfr opration, th pos paramtr quals to th RfrncAddr paramtr. In all Cursor_mov paramtrs, only RfrncAddr is includd in th Rfr opration bcaus only this paramtr may b affctd by concurrnt oprations and is rlvant to OT. With th mdiation of th Rfr opration, transformation of a Cursor_mov opration is don in th following stps: 1. Th Cursor_mov opration is convrtd into a Rfr opration. 2. Th Rfr opration is transformd with OT. 3. Th RfrncAddr paramtr of th Cursor_mov opration is adjustd according to th pos paramtr of th transformd Rfr opration. In this way, th Cursor_mov is ffctivly transformd against concurrnt oprations. Th major rason for translating th Cursor_mov opration into a Rfr opration for OT procssing is that Rfr is mor gnric in th sns that, in addition to rprsnting objct rfrnc for tlpointing, it can also b usd to rprsnt othr oprations that only rfr to objcts in th documnt without gnrating any modification.

7 Th OT tchniqu consists of two sparat layrs [19]: th high-lvl transformation control layr and low-lvl transformation function layr. Th transformation control layr dtrmins which oprations ar transformd; th transformation function layr dtrmins how th oprations ar transformd. To add a nw opration into an xisting OT tchniqu, w only nd to dfin transformation functions for this nw opration whil kping th high-lvl control algorithm unchangd. Thr ar two kinds of transformation functions: inclusiv transformation (IT) and xclusiv transformation (ET) functions [19]. IT transforms opration Oa against opration Ob in such a way that th impact of Ob is ffctivly includd; ET transforms Oa against Ob in such a way that th impact of Ob is ffctivly xcludd. For th Rfr opration, ET functions ar not ndd, bcaus Rfr dos not hav ffcts on othr oprations and hnc is not savd in th HB. For th sam rason, IT functions that transform othr oprations against th Rfr opration ar not ndd ithr. Thrfor, w only nd to dfin IT functions for transforming Rfr against othr oprations. IT_RI(Or, Oi) if (Oi.pos <= Or.pos) Or.pos = Or.pos + Oi.ln; rturn Or; IT_RD(Or, Od) if (Od.pos + Od.ln < Or.pos) Or.pos = Or.pos - Od.ln; ls if ((Od.pos < Or.pos) && (Od.pos + Od.ln >= Or.pos)) Or.pos = Od.pos; rturn Or; IT_RU(Or, Ou) rturn Or; Figur 6: IT functions for th Rfr opration. Th OT tchniqu supports thr primitiv oprations, which ar Insrt, Dlt and Updat [20]. IT functions transforming Rfr against ths oprations ar shown in Figur 6. In th function dscription, w us th dot notion to rfr to a paramtr of an opration. For xampl, w us Oa.ln to rfr to th ln (lngth) paramtr of opration Oa and us Oa.pos to rfr to th pos (position) paramtr of th Oa. Whn a Rfr opration is transformd against an Insrt opration (IT_RI), th Rfr s position is shiftd to th right by th Insrt s lngth if th Insrt s position is to th lft of th Rfr s position, bcaus th rfrnc objct is pushd to th right. If th Insrt s position is to th right of th Rfr s position, thn th Rfr s position paramtr is not changd. Transforming a Rfr against a Dlt (IT_RD) is mor complx. If th rang of th Dlt opration is compltly to th lft of th Rfr s position, thn th Rfr s position is shiftd to th lft by th Dlt s lngth, bcaus th rfrnc objct is pulld to th lft. If th Dlt s rang covrs th Rfr s position, thn th position of th Rfr is st to th position of th Dlt, bcaus th original rfrnc objct is dltd by th Dlt opration and th objct at Od.pos bcoms th nw rfrnc objct. Finally, if th Dlt s position is to th right of th Rfr s position, thn th Rfr s position paramtr is not changd. Whn a Rfr is transformd against an Updat (IT_RU), th Rfr s position paramtr is not changd, bcaus an Updat opration dos not affct th position of th rfrnc objct. 5.2 Achiving th RPP Effct Th RPP ffct can b achivd by making us of th rlativ ratio position paramtrs of th Cursor_mov opration x (pixl) y (pixl) David Rsizing y (pixl) David 100 x (pixl) Figur 7: A scnario of achiving th RPP ffct with th rlativ ratio position paramtrs. Th initial stat; Th stat aftr rsizing. Whn th rfrnc objct has bn found from th documnt, coordinats of th nw tlpointr position can b calculatd by using its currnt status (position and siz) and th rlativ ratio position paramtrs. An xampl is shown in Figur 7. In th initial stat (Figur 7-), th rfrnc objct occupis an ara of 100 * 100 pixls. Usr s tlpointr is at th cntr of th rfrnc objct and th rlativ ratio position is <0.5, 0.5>, corrsponding to th rlativ pixl position <50, 50>. Usr David s tlpointr is outsid th rfrnc objct and th rlativ ratio position is <0.8, 1.2>, corrsponding to th rlativ pixl position <80, 120>. Aftr th rfrnc objct is rsizd to half of its original siz (Figur 7-), which is 50 * 50 pixls, positions of th two tlpointrs ar rcalculatd. Basd on th nw siz of th rfrnc objct and th rlativ ratio positions, th rlativ pixl position of usr s tlpointr is changd to <25, 25>, so that it is still at th cntr of th rfrnc objct; th rlativ pixl position of usr David s tlpointr is changd to <40, 60>, so that it is still at th sam position rlativ to th rfrnc objct.

8 5.3 Supporting th Virtual Local Cursor To provid th local usr with fdback of rmot tlpointr rlocation, th local cursor is accompanid with a virtual local cursor. Normally, th virtual local cursor is not visibl sinc it points at xactly th sam position as th ral local cursor dos. Whn rmot tlpointrs ar rlocatd to track th movd rfrnc objct, th virtual local cursor nds to b rlocatd accordingly and thn points to a diffrnt position from th ral local cursor. Only in this circumstanc, th virtual local cursor appars. To kp th virtual local cursor consistnt with rmot tlpointrs, th PRA and RPP ffcts should also b achivd whil rlocating th virtual local cursor. Whn th usr movs th local cursor, th virtual cursor shall disappar. Whil moving th local cursor, th usr may want to point to anothr objct. In this cas th nw rfrnc objct should b idntifid and associatd with th local cursor and tlpointrs, and rmot tlpointrs ar rlocatd accordingly. Th usr may also want to mov th local cursor to point to th original rfrnc objct. With th virtual cursor associatd with th original rfrnc objct, it is much asir for th usr to find this movd objct from th documnt. In this cas, thr is no nd to rlocat rmot tlpointrs. 5.4 Impacts of Subsqunt Editing Oprations Apart from concurrnt diting oprations, squntial diting oprations xcutd aftr a Cursor_mov opration may also chang th position or siz of th rfrnc objct and hnc invalidat th association btwn th tlpointr and th rfrnc objct. Editing oprations xcutd at any addrss could affct th position or siz of th rfrnc objct. First, oprations targting on a rfrnc objct could dirctly chang its position or siz attribut. Scond, oprations xcutd bfor a rfrnc objct could chang its linar addrss in th documnt and thn affct its position (s Figur 2- and (c)). Third, oprations xcutd aftr th rfrnc objct could chang th layout of th documnt viw, and th rfrnc objct position may also b affctd. To solv ths problms, th following tlpointr rlocation schm for accommodating changs causd by subsqunt diting oprations is ndd. 1. Addrsss of all rfrnc objcts ar adjustd to accommodat th ffct of th diting opration. 2. Nw positions of all tlpointrs ar rcalculatd basd on th nw status of rfrnc objcts. 3. Tlpointrs ar movd to nw positions if ncssary. It should b pointd out that this rlocation schm is also applid to th virtual local cursor so that it can also achiv th objct-associatd ffcts in th fac of dynamic changs causd by subsqunt diting oprations. 6. Prsrving Objct-Associatd Effcts in th Fac of Viw Changs Tlpointrs ar displayd in a layr diffrnt from th documnt viw window. Thrfor, viw changs (.g. scrolling up and down, zooming in and out) could also affct th association btwn tlpointrs and rfrnc objcts. To prsrv th objct-associatd ffcts in th fac of viw changs, tlpointrs and th virtual local cursor nd to b rlocatd as wll. An xampl is shown in Figur 8. Scrolling Figur 8: A scnario for prsrving th objct-associatd ffcts in th fac of viw chang. Th initial stat; Th stat aftr scrolling. In th initial stat (Figur 8-), th tlpointr is pointing at th cntr of th charactr A. Aftr th documnt viw is scrolld up, th position of th rfrnc objct, charactr A is movd. To prsrv th PRA and RPP ffcts, th tlpointr nds to b movd up as wll so that it still points at th cntr of th charactr A (Figur 8-). Lik diting oprations, viw changs could also b concurrnt with or squntial to th Cursor_mov opration. Thrfor, both situations should b considrd in th solution. Th ffct of th concurrnt viw chang is accommodatd by th tlpointr position calculation. Whn a Cursor_mov opration is xcutd at a rmot sit, th currnt status (aftr th viw chang) of th rfrnc objct is usd to calculat th tlpointr position. In this way, th tlpointr position has takn th ffct of viw changs into account. Viw changs affct th position and siz of th rfrnc objct without changing thir intrnal stat in th documnt. Thrfor, whn a subsqunt viw chang occurs, addrsss of rfrnc objcts do not nd to b adjustd. Only th tlpointr position rcalculation and moving ar ndd. In othr words, th tlpointr rlocation schm for subsqunt viw changs only

9 includs th scond and third stps of th tlpointr rlocation schm for subsqunt diting oprations. For th xampl shown in Figur 8, aftr th viw has bn scrolld up, th nw status of th rfrnc objct th charactr A is obtaind. Th nw position of th tlpointr is calculatd basd on th position and siz of th rfrnc objct, and th rlativ position paramtrs of th tlpointr. Finally, th tlpointr is movd to th nw position. 7. Implmnting OAT in CoWord W hav implmntd th OAT tchniqu in th CoWord systm ( which has convrtd MS Word into a ral-tim collaborativ word procssor with th TA approach, as shown in Figur 1. Th main bnfits of implmnting th OAT tchniqu in th TA framwork includ (1) th availability of GCE for transforming th Cursor_mov opration with OT, (2) th availability of th SA s API for qurying and updating documnt objct information, and (3) th ability to rus th OAT tchniqu in a rang of xisting off-th-shlf commrcial singl-usr applications. 7.1 API Support of MS Word With th OAT tchniqu, collaborating sits communicat th tlpointr information by mans of th Cursor_mov opration. Whil implmnting th OAT tchniqu in a TA-basd systm, th SA s API should provid adquat functionality for th gnration and xcution of th Cursor_mov opration. MS Word provids an objct-orintd API ( for accssing objcts in a documnt. Th documnt is rprsntd with a Documnt objct, and contnt objcts in a documnt can b accssd by a Rang objct, which rprsnts a linar squnc of objcts. A documnt can b displayd in multipl windows. Each window is rprsntd with a Window objct. Mthods and proprtis of ths objcts provid th ncssary information for th gnration and xcution of th Cursor_mov opration. To gnrat a Cursor_mov opration at th local sit, th addrss, scrn position and siz of th rfrnc objct ar ndd to dtrmin th opration paramtrs. Th rfrnc objct can b idntifid by passing th local cursor position to th RangFromPoint mthod of th Window objct corrsponding to th documnt window containing th local cursor. This mthod rturns a Rang objct containing th rfrnc objct basd on th cursor position. Th scrn position of th rfrnc objct can b obtaind by passing th Rang objct into th GtPoint mthod of th Window objct. Proprtis of th Rang objct can provid th siz and addrss of th rfrnc objct. To xcut a Cursor_mov opration at a rmot sit, th scrn position and siz of th rfrnc objct ar ndd to calculat th tlpointr position. Th RfrncAddr paramtr is first usd to accss th rfrnc objct (containd in a Rang objct) from th Documnt objct. Thn, th GtPoint mthod of th Window objct and attributs of th Rang objct can provid information ndd to calculat th tlpointr position. 7.2 Th OAT Tabl Th main data structur for supporting OAT is an OAT Tabl (OATT). Th OATT contains ntris for all cursors displayd on th scrn (including tlpointrs and th local virtual cursor). Each ntry contains a Rfr opration, a rlativ ratio position from th Cursor_mov opration and an absolut scrn position of th tlpointr. Whn a Cursor_mov opration is xcutd, th rlativ and absolut positions of th tlpointr ar savd in th ntry for th targt tlpointr. Whn a Cursor_mov opration is propagatd, a Rfr opration convrtd from th Cursor_mov, th rlativ and absolut positions of th local cursor ar savd in th local cursor ntry. Whn an diting opration is xcutd or a viw chang happns, th oprations and positions in all ntris ar updatd. Whn a sit joins a collaboration sssion, a nw ntry for th nw sit is cratd in th OATTs of all xisting sits; and th nw sit inhrits th OATT of on xisting sit. Whn a sit lavs a collaboration sssion, its corrsponding ntry is rmovd from OATTs of all xisting sits. In th TA framwork, th OATT is maintaind in th WA modul of th GCE (s Figur 1). 7.3 Handling th Cursor_mov Opration In th CoWord systm, th Cursor_mov opration is only usd to rprsnt movmnts of th local cursor, so it is dfind as an AO. On th othr hand, th Rfr opration is dirctly supportd by OT and is usd to rprsnt Cursor_mov in OT, so it is dfind as a PO. Whn th local usr movs th mous cursor, th cursor movmnts ar intrcptd by th LOH modul (s Figur 1). Thn LOH calls th API-AO Adaptation modul to gt information about th rfrnc objct from th Word API. Basd on ths paramtrs, a Cursor_mov AO is gnratd. Nxt, th virtual local cursor is hiddn if it is visibl. Finally th Cursor_mov AO is propagatd to rmot sits by th ROH modul. Manwhil, th rlativ and absolut positions of th local cursor and a Rfr PO convrtd from th

10 propagatd Cursor_mov ar savd in th local cursor ntry of th OATT. Whn th Cursor_mov AO is rcivd by th ROH modul of a rmot sit, it is convrtd to a Rfr PO by th AO-PO Adaptation modul and transformd by OT in GCE. Thn th rfrnc objct is idntifid from th documnt and its status paramtrs ar qurid from th Word API by calling th API-AO Adaptation modul. Nxt, th scrn position of th tlpointr is calculatd basd on ths paramtrs and th rlativ position paramtrs in th Cursor_mov opration. And thn, th tlpointr is movd to th nw position. Finally, th nw rlativ and absolut positions of th tlpointr and th transformd Rfr PO ar savd in th OATT. 7.4 Rlocating Tlpointrs aftr Editing Oprations Editing oprations ar gnratd at th local sit and propagatd to rmot sits and applid to rmot documnt rplicas. Thrfor, diting oprations affct tlpointrs at both th local and rmot sits. Aftr a local or rmot diting opration is xcutd, all ntris in th OATT should b rcalculatd to adapt th chang causd by th diting opration. This is don by calling th AdjustOATT routin shown in Figur 9. AdjustOATT (O, OATT) for (i = 0; i < OATT.ntris.count; i++) IT (OATT.ntris[i].Or, O); Figur 9: Th routin for adjusting th OATT. This routin transforms th Rfr opration savd in ach ntry against th nw diting opration. Positions of Rfr oprations ar adjustd to rflct th ffct of th nw diting opration. Th IT invocation in this routin chooss suitabl IT functions shown in Figur 6 according to th nw diting opration s typ. Nxt, all tlpointr positions should b rlocatd by calling th routin RlocatAllTlpointrs shown in Figur 10. Th absolut scrn position of th tlpointr in ach ntry (including th local virtual cursor ntry) is rcalculatd for its nw scrn position aftr th xcution of th nw diting opration. This rcalculation involvs th Rfr opration, th rlativ ratio position savd in OATT ntris and status information of th rfrnc objct that can b obtaind from th Word API. If th nw scrn position is diffrnt from th currnt on, thn th currnt scrn position is rplacd and th tlpointr is movd to th nw position. If th nw local virtual cursor s position is diffrnt from th local ral cursor s position, thn th virtual local cursor bcoms visibl and is movd to th nw position. RlocatAllTlpointrs(OATT) for (i = 0; i < OATT.ntris.count; i++) nw_scrn_pos = CalculatPos(OATT.ntris[i]); if (PositionChangd(OATT.ntris[i], nw_scrn_pos) SavScrnPos(OATT.ntris[i], nw_scrn_pos); if ( IsLocalCursor(OATT.ntris[i]) && VirtualCursorIsHiddn( ) ) ShowVirtualCursor( ); MovTlpointr(OATT.ntris[i]); Figur 10: Th routin for rlocating all tlpointrs. Unlik diting oprations, viw changs ar not propagatd to rmot sits, so thy only affct tlpointrs locally. Aftr a viw chang happns at th local sit, only th RlocatAllTlpointrs routin is invokd to calculat th nw positions of cursors and to mov th tlpointrs if ncssary. This is corrct bcaus viw changs do not affct addrsss of rfrnc objcts. 8. Conclusion and Futur Work In this papr, w hav contributd a novl objctassociatd tlpointing tchniqu OAT that is suitabl for ral-tim collaborativ documnt diting. Th most important fatur of th OAT tchniqu is that it is abl to track th rfrnc objct and maintain th rlativ position to th rfrnc objct in th fac of dynamic contnt and viw changs in collaborativ documnt diting systms. W dfind th dsird OAT ffcts, analyzd th rlatd consistncy and rlocation issus, and dvisd solutions basd on th Oprational Transformation (OT) tchniqu. W hav xtndd th OT tchniqu with a nw gnric primitiv opration Rfr, which can b usd to modl various objctrfrncing activitis from th usr intrfac. Morovr, th OAT tchniqu has bn implmntd in th CoWord systm to provid OAT support to ral-tim collaborativ word procssing. W ar in th procss of applying th TA approach and th OAT tchniqu to othr applications, including wb pag dsignrs, graphic and imag ditors, and CAD/CASE tools. Apart from studying th implmntation tchniqus for supporting th known group-awarnss tchniqus (.g. tlpointrs, radar viws, tc.) in TA-basd systms, w also plan to invstigat th spcial group-awarnss rquirmnts and nw supporting tchniqus in ths systms and conduct usability study of thm.

11 9. Rfrnc [1] Bgol, J., Rosson, M., and Shaffr, C. Flxibl collaboration transparncy: supporting workr indpndnc in rplicatd application-sharing systms, ACM Transactions on Computr Human Intraction, 6(2), 1999, pp [2] Birman, K., Schipr, A., and Stphnson, P. Lightwight causal and atomic group multicast. ACM Transaction on Computr Systms, 9(3), 1993, pp [3] Crowly, T., Milazzo, P., Bakr, E., Forsdick, H., and Tomlinson, R. MMConf: An infrastructur for building shard multimdia applications. In Procdings of th ACM Confrnc on Computr- Supportd Cooprativ Work, 1990, pp [4] Ellis, C. A., and Gibbs, S. J. Concurrncy control in groupwar systms, In Procdings of ACM Confrnc on Managmnt of Data, May 1989, pp [5] Englbart, D. and English, W. A rsarch cntr for augmnting human intllct. In Procdings of th Fall Joint Computing Confrnc, vol. 33, 1968, pp [6] Grnbrg, S., Gutwin, C., and Rosman, M. Smantic tlpointrs for groupwars. In Procdings of Australian Confrnc on Computr- Human Intraction, 1996, pp [7] Grnbrg, S., and Rosman, M. Groupwb, a www browsr as ral tim groupwar. Confrnc companion on Human factors in computing systms: common ground, 1996, pp [8] Gutwin, C. Workspac awarnss in ral-tim distributd groupwar. PhD thsis, Univrsity of Calgary, Calgary, Canada. [9] Gutwin, C., and Pnnr, R. Improving intrprtation of rmot gsturs with tlpointr tracs. In Procdings of ACM Confrnc on Computr- Supportd Cooprativ Work, Novmbr 2002, pp [10]Gutwin, C., Dyck, J., and Burkitt, J. Using cursor prdiction to smooth tlpointr jittr. In Procdings of ACM SIGGROUP confrnc on supporting group work, Novmbr 2003, pp [11]Gutwin, C., and Grnbrg, S. Th ffcts of workspac awarnss support on th usability of raltim distributd groupwar. ACM Transactions on Computr-Human Intraction, 6(3), Sptmbr 1999, pp [12]Li, D. and Li, R. Transparnt sharing and intropration of htrognous singl-usr applications, In Procdings of ACM Confrnc on Computr Supportd Cooprativ Work, Novmbr 2002, pp [13]Ranynal, M. and Singhal, M. Logical tim: Capturing causality in distributd systms. IEEE Comput. 29(2), Fbruary 1996, pp [14]Rodham, K. J., and Olsn D. R. Smart tlpointrs: maintaining tlpointr consistncy in th prsnc of usr intrfac customization. ACM Transactions on Graphics, 13(3), July 1994, pp [15]Rosman, M., and Grnbrg, S. Building ral-tim groupwar with groupkit, a groupwar toolkit. ACM Transactions on Computr-Human Intraction, 3(1), March 1996, pp [16]Stfik, M., Fostr, G., Bobrow, D. G., Kahn, K., Lanning, S., and Suchman, L. Byond th chalkboard: computr support for collaboration and problm solving in mtings. Communication of th ACM, 30(1), January 1987, pp [17]Sun, C. and Ellis, C. A. Oprational transformation in ral-tim group ditors: issus, algorithms, and achivmnts. In Procdings of th ACM Confrnc on Computr Supportd Cooprativ Work, 1998, pp [18]Sun, C. Undo as concurrnt invrs in group ditors. ACM Transactions on Computr-Human Intraction, 9(4), Dcmbr 2002, pp [19]Sun, C., Jia, X., Zhang, Y., Yang, Y., and Chn, D. Achiving convrgnc, causality-prsrvation, and intntion-prsrvation in ral-tim cooprativ diting systms, ACM Transactions on Computr- Human Intraction, 5(1), March 1998, pp [20]Sun, D., Xia, S., Sun, C., and Chn, D. Oprational transformation for collaborativ word procssing, In Procdings of ACM Confrnc on Computr Supportd Cooprativ Work, Novmbr 2004, pp [21]Xia, S., Sun, D., Sun, C., and Chn, D. Lvraging singl-usr applications for multi-usr collaboration: th CoWord approach. In Procdings of ACM Confrnc on Computr-Supportd Cooprativ Work, Novmbr pp