My own wishlist

[David Faught]

1. Further integration of Tweak and Croquet where:
  a. Morphic is gone, both graphical and event handing,
  b. Multiple Tweak projects can be used simultaneously, both in
windows/portals in the 3D space and as overlays as described in the
"Filters And Tasks in Croquet" whitepaper, and
  c. Event handling is structured from the ground up to support this
environment.

2. The Tweak-based Croquet UI could be more context-oriented (kill the
top of the screen menu bar!) and make use of all the Tweak facilities,
including EToys, in the overlays.

3. Not having used it, I'm not sure how Brie fits with this, but I
speculate that having both a 2D and 3D UI would be useful.

4. Performance is improved so that 3D interaction is more natural.  I
speculate that putting everything through the harness-router-island
slows down interaction with the scene quite a bit.  Some things that
ran okay in Jasmine are really slow in Hedgehog.  I have thought about
building a separate Tweak project-based OpenGL window (Bert provided
some basic code for this a while ago) so that quicker, more direct
interaction with the 3D scene could be used for something like a 3D
modeling package.  Maybe this could actually be an expansion of the
public/private separation of objects so that you could do model
building in a local private 3D space and then easily share it with the
public island.  Not sure if this would really be a good alternative to
solving performance issues.

5. (running out of time to write this) World peace, an end to hunger
in the world, and free flash drives and webcams for everyone,
especially the OLPC recipients.

More some other day.

[Howard Stearns]

David Faught wrote:
> ..
>
> 3. Not having used it, I'm not sure how Brie fits with this, but I
> speculate that having both a 2D and 3D UI would be useful.
> ...

Brie has not yet been re-implemented on Hedgehog. It is wise in any
distributed project to not be dependent on new work by other projects --
  Brie, Tweak, or anything else. Croquet will certainly not be dependent
on Brie. (Forever, I hope!)

That said, the intention with Brie is to provide a fully replicatable
(!) model -- i.e., everything of interest is a Brie object and therefore
replicates and persists as a Croquet object.  At that level, it is
independent of Tweak and is (conceptually) all 3D.

At this level, there is no "event handling", but there are "gestures"
that are also Brie objects. (Your 1.c.)

I'm not thinking of any support nor integration with EToys. UI elements
are displayed on an Interactor (think Portal). Those UI elements may be
there all the time (e.g., the Wicket buttons on DAS' Filters and Task
paper), or not. You can "wear" an interactor as your toplevel screen
frame. So whether there is a permanent menu, and where it appears, is up
to you. (Your 2).

You will certainly be able to build stuff in one place and carry it to
another. (Your 3) KidsFirst doesn't have Brie in it at all, but
KidsFirst is a baseline or minimal functional prototype of the user
functionality of Brie. (E.g., KidsFirst doesn't have interactors, but it
does have pervasive copy/cary/paste both within a world and between
worlds.  And it has wiki-like "built it while you use it in a shared
party and it will be there when you leave and come back.")

--
Howard Stearns
University of Wisconsin - Madison
Division of Information Technology
mailto:[hidden email]
voice:[phone]

[David Faught]

On 10/20/06, Howard Stearns <[hidden email]> wrote:
> David Faught wrote:

> > 3. Not having used it, I'm not sure how Brie fits with this, but I
> > speculate that having both a 2D and 3D UI would be useful.

> Brie has not yet been re-implemented on Hedgehog. It is wise in any
> distributed project to not be dependent on new work by other projects --
>  Brie, Tweak, or anything else. Croquet will certainly not be dependent
> on Brie. (Forever, I hope!)

Yes, I'm familiar with Error 33.  Ever hear of the Procedural Textures
project?   Or the MediaViewer web browser?  As long as the various
efforts in Croquet are deemed to be research, Error 33 certainly
applies.  But there are also potential synergies, as I attempted to
describe, that could be extremely powerful and elegant.  If there were
a single, more commercial Croquet project, rather than many individual
areas of research, the kind of duplication of effort that avoiding
Error 33 generates would be unthinkable.  Dependence on other parts of
the project would be the rule, in order to make the overall effort as
efficient as possible.  Large complicated interdependent projects are
well known in the commercial sector, as well as other areas.  Maybe
this is where Qwaq, Inc. is headed, I don't know.  Of course if
Croquet were a more commercial project, it would probably be
unavailable to people like me.

> That said, the intention with Brie is to provide a fully replicatable
> (!) model -- i.e., everything of interest is a Brie object and therefore
> replicates and persists as a Croquet object.  At that level, it is
> independent of Tweak and is (conceptually) all 3D.

This goes long with my limited understanding of Brie, where it would
be a primarily 3D UI, independent and yet complementary of the 2D
Tweak.  Or maybe an orthagonal alternative.

> I'm not thinking of any support nor integration with EToys. UI elements
> are displayed on an Interactor (think Portal). Those UI elements may be
> there all the time (e.g., the Wicket buttons on DAS' Filters and Task
> paper), or not. You can "wear" an interactor as your toplevel screen
> frame. So whether there is a permanent menu, and where it appears, is up
> to you. (Your 2).

Now you are back to talking about a 2D interface, I think.  I'm not
sure what "wearing" a 3D portal/interactor would be like. Multiple
overlaid avatar costumes?  Making use of Tweak would be an obvious
choice in the 2D area.  This is what I'm trying to get at.  Why
reinvent the wheel in the 2D interface when Tweak is already there
(almost)?  The only reason I mentioned EToys is that it is present in
Tweak and can be useful and fun in building interfaces.

> You will certainly be able to build stuff in one place and carry it to
> another. (Your 3) KidsFirst doesn't have Brie in it at all, but
> KidsFirst is a baseline or minimal functional prototype of the user
> functionality of Brie. (E.g., KidsFirst doesn't have interactors, but it
> does have pervasive copy/cary/paste both within a world and between
> worlds.  And it has wiki-like "built it while you use it in a shared
> party and it will be there when you leave and come back.")

Looking forward to it!

[Howard Stearns]

On Oct 21, 2006, at 11:02 AM, David Faught wrote:

> On 10/20/06, Howard Stearns <[hidden email]> wrote:
>
>> ...[In Brie,] UI elements
>> are displayed on an Interactor (think Portal). Those UI elements  
>> may be
>> there all the time (e.g., the Wicket buttons on DAS' Filters and Task
>> paper), or not. You can "wear" an interactor as your toplevel screen
>> frame. So whether there is a permanent menu, and where it appears,  
>> is up
>> to you. (Your 2).
>
> Now you are back to talking about a 2D interface, I think.  I'm not
> sure what "wearing" a 3D portal/interactor would be like. Multiple
> overlaid avatar costumes?  Making use of Tweak would be an obvious
> choice in the 2D area.  This is what I'm trying to get at.  Why
> reinvent the wheel in the 2D interface when Tweak is already there
> (almost)?  ...

Picture buttons and text floating in 3D.  But this is part of a  
particular UI, and not part of some other UI, so they are only  
visible when looking through a particular Interactor (in a TWindow).  
In some cases, the buttons and text might be associated with some  
object in the scene and so appear to be located in the same 3-space  
as the object they are associated with.  In other cases, it's more  
intuitive to think of the UI elements as part of the Interactor --  
sort of a heads-up display -- so the buttons and text look like they  
are pasted onto the TWindow like decals.  This is how the buttons  
appear in David's paper.

We're used to driving through TWindows that have TPortals to other  
spaces -- you enter the space.  What happens when you drive through a  
TWindow that contains an Interactor?  Well, imagine you're driving  
your car through a bunch of newspapers blowing around in the breeze:  
the newspapers stick to your windshield.  In the case of an  
Interactor, the button and text "decals" stick to your camera lens as  
you continue to drive around.

The Brie proof-of-concept built for Jasmine/Dormouse worked this way.

We were more interested in exploring the range of effects from  
"clearly pasted onto the Interactor," to "clearly in the space with  
the other objects," and everything in between.  Since everything of  
interest in Brie is (conceptually) an observable "concrete" object at  
all times, we used a single 3D object in all these different  
variations. However, in the case where the UI element is pasted flat  
on the camera lens, there is the opportunity to render using the  
higher fidelity 2D text (or other widgets) from the operating system  
(or from any other "lower" level 2D UI toolkit). We didn't do this  
visual optimization at that time.

Also, any element, whether part of the UI or not, could be arranged  
by direct manipulation while you were using it. (A right-click  
selected the object in a special minor mode -- "builder's mode" --  
that suppressed the usual click behaviors of buttons and menu items  
so that they could be dragged around.  Remember, everything was a  
"concrete" object.  Any other select or deselect (left-click on  
anything, including empty space)  cleared the minor mode.)  Since we  
were exploring the creation of compound objects in 3D, we created our  
UI elements the same way. However, there's a special case here, too,  
where a bunch of flat UI elements all happen to be in the same plane  
-- even if that plane is not perpendicular to the camera.  In such  
cases, we might have used a 2D toolkit (like Tweak) within a texture  
in 3-space.  But we didn't do that either.  I'm not sure whether it's  
worth shifting user models and mechanisms just to take advantage of  
that.

Is it worth unifying the models? Let Tweak do 3D?  Let Brie use  
existent Tweak players and costumes?  Maybe. Eventually.  But they  
are really entirely different things with very little overlap.  For  
now, I feel that it's best for each to evolve separately to best  
explore their own very different domain, purpose, and scope -- and  
not coincidentally to avoid being dependent on each other.  As more  
of Tweak becomes an integrated part of Croquet, the way Islands and  
signals are now, Brie will certainly take advantage of more.  Other  
parts, such as fields and annotations, are really a very different  
approach -- conflicting even -- to Brie's purposes.