Status of Xpop 5/8/90           R.Evans

The emerging new xpop implementation lives in $poptestmaster/P.xpop and
can be accessed by specifying popversion 'xpop'. This system is an
overlay of the sun4 testmaster system ($poptestmaster/M.sun4, popversion
'master' on a sun4), which is in turn an overlay of the pop master tree.
Currently the master still contains 13.82 xpop, and P.xpop overlays all
the new facilities. It is intended as a site for drawing together all
the strands of xpop development and for testing by cooperative
knowledgeable people, prior to installation into the master. This
document describes briefly what is in this system, what is still
missing, and what needs to be done. Its largely a brain dump (this is an
excuse for it not being properly structured!) but hopefully some of the
text here might be useful in real documentation. Items are numbered for
ease of reference only - the ordering signifies nothing more than the
order in which things came into my head.

1) Documentation: Some of the files previously found on
~rogere/poplog/Xpop are now to be found in the ref/help directories as
skeletal ref/help files.  These are REF xtoolkit REF xt_procs and HELP
xt_compat.  Clearly these need expansion and completion.  Old xpop help
files are still in the help directory and are probably obsolete, though
its so long since I looked at them I can't be sure - there may be some
useful words there still.  In addition adrian has started roughing up
ref files for the libraries he's been working on but so far none of
adrian's stuff has been incorporated into the system.

2) REF xtoolkit: unfinished - read 'hardly started' really, since most
of what's there is pretty inaccurate or unsatisfactory. Some of what's
below might go in here.

3) REF xt_procs: a list of all the procedures and what libraries they
are in. All the procedures listed are intended to be in xpop, and indeed
many of the are now. But there are quite a number of other procedures
which are not in the ref file yet - mostly Xpt procedures I hadn't
thought of when that list was made. It would be useful for someone to
run over the $Xpop/src files and compare what's exported with what ref
xt_procs says is exported (there will be omissions both ways).

4) HELP xt_compat: intended to be a help file for lib xt_compat, the
library which provides access tot he 'old xpop' compatibility stuff.
Currently its a rough list of facilities which have disappeared in the
new xpop and possible workarounds. Its far from complete even as that,
but probably important to anyone trying to use xpop.

5) new xpop executable: this contains fast versions of the routines
marked with a '+' in ref xt_procs, plus a number of additional Xpt
routines not documented anywhere yet.  It introduces two new datatypes:
XptDescriptors and XptProcedures and uses them exclusively for its
representation of toolkit objects.  The old widget and widgetclass keys
and all the external_shadow, external_coerce, external_object etc.
stuff have gone.  In addition many of the previously core toolkit
routines (eg XtInitialize, XtCreateApplicationShell) have gone, because
they are no longer core - they will be defined instead in lib
xt_r2compat, along with the other X11R2 compatibility routines.  Note
however that this library does not exist yet.  I don't know off-hand
whether Adrian has done any work on it.  It might be worth getting it
together (see item 6) because it defines a lot of routines people are
probably using - although they should in the long term stop using them.
In particular the mkxpop11 (etc) scripts uses such routines and for this
reason its not currently possible to build xpop11 - see item 7.

6) lib xt_r2compat: implementing this might be slightly tricky.  In the
C library they are implemented in terms of new core routines in a
relatively straightforward fashion.  In principle we could try and
implement them ourselves as pop routines using the corresponding pop
versions of the core routines, but without a precise functional
specification this is a little dangerous.  The simplest thing to do is
just externally load them, but then we have to make sure that the
associated poplog work gets done correctly.  For example XtInitialize
calls XtToolkitInitialize.  The pop version of XtToolkitInitialize does
some pop-specific initialization, and so we must make sure that
XtInitialize does this too.  For this reason the system provides the
routine XptToolkitInitialize which does just the pop-specific part, and
can be called by the XtInitialize wrapper.  There may be other similar
cases we need to provide additional support for too.  A different kind
of example: the C compat routines create an application context but keep
it private, since anyone using these routines is (presumably) not
interested in application contexts).  But some of the xpop functionality
is only available via application contexts (XptAppTryEvents,
asynchronous event handling etc.).  Rather than provide r2compat
versions of these routines, I suggest lib xt_r2compat pulls out the
appcon created by XtIntialize (using XtWidgetToApplicationContext on the
widget returned) and puts it in a pop global variable, say
XptDefaultAppContext.

7) xpop11, xprolog etc: As mentioned in (5), these saved images have not
been built. The make scripts as they stand require lib xt_r2compat, but
this is not the real reason for not building them - I could just change
the scripts to use r4 routines instead. The real reason is a design one
- these scripts are not as NECESSARY now as they used to be. Under old
xpop there was this irritating asymmetry whereby the first widget you
created took completely different args from all the others. One of the
principle reasons for the xpop11 image was in fact to create this first
widget sensibly, so users saw a more uniform interface to widget
creation. In r4 xpop the asymmetry has disappeared, or rather the
information which used to be passed to the first widget created is now
passed independently (in XtOpenDisplay etc.) from actual widget
creation. So we don't need these images any more for that reason. BUT we
have discussed other reasons for having saved images - to facilitate
nice subsystem-specific x startup. This is discussed more fully  under
(8).

8) Executables and images: xpop is still a separate executable with
exactly the same directory structure etc. as before. So the xpop
executable, stb, links etc. all live in $Xpop/bin. I reckon it was more
or less agreed to have a single poplog executable including x. This
probably means it should live in $popsys and $Xpop/bin can disappear
completely (also all the popexlinkbase checks, in lib popxlib and compat
lib xt_check_exlinkbase, become redundant too). My favourite picture of
the world is still more or less as described im my recent mail message.
Have a base image which looks for special x startup args and puts them
in popxtarglist if found but doesn't actually do an x startup. Other
things can then check for popxtarglist non-false to know they are
starting up under x. In particular, subsystem_initialise (or whatever
its called) can look for this and run subsystem-specific x startup stuff
(eg a new procedure in the subsystem table). So we have saved images for
pop11, prolog, lisp, ml and any required combinations, each doing their
x startup if and only if they are started under x. The xpop11, xprolog
etc are redundant. Most of this structure can be established without any
knowledge of x (use of which is localised to the new procedures in the
subsystem table), which might make it easier to find someone to do it!

9) XptDescriptors: XptDescriptors are one of the two new data types
added to poplog for the x interface (the other is XptProcedures,
discussed below). To the user they are opaque handles on toolkit objects
like widgets widgetclasses application contexts etc. they have no
accessible internal structure, although the toolkit itself provides
access routines for some fields (eg the procedure XtWidgetClass). Their
only apparent characteristics are their types, their props and their
external values. The type is distinct from the key - descriptors all
have the same key - it is simply a word (eg "Widget", "Widgetclass) and
is stored in the external_ptr_props field. It is used for printing the
descriptor, and is accessible using the procedure -XptDataType-. The
props is also used for printing the descriptor and is typically its name
etc. If a props is present, the external_ptr_props field is a pair whose
front is the type and whose back is the props. The external value is in
the external_ptr field. (Question for johng - the file extern_generic.p
is older than your stuff and apopears to duplicate newer functionality.
Should it disappear? If so, are there generic routines for pulling
out/putting back the XP_PTR field of any externalclass object as an
external_ptr record - I don't recall any).

However XptDescriptors do in fact have internal structure, used to
maintain gc-related dependencies among descriptors. Basically they have
a single additional full field containing dependent data (either the
data itself, or a vector in the case of many dependents). (You can look
at these with fast_subscrv(2,desc)). They are managed by the fast_
routines themselves and normally can (and should) be ignored. But
knowledge of what the links are may be useful, so you know how much you
have to keep hold of. In brief the descriptors currently maintain
pointers to related objects as follows:

    widgets - class, parent widget or appcontext, children,
              callback procedures and their args
    widgetclasses - superclass
    appcontexts - displays, input ID's, timer ID's
    displays - appcontext, device


10) XptProcedures: XptProcedures are a new class of external procedure.
They are structurally and functionally identical to external_procedures,
the only difference is that they have a different class_apply, the
procedure -XptProcedure_apply-. They can be created with the routine
XptImportProcedure which is an 'implicit access procedure' (as in ref
external_data). It takes an external ptr record and returns an
XptProcedure, its updater does nothing. The reason for needing
XptProcedures is as follows. When an external (C) intrinsic procedure is
running, it might decide to talk to the server. When the server replies,
it triggers a pop interrupt which now (post 13.82) gets trapped before
the intrisics get a look in. Pop gaily attempts to handle the server
response by invoking the intrisics, the result being that the intrisics
get called recursively and get rather upset. SO, XptProcedures invoke
the underlying c routine but first locally set a flag to tell pop not to
handle any X events on interrupt. They should be used for any external
X procedure which might generate a call to the server. Generally, they
should be used for all external x procedures. So the externally loaded
ones should use them, and ideally the lib xlib stuff should too -
hopefully not too hard to achieve.

Part of Tom's message of 9/8/90
-------------------------------

The general areas still needing work are (as dictated by Roger
last Sunday):

-   Input, Timer and Workproc events.
    Need to add things like XtAppAddInput, XtAppAddTimer etc.
    (see ~rogere/poplog/Xpop/xt_procs)
    This will involve creating new XptDescriptor types for input_id,
    timer_id etc.
    It will also need support for event structures.

-   Some coercion routines
    Adrian - do you have a list of the coercions you have needed so far?

-   Actions (also needs support for event structures.)

-   Long-term wait problem (this is probably best left till Roger gets
    back).

-   Various other data-structure support
    e.g. Arglists, Args etc. (these are currently done using vectors).
    These should probably all be externam_ptr types; in this case they
    can use their props field to aid checking as XptDescriptors do.
    (see ~rogere/poplog/Xpop/xt_procs)

-   Replacements for xt_compat stuff; in particular:

    xt_value    becomes XptValue - uses Arglist support
                needs to know resource types to do coercion?

    xt_load_wc  becomes XptLoadWidgetClass
                The main complication here is that we need a mechanism
                (library) to maintain the mapping from widget-class
                names to widget-class objects, given that different
                widget-sets may contain widget-classes of the same name.
                In other words we need a widget-set support library.
                Other than this, xt_load_wc is just an interface to
                external_do_load.

    xt_arglist  becomes:
                XptArglist( [ [name1 val1] [name2 val2] ...] )
                    -> Arglist (vectore?) -> Nargs
                and
                XptVaArglist - the varargs version - dumps
                    name/value pairs on the stack and then Nargs.

-   Startup, executables, saved images etc.
    This needs to be organised so that we have a single base binary
    (called base_poplog ?) which includes the basic xpop stuff,
    and saved images for each language subsystem etc which do both
    normal and if appropriate X startup actions.
    The subsystem table should contain a special slot for X subsystem
    startup.

-   Garbage widget
    -   needs rewriting for the new regime.
    Jon - do we have anything in the Poplog widget-set which would
    provide us with the single-button functionality we need here?
