SYSDOC xplans                   R.Evans April 1990

This file discusses an overall strategy for XPop development, and in
particular, what has be done for V14.


         CONTENTS - (Use <ENTER> g to access required sections)

 --  General Strategy
 --  Naming schemes
 --  Work to do


General Strategy
----------------

Version 14 Poplog will include an Xpop system which supports use of the
X11R4 toolkit by the applications programmer.

Issues arising from this statement:

1)  Should Xpop be a separate executable from the standard Pop11 image?
    The general development strategy has been to enhance Poplog in a
general way to support the needs of Xpop, among other things. We are now
pretty close to being able to put the whole of Xpop in libraries if we
want to. Its not clear to me whether we want to or not. In favour:

    a)  only one executable (pop11 as now) - simpler all round
    b)  xpop would necessarily be vanilla - no commitment to particular
        widgetsets etc.. So users can't really avoid doing at least one
        external_load anyway, and once established they'll probably have
        a saved image anyway. So the value in those terms of putting
        stuff into the system is marginal.

Against (ie in favour of keeping separate xpop as now)

    c)  system issues like sharing, demand paging etc. of core system
    d)  core routines are faster (probably not very important)
    e)  explicit control over interrupt handling etc.
    f)  access to syspop if we need to play dirty at all
    g)  continuity from existing relatively stable, bug-free xpop system
    h)  easier to accomodate other system dependancies on X (eg gc
        feedback)
    i)  external_load structure more straightforward

I'm not sure about the right final position on this. But I think that
for now we should keep an Xpop system but protect ourselves from
problems if we decide to alter the distribution by creating libraries,
possibly dummy ones, which users should load if they want to be
sure their code will be upwardly compatible.

2)  How far do we want R3/current Xpop compatibility? The publicity says
    its R4 and, having looked at the R4 intrinsics manual, there are
some R4 features which would be very useful from the outset, especially
for higher level libraries.  Furthermore, the details of various struct
definitions etc.  may have changed - this is transparent to C
programmers but since we can't directly read C header files, its an
issue for us.  So providing continued support for R3 even into V14 would
involve quite a lot of work for us.  So I think we should go straight
for R4. Some things would work under R3, others might be made to work
with a little additional user work. if the differences are substantial
enough, maybe a doc file should discuss it? Or maybe we should just junk
R3 (the current code hasn't actually juinked R2 yet!)

Compatibility with current XPop is tricky - there aren't many users yet
but we may not want to leave them completely stranded. BUT most of the
existing libraries are very rough and ready and should be scrapped. I
propose establishing a compatibility library into which any which can be
easily supported will be put, but probably only until v15.

3)  How much of the toolkit functionality should we provide? In V14 we
    aim only to support the application programmer, as opposed to the
widget writer.  That is, we only need routines for using pre-defined
widgetclass, not for constructing new ones.  So priority is given to
those routines.  Bearing in mind any additonal routines can be
externally loaded fairly easily, this should not be too great a problem,
even for people who want to do more esoteric things (eg applications
programmers sometimes need access to lower level routine to workaround
bugs in widgetclass definitions). Full support for widgetclass creation
can be thought about for V15.

The intrinsics manual itself draws this distinction, so its fairly easy
to pick out which routines we're talking about. Not all of these will be
simple imports, and indeed there will be one or two additional routines
provided by Poplog.

In addition to intrinsics support, of course, we are providing support
for some widgetsets, including our own set.  To date the pop widgetset
has had its own libraries to support. It might be cleaner and in the
long run easier to support the pop widgetset at the same level as any
other (eg the Motif set).

4)  What level of support should we provide? This is another tricky one.
    I have identified three levels of support one might consider:
        a)  basic procedures and structs as provided by lib external
        b)  basic procedures and data structures in a poplog idiom
        c)  high level libraries in a poplog idiom
(actually I think there's a fourth - the haphazard support provided by
the current xpop libraries comes somewhere between b and c). The phrase
'in a poplog idiom' is intended to mean 'in a fashion consistent with
ordinary use of poplog'. For pop11 this would mean manipulating ordinary
pop11 procedures and data structures, without knowing anything about C,
external procedures, data etc.. Similarly for prolog lisp or ML etc.
This seems conceptually higher level than lib external but not by much
(and probably only cos pop11 is higher level than C) - they're largely
just parallel ways of building on top of raw external procedures.

My view of what we should do runs like this: level (a) alone does not
provide enough support to qualify as a proper interface to the X
toolkit.  Core routines and data structures should be provided at level
(b) (as Xpop currently does) for pop11 at least.  Ideally I'd like to
see this level of support for all supported routines, but there are a
number of self contained groups of routines which are relatively
specialist and hence might be candidates for only level (a) support in
V14, with level (b) support coming later. This has the long-term drawback of
having two distinct styles of interface to some routines, but I firmly
believe level (b) support is required for the core, but we probably
don't have time to provide it across the board.

High level support would come in the form of libraries built upon the
lower layers.  I have been working on one such library for pop11
(described briefly below) and I believe it would provide adequate
support at this level for V14.  In addition there are a few
self-contained packages around which could also be distributed.

There remains the issue of support in languages other than pop11.

Widgetset support is similarly problematic.  Widgetsets tend to have
associated procedures and data structures and these have to be supported
at some level.  Again levels (b) and (c) seem preferable to me for any
set we really want to claim to support, with maybe some level (a) as a
convenience (to save you having to write them yourself).

NB: I've said nothing about the Xlib directory. This is deliberate. I
think we should view the Xlib libraries as something entirely separate,
just as, say a Nag interface would be. They are there and available but
not directly anything to do with Xpop.

Naming schemes
--------------

The following suggestion is a combination of what I was loosely doing in
the current Xpop, plus suggestions from various sources, plus new things
to accommodate new ideas.

Intrinisic routines
    C names (-XtCreateWidget- etc.) refer to Pop idiom, arg checking
    routines, or functionally equivalent routines loaded using lib
    external (or whatever). These provide a consistent 'safe' set
    of routines for low level development.

    The prefix fast_ (-fast_XtCreateWidget- etc.) is used for operationally
    equivalent but possibly non-arg checking variants (defined for all
    routines but not compelled to differ from slow version?).

    The prefix raw_ (-raw_XtCreateWidget-) is used for routines loaded
    using lib external (or whatever) but NOT functionally equivalent to
    the pop idiom routine. These may or may not be defined for a given routine.

Intrinsic Extensions
    Adopt the same convensions as for Intrinsic routines, except the
    prefix Xt (in the C name and all derivatives) becomes Xpt, thus
    -XptTryEvents- etc.

Pop widgetset class and procedure names (in C code)
    Adopt the prefix Xpw, thus -XpwFooWidgetClass-, -XpwBazObjectClass-
    etc. Similarly for associated routines.

Imported widgetclass and procedure names
    Use full widgetclass name for variable holding widgetclass rec (if
    needed). Use same conventions as intrinsics for procedures.

Associated Poplog identifiers
    Currently these are mostly 'xt_', which was intended to be viewed on
    a par with 'sys_', 'pop_' etc.. But I don't think we can draw  a
    fine enough distinction between these and the 'Xpt' routines, so I
    vote for using 'Xpt' exclusively. However I think we could retain
    the prefix 'xt_' for names of libraries (ie filenames rather than
    identifiers) which define Xt routines (maybe also 'xpt_ for pop
    extenstions?).

X toolkit constants
    Use their C names. Its not clear to me what our attitude to these
    should be - in particular the toolkit has a large number of string
    valued constants which are completely predictable (XtNcallback =
    "callback" etc.). Seems silly to define them all somewhere and fill
    up Pop with lots of useless symbols. I guess individual libraries
    can define the ones they use.


Work to do
----------

(NB: this bit is essentially my response to Tom's plan - hence the
     egocentric nature of some of the comments!  RE)

-   XptDescriptor apparatus

This is the support for the opaque handles on Xt objects discussed in
ref xtoolkit. Not difficult and indeed almost done - most of the effort
was deciding it was the right thing to do! As well as the datatype
itself, a couple of coercion and type checking routines.

-   Coercion routines for external_load

Externally loaded widgetclasses will be coerced to XptDescriptors. Once
the routines for this are available, xpop won't need a separate
extern_load.p, and the master version will start working again.

-  sys_async_input.p and sys_input_waiting.p into the main sources

These files in the current xpop source should really be in the master
source. I've tidied sys_input_waiting.p but not tested it, but I believe
its ready to go. I think sys_async_input.p can just go as is (no
testing beyond what its had in xpop thse past two years!!). SOMEONE ELSE
COULD DO THIS

-   Datatype coercion routines for
    some fast_ procedures.

Hiding in here is callback (coercion for pop procedures passed as
arguments to C code). Not problematic I don't think, but has a few
knock-on effects (eg destroy callbacks become much easier...) which need
chasing. NB: public coercion routines needed in some cases (for
externally loaded Xt routines).

-   throwing away obsolete files

most of the 'old' new etxernal stuff (extern_shadow, extern_object etc.)
is now obsolete. I think the core system only uses it in one place
(callback), and won't do even that when 'real' callback is in. Then in
principal it can probably just disappear, but there are libraries (eg
xt_shadow) which use it and need checking out. Probably needs me to give
a helping hand, if not do it.

-   fast_ (non-checking) versions of xt routines

Fast routines do no checking so its just a case of necessary coercions
to args and results. The ones in the system either have very
non-trivial coercions or are just there 'because they should be'.
There are 64, but probably only about 5 with any real substance! The
ones in libraries (79), will be externally loaded with
-external_require- and should be entiresly straightforward (SOMEONE
ELSE)

-   Type-checking routines

The checking version of routines use a common set of type-checking
routines. Not very many, but I'll do it cos I know what I want and it
will be quicker!

-   Checking versions of xt routines    (143)
    (in libraries)

Checking routines are just totally boring wrappers around non checking
routines. Someone else can do this once we've established the basic
form.


-   Datatype libraries
        (interfaces to C datatypes)     (45)

Not really 45 distinct cases I don't think. Maybe a little thought
needed here to pin down exactly how to do this. Then some help
appreciated to actually do it.

-   Varargs mechanism (library)

Little library, discussed already with some. Quicker to do than to
explain again (unless someone who knows beats me to it).


-   Long term wait handling

I fear this one's for me, though I'd rather it wasn't. I've never
managed to explain it properly to anyone, but I know what has to be done
now.

    Currently we are servicing X events using io interrupts.  This
performs reasonably well (except on systems without sigio, where we
simulate it with a polling timer!), but denies us a crucial bit of X
toolkit functionality - namely the timer functions.  This is because the
timer operation is predicated on the fact that the application spends
all its long term waits, indeed most of its time, inside the toolkit in
in a -select- with a timeout set for the next timer event. But Poplog
currently doesn't. It might spend a substantial time processing (not
sure how to handle toolkit timers during this - suggestions?), but it
generally spends time in long term waits, ie interactive reads,
hibernation, or waiting for children. We need to modify these waits to
make them wait inside the toolkit's select, instead of where they
thought they would wait(!). I think this is too complicated to explain
here...

-   Fork problem

Modification to -sysfork- and -sysvfork- so that they behave with
respect to SIGIO at least. Currently when you fork you can end up with
two processes responding to sigio on the same file descriptors (eg both
servicing the SAME X connection!). Not clear what the general solution
is, an easy patch is to redefine io_interrupt to identfn inside sysfork
BEFORE forking, and only restore it in the parent. (NB: can't do this
with dlocal or the child will restore too?) SOMEONE ELSE COULD DO THIS.

-   Tidying up Poplog "garbage widget"

The main thing here is deciding how to provide a gc-feedback hook. The
current one is a little too hardwired.


(III) Software organisation


(IV) Documentation


(V) High-level tasks

-   define:widget package
    (high-level interface to external_load for
        widgets, widget-classes etc.)

Yes I'd like to spend some time on this if possible. Its really nearly
there! Probably get an athena widget interface (at least) for free, cos
that's what I'm using to test it with.


-   Flavours interface

Not me at all!

-   lib rc_graphic may need some work to fit in with the Pop idiom

(Probably not much work)


-   Widget set interfaces:
    o   Motif
    o   DECwindows
    o   Xview/OpenWindows
    o   Athena
    o   Poplog widgets


There are several issues here - interfaces to what, for a start
(intrinsics, define:widget, C idiom stuff...)
