REF POPC                                            John Gibson Dec 1994

        COPYRIGHT University of Sussex 1994. All Rights Reserved.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
<<<<<<<<<<<<<<<<<<<                              >>>>>>>>>>>>>>>>>>>>>>>
<<<<<<<<<<<<<<<<<<<     POP-11 OBJECT-MODULE     >>>>>>>>>>>>>>>>>>>>>>>
<<<<<<<<<<<<<<<<<<<         COMPILATION          >>>>>>>>>>>>>>>>>>>>>>>
<<<<<<<<<<<<<<<<<<<                              >>>>>>>>>>>>>>>>>>>>>>>
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

This file is a reference manual for the Pop-11 object-module compilation
and linking  system.  It describes  in  detail Popc,  the  object-module
compiler  for  Pop-11  programs,  and  also  discusses  the   associated
utilities Poplink and  Poplibr. It  should be read  in conjunction  with
HELP * POPC, which is a user-guide  for the three programs (listing  all
their available options, etc).

         CONTENTS - (Use <ENTER> g to access required sections)

  1   Overview

  2   Use of +strict compile_mode
      2.1   Effect on Declarations
      2.2   Effect on Writeability of Structures

  3   How Popc Compiles a File
      3.1   Compile Phase
            ... Shadow Values for Permanent Identifiers
            ... Use of Libraries
            ... Testing for Popc
      3.2   Value Generation Phase
            ... Structures Generated by Poplink
            ... Constants vs Variables
            ... Value-Generation Errors
            ... Restriction on Constant Values

  4   Changing Pop-11 Files to Work with Popc
      4.1   Permanent Identifiers: Declarations & References
            ... Poplog System Identifiers and popsyslist
            ... Multi-File Programs
            ... Record and Vector Classes (defclass-declare)
            ... Other Declarations (declare_incremental and declare_updater)
            ... -include Option
      4.2   Use of uses-now Instead of uses
      4.3   Restrictions on Compile-Time Executed Code
      4.4   Syntax, Macro & Other Compile-Time Executed Procedures
            ... General
            ... Permanent
            ... Lexical

  5   Additional Syntax For Popc
      5.1   Incremental Identifiers
            ... Syntax of declare_incremental
            ... Assignment Code: Basic Initialisation
            ... Lists
            ... Procedures
            ... Properties
      5.2   Building W-Libraries
      5.3   Other

  6   Program Linking
      6.1   Exporting of Identifiers and Sections
      6.2   Run-Time Autoloading of Identifiers
      6.3   Weakly-Referenced Identifiers
      6.4   testdef
      6.5   Entry Procedure
      6.6   Embedding Pop in C
            ... Initialising Pop
            ... Unix Signals

  7   Miscellaneous
      7.1   Allocation of External Symbols




-----------
1  Overview
-----------

In conventional object-module (or 'batch') compilation, each source file
of program  code  is compiled  to  a corresponding  object  file,  which
contains a translation  of the program  into host-machine  instructions;
the set of  object files is  then linked by  the operating system  (O/S)
linker to produce an executable image file. This usually also results in
the incorporation of one or library object modules, which are  extracted
automatically by the linker on demand.

With Popc the situation  is similar, but with  one major difference:  in
order to  handle the  linking of  global data  structures within  a  Pop
program, a second file is produced for each source file compiled.  These
extra files  (called w-files,  from their  .w extension)  are in  binary
format, and describe the  usage by their  corresponding object files  of
such things as Pop identifiers, quoted words, and other data.

Thus a w-file together with its associated object file can be thought of
as a composite 'Pop  object file'. (Ideally, the  two would actually  be
combined  into  a  single  file,   but  O/S  considerations  make   this
impossible.) Similarly, just as an O/S library archive can be built from
a set of object files (e.g. with  the ar utility in Unix, or library  in
VMS), so  Poplibr  can  be  used to  create  and  maintain  'Pop  object
libraries' from  sets  of  w-  and object-  files,  where  each  library
consists of a w-library file plus an ordinary O/S object library.

All the normal  Poplog system  libraries are  compiled to  corresponding
w+object libraries; these reside in the directory $popobjlib.

Another respect in which Popc differs from conventional object compilers
is that library  archives are used  not only at  link-time, but also  at
compile-time, to extract  declarations for library  identifiers used  by
the files being compiled. This makes it unnecessary for programs to have
to provide declarations for system identifiers. For each file  compiled,
the  libraries  so  used  are  remembered  on  the  output  w-file,  and
automatically supplied to the linking process.

Linking of a program is performed by the Pop linker, Poplink. This takes
as input the set of all w-files for the program, automatically  extracts
any  Pop  system  library  modules  used,  and  performs  the  necessary
global-structure  linking.  The  linked   global  data  structures   are
generated as some  additional object files,  which Poplink then  passes,
together  with   all  object   files/libraries  corresponding   to   the
w-files/libraries used, as input to the O/S linker.

The diagram below illustrates the compilation and linking process  for a
program consisting of two source files 'foo.p' and 'baz.p'. In this, the
dotted lines represent the usage  of w-libraries by Popc for  extracting
identifier declarations,  and  add.o represents  the  additional  object
files produced by Poplink:

             Popc              Poplink            O/S linker
                    ---------
   ---------        | foo.w | -----
   | foo.p | --O--> |-------|     |
   ---------   |    | foo.o | ----+--------------------
               |    ---------     |                   |
               |    ---------     |                   |
   ---------   |    | baz.w | ----|                   |
   | baz.p | --O--> |-------|     |                   |
   ---------   |    | baz.o | ----+-------------------|
               |    ---------     |                   |
               |                  |     ---------     |     ----------
               |                  O---> | add.o | ----O---> | image  |
               |                  |     ---------     |     ----------
               |    ---------     |                   |
   w-library   --<- | lib.w | -----                   |
    (.wlb)          |  ...  |                         |
                    |-------|                         |
   o-library        | lib.o | -------------------------
    (.olb)          |  ...  |
                    ---------

As  the  description  above  implies,   Poplink  runs  the  O/S   linker
automatically. In addition (as with Unix C compilers), Popc runs Poplink
automatically for you (unless you specify otherwise with the -c option).
Thus Popc is used for both compilation and linking.




------------------------------
2  Use of +strict compile_mode
------------------------------

2.1  Effect on Declarations
---------------------------
In compiling  Pop-11 programs  with Popc,  you are  strongly advised  to
follow Poplog system and library source code by using

        compile_mode :pop11 +strict;

in all Pop-11 source files (see HELP * COMPILE_MODE).

First, this disallows  the use  of 'old-style' Pop-11  where the  formal
argument   and   result   identifiers    of   procedures   default    to
dynamically-local vars,  and other  procedure locals  are declared  with
vars. It forces all procedure formals to be declared explicitly (usually
as lvars).

For example, without +strict a procedure such as

        define old_style(x, y, z);
            vars u, v, w;
            ...
        enddefine;

is allowed; all of its locals (x,  y, z, u, v, w) are  dynamically-local
permanent variables, and it is thus the same as

        vars x, y, z, u, v, w;

        define old_style(x, y, z);
            dlocal x, y, z, u, v, w;
            ...
        enddefine;

Compared to a  similar procedure using  lvars, the above  is (a)  highly
non-modular, (b)  generates  much  less efficient  code,  and  (c)  in a
batch-compiled environment will lead to much slower linking times (since
every permanent identifier corresponds to a global external symbol).

Second,  +strict  establishes  the  default  declaration  for  top-level
defines as 'constant procedure' (note that without +strict, the form

        define fred();
            ...
        enddefine;

declares fred as  an untyped variable).  With Popc, it  is important  to
have  as  many  procedures   as  possible  declared  as   procedure-type
constants, both for efficiency reasons (a procedure identifier generates
much more efficient code), and because of restrictions on  execute-level
cross-file references to variables (described below).

+strict also disallows the vars default for nested defines, forcing them
to be declared explicitly. (Except  for the occasional dlocal or  lvars,
all nested procedures should be lconstant.)

Finally, +strict causes  all permanent identifiers  to be  automatically
declared global.


2.2  Effect on Writeability of Structures
-----------------------------------------
As   described   by   Creating   Layered   Systems   in    REF * SYSTEM,
'non-specialised' data structures  such as pairs,  lists, strings,  full
vectors, etc,  can  have a  general  writeable or  nonwriteable  default
specified for them  when creating  a saved  image with  sys_lock_system.
There is also a separate default for (user-created) closures.

The corresponding behaviour for Popc  is controlled by the  compile_mode
:popc options  wrdflt and  wrclos (see  HELP * COMPILE_MODE). Since  the
composite +strict mode includes  both -wrdflt and  -wrclos for Popc,  an
additional effect of using

        compile_mode :pop11 +strict;

in programs is to force a nonwriteable default for all such  structures,
including closures. That is,  Popc will compile execute-level  instances
of these structures into nonwriteable areas of memory.

For batch-compiled  programs,  this  has  similar  advantages  to  those
described in Creating Layered Systems: reduced garbage collection  time,
greater shareability of the executable image, etc. But it does mean  you
will need to apply  writeable to any structures  of this type that  your
program requires to update.

Alternatively, you can add another compile_mode statement such as

        compile_mode :popc +wrdflt +wrclos;

after the  one for  +strict  (note that  the  two statements  cannot  be
directly combined, since  they specify contradictory  values for  wrdflt
and wrclos).




---------------------------
3  How Popc Compiles a File
---------------------------

3.1  Compile Phase
------------------
Popc compiles a file by calling the ordinary Pop-11 compiler, but  after
setting the  system  variable  pop_pas_mode to  the  word  "popc".  This
variable having a true value causes  the Poplog Virtual Machine (VM)  to
divert VM code-planting and permanent identifier declarations to various
hook procedures, for which Popc provides appropriate definitions.

This is 'Popc  compile mode': all  compilation in this  mode results  in
procedures containing Popc's own representation of the code, which  will
be interpreted  if necessary,  i.e. when  code is  actually executed  at
compile-time.


...  Shadow Values for Permanent Identifiers
--------------------------------------------
In particular,  Popc-compiled code  traps all  assignments to  permanent
identifiers, and stores them  on a global list.  This list is the  basic
target of Popc's  operation, and is  the only source  for generation  of
output object code.

However, the  'real'  identifiers  in  the  underlying  system  are  not
actually assigned to.  This is  necessary because  Popc can  be used  to
compile source code that makes assignments to identifiers actually being
used by the underlying Pop-11 system (in which Popc is running); if  the
real identifiers  were  assigned to,  the  operation of  the  underlying
system would be changed or corrupted.

The point is that  the purpose of a  Popc-compiled assignment is to  set
the value of an identifier in the  output object file (and hence in  the
linked executable image), not to set  its value in the Pop-11 system  in
which Popc is running.

On the other hand, a Popc-compiled  access to an identifier returns  any
Popc value assigned to that identifier  (if it has one -- otherwise  the
current value of the real identifier is returned).

In summary, Popc maintains shadow values for permanent identifiers  that
have been assigned to.  The Popc shadow value  of an identifier is  also
accessed and updated by sys_current_val when Popc mode is in force.


...  Use of Libraries
---------------------
Not all  code  compiled  during Popc's  operation  will  necessarily  be
compiled in Popc  mode. When  compilation of library  files is  invoked,
either by autoloading or with uses-now,  Popc mode is turned off.  (Note
that an ordinary uses does not compile libraries -- see Use of  uses-now
Instead of uses below.) Popc mode can also be explicitly turned off with
the syntax construct normal_compile ... end_normal_compile.

The principal  problem that  arises  with ordinary-mode  compilation  of
libraries is that Popc runs in a Pop-11 system containing neither  the X
nor Ved parts of Poplog. Attempting to compile libraries that use  these
facilities will  generally lead  to  multiple 'DECLARING  VARIABLE  ...'
messages, or  mishaps.  It is  thus  essential to  restrict  loading  of
libraries to those  situations where  the code is  actually required  at
compile-time.

On the one  hand, if a  library is explicitly  specified with  uses-now,
Popc has no choice  but to compile it  (it is up to  you to ensure  that
only suitable libraries are so specified).

On the  other  hand,  Popc  will  not  attempt  to  autoload  a  library
identifier unless absolutely necessary. Moreover, in certain cases  Popc
limits the library directories that are searched for autoloadable files,
as follows:

As in the normal interactive system, autoloading will happen only for an
identifier that is strongly referenced but not declared (or only  weakly
declared). Unlike  the  normal  system, however,  autoloading  will  not
generally happen if a  declaration for the  identifier can be  extracted
from a w-library. The exceptions to this are where

    (a) the identifier is a syntax word or macro (and therefore needs to
        be executed immediately);

    (b) the  value   of  the   identifier  is   actually  accessed   (by
        execute-level code).

In case (a),  the assumption is  made that (being  syntax or macro)  the
library will not contain  any 'hard' references to  Ved or X  facilities
(the section  Syntax, Macro  &  Other Compile-Time  Executed  Procedures
below discusses  this  in  more  detail).  Therefore  the  full  set  of
popautolist directories  are searched,  as normal.  The case  where  the
identifier is completely undeclared has  to be treated similarly  (since
it might be syntax or macro).

However, in case (b) (that  is, where a non-syntax/macro declaration  is
available), it is in many circumstances good enough to simply allow  the
undef value  of the  identifier to  be accessed,  without attempting  to
autoload it. (The reason  for this is explained  more fully under  Value
Generation Phase  below; basically,  providing the  undef value  is  not
executed or otherwise  used in  some computation, it  will be  correctly
interpreted during generation of output code.)

Hence in this case, only a restricted set of directories  ($popautolib +
$poplocalauto) are searched, instead of the full popautolist. While this
prevents many potentially problematic (i.e. Ved and X) identifiers being
autoloaded unnecessarily, it does means that uses-now will be needed for
a small number of  library identifiers which  are not syntax/macro,  but
which nevertheless can be successfully used at execute-level with  Popc.
(The X procedures XptSpecifyResourceInfo and XptSpecifyPopValueTypes are
examples of this.)


...  Testing for Popc
---------------------
The definition

        lconstant macro POPC_COMPILING = true;

is automatically  added to  the  beginning of  every source  file.  This
enables a file to use

        #_IF DEF POPC_COMPILING
            ...
        #_ENDIF


etc, to test  for being compiled  with Popc  (either in or  out of  Popc
mode).

Occasionally, some syntax and macro code may need to behave  differently
when being run in Popc compile mode. In these cases the test

        if pop_pas_mode == "popc" then
            ...
        endif

is appropriate.



3.2  Value Generation Phase
---------------------------
After compiling a file,  Popc works through the  list of assignments  to
permanent identifiers, and  generates object (assembler)  code for  each
value assigned.  The output  assembler  file is  then translated  to  an
object file with the operating-system assembler.

Permanent  assignments  are  the   only  source  for  code   generation.
Assignments to lexical identifiers  will be ignored  unless they are  in
some way referenced by structure(s) assigned to permanent identifier(s).


...  Structures Generated by Poplink
------------------------------------
In a Pop-11 program, certain data  structures must be unique across  the
system as a whole: for example,  the dictionary word "abcd" must  always
be the same object, wherever referred to. Generation of such  structures
cannot be done by  individual object modules, since  there is no way  of
deciding that one module rather than another should do the job.

Thus (as described in Overview above), a linking process is required, to
generate these structures centrally: this is the purpose of w-files  and
Poplink. The structures which are always generated by Poplink are:

    #   dictionary words
    #   permanent identifiers
    #   permanent word-identifiers
    #   anonymous undef values (having a false undefword)
    #   the unique objects in false, true, and nil

(Poplink also generates other  structures when these  are the values  of
incremental identifiers. See Incremental Identifiers below.)


...  Constants vs Variables
---------------------------
A constant identifier can  only be assigned a  value in one file,  and a
(strongly-referenced) constant must always have a value. Poplink  treats
a violation of either of these conditions as an error.

On the other  hand, Poplink allows  some leeway in  assigning values  to
vars. For example,  an assignment for  a variable in  an object file  is
allowed to override  one in a  library module; also,  a variable may  be
unassigned (and default to an undef value).

These (and  other)  considerations  mean that,  while  the  Popc/Poplink
system supports inter-file  values for  constants, it  does not  support
inter-file values for vars. In practice this means the following:

    Except for the  Poplink-generated structures  listed above,  the
    only compile-time  cross-file  references  allowed  are  to  the
    values  of  constants.   Thus  the  values   of  constants   are
    compile-time  accessible  between  files,   but  the  values  of
    variables are not.

Note that  this  restriction  does not  concern  run-time  accessing  of
variable values; such accesses (i.e. in procedure code), will always  be
to variable identifiers (Poplink-generated structures). It concerns only
Popc's generation  of  object code  from  the values  in  the  permanent
assignment list -- that is, 'static' references between files  resulting
from compile-time executed code.


...  Value-Generation Errors
----------------------------
To use Popc  successfully, you need  to know how  the above  restriction
operates in practice, and how it relates to errors of the form

        NON-GENERATABLE VALUE (optional-qualifying-message)
            INVOLVING: <object>

which may occur during Popc's value generation phase.

When presented with a ("full")  location containing a value to generate,
Popc does one of three things:

    (a) If the  value  is  a  simple object  (integer  or  decimal),  it
        generates the appropriate bit pattern in the location;

    (b) If the value is a Poplink-generated structure, OR if there  is a
        constant identifier in the dictionary whose idval is the object,
        it assumes that the structure is generated elsewhere, and merely
        outputs a reference  to it (that  is, outputs its  corresponding
        external symbol);

    (c) Otherwise, it  actually  generates  the structure  --  that  is,
        allocates  memory  for  it  in  the  current  output  file  (and
        recursively generates its sub-structures, etc).

Note that (b) includes the case where the object is a named undef record
contained in some constant  identifier in the  dictionary. Such a  value
may have resulted by  accessing the value of  a constant which has  been
declared but not assigned  to: this is the  way in which constants  from
other files are usually dealt with (hence Popc does not need to know the
actual value of the constant).

For example, if  baz is a  constant procedure defined  in another  file,
then the execute-level code

        constant procedure baz;

        define foo = baz(% 1, 2, 3 %) enddefine;

will result  in the  Popc shadow-value  for foo  being a  closure  whose
pdpart is <undef  baz>; this will  be mapped correctly  to the  external
symbol for the constant baz.

On the other  hand (because  of the restriction  on variables  described
above), Popc will refuse  to generate a named  undef record that is  the
idval of  any variable  in  the dictionary.  Thus  if baz  were  instead
declared

        vars procedure baz;

the error

        NON-GENERATABLE VALUE (undef value of perm variable)
            INVOLVING: <undef baz>

would result.

For other structures (and where no suitable constant can be found), case
(c) comes into  play, i.e.  Popc actually generates  the structure.  The
following kinds of structure cannot be generated, however:

    (1) Proper procedures that have not been compiled in Popc mode;

    (2) Currently, instances of  shadowclass structures  (but note  that
        shadowkeys are OK);

    (3) Currently, XptDescriptor structures;

    (4) System devices (user devices are OK);

    (5) Processes.

For example, (1) will give an error like

        NON-GENERATABLE VALUE (procedure not compiled in popc mode)
            INVOLVING: <procedure>

In general, an error of the form

        NON-GENERATABLE VALUE (optional-qualifying-message)
            INVOLVING: <object>

means that the object is not  a generatable structure, and moreover,  is
not the value of any constant in the dictionary. Note that a value  does
not have to be a generatable structure, providing it can be mapped  to a
constant; this enables a structure such as

        vars list = [% hd, tl, lmember %];

to be generated,  even though the  accesses to hd,  tl and lmember  will
result in  the  list  containing the  actual  Pop-11  system  procedures
(which, of course, are not Popc-mode compiled). Compare, however

        vars list = [% hd, tl, member %];

This will give

        NON-GENERATABLE VALUE (procedure not compiled in popc mode)
            INVOLVING: <procedure member>

because the procedure member is not a constant (unfortunately).


...  Restriction on Constant Values
-----------------------------------
Another restriction not covered by the last section is the following:

    You cannot assign a constant from another file to be the  direct
    value of a constant in the current file.

For example, if baz  is defined in  a file other  than the current,  the
following is invalid:

        constant procedure baz;

        constant procedure foo = baz;

and will result in the error

        INVALID INITIALISATION FOR PERM CONSTANT foo

This only  concerns setting  foo  directly equal  to  baz; there  is  no
problem when baz is  some sub-element of a  structure being assigned  to
foo (that is, when the reference to baz is in a memory location).  There
is also no problem when setting a variable equal to a constant, e.g.

        vars procedure foo = baz;

is correct.  (Once again,  the reference  to  baz will  be in  a  memory
location.)

You can,  however,  define  a constant  to  have  any  Poplink-generated
structure as its value. E.g.

        constant
            XXXX    = true,
            a_word  = "abcd",
            baz_id  = ident baz,
            a_list  = [];

are all valid.

(The above restriction  arises because  of the inability  of O/S  object
code to set the value of one external symbol equal to another, where the
other is not defined in the current file. Note that it does not apply to
lconstants, because these have scope only in the current file and  hence
do not need to define external symbols.)




------------------------------------------
4  Changing Pop-11 Files to Work with Popc
------------------------------------------

Many Pop-11  programs that  compile  and work  correctly in  the  normal
interactive Poplog system  will require changes  to compile under  Popc.
However, these changes can  always be made  in such a  way that the  end
result will work with either system.

The main  respects  in  which  programs will  require  changing  can  be
summarised as:

    Permanent Identifier Declarations
        All  permanent  identifiers  must   be  declared  before   being
        referenced. All declarations must be consistent, both within and
        across files.

    Use of uses-now instead of uses
        uses for libraries that are required at compile-time must employ
        uses-now instead.

    Compile-Time Executed Code
        There are certain restrictions on  the code that be executed  at
        compile-time.

    Keeping Compile-Time and Run-Time Code Separate.
        Permanent syntax  words  and macros  should  not be  mixed  with
        run-time code in the same file.

These areas are dealt with in the following sections.


4.1  Permanent Identifiers: Declarations & References
-----------------------------------------------------
In a file  compiled with Popc,  all user permanent  identifiers must  be
declared before being referenced.  An undeclared identifier will  result
in the error

        UNDECLARED IDENTIFIER xxxx

Moreover, any re-declaration of an  identifier must be identical to  its
existing  declaration  (in  terms  of  constant/vars  and   identprops),
otherwise the error

        REDECLARING STATUS OF PERM IDENTIFIER xxxx

results.

If an  identifier is  referenced with  weakref  in a  file, it  must  be
referenced everywhere in that file with weakref, i.e. all references  in
a single file must be either weak or strong. Referring to an  identifier
both weakly and strongly in the same file will produce the error

        MIXED WEAK/STRONG REFERENCES TO IDENTIFIER xxxx

(Note, however, that Popc does  not distinguish between strong and  weak
declarations: you may declare an identifier as either -- although it may
generally be clearer  to use weak  for a weakly-referenced  identifiers,
etc.)

Only identifiers actually assigned to at execute-level or referenced  in
procedure code are recorded on the output w-file; identifiers which  are
declared but not used anywhere are simply ignored.

Both Poplink  and Poplibr  also enforce  the consistency  of  identifier
declarations across files: all permanent identifiers must have identical
declarations in all files. You cannot declare an identifier as  constant
in one file, and vars in another, or as procedure-type in one file,  but
untyped in another, etc.  (An identifier may,  of course, be  referenced
weakly in one file and strongly in another.)


...  Poplog System Identifiers and popsyslist
---------------------------------------------
You do not need to supply declarations for Poplog system identifiers. As
described in Overview  above, Popc  automatically extracts  declarations
for these  from the  system  w-libraries, that  is,  .wlb files  in  the
directory $popobjlib.

However, which of these  files are actually  inspected is controlled  by
the current  value of  * popsyslist (which  contains * popuseslist  as a
sub-list). The names of files in $popobjlib just replicate the directory
pathnames in which the corresponding source files reside, e.g.

        $usepop/pop/src/        -->  src.wlb
        $usepop/pop/lib/auto/   -->  libauto.wlb
        $usepop/pop/x/ved/src/  -->  xvedsrc.wlb

Using this name mapping, Popc  will search the .wlb files  corresponding
to the  directories in  popsyslist.  Thus for  example, a  program  that
normally requires

        uses popxlib;

to extend popautolist  and popuseslist with  the X library  directories,
will require the  same with Popc.  (But note that  for this to  function
correctly, you must have

        uses-now popxlib;

instead. See Use of uses-now Instead of uses below.)

If you have  your own w-libraries,  Popc can  be told to  use these  for
declaration-extraction with the -wlib option. (You can also prevent  the
use  of  popsyslist  in  this  respect  with  the  -nosys  option.)  See
HELP * POPC.


...  Multi-File Programs
------------------------
The problem  of declarations  will  mainly affect  multi-file  programs,
which when  loaded  in  the  ordinary  Poplog  system  usually  rely  on
previously-compiled files supplying  declarations for  later ones.  This
technique  does  not  work  with  Popc  since  each  file  is   compiled
individually. (This applies even when compiling a number of source files
in a single run, e.g. in

        popc -c foo.p baz.p ...

all declarations for baz.p and subsequent  files must be supplied as  if
each were being compiled alone.)

(However, for special uses Popc provides the -g option, which  enables a
group of files to be compiled to a single object module. With this,  the
source files  are  effectively  just  concatenated,  and  so  identifier
declarations in earlier files are valid for references in later ones.)


...  Record and Vector Classes (defclass-declare)
-------------------------------------------------
If you use defclass  to define a (permanent)  record or vector class  in
one file, and wish to use one  or more of its associated identifiers  in
other files, then you must provide declarations for those identifiers in
the other files.

This can  be done  using  ordinary declarations,  e.g. if  file  'foo.p'
defines a class

        defclass fred {fred_f1 :full, fred_f2 :full};

then file 'baz.p' can make use of the associated procedures by declaring
them appropriately:

        constant fred_key;
        constant procedure (consfred, destfred, fred_f1, fred_f2);

(Note that +strict is  assumed here, i.e. that defclass will default  to
constant, with type procedure for all the identifiers except fred_key).

There is, however, one situation in  which the above is not  sufficient:
namely, where  a file  (other than  that defining  the class)  uses  the
constructor procedure to make an instance of the class at  compile-time.
For instance, baz.p might contain something like

        lconstant a_fred = consfred(1,2);

In this case, merely having consfred declared is clearly not sufficient;
the proper constructor procedure must actually be available.

In order to  achieve this,  note that  you cannot  simply replicate  the
ordinary definition of the fred class in baz.p, because that would cause
all the associated identifiers to be multiply-defined at link-time (only
one file may actually define the class). Instead, use defclass-declare:

        defclass-declare fred {fred_f1 :full, fred_f2 :full};

This  is  identical  to  an  ordinary  defclass,  except  that  all  the
identifiers -- while declared in Popc mode -- have their values assigned
in non-Popc mode. Thus  as far as Popc  is concerned, they are  declared
but not initialised;  instead, the  real identifiers  in the  underlying
system are assigned to (making  the constructor procedure available  for
execution).

(Note also another point  concerning this 'instance-constructing'  case.
As with any other structure, the  instance will be generated by Popc  as
described in Value Generation Phase above. You might think the key field
of the  structure would  somehow be  special, but  in fact,  it  follows
exactly the same rules as for any other field: Its value is fred_key  --
a structure  defined  in another  file  --  and so  fred_key  must  be a
constant. Thus you  cannot construct compile-time  instances of a  class
from another file for which the key identifier is a variable.)

While defclass-declare is essential  in the instance-constructing  case,
whether you use  it as  a substitute  for simple  declarations in  other
cases is a matter of taste.


...  Other Declarations (declare_incremental and declare_updater)
-----------------------------------------------------------------
Popc provides two special  declarations for permanent identifiers  which
you   may   need   to   use   (usually   with   multi-file    programs):
declare_incremental and declare_updater. Both of these declarations  are
also available in ordinary Pop-11 (where they essentially do nothing).

The first (declare_incremental)  is required  where a  program needs  to
build up some global structure (such as a list) across several files  --
that is, in the normal  interactive system the structure is  initialised
to an empty  state in  (say) the first  file loaded,  after which  later
files add increments to it.

This technique is supported with  Popc, but only for lists,  procedures,
and properties. See Incremental Identifiers below for full details.

The second declaration (declare_updater) may be required when a constant
procedure from one file is used (for example) as the pdpart of a closure
in another. E.g.

        constant procedure baz;     ;;; defined in some other file

        define foo = baz(% 1, 2, 3 %) enddefine;

When generating code for the  closure foo, Popc has  no a priori way  of
knowing whether the pdpart procedure baz has an updater or not, and thus
whether foo should have one: the default is to assume not. Hence if  baz
does have an updater (and you want foo to have one), then

        declare_updater baz;

is needed (following the ordinary declaration of baz).

See * declare_updater below.


...  -include Option
--------------------
To make it easy to supply identifier declarations that are not  required
for ordinary interactive compilation, Popc provides the -include option.
This automatically  #_INCLUDEs  a file  of  declarations in  every  file
compiled, e.g.

        popc -c -include popc_declare.ph foo.p baz.p ...

will #_INCLUDE 'declare.ph' at the beginning of every source file.

This and other initial arguments  to Popc can be conveniently  specified
for all source files in a given directory by means of a 'popc.args' file
in that directory (see HELP * POPC).



4.2  Use of uses-now Instead of uses
------------------------------------
In Popc compile mode, an ordinary uses statement such as

        uses foo, baz, ... ;

does not  actually cause  any  files to  be  loaded. Rather,  it  merely
records that the  identifiers foo, baz,  ... have been  'used'. At  link
time, this will force the extraction of the library modules defining the
identifiers even if they have not actually been referred to anywhere  in
the current (or any other) file.

This behaviour means,  however, that an  ordinary uses is  inappropriate
for a library which is actually required at compile-time -- that is, one
which (for example)  defines syntax  words or macros,  or which  extends
popuseslist (such as  LIB * POPXLIB). In these  cases, uses-now must  be
employed:

        uses-now popxlib, Xm, ... ;

This is  effectively  the same  as  surrounding an  ordinary  uses  with
normal_compile  ...  end_normal_compile,  i.e.  Popc  compile  mode   is
temporarily turned off, and the files are compiled as normal. (In normal
interactive compilation, uses-now is the same as uses.)



4.3  Restrictions on Compile-Time Executed Code
-----------------------------------------------
Almost all run-time executed code inside procedures will work  unchanged
with Popc, providing permanent  identifiers are declared correctly.  The
majority of problems in converting files  for Popc will occur with  code
executed at  compile-time;  that  is, either  execute-level  code  (code
outside of procedures,  lconstant expressions, code  inside #_< ...  >_#
etc), or syntax and  macro procedure code which  gets executed when  the
syntax word or macro is run during compilation.

(Note that  while execute-level  code will  always be  compiled in  Popc
mode, syntax/macro  code may  be compiled  either in  Popc mode,  or  in
ordinary mode when from a library.)

The principal problems you  will encounter will  result from the  errors
already described in Value Generation Phase above. In summary, they are:

  # You cannot reference the values of permanent variables assigned in
    other files (including system ones).

        Or rather, you  can reference them,  but the value  generation
        phase will not be able to generate their values. Probably  the
        most frequent problem in this respect is attempting to  create
        closures of variable procedures, e.g.

            define is_a_member =
                member(% [a list of words] %)
            enddefine;

        This will fail because  the procedure member  is not a  constant
        (using lmember_= instead would be  valid, because that is).  The
        solution in such cases is to define a proper procedure, i.e.

            define is_a_member(item);
                lvars item;
                member(item, [a list of words])
            enddefine;

  # You cannot  set one  permanent constant  directly equal  to  another
    (unless the latter's value has already been assigned in the  current
    file).

        For example,

            constant procedure foo = hd;

        is invalid. In this case, a dummy closure could be created:

            define foo = hd(%%) enddefine;

  # Popc runs in a Pop-11 that contains neither the Ved nor X parts of
    the system.

        Thus (with  the exception  of associated  syntax procedures  and
        macros) you cannot execute  any Ved or  X procedures at  compile
        time.

        (Note that  there  are  (at least)  two  X  library  procedures,
        XptSpecifyResourceInfo and  XptSpecifyPopValueTypes,  which  are
        not syntax  or macro,  but  which can  nevertheless be  used  at
        compile-time since they do not  make any 'hard' references  to X
        facilities (that  is, they  only refer  to them  'by name').  As
        described in Use of Libraries  above, you need uses-now to  load
        these procedures.)

  # Certain structures are not generatable.

        Currently, this  involves  instances of  shadowclass  structures
        (shadowkeys are OK  however), and  XptDescriptor structures.  To
        surmount such  problems,  generate the  structures  at run-time,
        either with a runtime action or in some other way.

        For example,  LIB * XptArgList requires  a constant  zero-length
        XptArgList structure (a  shadowclass instance).  This cannot  be
        defined as

            lconstant empty_arglist = consXptArgList(0);

        since the structure  is not generatable;  instead it must  use a
        top-level lvar  initialised  to  false, which  then  caches  the
        structure the first time it is used:

            lvars empty_arglist = false;

            define XptArgList(list);
                lvars list, size;
                if list == [] then
                    empty_arglist or
                        (consXptArgList(0) ->> empty_arglist), 0;
                ...

        A similar effect could be achieved with a runtime action:

            lvars empty_arglist;

            define :runtime_action;
                consXptArgList(0) -> empty_arglist;
            enddefine;

            define XptArgList(list);
                lvars list, size;
                if list == [] then
                    empty_arglist, 0
                ...



4.4  Syntax, Macro & Other Compile-Time Executed Procedures
-----------------------------------------------------------
In this section, compile-time  procedure (CTP) means  a syntax or  macro
procedure,  or  any  other  procedure   that  could  be  executed   in a
compile-time context.

...  General
------------
The points in the last section apply to the code planted or generated by
all CTPs, whether lexical or permanent identifiers.

CTPs of  any kind  should not  make any  direct references  to Ved  or X
run-time procedures, or any other  facilities that are not present  in a
basic Pop-11 system (because such code  will not load inside Popc).  All
such references should be 'by name', that is:

    #   For VM code-planting, always use sysPUSH and sysCALL on a quoted
        word or word-identifier rather than using sysPUSHQ or sysCALLQ;

    #   For other contexts,  access an object  by using  sys_current_val
        applied to the name as a quoted word/word-identifier.

In fact, references in CTPs to any run-time facilities should  generally
use the above --  for example, a  particular library run-time  procedure
may be  loadable  in Popc,  but  referring to  it  by name  rather  than
directly will obviate the need to load the procedure at compile-time.

...  Permanent
--------------
There are  additional factors  to bear  in mind  when writing  permanent
compile-time procedures (PCTPs).

If you  are  making a  CTP  a permanent  identifier,  you are  doing  so
(presumably) because you  want it to  be usable in  more than one  file.
Since with Popc, each file is compiled individually, you cannot rely  on
the PCTP having been loaded from some previous file (as often happens in
ordinary Pop-11). Thus

    (a) the  file  containing  the   PCTP  definition  must  either   be
        autoloadable, or  be loaded  explicitly with  uses-now by  every
        file that uses it;

    (b) as described in Use of Libraries above, this will load the  file
        in non-Popc  mode; hence  the  file defining  the PCTP  must  be
        capable of being loaded (in non-Popc mode) inside Popc.

As usual, (b) means  that the file containing  the PCTP definition  must
not make any  direct references to  Ved or X  run-time facilities,  etc.
This applies both to the PCTP itself,  and anything else that may be  in
the same file.  If the  PCTP needs  to reference  these facilities,  the
references must always  be via  quoted words  or word-identifiers.  (For
examples of  this,  see  LIB * vedset, * XptVal; these  are  two  syntax
constructs which plant calls to  Ved and X procedures respectively,  but
always  using  sysCALL  on  procedure  names,  never  sysCALLQ  on   the
procedures directly.)

Use of quoted words/word-identifiers  in this way is  not a problem  for
syntax constructs, macros,  etc, because such  things usually expect  to
have to use  the names  of identifiers,  and efficiency  issues are  not
involved. (For instance, a sysCALL on the name of a constant will result
in the same code as a sysCALLQ on its value.)

On the other  hand, run-time  code will  generally want  to make  'hard'
references to  identifiers  (and  not refer  to  names)  for  efficiency
reasons. Hence including other, run-time, procedures in the same file as
a PCTP may compromise the latter's ability to load inside Popc.

The above considerations lead to this general rule:

    Do not mix  permanent syntax  and macro  definitions (and  other
    procedures that  need  to  be  executed  at  compile-time)  with
    run-time code  in  the same  file  -- keep  the  two  completely
    separate.

A special case of the above comes  where a PCTP (such as a syntax  word)
plants code  for  run-time procedure(s)  that  are specific  to  itself.
Because they are  used only by  a particular PCTP,  such procedures  are
often defined as  lconstants in  the same file.  Although violating  the
rule above,  this  in itself  would  not necessarily  prevent  the  PCTP
loading in  Popc (it  would depend  on what  the lconstant  procedure(s)
contain); however,  the  practice  will not  work  anyway,  for  another
reason:

As described in  Value-Generation Errors above,  proper procedures  that
have not been compiled  in Popc mode cannot  be generated in the  output
object file. But (when used in a file other than its own), the PCTP file
will be compiled in non-Popc mode;  thus all the procedures it  contains
will be non-generatable.

To avoid this problem, run-time  procedures for which calls are  planted
by PCTPs must  always be  permanent identifiers. Popc  will then  merely
make the planted calls  reference the corresponding identifiers  without
attempting to generate the procedures  they contain. (Note that even  if
Popc  could  generate  lconstant  procedures  from  the   (non-Popc-mode
compiled) PCTP file, this would be highly undesirable anyway, since then
every file  using the  PCTP would  have its  own private  copy of  those
procedures.)

Providing the run-time  procedures are permanent  identifiers, there  is
nothing to prevent  them being in  the same file  as the PCTP,  assuming
they do  not reference  Ved or  X facilities.  However, this  is  always
undesirable for yet another reason: being in the same object module, the
run-time parts cannot be linked separately from the compile-time  parts.
(Not only  will linking  the run-time  parts into  a program  force  the
compile-time parts to be included as well, but the latter will often use
system compiler procedures, which  in turn will have  to be linked  in.)
Thus a linked program could end  up containing large amounts of code  it
does not actually require -- yet another reason for adhering to the rule
above.

Note that both  the example  syntax procedures  mentioned already  (i.e.
LIB * vedset, * XptVal) also  contain instances of  such 'own'  run-time
procedures that  are  permanent  constants  defined  in  separate  files
(LIB * vedset_pcompose  and   * XptValTestTypedArg  respectively).   For
ordinary Pop-11 use, there is no reason why these procedures should  not
simply be defined as  lconstants in their  respective syntax files;  but
they have to be permanent identifiers, and in separate files in order to
work with Popc.

...  Lexical
------------
By definition, lexical CTPs (e.g.  lconstant syntax and macro) can  only
be used in the files they are defined in, that is, can operate only when
Popc is being used to directly compile their files. Hence any  lconstant
run-time procedures they employ must also be compiled in Popc mode, etc.

Unless specifically  referenced  in  a non  syntax-  or  macro-expanding
context, lexical compile-time code will not be output on the object file
at all: thus the issue of mixing it with run-time code in the same  file
does not apply.




-----------------------------
5  Additional Syntax For Popc
-----------------------------

5.1  Incremental Identifiers
----------------------------
A technique commonly used in multi-file  programs is to build up  global
data structures  across a  number of  files. In  the normal  interactive
system, a structure  (such as  a list) may  be initialised  to an  empty
state in (say) the first file loaded; later files then add increments to
the structure,  as  appropriate.  A  permanent  identifier  containing a
structure created in this way is called an incremental identifier.

In an environment where each file is compiled individually,  incremental
identifiers necessitate a global linking process. This is so because  an
individual object  module can  do  no more  than record  its  particular
contribution to each structure; the  final structures must be  assembled
from the various parts when the program is linked.

Popc provides support for incremental identifiers containing one of  the
following types of structure:

    # lists
    # procedures
    # properties

The basis of the facility is the declare_incremental declaration,  which
declares a permanent identifier as being  incremental and of one of  the
three types listed above,  viz: incremental list, incremental  procedure
or incremental  property.  (Note  that  the  identifier  must  first  be
declared  in  the   normal  way,  i.e.   declare_incremental  does   not
subsume the basic constant/vars declaration.)

When a  permanent identifier  is  declared incremental,  Popc  records a
top-level assignment  to  it as  being  merely  a part  of  the  overall
structure in  the  identifier.  At program  link  time,  various  object
modules  containing  increments  for   the  identifier  may  have   been
incorporated in the program; Poplink then assembles the final  structure
from all the parts included.

Note that various system identifiers are incremental, e.g.  popautolist,
popuseslist and  pop_runtime_actions (lists),  popexit (procedure),  and
vedgetsysfilepdr (property). When Popc  extracts declarations for  these
identifiers from system w-libraries, it also recovers the  corresponding
incremental declarations, thus allowing user files to define  additional
increments to the standard values of these identifiers.


...  Syntax of declare_incremental
----------------------------------

declare_incremental                                             [syntax]
        This construct has the form

          declare_incremental
                 list [attributes] (perm_identifier, ... ),
            procedure [attributes] (perm_identifier, ... ),
                          property (perm_identifier = propexpr, ... ),
          ;

        The body may contain repeated  clauses, each beginning with  one
        of the keywords list, procedure or property, separated from  the
        next keyword clause by comma, or terminated by semicolon.

        For list or procedure, each keyword clause consists of  either a
        single perm_identifier name, or  a comma-separated list of  them
        in parentheses. The identifiers may be optionally preceded  by a
        comma-separated sequence of attributes in square brackets [...],
        where each attribute is one of

                prec=integer
                sublists
                writeable

        For prec, integer is a positive or negative integer in the range
        -16:8000 to 16:7FFF. E.g.

           declare_incremental list [sublists,prec= -200] (foo, baz);

        (N.B. A negative integer currently requires a space between  the
        = and the minus sign.)

        A property clause consists of either a single

            perm_identifier = propexpr

        spec, or  a  comma-separated list  of  them in  parentheses  (no
        attributes are allowed).  In each case,  propexpr is any  Pop-11
        expression which evaluates to a property. E.g.

            declare_incremental property (
                    xxxx = newassoc([]),
                    yyyy = newproperty([], 8, false, false),
                );

        The  semantics  of  declare_incremental  are  explained  in  the
        following sections.


...  Assignment Code: Basic Initialisation
------------------------------------------
After its (first) declaration with declare_incremental, the Popc  shadow
value of an incremental identifier  is set to its corresponding  'empty'
value, namely [] for  lists, identfn for  procedures, and the  evaluated
propexpr for properties.

This allows execute-level code in a file to add to the identifier,  i.e.
to add further elements to its 'existing' value (as if it had one).  For
example, if incr_list is an incremental list, then the code

        [abcd] <> incr_list -> incr_list;

'adds' the word  "abcd" to the  list. It is  important to note  however,
that  (even  though  the   end  result  may   appear  the  same),   Popc
distinguishes the above from

        [abcd] -> incr_list;

which -- by  virtue of the  fact that  it does not  access the  existing
value of the identifier before assigning  to it -- is deemed to  provide
the basic initialisation for the identifier.

The basic  initialisation  for  an incremental  identifier  is  like  an
ordinary initialisation for a non-incremental identifier (only one  file
can make the initialisation, etc). It is 'the part that should always be
present', as opposed to additions by other modules, which are  'optional
parts' (i.e. will be  added only if the  modules have been  incorporated
in the linked program for other reasons).

...  Lists
----------
An incremental list identifier must  always be a variable (this  applies
also to incremental  procedures, but  not to properties  -- see  below).
After declaration  with declare_incremental  list ...,  the Popc  shadow
value of the identifier is set to []; thus execute-level code that  adds
to the identifier, e.g.

        declare_incremental list incr_list;

        "abcd" :: incr_list -> incr_list;
        incr_list <> [abcd] -> incr_list;

will work as expected.

The linked value of an incremental list is a list whose elements are the
element(s) from each object module.

You can  use  the prec  (precedence)  integer attribute  with  different
values in different files  to control the order  in which elements  from
the various object modules will appear in the final list. The increments
from different  modules are  sorted in  numerical prec  order, with  the
lowest (most negative  value) coming  first. Note  that if  prec is  not
specified in a particular  module then it defaults  to 0 (thus  negative
values will come before any defaults, and positive values after them).

Elements from each individual  module will appear in  the final list  in
the same order  they appeared in  the module. However,  if the  sublists
attribute is specified  in a  module, then  all the  elements from  that
module will appear as a single  sub-list of the final list (rather  than
as separate elements).

The writeable  attribute controls  whether the  final list  is put  into
writeable memory: if  writeable is present  in any module,  the list  is
made writeable.

...  Procedures
---------------
As with  lists, incremental  procedure  identifiers must  be  variables.
After declaration  with  declare_incremental  procedure  ...,  the  Popc
shadow value of  the identifier  is set to  identfn; thus  execute-level
code that adds to the identifier, e.g.

        declare_incremental procedure incr_pdr;

        pdr1 <> incr_pdr -> incr_pdr;

will work as expected.

The linked  value  of an  incremental  procedure is  a  procedure  which
executes in turn  the procedures  from each  object module.  (Currently,
Poplink implements this as a closure

        appdata(% procedure_vec, fast_apply %)

where  procedure_vec  is   a  full  vector   containing  the   component
procedures.)

As with lists, the  prec integer attribute controls  the order in  which
procedures from  the various  object modules  will appear  in the  final
procedure.

...  Properties
---------------
Incremental properties are slightly different from the other two  types,
since the  identifier itself  contains a  value (a  property  procedure)
which is independent  of whatever  increments are  actually supplied  to
build the property (its entries). For this reason, the identifier may be
constant or variable, as desired (it may also be procedure-type).

After declaration with declare_incremental property ..., the Popc shadow
value of  the  identifier  is  set  to  the  evaluation  of  the  Pop-11
expression supplied  (which must, of course, produce a  property).  Thus
execute-level code that adds to the identifier, e.g.

        declare_incremental property incr_prop = newassoc([]);

        27 -> incr_prop("abcd");
        28 -> incr_prop("efgh");

will work as  expected. Note  that whereas with  list and  procedures, a
file is deemed to initialise an  identifier if it assigns to it  without
accessing it first,  in the  property case  this is  done by  explicitly
assigning a  property  to the  identifier.  For example,  if  the  above
assignments are replaced by

        define incr_prop =
            newassoc([[abcd 27] [efgh 28]])
        enddefine;

etc,  then  the  entries  in  incr_prop  will  be  taken  as  its  basic
initialisation.

There is  no  precedence relationship  for  a property;  the  increments
generated by each object module are  simply a set of entries, which  are
arranged inside the property in the usual way.

The  property  expression  supplied  to  declare_incremental  for   each
identifier may be any kind of  property. However, it must be capable  of
evaluation in an  arbitrary Pop-11 context  (since incremental  property
declarations  from library  modules   must  be   resurrectable  in   any
environment). This means  (for example) that  the property's default  or
equal_p values can only be  standard system values/procedures.

The same  expression must  be  used for  an  identifier in  all  modules
(Poplink treats any discrepancy between modules as an error).


5.2  Building W-Libraries
-------------------------
As described  in Overview  above  (and also  by  Use of  W-Libraries  in
HELP * POPC), each  w-file  records  which  w-libraries  were  used  for
extracting permanent identifier declarations when it was compiled.  Thus
every w-file knows  which libraries it  needs when linked  as part  of a
program.

When compiling  a  set  of  source  files with  which  to  build  a  new
w-library, you cannot (of  course) rely on this  mechanism to deal  with
identifier references between the source  files of the new library  (for
the fairly  obvious  reason  that  the  w-library  cannot  be  used  for
extracting declarations until after it is built).

As described in  earlier sections, therefore,  you must supply  explicit
declarations for all permanent identifiers referenced across the  source
files of the  library. However,  to enable each  w-file that  references
other library parts to  record that it 'uses'  this library, the  syntax
construct library_declare_section is provided: this informs Popc  that a
set of  identifiers  belong to  a  particular library  (even  though  it
doesn't yet exist).

Essentially, library_declare_section  is  a  'forward'  declaration  for
w-libraries. It is also  required in the  more complicated situation  of
building  a  whole  collection  of   libraries  which  use  each   other
recursively (as with many  of the Poplog libraries).  In this case,  the
Popc   -wlib   option    can   be   used    to   make   available    for
declaration-extraction those  libraries that  have already  been  built,
while library_declare_section  is used  for those  that have  yet to  be
built (including the one currently being compiled).


library_declare_section                                         [syntax]
        This construct has the form

            library_declare_section w-library-name-string
                statement-sequence
            end_library_declare_section;

        where  statement-sequence  is  any  sequence  of   execute-level
        statements. Its  effect is  to  mark all  permanent  identifiers
        declared inside statement-sequence as belonging to the w-library
        given by (the quoted string) w-library-name-string.

        w-library-name-string follows  the  same  rules  for  specifying
        w-library names to  Popc, Poplink and  Poplibr, as described  in
        Object & Library File Arguments in HELP * POPC.

        Note  that  if  w-library-name-string  contains  an  environment
        variable/logical name, it will NOT be expanded when the name  is
        stored on w-files. This is important, since it means environment
        variables will  remain  unexpanded  until  the  w-libraries  are
        actually used at link-time, allowing their locations to  change.
        (For example,  all  Poplog  libraries are  referenced  by  names
        starting with (at least)  $usepop/, and others as  $popautolib/,
        $popliblib/, etc.)



5.3  Other
----------

declare_updater                                                 [syntax]
        Where a permanent identifier contains an undef procedure defined
        in some  other  file, this  declaration  informs Popc  that  the
        procedure has an updater. The format is

            declare_updater perm_identifier, perm_identifier, ... ;

        It is usually required only  when a constant procedure from  one
        file is used as the pdpart of a closure in another. E.g.

            constant procedure baz;     ;;; defined in some other file
            declare_updater baz;        ;;; say it has an updater

            define foo = baz(% true %) enddefine;

        When generating code for the closure foo, Popc will generate  an
        updater for it in  the normal way (i.e.  to call the updater  of
        baz). Without the declare_updater, baz is assumed not to have an
        updater, and so the  closure foo will not  be given one  either.
        (Note that  declare_updater declarations  are not  required  for
        system  procedures,   since   the   information   is   extracted
        automatically from system w-libraries.)

        In ordinary  Pop-11, this  declaration merely  checks that  each
        perm_identifier contains a procedure with an updater.


normal_compile                                                  [syntax]
        This construct allows code to be executed in non-Popc mode while
        compiling a file with Popc. Its format is

                normal_compile
                    statement-sequence
                end_normal_compile;

        It locally sets pop_pas_mode  to its usual  value of false,  and
        then does

            pop11_exec_stmnt_seq_to("end_normal_compile") -> ;

        i.e.  execute  statements  until  the  closer  is  reached.  (In
        ordinary Pop-11, this is therefore effectively a no-op.)




------------------
6  Program Linking
------------------

6.1  Exporting of Identifiers and Sections
------------------------------------------
One of the functions  performed by Poplink is  to generate word  records
for all quoted  words used  in a  program, and  to arrange  them in  the
Pop-11 dictionary.

In the normal Pop-11 system, when you declare a permanent identifier  it
automatically becomes associated with the word corresponding to its name
(either at top-level  or inside  a section). This  makes the  identifier
accessible from the dictionary, either for further program  compilation,
or for use with valof.

When Poplink  links  a program  it  assumes  that in  general,  no  more
compilation will  take  place.  Hence by  default,  the  only  permanent
identifiers 'exported'  to  the  dictionary  are  those  for  which  the
corresponding quoted  word  was used  somewhere  in the  program.  (This
allows valof to be used  on such words; note  however, that it does  not
cope with the  case of  a program  constructing words  at run-time  with
consword.)

Exporting of  other identifiers must be requested explicitly, in one of
two ways:

    (1) By section, with the -s option to Poplink (see HELP * POPC);

    (2) By individual identifier,  either with  uses-by_name in  program
        files, or  by Poplink  options  -ubn and  -idexp etc.  (See  the
        description of uses-by_name in HELP * uses.)

A section is generated  in the section tree  either when an explicit  -s
option is given for it, or when one or more identifiers in that  section
are exported.


6.2  Run-Time Autoloading of Identifiers
----------------------------------------
In addition to using  valof on quoted words  and word identifiers,  some
programs and  facilities also  rely on  being able  to autoload  library
identifiers at run-time  (i.e. before accessing  them with valof).  In a
standalone  linked  program  this  is  probably  undesirable,  since  it
requires that the corresponding library source directories and files  be
present in the  program's execution  environment. It  also requires  the
incorporation into the linked program of sys_autoload, together with the
whole Pop-11 compiler and Poplog Virtual Machine.

Apart from those parts directly connected with the Pop-11 compiler,  all
system and library code  that uses valof makes  only weak references  to
sys_autoload (including valof itself).  If sys_autoload  is only  weakly
defined, these facilities will  not attempt to use  it. Thus, if you  do
require your program to perform run-time autoloading, you must include

    uses (sys_autoload, pop11_compile);

somewhere within the code.

Without this, run-time autoloading will not occur. You must then  ensure
that all library identifiers to which valof may be applied are  included
at link-time, and exported to the dictionary. The best way to do this is
with uses-by_name statements in your program.

For  example   (as  described   in  Class   Apply  Of   Descriptors   in
REF * XptDescriptor), the general apply mechanism for the  XptDescriptor
datatype constructs the name "XpttypeApply" when given an  XptDescriptor
to apply, and then calls the  apply procedure got from valof applied  to
the name; thus (e.g), applying a Widget will call XptWidgetApply. In the
normal system this  would be autoloaded,  but autoloading cannot  happen
without sys_autoload  being  present.  Hence to  apply  a  Widget,  your
program must force the inclusion of the procedure with

    uses-by_name (XptWidgetApply);


6.3  Weakly-Referenced Identifiers
----------------------------------
If at any point  in the linking  process all the  references to a  given
identifier are weak  (made with  weakref), Poplink will  not search  for
that identifier in libraries.  For all identifiers  that remain weak  at
the end, Poplink  'plugs' references to  them by generating  appropriate
dummy structures.

To avoid generating the same dummy structure repeatedly, only one  dummy
is produced for  each different class  of identifier, i.e.  one for  all
untyped  ordinary   identifiers,   one   for   procedure-type   ordinary
identifiers, one for active variables of each multiplicity M, and so on.

For constants, the dummies generated are anonymous undef records of  the
appropriate type -- that  is, all constants  of a particular  identifier
class are set to have their idval equal to the same undef record.

For variables, a single  dummy identifier is  generated for each  class.
The idval of each of these identifiers is initialised to the appropriate
constant undef record (although you should never rely on a weak variable
having any particular value, because dummy program assignments to it may
alter the initial value).


6.4  testdef
------------
In the ordinary system (as described in REF * IDENT), the expression

        testdef idname

is equivalent to the truthvalue of the expression

        isdefined(ident weakref idname)

However, when a program is  compiled and linked with Popc/Poplink,  this
is so only in relation to the state at link-time. In order to make  such
tests as  efficient as  possible, Popc  compiles them  to  zero/non-zero
tests on  the values  of special  external symbols,  which are  assigned
values by Poplink  after the  'strengths' of the  identifiers have  been
determined at link-time.

Hence the  truthvalues of  testdef are  permanently frozen,  i.e. if  an
identifier is weak at link-time, but then becomes defined at run-time, a
Popc-compiled testdef for it will remain unaffected.


6.5  Entry Procedure
--------------------
The -e option to Poplink specifies the name of the entry procedure to be
called on image startup; if omitted, it defaults to $-Pop$-Main (as  for
the -entrymain  argument  to * MKIMAGE).  The  entry procedure  must  be
defined as a procedure-type constant.

On startup,  arguments  in poparglist0  and  poparglist are  set  up  as
described in General System Startup in REF * SYSTEM.

Saved image arguments (starting with +) are interpreted only if  syssave
and/or sys_lock_system has been linked  into the program (these will  be
incorporated only if a  strong reference occurs for  either one --  both
bring  in   sysrestore   automatically).   You  can   use   syssave   or
sys_lock_system to save an  image on top of  your program image;  such a
saved image will  of course  only restore  into your  program, not  into
normal Poplog.  If  one or  more  +  arguments are  present,  the  saved
images(s) are restored and control is  passed to the last one (i.e.  the
syssave or sys_lock_system that created it will return true).

If no saved images are present, the specified entry procedure is  called
(from inside setpop). Before doing this, or before handing control  to a
saved image, special flag arguments  in poparglist beginning with %  are
processed in the normal way.



6.6  Embedding Pop in C
-----------------------
Popc-compiled code may be linked into  and called from a C program.  The
linking aspect is controlled by the Poplink options -exmain and -exlink,
which provide  for either  ordinary Pop  linking with  external  startup
(-exmain), or  independent  external  linking  (-exlink).  See  External
Startup & Linking in HELP * POPC for details.

The interface  for calling  Pop procedures  from C  is the  same as  for
external  callback,  described  in  REF * EXTERNAL.  Use  the   callback
function pop_get_ident to access  procedures from Pop  you wish to  call
from outside. Note that you will generally need to specify  uses-by_name
for such procedures, to ensure that they are exported to the  dictionary
(otherwise pop_get_ident will not be able to access them).

...  Initialising Pop
---------------------
Before any  callback  functions  are  called, the  Pop  system  must  be
initialised by  calling  pop_sys_init (this  function  is in  a  library
supplied by Poplink):

    #include "callback.h"           /* $popexternlib/callback.h */
    int pop_sys_init(unsigned plog_nwords, unused);

As described in Callback and Prolog in REF * EXTERNAL, the Poplog Prolog
area cannot be expanded  during a call from  external code. When  Prolog
routines are incorporated  into a standalone-linked  C program,  similar
restrictions apply; moreover, the Prolog  area cannot be located in  its
normal position on  the callstack,  and must be  allocated initially  in
ordinary memory. The  plog_nwords argument  enables you  to specify  the
size of the area in words; it effectively assigns the (afterwards fixed)
value of pop_prolog_size. If plog_nwords  is 0, the normal default  size
is used.

The unused argument is reserved for future use, and should be passed  as
0.

After initialisation, interrupt is set to sysexit; hence by default, any
mishaps occurring  in Pop  will cause  image exit.  However, a  callback
function called from  outside Pop always  sets PEF_CATCH_ABEXIT_NEXT  in
*pop_external_flags, so if  you reset  interrupt to setpop  in your  Pop
code, any mishaps or abnormal exits  will simply result in the  callback
function returning a 0 result.

...  Unix Signals
-----------------
In a system where it has full control, Pop normally assigns handlers for
all Unix signals (so  as to implement  Ctrl-C interrupts, timers,  child
process control, etc, etc).

A controlling  C program  may,  however, wish  to  have its  own  signal
handlers, and not want Pop to interfere with them. The C variable

    sigset_t * _pop_exclude_sigset;

is therefore provided; if  set to a  pointer to a  signal set (see  e.g.
UNIX * sigsetops),  then   * sys_reset_signal   will  not   change   the
underlying handler for any signal in the given set.

Needless to say,  if you use  this to prevent  Pop setting handlers  for
things like SIGALRM and SIGVTALRM, you can expect the corresponding  Pop
facilities such as  sys_timer to  stop working. In  terms of  modularity
Unix signals are a total disaster,  and there is no general solution  to
the problem.

(No corresponding problem exists in  VMS, where handlers for  conditions
and timer interrupts etc are specified in a modular way.)




----------------
7  Miscellaneous
----------------

7.1  Allocation of External Symbols
-----------------------------------
When Popc  generates  object  code,  it allocates  one  or  more  global
external symbols to each permanent identifier. An identifier whose  full
pathname is

        $-foo$-baz$-grunge

will have associated external symbol(s) of the form

        prefix.foo.baz.grunge

where prefix is 1-3 characters, e.g.

       i.foo.baz.grunge    (identifier record, when required)
       c.foo.baz.grunge    (value of a constant)
      xc.foo.baz.grunge    (execute label of a constant procedure)
      uc.foo.baz.grunge    (constant procedure updater)
     xuc.foo.baz.grunge    (execute label of constant procedure updater)

(N.B. On systems where '.' is not allowed in symbol names, '__' is  used
instead.)



--- C.all/ref/popc
--- Copyright University of Sussex 1994. All rights reserved.
