SYSDOC POPMAKE                                      R.Evans  July 1988


         CONTENTS - (Use <ENTER> g to access required sections)

 --  Introduction
 --  Setting up your environment to use POPMAKE
 --  A simple example of use
 --  Basic POPMAKE commands
 --  POPMAKE's make command
 --  Working with multiple versions of Poplog
 --  How POPMAKE works
 --  POPMAKE with SCCS
 --  Additional SCCS Functionality

Introduction
------------

POPMAKE is a collection of Unix utilities designed to support modular
Poplog development work. It provides the following facilities:

-   lets you develop and test Poplog system code without having to
    copy/link a whole source directory into a private directory
-   frees you from having to keep track (in W_FILES or wherever) of
    which files you have changed.
-   provides support for testing development under multiple versions
    of Poplog (in particular, different processor versions etc.)
-   provides 'make' facilities for automatic optimal rebuilding of a pop
    system.
-   accomodates the use of SCCS in your program development


Setting up your environment to use POPMAKE
------------------------------------------

POPMAKE is directly accessible in the poplocal tree on COGS suns.
It lives in $poplocal/popmake, with the 'pm' command linked into
$poplocal/local/com.

To make it available in other environments (from which the popmake
directory is accessible), you need to get the command script 'pm' (in
the popmake directory) into your command path. The recommended way of
doing this is to symbolically link it into your private commands
directory. Eg, if your commands directory is ~/com, do this:

    % ln -s $poplocal/popmake/pm ~/com
    % rehash

The rehash may be unnecessary, but it doesn't hurt.

Note: it is no longer necessary to define $POPMAKE in your .login
However, if you do so, make sure you set it to the popmake directory,
ie:

    % setenv POPMAKE $poplocal/popmake


A simple example of use
-----------------------

Let us suppose you want to make a small change to a system source file,
and you (naturally) want to test it before installing it in the masters.
For simplicity, lets say you are changing one of the mishap messages in
errors.p. Proceed as follows:

Create a directory for this development task, and cd into it:

    % mkdir ~/errors
    % cd ~/errors

Initialise the directory using the POPMAKE 'init' command:

    % pm init

POPMAKE will create a number of files in the directory, which I'll
discuss below. Now fetch a copy of errors.p:

    % pm fetch errors.p

NB: POPMAKE does NOT currently lock the file in the system masters.
Edit it as required, and then run POPMAKE's make command:

    % ved errors.p
    ...
    % pm make

POPMAKE will build a Poplog image, using the local errors.p, but taking
everything else from /pop/csuna/here. It will print messages as it goes
along to say what is happening:                    

    pgcomp: errors.p
    pgcomp: errors.w changed
    pglink: doing full link
    pglink: done

The first tells you which files it is recompiling (it only does those
that have changed since the last build). The second informs you that the
.w file changed (in this case because there wasn't a .w file before
compilation). When this happens a full link is required (ie pglink,
rather than pglink -q). You are told this just for information - in case
you get errors and later decide to do the link manually. In this case,
with no errors, POPMAKE goes on to do a full link.

Once the make is done, you will find a 'newpop11' that you can run to
test out your changes. You repeat the edit/make cycle until everything
is working. You can fetch additional files, including brand new ones if
desired, and when everything is done, you install your changes in the
masters. Then you can delete the whole directory.

You might get errors at any stage - syntax errors from pgcomp, poplink
or link errors from pglink. But POPMAKE's make file keeps track of what
still needs to be done, so the 'pm make' command is almost always all
you need to build the system (although explicit command exist for your
safety and convenience).


Basic POPMAKE commands
----------------------

There is in fact only one POPMAKE command - the 'pm' script you linked
into your commands directory. But its first argument is actually the
name of a command script in the $POPMAKE directory (feel free to look at
these and suggest changes), and these are the real commands. Thus all
POPMAKE commands start with 'pm '. (Some of the command scripts in
$POPMAKE end in .pm - for these 'pm' will complain if you try and run
them in a directory not configured for POPMAKE).


pm init [-sccs] [-r rdir] [-l ldir] [-s sdir] ["id string"]

    This command initialises a directory for use with POPMAKE. It
symbolically links all the .ph files from the remote master directory,
and creates a number of control files, described below. With the -sccs
flag, it initialises for use with SCCS. If rdir is specified, this is
taken as the remote source directory (default is /pop/csuna/here). If
ldir is specified, this is taken as the directory to initialise for
POPMAKE (default is the current directory). If sdir is specified, this
is taken as the local source directory (see below) (default is current
directory). If the id string is specified, it will be inserted in the
-popversion- string for pop systems created in this directory. If it is
"mytest", then popversion might be:

    (Version 13.52 mytest Thu Jun 30 13.22.52 BST 1988)

The default id string is "--- fredb ---" where fredb is your user name.


pm fetch [-e] [file ...]

    This command fetches the named file or files. If the file is a .p or
.s file in the remote master directory, it is copied to the local
directory, if it is a .ph file in the remote master, the local symbolic
link is replaced by a copy of the file. If it is a new file, it simply
creates it in the local directory (and tells you that it is a new file).
Note: if the file already exists locally, fetch does nothing (but tells
you so), so for new files in particular, you should fetch them BEFORE
you put code in them.

    Fetch also does POPMAKE bookkeeping, so you MUST use it to get new
files.

    The -e flag is relevant only if SCCS is being used - it tells
POPMAKE to make the file(s) ready for editing once created.

pm vfetch [file ...]

    This command does a 'virtual' fetch on the filenames given. If the
file is a .p or .s file in the remote master directory, a symbolic link
is created to it from the local directory. POPMAKE's bookkeeping
functions are also carried out for the named files. The effect of this
is to force POPMAKE to compile local versions of the object files, even
though you don't actually want a local copy of the source to edit. A
typical use of this might be where you have installed a file into the
system master which has some compile-time conditional switches in it:
normal system compilation sets them one way, but in your POPMAKE
directory you might compile the same file with the switches set
differently.

pm unfetch [file ...]

    This command undoes the effect of fetch or vfetch. It removes local
versions of the file (but not .w, .o files currently), recreates links
for remote .ph files and does bookkeeping. Generally only used if you
make mistakes (eg mistype filenames).


pm make [-m mac[=val] ...] [args]

    This command runs 'make' on POPMAKE's makefile. For full details see
below. The -m arguments are passed through to pgcomp, and so can be used
for popc macro definitions (eg -m SUN=true). Such macro definitions can
also be placed in the PM_CONFIG file using the PM_MACROS specification
(see below). The args, if any are passed to the make command. If no args
are given, make will assume the target 'newpop11'.


pm pgcomp
pm pglink
pm asm

    These commands provide a direct interface to the pgcomp, pglink and
asm scripts. These are basically the same as the system ones, except
they are more chatty, and they know that the main system archive
libraries live in a remote directory. They are entirely compatible
with the rest of POPMAKE and can be used freely if wished.


pm makebase

    This command makes the external link base file 'newpop11.stb' given
the executable image 'newpop11'


pm makemap

    This command makes the symbol map file 'newpop11.map' given the
executable image 'newpop11'


pm redo

    This command converts a POPMAKE directory from old format to current
format. Its definition changes whenever the format changes. Its current
function is to create the file M_FILES if it doesn't exist already.


pm help

    This command displays this help file (using 'more').


POPMAKE's make command
----------------------

POPMAKE has a special 'make' command which executes Unix 'make' (see MAN
(1) make), on a make script which knows about Poplog system file
dependencies. The useful make targets are listed below and are invoked
simply by giving the target name as an argument to 'pm make' (eg 'pm
make stb'). No target is the same as specifying newpop11 as target.

    newpop11    - rebuild the newpop11 executable image
    stb         - rebuild newpop11.stb
    map         - rebuild newpop11.map
    objects     - recompile any object files that need it
    dict        - rebuild the dictionary files

Make keeps track of which files have changed and what work needs to be
done to rebuild the requested target. For example, it will decide for
itself whether to recompile particular object files, whether to do a
quick or a full link etc., in order to build newpop11. Sometimes it will
simply report that no work needs to be done ('`newpop11' is up to date'
etc.).

Note that although make will accept a single object file name (foo.o
etc.) as a target, it will not rebuild it correctly. (This is a bug, but
not obviously fixable.) Use the target 'objects' to rebuild all the
object files that need it instead. Also, make doesn't know anything
about .ph files, so if you change one, you will have to rebuild any
source files that include it yourself (using 'pm pgcomp').


Working with multiple versions of Poplog
----------------------------------------

In its standard configuration, POPMAKE allows you to create a directory
of Poplog source files and build an executable image in that directory,
bringing in files from the remote master directory where needed.
Sometimes, however, this degree of modularity is not enough. Suppose you
are developing some source files and want to test them with more than
one version of Poplog (eg two distinct version numbers, or versions for
different machines). You could have two ordinary POPMAKE directories
with different remote source directories (using the -r option to pm
init), but that would entail having two copies of your development
sources, keeping their popmake status compatible (fetching in both
directories etc.), fiddling with SCCS etc..

The -s option to pm init provides a way of avoiding this complexity in
such cases. It allows you to create a POPMAKE directory whose LOCAL
source directories (as well as the remote ones) are somewhere else. The
current directory is used only for objects, executable images and the
Makefile bookkeeping files. All the popmake COMPILATION commands (make,
pgcomp etc.) will work but sourcefile commands (fetch etc.) will not.

Typically you would use -s in conjunction with -r, to change the remote
master directory too. Suppose you create a development Poplog system
on a Sun running SunOS 3.4 with:

    % mkdir test; cd test;
    % pm init "test"

Now you want to test your development under SunOS 4.0. Lets assume there
is a 4.0 source directory in /pop/csunb/here (there isn't currently).
Then you could do:

    % cd ..     # back to where we were
    % mkdir test.bob; cd test.bob
    % pm init -r /pop/csunb/here -s ../test "bobcat test"

For compilation, this directory would behave like any other POPMAKE
directory (but the POPMAKE commands must be run from a SunOS 4.0
machine!). But it uses the source files of 'test', so actual
development happens there and only there.

Similar set-ups for other machines (vax bobcat etc.) depend only on (a)
POPMAKE existing on both machine types and (b) there being a common
directory for the development sources (NB: the -s popmake directory
and its remote masters don't have to be in a common area, since they are
only accessed from their own machine type.



How POPMAKE works
-----------------

The core of POPMAKE consists of slightly doctored versions of the
standard system build scripts, a makefile, and a few configuration
control files which live in the particular development directory.

The system build scripts have been modified to assume that the
popc/poplink saved images, and the main system masters, are in a remote
directory ($PM_REMOTE) rather than the current directory. But source
files in $PM_SRC (default, current directory) override those in
$PM_REMOTE. They build (.o files, newpop11 etc.) in the current
directory. Definitions for $PM_REMOTE, $PM_SRC etc. are taken from a
file PM_CONFIG in the current directory. There are a few other minor
changes, but those are the main ones.

The makefile know about the major interdependencies of the Poplog
system. It uses some control files (WORDS.time, OBJS.todo, LINK.time,
LINK.done) to help it keep track of more difficult dependencies (eg if
we've done a full link, we don't need to do a link -q afterwards). So if
you notice these files, just ignore them (though if you accidentally
remove them, make will generally cope - it may just do more work than
necessary).

The main control files are PM_CONFIG, W_FILES, O_FILES and M_FILES.
PM_CONFIG tells POPMAKE (a) where to find the remote directory, (b) what
the id string for this pop system is, and sometimes (c) where to find
the local sources (d) which additional object modules to exclude from
the poplink and (e) what other object modules should be added to the
final ld (eg the X windows libraries). The file's format is a bourne
shell script to set shell variables. The default version is:

    PM_REMOTE=/pop/csuna/here
    PM_ID="--- $user ---"

but PM_ID may be (and generally is) overridden in the pm init command. A
line defining PM_SRC is also added if the -s flag is given to pm init.
Further (optional) lines are passed to the poplink command:
a line defining PM_EXCL specifies Poplog object modules (.w files) to
omit from the poplink, and a line defining PM_LIBS specifies trailing
arguments to the poplink command. In other words, the poplink command
gets built like this:

    safepop11 +poplink .... src.wlb -ex ( ... $PM_EXCL ) $PM_LIBS

For example to link in the X libraries you might have a line: (this
won't actually work, currently).

    PM_LIBS='-lo ( -lXaw -lXt -lX )'

To omit the PWM from the link you might say (this won't work either,
unless you leave out VED too!):

    PM_EXCL='pwmio.w vdpwm.w'

It is perfectly acceptable for you to hand edit PM_CONFIG, if you want
to change something. In particular, there is currently no other way to
add a PM_EXCL or PM_LIBS line. A further optional line specifies popc
macro arguments, as you would specify to the pm 'make' command. For
example, Xpop has a popc macro 'XT_REVISION' which says which version of
the X toolkit it is using. This defaults to 3, but to build a revison 2
system, one might have a PM_CONFIG file with the line:

    PM_MACROS='-m XT_REVISION=2'

The file W_FILES is as used by pglink - it is a list of the .w files not
to be located in the archive. O_FILES is a corresponding list of .o
files, and is used by the makefile. M_FILES is a collection of 'make'
commands to tell 'make' how to build object files. These files are
maintained by fetch/unfetch and should generally be left alone.


POPMAKE with SCCS
-----------------

If you give the -sccs flag to pm init, POPMAKE sets up the directory for
use with SCCS. Some of its control files (notably PM_CONFIG and W_FILES)
are placed under SCCS control, as are any files fetched. This means you
have to explicitly edit, delta and get (or delget) your sourcefiles as
you work. Note however, that pm make will cope with the files in more or
less any SCCS state - you don't have to delget before making.

To simplify life a little, the pm command accepts SCCS commands as well
as those described above. Thus

    % pm delta foo.p

is entirely equivalent to

    % sccs delta foo.p

(One sccs command (deledit) is actually overriden by a pm command, but
this is just a bug patch - it works exactly like deledit should.) In
addition to the standard SCCS commands, two other commands are available
for use under SCCS:

pm log [-R Rel] [-L] [-arg ...] [files]

    This command prints out a log for all the files specified,
consisting of version number and comments added when the version was
created. The -R option makes it print only information more recent that
release Rel, the -L option makes it send the output to log files rather
than to the terminal (eg log for foo.p goes to foo.log). Any other args
are passed through to the SCCS 'prs' command on which this is based.

pm tidy

    This command tidies up SCCS files at the end of a session. The idea
is that you have  created your files and tested them and now have an up
to date working system. tidy delget's all the files that need
it (prompting only once for comment), and also touches the corresponding
.o files (otherwise make would think they need recompiling - the delget
makes the source more recent than the object).


Additional SCCS Functionality
-----------------------------

POPMAKE provides two useful pieces of functionality over the basic
sccs command. Firstly, suppose you want to create subdirectories of your
source directory, for example containg documentation files, or test
files. You can use SCCS on these, simply by creating an SCCS
sub-directory within them. But then to access the files you either have
to cd into the subdirectory, and do sccs edit (or whatever), or else use
the -d flag to sccs. The former is inconvenient, if you are working in
several directories at once, the latter creates files for editing in the
CURRENT directory, not the one specified with the -d (which I think
would be more natural). So pm takes an optional directory specification
as first arg, starting with a /. If the directory is a subdirectory of
the current directory, the pm command that follows is executed in that
directory, rather than the current one. So you can control files in a
subdirectory 'help' of your development directory by giving commands
like:

    pm /help edit foo

All the SCCS commands can be used in this way, and they will manipulate
files entierly within the specified supdirectory. In fact, all the
POPMAKE commands will work, as long as the subdirectory contains
appropriate configuration files (PM_CONFIG, W_FILES, O_FILES etc.).

Secondly, POPMAKE uses the environment variable $PM_SCCS as the name of
the SCCS control subdirectory. It defaults to SCCS, but the user may set
it globally if desired (it is passed as the '-p' argument to all sccs
commands). For example, setting it to '.SCCS' will hide the directory -
which might make installation scripts easier to write.
