The doc-creation process does not need any information from the configure-process - it works independently. The toplevel doc-targets will forward usually to the doc/Makefile, and it will do so for quite a few other targets as well. The 'make install' will indeed not install these docs (which is a megabyte of html files), so you have to call 'make install-doc'. For the latter, we use a little trick - the install-doc is forwarded to the configure'd Makefile (which knows the configure'd prefix, e.g. /usr/local or /programs), and the configure'd Makefile forward this target in turn to doc/Makefile but adding a makfile-override prefix=@prefix@ - in makefile-speak: '$(MAKE) -C $(srcdir)/../doc/Makefile prefix=$(prefix) [sadly, bsd pmake does not support "-C"]

The doc-tarball created in the doc-subdir (e.g. words.tar, wordsets.tar and doc.tar) can be rolled into a toplevel dist-archive for the doc. This is also needed since the generated docs are so large (around a megabyte of data) and the doc-generation requires perl and a perl-based helper-package xm-tool (from xm-tool.sourceforge.net) to be made.

The "make rpm" is currently a bit weird in its assumptions - it looks in the local, parent, parentparent directory for a subdir called "packages". It then copy the "pfe-current.tgz" to "packages/SOURCES/" and call 'rpm -ba pfe.spec'", i.e. rpm-build-all

It is interesting to see that this scheme works for most people who create "rpm" files, but it could be just as well be a bit annoying to some other people.

The debian distro files are currently not shipped with the base pfe.

The windows archive-packages are not fully ready, currently it creates a simple zip-archive with prebuilt binaries for mingw32, but this target is not yet supported - I did this target largely to speed up the development process for this target (cygwin windows-pfe works fine for years).

The most time critical piece of code in pfe is the inner interpreter, a tight loop calling all primitives compiled into a high-level definition. You find it in file support.c, function run_forth().

On some CPU's it significantly saves time when the code of the inner interpreter is unrolled several times without the need to jump back to the start of the loop after every primitive is executed. On other CPU's it doesn't help or even makes it slower.

For example the benchmark-performance of pfe on a 486 is about 15% better with unrolled NEXT, while the performance on a Pentium becomes slightly worse.

You'll have to try it, what is better on your machine. To enable the feature, you need the following compiler option in Makefile:

	 -DUNROLL_NEXT   

which you can achieve now at `configure`-time too by using

   --with-user-config=UNROLL_NEXT   

pfe is designed for best portability. This means it can be compiled with a variety of compilers on many systems. Obviously this prevented me from squeezing the last bit of performance out of any special system.

Fortunately there's a way to tune it up significantly with only little effort provided you have GNU-C at hand.

Let me explain: As most of you probably know, a Forth-interpreter traditionally contains a so-called virtual machine. PFE does. This virtual machine consists of several virtual registers and a basic set of operations. The virtual registers are:

	IP	an instruction pointer
	SP	the data stack pointer
	RP	the return stack pointer
	WP	an auxiliary word pointer
 

in pfe there are additionally:

	LP	pointer to local variables
	FP	floating point stack pointer
 

In a traditional assembler-based Forth implementation these virtual registers would be mapped to physical registers of the CPU at hand. How efficient such an implementation is depends heavily on how cleverly this mapping is done.

pfe has no other choice than to declare C-language global variables to represent these virtual registers. These variables are accessed very frequently.

Now GNU-C allows us to put global variables in registers! Obviously the number of registers in a CPU is limited and the use of registers by library functions and the compiler itself interferes.

In spite of these restrictions it is possible to find a niche even in an i386 where to place the two most important virtual registers resulting in a performance boost of about 50%. (Just one more detail that shows what a great job the GNU-C developers did.)

If your system is one of those known by the config-script then all provisions to use global register variables are already taken. You can enable and disable the usage of global register variables in by specifying the option '-DPFE_USE_REGS' in the Makefile, or using the configure-option '--with-regs'.

If your system isn't known by the config script, then first make sure you have a stable port according to the instructions in the file `INSTALL'. Then read the next section to enable the usage of register variables on your system. If all works well please send me your changes.

When you use global register variables in GNU-C then you have to explicitly state which machine register to use for the global variable to declare "register". The syntax is like this:

	register type variable_name asm ("machine register name");
 

instead of just

	type variable_name;
 

As far as I see choosing machine registers to use for global register variables is just a matter of trial and error.

First find out how registers are named on your machine. Not how the CPU-manufacturer names them but how the assembler used by gcc (as or gas depending on the configuration of gcc) names them. It's easy: simply use gcc to compile one of the C files with option -S. I changed the `makefile' to allow this by simply `make core.s'.

Then look at `core.s': You don't have to know much of assembly language programming and even less of the particular CPU. All you are interested in is: what are the registers? In `core.s' search for the label `dupe_' i.e. the compiled function that does the work of the Forth word `DUP'. The C-source for DUP is:

	FCode (p4_dup)
	{
  	  --SP;
  	  SP[0] = SP[1];
	}
 

On an RS/6000 (where you won't have to do this because I did it already) using gcc you'd find the following assembler lines generated for p4_dup_:

 .p4_dup_:
        l 11,LC..106(2)
        l 9,0(11)
        cal 0,-4(9)
        st 0,0(11)
        l 0,0(9)
        st 0,-4(9)
        br
 

Reading more of the generated assembler source allowed a guess that

Next edit the file `pfe/def-regs.h'. Add a system specific section of preprocessor definitions naming CPU registers to use for virtual machine registers like this:

	...
	#elif __target_os_aix3

	#  define P4_REGIP "13"
	#  define P4_REGSP "14"
	#  define P4_REGRP "15"
	#  define P4_REGW  "16"
	#  define P4_REGLP "17"
	#  define P4_REGFP "18"

	#elif...
 

Ok, the full set needed a little more experimentation. Maybe start with only P4_REGSP or P4_REGIP.

After enabeling these declarations with the -DPFE_USE_REGS command line option another `make core.s' yields the following translation for DUP:

 .p4_dup_:
	cal 14,-4(14)
	l 0,4(14)
	st 0,0(14)
	br
 

Quite a difference!

If your CPU has different types of registers for data and for pointers then the pointers are needed in pfe. (On M68k the Ax not the Dx.)

If you don't have enough free registers in your CPU then serve the first virtual registers in the above list first. They are ordered by their importance.

Then do a `make new' with option -DPFE_USE_REGS. If you get compiler errors and warnings about `spilled' or `clobbered' registers then change the mapping until it compiles quietly. There's a good chance that it still runs now and if it does it runs significantly faster than before.

Good luck!
Dirk

currently, building of a module has only been done within the pfe source tree. You have to add a few lines to the Makefile.am so it is build and installed. This howto will guide you through the process of creating a new module for the pfe source tree, i.e. a pfe extension module.

the first thing you have to do of course: be creative and invent a name. This name will be used in many many occasions as a reference symbol and signon identifier. In this example the module is named 'example' which is creative enough here.

this name is called a 'wordset-name' since it will be used as that. It can even be queried with ENVIRONMENT, and it is listed in the LOADED wordlist of pfe.

the filename shall *not* be example.c, since I am compiling the pfe for some embedded/kernel targets which only need a '.o'-file, just think of a linux kernel module. Since the intermediate objects are '.o'-files and the 'ld -r' target of several intermediate objectfiles is also an '.o'-file, well, it must be assured that the intermediate objectfiles have a different name than the product '.o'-file.

If you did not understand what I want, well, don't think about it too long and add a "-ext" to the filestem, so that in here, the extension module 'example' is build from the source file 'example-ext.c'.

Have a look at the 'Makefile.am' and its 'toolbelt-ext.c'. You will instantly see what is to be done: first, add the module name to the 'pkglib_LTLIBLIBRARIES, ie.

 -pkglib_LTLIBRARIES = edit.la toolbelt.la
 +pkglib_LTLIBRARIES = edit.la toolbelt.la example.la

and then add a new _la_ section that `automake` can see, since you will probably build from just one sourcefile, it will just look like the others, ie.

 +example_la_SOURCES = example-ext.c
 +example_la_LDFLAGS = -export-dynamic -module -avoid-version
 +example-ext.lo : example-ext.c
	$(LTCOMPILE) -DMODULE -c $<

where the third line should go away in the next PFE generation, and in fact since generation 32.x there is no need for a MODULE-define anymore - the .o file is the same for being either external or internal to the pfe binary. Instead we link a loader-part into the module.so. The 32.x module automake snippet looks now like:

 +example_la_SOURCES = example-ext.c example-dll.c
 +example_la_LDFLAGS = -export-dynamic -module -avoid-version
 +example-dll.c : $(srcdir)/module-dll.c
	sed s/module/example/ $< > $@

and it allows us to modify the module glue code independently of the module source code later on.

this is it for Makefile.am, now go ahead and create the file, i.e. 'example-ext.c'

I do strongly suggest that you include a header comment that goes right at the start of the file. The autodoc system of pfe will see it as a special section that should be treated specially and included in the documentation file. Just explain everything that you want to point out to anyone who would want to use your wordset. Do also include your name and a copyright information. Remember, it is the most easiest for you to send me the file, so it can be distributed along with PFE, so it can get compiled on many many platforms, and so it can get maintained over some internal changes in PFE. And actually, this very source file is stored also in the Tek/MPT Source Repository, where you don't want that some Tekkie simply adds a Tektronix Copyright in there - the files are writable by other Tek developers too, not just me.

 /** 
  *  Artistic License (C) 2000 Julius Caesar
  *
  *  @description
  *      An example module for my personal experimentation.
  */

next you need to include some headers from the pfe base system. These headers are made namespace clean, ie. they all have a prefix like 'FX_' or mostly 'p4_'. For a real programmer, this is inconvenient, and it makes the code not very readable. If you look closer, you will see that in most headers there are '#ifdef _P4_SOURCE' sections (expecially in def-types.h) which do include things like

 #define DP    p4_DP
 #define BASE  p4_BASE
 #define SP    p4_SP
 #define STATE p4_STATE

In general, most sources handwritten by users will want to have these. This is however not a good recommendation if the extension module is derived from some other source, e.g. Tek/MPT has a SWIG extension to convert C headers to pfe modules. Anyway, your file will most start with:

 #define P4_SOURCE 1
 #include <pfe/pfe-base.h>

make sure to include one of the pfe headers first, so that the gcc register allocation may work (--with-regs is greater 0). For a single wordset, you need also to include pfe/def-words.h, but I recommend to do that last, after all other includes, since there are a lot of two-char #defines (if you specified P4_SOURCE).

Now, let's have a look at a simple word, e.g. the 2NIP as implemented in 'toolbelt-ext.c'. Please add a javadoc like comment before, and make the first line of that comment show the Forth Stack Notation.

 /** 2NIP ( w x y z -- y z )
  * Drop the third and fourth elements from the stack.
  */

Now everyone knows what that word should do. All wordset words in PFE should then be declared with a prototype macro as FCode. On most systems, a 'FCode(example)' will expand to 'void example_ (void)' - note the underscore at the end that distinguishes the pfe symbol from other C symbols.

Write the body of the function. Inside of an 'FCode' word, you are assured to access the forth stacks and dictionary directly - via its pointer macros. The most common pointers are:

 SP - Parameter stack pointer (downwards)
 RP - Return stack pointer (downwards)
 FP - Floating stack pointer (downwards, not always compiled in).
 IP - Colon Instruction pointer (upwards) 
      points to the next token to be executed by the innner
      interpreter (known as NEXT in other forth systems).
 DP - dictionary pointer, the values is otherwise known as HERE.
 LAST - pointer to NFA of the last CREATE word, null after FORGET

most of the important ones are declared in def-types.h, and most of the important macros to access them are declared in def-macro.h, e.g.

 FX_DROP FX_POP FX_PUSH(x) FX_DUP FX_OVER FX_NIP ... changing SP
 and FX_COMMA to compile to HERE 
 (and comma is defined as '*DP = x, DP += sizeof(p4cell)' )

the 2NIP implementation is of course a short one. We just want to nip the third and fourth item in the parameter stack, and just as you would expect from 'PICK', the values in the SP-stack are called SP[0] SP[1] SP[2] SP[3], where SP[0] is of course the top of stack. Here we copy [0]->[2] and [1]->[3] and decrease then the stack depth by increasing the stack pointer by 2 - remember that the parameter stack is a "(p4cell*)" and it increments downward.

  SP[2] = SP[0];
  SP[3] = SP[1];
  SP += 2;

you can then declare other such words, and finally you need to make them known to forth. This is done by assembling all the words in a Wordset-table. A Wordset table is really two C strutures, where the first lists the entries and the second gives some more information. They are always written shoulder on shoulder, so it looks like

  P4_LISTWORDS(example) =
  {
	CO ("2NIP",   p4_two_nip),
  };
  P4_COUNTWORDS(example, "EXAMPLE - my own example words");

but the two-letter entries will be removed from PFE shortly, so use the new style with longer names. Since 31.x generation, you should use this style:

  P4_LISTWORDS(example) =
  {
	P4_FXco ("2NIP",   p4_two_nip),
  };
  P4_COUNTWORDS(example, "EXAMPLE - my own example words");

and in the following sections, only the longer names are referenced. Have a look into the header file to get the old-style words.

note that P4_FXco is a macro from 'pfe-words.h' that does all the relevant things. So just give it a name with a C-string and the name you used in FCode. The P4_COUNTWORDS macro has a string - and the first part (upto the first space) is used to identify the wordset in ENVIRONMENT queries. It will also show up in the LOADED WORDS.

the macro (e.g. P4_FXco) will define what the symbol should be look like in forth - P4_FXco is a subroutine code reference, i.e. a primitive. P4_IXco is the same, but immediate. There are lots of other macros, just have a look at 'def-words.h'

note: the following paragraph is outdated since the 31.x generation which has this LOADLIST table in the referenced module-dll.c. No need to declare it by hand anymore, just link it to your original source code. Anyway, you still have the option to declare an explicit loadlist, but you can not be assured that this will stay in the next generation of PFE, where the loadlist code might be removed completly. The single-level load-scheme does not need it anymore, as all kinds of loader-commands are available in wordsets too. Avoid this one.

Anyway, here's what it looked like:

 P4_LOADLIST (example) =
 {
	P4_LOAD_INTO, "EXTENSIONS",
	&P4WORDS(example),
	P4_LOAD_END
 };
 P4_MODULE_LIST (example);

And now you are basically through with it. Just compile, and when `pfe` is started, type 'LOADM example' to get access to the words in the 'EXTENSIONS' vocabulary.

The Semant words are one of the nicest features of PFE. Without much horrors, you get compiling words and state-smart words ... and it will also be nicely decompiled by `SEE` without any further problem.

Let's have a look now at p4_literal, i.e. LITERAL

 /** LITERAL ( value -- )
  * compiling takes the value from CS-STACK and puts
  * it into the dictionary. Upon execution, it will 
  * visible the parameter stack. In exec mode, the
  * value is just left on the CS-STACK - which simply
  * is the parameter stack itself.
  */
 FCode (p4_literal)
 {
	if (STATE)
	{
		FX_COMPILE (p4_literal);
		FX_COMMA (FX_POP);
	}
 }
 FCode (p4_literal_execution)
 {
	FX_PUSH (P4_POP (IP));
 }
 P4COMPILES (p4_literal, p4_literal_execution,
	P4_SKIPS_CELL, P4_DEFAULT_STYLE);

The last COMPILES-declaration is the binding link between everything and all about Semant-words. The first parameter references the original compiling FCode. The FX_COMPILE in the compiling FCode will in turn reference this semant declaration.

The second parameter of COMPILES is of course the execution that should be COMMA into the dictionary. Since pfe is indirect threaded, you cannot just use FX_COMMA(p4_literal_execution), instead you compile the address of the pointer to p4_literal_execution that is given by the static Semant-structure. The advantage is, that the decompiler knows the address of this COMPILES-structure, and so there are some hints for the decompiler. SKIPS_CELL should be very obvious - the decompiler shall not interpret the next token in the colon-definition. And the default-style is, well, just nothing. All kinds of indentations for IF and LOOP style words could be given. See 'def-const.h' for some of them.

The compiling word should now be understandable: if in compiling mode, compile a execution-token (the address to a pointer to a C-function), and the value on the stack into the dictionary at HERE. The POP will also consume the value off the paramter stack.

The execution is supposed to do the reverse of it, so PUSH will insert the value on top of the parameter stack, and the value is retrieved by looking at IP. Remember, IP points to the next token that the colon-inner-interpreter will execute if the current C-function returns. Therefore, the value is fetched from there (i.e. *IP) and afterwards increased to the next token (i.e. IP++) which can be expressed with a single statement as in *IP++. You could however use the macro P4_POP(IP) to make for a bit of literal programming here.

Now that the implementation is done, export the semant-word in the wordset-table - and be sure to use 'CS'. All 'CS' words are of course immediate, and it does not reference the compiling word, but the semant-structure. Here you would write...

  P4_LISTWORDS(example) =
  {
	P4_SXco ("LITERAL",   p4_literal),
  };
  P4_COUNTWORDS(example, "EXAMPLE - my own example words");

The real benefit will be obvious when you make a colon-definition with a semant-word, and when done, use SEE to see what is in there. It will produce some very fine output. Well, the SEE words are of course in debug-ext.c, since decompiling is used usually during debugging or even single-stepping.

The previous section dealt with the execution semantics of compiling words which add their execution vector to the current colon definition under creation. Here we present the style of creating new HEADER entries in the dictionary and setting up its runtime code for the new words.

Up to the 31.x generation, this was very simple - one would simply call a word that creates a header entry (or just skip that part of noname entries), and the CFA runtime vector had been simply COMMAd into its place, followed by more COMMAs to set up the parameter field. Here's a typical snippet of that style:

 FCode(p4_constant)
 {
    p4_header (PFX(p4_constant_RT), 0);
    FX_COMMA(FX_POP);
 }

With the current generation of 32.x this is not quite recommended, even that you can still use this scheme. However, use a new style for it which is much more obvious about what you want to do, so let it look like this:

 FCode(p4_constant)
 {
   FX_HEADER;   // create header up to but not including CFA
   FX_COMMA(PFX(p4_constant_RT));     // setup the CFA value
   FX_COMMA(FX_POP);
 }

The disadvantage is that it makes a specific assumption about the setting of a runtime vector of a codefield, and it even declared the codefield to be just the address of the runtime C-routine. This is true for the default indirect-threaded model.

In order to widen the range of possible threading-models, we go the same way as for the semant-words - we create an runtime info-block and the CFA-setup is done by referencing this info-block - in the default indirect-threaded model it will simply fetch the value that points to the C-routine, and COMMA it where the definition is so far. Here is the style that is recommended in the 32.x generation:

 FCode(p4_constant)
 {
    FX_HEADER;
    FX_RUNTIME1(p4_constant);
    FX_COMMA(FX_POP);
 }
 P4RUNTIME1(p4_constant, p4_constant_RT);

and unlike the traditional code, this one is not anymore just an FXco or IXco (that would get at the FCode address) in the LISTWORDS table, instead we use now the RTco designation in the LISTWORDS table that will reference the name of the P4RUNTIME info-block (just like SXco references the name of the P4COMPILES info-block).

Actually, this new style with an FX_RUNTIME macro, makes the C source much more obvious as the macro name FX_RUNTIME does point out what shall be done at this point, to setup a runtime-vector for the header just created before. But there is also another need around here which circulates around the decompiling of words. Up to here, the debug-ext will contain a large table of all known runtime-vector values and associate it with the C code to decompile its parameter area, including the colonwords. Using this new scheme, the moduleloader has the chance to see new runtime vectors, and register them dynamically. This is not done up to now, but it will be used in the 33.x generation.

In the 32.x generation, the style of the runtime implementation has also changed, although the old style is still supported. The traditional scheme for the forth systems is the use of a word-pointer, short WP, that is either an explicit variable in the inner interpreter, or it can be fetched indirectly by looking for IP[-1][] (the inner interpreter will fetch the current value from IP and increment it. Then the execution-token is executed by jumping indirectly. The value IP[-1] points to the CFA of the current word, adding one cell lets us see the PFA of the current word executed in the inner interpreter).

To access the parameter values, one can simply use the WP macros and address it with a normal C-style array index. A typical runtime would therefore look like:

 FCode(p4_constant_RT)
 {
   FX_PUSH( P4_WP_PFA[0] );
 }

To make it easier to support native-cpu sbr-threading and portable call-thereading, there is a change about here, since for either of thse we can not just fetch the IP[-1] to get at the wordpointer, nor is the latest value be fetchable from the cpu register in sbr-threading mode (atleast not without some assembler snippets). Instead of the assumption of a global wordpointer (either explicit or implicit through IP), we create a local wordpointer in the runtime-definition and a new macro can be used to capsule the needed setup-code. The new style looks like:

 FCode (p4_constant_RT)
 {
    FX_POP_RT_BODY_(pfa);
    FX_PUSH( pfa[0] );
 }

but for a single fetch, the same source code can be written a bit shorter - which will actually resolve the same code as with P4_WP_PFA, see here:

 FCode (p4_constant_RT)
 {
    FX_PUSH( FX_POP_RT_BODY );
 }

The base to call this macro something like _POP_ has a simple reason that lies in the call-threading model. Since a colonword will be made up of pointer to C-code (instead of pointers to pointers of C-code as is in the indirect-threaded model), there is no easy way to get at the address of the parameter field - unless one would use direct threading that would jump directly into a copy of native-code in the codefield of each word, and that native-code snippet would be required to set up the wordpointer then. For call-threading however we jump directly into the C-compiler generated routine, so that DATA and CODE are fully seperated in different segments, with the possibly of an unwriteable CODE segment.

To get the parameterfield address, we have to add that one explicitly into the colonword - each word that needs to access parameters will be compiled with two cells in a call-threaded colondef, the first one is the runtime-vector and the second the parameter-vector. The inner interpreter will fetch the runtime-vector, increment the IP (instruction pointer of the current colondef), and jump directly in the C code. The runtime C code will then have to fetch the parameter-vector and thereby adjust the IP to point to the next runtime-vector following the current tuple. This would not needed to be done for primitives, and well, that's what the name comes from - primitives don't have a parameter field.

Unlike the traditional indirect-threaded forth, the codefield of words in call-threaded mode do not contain a code address, instead they point to a code info-block which could actually be just the same as the info-record that is also available in the WORDSET table to export definitions. The executions done in the indirect-threaded listloader will simply be postponed to compile time.

Well, the call-threading mode of this style is not very consise w.r.t. to the memory consumption - each call-threaded colondef would get compiled as two cells, one for the colondef-runtime and one parameter-vector point to the list of exec-vectors that make upt that definition. Only the primitives being compiled from C source would be one cell entries. However this restriction can be lifted when going from call-threaded colondefs to sbr-threaded colondefs for the cpu architectures that we know about. Each runtime-vector would simply be preceded with the cpu-native code for call-subroutine, and the complete colondef would then be a native-code primitive in the end that does not need a parameter vector when compiled. Unlike direct-threading forth systems, just two native-code bitpatterns must be discovered to make it work - sbr-call and sbr-return. The rest would be just native-code optimizations.

The PFE has a generic extension system that a programmer can use to write his own forth extension in C. The lower C-level part is described on the page "how to write a PFE module". This external module contains a wordset loadlist table, and in fact the same tables are used internally in PFE. Traditionally, in PFE one did load the external modules via LOADM like

 LOADM gforth

however this will only work if the resp. module gforth.so had been living in a real external module. On some platforms however there is no good binary module provided by the operating system, so that the external modules are pre-linked to the PFE and all that is needed is to make the wordset entries available to the PFE runtime. This is not just an activation, the wordset table can have contructor instructions that can heavily modify the current PFE process instance.

The pre-linking does not bind anything of the external module to the basic PFE runtime system, so that (using the resp. LGPL exception rule about relinkable process binaries) these modules do not fall under LGPL themselves. The only reference is made in dl-internal to reference the data-like wordset-table of the extra module, and the table is searched later on, it is not activated in the default boot phase of PFE.

Apart from using this scheme for system that do not support a good dynaload system for binary modules, it can also be used for optimization of the system - for instance the zchar-ext wordset is usually pre-linked to the PFE core library. It implements the zero-terminated strings that are widely used in C, and it brings about the backslashstring definitions that are needed to use C-like strings with embedded newlines and other control-characters.

To make it easier to provide for the different activation routines for non-core wordsets of PFE, the PFE of 31.x and higher is now using the EXTENSION-query mechanism to activate wordsets - this follows the guidelines of the ANS Forth standard to activate additinal wordsets. Forth applications shall check the environment for a specific wordset with a query. The wordset might not be present before this call, so that references to the words in that wordset will fail (with an `undefined` exceptioin), but after a query to the environment did return true, it must stay that way and a wordset shall not be unloaded/deactivated thereafter.

The PFE adopts this mechanism with some small restrictions. First of all, the ENVIRONMENT may contain a lot more definitions than just wordset assertions - if a query is not yet defined in the ENVIRONMENT, it is hard to decide whether it is part of another wordset or a reference to a wordset assertion itself that should trigger an implicit LOADM call. Therefore, if an application writer wants the PFE to check for a wordset then it should always be done for the -EXT variant. If the resp. wordset is already loaded, it will return the value of its assert constant. Otherwise, the PFE will replace the "-EXT" part with the module file extension being used on the current operation system, and if there is such an external module, it will loaded. When loaded succesfully, the environment-query will check again whether the newest module has been pushing an assert constant into ENVIRONMENT. If yes, then this value is returned, if no then an assert constant is generated that contains null as the value but makes the next environment query to succeed.

This scheme is used for internal and external modules, it even applies to the floating wordset. Since 31.x, the floating wordset is normally compiled as a wordset module, and the basic pfe runtime system does not contain the floating definitions after boot. They have to be activated. Try the following lines to see what it does:

 : EXIST? IF ." EXIST=" .  ELSE ." NOT EXIST!" THEN CR ;

 s" floating" ENVIRONMENT? EXIST? \ no
 s" floating-ext" ENVIRONMENT? EXIST? \ yes, loaded on the fly
 s" floating" ENVIRONMENT? EXIST? \ yes, matches floating-ext 

Note that there is not even a floating-stack before the floating-ext wordset has been loaded, - it gets some memory from the forth dictionary via a constrctor, and it places a deconstructor, and it places a hook into the ABORT cleanup routine to have the floatingstack cleared when an error occurs.

The pfe searches its lib-directory under "$prefix/lib/pfe/" for available modules - as an example, the activation of the FLOATING-EXT wordset is looked for in the binary "$prefix/lib/pfe/floating.so". The main LGPL:ed pfe core does ship with a number of external modules, and with each new generation more modules become available. These modules use the C compiler to generate optimized native code for the primitives they provide, and they can be easily used to interface with any other C-level code.

Among the wordset modules that are currently available, the application programmer can use these for example:

this section applies up to and including the 31.x generation of pfe.

The ANSI standard on forth requires that a DOES> code changes the CFA-vector of the latest CREATE to the colon's token-list right after the DOES>. However, PFE does actually regard the value in the CFA as the address of an actuall C-routine, so it does simply jump there - the bytes at the DOES> address need to be actual cpu asm code, but PFE can not compile cpu specific code as it would require knowledge about the cpu beyond that of the C compiler.

Actually, the CFA of the last CREATE word is changed to a C routine in the standard .text body that shall start to execute the colon's token-list right after DOES>. However, if the CFA of CREATE points to a common routine the .text section, how shall the DODOES code know where to start execution?

In the forth-83 and earlier forth dialects, the address where to actually jump to was stored in the field directly following the CREATE address, i.e. in the first PFA cell. Therefore they used the word <BUILDS (defined as : <BUILDS CREATE 0 , ;) to ALLOT the extra field, and the sequence after DOES> did receive the address of CREATE plus one cell. The forth'94 standard does not allow this - the sequence after DOES> gets the actual PFA address, and <BUILDS is not defined in the standard - just use CREATE.

All non-asm implementation of forth do the same trick - they add the extra field before the CFA, in PFE called the AUX-field. It is only used by DOES-words but by its implementation issues it will be allocated for every word. The header size of a word in a non-asm implementation is therefore bigger than in in asm implementations of forth just for these DOES-words.

To sum it up - between the LINK-field and the CODE-field is an AUX-field. All forth languages base on C do this.

The forth'94 does not do any requirements on the structure of a name-field, it does not even need to exists. The forth'83 implementations had a atleast a header field, and the traditional structure of a name field from FIG-times is a counted string whose upper bits have been (mis-)used for the flag-bits. The highest bit marks it as the the flagfield (it is always set), followed by the immediate-bit and the smudge-bit. This leaves just 5 bit for the actual count of the name in the lower bits, and the names were hence limited to 31 chars.

However, 31 chars is clearly inappropriate for names more than twenty years later - the PFE is often used to interface to C-defined API and whereas old implementations of C had a 32-char limit too, they do not now anymore. The limit had to be raised for PFE, and instead of making extremely radical changes to the name-structure, the trick of F-PC was used - to move the the flags just before the count-byte of the name.

Due to some other implemenation issues (the link-to-name routine looks for the highbit that marks the flag-field) the limit is now 128 chars. As an advantage, the structure of the namefield is now a plain counted string without any flags mangled in. As always, the PFE allows to change back to the traditional ways, and in the source code a macro is used to enhance readability - "_FFA". And the other macro is called "NFACNT" which will return (*nfa&31) for the traditional mode to mask out the flag-bits from the count-value - in the default mode of pfe however, it is just (*nfa). Use these macros to calm down the #ifdef-noise.

To sum it up - the flags live in an extra byte just before the name field which is a normal counted string.

  structure Head                   structure Head
    1 chars field flags              1 chars field FFA
    x chars field name               x chars field NFA
    1 cells field link               1 cells field LFA
    1 cells field aux                1 cells field AUX
    1 cells field code               1 cells field CFA
    ( followed by body)              ( followed by PFA)
  endstructure                     endstructure

Using `configure` options, both the flags-field and the aux-field can be cut out again to arrive at a header-structure that is FIG-forth compatible. The use of `with-fig` will cut out both fields.

...

this section applies to PFE generation 32.x or later.

The first PFE generation took the option to add an extra aux field into the header of a PFE word, and each word in pfe had the exact same layout which did closely follow the FIG model in earlier times. Just the flag.byte was not folded anymore with the count.byte of the name.

As explained above, there is a fundamental problem between the FIG forth and ANS forth systems about the usage of <BUILDS and CREATE related to DOES words. The ANS Forth expects that the code-field of a CREATed word is modified to point to the DOES-field which then contains a trampolin to execute the colon XTs just following it. However, this is impossible to achieve with a traditional indirect-threaded model using no assumptions about the target processor ISA. That's why the PFE does follow the habit of FIG forth to place a second field near the CFA of the CREATEd word in ANS Forth, which is simply the address of the DOES code. Therefore, the DOES-address does effectivly not need to carry an additional code address.

However, there is a problem with compatibility about the second argument, as this one is neither the actual runtime code value of the CREATEd word and it is not data parameter value to be used by the execution of the word. It constitutes its own categorization as a helper field.

In FIG forth, this was different as one was supposed to use the word <BUILDS to CREATE new words that could be extended with DOES>, and the BUILDS word did boil down to be a combination of a basic CREATE followed by " 0 , " followed by DOES> to put its HERE value into the zero just following the CREATE header. The two words CFA and PFA were simply constant operators that did always add/substract a CELL which is simply the size of the runtime time vector in an indirected forth implementation.

Well, there is a problem attached to this mode, since there might be words that want to adjust the first parameter for the DOES code to see, however the PFA[0] was occupied by the DOES-vector, so the first real parameter for the DOES-code would be in PFA[1]. The FIG programs knew this, so they added another CELL for those words being derived via a <BUILDS/DOES> creation. But - ANS Forth does explicitly disallow this, <BUILDS is identical to CREATE here, and the word >BODY is supposed to convert a CFA (code-field address) into a PFA (parameter-field address), not matter if that is a VARIABLE or a DOES word.

There are effectivly three solutions, each having its own derived problems:

The PFE up to 31.x did use the second option, one-plus-one, which is also the model of many other portable forth systems, e.g. gforth. Since generation 32.x, we use the third model with a variable to-body execution. Among the advantages is the result of some internal code cleanup - so far the FIG variant of forth did have another header layout of the words, so there were some defines and ifdefs needed in the PFE forth implementation. With the non-const to-body variant however, the FIG and ANS header layout is identical, so it makes it easier to port FIG forth sources to the ANS mode of PFE, and it even opens the possibility to mix ANS-converted code with non-converted code, as it just boils down to replace a wordlist on the search-order. The binary result is identical (unless the flag.byte is used, which is an extra option however).

This advantage of the internal binary layout being identical is countered by some disadvantages. First of all, the ANS forth programs may not have made an assumption that the offset of the parameter field is always the same for both variables and does-words. They must have used >BODY in all places, and not memorized the offset of this word somewhere, as if the >BODY is a constant function. So far, there was no incident about this, and the people on comp.lang.forth did give their consent to this model as being fair - perhaps some native code forth systems with direct threading had been using variable size of the code field anyway, to lower memory consumption - that's because in direct threading, the code-field contains native cpu code being either just the complete runtime or a trampolin into an external runtime code. For a few things like a variable-runtime, this can be quite short for may cpu ISAs.

Secondly, the non-const nature of the to-body execution implies some runtime restrictions. First of all, it definitly needs an if-code, making it a bit slower, and the if-value to be checked is the first cell of the CFA on which to decide what size the complete codefield has, and to add this size of the codefield. The latter implies that there is an additional memory access, which is not only slower than a constant to-body offsetword, but constitutes the availability of a SIGBUS in to-body for bad addresses. The value supplied to that to-body must be a valid CFA, and nothing else. (one can however add some backward compatilibity to check for zero. It returns the size of the variable and colonword codefield).

And thirdly, there is some confusion about CREATE - in ANS Forth we must ensure that a DOES vector can be attached, so it must have the two fields ready, while in FIG Forth mode just one cell is needed, and only <BUILDS puts the two-cell type of thing. To get around this, the word CREATE is only a SYNONYM - in an ANS-Forth section, it will resolve to <BUILDS, and show this word on decompilation. The forth application writers are furthermore encouraged to use this word directly in their sources, and add a section like

  [UNDEFINED] <BUILDS [IF] : <BUILDS CREATE ; [THEN]
  

somewhere in the header of their applications for portability reasons. In a FIG Forth section, the CREATE synonym will resolve to CREATE: which makes a word that is not supposed to be extended with some DOES code later on.

The use of a variable codefield size opens new ways to create optimized code (and codefield areas) for use in native-threading systems, and it seems to be a feature well covered by ANS Forth rules. At the same time, the default pfe internal layout does exactly match that of the FIG, so it makes it a lot easier to convert legacy fig applications to the ANS Forth world. This mode is also recommended to all forth system implementors - do not anymore use the does-field model as it was used in pfe up to 31.x, and adapt your applications to expect the minor restrictions about the usage of words in this mode. It does even save you a few cells in the memory footprint.

The ANS Forth provides an extra wordset dedicated to LOCALS which happens to be critized by about every forth developer around. The problem is the inverse order of declarations compared with the normal stack comment. So, if your word expects three values as input parameters, you would usually declare it with a stack comment like this:

 : MY-WORD ( a b c -- x )
     whatever instructions
 ;

however if you want these three values to end up in named places on the return-stack, you have to write this one with the following LOCALS syntax:

 : MY-WORD LOCALS| c b a |  ( a b c -- x )
   whatever instructions
 ;

To understand this scheme, we have to look at the underlying word (LOCAL) which is also provided in the ANS Forth LOCALS EXT wordset. This word is not quite useful for the user, it is meant to be used by compiling words like LVALUE to declare new LOCALS to the LOCALS machine in the system. The PFE has such an LVALUE word that declares the following name to be a new LOCAL, and it initializes it in the course. To declare three values, we would have to write this:

 : MY-WORD ( a b c -- x )
   LVALUE c
   LVALUE b
   LVALUE a
   whatever instructions
 ;

and of course, this order is necessary since the first value is the one on top of the stack being the rightmost in the stack comment declaration. The ANS Forth LOCALS|-word does simply follow this order, making the name "c" to be in the first place and therefore leftmost in its simple syntax.

Please note that PFE does not follow the solution that some other forth systems implement - using another syntax extension that does the locals declaration in the "correct" order. However, a scheme like the John Hopkins Scheme adds a lot of complexity - the declaration must be parsed and reordered to match the internal order of initializations, and in order to provide LOCALS types other than single-value items, they must extend the parsing code even further. Instead, just skip that and follow the forthish style, using the simplest syntax that is still best to read - declare each local-value seperately.

One of the explicit locals has been describe above - called LVALUE to remind you of VALUE which has the same stack execution in declaring global values. The same has been done for the other local declarators - their names shall match the declarators that do the same thing for global items with a single "L" prepended. Currently, the following two local declarators are available in PFE

  LVALUE ( a [name] -- ) : [name] ( -- a )
  LBUFFER: ( size [name] -- ) : [name] ( -- addr-of-buffer )

feel free add more. I know one project that had a good use for LFCONSTANT but there is no floating extensions module so far - although I would like to include one if you have one written.

One nice side effect of explicit Locals is the availability of init computations that you can not have with any of the compound declarations. Have a look at this example to get an idea:

  : SAVE-BUFFER-TO-FILE ( buffer-addr buffer-len file-name -- done? )
     COUNT R/W OPEN-FILE IF DROP FALSE EXIT THEN
     LVALUE FILE-ID
     LVALUE BUFFER-LEN
     LVALUE BUFFER-ADDR
     whatever needs to be done else
     BUFFER-ADDR BUFFER-LEN FILE-ID WRITE-FILE
     and so on
  ;

well, this program does nothing so complex that it would require the use of locals to enhance readibility and maintainability. Just get the idea that we did more that the pure ANS Forth locals can provide for. However, be aware that PFE does not do sanity checks so far - putting an LVALUE inside a LOOP would not be that wrong (it would just reinitialize the value at that point over and over again) but the size argument to LBUFFER: is taken at runtime, and the needed space is carved from the return stack just then. Running this one inside a LOOP or mixing such a declarator with words that affect the return-stack (like >R or R>) could lead to undesired effects that are not warned for at compile time.

Looking close at LBUFFER: a C programmer could compare the functionality to that of alloca(). The declared name is actually a simpel LVALUE that will be assigned space carved dynamically form the return-stack - and automatically FREEd on any EXIT point in the function since the LOCALS frame is given back. This can be used to change the habit of many programmers to write non-reentrant forth words whenever a local buffer is needed, for instance to combine a file name with a prefix in a local string buffer.

Looking at the ANS Forth standard, one will find another reason for this habit since the standard does not require the return-stack of forth to be very large - that was done so the complete return-stack could be left in some hardware on special Forth CPUs - the threading model of forth does benefit a lot from call/return optimizations on the CPU, making it a lot faster. And if you look through the ANS Forth documents, you will see that there is no requirement for RP@ to exist, or that any item on the return-stack has an address. All this is purely optional. However - all the forth system running inside a host OS on general-purpose CPUs will provide a proper return-stack plus and the availability to take references of the values put there.

There is another way to help around the problem of a temporary string buffer to do some concatenations and splitting - it is traditionally called a POCKET. The first forth systems had just one such global string buffer that was reused in every word. Of course, that could easily to lead to problems with a word using it and calling a word thereafter that uses the POCKET too. Plus you can not have two string buffers with a single POCKET.

That's why quite some forth systems started to provide more than one POCKET, and a developped system like PFE provides a word to automatically access each temporary buffer in a round-robin fashion to lower the potential problems of called-words to overwrite the current buffer. Be aware that this can still happen here, and that many of the PFE system words do actually use the dynamic POCKETs internally to lower the return-stack pressure.

The word to get a new POCKET is simply called POCKET-PAD to remined you that the returned pocket buffer is not exclusive to the function requiring it. And it is sometimes a good idea to put this address directly into a LOCAL value to have name for it. It will look like this:

  : MY-WORD ( filename -- )
    POCKET-PAD LVALUE filename
    S" /prefix/path" filename PLACE
    count filename +PLACE
    (USE-WORD)
  ;

and please notice how used the pocket-pad for just a short time to construct an argument to (USE-WORD). This is the recommended use, since it is hard to guess what some of the called words in MY-WORD will actually do, and whether they request POCKETs too, and finally getting hold of the same one that was used here. In most of the forth programs one can come up with a good expected value of POCKET allocations in lower words however, and a pocket-pad does lower memory pressure quite a bit. And a POCKET method scheme can be easily ported to systems that have a non-addressable hardware return-stack.

The ansforth'94 standard describes a word to detect features of the forth environment. Forth scripts can use these hints to check whether the forth system is appropriate, and in some cases the script can try to use a different compilation path. However, the forth standard does not describe a way to extend the environment-hints - the 'environment?' word can only return the values. This is preventing a forthscript library to implement missing features, and advertise the newly gained system feature to independent application programs.

In december 2000, the developers of gforth (Anton Ertl), win32forth (Tom Zimmer) and pfe (Guido Draheim) agreed on a common approach for environment-query extensions based on a proper vocabulary named 'environment'. The implementation of 'environment?'-queries is based on a 'search-wordlist'-call. If the search fails, 'false' is returned, if it succeeds, the returned execution-token is 'execute'd and a 'true' flag is placed above.

 : ENVIRONMENT?
   ['] ENVIRONMENT >WORDLIST SEARCH-WORDLIST
   IF  EXECUTE TRUE ELSE  FALSE THEN ;

The environment hints themselves are implemented by adding word definitions to this vocabulary, in most cases these are 'constant' or '2constant' definitions.

 also environment definitions
 1994 constant locals-ext
 false constant /floored
 32 constant max-files
 -1 -1 2constant max-ud
 previous definitions

This implementation strategy allows also to place internal variable/values into this vocabulary that can be easily referenced in the actual implementation by simply placing the environment-vocabulary in the search-order (by using the name) and to let the outer interpreter see words like '/hold' and 'max-files' directly. In conjunction with the tool-ext's 'words', the user can get an overview which system features are available.

 environment words ( returns: )
  C GFORTH-DIR              C SYSTEM-EXT              C CLK_TCK 
  p PFE-DEBUG               C FORTH-83                p PFE-VERSION 
  p LICENSE                 p HOST-SYSTEM             C ENVIRON-EXT 
  C STRING-EXT              C TOOLS-EXT               C MEMORY-ALLOC-EXT 
  C #LOCALS                 C LOCALS-EXT              p MAX-FLOAT 
  p FLOATING-STACK          C FLOATING-EXT            p MAX-FILES 
  C FILE-EXT                C FACILITY-EXT            C EXCEPTION-EXT 
  p MAX-UD                  p MAX-D                   C DOUBLE-EXT 
  C BLOCK-EXT               p RETURN-STACK-CELLS      p STACK-CELLS 
  C MAX-U                   C MAX-N                   C MAX-CHAR 
  C FLOORED                 C ADDRESS-UNIT-BITS       C /PAD 
  C /HOLD                   C /COUNTED-STRING         C CORE-EXT 
  C CHAIN-WORDLISTS         C WORDLISTS               C SEARCH-ORDER-EXT ok 

In pfe, the reduced question is also fullfilled by an "-EXT"-hint in the environment-wordlist, i.e. a call like

 s" core" enviroment? .    -1 ok   

returns true since it matches the 'core-ext' hint. This is a kind of convenvience behaviour.

Another issue in forthscripted librarysections is the question whether a specific word is already present. The ansforth'94 standard describes the precompiler-like [IF]-[ELSE]-[THEN] words, however it is not described how to reliably deduce flags for these. A discussion on comp.lang.forth revealed, that even the set of 'find' and 'search-wordlist' is not so strictly standardized to accomodate the need for these ifdef-words.

Over time, every forth system has implemented means for a precompiler-type ifdef-section. In pfe, the early tek-mforth ifdefs are present in a seperate cdecl-ext wordset that does implement a subset of a C-compiler's precompiler words, including '#ifdef-#else-#endif' and '#define'. Other forth system have implemented '[ifdef]'-words, however these words do all suffer from having no easily portable implementation - with the greatest problem that the standard-implementation of '[if]-[else]' must count [if]'s and [then]'s to detect proper nesting. A system which had no [ifdef]'s does not see a newly-defined [ifdef] in the execution of [if].

Therefore the usage of [ifdef] should be depracated. Instead an independent word should be used to construct the flag for [if], and from the reference to the Forth Programmers Handbook (FPH), many forth system have chosen to use the terms [defined] and [undefined]. Note that some systems still use 'defined' (which is not immediate) or 'defined?' or 'have?', or perhaps they have '[not]' instead of the '[undefined]'-call. However the pair [defined]/[undefined] has found widespread acceptance and it is therefore implemented in pfe along with the [void]-call to place a reliable immediate false-flag for usage by [if].

 [void] [if] 
    here can be any text as this if can not get true.
 [else]
    [undefined] rdrop [if] \ check for non-existance of a word
       [defined] r>drop [if] \ but found a synonym
         : rdrop postpone r>drop ; immediate
       [else]
         : rdrop r> r> drop >r ; \ this is not standard-conform
       [then]
    [then] \ the standard- bracket-if/then call is properly nested
 [then]

Although these words are not (yet) standard, with some knowledge about the target systems search-order implementation, a highlevel definition of [defined]/[undefined] can be created that does properly work with the system-provided [if]/[else] construct. About twothird of the mature forth systems do already have the [defined]/[undefined] pair, so that their usage is reasonably portable across systems, and application software can be assured to compile in future system implementations.

Older forth system have often used a single word for doing both, exec-mode and compile-mode. In exec-mode, it would leave the parsed-value of the following word on the stack, while in comp-mode it would compile an execution-token and the value in a way that the value would pop up at runtime again. The ansforth'94 does however prefer to define two words, one for exec-mode and the other for comp-mode, so they are not state-smart anymore. In some compiling-environments (with preprocessing functionalities) this may eventually be a nice win, however it does sometimes confuse the the average forth programmer. The programmer can not anymore use a word on the commandline, and when satisfied enclose it into a ":"-";"-pair to memorize it as a colon-sequence.

In pfe, some of the immediate or exec-only/comp-only words have been extended to be state-smart. Where the ansforth'94 will make the behaviour of the word rather undefined or not applicable, in pfe it is just state-smart, and it will hence behave in traditional ways. The traditional forth programmer has simply to watch out for the cases where a newly-invented word from the ansforth'94 standard has been overloaded instead of the traditional word which had somehow been defined to be just not state-smart.

  .( whatever ) \ not state-smart, immediate
  ." whatever ) \ state-smart, orginally comp-only
  '             \ not state-smart, not immediate
  ['] name      \ not state-smart, immediate & compiles always
  '> name       \ state-smart version
  char woo      \ not state-smart, not immediate
  [char] woo    \ not state-smart, immediate & compiles always
  ascii         \ state-smart, very common in forth systems
  control       \ also state-smart and very common. Subtracts 32.

CFA Indirect Threading is the usual way in PFE and the traditional variant being the only one compatible with the FIG model. A colon word is a list of CFA addresses where the CFA contains the address of a C subroutine and the BODY of this word directly following this Code-Field (where CFA has its name from, the body being referred to as the Parameter-Field (PFA).

 NFA       LFA      CFA       PFA
 +---------+--------+---------+-------
 | name[n] | link*  | code*   | data....
 +---------+--------+---------+-------
            nextNFA  C call    body(only for non-prims)

The DOES words will put a DOES-subroutine into the CFA of a CREATEd word - this DOES-subroutine will use the first PFA[0] field to get the indirect address of the real (modified) behaviour of the word. The DOES_CODE field exhibits itself as just another CFA directly followed by its DOES_BODY. The current TO_BODY will detect the situation for a DOES-word and return DOES_BODY when someone asks for the BODY of a DOES-word - which means to return the address of PFA[1] instead of PFA[0].

 NFA       LFA      CFA       PFA[0]   PFA[1]
 +---------+--------+---------+--------+-------
 | name[n] | link*  | code*   | does*  | data...
 +---------+--------+---------+--------+-------
            nextNFA  do-does()

Before the DEFER words (including DOER/MAKE) became part of the inner system, they were just DOES-words who stored the DEFERred execution vector in the DOES_BODY. Therefore, the current implementation will store the IS-vector in PFA[1] of a DEFER-word. This is in contrast with the behaviour of the TO-var word which will store its value into PFA[0].

 NFA       LFA      CFA       PFA[0]   PFA[1]
 +---------+--------+---------+--------+-------
 | name[n] | link*  | code*   | unused | target
 +---------+--------+---------+--------+-------
            nextNFA  do-defer()

Call-Threading is a newer model for execution. The colon word is a list of addresses of C subroutines. As long as theses are primitives (without a BODY area) it will obviously be as double as fast in execution since there is on memory access instead of two in the CFA Threading model - the latter would have to fetch the CFA from the colonword and the C subroutine address from the referenced Code-Field - while in Call-Threading it would just have to fetch the C subroutine address and use a normal indirect function call to get it executed.

However this would not quite allow parametric words other than in the Direct-Threading case - in Direct-Threading the BODY would be be prepended with a longer Code-Field which contains a complete subroutine as if assembled by a C subroutine compiler. Therefore in the Direct-Threading case, the inner interpreter could again just jump there as it would for primitives. Most of the Direct- Threading systems would just jump to the real common behaviour of the BODY-word - via the typical IP[-1] we still get the original CFA and can derive the PFA = IP[-1]+CodeFieldSize - where the CodeFieldSize is usuall larger than just a pointer since it would consist of cpu-native jump-instruction followed by the behaviour address compiled by the C compiler.

Well, this explains the model of Direct-Threading which would not be quite portable since it assumes knowledge about the cpu-native jump-instruction which would have to get pasted into the data-area of the application which gets then executed as code atleast for the CodeField part. On processors with different on-die caches for Data and Code, this would yield to considerable cache-thrashing, and some Forth implementations tried to overcome the problem by ensuring that the corresponding Data-area of a Direct-Threaded BODY-word gets put a bit away - with the distance being the cache-line length in the Instruction-Cache. Which of course assumes another set of knowledge for the underlying native cpu architecture. Therefore: Call-Threading is not Direct-Threading.

To enable BODY-words in Call-Threading we use another trick - for each BODY-word, an XT-compile ("compile,") will detect the situation and compile two addresses. The first one is the C-subroutine of the common behaviour of the BODY-word directly followed by the BODY-address of it. The behaviour-subroutine will then just have to fetch that BODY-address away so that the inner-interpreter will have its IP put onto the next real C-subroutine (and not a BODY-address). That's why you will see those POP_RT_BODY in each of the _RT-routines in the forth-implementation which will be a (*IP++) in Call-Threading model. Unlike the Direct-Threading model, the Call-Threading model is portable: the "compile," word will simply copy the address contained in the CFA to the new colonword and then add the BODY-address right thereafter.

  VARIABLE XX
  : AA XX @ ;

  in ITC mode (indirected threaded via CFA subroutine calls)
  | name... | link | code'do-colon | cfa'XX | cfa'@ | cfa'; |
    n bytes   LFA    CFA             cell     cell    cell
  
  in CTC mode (call threaded with conditional body pointer comma) 
  | name... | link | code'do-colon | code'do-var | pfa'XX | code'@ | code'; |
    n bytes   LFA    CFA             cell          cell     cell     cell

One could think that one would be able to mix FIG-style CFA-Threaded colonwords and PFE-style Call-Threaded colonwords - the outer interpreter could in fact do both and runtime switchable - we would just have to swap the behaviour of "compile," while the rest could effectly stay the same. However, we can not do this in full FIG-style mode - the outer interpreter's "compile," word does need an additional information about each word telling whether it is a primitive or a body-word. As a side note: of course, a simplistic approach would just always add a body-field for each call and the inner interpreter fetches the code-address and the data-address (even when the data-adress is just null). But this would simply make two memory-accesses for all words including primitives and there is no benefit over CFA-Threading here.

Instead, we use an extra bit in the FlagField of each word which gets set when the word is not-a-primitive. In the pfe C-level sourcecode you will see two words for creating a HEADER: FX_HEADER and FX_RUNTIME_HEADER where the latter is just a combination of FX_HEADER followed by FX_RUNTIME_WORD where this one will just set a bit in the LAST word and this bit being called P4xISxRUNTIME. - Now, the "compile," word will check the FlagField for ISxRUNTIME being set, and only in this case a BODY-address gets compiled. When no word has this FlagBit set then a Call-Threaded colonword will just be a list of C-subroutine addresses.

This makes it a very fast implementation - each primitive gets called with just an indirect-functioncall (instead of a double-indirect one), and only BODY-words will use more memory-accesses, which they would do anyway since they will surely access the data in their body. However, there is even another surplus - it makes it easier to have forth integrate with C-level projects since we are now able to construct new colonwords directly in C without the help of an inner interpreter or even "compile,". The C-level app would just have to create a list of function-pointers, adding body-address for each reference to an _RT-named routine. When the C-level-created forth-colonword has been created (as a ":noname"-colonword) the it can just be p4_call'ed.

(FIXME: stick note about implemenation of DOES-words and DEFER-words in here)

From aboves model for Call-Threading we can now derive the model for SBR-Threading - there is only one difference between the two here: in SBR-Threading each of the CALL-addresses gets prepended with the cpu-native bits for an SBR-functioncall. And be noted: on many RISC- machines these bits are mangled into the CALL-address itself, usually the upper four bits so that on a 32-bit machine only the lower 28-bit addresses are reachable for CODE jumps. On machines like the i386 architecture, the cpu-native SBR-functioncall is a single byte (9E) so the code-size will increase by another 25% (CFA-Threading vs. CALL-Threading had increased the executable size by about 20%).

This derivation is largely different than SBR-Threading derived from Direct-Threading - and it does more closely follow the nativecode scheme for creating BODY-words like accessing Variables and Constants. The nativecode would just put another cpu-native bit followed by the address of the Variable to fetch. This leaves room for a nativecode optimizer which could replace the sequence (call-nativecode + varfetch-routine + vardata-address) by an optimized sequence of (varfetch-nativecode + vardata-address).

The same optimizer can be used for DOES-words: the sequence of (call-nativcode + doescall-routine + doescode-address) could get replaced with (indirectcall-nativecode + doescode-address). The doesdata-field is still easily derived as it still follows the doescode-field.

And a note about DEFERred words - without Direct-Threading, a deferred word will be executed as a doublyindirect call and there is no cpu-nativecode for that. Actually, this is good since a) we want to have the DEFERred vector to be no in the first paramfield and b) we want to check for null in the defervector and warn instead of segfault (or call a routine that CRASHes).

  VARIABLE XX
  : AA XX @ ;

  in CTC mode (call threaded with conditional body pointer comma) 
  | name... | link | code'do-colon | code'do-var | pfa'XX | code'@ | code'; |
    n bytes   LFA    CFA (a cell)    cell          cell     cell     cell

  in SBR mode (subroutine threaded with cpu-native SBR assembler snippets)
  | name... | link | colon-info | SBR-setupcode | .....
    n bytes   LFA    to infoblock (nothing on x86, 16bytes on powerpc)

  .... SBR-call + code'do-var | pfa'XX | SBR-call + code'@ | SBR-exitcode |
       1 byte   + cell          cell    1 byte   + cell     1 byte (12 on ppc)
 

The above code will make do-var to fetch the following pfa'XX address from the colon-word with an extra call, thereby adjusting the subroutine return-adress to point to just after the body-arg. This will make some CPU variants picky where the L1 cache is divided into Code-cache and Data-cache. In CTC-derived SBR-threading it means alternating access as Code and Data, and some K6 are known to be panicking as soon as they see a Data-access of an address in the Code-L1-cache and do a complete flush of the Code-cache.

The solution uses arg-threading. Instead of the do-var code implementation to fetch the body-adress being compiled after its call, the SBR-ARG threading will compile an immediate arg-push before the call - which is actually the pfa-address prepended with an SBR-snippet to push it into the right place. The called do-var routine can then just use the value in place.

  in SBR-threading
  .... | to infoblock | .. | SBR-call + code'do-var | pfa'XX | ...
         cell           0    1 byte   + cell          cell

  in SBR-ARG-threading
  .... | to infoblock | .. | SBR-push + pfa'XX | SBR-call + code'do-var | ...
         cell           0    1 byte     cell     1 byte   + cell
 

Even that we consume yet another byte more, the code will actually run faster on many CPU types - but it depends on the actual implementation of the L1 cache. Another problem is to find a place to bring the value from the colon-routine into the do-var routine without the C compiler to trash the place innocently during its SBR setup. Using a register with the gcc fixed register allocation is one means and often a good one.

On a register-starved architecture like x86, it helps greatly to use the CPU-native SBR-code. These are very fast calls specialized to make a subroutine call. However an architecture like powerpc does not have any specialized subroutine-call instruction, and the return-stack can be referenced from any of its CPU registers.

The problem with the CPU-native SBR-code and RP-register has to do with the habit of forth programs to use to-RP and RP-from instructions to move values from and to the return-stack as temporary values. This will require the implementation of these routines to pop-off the return-address, push/pop the value, and put the return-address back. That can heavily interfere with what the C compiler assumes about the RP depth, not to speak of some CPU architectures to have a special return-address cache that will get flushed hereon.

Instead one make up a three-stack machine, where colon-word return-address are stored on another stack than the C compiler will assume them for. It makes it quite the same as the normal token-threading but the address-tokens are interpreted with some native-code snippet - just not the SBR-call snippet but some other - and it is still native-code SBR threading.

The ARG-register should be a scratch-register in the local ABI (application binary interface - the description about the structure of call-frames and the registers possibly affected by all into a routine) - if you choose a callee-saved register then it has the effect that the compiler will generate some code in the procedure header to save away that register to the return stack which is absolulty unncessary and possibly counterproductive.

About all cpu ABI do know some scratch register for local computations whose value must not be saved with a procedure and whose value may also not be assumed to be left untouched with an intervening function call. The latter limits the possible use of the value in that register and one should better check if the compiler is clever enough to save value around function calls - there are numourous bugs in compilers for such explicit register allocations.

When looking for a scratch register, the ABI's return register is often a good choice since all the forth words are void-routines. The return-register is generally considered to be touched by function calls, so it falls into the category of a real scratch register. However, the return-register is often the accumulator of traditional cpus, and in many cases that cpu ISA (instruction set architecture - the possible instructions and which registers they can take for operands) must take this register for some instructions to be possible. In most such cases, the displacement- register might be an alternative.

Looking at the current choice, the "%eax" accumulator register is chosen on i386 which actually works, so one does not need to look at the "%edx" register. For the m68k, the return-register "%d0" is not a good choice as the ARG to the routines is always a pointer that points to the actual data - the pointer is used to fetch data with constant displacements, and it is incremented by one cell in many cases - the m68k has some optimized opcodes for these address operations but which work on "%a*" registers only. The compiler would therefore move the "%d0" arg to the scratch "%a0" register for operations. The latter however is not a good choice either, since the compiler does want to use it for indirection function calls like "(*)()" on C level, and does not push it elsewhere. However, the "%a1" register showed to be also a scratch register in this ABI, and it works. Similar decision chains can be made for the powerpc ABI where the "r0" register is used for many hardcoded temporaries, and "r1" is used for the program return stack. Using "r2" succeeds here.

Be aware of the diffences between an ABI and an ISA for a given cpu - in the def-regs section, we do currently one differentiate by cpu-type, which does match with the ISA type. Most cpu makers hand out docs about the recommended ABI for their processor, and in most cases they are overtaken - which leads to a single matching ABI. But this one-to-one relationship can not be assured, see as an example the powerpc ISA which has no dedicated register for the return-stack (it does not even have a dedicated instruction for subroutine calls) and any of the 32 general registers can be used as the stack pointer for an ABI on top of this ISA.

A traditional CPU however has a dedicated CALL instruction, a register to save the return address, and a RTS instruction to return-from-subroutine. In this mode, it is best to alias the forth RP and the cpu RP - the current IP in the caller's routine can be deduced by looking at the value in the return-stack. This mode saves these two registers from being needed to be assigned globally.

The RP is declared locally within the USE_CODE_ADDR macro in the C-sources, and the RP value is given as the ARG to this routine (_XE routines in the sources). The current forth-RP is simply the value of the cpu-RP before the call instruction which pushes the IP on the very same stack. Within the _XE routine, the IP can therefore be modified and by using RP[-1] (in general, the offset is indeed just minus one - but most ISAs know some flavour of calls that saves more data to the stack, e.g. call-interrupt-handler).

In this mode, the RP-pointer itself must be left constant since otherwise the CPU will not find its return-adress on the stack. When some _XE routine needs to modify the RP, it must ensure to save the return-address (the IP value), then modify the RP (PUSH, POP, ROOM, DROP), and then restore the return-address as the uppermost value of the real cpu-RP. See the _NEW_RP macros around that handle this problem.

To change the IP alone, one can simply write to the value inside the call-frame - the next RTS will find that value.

As an example, take a look at this m68k assembler of the IF-execution which knows that directly after the IF-call a target-address follows which points to the resp ELSE. Here we chose the "%a5" register to hold the forth parameter-stack:

0000095e _p4_if_execution_:
     95e:       4a9d            tstl %a5@+
     960:       660a            bnes 96c
     962:       2069 fffc       moveal %a1@(-4),%a0
     966:       2350 fffc       movel %a0@,%a1@(-4)
     96a:       4e75            rts
     96c:       58a9 fffc       addql #4,%a1@(-4)
     970:       4e75            rts

reading it: pop the value and test it, if it is non-zero, jump to 96c which skips the target-address by advancing the IP by one cell. The %a1 register has been given as an argument and points into the real return-stack just below the return-address of this function - the add-instruction on "%a1@(-4)" is therefore identical with "%sp@" and you can read the code as "addql #4,%sp@" to modify the address that the following "rts" will jump to. The same accounts for the false-case - here we just fetch the value at the caller's IP which has the target-address, and then store this value into the IP so that the following "rts" will find the new value and BRANCH to it.

One might be tempted here to tell the compiler to just use "sp@" as the IP value, but this is only true if the compiler did not generate a local variable frame - in which case the "sp" would be well above the address of the return-address. The gcc supports the __builtin_return_address but it is not an lvalue, so it can not be modified nor can one take the address of it, and the other __builtin_frame_address is known to carry wrong values when compiled as -fomit-frame-pointer and the local function did not need a frame for real. Therefore, it is safest to just pass down the sp-value from just before the call to the routine and get the IP as RP[-1]. It just needs to put another opcode before the CALL-part to the _XE routine, representing "movl sp, a1" on m68k.

The powerpc architecture does no have a dedicated return-stack or instructions to call a subroutine or return from a subroutine. Instead it has a special Link Register (LR) and the subroutine calls are derived from the branch-instructions. The branch-instruction (with the L-bit set), will first copy the next code address to the LR and then jump to the target. The code at the target address will then be responsible to fetch the value from the LR and store it on the stack - in the presented case, this stack is referenced by the "r1" global register. The same is done on return-from-subroutine - the return-address is restored into the LR and then a "blr" = branch to address in link-register is compiled.

Therefore, the setup sequence for a new procedure is quite long and simulates the behaviour of a single CALL-instruction on traditional CISC architectures - the same happens as a simulation for the RTS-instruction. This is good practice but imposes a serious problem: how to define the IP. On traditional ISAs, the IP will always be right above the last good RP value, but in this mode, the return-address is nothing more than a local variable of the called routine.

Furthermore, even very simple functions will actually have a local-frame, where they will store the RP of the previous caller, so that the current cpu-RP value is not actually the pointer to the forth-RP data - again, some assumptions may break here if just adding a displacement or trying to fiddle with that __builtin_frame_address - and an RP-change is even more problematic as not just one cell must be saved before the change, but the whole locals-frame of the caller and the callee routine.

It took several hours of experimenting that there are always cases that it will fail to describe the forth-RP as an alias of the cpu-RP. Instead, it is best to not alias the two, and allocate a separate forth-RP as a global register allocation

It turns out that this makes the setup-code for a colon-routine even shorter and faster as we are not bound to the system ABI - in the example powerpc ABI it became visible that the system ABI did usually save the return-address into a place in the callers stack-frame - that simply means that such RP space must be reserved when calling any of the primitives compiled by the C compiler. Using an extra forth RP, we do not need to deal with these specialties, and keep the setup-code down to the minimum.

Since the forth-RP is an extra independent global register on the powerpc, another question must be answered - where to get the callers IP from. One way might be to give it as an ARG to the routine, in the place of the RP that would be normally pushed down, but it happens to be not necessary - instead of suppling some setup-code in the colon-routine just before the call into the forth-subroutine, the IP arg can be setup within the forth-subroutine by creating a copy of the special LR-register in a general-purpose register where it can be easily accessed and modified. The lead-out code then pastes move-arg-to-lr and branch-via-lr into the C-routine (you may notice some garbage left over in the assembler output since the C compiler will then still compile its own lead-out code containing another branch-via-lr).

To alias the forth-RP and the cpu-RP has the advantage of saving another cpu register for local computations - and on a register-starved ISA like intel-32bit, this happens to an essential requirement for good native speed. The contrary is true for an LR-type RISC cpu where the ABI's locals-frame interferes with the forth-operations on an RP-stack. A CPU with enough cpu general registers and traditional return-stack ISA might choose either way.

This package contains all neccessary ANSI-C source files to build a running Forth-environment on most UNIX machines, on DOS and on OS/2.

The Forth-system is closely modeled along the new American National Standard for the programming language Forth. With the exception of the assembler words I implemented every word of every word set mentioned in the dpANS-6 document of june 1993. Additionally it is compatible to Forth-83.

This set of source files is distributed under the GNU general public license for libraries. See the file <file>COPYING.LIB</file> for conditions.

I chose that one to point out that I don't consider programs you write on top of it a `derived work' of the portable Forth environment. To violate these conditions you have to do two things together:

Should be easy to avoid.

For fun. As an excercise in unix programming. And there was no such thing. See below, design objectives.

With two elaborate standards at hand, one for C, one for Forth, it should be possible to build one language in terms of the other and thus provide both where at least one is available.

While I leave the writing of an ANSI-C compiler in Forth to those who really believe in Forth's superiority and universality, I concentrated on the manageable part: Providing a Forth-system in ANSI-C that is

Maybe you miss the design objective SPEED. It was not my goal to provide the fastest C-based Forth-environment. This would have led to conflicts with more important goals. We all have fast computers, haven't we?

After all pfe isn't slow. With a little tuning using GNU-C's global register variable feature it is pretty fast. On an ix86-based system it seems to run with about two thirds the speed of a direct threaded assembler Forth (eforth) i.e. benchmarks take 50% more time.

Did I achieve the above objectives? Some of them. I'll continue -- slowly -- working on it.

The system is in use for eighteen month now. Several quite obvious and several quite subtle bugs have been fixed. While there surely are some more, they are not as obvious since the system passes several test programs, some of them rather sophisticated.

Once you get the system running, you'll have

Thus you'll be able to edit, compile an run programs in a moderately comfortable way. For the final design of the development environment your suggestions still are welcome!

If you try this system, please keep in mind that it is still under development. Sometimes new --even stupid-- bugs are introduced while enhancing functionality or while fixing old ones. I appreciate every hint to a bug and I fixed every bug I've been told about in the last months. So please don't hesitate to tell me about whatever seems wrong. Please check for the latest version via anonymous ftp from

	roxi.rz.fht-mannheim.de:/pub/languages/forth/pfe-?.?.?.tar.gz
 

(accessible from germany only) or

	sunsite.unc.edu:/pub/linux/devel/forth/pfe-?.?.?.tar.gz
 

Please send suggestions and bug reports via e-mail to

	duz@roxi.rz.fht-mannheim.de
 

UPDATE: The new PFE can be found at http://pfe.sourceforge.net which is maintained by Guido Draheim, send suggestions and reports via e-mail to guidod@gmx.de (the new website from Dirk-Zoller can be found at www.dirk-zoller.de)

For installation refer to the files `INSTALL' and `TUNING'. (update: the new pfe uses autoconf/automake, i.e. gnu-style)

Once you have it running and see the "ok" prompt after typing return you can interactively type in forth words. If you mistype, you can edit the command line and recall old command lines with the arrow keys. (If you can't then your termcap doesn't work all right and you can resort to wordstar-like control keys.)

To write some more statements try "EDIT-TEXT filename". This will invoke your favorite text-file editor on the given file. If it doesn't, first check the environment variable EDITOR, then check the file "const.h" for the #defined symbol "EDITOR". (update: the new pfe has the editor not builtin by default)

Having written some code you can load it by "INCLUDE filename".

If you prefer the old style block files, give a file to use as block-file with the -b commandline option. Alternatively you can say `USING filename' or if the file doesn't exist yet: `USING-NEW filename'. Then you can edit a block by `n EDIT-BLOCK'.

If your termcap-mechanism works well, the arrow keys and some other function keys should be active. Quit the editor with ^U and load blocks with "n LOAD". If it doesn't work well you might not even get a picture.

For more commandline options try the option -h.

The interrupt key is remapped to ^U (normally it's ^C) and leads back to the FORTH input loop. Use it to break out of infinite loops. If you find the ^U annoying see src/options.h for how to change it. I needed the ^C for the Wordstar look-and-feel.

To terminate the system, type BYE at the command-prompt or press the keyboard quit key of your system (usually ^\).

I started writing a documentation in texinfo format. This will allow you to view it online or print it in good quality. There's also an outdated and hopefully soon superseded man-page that explains some of the command line options. All documentation is highly unfinished. (update: the new pfe uses docbook format to achieve the same goals)

For more information please try to get the dpANS-document, which is an EXCELLENT REFERENCE to this system! You can (could?) ftp it at ftp.uu.net in the directory /vendor/minerva/x3j14.

Recently the ANSI standard (or the last draft) bacame available as Word document and in HTML format. From a post of Norm Smith to c.l.f:

  "I have uploaded a new HTML version of the DPANS94 document
   to Taygeta.oc.nps.navy.mil. The contents are identical to the
   original posting. The new version, V2.0, has many more hyperlinks.
   Specifically, all (well, at least most :-) of the See: references
   are now linked. Anchors have been added for each section in
   my working copy. This will faculitate filling in many additional
   links in the next version of the document."
 

(update: the best place to view the dpans is the FIG's website at forth.org - see forth.org/dpans/dpans.html htmlpage)

are welcome! After the kernel is finished now it makes much sense to share the burden of creating and improving a well rounded programming environment for all tasks a Forth programmer wants to do. PFE is YOUR tool. Get involved in it's design!

There is a mailing list on pfe which you should subscribe to if you want to be up to date with the development of the system. Send e-mail to duz@roxi.rz.fht-mannheim.de if you want to subscribe to that mailing list. (update: there is a new maintainer and a new mailinglist at pfe.sourceforge.net )

I want to express my gratitude to the people who put their efforts in the precise descriptions I found in these documents:

Thanks for providing superb development tools:

Several nice people on the net continuously gave me valuable hints to bugs and possible improvements, were patient enough to try the very first releases, made ports to machines I never saw myself and thus kept me from frustration. If pfe is stable now it is thanks to it's users. Most notably:

   Lennert Benschop, Sean Conner, Holger Dietze, Kevin Haddock,
   Rob Hooft, Giorgio Richelli, Marko Teiste, Guenther Thomsen.
 

Thank You.

In the traditional pfe and in the case of an ABORT (or any other THROW that is not CATCHed), the simple approach was use that the current input-buffer was printed and a single "^" did mark the current position of the ">IN" pointer, which is the index within the input-buffer where the next PARSE will start off.

However, the gforth showed that it is better for the user to see the last WORD being PARSEd last from the input - that is, the whole last charlenght shall be underlined. That is almost simple since there is an internal SPAN variable in forth that gives us the length of the last PARSE value, and this SPAN variable is not transient (temporary in a way).

Now, what's the problem - the outer interpreter will PARSE the next word from the input buffer using a BL space. The PARSE will read from the input buffer until it hits either that space OR the end of the input buffer. If there is a space then the ">IN" index will be left pointing to the point one beyond that space, that is two chars beyond the last valid character of the WORD just parsed out. However, if the PARSE hits the end of the input-area, the ">IN" pointer will be just after the end of the input-area, that is one char beyond the last valid character of the word just parsed out.

Now, how to detect the difference - the difference whether an additional delimiter has been parsed away that was not seen by the word calling PARSE (or any other special purpose parsing function) and which therefore does not add to the SPAN returned. The answer is that it is very hard - we could add the heuristic that at the end of the input buffer we just add one, but there are very different input SOURCEs each having a different idea of the actual end (think of a block file that is read in one go), and it we would have to check the buffer whether there might be a single delimiter just before the actual end.

So actually, the real answer would be to place additional variables to the parsing functions that can be checked at the next THROW. However, it simply means to slow down the parsing functions a bit, and all of them must follow this standard to publish what they've been returning from the input buffer.

Here I took the decision to not add either of additional complexity in the internal parsing or execption facilities. Instead, (at an ABORT) the underline of the last PARSE simply shows SPAN+1 characters - inside the input buffer, that will include the delimiter just after the last WORD, and if we had been at the end of the input buffer, it will underline the word plus the delimiter just before the last WORD. That is good enough to give the user an idea what characters in the input buffer had been affected in the last PARSE - a human mind can easily subtract addtional delimiters before and after such a word, which is usually a space or a doublequote.

In the end, the PFE does now show the complete SPAN of the last PARSEd word without adding much of additional algorithmic complexity to the forth engine. That keeps the PFE code readable and maintainable at the cost of users who sometimes wonder why the space after word is underlined too. Well, now you know...

There is a difference between an include-file that is given as the bootscript to the pfe binary via the shell commandline, and the loading of the same file via "INCLUDE >file<" directly from the forth commandline. This is most obvious with the ORDER that is forgotten when being set in a bootscript as soon as it hits the the mainloop - when the bootscript has finished, the inner endless loop is setup and will run first through the QUIT-word and ABORT-word initialization sequences which includes a RESET-ORDER.

To give you an idea, here is what is going on: first the the forth dictionary is allocated, then it gets initialized from the wordsets being compiled into the main forth binary, then the bootimage is loaded, then the bootscript is loaded, then we look for APPLICATION and jump into it. The default APPLICATION is the forth-internal MAINLOOP. The mainloop does build a CATCH-domain around the traditional forth-routine QUIT. Calling QUIT will first go through some initializations that will ensure that the forth machine is in a sane state even when it is run again explictly or implictly from an ABORT that did end in the CATCH-domain of the MAINLOOP.

That's why you can not leave values from the bootscript on the parameter stack - they will be lost. There can not be any files left open from the bootscript either, they will be closed automatically. While that does not hurt many people, the case about the order-reset can be problematic in some cases as it might change the parsing behaviour of the outer interpreter as soon as the mainloop's QUIT is reached. To get around this, you can use the PFE-specific word DEFAULT-ORDER to explictly set the order that RESET-ORDER will see.

Shall we implicitly do a DEFAULT-ORDER after loading the bootscript? That might help us but in the real world there are far too many forth scripts that change the ORDER via a lot of "ALSO" additions starting off with an "ONLY FORTH" but they never call any "PREVIOUS" at the end as they expect that the next module script will do the same and start off at "ONLY FORTH". Furthermore, there is a basic differene between a bootscript and a normal include-file since the APPLICATION variable is only checked after the bootscript but not after each include-file. In a way the include-script on the shell commandline will not be loaded through INCLUDE but through BOOTFROM which builds on top of INCLUDE.

Just make sure that an application bootscript is ready to take this difference - one of the differences is also that any error in a bootscript will let the PFE instance simply die since there is no CATCH-domain inside an endless loop, and it is up to the APPLICATION to create a CATCH-domain and a PARSE-loop if needed. It could be that a forth program is only used as a filter for some input from the shell commandline so that it shall never reach the forth commandline. That is up to the bootscript to flag (well, you can also use the comandline option "-y"/"--bye" to let APPLICATION point to BYE instead of MAINLOOP).

After all - the forthscript on the shell commandline is not exactly the same as loading the same file from within the forth commandline. In general, there should be no difference, in other cases it might lead to some unexpected differences. Now you know why...

P.S. in the Tek/MPT embedded variant, there is no use for a PFE that quits immediatly, in fact it doesn't know BYE, it can just do a COLD reboot. Here the bootscript is sent to the PFE process via putting the resp. forth commands into the terminal input queue, so it is seen within the endless-loop, and all scripts will make use of the PFE-specific DEFAULT-ORDER word to ensure a good state at an ABORT.

COLD does not reset ORDER ... A problem that existed since a change in 0.32.x and which should go away. It has to do with COLD going not deep enough due to the bootscript mechanics vs include-files.

Call-threading has no debugger ... new to 0.32.x are some more threading modes including SBR-threading. While the latter is largely unstable and not available on all platforms, the CTC threading however is completly portable and should be supported by a debugger too.

Version 2.1, February 1999

(The master copy of this license lives on the GNU website.)

Copyright (C) 1991, 1999 Free Software Foundation, Inc.
59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.

[This is the first released version of the Lesser GPL.  It also counts
 as the successor of the GNU Library Public License, version 2, hence
 the version number 2.1.]

The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public Licenses are intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users.

This license, the Lesser General Public License, applies to some specially designated software packages--typically libraries--of the Free Software Foundation and other authors who decide to use it. You can use it too, but we suggest you first think carefully about whether this license or the ordinary General Public License is the better strategy to use in any particular case, based on the explanations below.

When we speak of free software, we are referring to freedom of use, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish); that you receive source code or can get it if you want it; that you can change the software and use pieces of it in new free programs; and that you are informed that you can do these things.

To protect your rights, we need to make restrictions that forbid distributors to deny you these rights or to ask you to surrender these rights. These restrictions translate to certain responsibilities for you if you distribute copies of the library or if you modify it.

For example, if you distribute copies of the library, whether gratis or for a fee, you must give the recipients all the rights that we gave you. You must make sure that they, too, receive or can get the source code. If you link other code with the library, you must provide complete object files to the recipients, so that they can relink them with the library after making changes to the library and recompiling it. And you must show them these terms so they know their rights.

We protect your rights with a two-step method: (1) we copyright the library, and (2) we offer you this license, which gives you legal permission to copy, distribute and/or modify the library.

To protect each distributor, we want to make it very clear that there is no warranty for the free library. Also, if the library is modified by someone else and passed on, the recipients should know that what they have is not the original version, so that the original author's reputation will not be affected by problems that might be introduced by others.

Finally, software patents pose a constant threat to the existence of any free program. We wish to make sure that a company cannot effectively restrict the users of a free program by obtaining a restrictive license from a patent holder. Therefore, we insist that any patent license obtained for a version of the library must be consistent with the full freedom of use specified in this license.

Most GNU software, including some libraries, is covered by the ordinary GNU General Public License. This license, the GNU Lesser General Public License, applies to certain designated libraries, and is quite different from the ordinary General Public License. We use this license for certain libraries in order to permit linking those libraries into non-free programs.

When a program is linked with a library, whether statically or using a shared library, the combination of the two is legally speaking a combined work, a derivative of the original library. The ordinary General Public License therefore permits such linking only if the entire combination fits its criteria of freedom. The Lesser General Public License permits more lax criteria for linking other code with the library.

We call this license the "Lesser" General Public License because it does Less to protect the user's freedom than the ordinary General Public License. It also provides other free software developers Less of an advantage over competing non-free programs. These disadvantages are the reason we use the ordinary General Public License for many libraries. However, the Lesser license provides advantages in certain special circumstances.

For example, on rare occasions, there may be a special need to encourage the widest possible use of a certain library, so that it becomes a de-facto standard. To achieve this, non-free programs must be allowed to use the library. A more frequent case is that a free library does the same job as widely used non-free libraries. In this case, there is little to gain by limiting the free library to free software only, so we use the Lesser General Public License.

In other cases, permission to use a particular library in non-free programs enables a greater number of people to use a large body of free software. For example, permission to use the GNU C Library in non-free programs enables many more people to use the whole GNU operating system, as well as its variant, the GNU/Linux operating system.

Although the Lesser General Public License is Less protective of the users' freedom, it does ensure that the user of a program that is linked with the Library has the freedom and the wherewithal to run that program using a modified version of the Library.

The precise terms and conditions for copying, distribution and modification follow. Pay close attention to the difference between a "work based on the library" and a "work that uses the library". The former contains code derived from the library, whereas the latter must be combined with the library in order to run.

0. This License Agreement applies to any software library or other program which contains a notice placed by the copyright holder or other authorized party saying it may be distributed under the terms of this Lesser General Public License (also called "this License"). Each licensee is addressed as "you".

A "library" means a collection of software functions and/or data prepared so as to be conveniently linked with application programs (which use some of those functions and data) to form executables.

The "Library", below, refers to any such software library or work which has been distributed under these terms. A "work based on the Library" means either the Library or any derivative work under copyright law: that is to say, a work containing the Library or a portion of it, either verbatim or with modifications and/or translated straightforwardly into another language. (Hereinafter, translation is included without limitation in the term "modification".)

"Source code" for a work means the preferred form of the work for making modifications to it. For a library, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the library.

Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running a program using the Library is not restricted, and output from such a program is covered only if its contents constitute a work based on the Library (independent of the use of the Library in a tool for writing it). Whether that is true depends on what the Library does and what the program that uses the Library does.

1. You may copy and distribute verbatim copies of the Library's complete source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and distribute a copy of this License along with the Library.

You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee.

2. You may modify your copy or copies of the Library or any portion of it, thus forming a work based on the Library, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions:

3. You may opt to apply the terms of the ordinary GNU General Public License instead of this License to a given copy of the Library. To do this, you must alter all the notices that refer to this License, so that they refer to the ordinary GNU General Public License, version 2, instead of to this License. (If a newer version than version 2 of the ordinary GNU General Public License has appeared, then you can specify that version instead if you wish.) Do not make any other change in these notices.

Once this change is made in a given copy, it is irreversible for that copy, so the ordinary GNU General Public License applies to all subsequent copies and derivative works made from that copy.

This option is useful when you wish to copy part of the code of the Library into a program that is not a library.

4. You may copy and distribute the Library (or a portion or derivative of it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange.

If distribution of object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place satisfies the requirement to distribute the source code, even though third parties are not compelled to copy the source along with the object code.

5. A program that contains no derivative of any portion of the Library, but is designed to work with the Library by being compiled or linked with it, is called a "work that uses the Library". Such a work, in isolation, is not a derivative work of the Library, and therefore falls outside the scope of this License.

However, linking a "work that uses the Library" with the Library creates an executable that is a derivative of the Library (because it contains portions of the Library), rather than a "work that uses the library". The executable is therefore covered by this License. Section 6 states terms for distribution of such executables.

When a "work that uses the Library" uses material from a header file that is part of the Library, the object code for the work may be a derivative work of the Library even though the source code is not. Whether this is true is especially significant if the work can be linked without the Library, or if the work is itself a library. The threshold for this to be true is not precisely defined by law.

If such an object file uses only numerical parameters, data structure layouts and accessors, and small macros and small inline functions (ten lines or less in length), then the use of the object file is unrestricted, regardless of whether it is legally a derivative work. (Executables containing this object code plus portions of the Library will still fall under Section 6.)

Otherwise, if the work is a derivative of the Library, you may distribute the object code for the work under the terms of Section 6. Any executables containing that work also fall under Section 6, whether or not they are linked directly with the Library itself.

6. As an exception to the Sections above, you may also combine or link a "work that uses the Library" with the Library to produce a work containing portions of the Library, and distribute that work under terms of your choice, provided that the terms permit modification of the work for the customer's own use and reverse engineering for debugging such modifications.

You must give prominent notice with each copy of the work that the Library is used in it and that the Library and its use are covered by this License. You must supply a copy of this License. If the work during execution displays copyright notices, you must include the copyright notice for the Library among them, as well as a reference directing the user to the copy of this License. Also, you must do one of these things:

For an executable, the required form of the "work that uses the Library" must include any data and utility programs needed for reproducing the executable from it. However, as a special exception, the materials to be distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable.

It may happen that this requirement contradicts the license restrictions of other proprietary libraries that do not normally accompany the operating system. Such a contradiction means you cannot use both them and the Library together in an executable that you distribute.

7. You may place library facilities that are a work based on the Library side-by-side in a single library together with other library facilities not covered by this License, and distribute such a combined library, provided that the separate distribution of the work based on the Library and of the other library facilities is otherwise permitted, and provided that you do these two things:

8. You may not copy, modify, sublicense, link with, or distribute the Library except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, link with, or distribute the Library is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

9. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Library or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Library (or any work based on the Library), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Library or works based on it.

10. Each time you redistribute the Library (or any work based on the Library), the recipient automatically receives a license from the original licensor to copy, distribute, link with or modify the Library subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties with this License.

11. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Library at all. For example, if a patent license would not permit royalty-free redistribution of the Library by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Library.

If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply, and the section as a whole is intended to apply in other circumstances.

It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice.

This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License.

12. If the distribution and/or use of the Library is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Library under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License.

13. The Free Software Foundation may publish revised and/or new versions of the Lesser General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.

Each version is given a distinguishing version number. If the Library specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Library does not specify a license version number, you may choose any version ever published by the Free Software Foundation.

14. If you wish to incorporate parts of the Library into other free programs whose distribution conditions are incompatible with these, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally.

15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.

16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

END OF TERMS AND CONDITIONS