Amiga Development > AmigaOS 3.2

Development updates

(1/5) > >>

4pLaY:
AmigaOS 3.2 is currently in development and there is no set release date. Here I will, with the permission of Thomas Richter, post some of his personal updates on the development.

4pLaY:
Time to start with a series of posts describing the changes for 3.2. Exec is first.

There is not so much about exec, really. 3.1.4 brought one new bug into exec, namely the Alert AN_BadFreeAdr, which was triggered on a bad memory release, forgot to restore one register and hence caused another crash afterwards. This was fixed by SetPatch of 3.1.4.1, and now the fix was migrated into the code.

There is another series of fixes, though of old bugs. Alert() in general had a series of problems. First, it did not save all registers. Thus, in case ROMWack was entered later on, the register file was partially overwritten and thus made a post-mortem debug with ROMWack rather pointless. Also, ROMWack tried to detect the CPU type - by the same function exec uses during bootstrap. This sounds fine, but it is customary these days to move the vector base register out of the zero-page, and this is something the mentioned CPU-detection algorithm did not expect - it rather moved it back. Sometimes with fatal side effects. The code is now more careful in not touching VBR if it does not have to. The problem of the lost registers must have been introduced somewhere in the v37 series because the 1.2 kickstart did better.

Unfortunately, this is not sufficient to make ROMWack happy because there is another way how to enter it - namely through the dos.library "Alert" requester. ("Software error - task held..."). Unfortunately, this requester *also* overwrote the registers, and it neither saved the stackframe of the crash. The latter is, however, important to be able to debug a crash with the ROMWack. Thus, the dos.library exception vector - aka the error requester - was improved by saving all registers, and by also making a copy of the exception stack frame. If the user presses "cancel", the stack frame and the registers are restored, and the code enters the default exec error handler, which again points by default to the ROMWack.

Thus, ROMWack should have been become a lot more useful than before as it has access to the important registers and the stack frame causing the problem.
Or, use a debugger in first place...

There is much more to be said about the dos.library, but this is for a later episode.

4pLaY:
More progress: Expansion.

There were still some interesting issues in expansion. In case a RAM-board was unsized and requested auto-sizing by the Os, the Os overwrote the last byte behind the indicated slot size and thus might have trashed another expansion sitting behind it. This was still an old bug from 3.x times of which I do not know when exactly it was made.

Another old bug was the A3000 superkickstart support. What this does is that the code attempts to check whether the MMU is active, and maps the upper 512K RAM to a kickstart image. If so, the A3000 expansion skips this memory region, and does not insert it into the memory list. Or at least, it attempted to do so. Due to a logic error, expansion tested the wrong bit in the MMU configuration register and thus tested the bit whether there is a supervisor MMU table present, and only performed the mapping *if not so*. Instead, it should have tested the MMU-enable bit, and perform the superkickstart logic only if this bit is *set* (not clear). This stole 512K for A3000 users using the mmu.library which does enable the MMU, and uses "supervisor" MMU tables. The 3.1.4.1 SetPatch installs a workaround to merge the upper 512K into the memory list in such cases, though 3.2 will fix the problem at its root. Again, an old bug that survived.

Then, another issue with file systems. Expansion configures file systems with 600 bytes of stack, which is a bit short. This increases now to 1024 bytes. Actually, it is more complicated that this: The stack size in the DosList structure is measured in bytes for C-style handlers, and in longwords for BCPL-style handlers. Thus, if expansion leaves there a "256", this is fine for BCPL style handlers (256*4=1024), but way, way, way too short for C-style handlers (256 bytes!). This problem was addressed in the dos.library and system-startup, though.

4pLaY:
The next ROM component: Utility.

The utility.library contains several utility functions for tag lists, arithmetics, ID generation and so on. We haven't found any old or new bug in it, but extended its capabilities. One of the things that happened is that the long division functions (32/32 division) for the 68K processor were not very adequate as they used a rather slow "egyptian" division algorithm that generates its results by shifting and subtraction. Well, it works, but it is slow, so it was replaced by a better performing "Algorithm D" implementation by Knuth.

It is also weird that we do have a 32x32->64 multiplication, but no 64/32 division function in utility. The latter is, in fact, required by multiple Os components that handle large disk drives, so two functions for that were added, a signed and a unsigned quad-word division. For the 68020 to 68040, it uses the long division of the processors, for the 68000, a long Algorithm D implementation is used, and for the 68060, a similar algorithm D implementation as well as this processor lacks a 64 divide.

Utility also includes a set of new string functions. Strncpy() which is a length-limited string copy receiving a target buffer and a target buffer size. Unlike its ANSI counterpart strncpy, it always NUL-terminates its destination, and returns in case of success the pointer to the terminating NUL character for easy string concatenation, or NULL for buffer overflow, with a partial string then having been copied and terminated.

Strncat() concatenates strings into a length-limited buffer, using the same type of interface, and Snprintf() is a string-formatting function into a lenght-limited buffer on top of the exec RawDoFmt() function, quite similar to ANSI snprintf(), just that it returns the required buffer size, not the size of the result string. This helps to prevent the usual "off-by-one" error one typically runs into by confusing buffer sizes and string lengths.

4pLaY:
More components: dos.

The dos.library was actually Tripos, an operating system of its own until CBM bought it, and integrated it into AmigaOs. For Os 2.0, the BCPL source was completely replaced by an assembler/C implementation that came from "arp", the "Amiga Replacement Project", available independently for Kickstart 1.3.

For 3.2, we made more modifications to this antique core. First, we have a new dos.library function that acts as a "hook" whenever the dos.library attempts to bring up the "Please insert..." requester. An external program can hook in there, and can improve the requester, or offer more choices. In fact, Os 3.2 ships with such a program, namely "AssignWedge". The AssignWedge offers the functionality to also mount a handler, or deny a request for a specific volume from that point on. The "Assign" command can then be used to remove such a permanent denial.

Then we have an increased stack size for built-in handlers. In 3.1.4 and before, SetPatch took care of increasing their stack, but the defaults have never been touched. In 3.2, these defaults grew. It is, however, no longer the dos.library that mounts them, see below

The System() call received new tags: SYS_Error specifies an error output stream, which is mirrored by the CreateNewProc tags NP_Error and NP_CloseError that never worked before. This error ouptut stream defaults to the standard output if not defined, but may be given to redirect error messages to a separate stream. The shell offered support for error redirection since ever by means of the "*>" redirection token, but it wasn't very useful since many commands did not use it.

This changes for 3.2: First, the dos.library internal function PrintFault() uses it now, and we have also new calls to receive it (ErrorOutput()), to write a string through it (PutErrStr()) or to change it (SelectError()). All internal commands and commands in C: have been extended to use this new error stream, allowing you to separate error from regular output. Note well: If there is no error output redirection, the error output goes into the standard output for compatibility reasons, but if "*>" is explicitly given, stderr goes to some separate file.

Then, we have SYS_CmdStream as a new tag for System(), which tells the call that instead of taking the commands to be executed from a file rather than its first argument. This sounds fairly harmless, but provides an important new feature. Before that, let me also say that SYS_Asynch now also works for System() in case the command string is empty, and thus if SYS_CmdStream is used instead.

Now, what is all the fuzz about it? With SYS_CMdStream = Open("S:Shell-Startpup"), SYS_Output=Open("CON:") and SYS_ASync=TRUE, you can now create a new CLI, similar to "NewShell". Actually, the "NewShell" command and SYS:System/CLI do now exactly that, and the archaic and undocumented method by which NewShell used to operate is opsolete and has been removed. One piece of BCPL junk less, yay!

This said, the whole interface towards the shell has been simplified. We used to have three (actually, four!) methods how a shell can be launched: One at system startup, one when the user used the "Run" command or the "System()" dos.library function, and one for "NewShell", and the shell had to fiddle out what the user wanted to do, and then had to call into three also mostly or completely undocumented functions, CliInit(), CliInitNewCli() and CliInitRun() to get initialized.

This is of course utterly complex and error prone. Now that "NewCli" goes through "System()", "CliInitNewCli()" is obsolete and no longer needed. And, due to another change, "CliInit()" is also obsolete. I come to that in a minute.

So, what happens if the system boots up: Well, the dos.library is initialized, then the dos.library used to initalize the shell, then starts the shell, the shell is supposed to understand what the system wants from it through an archaic and mysterious interface whose documenation is hard to find and to digest, then calls into CliInit(). CliInit() then mounts all the handlers, opens the startup-sequence, and returns to the shell, which then runs the startup-sequence. Thus, this used to be like "ping-poing":

Dos->(CreateProc())->Shell->(CliInit()->)Dos->Shell

Yummie. All this changed in 3.2. The system startup procedure which was part of the mysterious and undocumented CliInit() function was removed from the dos.library whatsoever, and moved into a separate system component, "System-Startup". It resides in ROM, or in L: for those that go for a software-only upgrade, and is loaded there by LoadModule. So the procedure looks like this now:

Dos->(CreateProc())-System-Startup->Shell.

No ping-pong. What System-Startup does is what CliInit() used to do, and a lot more. So it mounts all the handlers, all the volumes, creates the default assigns like S:, DEVS:, L: and C:, and then creates the boot shell, by the mechanism discussed above: System(), with SYS_CMDStream=Open("S:Startup-Sequence") and SYS_OUtput=Open("CON:...").

Thus, CliInitNewCli() done for good, CliInit() done for good, and the only init mechanism the shell is left with is CliInitRun(), everything else was removed. Thus, no longer any guesswork as what to call, and a very simple mechanism to get the shell going.

We have a couple of other improvements as well:
- ExAll() had a bug by forgetting to pass an argument to a potential error requester and thus could crash or show nonsense on errors.

- All the BCPL inherited I/O functions such as FWrite or FRead were terribly slow. They made single-byte-I/O, going through three function layers to get data, and thus delayed all operations down that depended on them, such as the icon.library which uses FRead to read icons. FRead() and FWrite() have been rewritten to make fast block I/O, and "burst" directly into the user buffer if possible without going through an internal buffer, hopefully also speeding up the icon.library and many other system components that use them.

- System() ignored the NP_Name tag. Instead, a new process created by it was always named "Background CLI". Now you can name it as you like. For example, "Initial CLI", the name used by System-Startup (obviously!).

- System() had more problems in erraneously releasing file handles in case of failure. The interface specification say that the file handles remain open and untouched in case of failure, but did not always do so. This might have caused hard to trace bugs in low-memory situations.

- There was some confusion about the stack size of file handlers. Unfortunately, the default stack size of a handler is measured in bytes for C-style handlers, and in long words for BCPL-style handlers, thus caused some mismatch if the dos.library adjusted them. In fact, the 3.1 SetPatch increased the stack size of the internal handlers to something like 16K instead of 4K due to this mismatch in units. We fixed that.

- Speaking of stack sizes, handlers may now contain a magic "$STACK:" token that indicates a minimum stack size they want to have. If such a stack token is present, the default stack size is increased to this minimum size. It is always measured in bytes, regardless of the handler type. Thus, there is no longer any guessing necessary as what to put into the "Stack" mount line as the handler can communicate this now itself.

- WFPrintf() has some issues with invalid templates for its arguments.

- Speaking about the ROMWack, I already mentioned in exec that the exec default exception handler preserves now registers for the system debugger. Unfortunately, the dos.library default Alert requester ("Software error, Task held...") did not. Now, 3.2 will fix that and the Alert requester will store the stack frame away and restore it when the system debugger kicks in. Thus, post-mortem debugging with ROMWack should finally work again. This was broken somewhere on the road from Kickstart 1.2 to 1.3 (yes, really that old!).

- Mounting BCPL handlers leaked 20 bytes of memory. Another fix of an ancient bug.

- Finally, the CLI structure was extended. It now contains also the history. The new ExtendedCommandLineInterface structure is created by AllocDosObject(), and you better use that function to create CLIs as otherwise the "history" command of the shell does not work. Of course, all system functions use it, and as of 3.2, ARexx was updated to go through AllcoDosObject() instead of rolling its own CLI. As said, legacy code that does not will continue to work, but then "history" will print an error as the CLI is unextended. The shell will continue to work, though. I haven't observed this case in the wild, yet, as most programs create CLIs through either System() or "NewCLI" and both are of course fine.

There is more that needs to be said about System-Startup as this new system module does now a lot more than CliInit(), but I leave this for a later episode. This one is long enough.

Navigation

[0] Message Index

[#] Next page

Go to full version