[ColorForth] Philosophy (part 2 of 3)

[David Govoni <dgovoni@xxxxxxxxxxxx>]

Continuing,

Reading Jeff's writings on 1x programming, as well as Terry Loveall's 
explorations, and stewing for a while I've come up a notion.

There are two reasons for big complicated bloated code.

1) Cohabitation and
2) The General Case


1) Cohabitation

Cohabitation is what chip level languages and floppy booting systems can 
completely avoid. If you 'own' the machine and can do to it what you want, 
you're free to write whatever code you wish.

If you wish to share hardware with other programs you have to do things 
differently. And if you wish to share a system with other currently running 
programs it gets even worse.

In the early days of PCs and DOS there was an interesting way of handling 
files. Most commercial programs did seem to use slow OS level commands to 
read and write files. Programs wrote their own routines to have the 
hardware access the drives. It was faster and more efficient. But the code 
still had to put information on the disk in the same format. FAT files and 
directories. This way data wouldn't conflict with the data from other 
programs.

On DOS systems, Lotus 1-2-3 ran and saved its files on the shared file 
system. Then WordPerfect would run and be reasonably sure that Lotus' files 
hadn't overwritten its own files. Both could implement their own code to 
control the hardware because there was no multitasking. But both had to 
play by the 'accepted' shared format of FAT disks though. Sure there was 
more efficient ways for programs to store data, but if you wanted to share 
a disk you couldn't implement them.

When PCs became multitasking GUI systems they evolved into machines where 
it is almost impossible for a program to access the hardware directly. The 
OS had to become the broker between the running programs and the hardware. 
Code became bigger and more bloated as it had to handle talking to the OS 
and dealing with the problems that could result.

Its interesting how many subsystems have evolved to allow limited 'direct' 
access to hardware to improve performance. Most 3D graphic systems allow 
one program at a time direct access to the hardware. Thus letting the 
program get the performance it needs by being the only program currently 
accessing that hardware. In essence if you need to speed up access to some 
hardware then don't allow the system to share it.

The result is complex systems that has to broker the hardware between 
running programs. In the world of mainframes and minis this isn't a new 
idea. For PCs it was a shock for programmers. Why write complex event 
driven programs? Why get all input from the OS? Why ask the OS to do most 
of your work for you? Because otherwise you couldn't share the machine with 
the other programs.

(By the way, ever notice how many programs cause problems on PCs by not 
following the rules? In a Unix style system its hard to break these rules 
and therefore its hard for programs to foul up the entire system. However 
in the windows world its still quite easy to break the rules. And system 
stability suffers because programs not playing by the rules).

If you're building an embedded device or running a system that takes over 
the hardware you can remove all of this added complexity.

However, if you want to write a program that runs on a multitasking OS this 
isn't possible. Its a trade off.


2) The General Case

The second reason for code bloat and complexity is the need to compile code 
that can handle any situation.

I use Charles' BMP code as an example. (Yes I know the code isn't this 
complicated, and the BMP code assumes a lot of the things I'm going to talk 
about, but let me take his example a bit further...)

Let's have a programmer create a BMP creation routine for a language that 
has to be pre-compiled. Where it has to be fixed in binary form before the 
user runs the program.

You'd create a routine that could be given any memory location since you 
don't know where the image is.

You'd write a routine to save any size image since you may not know how big 
it is.

You'd write a routine that would have to take in all possible options and 
settings and constantly check to see which parts of the routine are needed 
and which aren't. (Is this a TrueColor image or a color mapped image? Is it 
being compressed or not? How is it being compressed.)

The routine would end up being a collection of IFs and CASEs that direct 
the routine to the specific code. Or the programmer could use objects or 
other techniques to hide the complexity.

In Charles' case the code is compiled into a specific routine that is run 
once. You end up with a routine that takes a BMP of a fixed size in a fixed 
location of memory and stores it in a fixed manner.

No unnecessary variables (where is the image? how large?) only hardcoded 
values that improve performance.

No unnecessary decisions (pixel depth of image? how are pixels stored in 
the memory space?) only the code necessary to do the specific work. (This 
is where I extend the example. Charles' avoids these questions by making 
decisions as a programmer ahead of time, his code assumes the universal 
answers to these questions)

Unlike the hard coded general purpose routine that is called for every BMP 
created, this code is compiled, run, and thrown away for each invocation.

And as long as the compiler is ultra efficient the trade off is worth it. 
In fact the longer the time spent in the routine the bigger the payoff.


My machines spend an incredible amount of time ray tracing. The program I 
most frequently use is POV-Ray. Freely available and amazingly efficient. 
Yet it is precompiled code designed to ray trace any scene its given.

Image what would happen if given a scene description a ray tracer was 
compiled to render that one scene only. Any assumptions that could be made 
wouldn't have to be tested for in the code. Why have one routine for 
working on any object in the scene if you could have a routine specific for 
each object in the scene? Remove as many indirect memory accesses as 
possible. Don't have the code look up a variable. Have its value hardcoded.

Given some images can take 48 hours on a modern machine there has to be 
room for improvement in render times.

This is the huge advantage of compiling code on the fly.

(hang on, there's more one more part)

David Govoni

------------------------

To Unsubscribe from this list, send mail to Mdaemon@xxxxxxxxxxxxxxxxxx with:
unsubscribe ColorForth
as the first and only line within the message body
Problems   -   List-Admin@xxxxxxxxxxxxxxxxxx
Main ColorForth site   -   http://www.colorforth.com