The Long Version

You could not build a GUI application without a framework, (the Frame, Window, Container, and Component Classes). You should not build a backend application without a framework.

A backend [activatable] remote/non-remote object runs in a plain brown box in a dark room with no one watching it. If you simply add application code to this empty box you will run into insurmountable problems.

Threads

The first problem that comes up is timing the request. The client sends a request to the Backend Server for information contained in a private resource. If a horde of other users are also updating that private resource, by the time the request completes, the original user has gone home. If the private resource is nonfunctioning to the point that the request cannot complete, not only has the original user gone home, but the Client thread hangs forever.

How does one do an autonomous request, like a callback?  That is -- send in a request, have it processed by a background thread that contacts the original Client when complete. If we simply create a new application thread for every request, the create/destroy overhead and the number of application threads will put a severe strain on the Backend Server, and the Server will eventually run out of resources.

The practical solution to these and many more problems is to separate the Client thread activity from the application processing. You can do this by creating an application queuing and threading structure on the server side. This is the way highly reliable, fully mission-critical software products work, and that structure can be available for any application.

For a Client that needs an immediate response, the Client thread contacts an application thread.  If the application thread does not respond within the time limit, then the requesting thread returns to the Client with a timeout message.

For an autonomous request, the Client thread contacts an application thread. The backend application executes asynchronously.

Now the only problem is how does one design a queuing and application threading environment so that:

• The requesting threads and the application threads can talk to each other.
• The application environment can know about Client timeouts and recover therefrom.
• A thread overload problem does not occur, (i.e. where so many application threads are executing that the JVM cannot sustain anymore threads or these threads cause so much competition for resources that the environment effectively stalls).
• The application thread create/destroy overhead does not bog down the application processing.
• The threading environment is monitored to pinpoint stalls.
• An abnormally terminating thread can recover and restart.
• Partial scheduling failures can back-out.
• The entire threading environment may quiesce and shut down gracefully.
• Are you getting the picture? This list can go on and on.

When dealing with threads there are no quick solutions, only thorough designs.

Components

Having established a queuing environment, it is only a short walk to having multiple queues for each request. This is known as request brokering, that is -- breaking the request into its component parts and placing each component into a separate queue.

For a request with multiple access to resources, each access goes into a separate queue with its own threads. As each access completes, the response concatenates with all other responses. When all access complete, the system returns the total response to the originator.

This is not very difficult to do. However, what about the autonomous request?   After all access complete, there is no originator waiting for the reply, what happens then?  This is where the independent agent comes in. The system must create a new process to handle this situation. What seemed easy has now become difficult.

Recursion

Recursion is one of the most useful techniques in computer programming. Think of the parentheses processor in a compiler:  x = (((a + b) * c) / d)  Every inside parentheses pair must process first. This means that the parentheses processor must call itself.

It is this ability to stop here, call itself and pick up where it left off that makes recursion so useful. However, the effort required in keeping track of allocated storage, levels, etc., means that designers usually chose some other, less efficient method.

There is simply no way to add recursion to a fully built backend process without ripping it completely apart.  This must be designed in at the beginning.

Logging

Anyone who has ever worked with a background process knows how important it is to log errors.  How else can anyone know what happened after a failure.  There are three kinds of logs:

1. The individual application's log.
2. A normal application process such as a start up sequence. At each stage of the start up, the process logs its status so that if any total failure occurs, at least the team knows about where to look.
3. The failure log is for application queuing and threading errors.

The first is application dependent. Usually, no general purpose log will suffice.   Commercial products are available today and the standard language will support logging in the near future.

The other two kinds of logging are part of the framework and usually share a common usage. They do not need a fancy structure; a simple file or DBMS table is sufficient. It is because of this that the framework log should be separate from the application log.  This makes it another detail for the application developer.

Notification of errors

So what does one do when one detects an unrecoverable error? There is usually some standard messaging system in place but how does the framework and each application interface to that procedure? It is better to put a common procedure in place at the framework level. In this way, adherence to the standard is easy.

How to detect and recover from stalls

A stall is when a request takes so long that the response is no longer needed, a thread is waiting on a locked resource that can never be unlocked or an application has ended abnormally.

The only way to detect such an event is to time each function in the life of the request.  These include, but are not limited to:

• Thread processing -- that time outside the pure application code where the thread is executing "synchronized" statements, or other such code.
• Thread activation -- that time when a new thread is required from the JVM but before the thread actually begins executing.
• Thread wait time -- how long a thread will wait for resources or new requests.
• Application processing -- the actual application code (this is tough since each application may have different requirements).
• All the other little parts that make a normal life cycle.

Now, having timed these events, one needs to write a daemon thread that lives on the Server to actually check the timing.

Real time alteration of the environment

No server runs the same all day, every day. There are always peaks and valleys of processing. Sometimes more time is necessary to process a request because of seasonal requirements. Sometimes a longer queue is necessary because of more requests.

A properly designed server can be dynamically tweaked in all the right places to function in a wide range of elements throughout the day.

This must be generic and designed in at the beginning.

You wrote it.  Now you have to tune it.

Tuning starts with good design. Not even we (and we're good) can tune a can of worms.

The basis for tuning is statistics.  How many happened where and for how long.   You are not going to get this by adding it in after the fact.

Taken together with dynamically altering the environment and an audit trail (log) you have a viable system that is capable of being tuned.

What's happening?

So,  you sent a request to the Server and ... and ... and ...

As the saying goes, "A picture is worth a thousand words." To be truly effective, not only should there be a way for Clients to inquire about autonomous requests but a way for any user to inquire about the overall health of the Backend Server.

This means that you must build GUI and non-GUI interfaces to the Server so that it is easy for system administrators to get a picture of the executing environment.

What about the next application

Now that you built one application with all the underlying framework, can you reuse the framework for another application?

Think of this like an office building. Besides the superstructure of concrete and steel every office needs:

• heating/air conditioning,
• hot/cold water,
• sewerage,
• electricity,
• telephony,
• satellite/microwave hookups,
• cabling between floors,
• removable windows to facilitate large equipment installs and
• other industry specific requirements.

If you have a generic framework in place, then any type of application can live inside.

Hooks for adding management services

No generic framework can work for all application environments. This is where the hook or user exit comes in.

Backend Servers are persistent. Applications can take advantage of this environment to store common variables between threads (i.e. connection pools are the most common). By using a start up exit, each application can prime its own block of storage and even start daemon threads (customization).

Need global notification when the server goes down? This is a use for a shut down exit.

There is no way to add this functionality in after the server is written without ripping it apart. Patch a piece here and it is sure to fail over there. There is no beating good, up-front design.

Extensibility

The first law of computer programming states that:
Any code written today will be enhanced tomorrow.

Building a static application environment is a ticket to disaster. As soon as people use a system they find other, and sometimes better, ways to use that system. This means changes.

Anybody can build an ad hoc or single use system. The true professional builds dynamic systems. Systems capable of using plug-in code (components) that are easily extensible and changeable. The extra weeks up front in design can save months of work down the road.

Problems, problems, problems. Think of the above as the dirty dozen. If you never designed a mission-critical server application before then this is all new. If you have, then you should be happy that you will not have to do all those painstaking details again.

We've done this before, many times.  We solved the problems.  We've made it easy for you.

Welcome to Tymeac