Batch Window (Proposal)

[sotretus]

Most batch programs I know of execute when the "online" is off (i.e. midnight) and must be stopped before the online begins executing: a "Batch Window".

Still I can't seem to find the BatchWindow concept in the Spring Batch architecture.

So for our little project we created a BatchWindow hierarchy Strategy for handling this. This is our approach, and the idea is to kick-start a brainstorming session to see how the BatchWindow concept could fit in Spring-Batch.

(All code is pseudocode).


interface BatchWindow{
//is in window right now
boolean inWindow();
//is the following date inside the window
boolean inWindow(Date);

//when the window will be finished, based on the
//current time.
Date nextFinishTime();
}


Particularly we have a CronBatchWindow impl.

class CronBatchWindow{

String startCronExpression;
String stopCronExpression;

}


Now we can Identify if we are inside a batch window or not. Notice that we require two cron expressions to define a batch window (We are not allowing to create cron expressions with "-" inside them)

Next we have a CompletionPolicy based on batch windows, called WindowBasedStepCompletionPolicy:


class WindowBasedStepCompletionPolicy{

BatchWindow window = null;

//remember, just the important stuff here
public boolean isComplete(...){
boolean inWindow = this.window.inWindow();
...
}

}



Regards
AB

[Dave Syer]

That's pretty much exactly how we imagined a Batch Window would be implemented in Spring Batch (through a CompletionPolicy). N.B. there are two possible levels of CompletionPolicy for the inner and outer loop - both could have a custom CompletionPolicy for this scenario - the inner loop would get shorter as the batch window approached to limit the number of items in a transaction and give the outer loop more chances to end. The outer loop would have a hard deadline where it would complete at the end of the window.

If you put something in JIRA we will add it as a standard feature at some point.

[sotretus]

Ok I will, just have to change some package names and write the comments in English.

For the "hard" and "soft" deadlines, it seems that the inner loop should have no deadline at all, right?. Or are you thinking in something like ("if I only have to process the remaining 5%, then skip the completion policy") ?

As a side note:
We were also able to reuse the BatchWindow to specify a maximum time to wait on a remote resource (some business processing) in order to avoid hitting the max limit.