Batch processing – memory of the future

An important reason for the introduction of writing 6500 years ago was, besides religious and ritual reasons as well as writing chronicles, the administration of lists and tables in the context of bookkeeping. In those days there were certainly moments, in which the scriveners could not perform their tasks in a timely matter and created that way piles of unfinished tasks. In this case the employees had to invent accelerated transactions, in order to process the piles. Even if there were no electronic data processing at that time, with no doubt they already had invented the batch processing.


On the mainframes of today the batch processes are running over night, which transfer the accrued data of the day into the respective systems and databases. If the night is not long enough, the appropriate batch window must be expanded by additional computer capacities. The processing follows thereby different patterns.

  • First come, first serve
    In medieval times strict grinding rules were defined, in order to prevent fiddles. Until today this rule provides fair handling for requests that are submitted one after the other. The sequence results from the entry. The pile grows and the affairs are processed from bottom up.
  • Nip the things in the bud
    The advantage of preferring the last entry comes from the fact that the file already lies on the table and thus does not have to be sorted. As long as you have sufficient resources, you can avoid this way any piles. Since this leads to a waste of capacities, if there are no orders, this approach is only economical, if it saves costs of pile administration or storage space.
  • Burning issues
    Already Eisenhower was concerned with the question, what should be first dealt with. The matrix that was named after him, recommends to immediately take care of the urgent things or at least to delegate them to competent employees. The sequence depends on the expiration date of the task. For this purpose you need a good overview of the due dates in the pile.
  • The more danger, the more honor
    In this case you engage with the most extensive tasks first. They can stem from the value of the result, from the required effort for processing or the number of involved people. Naturally they take especially long. With the earliest possible beginning, the probability rises to finish timely. For this purpose the procedures must be evaluated according to their extent and then be sorted in descending order.
  • Many pennies make a dollar
    In this case you emphasize on the tasks with the fewest extent first. Also here, as within the last bullet point, the importance is derived from the value of the result, from the required effort for processing or the number of involved people. These tasks will probably need less effort. By the preferential processing of these tasks the pile quickly gets significantly smaller. The procedures are also evaluated according to their extent, but then be sorted in ascending order.

In data processing the data are not processed with one of the above procedures, but after the logical relationships of the databases and files. The goal is to have a consistent volume of data the next day. The height of the pile depends thereby on the IT for collecting the procedures. Appropriate rules bring the data already into a format that can be processed faster, when they are typed in.

In the agile projects of today you use piles, the so-called Backlogs, likewise. Here you simply collect the tasks that are not yet realized. The Product Owner, i.e. a decisive representative of the field, determines after a sprint, which tasks are developed in the next sprint. A sprint is a two to four-week work cycle, in which the developers have full control of the conversion.

Thus, the night window for the completion of batch runs corresponds to a sprint. The substantial difference is that the batch window must complete all planned procedures, since otherwise the consistency of the data is no longer secured. If a task is not finished in a sprint, you can use the subsequent sprint or put it pack into the backlog.

Bottom line: Batch processing was always needed. The difference results from processing and the sorting of the piles – FIFO, LIFO, FEFO, HIFO, and LOFO. In the context of the agile organization the Product Owner makes the decision, but they also still follow the same patterns. That makes the batch processing a memory of the future.