Tester Jobs Requirements (VIDEO GAME)

Groningen is a Java based application that automatically tunes Java virtual machine garbage collector settings for cloud based services. It uses a generational genetic algorithm to produce sets of modified JVM settings. Experimental tasks are run with these settings for a specified duration. At the end of the experiments, garbage collector logs are collected, processed and the experiments are scored. Next generation of settings and experiments are generated based on these scores. This way, Groningen evolves a population of experimental tasks under real front end load to find the most fit JVM settings. It continues to iterate through subsequent generations of experiments until near optimal settings are found.

Groningen is designed to be pointed at an application which it will tune on an on-going basis. A search space is defined with all possible JVM settings and it will automatically search through those settings to find the best performance sets. It will work on the problem without human interaction and eventually inform its human operator via console of near optimal settings that are likely to improve application performance.

However, the process is intrusive in a way that application instances have to be restarted many times to change JVM settings before arriving to optimal settings. At the same time, real system stimuli is required to properly indentify optimal settings.

For these reasons, instances that are beeing tested should be a subset of the live application, possibly as a duplicate set of instances processing replicated traffic on unused infrastructure such as a QA environment replaying representative traffic.

As seen in the image above, major Groningen components are: Hypothesizer, Generator, Executor, Extractor, Validator and the Experiment Database.

When Groningen starts, it creates an initial experimental population and runs each experiment under real load to produce real world data. Garbage collector logs are collected by the Extractor object immediately following experiments by the Executor object when tasks have restarted. Collected logs are opened, parsed and the collected data is stored inside the Experiment Database. Data from the logs is recorded in as much detail as posible so the fitness function can determine how successful a specific mutation was. After the data was collected, Validator object checks if the collected data qualifies as a successful experminental run and if it can be trusted. This is done to prevent abnormal or degenerate behavior to propagate to the next generation. In these cases, Validator sets a false valdation flag in the Experiment Database. For the very first iteration, Hypothesizer will create a random set of settings. Later, however, it will use the fitness function and the collected data to score each experiment and using generational genetic algorithm it will create new set of experimental settings. Fitness function is configurable according to the application type beeing tuned. There are three optimization goals: latency, throughput and footprint. Higher score means that a particular experiment is more likely to be crossed with another experiment to produce the next generation. Chromosomes are mutated with a slight probability to avoid getting stuck in a sub-optimal local maximum. At this point Groningen may decide that a near optimal configuration has been found and exit, or the Generator object will generate another experiment. Executor will then start experiments and monitor them in case of problems during execution. If an experimental task encounters a problem than it is reset to the default JVM settings and is marked accordingly so the Validator object can set a false valdation flag.