BOR Framework for the execution and analysis of empirical investigations in machine learning
Download

BOR Ranking & Summary

BOR Tags

execute machine learning empirical investigation

BOR Description

Framework for the execution and analysis of empirical investigations in machine learning Bechmarking on Rails (BOR) is a free and open source framework built for rapid specification, execution, and analysis of empirical investigations in the area of machine learning and signal processing. BOR is written in python and can be combined with popular toolkits like the Weka framework, the Modular toolkit for Data Processing and the MMLF. BOR supports the massive parallel execution of benchmarking experiment on grids like Apples XGrid. BOR has an object-oriented design, providing classes for important entities like Datasets, Processing, and Modality.Datasets are structured in so-called BOR Collections. BOR Collections should comprise data that originates from the same source, i.e. the process that generated them should be the same. Several BOR-Collections that orginate from different sources can be bundled together into a BOR Bundle. A typical example of a BOR Collection is an EEG measured from one subject in one session. In contrast, a BOR Bundle can contain the measurements of several different subjects or from different sessions. Both BOR Collections and BOR Bundles consist of the actual data and some meta data stored in the file “collection.yaml”. BOR Collections have a type e.g. “time series collection” or “feature vector collection”. A BOR bundle inherits the type of the collections it comprises, i.e. a bundle must be homogenous containing only one type of collection.Processing describes any kind of computation that transforms one kind of data into an other. In BOR, there exist different conceptualizations of processing for different levels of granularity. On the middle level, a BOR operation is a process that takes one BOR bundle as input and produces a second BOR bundle as output. On the highest level, a BOR campaign is a sequence of BOR operations. The input of the BOR campaign is processed by the first BOR operation of the campaign. The output of a BOR operation acts as the input for the subsequent operation of the campaign. The output bundle of the last operation is the result of the campaign.Each BOR operation consists internally of a set of BOR processes. While the operations of a campaign are dependent and thus processed sequentially, the processes of an operation are indepedent and can thus be processed in parallel. The way an operation is divided into processes is not fixed, for instance an operation might have one process per collection of the input bundle or one process per run applied to the input bundle.Both BOR processes and BOR operations have a type. Currently, most processes are internally implemented by using algorithms implemented in the aBRI-DP/MDP or in Weka. Correspondingly, there is one MDP-Operation/MDP-Process type. Furthermore, there are currently two processes based on Weka. One type is the WekaFilter-Process/Operation, which is defined by its property to transform one bundle of type “feature vector” into another bundle of the same type. It might internally apply some kind of feature selection, normalization etc. The second Weka-based type is the WekaClassification-Process/Operation. This type is defined by its property of transforming one bundle of type “feature_vector” into a bundle of type “result_set”. Usually, it applies internally a set of classifiers to the bundle and stores several statistics concering their performance (accuracy, precision, recall etc.) as well as some properties of the input data into a result file. Furthermore, there is also one AnalysisProcess/Operation that analyzes the data contained in a “result_set” bundle and creates a set of plots that visualize and evaluates the effect of various parameters on several metrics. All processes of a BOR operation must have the same type. In contrast, the operations of a BOR campaign have typically different types. The restriction is that each operation of a campaign must be able to process a bundle produced by the preceding operation. Requirements: · Java 1.5 or later

BOR Related Software