As described in a previous research report: “Cornerstone Big Data Analysis through Apache Spark”, the systematic analysis of ever-expanding data collections presents companies with ever-greater challenges. However, some of the know-how is simply missing in order to be able to carry out big data projects successfully. Therefore one simply follows the current trends and buzz words and adopts approaches which are currently en vogue. This approach often leads to less successful projects and several regularly reoccurring patterns of misconceptions can be identified.

This research report highlights some of these unsuccessful patterns and introduces some of the work done in the PRO-OPT SMART-DATA research project. The project experimented with various approaches for the modeling of production data of an automotive supplier. One objective was to be able to apply and compare statistically grounded analysis and classification procedures as well as new procedures from the AI eco-system. Different tool stacks were used. This report focuses on Cornerstone’s direct access to Big Data databases.