Integrated Artificial Intelligence (AI) & Machine Learning - Deep Learning with CFD & FEA Simulation
Machine learning is a method of data analysis that automates analytical model building. It is a branch of Artificial Intelligence based on the idea that systems can learn from data, identify patterns and make decisions with minimal human intervention. With Artificial Intelligence (AI) applications in CAE, that is Mechanical Engineering and FEA and CFD Simulations as design tools, our CAE engineers evaluate the possible changes (and limits) coming from Machine learning, whether Deep Learning (DL), or Support vector machine (SVM) or even Genetic algorithms to specify definitive influence in some optimization problems and the solution of complex systems.
Artificial Intelligence (AI) Twins can predict the outcome of simulation studies and can be used in the product development lifecycle when performing the traditional simulation is too costly or takes too much time.
Our AI team at Enteknograte uses the advanced CFD and FEA software in combination with Artificial Intelligence (AI) and Machine Learning tools Twins with the goal to train AI to learn from simulations, to extend the knowledge over time, and increase the performance and efficiency in the modeling process.
Machine Learning Algorithms
Machine learning algorithms all aim to learn and improve their accuracy as they process more datasets. One way that we can classify the tasks that machine learning algorithms solve is by how much feedback they present to the system. In some scenarios, the computer is provided a significant amount of labelled training data is provided, which is called supervised learning.
In other cases, no labelled data is provided and this is known as unsupervised learning. Lastly, in semi-supervised learning, some labelled training data is provided, but most of the training data is unlabelled. Let’s review each type in more detail:
- Supervised Learning
- Semi-supervised Learning
- Unsupervised Learning
Artificial Intelligence (AI) and Machine Learning (ML) in CFD & FEA
Finite Element Method and CFD has become the top physics-based simulation technique and the number of elements involved in a FEM and CFD simulation have increased by a factor of ten every decade. As a result of the increased problem size, the computing resources needed for FEA and CFD simulation in for example structural integrity, computational fluid dynamics (CFD), electromagnetic analysis, and structural topology optimization has grown dramatically and represent a non-trivial cost element in the design process. Artificial Intelligence (AI) and machine learning (ML) has been advancing and inventing new methods that address the complexity of the same design problems in FEA and CFD.
Recent advances in deep learning and the implementation of these methods using specially designed platforms running on GPU-based clusters are allowing ML models to shortcut the simulation process by summarizing the results of simulations. In doing so, the ML model serves as a repository of the wisdom gained from multiple simulation runs. The clear benefit of using ML is the reduction of number of simulation runs during the design of a new, but similar, product.
With AI & ML, FEA and CFD simulation changes from being a tool in the design cycle to a tool of data generation. Transforming from a platform of managing data, to a platform in which the product design lives and functions.
FEA and CFD simulation allow the modeling of the most complex systems, while ML can help optimize the use of simulation resources to make product designs more efficient without sacrificing accuracy.
Enteknograte team consist of talented engineers predominantly use a Python based stack for high-performance and low latency Machine Learning development with CFD and FEA based results training.
The people standing behind the Python ecosystem are truly amazing, and we wish them (and us) to continue their productive work to make the world better!
Applying AI and machine learning tools in the technological applications can enhance simulation efficiency, improve product quality and reduce production costs.
The combination of computational fluid dynamics (CFD) with machine learning (ML) is a recently emerging research direction with the potential to enable the solution of so far unsolved problems in many application domains. Machine learning is already applied to a number of problems in CFD, such as the identification and extraction of hidden features in large-scale flow computations, finding undetected correlations between dynamical features of the flow, and generating synthetic CFD datasets through high-fidelity simulations. These approaches are forming a paradigm shift to change the focus of CFD from time-consuming feature detection to in-depth examinations of such features, and enabling deeper insight into the physics involved in complex natural processes.
Deep Learning
Deep learning is an ambiguous term used to denote a collection of models mainly implementing neural networks with many layers to challenging classification and estimation problems. The rapid growth of the power of deep learning techniques can be attributed to the development of parallelized versions of the deep learning models that can be run on GPU-based computer clusters.
This allows them to tackle problems of high complexity and simultaneously achieve high accuracy by being able to use large complex training datasets efficiently. This boosted the applicability of deep learning models to the difficult problem of representing the simulation results of FEM analysis used in product design on an industrial scale for real complex problems.
Reinforcement Learning
Reinforcement learning refers to an area of machine learning where the feedback provided to the system comes in the form of rewards and punishments, rather than being told explicitly, “right” or “wrong”. This comes into play when finding the correct answer is important, but finding it in a timely manner is also important.
So a large element of reinforcement learning is finding a balance between “exploration” and “exploitation”. How often should the program “explore” for new information versus taking advantage of the information that it already has available? By “rewarding” the learning agent for behaving in a desirable way, the program can optimize its approach to achieve the best balance between exploration and exploitation.
Genetic and Evolutionary Algorithms
Although machine learning has been very helpful in studying the human genome and related areas of science, the phrase “genetic algorithms” refers to a class of machine learning algorithms and the approach they take to problem solving, and not the genetics-related applications of machine learning. Genetic algorithms actually draw inspiration from the biological process of natural selection. These algorithms use mathematical equivalents of mutation, selection, and crossover to build many variations of possible solutions.
Decision Tree Learning
Decision tree learning is a machine learning approach that processes inputs using a series of classifications which lead to an output or answer. Typically such decision trees, or classification trees, output a discrete answer; however, using regression trees, the output can take continuous values (usually a real number).
Artificial Neural Networks
An artificial neural network is a computational model based on biological neural networks, like the human brain. It uses a series of functions to process an input signal or file and translate it over several stages into the expected output. This method is often used in image recognition, language translation, and other common applications today.
Cluster Analysis
A cluster analysis attempts to group objects into “clusters” of items that are more similar to each other than items in other clusters. The way that the items are similar depends on the data inputs that are provided to the computer program. Because cluster analyses are most often used in unsupervised learning problems, no training is provided.
The program will use whatever data points are provided to describe each input object and compare the values to data about objects that it has already analyzed. Once enough objects have been analyze to spot groupings in data points and objects, the program can begin to group objects and identify clusters.
Clustering is not actually one specific algorithm; in fact, there are many different paths to performing a cluster analysis. It is a common task in statistical analysis and data mining.
Bayesian Networks
A Bayesian network is a graphical model of variables and their dependencies on one another. Machine learning algorithms might use a bayesian network to build and describe its belief system. One example where bayesian networks are used is in programs designed to compute the probability of given diseases. Symptoms can be taken as input and the probability of diseases output.
Rule-based Machine Learning
Rule-based machine learning refers to a class of machine learning methods that generates “rules” to analyze models, applies those rules while analyzing models, and adapts the rules to improve performance (learn). This technique is used in artificial immune systems and to create associate rule learning algorithms, which is covered next.
Association Rule Learning
Association rule learning is a method of machine learning focused on identifying relationships between variables in a database. One example of applied association rule learning is the case where marketers use large sets of super market transaction data to determine correlations between different product purchases. For instance, “customers buying pickles and lettuce are also likely to buy sliced cheese.” Correlations or “association rules” like this can be discovered using association rule learning.
Inductive Logic Programming
To understand inductive logic programming, it is important to first understand “logic programming”. Logic programming is a paradigm in computer programming in which programs are written as a set of expressions which state facts or rules, often in “if
this, then that” form. Understanding that “logic programming” revolves around using a set of logical rules, we can begin to understand inductive logic programming.
Inductive logic programming is an area of research that makes use of both machine learning and logic programming. In ILP problems, the background knowledge that the program uses is remembered as a set of logical rules, which the program uses to derive its hypothesis for solving problems.
Applications of inductive logic programming today can be found in natural language processing and bioinformatics.
Support Vector Machines
Support vector machines are a supervised learning tool commonly used in classification and regression problems. An computer program that uses support vector machines may be asked to classify an input into one of two classes. The program will be provided with training examples of each class that can be represented as mathematical models plotted in a multidimensional space (with the number of dimensions being the number of features of the input that the program will assess).
The program plots representations of each class in the multidimensional space and identifies a “hyperplane” or boundary which separates each class. When a new input is analyzed, its output will fall on one side of this hyperplane. The side of the hyperplane where the output lies determines which class the input is. This hyperplane is the support vector machine.
Representation Learning
Representation learning, also called feature learning, is a set of techniques within machine learning that enables the system to automatically create representations of objects that will best allow them to recognize and detect features and then distinguish different objects. So the features are also used to perform analysis after they are identified by the system.
Feature learning is very common in classification problems of images and other media. Because images, videos, and other kinds of signals don’t always have mathematically convenient models, it is usually beneficial to allow the computer program to create its own representation with which to perform the next level of analysis.
Similarity Learning
Similarity learning is a representation learning method and an area of supervised learning that is very closely related to classification and regression. However, the goal of a similarity learning algorithm is to identify how similar or different two or more objects are, rather than merely classifying an object. This has many different applications today, including facial recognition on phones, ranking/recommendation systems, and voice verification.
Sparse Dictionary Learning
Sparse dictionary learning is merely the intersection of dictionary learning and sparse representation, or sparse coding. The computer program aims to build a representation of the input data, which is called a dictionary. By applying sparse representation principles, sparse dictionary learning algorithms attempt to maintain the most succinct possible dictionary that can still completing the task effectively.
Deep Learning with PyTorch
PyTorch is an open source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing, primarily developed by Facebook's AI Research lab. PyTorch is a Python-based scientific computing package serving two broad purposes: A replacement for NumPy to use the power of GPUs and other accelerators. An automatic differentiation library that is useful to implement neural networks.
TensorFlow
TensorFlow is a free and open-source software library for machine learning. It can be used across a range of tasks but has a particular focus on training and inference of deep neural networks. Tensorflow is a symbolic math library based on dataflow and differentiable programming. TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications.
ML & DL with Keras
Keras is an open-source software library that provides a Python interface for artificial neural networks. Keras acts as an interface for the TensorFlow library. Up until version 2.3 Keras supported multiple backends, including TensorFlow, Microsoft Cognitive Toolkit, Theano, and PlaidML. Keras is used by CERN, NASA, NIH, and many more scientific organizations around the world (and yes, Keras is used at the LHC). Keras has the low-level flexibility to implement arbitrary research ideas while offering optional high-level convenience features to speed up experimentation cycles.
Scikit-Learn
Scikit-learn (formerly scikits.learn and also known as sklearn) is a free software machine learning library for the Python programming language. It features various classification, regression and clustering algorithms including support vector machines, random forests, gradient boosting, k-means and DBSCAN, and is designed to interoperate with the Python numerical and scientific libraries NumPy and SciPy.
