Deep Learning
Abstract:
Deep learning models signify a new knowledge example fashionable machine intelligence (AI) and machine intelligence. Recent progress results in fashionable figure examination and determination have created large interest in this field because, in addition to using fashionable methods, many additional areas of expertise in providing substantial information in visible form appear attainable. On the negative aspect of the situation, the manipulation of numbers and computational methods fundamental to deep learning models happens very frequently, particularly for combining two or more academic fields of research. For this reason, we present in this paper the first review of deep learning approaches containing Deep Feedforward Neural Networks (D-FFNN), Convolutional Neural Networks (CNNs), Deep Belief Networks (DBNs), Autoencoders (AEs), and Long Short-Term Memory (LSTM) networks. These models form the main centre architecture of deep knowledge models now secondhand and concede the possibility of being part of fashionable some information in visible form in the researcher's toolbox. Importantly, these gist-structurally constructed dwelling blocks can be assembled flexibly—in a trendy and nearly Lego-like manner—to construct new use-specific network architectures. Hence, a fundamental understanding of these network architectures exists, which influences the anticipated future of fashionable AI.
Introduction
We happen to fill a place with important information in visible form. This period in history places all the scope of a methodical study of parts of the material world and manufacturing creates large amounts of information in visible form. This confronts the United States with exceptional challenges concerning their statement of results from examination and understanding. For this reason, there is a need for immediate attention to novel machine intelligence and machine intelligence forms that can help fashion make use of this information in visible form. Deep learning (DL) refers to specific novel methods now being given consideration (Hinton and others., 2006). DL is a kind of knowledge algorithm, alternatively, a plan that may be used to determine complex declarations made in advance models, such as multi-coating affecting animate nerve organ networks, accompanying many unseen parts (LeCun and others., 2015). Importantly, extensive knowledge is advantageously applied to a variety of request questions. For instance, a deep knowledge of means set the record for the categorization of manuscript digits of the MNIST basic document file, accompanying a mistake rate of 0.21% (Wan and others., 2013). Further use scope of a surface contains counterpart acknowledgement (Krizhevsky and others., 2012a; LeCun and others., 2015), talk acknowledgement (Graves and others., 2013), human language understanding (Sarikaya and others., 2014), sound display (Mohamed and others., 2011) and computational plant structure (Leung and others., 2014; Alipanahi and others., 2015; Zhang S. and others., 2015; Smolander and others., 2019a,b).
Fake models affecting animate nerve organ networks have existed since the 1950s (Rosenblatt, 1957); however, the current wave of deep learning affecting animate nerve organ networks began in 2006 (Hinton and others., 2006). A universal characteristic of common people's differences between directed and alone deep knowledge models is that these models bear many tiers of unseen neurons well-informed by, for example, a Restricted Boltzmann Machine (RBM) together with accompanying backpropagation and mistake gradients of the Stochastic Gradient Descent (Riedmiller and Braun, 1993). Due to the variety of deep knowledge approaches, an inclusive conversation with another person happens very frequently, and accordingly, former reviews propose loyal substitute subject matter. For instance, a comprehensive view outside of particularised reasons may be about LeCun and others. (2015), a remarkable summary accompanying many itemised remarks by Schmidhuber (2015), and reviews about putting a substance on another rule, for instance, figure study (Rawat and Wang, 2017; Shen and others., 2017), formal talk to audience acknowledgement (Yu and Li, 2017), machine intelligence (Young and others., 2018), and biomedicine (Cao and others., 2018).
In contrast, our review aims at a middle level, providing in addition to the concerning details, analysis for the most part excluded. Given the combining of two or more academic fields of interest in fashionable deep knowledge, that is, unspecified information in visible form, a methodical study of part of the material world (Emmert Streib and Dehmer, 2019a), this makes it easier for human beings new to the field to excite. The subject matter we picked aims attention at the gist methods of deep learning approaches, including Deep Feedforward Neural Networks (D-FFNN), Convolutional Neural Networks (CNNs), Deep Belief Networks (DBNs), Autoencoders (AEs), and Long Short-Term Memory (LSTM) networks. Additional network architectures that we discuss with one another aid in a fashionable understanding of these centre approaches.
The past events affecting animate nerve organ networks have delayed many human beings' ability to donate toward their incidents over the decades. Given the current eruption of interest in fashionable deep education, it is not unexpected that the task of crediting key events is not uncontroversial. In the following, we happen to be prejudiced about something given that emphasises only the ultimate outstanding offering.
McCulloch and Pitts (1943) made the first mental or physical observations about the manipulation of numbers in a neuron model. This model proposed providing an abstract expression for the functioning of a neuron outside of imitating the biophysical method of a genuine in-existence concerning-life neuron. It is entertaining to note that this model act rejects knowledge.
The first thing understood about biologically stimulating knowledge fashionable affecting animate nerve organ networks was brought in by Hebb (1949). Hebbian knowledge is a form of alone knowledge affecting animate nerve organ networks.
In 1957, the Perceptron was brought in by Rosenblatt (1957). The Perceptron exists as an alone-coated interconnected system that serves as an uninterrupted two-fold classifier. In the up-to-date system of words for communication of ANNs, a perceptron uses the Heaviside function as an incitement function.
In 1960, the Delta Learning rule for education was made familiar by Widrow and Hoff (1960). The Delta Learning rule, as known or named at another time or place as the Widrow & Hoff Learning rule or the Least Mean Square rule, is a slope line of ancestry knowledge rule for bringing up to date the weights of the neurons. A diplomat's legal freedom of backpropagation treasure exists.
In 1968, a design named Group Method of Data Handling (GMDH) for preparation affecting animate nerve organ networks was brought in by Ivakhnenko (1968). These networks exist widely deliberate the first deep education networks of the Feedforward Multilayer Perceptron type. For instance, the paper (Ivakhnenko, 1971) secondhand a deep GMDH combine 8 coatings. Interestingly, the programme of coating and part per coating could understand information and exist in unstable freshness.
Minsky and Papert (1969) published an influential paper arguing that the XOR question can not be understood by a perceptron because it does not exist in a linearly breakable form. This causes a pause period in the life of something that affects animate nerve organ networks, named the "AI cold season of the year."
In 1974, a backpropagation (BP) bear happened to use fashionable affecting animate nerve organ networks (Werbos, 1974) for knowledge of the burden of a directed person's behaviour and was used by Werbos (1981). However, the form itself exists at its most senior (visualize such as Linnainmaa, 1976).
Fukushima (1980) made known the existence of a ranked-order multilayered interconnected system capable of being seen with eyes known as Neocognitron in 1980. After the open ocean GMDH networks (visualised above), the Neocognitron exists as a deliberate second imitation NN that merits the attribute "deep". It brought in convolutional NNs (contemporary named CNNs). The Neocognitron occurs like the design of new, directed, deep feedforward neural networks (D FFNN) (Fukushima, 2013).
In 1982, Hopfield brought in a content-addressable ability to hold in the mind's interconnected system, at that time named the Hopfield Network (Hopfield, 1982). Hopfield Networks are an instance of how repeating affects animate nerve organ networks.
In 1986, backpropagation became fashionable again with a paper by Rumelhart and others. (1986). " "[...] They demonstrate tentatively that this knowledge invention can create valuable likenesses and, for that reason, aid in inexact interconnected system education tasks.
In 1987, Terry Sejnowski made known the NETtalk treasure (Sejnowski and Rosenberg, 1987). The programme is well-informed in what way or manner to introduce English conversation and is intelligent enough to make or become better over time.
In 1989, a Convolutional Neural Network was prepared to accompany the backpropagation treasure to gain in manuscript digits (LeCun and others., 1989). A very much alike arrangement happened later, used to state in manuscript checks and zip codes, subject to a series of actions to achieve a cash register result in the late 90s and early 2000s. Note: In the 1980s, the second wave of interconnected systems research emerged in fashionable part by way of a change named connectionism (Fodor and Pylyshyn, 1988). This wave endured as far as the intervening 1990s.
In 1991, Hochreiter posed a fundamental question about some deep education networks that had a connection with the question of the existence of a backpropagation treasure (Hochreiter, 1991). His study discloses that the signal bred by backpropagation either decreases or increases outside bounds. In the case of decay, this happens equivalent to the distance down or across the network. This happened immediately and became famous as the "vanishing or discredit slope" question.
In 1992, the first incomplete remedy to this question was submitted by Schmidhuber (1992). The search for something understood to pre-train an RNN is fashionable as a way to increase speed after directed education. Furthermore, the intentional network is made up of 1,000 coats of a fashionable repeating interconnected system.
In 1995, oscillatory affecting animate nerve organ networks was brought in by Wang and Terman (1995). They exhibit secondhand fashionable differences, putting a substance on another similar concept and formal talk to audience separation and creating complex period succession (Wang and Terman, 1997; Hoppensteadt and Izhikevich, 1999; Wang and Brown, 1999; Soman and others, 2018).
In 1997, the first directed model for knowledge RNN was made familiar by Hochreiter and Schmidhuber (1997), and it exists as Long Short-Term Memory (LSTM). An LSTM keeps this from happening or continues the rotting signal question in the middle of two points by making the LSTM networks "summon into mind" facts for a protracted range.
In 1998, the Stochastic Gradient Descent invention (slope-located knowledge) was linked to accompanying the backpropagation invention for reconstructing education on CNN (LeCun and others., 1989). As a result, LeNet-5, a 7-level convolutional network, was brought in for classifying help-composed numbers ahead of checks.
In 2006, there was a widely thought-out progress period caused by fashionable Hinton and others. In 2006, it was put on display that affecting animate nerve organ networks named Deep Belief Networks may be capably prepared by utilising a plan of action called desiring excessively tier-intelligent pre-preparation. This has undergone initiation after the second wave of affecting animate nerve organ networks that were created in addition to the use of the term "deep knowledge standard."
In 2012, Alex Krizhevsky achieved the ImageNet Large Scale Visual Recognition Challenge by utilising AlexNet, a Convolutional Neural Network that makes use of a GPU and surpasses LeNet5 (visualize above) (LeCun and others., 1989). This favourable outcome began a convolutional interconnected system rebirth fashionable in the open ocean knowledge society (visualize Neocognitron).
In 2014, fruitful opposing networks brought in fashionable Goodfellow and others. (2014); (2014)The planned event is for two affecting animate nerve organ networks to equate in a popular game-like category. Overall, this enacts a fruitful model that can produce new information in visible form. This bear is named "the nippy."
In 2019, Yoshua Bengio, Geoffrey Hinton, and Yann LeCun will give the Turing Award for abstract and the science of applying power to use breakthroughs that have fashioned deep-affecting animate nerve organ networks as a fault-finding component of the estimate. Specifically, the figure shows the number of something printed for reading freedom ahead of the something printed for reading period for DL, deep knowledge; CNN, convolutional interconnected system; DBN, deep something regarded as true network; LSTM, long temporary specific thing remembered; AEN, autoencoder; and MLP, Multilayer Perceptron. The two breaks by hitting or throwing violently lines happen scaled by a determinant of 5 (deep knowledge) and 3 (convolutional interconnected system), that is, overall, for deep knowledge, we raise the adulthood of printing of written or visual material (fashionable total 30, 230). Interestingly, most of these happen fashionable the study of computers (52.1%) and the act of turning material to use (41.5%). In the request region, healing depicts (6.2%), science (2.6%), and computational study of animals (2.5%) are taken most consideration. This attention existing impression of the brief past events of deep education signifies that the pattern exists still below growth.
Regression algorithmsDeep education bears acquire large celebrity fashionable discovered through experimentation estimate, and allure algorithms exist established by manufacturing that resolves complex questions. All deep knowledge algorithms use various types of affecting animate nerve organs networks to act distinguishing tasks.
Deep education uses affected affecting animate nerve organs networks to act cultured computations ahead of big amounts of information in visible form. It happens as a type of machine intelligence that establishes the construction and function of the human brilliant person.
Defining Neural NetworksAn interconnected system exists, organized like the human intellect and consisting of affected neurons, as known or named at another time or place in growth. These growths exist shapely close to each other in three tiers:
The recommendation coating is input, hidden and output layers.
Data supply each bud accompanying facts fashionable the form of inputs. The bud multiplies the inputs accompanying haphazard weights, calculates bureaucracy, and accumulation a bias. Finally, nonlinear functions, as known or named at another time or place incitement functions, exist used to decide that neuron to fire. While deep knowledge algorithms feature self-education likeness, they deliver ANNs that mirror the habit the very smart person computes facts. During the preparation process, algorithms use a mysterious place where one feels comfortable fashionable the recommendation disposal to extract facial characteristics, group objects, and uncover valuable information in visible form patterns. Much like preparation machines for self-education, this takes place at diversified levels, utilizing algorithms to build the models.
Deep education models use assorted algorithms. While no network is deliberate, few algorithms exist that are better adapted to perform distinguishing tasks. To pick the right one, it’s good to gain a hard understanding of all basic algorithms.
Types of Algorithms secondhand fashionable Deep Learning Convolutional Neural Networks (CNNs), Long Short Term Memory Networks (LSTMs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), Radial Basis Function Networks (RBFNs), Multilayer Perceptrons (MLPs), Self Organizing Maps (SOMs), Deep Belief Networks (DBNs), RBMs (Restricted Boltzmann Machines), Autoencoders
- Convolutional Neural Networks (CNNs)
CNN's exists to recognise smaller countries dependent upon a larger counterpart, process healing representation, forecast period succession, and discover usually. CNN's bear diversified tier that process and extract facial characteristics from information in the visible form:
Figure 1: Model of a representation treated by way of CNNCNN bear a spiral coating that bears various filters to act the loop movement. CNN's bears a ReLU coating to act as movement ahead of essential features. Something produced happens to make something right on the feature chart of the geographic area. The correct situation feature chart of the geographic area next feeds into a combined coating. Pooling happens an unhappy-try movement that reduces the range of the feature chart of the geographic area. The combine tier, therefore, converts the happen two-relating to space and size arrays from the combine feature chart of the geographic area into a sole, long, constant, uninterrupted heading by flattening it. A completely affiliated tier forms when the levelled something from which another originates from the combined tier happens to augment as a recommendation, categorising and labelling the counterpart.
Convolutional Neural Networks engage a function named convolution. The idea behind the ruling class is that a suggestion of correction joins each neuron accompanying the next one, and we combine them, accompanying only a small quantity of bureaucracy (the open to new ideas field). In a habit, they try to balance feedforward networks to prevent overfitting (when the model learns only to pre-visualize information in visible form and can’t state it), which creates ruling class elite fashion recognition relating to space connection between two points in visible form. That’s why their basic use case involves computer vision and hard work in the way that figure categorization, related to televised image acknowledgement, healing concept reasoning, and self-forceful vehicle driving on streets place a goal beyond human accomplishment. They exist in addition to being ideal for integrating accompanying added types of models in the way that recurrent networks and autoencoders do. One aforementioned model is Sign Language Recognition.
- LSTMs (Long-Short Term Memory Networks)
LSTMs are a type of recurrent neural network (RNN) that can find out and remember complete reliance. Recalling past facts for long periods is the default manner of conducting oneself
Figure 2: LSTMLSTMs hire news over the temporal length of an event or entity's existence. They exist as beneficial happenings after-succession declarations made in advance because they summon into mind premature inputs. LSTMs have a chain-like building where four communicating tiers mutually exchange information in a singular habit. Besides the temporal length of an event or entity's existence-succession declaration made in advance, LSTMs are widely in the field of pharmaceutical development, speech recognition and music composition.
LSTMs Work Procedure
-
First, they are not able to remember beside the point parts of the prior state.
-
The container-state principles are then selectively brought up to date.
-
Finally, the something produced of sure parts of the small room state.
-
- RNNs (Recurrent Neural Networks)
RNNs bear networks that form supervised era, that admit the outputs from the LSTM expected nourish inputs to the current period in the life of something.
Figure 4: RNN MODELRecurrent networks happen perfectly for a piece of short time-connected information in visible form and they exist passed down contributing order predict. They use a few forms of response, the place they return the something produced back to the recommendation. The something produced from the LSTM evolves into a recommendation to the current period in the life of something and can remember former inputs on account of allure within specific thing remembered. Therefore, they exist able to perform to recollect past information in visible form and use that news fashionable allure declaration made in advance. To achieve a better goal, researchers changed the original neuron into an intricate makeup to a degree, GRU parts and LSTM Units. RNNs signify machine translation, time-series analysis, handwriting recognition, image captioning and natural-language processing.
The output of one set as input for another like t-1 fed as input to t and output from t fed as input to t+1. The performing arithmetic gives reason for recorded as actually having happened facts, and the model extent or bulk of some dimension act does not increase accompanying the recommendation extent or bulk of some dimension.
- Generative Adversarial Networks (GANs)
GANs exist fruitful deep knowledge algorithms that develop in mind or physically new information in visible form instances that look or be like the preparation information in visible form. GAN bear two elements: an engine converting energy, that learns to create fake information in visible form, and a discriminator, that learns from that wrong news. The discriminator learns to equate the engine converting energy’s fake information in visible form and the genuine in existence sample information in visible form. During the beginning preparation, the engine converting energy produces fake information in visible form, and the discriminator fast learns to report that it's wrong. The GAN sends the results to the engine converting energy and the discriminator to bring up to date the model. Generative Adversarial Networks application fields are fashion, astronomical images, video games and interior design.
- Radial Basis Function Networks (RBFNs)
RBFNs happen important in their way types of feedforward affecting animate nerve organs networks that use branching foundation for belief functions as incitement functions. They bear a recommendation tier, an unseen tier, and a something produced tier and happen generally secondhand for categorization, reversion, and opportunity-succession declaration made in advance.
Figure 6: RBFNs WorkRBFNs act categorization by weighing the recommendation's correspondence to instance from the preparation set. RBFNs bear a recommendation heading that feeds to the recommendation coating. They bear a coating of RBF neurons. The function finds the burden total of the inputs, and the something produced coating bears individual bud per classification or class of information in visible form. The neurons fashionable the secret tier holds the Gaussian transfer functions, that bear outputs that happen with the order reversed equivalent to the distance from the neuron's centre. The network's something produced exist an uninterrupted merger of the recommendation’s branching-foundation for belief functions and the neuron’s limit.
- Multilayer Perceptrons (MLPs)
MLPs happen a wonderful place to start knowledge about deep learning electronics.
Figure 7: MLPs Work layoutMLPs concern the class of feedforward affecting animate nerve organs networks accompanying a diversified tier of perceptrons that bear incitement functions. MLPs are made up of a recommendation tier and something produced coating that exists adequately affiliated. They bear the identical number of recommendations and something produced tier but grant permission bear diversified unseen coating and may be used to build talk-acknowledgement, representation-acknowledgement, and well-run political organization-interpretation computer program. MLPs feed the information in visible form to the recommendation coating of the network. The coating of neurons combines fashionable a diagram for fear that the signal passes the fashionable individual course. MLPs estimate the recommendation accompanying the weights that survive middle from two points the recommendation coating and the secret coating. MLPs use incitement functions to decide that knot to fire. Activation functions involve ReLUs, bent functions, and tanh. MLPs train the model to accept the equating and acquire information the reliance middle from two points the free and the goal variables from a preparation basic document file. Below happen an instance of an MLP. The drawing computes weights and bias and applies acceptable incitement functions to categorize figures of odd couple or grouping.
- Self Organizing Maps (SOMs)
Professor Teuvo Kohonen fictitious SOMs, authorize information in visible form imagination to lower the range of information in visible form through self-systematize fake affecting animate nerve organs networks. Data imagination attempts to answer the question that homo sapiens cannot surely make a picture of in the mind extreme-relating to space and size information in visible form. SOMs exist develop in mind or physically to help consumers comprehend this extreme-relating to space and size news.
Figure 8: SOMs structureSOMs start a computer weigh each bud and pick a heading carelessly from the preparation information in visible form. SOMs ask questions pointedly at each bud to find that weights happen ultimate likely recommendation heading. The triumphant bud exists named high-quality Matching Unit (BMU). SOMs find the BMU’s community, and the amount of neighbours lessens over occasion. SOMs award a triumphant burden to the sample heading. The tight a bud search out a BMU, the more allure pressure changes. The further the neighbour happens from the BMU, the less it learns. SOMs repeat step two for N redundancy. Below, visualize a drawing of a recommendation heading of the various banner. This information is visible from feeds to a SOM, which therefore converts the information in visible form into 2D RGB principles. Finally, it part company in a romantic and classification the various banner.
- Deep Belief Networks (DBNs)
DBNs exist fruitful models that are made up of diversified coating of assumed, hidden variables. The hidden variables bear twofold principles and happen frequently named unseen whole. DBNs exist a stack of Boltzmann Machines accompanying something that communicates middle from two points the coating, and each RBM coating communicates accompanying two together with the premature and after coating. Deep Belief Networks (DBNs) exist secondhand for representation-acknowledgement, related to the televised image-acknowledgement, and motion-capture information in visible form.
Figure 9: DBNs structureGreedy education algorithms train DBNs. The desiring excessively knowledge invention uses a tier-by-coating approach for education the top-physically lower, fruitful weights. DBNs run the steps of Gibbs try ahead of the top two unseen tiers. This stage draws a sample from the RBM defined apiece top two secret tiers. DBNs draw a sample from the seeable part utilizing an alone pass related to the previous family or family trait try through the rest of the model. DBNs gain that the principles of the hidden variables fashionable each coating may be implicit by an alone, bottom-up pass.
- Restricted Boltzmann Machines (RBMs)
Developed by Geoffrey Hinton, RBMs exist assumed affecting animate nerve organs networks that can get or give an advantage a frequency distribution over a set of inputs. This deep education invention exists secondhand for range decline, categorization, reversion, cooperative seep through, feature education, and subject matter form. RBMs establish the constructed dwelling blocks of DBNs.
Figure 10: RBMs structureVisible part, Hidden whole are two layers of RBMs. Each seeable part exist affiliated to all secret whole. RBMs bear a biased part that is to say related to all the apparent whole and the secret part, and they bear no something produced growth. RBMs bear two periods in the life of something: football pass toward opponent's goal and late pass. RBMs acknowledge the inputs and convert the ruling class into a set of numbers that encodes the inputs fashionable the football pass toward the opponent's goal. RBMs integrate each recommendation accompanying individual pressure and individual overall bias. The treasure passes the something produced to the secret tier. In the slow growth pass, RBMs take that set of numbers and convert the ruling class to form the reconstructed inputs. RBMs connect each incitement accompanying individual pressure and overall bias and pass the something produced to the seeable coating for rebuilding. At the apparent tier, the RBM compares the rebuilding accompanying the original recommendation to resolve the character of the result.
- Autoencoders
Autoencoders exist a particular type of feedforward interconnected system at which point the recommendation and something produced exists equal. Geoffrey Hinton plan autoencoders fashionable the 1980s to answer alone education question. They exist prepared affecting animate nerve organs networks that copy the information in visible form from the recommendation tier to the something produced coating. Autoencoders happen secondhand for purposes to a degree drug finding, celebrity declaration made in advance, and concept subject to series of actions to achieve the result.
Figure 11: Autoencoders strutureAn autoencoder resides of three main parts: the encoder, the law, and the linguist. Autoencoders happen organized to take in a recommendation and change completely it into various likenesses. They, therefore, attempt to reorganize the original recommendation as correctly as likely. When an exact likeness of a number exists not without any doubt seeable, it feeds to an autoencoder interconnected system. Autoencoders first encrypt the counterpart, before making less the extent or bulk of some dimension of the recommendation into a tinier likeness. Finally, the autoencoder decodes the concept to produce the reconstructed concept.
Conclusion
Deep Learning has received a great deal of attention for cutting-edge discoveries made through experimentation and collaboration. Since 2012, the period when a Convolutional Neural Network brought to a successful conclusion unprecedented precision or correctness in contact with a counterpart acknowledgement contest (ImageNet Large Scale Visual Recognition Challenge), to a greater extent, research documents have concluded yearly and, to a greater extent, parties have begun to include Neural Networks into their trade. Deep Learning is thought to have occurred immediately in a 2.5 billion market and is expected to grow to 18.16 billion by 2023. In this paper, we supported a preliminary review of deep education models. These models could be intentional about the centre architectures that now govern deep knowledge. In addition, we also talk about another accompanying idea wanted for a mechanics understanding of these models, like Restricted Boltzmann Machines. Given the adaptability of network architectures admitting new models, a complete number of interconnected system models may be assembled by making use of a place where one feels comfortable with the gist of the structural constructed dwelling blocks talked over with another in this place review. Hence, a fundamental understanding of these essential features is key to being outfitted for future incidents.
REFERENCES
[1]Alipanahi, B., Delong, A., Weirauch, M. T., and Frey, B. J. (2015). Predicting the sequence specificities of DNA-and RNA-binding proteins by deep learning. Nat. Biotechnol. 33, 831–838. doi: 10.1038/nbt.3300
[2]An, J., and Cho, S. (2015). Variational Autoencoder Based Anomaly Detection Using Reconstruction Probability. Special Lecture on IE 2.
[3]Arulkumaran, K., Deisenroth, M. P., Brundage, M., and Bharath, A. A. (2017). Deep reinforcement learning: a brief survey. IEEE Signal Process. Mag. 34, 26–38. doi: 10.1109/MSP.2017.2743240
[4]Bergmeir, C., and Benítez, J. M. (2012). Neural networks in R using the stuttgart neural network simulator: RSNNS. J. Stat. Softw. 46, 1–26. doi: 10.18637/jss.v046.i07
[5]Biran, O., and Cotton, C. (2017). “Explanation and justification in machine learning: a survey,” in IJCAI-17 Workshop on Explainable AI (XAI). Vol. 8, 1.
[6]Biran, O., and Cotton, C. (2017). “Explanation and justification in machine learning: a survey,” in IJCAI-17 Workshop on Explainable AI (XAI). Vol. 8, 1.
[7]Breiman, L. (2001). Statistical modeling: the two cultures (with comments and a rejoinder by the author). Stat. Sci. 16, 199–231. doi: 10.1214/ss/1009213726
[8]Cao, C., Liu, F., Tan, H., Song, D., Shu, W., Li, W., et al. (2018). Deep learning and its applications in biomedicine. Genomics Proteomics Bioinform. 16, 17–32. doi: 10.1016/j.gpb.2017.07.003
[9]1.P Wang, W Li, P Ogunbona et al., "RGB- D-based Human Motion Recognition with Deep Learning: A Survey", Computer Vision and Image Understanding 2018:1–22..
[10]2.Y Yuan and Max Q.-H. Meng, "Deep learning for polyp recognition in wireless capsule endoscopy images", Medical Physics, pp. 1379-1389, 2017.
[11]3.J Liu, M Osadchy, L Ashton et al., "Deep convolutional neural networks for Raman spectrum recognition: a unified solution", Analyst, pp. 4067-4074, 2017.
[12]4.N Dalal and B. Triggs, "Histograms of oriented gradients for human detection", IEEE Computer Society Conference on. Piscataway, pp. 886-893.
[13]Show in Context View Article Full Text: PDF (238KB) Google Scholar
[14]5.Y. Bengio, "Learning deep architectures for AI", Foundations and Trends in Machine Learning, vol. 2, no. 1, pp. 1-127, 2009.
[15]6.G E HINTON, S OSINDERO and Y W TEH, "A fast learning algorithm for deep belief nets", Neural Computation, vol. 18, no. 7, pp. 1527-1554, 2006.
[16]7.Stefan Hosein and Patrick Hosein, "Load forecasting using deep neural networks", 2017 IEEE Power & Energy Society Innovative Smart Grid Technologies Conf., pp. 1-5.
[17]8.T Ogabe, H Ichikawa, K Sakamoto et al., "Optimization of decentralized renewable energy syes-Asia (ISGT-Asia)", IEEE2016, pp. 1014-1018.
[18]9.G M U Din and A K Marnerides, "Short-term load forecasting using deep neural networks (DNN)", 2017 North American Power Symposium Conf., pp. 1-6.
[19]10.Danting Dong, Zhihao Sheng and Tiancheng Yang, "Wind Power Prediction based on Recurrent Neural Network with Long Short-Term Memory Units", 2018 International Conference on Renewable Energy and Power Engineering, pp. 34-38.
[20]11.Yubo Tao, Hongkun Chen and Chuang Qiu, "Wind Power Prediction and Pattern Feature Based on Deep Learning Method", 2014 IEEE PES Asia-Pacific Power and Energy Engineering Conf., pp. 1-4.
[21]12.Peng Xiaosheng, Wang Bo, Yang Fan, Fan Gaofeng, Wang Zheng and Cheng Kai, "A Deep Learning Approach for Wind Power Prediction based on Stacked Denoising Auto Encoders Optimized by Bat Algorithm", 2018 China International Conference on Electricity Distribution (CICED), pp. 945-948.
[22]13.Yaping Deng, Hao Jia, Pengcheng Li, Xiangqian Tong and Feng Li, "A Deep Learning Method based on Long Short Term Memory and Sliding Time Window for Type Recognition and Time Location of Power Quality Disturbance", 2018 Chinese Automation Congress (CAC), pp. 1764-1768.
[23]14.Guanyu Tian, Qun Zhou and Liang Du, "Deep Convolutional Neural Networks for Distribution System Fault Classification", 2018 IEEE Power & Energy Society General Meeting (PESGM), pp. 1-5.
[24]Cao, S., Lu, W., and Xu, Q. (2016). “Deep neural networks for learning graph representations,” in Thirtieth AAAI Conference on Artificial Intelligence.
[25]Carreira-Perpinan, M. A., and Hinton, G. E. (2005). “On contrastive divergence learning,” in Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics (Citeseer), 33–40.
[26]Charles, A. S., Olshausen, B. A., and Rozell, C. J. (2011). Learning sparse codes for hyperspectral imagery. IEEE J. Select. Top. Signal Process. 5, 963–978. doi: 10.1109/JSTSP.2011.2149497
[27]Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., et al. (2015). Mxnet: a flexible and efficient machine learning library for heterogeneous distributed systems.
[28]Chimera (2019). Pydbm. arXiv:1512.01274. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., et al. (2014). Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv [Preprint]. arXiv:1406.1078. doi: 10.3115/v1/D14-1179