machine learning - Advantages of RNN over DNN in prediction -
i going work on problem needs addressed either rnn or deep neural nets. in general, problem predicting financial values. so, because given sequence of financial data input, thought rnn better. on other hand, think if can fit data structure, can train dnn better because training phase easier in dnn rnn. example, last 1 month info , keep 30 inputs , predict 31'th day while using dnn. dont understand advantage of rnn on dnn in perspective. first question proper usage of rnn or dnn in problem.
my second questions somehow basic. while training rnn, isnt possible network "confused"? mean, consider following input: 10101111, , our inputs 1 digits 0 or 1 , have 2-sequences (1-0,1-0,1-1,1-1) here after 1, comes 0 several times. , @ end, after 1 comes 1. while training, wouldnt become major problem? is, why system not gets confused while training sequence?
thank help
i think question phrased bit problematically.
first, dnns class of architectures. convolutional neural network differs deep belief network or simple deep mlp. there feed forward architectures (e.g. tdnn) fit timeseries prediction depends on you, whether you're more interested in research or solving problem.
second, rnns "deep" gets. considering basic rnn, elman network: during training backpropagation through time (bptt) unfolded in time - backpropagating on t timesteps. since backpropagation done not vertically in standard dnn horizontally on t-1 context layers, past activations of hidden layers t-1 timesteps before present considered activation @ current timestep. illustration of unfolded net might in understanding wrote (source):
this makes rnns powerful timeseries prediction (and should answer both questions). if have more questions, read elman networks. lstms etc. confuse you. understanding elman networks , bptt needed foundation understand any other rnn.
and 1 last thing you'll need out for: vanishing gradient problem. while it's tempting let's make t=infinity , give our rnn memory possible: doesn't work. there many ways working around problem, lstms quite popular @ moment , there proper lstm implementations around nowadays. it's important know basic elman network could struggle t=30.
Comments
Post a Comment