voladoddi

Basic questions (for someone who works with NNs day in and day out) -

1) When you say "tune the parameters" of a deep NN, what do you mean, more specifically, what parameters are you referring to? Isn't NN a kind of black box ?

2) How do you decide which parameters need to be tuned?for e.g. statistical tests?

Thank you! So for instance, in terms of SGD, is there a system in place to tune alpha value? In class I recollect we had to observe how system diverged when it was high enough, and there was no fast convergence rate like you said in case alpha was too low, but I never got an answer for how these numbers came up. Is it more like a Monte Carlo testing where you randomly throw in learning rate values and see how the system performs?

Because I'd reckon that would take a hell lot of time.

P.S. Apologies if this question sounds dumb.

Could you provide a list of most effective resources for starting with deep neural networks for someone who has training (pun unintended) or has already completed an entry level ML course at grad school?

ImageNet challenges are a good place to see how folks use already developed libraries for utilising DNNs but what if one has to get their feet wet with coding DNNs using a programming language ?