Neural network for letter recognition

I'm trying to add to the code for a single layer neural network which takes a bitmap as input and has 26 outputs for the likelihood of each letter in the alphabet.

The first question I have is regarding the single hidden layer that is being added. Am I correct in thinking that the hidden layer will have it's own set of output values and weights only? It doesn't need to have it's own bias'?

Can I also confirm that I'm thinking about the feedforward aspect correctly? Here's some pseudocode:

// input => hidden
for j in hiddenOutput.length:
    sum=inputs*hiddenWeights
    hiddenOutput[j] = activationFunction(sum)
// hidden => output
for 开发者_高级运维j in output.length:
    sum=hiddenOutputs*weights
    output[j] = activationFunction(sum)

Assuming that is correct, would the training be something like this?

def train(input[], desired[]):
    iterate through output and determine errors[]
    update weights & bias accordingly
    iterate through hiddenOutput and determine hiddenErrors[]
    update hiddenWeights & (same bias?) accordingly

Thanks in advance for any help, I've read so many examples and tutorials and I'm still having trouble determining how to do everything correctly.

---

Dylan, this is probably long after your homework assignment was due, but I do have a few thoughts about what you've posted.

• Make the hidden layer much bigger than the size of the input bitmaps.
• You should have different weights and biases from input -> hidden than from hidden -> output.
• Spend a lot of time on your error function (discriminator).
• Understand that neural nets have a tendency to get quickly locked in to a set of weights (usually incorrect). You'll need to start over and train in a different order.

The thing I learned about neural nets is that you never know why they're working (or not working). That alone is reason to keep it out of the realms of medicine and finance.

---

you might want to read http://www.ai-junkie.com/ann/evolved/nnt1.html . there it mentions exactly something about what you are doing. It also provided code along with a (mostly) simple explanation of how it learns. Although the learning aspect is completely different from feed forward this should hopefully give you some ideas about the nature of NN.

It is my belief that even the hidden and output layers should have a bias.

Also NN can be tricky, try first identifying only 1 letter. Getting a consistent high/low signal from only a single output. Then try to keep that signal with different variations of the same letter. Then you can progress and add more. You might do that by teaching 26 different networks that give an output only on a match. Or maybe you make it as one large NN with 26 outputs. Two different approaches.

---

As far as the use of bias terms is concerned I found the section Why use a bias/threshold? in the comp.ai.neural-nets FAQ very useful. I highly recommend reading that FAQ.